2024-09-16T14:49:36.322 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T14:49:36.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:49:36.421 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784 branch: reef description: rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906784' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: mon ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMlT7dfvhv6nDcP9NxoDsK3UuClKzMjmYbRwgwAAuolkrDvJljKTmXAPOir0lnmYHI8KMW+WJ16eBke4F+qjxhg= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBaksl13L6Xphrva4Bv21Cdq8BN6JHOLBG1DV1UHYmRV3iYyFyH/F1H7AwJgTwU1m3qDEG5FTySWUj5DIaa/Qk= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 thrash_many: true - rados: clients: - client.0 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T14:49:36.422 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T14:49:36.422 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T14:49:36.422 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T14:49:36.424 INFO:teuthology.task.internal:Checking packages... 2024-09-16T14:49:36.446 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T14:49:36.446 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T14:49:36.446 INFO:teuthology.packaging:ref: None 2024-09-16T14:49:36.446 INFO:teuthology.packaging:tag: None 2024-09-16T14:49:36.446 INFO:teuthology.packaging:branch: reef 2024-09-16T14:49:36.446 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:49:36.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T14:49:36.721 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853.gae7f3733 2024-09-16T14:49:36.723 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T14:49:36.730 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T14:49:36.730 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T14:49:36.940 INFO:teuthology.task.internal:Saving configuration 2024-09-16T14:49:36.954 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T14:49:37.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T14:49:37.467 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:44:32.828787', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMlT7dfvhv6nDcP9NxoDsK3UuClKzMjmYbRwgwAAuolkrDvJljKTmXAPOir0lnmYHI8KMW+WJ16eBke4F+qjxhg='} 2024-09-16T14:49:37.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:44:32.830198', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBaksl13L6Xphrva4Bv21Cdq8BN6JHOLBG1DV1UHYmRV3iYyFyH/F1H7AwJgTwU1m3qDEG5FTySWUj5DIaa/Qk='} 2024-09-16T14:49:37.494 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T14:49:37.618 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T14:49:37.619 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-16T14:49:37.619 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T14:49:37.816 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcac1f3bac0>, signals=[15]) 2024-09-16T14:49:37.817 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T14:49:37.903 INFO:teuthology.task.internal:Opening connections... 2024-09-16T14:49:37.903 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T14:49:37.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:49:37.981 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T14:49:37.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:49:38.059 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T14:49:38.258 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-16T14:49:38.277 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-16T14:49:38.277 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-16T14:49:38.335 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-16T14:49:38.335 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-16T14:49:38.335 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-16T14:49:38.335 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:49:38.335 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:49:38.336 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:49:38.337 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-16T14:49:38.364 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-09-16T14:49:38.383 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-09-16T14:49:38.383 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:49:38.442 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:49:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:49:38.443 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-09-16T14:49:38.464 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T14:49:38.473 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T14:49:38.536 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T14:49:38.537 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:49:38.539 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:49:38.557 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T14:49:38.601 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T14:49:38.601 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:49:38.604 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:49:38.705 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T14:49:38.855 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T14:49:39.029 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T14:49:39.029 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T14:49:39.029 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:49:39.030 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:49:39.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:49:39.047 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-16T14:49:39.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:49:39.048 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-16T14:49:39.105 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:49:39.106 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.14.0-503.el9.x86_64 2024-09-16T14:49:39.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2024-09-16T14:49:39.107 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:49:39.107 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.14.0-503.el9.x86_64 2024-09-16T14:49:39.107 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum install -y kernel 2024-09-16T14:49:41.668 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-16T14:49:42.203 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.2 MB 00:01 2024-09-16T14:49:43.920 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-16T14:49:48.938 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.5 MB 00:01 2024-09-16T14:49:50.421 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-16T14:49:52.242 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-16T14:49:54.876 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-16T14:49:57.948 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-16T14:49:59.675 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:49:59.675 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:49:59.764 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:49:59.776 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-16T14:49:59.777 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:49:59.778 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 79 M 2024-09-16T14:49:59.778 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 126 M 2024-09-16T14:49:59.778 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:49:59.931 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-09-16T14:50:00.640 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-16T14:50:01.123 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T14:50:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-16T14:50:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:50:01.491 INFO:teuthology.orchestra.run.smithi162.stdout:Total 46 MB/s | 79 MB 00:01 2024-09-16T14:50:02.171 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:50:02.539 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:50:02.540 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:50:03.428 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-16T14:50:03.526 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:50:03.527 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:50:05.435 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:50:05.435 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:50:05.538 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:50:05.542 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:50:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-09-16T14:50:05.542 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:50:05.543 INFO:teuthology.orchestra.run.smithi134.stdout:Install 4 Packages 2024-09-16T14:50:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:50:05.544 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 79 M 2024-09-16T14:50:05.545 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 126 M 2024-09-16T14:50:05.545 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:50:05.946 INFO:teuthology.orchestra.run.smithi134.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 182 kB/s | 23 kB 00:00 2024-09-16T14:50:05.988 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:50:06.640 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:50:06.732 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:50:09.540 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:50:09.674 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:50:10.804 INFO:teuthology.orchestra.run.smithi134.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.9 MB/s | 29 MB 00:04 2024-09-16T14:50:12.929 INFO:teuthology.orchestra.run.smithi134.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-16T14:50:14.480 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:50:14.517 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:19.422 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:21.403 INFO:teuthology.orchestra.run.smithi134.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.2 MB/s | 35 MB 00:15 2024-09-16T14:50:21.404 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:50:21.404 INFO:teuthology.orchestra.run.smithi134.stdout:Total 5.0 MB/s | 79 MB 00:15 2024-09-16T14:50:22.089 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:50:22.422 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:50:22.422 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:50:23.387 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:50:23.387 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:50:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:50:26.455 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:50:26.540 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:50:29.327 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:50:29.460 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:50:34.303 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:50:34.577 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:39.408 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:50:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:50:58.214 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:50:59.004 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:50:59.360 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:50:59.451 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.451 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:50:59.451 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:50:59.451 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:50:59.451 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:50:59.451 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-09-16T14:50:59.451 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:50:59.452 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:50:59.511 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:50:59.512 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:50:59.512 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 82.el9 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8274670 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:50:59.554 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2024-09-16T14:50:59.555 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:50:59.556 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:50:59.556 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:50:59.556 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:50:59.557 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:51:00.229 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-09-16T14:51:01.533 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:51:01.564 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-09-16T14:51:01.585 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T14:51:01.601 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.dtCyhEeLr6 2024-09-16T14:51:01.602 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dtCyhEeLr6 2024-09-16T14:51:01.669 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.dtCyhEeLr6 2024-09-16T14:51:01.823 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.dtCyhEeLr6 is 6KB 2024-09-16T14:51:01.872 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.dtCyhEeLr6 2024-09-16T14:51:01.887 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:51:01.950 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:51:01.950 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:51:01.950 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:51:01.950 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:51:01.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:51:02.097 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-09-16T14:51:17.366 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:51:17.409 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:51:17.776 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:51:17.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:51:17.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:51:17.777 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:51:18.633 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:51:18.634 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:51:18.634 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:51:18.989 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:51:19.076 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:51:19.077 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:51:19.078 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:51:19.078 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:51:19.078 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:51:19.078 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T14:51:19.078 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:51:19.078 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:51:19.137 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:51:19.137 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:51:19.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2024-09-16T14:51:19.169 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2024-09-16T14:51:19.169 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2024-09-16T14:51:19.169 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.06 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 82.el9 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Group : Unspecified 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 8274670 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : builder@centos.org 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:51:19.170 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:51:19.171 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:51:19.172 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:51:19.172 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:51:19.172 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:51:19.801 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-09-16T14:51:21.185 INFO:teuthology.orchestra.run.smithi134.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:51:21.212 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-09-16T14:51:21.214 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T14:51:21.230 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.IPQ2POozAC 2024-09-16T14:51:21.230 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IPQ2POozAC 2024-09-16T14:51:21.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.IPQ2POozAC 2024-09-16T14:51:21.460 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.IPQ2POozAC is 6KB 2024-09-16T14:51:21.508 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.IPQ2POozAC 2024-09-16T14:51:21.523 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:51:21.586 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:51:21.586 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:51:21.586 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:51:21.586 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:51:21.587 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:51:21.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-09-16T14:51:32.109 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-09-16T14:51:32.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:51:32.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:51:50.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-16T14:51:51.767 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T14:51:51.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:51:51.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:51:59.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:51:59.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:52:02.525 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-16T14:52:10.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-16T14:52:14.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:52:14.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:52:19.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:52:19.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:52:22.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-16T14:52:34.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:52:34.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:52:52.669 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-09-16T14:53:07.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:53:07.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:53:08.169 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-16T14:53:08.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:53:08.556 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:53:08.556 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-16T14:53:08.571 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:53:08.571 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:53:08.571 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:53:08.571 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:53:09.572 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T14:53:09.572 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:53:09.647 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-09-16T14:53:09.672 DEBUG:teuthology.parallel:result is None 2024-09-16T14:53:14.532 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T14:53:29.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:53:29.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:53:29.886 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-16T14:53:30.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:53:30.241 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:53:30.241 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-16T14:53:30.258 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:53:30.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:53:30.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:53:30.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:53:31.259 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-09-16T14:53:31.260 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:53:31.350 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-09-16T14:53:31.370 DEBUG:teuthology.parallel:result is None 2024-09-16T14:53:31.370 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T14:53:31.377 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T14:53:31.377 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:53:31.380 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:53:31.398 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T14:53:31.473 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T14:53:31.677 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T14:53:31.677 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:53:31.680 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:53:31.707 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T14:53:31.826 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T14:53:31.826 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:53:31.828 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:53:31.858 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:53:31.866 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:53:31.869 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:53:31.878 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:53:31.879 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T14:53:31.915 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T14:53:31.915 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:53:31.918 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:53:31.948 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T14:53:32.026 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T14:53:32.027 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:53:32.029 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:53:32.046 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:53:32.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:53:32.184 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:53:32.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:53:32.312 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T14:53:32.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:53:32.378 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:53:32.427 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:53:32.476 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:53:32.561 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:53:32.604 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T14:53:32.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:53:32.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-16T14:53:32.673 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-09-16T14:53:32.757 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:53:32.758 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:53:33.226 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T14:53:33.234 INFO:teuthology.task.internal:Starting timer... 2024-09-16T14:53:33.234 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T14:53:33.322 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T14:53:33.434 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-16T14:53:33.435 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-16T14:53:33.508 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-16T14:53:33.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-09-16T14:53:33.570 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-09-16T14:53:33.572 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T14:53:33.572 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-16T14:53:33.593 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-16T14:53:33.593 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-09-16T14:53:33.620 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-09-16T14:53:33.620 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi134.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-09-16T14:53:33.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:53:33.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:53:33.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:53:33.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:53:33.691 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T14:53:33.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:53:33.754 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:53:33.792 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T14:53:33.803 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T14:53:33.816 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T14:53:33.817 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi134.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T14:57:18.574 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi134.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-09-16T14:57:18.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:57:18.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:57:18.667 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-16T14:57:18.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-16T14:57:18.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:57:18.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:57:18.823 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-16T14:57:18.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T14:57:18.904 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T14:57:18.914 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T14:57:18.915 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:57:18.915 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:57:18.918 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:57:18.918 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:57:18.945 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:57:18.959 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:57:18.979 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:57:18.984 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-16T14:57:18.995 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-16T14:57:18.995 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:57:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:57:19.022 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-09-16T14:57:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:57:19.034 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-09-16T14:57:19.048 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:57:19.065 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:57:19.081 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:57:19.086 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:57:19.086 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-16T14:57:19.086 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.086 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.086 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.087 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.121 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.125 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:57:19.125 INFO:teuthology.run_tasks:Running task install... 2024-09-16T14:57:19.134 DEBUG:teuthology.task.install:project ceph 2024-09-16T14:57:19.134 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}} 2024-09-16T14:57:19.134 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T14:57:19.134 INFO:teuthology.task.install:Using flavor: default 2024-09-16T14:57:19.145 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T14:57:19.145 INFO:teuthology.task.install:extra packages: [] 2024-09-16T14:57:19.145 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'tag': None, 'wait_for_package': False} 2024-09-16T14:57:19.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:57:19.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'tag': None, 'wait_for_package': False} 2024-09-16T14:57:19.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:57:19.298 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/ 2024-09-16T14:57:19.298 INFO:teuthology.task.install.rpm:Package version is 18.2.4-845.ge4e227b3 2024-09-16T14:57:19.303 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/ 2024-09-16T14:57:19.303 INFO:teuthology.task.install.rpm:Package version is 18.2.4-845.ge4e227b3 2024-09-16T14:57:19.411 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T14:57:19.411 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T14:57:19.411 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T14:57:19.418 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T14:57:19.418 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T14:57:19.418 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T14:57:19.436 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T14:57:19.437 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T14:57:19.444 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T14:57:19.444 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T14:57:19.502 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T14:57:19.511 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T14:57:19.578 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T14:57:19.587 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T14:57:19.643 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-16T14:57:19.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-16T14:57:19.651 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-09-16T14:57:19.653 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-09-16T14:57:19.871 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-16T14:57:19.884 INFO:teuthology.orchestra.run.smithi162.stdout:45 files removed 2024-09-16T14:57:19.898 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw 2024-09-16T14:57:19.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw 2024-09-16T14:57:20.455 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 301 kB/s | 77 kB 00:00 2024-09-16T14:57:20.461 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 278 kB/s | 77 kB 00:00 2024-09-16T14:57:20.745 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-16T14:57:20.748 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-16T14:57:21.012 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T14:57:21.015 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T14:57:21.562 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-16T14:57:22.740 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-16T14:57:30.682 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 20 MB 00:06 2024-09-16T14:57:35.074 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T14:57:36.575 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 110 kB/s | 19 kB 00:00 2024-09-16T14:57:38.354 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-16T14:57:42.720 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:20 2024-09-16T14:57:43.818 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-16T14:57:46.194 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 7.7 M 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.3 M 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:57:46.205 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 5.1 M 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 18 M 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 25 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 712 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T14:57:46.206 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 478 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.5 M 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 45 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 129 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 162 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 321 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 300 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 99 k 2024-09-16T14:57:46.207 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout:Install 28 Packages 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 2 Packages 2024-09-16T14:57:46.208 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:57:46.209 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 53 M 2024-09-16T14:57:46.209 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:57:46.877 INFO:teuthology.orchestra.run.smithi162.stdout:(1/30): ceph-base-18.2.4-845.ge4e227b3.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-16T14:57:46.985 INFO:teuthology.orchestra.run.smithi162.stdout:(2/30): ceph-radosgw-18.2.4-845.ge4e227b3.el9.x 14 MB/s | 7.7 MB 00:00 2024-09-16T14:57:47.011 INFO:teuthology.orchestra.run.smithi162.stdout:(3/30): ceph-selinux-18.2.4-845.ge4e227b3.el9.x 188 kB/s | 25 kB 00:00 2024-09-16T14:57:47.053 INFO:teuthology.orchestra.run.smithi162.stdout:(4/30): libcephfs2-18.2.4-845.ge4e227b3.el9.x86 10 MB/s | 712 kB 00:00 2024-09-16T14:57:47.086 INFO:teuthology.orchestra.run.smithi162.stdout:(5/30): libradosstriper1-18.2.4-845.ge4e227b3.e 6.2 MB/s | 478 kB 00:00 2024-09-16T14:57:47.137 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T14:57:47.162 INFO:teuthology.orchestra.run.smithi162.stdout:(6/30): python3-ceph-argparse-18.2.4-845.ge4e22 601 kB/s | 45 kB 00:00 2024-09-16T14:57:47.253 INFO:teuthology.orchestra.run.smithi162.stdout:(7/30): librgw2-18.2.4-845.ge4e227b3.el9.x86_64 22 MB/s | 4.5 MB 00:00 2024-09-16T14:57:47.279 INFO:teuthology.orchestra.run.smithi162.stdout:(8/30): python3-ceph-common-18.2.4-845.ge4e227b 1.1 MB/s | 129 kB 00:00 2024-09-16T14:57:47.304 INFO:teuthology.orchestra.run.smithi162.stdout:(9/30): python3-cephfs-18.2.4-845.ge4e227b3.el9 3.2 MB/s | 162 kB 00:00 2024-09-16T14:57:47.338 INFO:teuthology.orchestra.run.smithi162.stdout:(10/30): python3-rados-18.2.4-845.ge4e227b3.el9 5.4 MB/s | 321 kB 00:00 2024-09-16T14:57:47.363 INFO:teuthology.orchestra.run.smithi162.stdout:(11/30): python3-rbd-18.2.4-845.ge4e227b3.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-16T14:57:47.396 INFO:teuthology.orchestra.run.smithi162.stdout:(12/30): python3-rgw-18.2.4-845.ge4e227b3.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-16T14:57:47.622 INFO:teuthology.orchestra.run.smithi162.stdout:(13/30): ceph-common-18.2.4-845.ge4e227b3.el9.x 15 MB/s | 18 MB 00:01 2024-09-16T14:57:47.672 INFO:teuthology.orchestra.run.smithi162.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-16T14:57:47.706 INFO:teuthology.orchestra.run.smithi162.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 519 kB/s | 160 kB 00:00 2024-09-16T14:57:47.731 INFO:teuthology.orchestra.run.smithi162.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-16T14:57:47.798 INFO:teuthology.orchestra.run.smithi162.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 633 kB/s | 42 kB 00:00 2024-09-16T14:57:47.840 INFO:teuthology.orchestra.run.smithi162.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-16T14:57:47.873 INFO:teuthology.orchestra.run.smithi162.stdout:(19/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 9.1 MB/s | 308 kB 00:00 2024-09-16T14:57:47.915 INFO:teuthology.orchestra.run.smithi162.stdout:(20/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.4 MB/s | 292 kB 00:00 2024-09-16T14:57:47.949 INFO:teuthology.orchestra.run.smithi162.stdout:(21/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-16T14:57:48.001 INFO:teuthology.orchestra.run.smithi162.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 303 kB/s | 26 kB 00:00 2024-09-16T14:57:48.035 INFO:teuthology.orchestra.run.smithi162.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 618 kB/s | 49 kB 00:00 2024-09-16T14:57:48.060 INFO:teuthology.orchestra.run.smithi162.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-16T14:57:48.102 INFO:teuthology.orchestra.run.smithi162.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T14:57:48.194 INFO:teuthology.orchestra.run.smithi162.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-09-16T14:57:48.303 INFO:teuthology.orchestra.run.smithi162.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-09-16T14:57:48.370 INFO:teuthology.orchestra.run.smithi162.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-16T14:57:48.428 INFO:teuthology.orchestra.run.smithi162.stdout:(29/30): librados2-18.2.4-845.ge4e227b3.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-16T14:57:48.528 INFO:teuthology.orchestra.run.smithi162.stdout:(30/30): librbd1-18.2.4-845.ge4e227b3.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-16T14:57:48.534 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:57:48.534 INFO:teuthology.orchestra.run.smithi162.stdout:Total 23 MB/s | 53 MB 00:02 2024-09-16T14:57:48.546 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-16T14:57:48.605 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:57:48.640 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:57:48.640 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:57:49.262 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:57:49.263 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:57:50.341 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-16T14:57:50.635 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:57:50.712 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T14:57:50.750 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T14:57:50.801 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T14:57:51.112 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T14:57:51.224 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:57:51.316 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:57:51.408 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:57:51.434 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:57:51.497 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:57:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T14:57:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 9/32 2024-09-16T14:57:51.635 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:57:51.661 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:57:51.727 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:57:51.834 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T14:57:51.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T14:57:51.918 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T14:57:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T14:57:51.986 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T14:57:52.022 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T14:57:52.057 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T14:57:52.367 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T14:57:52.418 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:57:52.508 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:57:52.539 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 21/32 2024-09-16T14:57:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T14:57:52.983 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T14:57:53.012 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:57:53.083 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:57:53.152 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 25/32 2024-09-16T14:57:53.181 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 26/32 2024-09-16T14:57:55.281 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:57:55.320 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:57:55.781 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-16T14:57:55.872 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:57:55.955 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:57:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:57:56.038 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T14:57:56.038 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T14:57:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:57:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:57:58.131 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:57:58.142 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 7.7 M 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.3 M 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 5.1 M 2024-09-16T14:57:58.143 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 18 M 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 25 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 712 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 478 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.5 M 2024-09-16T14:57:58.144 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 45 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 129 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 162 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 321 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 300 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 99 k 2024-09-16T14:57:58.145 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Install 28 Packages 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 2 Packages 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:57:58.146 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 53 M 2024-09-16T14:57:58.147 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:57:58.890 INFO:teuthology.orchestra.run.smithi134.stdout:(1/30): ceph-base-18.2.4-845.ge4e227b3.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-16T14:57:58.949 INFO:teuthology.orchestra.run.smithi134.stdout:(2/30): ceph-selinux-18.2.4-845.ge4e227b3.el9.x 432 kB/s | 25 kB 00:00 2024-09-16T14:57:59.066 INFO:teuthology.orchestra.run.smithi134.stdout:(3/30): ceph-radosgw-18.2.4-845.ge4e227b3.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-16T14:57:59.099 INFO:teuthology.orchestra.run.smithi134.stdout:(4/30): libcephfs2-18.2.4-845.ge4e227b3.el9.x86 4.6 MB/s | 712 kB 00:00 2024-09-16T14:57:59.133 INFO:teuthology.orchestra.run.smithi134.stdout:(5/30): libradosstriper1-18.2.4-845.ge4e227b3.e 7.0 MB/s | 478 kB 00:00 2024-09-16T14:57:59.192 INFO:teuthology.orchestra.run.smithi134.stdout:(6/30): python3-ceph-argparse-18.2.4-845.ge4e22 770 kB/s | 45 kB 00:00 2024-09-16T14:57:59.275 INFO:teuthology.orchestra.run.smithi134.stdout:(7/30): python3-ceph-common-18.2.4-845.ge4e227b 1.5 MB/s | 129 kB 00:00 2024-09-16T14:57:59.342 INFO:teuthology.orchestra.run.smithi134.stdout:(8/30): librgw2-18.2.4-845.ge4e227b3.el9.x86_64 18 MB/s | 4.5 MB 00:00 2024-09-16T14:57:59.368 INFO:teuthology.orchestra.run.smithi134.stdout:(9/30): python3-cephfs-18.2.4-845.ge4e227b3.el9 1.7 MB/s | 162 kB 00:00 2024-09-16T14:57:59.401 INFO:teuthology.orchestra.run.smithi134.stdout:(10/30): python3-rados-18.2.4-845.ge4e227b3.el9 5.3 MB/s | 321 kB 00:00 2024-09-16T14:57:59.426 INFO:teuthology.orchestra.run.smithi134.stdout:(11/30): python3-rbd-18.2.4-845.ge4e227b3.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-16T14:57:59.452 INFO:teuthology.orchestra.run.smithi134.stdout:(12/30): python3-rgw-18.2.4-845.ge4e227b3.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-16T14:57:59.627 INFO:teuthology.orchestra.run.smithi134.stdout:(13/30): ceph-common-18.2.4-845.ge4e227b3.el9.x 15 MB/s | 18 MB 00:01 2024-09-16T14:57:59.744 INFO:teuthology.orchestra.run.smithi134.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-16T14:57:59.794 INFO:teuthology.orchestra.run.smithi134.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 291 kB/s | 107 kB 00:00 2024-09-16T14:57:59.820 INFO:teuthology.orchestra.run.smithi134.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-16T14:57:59.895 INFO:teuthology.orchestra.run.smithi134.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 565 kB/s | 42 kB 00:00 2024-09-16T14:57:59.928 INFO:teuthology.orchestra.run.smithi134.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-16T14:57:59.962 INFO:teuthology.orchestra.run.smithi134.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-16T14:58:00.012 INFO:teuthology.orchestra.run.smithi134.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 946 kB/s | 111 kB 00:00 2024-09-16T14:58:00.038 INFO:teuthology.orchestra.run.smithi134.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-16T14:58:00.062 INFO:teuthology.orchestra.run.smithi134.stdout:(22/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-16T14:58:00.080 INFO:teuthology.orchestra.run.smithi134.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-16T14:58:00.105 INFO:teuthology.orchestra.run.smithi134.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-16T14:58:00.155 INFO:teuthology.orchestra.run.smithi134.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-16T14:58:00.197 INFO:teuthology.orchestra.run.smithi134.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-16T14:58:00.364 INFO:teuthology.orchestra.run.smithi134.stdout:(27/30): librados2-18.2.4-845.ge4e227b3.el9.x86 20 MB/s | 3.3 MB 00:00 2024-09-16T14:58:00.406 INFO:teuthology.orchestra.run.smithi134.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-16T14:58:00.523 INFO:teuthology.orchestra.run.smithi134.stdout:(29/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-09-16T14:58:00.581 INFO:teuthology.orchestra.run.smithi134.stdout:(30/30): librbd1-18.2.4-845.ge4e227b3.el9.x86_6 14 MB/s | 3.0 MB 00:00 2024-09-16T14:58:00.586 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:00.586 INFO:teuthology.orchestra.run.smithi134.stdout:Total 22 MB/s | 53 MB 00:02 2024-09-16T14:58:00.659 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:00.695 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:00.696 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:01.348 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:01.349 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:02.795 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:02.892 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T14:58:02.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T14:58:02.978 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T14:58:03.293 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T14:58:03.385 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:58:03.468 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:58:03.552 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:58:03.578 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:58:03.624 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:58:03.653 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T14:58:03.696 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 9/32 2024-09-16T14:58:03.763 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:58:03.788 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:58:03.854 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:58:03.960 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T14:58:04.005 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T14:58:04.045 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T14:58:04.077 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T14:58:04.104 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T14:58:04.149 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T14:58:04.182 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T14:58:04.502 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T14:58:04.544 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:58:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:58:04.663 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 21/32 2024-09-16T14:58:04.702 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T14:58:05.108 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T14:58:05.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:58:05.201 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:58:05.270 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 25/32 2024-09-16T14:58:05.307 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 26/32 2024-09-16T14:58:05.867 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:58:05.867 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-09-16T14:58:05.867 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-09-16T14:58:05.867 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-09-16T14:58:05.868 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-09-16T14:58:05.868 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-09-16T14:58:05.868 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-09-16T14:58:05.868 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-09-16T14:58:05.868 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:06.639 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:58:06.665 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:06.666 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:58:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:58:06.742 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:07.449 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:58:07.487 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:58:07.885 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:07.885 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 1/32 2024-09-16T14:58:07.885 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2/32 2024-09-16T14:58:07.885 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 3/32 2024-09-16T14:58:07.885 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 4/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 8/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 9/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 12/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 13/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T14:58:07.886 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T14:58:07.887 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T14:58:07.888 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T14:58:07.888 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:58:07.888 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T14:58:07.888 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 31/32 2024-09-16T14:58:08.034 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:58:08.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:58:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:58:08.176 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T14:58:08.176 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T14:58:08.176 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:08.208 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:58:08.868 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:08.868 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:08.868 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T14:58:08.869 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T14:58:08.870 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:08.871 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:09.261 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-test 2024-09-16T14:58:09.832 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:26 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:09.963 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 40 M 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:09.964 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 166 k 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-16T14:58:09.965 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:09.966 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 40 M 2024-09-16T14:58:09.966 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 164 M 2024-09-16T14:58:09.966 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:10.158 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-16T14:58:10.192 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-16T14:58:10.242 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): libcephsqlite-18.2.4-845.ge4e227b3.el9.x 726 kB/s | 166 kB 00:00 2024-09-16T14:58:11.301 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): ceph-test-18.2.4-845.ge4e227b3.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-16T14:58:11.303 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:11.303 INFO:teuthology.orchestra.run.smithi162.stdout:Total 30 MB/s | 40 MB 00:01 2024-09-16T14:58:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:11.351 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:11.351 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:11.678 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:11.678 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:12.236 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T14:58:12.276 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T14:58:12.302 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:58:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:58:16.109 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:58:18.188 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:58:18.188 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 1/4 2024-09-16T14:58:18.188 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2/4 2024-09-16T14:58:18.188 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T14:58:18.248 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:58:18.248 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-16T14:58:18.249 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:18.499 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:18.829 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph 2024-09-16T14:58:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:58:19.055 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:58:19.055 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:19.056 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T14:58:19.056 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:58:19.056 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:58:19.056 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:19.057 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:58:19.140 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:58:19.140 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:19.401 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:36 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:19.538 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:19.540 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 6.5 k 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 2.1 M 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 1.5 M 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 246 k 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.7 M 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 17 M 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T14:58:19.541 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T14:58:19.542 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:19.543 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:19.544 INFO:teuthology.orchestra.run.smithi162.stdout:Install 33 Packages 2024-09-16T14:58:19.544 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:19.544 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 29 M 2024-09-16T14:58:19.544 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 104 M 2024-09-16T14:58:19.544 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:20.288 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:20.288 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 1/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 3/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 4/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 8/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 9/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:58:20.289 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 12/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 13/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T14:58:20.290 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:58:20.291 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T14:58:20.292 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 31/32 2024-09-16T14:58:20.328 INFO:teuthology.orchestra.run.smithi162.stdout:(1/33): ceph-18.2.4-845.ge4e227b3.el9.x86_64.rp 41 kB/s | 6.5 kB 00:00 2024-09-16T14:58:20.528 INFO:teuthology.orchestra.run.smithi162.stdout:(2/33): ceph-mgr-18.2.4-845.ge4e227b3.el9.x86_6 4.0 MB/s | 1.5 MB 00:00 2024-09-16T14:58:20.578 INFO:teuthology.orchestra.run.smithi162.stdout:(3/33): ceph-mds-18.2.4-845.ge4e227b3.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-09-16T14:58:20.678 INFO:teuthology.orchestra.run.smithi162.stdout:(4/33): ceph-mgr-modules-core-18.2.4-845.ge4e22 2.4 MB/s | 246 kB 00:00 2024-09-16T14:58:20.912 INFO:teuthology.orchestra.run.smithi162.stdout:(5/33): ceph-mon-18.2.4-845.ge4e227b3.el9.x86_6 8.0 MB/s | 4.7 MB 00:00 2024-09-16T14:58:20.937 INFO:teuthology.orchestra.run.smithi162.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 980 kB/s | 253 kB 00:00 2024-09-16T14:58:21.129 INFO:teuthology.orchestra.run.smithi162.stdout:(7/33): ceph-osd-18.2.4-845.ge4e227b3.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-16T14:58:21.154 INFO:teuthology.orchestra.run.smithi162.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-16T14:58:21.205 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:58:21.205 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:21.205 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-16T14:58:21.205 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.205 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T14:58:21.206 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.207 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:21.208 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:21.255 INFO:teuthology.orchestra.run.smithi162.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.2 MB/s | 125 kB 00:00 2024-09-16T14:58:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 705 kB/s | 135 kB 00:00 2024-09-16T14:58:21.347 INFO:teuthology.orchestra.run.smithi162.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-09-16T14:58:21.464 INFO:teuthology.orchestra.run.smithi162.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-16T14:58:21.539 INFO:teuthology.orchestra.run.smithi162.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 793 kB/s | 172 kB 00:00 2024-09-16T14:58:21.556 INFO:teuthology.orchestra.run.smithi162.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 554 kB/s | 51 kB 00:00 2024-09-16T14:58:21.577 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-test 2024-09-16T14:58:21.606 INFO:teuthology.orchestra.run.smithi162.stdout:(15/33): python3-cryptography-36.0.1-4.el9.x86_ 1.8 MB/s | 1.2 MB 00:00 2024-09-16T14:58:21.624 INFO:teuthology.orchestra.run.smithi162.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 495 kB/s | 42 kB 00:00 2024-09-16T14:58:21.649 INFO:teuthology.orchestra.run.smithi162.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 470 kB/s | 43 kB 00:00 2024-09-16T14:58:21.674 INFO:teuthology.orchestra.run.smithi162.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 689 kB/s | 11 kB 00:00 2024-09-16T14:58:21.699 INFO:teuthology.orchestra.run.smithi162.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2024-09-16T14:58:21.716 INFO:teuthology.orchestra.run.smithi162.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 440 kB/s | 18 kB 00:00 2024-09-16T14:58:21.733 INFO:teuthology.orchestra.run.smithi162.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 690 kB/s | 23 kB 00:00 2024-09-16T14:58:21.750 INFO:teuthology.orchestra.run.smithi162.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 578 kB/s | 19 kB 00:00 2024-09-16T14:58:21.767 INFO:teuthology.orchestra.run.smithi162.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 588 kB/s | 20 kB 00:00 2024-09-16T14:58:21.784 INFO:teuthology.orchestra.run.smithi162.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-16T14:58:21.809 INFO:teuthology.orchestra.run.smithi162.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-16T14:58:21.838 INFO:teuthology.orchestra.run.smithi162.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-16T14:58:21.855 INFO:teuthology.orchestra.run.smithi162.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 362 kB/s | 16 kB 00:00 2024-09-16T14:58:21.881 INFO:teuthology.orchestra.run.smithi162.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-16T14:58:21.897 INFO:teuthology.orchestra.run.smithi162.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 855 kB/s | 36 kB 00:00 2024-09-16T14:58:21.923 INFO:teuthology.orchestra.run.smithi162.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-16T14:58:21.940 INFO:teuthology.orchestra.run.smithi162.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-16T14:58:21.974 INFO:teuthology.orchestra.run.smithi162.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-16T14:58:21.999 INFO:teuthology.orchestra.run.smithi162.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-16T14:58:22.004 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:22.004 INFO:teuthology.orchestra.run.smithi162.stdout:Total 12 MB/s | 29 MB 00:02 2024-09-16T14:58:22.127 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:22.144 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:22.165 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:22.165 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:22.278 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:58:22.278 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 40 M 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 166 k 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout:Install 4 Packages 2024-09-16T14:58:22.279 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 40 M 2024-09-16T14:58:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 164 M 2024-09-16T14:58:22.280 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:22.577 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:22.578 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:22.604 INFO:teuthology.orchestra.run.smithi134.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T14:58:22.629 INFO:teuthology.orchestra.run.smithi134.stdout:(2/4): libcephsqlite-18.2.4-845.ge4e227b3.el9.x 682 kB/s | 166 kB 00:00 2024-09-16T14:58:22.662 INFO:teuthology.orchestra.run.smithi134.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-16T14:58:23.671 INFO:teuthology.orchestra.run.smithi134.stdout:(4/4): ceph-test-18.2.4-845.ge4e227b3.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-16T14:58:23.673 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:23.673 INFO:teuthology.orchestra.run.smithi134.stdout:Total 29 MB/s | 40 MB 00:01 2024-09-16T14:58:23.700 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:23.719 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:23.719 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:23.818 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:23.885 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T14:58:23.917 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T14:58:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T14:58:24.048 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:24.049 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T14:58:24.137 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T14:58:24.178 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T14:58:24.213 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T14:58:24.243 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T14:58:24.277 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T14:58:24.311 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T14:58:24.352 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T14:58:24.387 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T14:58:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T14:58:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T14:58:24.533 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T14:58:24.547 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:24.623 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T14:58:24.662 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T14:58:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T14:58:24.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:58:24.758 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T14:58:24.812 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T14:58:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T14:58:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T14:58:24.956 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T14:58:25.142 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T14:58:25.194 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T14:58:25.272 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T14:58:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T14:58:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T14:58:25.526 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T14:58:25.678 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 28/33 2024-09-16T14:58:25.704 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:58:25.730 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:27.181 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:58:27.209 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:58:27.210 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:27.210 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T14:58:27.210 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:58:27.210 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:58:27.210 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:27.656 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:58:27.681 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:27.890 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:58:27.914 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:58:27.914 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:27.915 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T14:58:27.915 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:58:27.915 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:58:27.915 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:58:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:58:28.453 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 1/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 3/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 4/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 5/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 6/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T14:58:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T14:58:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T14:58:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T14:58:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T14:58:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T14:58:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T14:58:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T14:58:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T14:58:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T14:58:30.446 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:58:30.446 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 1/4 2024-09-16T14:58:30.446 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2/4 2024-09-16T14:58:30.446 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T14:58:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T14:58:30.542 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T14:58:30.543 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T14:58:30.544 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:30.545 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T14:58:30.734 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:30.735 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:30.946 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-base 2024-09-16T14:58:31.070 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph 2024-09-16T14:58:31.514 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:31.603 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:58:31.635 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:36 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:31.645 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:31.646 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:58:31.646 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:31.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install cephadm 2024-09-16T14:58:31.774 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:31.776 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 6.5 k 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 2.1 M 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 1.5 M 2024-09-16T14:58:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 246 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.7 M 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 17 M 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T14:58:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T14:58:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Install 33 Packages 2024-09-16T14:58:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:31.781 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 29 M 2024-09-16T14:58:31.781 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 104 M 2024-09-16T14:58:31.781 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:32.078 INFO:teuthology.orchestra.run.smithi134.stdout:(1/33): ceph-18.2.4-845.ge4e227b3.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-16T14:58:32.296 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:49 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:32.429 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:32.429 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 226 k 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 226 k 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 217 k 2024-09-16T14:58:32.430 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:32.437 INFO:teuthology.orchestra.run.smithi134.stdout:(2/33): ceph-mds-18.2.4-845.ge4e227b3.el9.x86_6 4.4 MB/s | 2.1 MB 00:00 2024-09-16T14:58:32.470 INFO:teuthology.orchestra.run.smithi134.stdout:(3/33): ceph-mgr-18.2.4-845.ge4e227b3.el9.x86_6 2.8 MB/s | 1.5 MB 00:00 2024-09-16T14:58:32.546 INFO:teuthology.orchestra.run.smithi134.stdout:(4/33): ceph-mgr-modules-core-18.2.4-845.ge4e22 3.2 MB/s | 246 kB 00:00 2024-09-16T14:58:32.658 INFO:teuthology.orchestra.run.smithi162.stdout:cephadm-18.2.4-845.ge4e227b3.el9.noarch.rpm 995 kB/s | 226 kB 00:00 2024-09-16T14:58:32.658 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:32.659 INFO:teuthology.orchestra.run.smithi162.stdout:Total 988 kB/s | 226 kB 00:00 2024-09-16T14:58:32.659 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:32.664 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:32.664 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:32.667 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:32.667 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:32.739 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:32.754 INFO:teuthology.orchestra.run.smithi134.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-16T14:58:32.988 INFO:teuthology.orchestra.run.smithi134.stdout:(6/33): ceph-mon-18.2.4-845.ge4e227b3.el9.x86_6 5.1 MB/s | 4.7 MB 00:00 2024-09-16T14:58:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:33.138 INFO:teuthology.orchestra.run.smithi134.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 710 kB/s | 106 kB 00:00 2024-09-16T14:58:33.212 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:33.355 INFO:teuthology.orchestra.run.smithi134.stdout:(8/33): ceph-osd-18.2.4-845.ge4e227b3.el9.x86_6 18 MB/s | 17 MB 00:00 2024-09-16T14:58:33.380 INFO:teuthology.orchestra.run.smithi134.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 559 kB/s | 135 kB 00:00 2024-09-16T14:58:33.414 INFO:teuthology.orchestra.run.smithi134.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 1.9 MB/s | 1.2 MB 00:00 2024-09-16T14:58:33.464 INFO:teuthology.orchestra.run.smithi134.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-16T14:58:33.522 INFO:teuthology.orchestra.run.smithi134.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 747 kB/s | 125 kB 00:00 2024-09-16T14:58:33.545 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:33.564 INFO:teuthology.orchestra.run.smithi134.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-16T14:58:33.640 INFO:teuthology.orchestra.run.smithi134.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-09-16T14:58:33.665 INFO:teuthology.orchestra.run.smithi134.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 686 kB/s | 172 kB 00:00 2024-09-16T14:58:33.682 INFO:teuthology.orchestra.run.smithi134.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 320 kB/s | 51 kB 00:00 2024-09-16T14:58:33.732 INFO:teuthology.orchestra.run.smithi134.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 470 kB/s | 43 kB 00:00 2024-09-16T14:58:33.758 INFO:teuthology.orchestra.run.smithi134.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-16T14:58:33.783 INFO:teuthology.orchestra.run.smithi134.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-16T14:58:33.808 INFO:teuthology.orchestra.run.smithi134.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-16T14:58:33.820 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:33.820 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:33.821 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:33.821 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:33.821 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:33.821 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:33.833 INFO:teuthology.orchestra.run.smithi134.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-16T14:58:33.859 INFO:teuthology.orchestra.run.smithi134.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-16T14:58:33.884 INFO:teuthology.orchestra.run.smithi134.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-16T14:58:33.909 INFO:teuthology.orchestra.run.smithi134.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-16T14:58:33.934 INFO:teuthology.orchestra.run.smithi134.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T14:58:33.960 INFO:teuthology.orchestra.run.smithi134.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-16T14:58:33.985 INFO:teuthology.orchestra.run.smithi134.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-16T14:58:34.010 INFO:teuthology.orchestra.run.smithi134.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T14:58:34.035 INFO:teuthology.orchestra.run.smithi134.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-16T14:58:34.060 INFO:teuthology.orchestra.run.smithi134.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T14:58:34.076 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T14:58:34.094 INFO:teuthology.orchestra.run.smithi134.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T14:58:34.119 INFO:teuthology.orchestra.run.smithi134.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-16T14:58:34.145 INFO:teuthology.orchestra.run.smithi134.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T14:58:34.148 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:34.149 INFO:teuthology.orchestra.run.smithi134.stdout:Total 12 MB/s | 29 MB 00:02 2024-09-16T14:58:34.265 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:34.303 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:34.303 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:34.657 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:51 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:34.713 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:34.714 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 143 k 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:34.792 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 143 k 2024-09-16T14:58:34.793 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 395 k 2024-09-16T14:58:34.793 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:34.998 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-immutable-object-cache-18.2.4-845.ge4e227b 695 kB/s | 143 kB 00:00 2024-09-16T14:58:34.999 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:34.999 INFO:teuthology.orchestra.run.smithi162.stdout:Total 690 kB/s | 143 kB 00:00 2024-09-16T14:58:34.999 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:35.006 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:35.007 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:35.063 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:35.063 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:35.266 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:35.291 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:35.292 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:35.292 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T14:58:35.292 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:36.106 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T14:58:36.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T14:58:36.163 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T14:58:36.255 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T14:58:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:36.306 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:36.306 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:36.307 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:36.307 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:36.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T14:58:36.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T14:58:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T14:58:36.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T14:58:36.473 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T14:58:36.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T14:58:36.535 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr 2024-09-16T14:58:36.541 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T14:58:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T14:58:36.610 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T14:58:36.649 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T14:58:36.721 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T14:58:36.860 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T14:58:36.948 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T14:58:37.001 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T14:58:37.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T14:58:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T14:58:37.106 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:37.152 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T14:58:37.199 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:58:37.242 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:37.243 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:58:37.244 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:37.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T14:58:37.339 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T14:58:37.393 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T14:58:37.463 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T14:58:37.551 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T14:58:37.601 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T14:58:37.716 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T14:58:37.866 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 28/33 2024-09-16T14:58:37.891 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:37.894 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:58:37.919 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:58:37.919 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:37.919 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T14:58:37.919 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:58:37.920 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:58:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:38.023 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:38.023 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:38.023 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 3.5 M 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 24 k 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 16 k 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout:Install 5 Packages 2024-09-16T14:58:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:38.025 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 3.8 M 2024-09-16T14:58:38.025 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 87 M 2024-09-16T14:58:38.025 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:38.212 INFO:teuthology.orchestra.run.smithi162.stdout:(1/5): ceph-prometheus-alerts-18.2.4-845.ge4e22 107 kB/s | 16 kB 00:00 2024-09-16T14:58:38.229 INFO:teuthology.orchestra.run.smithi162.stdout:(2/5): ceph-grafana-dashboards-18.2.4-845.ge4e2 142 kB/s | 24 kB 00:00 2024-09-16T14:58:38.262 INFO:teuthology.orchestra.run.smithi162.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 612 kB/s | 31 kB 00:00 2024-09-16T14:58:38.296 INFO:teuthology.orchestra.run.smithi162.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-16T14:58:38.480 INFO:teuthology.orchestra.run.smithi162.stdout:(5/5): ceph-mgr-dashboard-18.2.4-845.ge4e227b3. 8.4 MB/s | 3.5 MB 00:00 2024-09-16T14:58:38.483 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:Total 8.2 MB/s | 3.8 MB 00:00 2024-09-16T14:58:38.484 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:38.491 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:38.491 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:38.569 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:38.569 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:38.820 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T14:58:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T14:58:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:58:39.395 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:58:39.425 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:39.871 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:58:39.895 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:58:39.896 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:39.896 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T14:58:39.896 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:58:39.896 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:58:39.896 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:40.020 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 4/5 2024-09-16T14:58:40.052 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:58:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:58:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:58:40.120 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:40.121 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T14:58:40.121 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:58:40.121 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:58:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:40.184 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:58:41.153 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:58:41.153 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 1/5 2024-09-16T14:58:41.153 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2/5 2024-09-16T14:58:41.153 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:58:41.153 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T14:58:41.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:41.436 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:41.658 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T14:58:41.934 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:58:41.934 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 1/33 2024-09-16T14:58:41.934 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2/33 2024-09-16T14:58:41.934 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 3/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 4/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 5/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 6/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T14:58:41.935 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T14:58:41.936 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T14:58:41.937 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T14:58:41.937 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T14:58:41.937 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T14:58:41.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T14:58:41.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T14:58:41.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T14:58:41.938 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T14:58:42.223 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:42.357 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:42.358 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 7.4 M 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T14:58:42.359 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout:Install 12 Packages 2024-09-16T14:58:42.360 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:42.361 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 38 M 2024-09-16T14:58:42.361 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 212 M 2024-09-16T14:58:42.361 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:42.922 INFO:teuthology.orchestra.run.smithi162.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-84 14 MB/s | 7.4 MB 00:00 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:42.963 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T14:58:42.964 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T14:58:42.965 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T14:58:42.966 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T14:58:42.966 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T14:58:42.966 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:42.966 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:42.966 INFO:teuthology.orchestra.run.smithi162.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 346 kB/s | 191 kB 00:00 2024-09-16T14:58:43.014 INFO:teuthology.orchestra.run.smithi162.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-16T14:58:43.081 INFO:teuthology.orchestra.run.smithi162.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-09-16T14:58:43.140 INFO:teuthology.orchestra.run.smithi162.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-16T14:58:43.224 INFO:teuthology.orchestra.run.smithi162.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-09-16T14:58:43.257 INFO:teuthology.orchestra.run.smithi162.stdout:(7/12): libgfortran-11.5.0-2.el9.x86_64.rpm 948 kB/s | 801 kB 00:00 2024-09-16T14:58:43.299 INFO:teuthology.orchestra.run.smithi162.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-16T14:58:43.360 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-base 2024-09-16T14:58:43.408 INFO:teuthology.orchestra.run.smithi162.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2024-09-16T14:58:43.433 INFO:teuthology.orchestra.run.smithi162.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-16T14:58:43.817 INFO:teuthology.orchestra.run.smithi162.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-09-16T14:58:43.934 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:44.024 INFO:teuthology.orchestra.run.smithi134.stdout:Package ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:58:44.066 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:44.067 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:58:44.067 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:44.150 INFO:teuthology.orchestra.run.smithi162.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.7 MB/s | 5.1 MB 00:00 2024-09-16T14:58:44.151 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:44.151 INFO:teuthology.orchestra.run.smithi162.stdout:Total 21 MB/s | 38 MB 00:01 2024-09-16T14:58:44.153 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install cephadm 2024-09-16T14:58:44.506 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:44.531 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:44.531 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:44.720 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:49 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:44.852 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 226 k 2024-09-16T14:58:44.853 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 226 k 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 217 k 2024-09-16T14:58:44.854 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:45.021 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:45.021 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:45.078 INFO:teuthology.orchestra.run.smithi134.stdout:cephadm-18.2.4-845.ge4e227b3.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-16T14:58:45.079 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:45.079 INFO:teuthology.orchestra.run.smithi134.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-16T14:58:45.079 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:45.084 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:45.084 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:45.087 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:45.087 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:45.153 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:45.615 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:45.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:45.774 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:45.836 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T14:58:45.873 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:58:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:58:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T14:58:46.057 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:46.205 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T14:58:46.232 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T14:58:46.310 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:46.333 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:46.567 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T14:58:46.989 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T14:58:47.104 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T14:58:47.137 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:47.273 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repo Size 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 143 k 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:58:47.274 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:47.275 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 143 k 2024-09-16T14:58:47.275 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 395 k 2024-09-16T14:58:47.275 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:47.500 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-immutable-object-cache-18.2.4-845.ge4e227b 635 kB/s | 143 kB 00:00 2024-09-16T14:58:47.500 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:47.500 INFO:teuthology.orchestra.run.smithi134.stdout:Total 630 kB/s | 143 kB 00:00 2024-09-16T14:58:47.500 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:47.508 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:47.508 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:47.565 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:47.565 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:47.717 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:47.784 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:47.808 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:47.808 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:58:47.808 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T14:58:47.808 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:48.751 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:48.978 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-mgr 2024-09-16T14:58:49.415 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T14:58:49.441 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:58:49.560 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:49.662 INFO:teuthology.orchestra.run.smithi134.stdout:Package ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:58:49.706 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:49.707 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:58:49.707 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:49.787 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T14:58:50.346 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:50.477 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:50.477 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:50.477 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:58:50.477 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 3.5 M 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 24 k 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 16 k 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout:Install 5 Packages 2024-09-16T14:58:50.478 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:50.479 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 3.8 M 2024-09-16T14:58:50.479 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 87 M 2024-09-16T14:58:50.479 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:51.044 INFO:teuthology.orchestra.run.smithi134.stdout:(1/5): ceph-grafana-dashboards-18.2.4-845.ge4e2 162 kB/s | 24 kB 00:00 2024-09-16T14:58:51.069 INFO:teuthology.orchestra.run.smithi134.stdout:(2/5): ceph-prometheus-alerts-18.2.4-845.ge4e22 94 kB/s | 16 kB 00:00 2024-09-16T14:58:51.119 INFO:teuthology.orchestra.run.smithi134.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 410 kB/s | 31 kB 00:00 2024-09-16T14:58:51.152 INFO:teuthology.orchestra.run.smithi134.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T14:58:51.294 INFO:teuthology.orchestra.run.smithi134.stdout:(5/5): ceph-mgr-dashboard-18.2.4-845.ge4e227b3. 8.8 MB/s | 3.5 MB 00:00 2024-09-16T14:58:51.298 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:51.298 INFO:teuthology.orchestra.run.smithi134.stdout:Total 4.6 MB/s | 3.8 MB 00:00 2024-09-16T14:58:51.299 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:58:51.306 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:58:51.306 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:58:51.389 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:58:51.389 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:58:51.644 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:58:51.725 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T14:58:51.757 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T14:58:51.786 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 1/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T14:58:52.005 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T14:58:52.006 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T14:58:52.006 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T14:58:52.006 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T14:58:52.006 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:58:52.006 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T14:58:52.410 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:52.411 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:52.812 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-rook 2024-09-16T14:58:52.818 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 4/5 2024-09-16T14:58:52.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:58:53.393 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:10 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:53.529 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 49 k 2024-09-16T14:58:53.530 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:53.531 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:53.532 INFO:teuthology.orchestra.run.smithi162.stdout:Install 13 Packages 2024-09-16T14:58:53.532 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:53.532 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.2 M 2024-09-16T14:58:53.532 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 26 M 2024-09-16T14:58:53.532 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:58:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 1/5 2024-09-16T14:58:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2/5 2024-09-16T14:58:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:58:53.928 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T14:58:53.972 INFO:teuthology.orchestra.run.smithi162.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 164 kB/s | 19 kB 00:00 2024-09-16T14:58:54.001 INFO:teuthology.orchestra.run.smithi162.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 175 kB/s | 26 kB 00:00 2024-09-16T14:58:54.060 INFO:teuthology.orchestra.run.smithi162.stdout:(3/13): ceph-mgr-rook-18.2.4-845.ge4e227b3.el9. 240 kB/s | 49 kB 00:00 2024-09-16T14:58:54.085 INFO:teuthology.orchestra.run.smithi162.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-16T14:58:54.110 INFO:teuthology.orchestra.run.smithi162.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-16T14:58:54.136 INFO:teuthology.orchestra.run.smithi162.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-16T14:58:54.186 INFO:teuthology.orchestra.run.smithi162.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 427 kB/s | 32 kB 00:00 2024-09-16T14:58:54.187 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T14:58:54.187 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:54.188 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:58:54.211 INFO:teuthology.orchestra.run.smithi162.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 188 kB/s | 14 kB 00:00 2024-09-16T14:58:54.237 INFO:teuthology.orchestra.run.smithi162.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-16T14:58:54.270 INFO:teuthology.orchestra.run.smithi162.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T14:58:54.304 INFO:teuthology.orchestra.run.smithi162.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-09-16T14:58:54.329 INFO:teuthology.orchestra.run.smithi162.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-16T14:58:54.363 INFO:teuthology.orchestra.run.smithi162.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T14:58:54.365 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:54.365 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-09-16T14:58:54.404 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T14:58:54.429 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:54.445 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:54.445 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:54.535 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:54.536 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:54.967 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:58:55.045 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:58:55.099 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 7.4 M 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:58:55.101 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:55.102 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:58:55.103 INFO:teuthology.orchestra.run.smithi134.stdout:====================================================================================== 2024-09-16T14:58:55.103 INFO:teuthology.orchestra.run.smithi134.stdout:Install 12 Packages 2024-09-16T14:58:55.103 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:58:55.104 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 38 M 2024-09-16T14:58:55.104 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 212 M 2024-09-16T14:58:55.104 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:58:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T14:58:55.282 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T14:58:55.348 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T14:58:55.382 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T14:58:55.423 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T14:58:55.528 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T14:58:55.601 INFO:teuthology.orchestra.run.smithi134.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 191 kB 00:00 2024-09-16T14:58:55.643 INFO:teuthology.orchestra.run.smithi134.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 5.3 MB/s | 801 kB 00:00 2024-09-16T14:58:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T14:58:55.701 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T14:58:55.885 INFO:teuthology.orchestra.run.smithi134.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-09-16T14:58:55.985 INFO:teuthology.orchestra.run.smithi134.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.4-84 15 MB/s | 7.4 MB 00:00 2024-09-16T14:58:56.010 INFO:teuthology.orchestra.run.smithi134.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-09-16T14:58:56.269 INFO:teuthology.orchestra.run.smithi134.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 136 kB/s | 38 kB 00:00 2024-09-16T14:58:56.443 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T14:58:56.490 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T14:58:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T14:58:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T14:58:56.618 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:58:56.653 INFO:teuthology.orchestra.run.smithi134.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 638 kB/s | 245 kB 00:00 2024-09-16T14:58:56.986 INFO:teuthology.orchestra.run.smithi134.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 5.1 MB/s | 4.9 MB 00:00 2024-09-16T14:58:57.362 INFO:teuthology.orchestra.run.smithi134.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 668 kB/s | 250 kB 00:00 2024-09-16T14:58:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:58:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 1/13 2024-09-16T14:58:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T14:58:57.390 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T14:58:57.391 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T14:58:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T14:58:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:57.724 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:58:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:57.725 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:58:57.992 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T14:58:58.078 INFO:teuthology.orchestra.run.smithi134.stdout:(10/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 1.2 MB/s | 3.0 MB 00:02 2024-09-16T14:58:58.584 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:15 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:58:58.721 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 139 k 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T14:58:58.722 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout:Install 6 Packages 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:58:58.723 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 7.0 M 2024-09-16T14:58:58.724 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 33 M 2024-09-16T14:58:58.724 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:58:58.928 INFO:teuthology.orchestra.run.smithi134.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.2 MB/s | 5.1 MB 00:02 2024-09-16T14:58:59.217 INFO:teuthology.orchestra.run.smithi162.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-16T14:58:59.242 INFO:teuthology.orchestra.run.smithi162.stdout:(2/6): ceph-mgr-cephadm-18.2.4-845.ge4e227b3.el 653 kB/s | 139 kB 00:00 2024-09-16T14:58:59.318 INFO:teuthology.orchestra.run.smithi162.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 770 kB/s | 58 kB 00:00 2024-09-16T14:58:59.351 INFO:teuthology.orchestra.run.smithi162.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-16T14:58:59.376 INFO:teuthology.orchestra.run.smithi162.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.4 MB/s | 548 kB 00:00 2024-09-16T14:58:59.453 INFO:teuthology.orchestra.run.smithi162.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-16T14:58:59.454 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:58:59.454 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.6 MB/s | 7.0 MB 00:00 2024-09-16T14:58:59.524 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:58:59.535 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:58:59.535 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:58:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:58:59.633 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:58:59.937 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:00.107 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T14:59:00.161 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T14:59:00.674 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T14:59:00.755 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T14:59:00.818 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T14:59:00.841 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:59:01.673 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:59:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/6 2024-09-16T14:59:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T14:59:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T14:59:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T14:59:01.674 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T14:59:01.932 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T14:59:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T14:59:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T14:59:01.933 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:01.933 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:02.234 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-fuse 2024-09-16T14:59:02.808 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:19 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:02.941 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:02.941 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:02.941 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:59:02.941 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 852 k 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout:Install 2 Packages 2024-09-16T14:59:02.942 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:02.943 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 932 k 2024-09-16T14:59:02.943 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 2.7 M 2024-09-16T14:59:02.943 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:03.304 INFO:teuthology.orchestra.run.smithi162.stdout:(1/2): ceph-fuse-18.2.4-845.ge4e227b3.el9.x86_6 3.2 MB/s | 852 kB 00:00 2024-09-16T14:59:03.329 INFO:teuthology.orchestra.run.smithi162.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 278 kB/s | 80 kB 00:00 2024-09-16T14:59:03.330 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:03.330 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.3 MB/s | 932 kB 00:00 2024-09-16T14:59:03.345 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:03.364 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:03.364 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:03.444 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:03.444 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:03.630 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:03.769 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T14:59:03.837 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:59:04.496 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:59:04.497 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/2 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:04.766 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:05.059 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-volume 2024-09-16T14:59:05.641 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:22 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:05.778 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:05.778 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:05.778 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-16T14:59:05.778 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:05.778 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 264 k 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout:Install 2 Packages 2024-09-16T14:59:05.779 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:05.780 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 342 k 2024-09-16T14:59:05.780 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 1.5 M 2024-09-16T14:59:05.780 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:05.969 INFO:teuthology.orchestra.run.smithi162.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 573 kB/s | 77 kB 00:00 2024-09-16T14:59:06.085 INFO:teuthology.orchestra.run.smithi162.stdout:(2/2): ceph-volume-18.2.4-845.ge4e227b3.el9.noa 1.0 MB/s | 264 kB 00:00 2024-09-16T14:59:06.086 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:06.086 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.1 MB/s | 342 kB 00:00 2024-09-16T14:59:06.098 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:06.107 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:06.108 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:06.142 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:06.143 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:06.291 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:06.470 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-16T14:59:06.504 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:59:06.520 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:59:06.521 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:59:06.521 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T14:59:06.521 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 1/2 2024-09-16T14:59:07.556 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:07.557 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:07.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados-devel 2024-09-16T14:59:08.481 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:08.615 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 127 k 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 127 k 2024-09-16T14:59:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 456 k 2024-09-16T14:59:08.617 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:08.842 INFO:teuthology.orchestra.run.smithi162.stdout:librados-devel-18.2.4-845.ge4e227b3.el9.x86_64. 562 kB/s | 127 kB 00:00 2024-09-16T14:59:08.843 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:08.843 INFO:teuthology.orchestra.run.smithi162.stdout:Total 558 kB/s | 127 kB 00:00 2024-09-16T14:59:08.843 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:08.849 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:08.849 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:08.912 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:08.912 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:09.062 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:09.569 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:09.823 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:10.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs2 2024-09-16T14:59:10.660 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:10.755 INFO:teuthology.orchestra.run.smithi162.stdout:Package libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:10.800 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:10.801 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:10.801 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:10.891 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs-devel 2024-09-16T14:59:11.461 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:11.593 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:11.594 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-16T14:59:11.594 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:11.594 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 31 k 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 31 k 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 139 k 2024-09-16T14:59:11.595 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:11.746 INFO:teuthology.orchestra.run.smithi162.stdout:libcephfs-devel-18.2.4-845.ge4e227b3.el9.x86_64 206 kB/s | 31 kB 00:00 2024-09-16T14:59:11.747 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:11.747 INFO:teuthology.orchestra.run.smithi162.stdout:Total 204 kB/s | 31 kB 00:00 2024-09-16T14:59:11.747 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:11.750 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:11.751 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:11.775 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:11.775 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:11.868 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:11.977 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:12.361 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:12.603 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:12.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados2 2024-09-16T14:59:13.349 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:13.440 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:13.483 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:13.484 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:13.484 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:13.563 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librbd1 2024-09-16T14:59:14.134 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:14.226 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:14.268 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:14.269 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:14.270 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:14.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rados 2024-09-16T14:59:14.913 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:15.003 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:15.046 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:15.047 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:15.047 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:15.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rgw 2024-09-16T14:59:15.683 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:15.773 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:15.817 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:15.817 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:15.817 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:15.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-cephfs 2024-09-16T14:59:16.467 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:16.558 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:16.602 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:16.603 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:16.603 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:16.678 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rbd 2024-09-16T14:59:17.243 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:17.334 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:17.377 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:17.378 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-16T14:59:17.378 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:17.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-fuse 2024-09-16T14:59:18.031 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:18.163 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 87 k 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 87 k 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 227 k 2024-09-16T14:59:18.165 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:18.357 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-fuse-18.2.4-845.ge4e227b3.el9.x86_64.rpm 453 kB/s | 87 kB 00:00 2024-09-16T14:59:18.357 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:18.358 INFO:teuthology.orchestra.run.smithi162.stdout:Total 450 kB/s | 87 kB 00:00 2024-09-16T14:59:18.358 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:18.365 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:18.365 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:18.432 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:18.610 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:19.037 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:19.287 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:19.559 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-mirror 2024-09-16T14:59:20.140 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:20.278 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:20.278 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:20.278 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 3.0 M 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 12 M 2024-09-16T14:59:20.279 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:20.611 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-mirror-18.2.4-845.ge4e227b3.el9.x86_64.rpm 9.0 MB/s | 3.0 MB 00:00 2024-09-16T14:59:20.612 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:20.613 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.0 MB/s | 3.0 MB 00:00 2024-09-16T14:59:20.613 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:20.622 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:20.622 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:20.703 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:20.704 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:20.976 INFO:teuthology.orchestra.run.smithi134.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 705 kB/s | 16 MB 00:23 2024-09-16T14:59:20.977 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:20.977 INFO:teuthology.orchestra.run.smithi134.stdout:Total 1.5 MB/s | 38 MB 00:25 2024-09-16T14:59:21.119 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:21.188 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:21.212 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:21.213 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:59:21.213 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T14:59:21.213 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:59:21.213 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:59:21.213 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:21.336 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:21.363 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:21.363 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:22.261 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:22.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-nbd 2024-09-16T14:59:22.618 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:22.704 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T14:59:22.748 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:59:22.851 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:59:22.883 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T14:59:23.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T14:59:23.090 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T14:59:23.107 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 02:57:43 PM UTC. 2024-09-16T14:59:23.167 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T14:59:23.242 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-16T14:59:23.242 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 172 k 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 172 k 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 494 k 2024-09-16T14:59:23.243 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-16T14:59:23.458 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-nbd-18.2.4-845.ge4e227b3.el9.x86_64.rpm 803 kB/s | 172 kB 00:00 2024-09-16T14:59:23.458 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:23.458 INFO:teuthology.orchestra.run.smithi162.stdout:Total 798 kB/s | 172 kB 00:00 2024-09-16T14:59:23.458 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-16T14:59:23.467 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-16T14:59:23.467 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-16T14:59:23.535 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-16T14:59:23.536 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-16T14:59:23.700 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-16T14:59:23.805 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:23.853 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T14:59:23.970 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T14:59:24.245 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T14:59:24.502 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-16T14:59:24.770 DEBUG:teuthology.parallel:result is None 2024-09-16T14:59:25.225 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:59:26.243 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T14:59:26.288 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:59:28.932 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:59:28.932 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 1/12 2024-09-16T14:59:28.932 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:59:28.932 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:59:28.932 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:59:28.933 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T14:59:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T14:59:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:29.371 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T14:59:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:29.373 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:29.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-mgr-rook 2024-09-16T14:59:30.329 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:30.462 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:30.463 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:30.463 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:59:30.463 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:30.463 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 49 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T14:59:30.464 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout:Install 13 Packages 2024-09-16T14:59:30.465 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:30.466 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 2.2 M 2024-09-16T14:59:30.466 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 26 M 2024-09-16T14:59:30.466 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:30.654 INFO:teuthology.orchestra.run.smithi134.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 321 kB/s | 26 kB 00:00 2024-09-16T14:59:30.679 INFO:teuthology.orchestra.run.smithi134.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 182 kB/s | 19 kB 00:00 2024-09-16T14:59:30.754 INFO:teuthology.orchestra.run.smithi134.stdout:(3/13): ceph-mgr-rook-18.2.4-845.ge4e227b3.el9. 273 kB/s | 49 kB 00:00 2024-09-16T14:59:30.845 INFO:teuthology.orchestra.run.smithi134.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-16T14:59:30.845 INFO:teuthology.orchestra.run.smithi134.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-16T14:59:30.848 INFO:teuthology.orchestra.run.smithi134.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 910 kB/s | 54 kB 00:00 2024-09-16T14:59:30.881 INFO:teuthology.orchestra.run.smithi134.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 479 kB/s | 32 kB 00:00 2024-09-16T14:59:30.914 INFO:teuthology.orchestra.run.smithi134.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 216 kB/s | 14 kB 00:00 2024-09-16T14:59:30.939 INFO:teuthology.orchestra.run.smithi134.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-16T14:59:30.965 INFO:teuthology.orchestra.run.smithi134.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.3 MB/s | 279 kB 00:00 2024-09-16T14:59:30.990 INFO:teuthology.orchestra.run.smithi134.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-16T14:59:31.040 INFO:teuthology.orchestra.run.smithi134.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-16T14:59:31.082 INFO:teuthology.orchestra.run.smithi134.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-09-16T14:59:31.084 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:31.085 INFO:teuthology.orchestra.run.smithi134.stdout:Total 3.6 MB/s | 2.2 MB 00:00 2024-09-16T14:59:31.148 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:31.163 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:31.163 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:31.251 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:31.251 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:31.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T14:59:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T14:59:32.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T14:59:32.076 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T14:59:32.117 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T14:59:32.219 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T14:59:32.329 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T14:59:32.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T14:59:33.101 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T14:59:33.149 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T14:59:33.183 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T14:59:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T14:59:33.252 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:59:34.068 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:59:34.068 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 1/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T14:59:34.069 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T14:59:34.392 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T14:59:34.393 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T14:59:34.394 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T14:59:34.394 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T14:59:34.394 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T14:59:34.394 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:34.394 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:34.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T14:59:35.220 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:35.353 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 139 k 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T14:59:35.354 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout:Install 6 Packages 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:35.355 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 7.0 M 2024-09-16T14:59:35.356 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 33 M 2024-09-16T14:59:35.356 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:35.958 INFO:teuthology.orchestra.run.smithi134.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-16T14:59:36.026 INFO:teuthology.orchestra.run.smithi134.stdout:(2/6): ceph-mgr-cephadm-18.2.4-845.ge4e227b3.el 627 kB/s | 139 kB 00:00 2024-09-16T14:59:36.059 INFO:teuthology.orchestra.run.smithi134.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.4 MB/s | 548 kB 00:00 2024-09-16T14:59:36.092 INFO:teuthology.orchestra.run.smithi134.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.8 MB/s | 74 kB 00:00 2024-09-16T14:59:36.167 INFO:teuthology.orchestra.run.smithi134.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-16T14:59:36.192 INFO:teuthology.orchestra.run.smithi134.stdout:(6/6): python3-natsort-7.1.1-5.el9.noarch.rpm 345 kB/s | 58 kB 00:00 2024-09-16T14:59:36.195 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:36.195 INFO:teuthology.orchestra.run.smithi134.stdout:Total 8.3 MB/s | 7.0 MB 00:00 2024-09-16T14:59:36.267 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:36.278 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:36.278 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:36.377 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:36.378 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:36.689 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:36.857 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T14:59:36.902 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T14:59:37.419 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T14:59:37.496 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T14:59:37.558 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T14:59:37.581 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:59:38.330 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:59:38.331 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/6 2024-09-16T14:59:38.331 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T14:59:38.331 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T14:59:38.331 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T14:59:38.331 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T14:59:38.616 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:38.617 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:38.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-fuse 2024-09-16T14:59:39.446 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:39.577 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 852 k 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:39.578 INFO:teuthology.orchestra.run.smithi134.stdout:Install 2 Packages 2024-09-16T14:59:39.579 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:39.579 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 932 k 2024-09-16T14:59:39.579 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 2.7 M 2024-09-16T14:59:39.579 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:39.694 INFO:teuthology.orchestra.run.smithi134.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 875 kB/s | 80 kB 00:00 2024-09-16T14:59:39.869 INFO:teuthology.orchestra.run.smithi134.stdout:(2/2): ceph-fuse-18.2.4-845.ge4e227b3.el9.x86_6 3.1 MB/s | 852 kB 00:00 2024-09-16T14:59:39.869 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:39.869 INFO:teuthology.orchestra.run.smithi134.stdout:Total 3.1 MB/s | 932 kB 00:00 2024-09-16T14:59:39.881 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:39.903 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:39.903 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:39.983 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:39.983 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:40.174 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:40.309 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T14:59:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:59:41.110 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:59:41.110 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/2 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:41.379 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:41.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-volume 2024-09-16T14:59:42.264 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:42.400 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 264 k 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:42.401 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:42.402 INFO:teuthology.orchestra.run.smithi134.stdout:Install 2 Packages 2024-09-16T14:59:42.402 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:42.402 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 342 k 2024-09-16T14:59:42.402 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 1.5 M 2024-09-16T14:59:42.402 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:42.657 INFO:teuthology.orchestra.run.smithi134.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 531 kB/s | 77 kB 00:00 2024-09-16T14:59:42.740 INFO:teuthology.orchestra.run.smithi134.stdout:(2/2): ceph-volume-18.2.4-845.ge4e227b3.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-16T14:59:42.740 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:42.740 INFO:teuthology.orchestra.run.smithi134.stdout:Total 1.0 MB/s | 342 kB 00:00 2024-09-16T14:59:42.752 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:42.761 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:42.762 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:42.799 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:42.800 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:42.943 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:43.123 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-16T14:59:43.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:59:43.174 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:59:43.175 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:59:43.175 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T14:59:43.175 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:43.851 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 1/2 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:44.157 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:44.461 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install librados-devel 2024-09-16T14:59:45.040 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:45.176 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repo Size 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 127 k 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 127 k 2024-09-16T14:59:45.177 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 456 k 2024-09-16T14:59:45.178 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:45.386 INFO:teuthology.orchestra.run.smithi134.stdout:librados-devel-18.2.4-845.ge4e227b3.el9.x86_64. 607 kB/s | 127 kB 00:00 2024-09-16T14:59:45.387 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:45.387 INFO:teuthology.orchestra.run.smithi134.stdout:Total 603 kB/s | 127 kB 00:00 2024-09-16T14:59:45.387 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:45.393 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:45.393 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:45.455 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:45.456 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:45.607 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:45.716 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:46.144 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:46.389 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:46.389 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:46.390 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:46.596 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install libcephfs2 2024-09-16T14:59:47.170 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:52 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:47.261 INFO:teuthology.orchestra.run.smithi134.stdout:Package libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:47.305 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:47.306 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:47.306 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:47.389 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install libcephfs-devel 2024-09-16T14:59:47.950 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:52 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:48.082 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:48.083 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:48.084 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repo Size 2024-09-16T14:59:48.084 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:48.084 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:48.084 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 31 k 2024-09-16T14:59:48.084 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 31 k 2024-09-16T14:59:48.085 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 139 k 2024-09-16T14:59:48.086 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:48.264 INFO:teuthology.orchestra.run.smithi134.stdout:libcephfs-devel-18.2.4-845.ge4e227b3.el9.x86_64 172 kB/s | 31 kB 00:00 2024-09-16T14:59:48.264 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:48.264 INFO:teuthology.orchestra.run.smithi134.stdout:Total 171 kB/s | 31 kB 00:00 2024-09-16T14:59:48.265 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:48.268 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:48.268 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:48.285 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:48.286 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:48.364 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:48.469 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:48.821 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:49.056 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:49.056 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:49.057 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:49.057 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:49.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install librados2 2024-09-16T14:59:49.811 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:54 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:49.900 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:49.943 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:49.944 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:49.944 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:50.025 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install librbd1 2024-09-16T14:59:50.593 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:55 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:50.682 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:50.725 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:50.726 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:50.726 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:50.806 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install python3-rados 2024-09-16T14:59:51.365 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:56 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:51.454 INFO:teuthology.orchestra.run.smithi134.stdout:Package python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:51.498 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:51.498 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:51.499 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:51.571 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install python3-rgw 2024-09-16T14:59:52.135 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:57 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:52.225 INFO:teuthology.orchestra.run.smithi134.stdout:Package python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:52.267 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:52.268 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:52.269 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:52.341 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install python3-cephfs 2024-09-16T14:59:52.904 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:57 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:52.995 INFO:teuthology.orchestra.run.smithi134.stdout:Package python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:53.037 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:53.038 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:53.038 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:53.128 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install python3-rbd 2024-09-16T14:59:53.689 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:58 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:53.778 INFO:teuthology.orchestra.run.smithi134.stdout:Package python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:59:53.820 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:53.821 INFO:teuthology.orchestra.run.smithi134.stdout:Nothing to do. 2024-09-16T14:59:53.821 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:53.910 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install rbd-fuse 2024-09-16T14:59:54.472 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:59 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:54.605 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 87 k 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:59:54.606 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:54.607 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 87 k 2024-09-16T14:59:54.607 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 227 k 2024-09-16T14:59:54.607 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:54.775 INFO:teuthology.orchestra.run.smithi134.stdout:rbd-fuse-18.2.4-845.ge4e227b3.el9.x86_64.rpm 516 kB/s | 87 kB 00:00 2024-09-16T14:59:54.776 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:54.776 INFO:teuthology.orchestra.run.smithi134.stdout:Total 512 kB/s | 87 kB 00:00 2024-09-16T14:59:54.776 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:54.783 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:54.783 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:54.850 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:54.850 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:55.004 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:55.105 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:55.492 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:55.831 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:55.832 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:55.832 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:55.832 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:55.832 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:55.832 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:56.073 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install rbd-mirror 2024-09-16T14:59:56.663 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:02:01 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:56.807 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:56.808 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:56.808 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repo Size 2024-09-16T14:59:56.808 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:56.808 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 3.0 M 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 12 M 2024-09-16T14:59:56.809 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T14:59:57.137 INFO:teuthology.orchestra.run.smithi134.stdout:rbd-mirror-18.2.4-845.ge4e227b3.el9.x86_64.rpm 9.1 MB/s | 3.0 MB 00:00 2024-09-16T14:59:57.137 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:59:57.138 INFO:teuthology.orchestra.run.smithi134.stdout:Total 9.1 MB/s | 3.0 MB 00:00 2024-09-16T14:59:57.138 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T14:59:57.147 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T14:59:57.147 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T14:59:57.230 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T14:59:57.230 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T14:59:57.667 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T14:59:57.721 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:57.747 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:57.747 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:59:57.747 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T14:59:57.747 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:59:57.747 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:59:57.748 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:58.831 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T14:59:59.074 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install rbd-nbd 2024-09-16T14:59:59.664 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:02:04 ago on Mon 16 Sep 2024 02:57:55 PM UTC. 2024-09-16T14:59:59.798 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 172 k 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:Install 1 Package 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 172 k 2024-09-16T14:59:59.799 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 494 k 2024-09-16T14:59:59.800 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-16T15:00:00.006 INFO:teuthology.orchestra.run.smithi134.stdout:rbd-nbd-18.2.4-845.ge4e227b3.el9.x86_64.rpm 833 kB/s | 172 kB 00:00 2024-09-16T15:00:00.007 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-16T15:00:00.007 INFO:teuthology.orchestra.run.smithi134.stdout:Total 827 kB/s | 172 kB 00:00 2024-09-16T15:00:00.007 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-16T15:00:00.015 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-16T15:00:00.015 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-16T15:00:00.082 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-16T15:00:00.082 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-16T15:00:00.248 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-16T15:00:00.345 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T15:00:00.792 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T15:00:01.082 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T15:00:01.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:01.082 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-16T15:00:01.082 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T15:00:01.082 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:01.083 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-16T15:00:01.331 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:01.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:00:01.477 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:00:01.553 INFO:teuthology.orchestra.run.smithi134.stdout:18.2.4-845.ge4e227b3.el9 2024-09-16T15:00:01.553 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845.ge4e227b3.el9 2024-09-16T15:00:01.554 INFO:teuthology.task.install:The correct ceph version 18.2.4-845.ge4e227b3 is installed. 2024-09-16T15:00:01.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:00:01.697 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:00:01.723 INFO:teuthology.orchestra.run.smithi162.stdout:18.2.4-845.ge4e227b3.el9 2024-09-16T15:00:01.723 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845.ge4e227b3.el9 2024-09-16T15:00:01.723 INFO:teuthology.task.install:The correct ceph version 18.2.4-845.ge4e227b3 is installed. 2024-09-16T15:00:01.728 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T15:00:01.728 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:01.728 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:00:01.754 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:01.754 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:00:01.794 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T15:00:01.795 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:01.795 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:00:01.819 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:00:01.884 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:01.884 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:00:01.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:00:01.982 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T15:00:01.982 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:01.982 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:00:02.007 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:00:02.071 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:02.071 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:00:02.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:00:02.167 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T15:00:02.167 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:02.168 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:00:02.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:00:02.257 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:02.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:00:02.282 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:00:02.348 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T15:00:02.430 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T15:00:02.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-09-16T15:00:02.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2024-09-16T15:00:02.457 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T15:00:02.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:00:02.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:00:02.523 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T15:00:02.524 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:00:02.561 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:00:02.600 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T15:00:02.600 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T15:00:02.600 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784', 'branch': 'reef', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906784', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 931, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 1390, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'sleep_before_teardown': 0, 'subset': '5319/100000', 'suite': 'rados', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6', 'targets': {'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMlT7dfvhv6nDcP9NxoDsK3UuClKzMjmYbRwgwAAuolkrDvJljKTmXAPOir0lnmYHI8KMW+WJ16eBke4F+qjxhg=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBaksl13L6Xphrva4Bv21Cdq8BN6JHOLBG1DV1UHYmRV3iYyFyH/F1H7AwJgTwU1m3qDEG5FTySWUj5DIaa/Qk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1, 'thrash_many': True}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_22:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T15:00:02.600 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:00:02.627 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:00:02.647 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:00:02.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:00:02.712 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:02.713 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:00:02.762 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:00:02.762 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-16T15:00:02.817 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-16 14:59:57.981864018 +0000 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-16 14:56:58.175480963 +0000 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-16 14:56:58.175480963 +0000 2024-09-16T15:00:02.818 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-16 14:56:58.175480963 +0000 2024-09-16T15:00:02.819 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:00:02.881 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-16T15:00:02.881 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-16T15:00:02.881 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000185536 s, 2.8 MB/s 2024-09-16T15:00:02.882 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:00:02.938 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-16T15:00:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:00:02.993 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-16 14:59:57.981864018 +0000 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-16 14:56:58.519472920 +0000 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-16 14:56:58.519472920 +0000 2024-09-16T15:00:02.994 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-16 14:56:58.519472920 +0000 2024-09-16T15:00:02.994 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:00:03.056 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-16T15:00:03.056 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-16T15:00:03.056 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00017962 s, 2.9 MB/s 2024-09-16T15:00:03.058 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:00:03.115 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-16 14:59:57.982863995 +0000 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-16 14:56:58.844465321 +0000 2024-09-16T15:00:03.171 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-16 14:56:58.844465321 +0000 2024-09-16T15:00:03.172 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-16 14:56:58.844465321 +0000 2024-09-16T15:00:03.172 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:00:03.234 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-16T15:00:03.234 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-16T15:00:03.235 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000210551 s, 2.4 MB/s 2024-09-16T15:00:03.236 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:00:03.292 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-16T15:00:03.348 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:00:03.348 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:03.348 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T15:00:03.348 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:03.348 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:03.349 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-16 14:59:57.982863995 +0000 2024-09-16T15:00:03.349 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-16 14:56:59.164457838 +0000 2024-09-16T15:00:03.349 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-16 14:56:59.164457838 +0000 2024-09-16T15:00:03.349 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-16 14:56:59.164457838 +0000 2024-09-16T15:00:03.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:00:03.411 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-16T15:00:03.411 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-16T15:00:03.412 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000207289 s, 2.5 MB/s 2024-09-16T15:00:03.413 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:00:03.469 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:03.469 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:03.469 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:00:03.486 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:00:03.486 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-09-16T15:00:03.541 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:00:03.541 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:03.541 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T15:00:03.541 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:03.542 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:03.542 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 14:59:21.465407591 +0000 2024-09-16T15:00:03.542 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 14:56:58.179760335 +0000 2024-09-16T15:00:03.542 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 14:56:58.179760335 +0000 2024-09-16T15:00:03.542 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-16 14:56:58.179760335 +0000 2024-09-16T15:00:03.542 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:00:03.604 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T15:00:03.604 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T15:00:03.605 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000197296 s, 2.6 MB/s 2024-09-16T15:00:03.606 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:00:03.664 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-09-16T15:00:03.719 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 14:59:21.465407591 +0000 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 14:56:58.518753592 +0000 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 14:56:58.518753592 +0000 2024-09-16T15:00:03.720 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-16 14:56:58.518753592 +0000 2024-09-16T15:00:03.720 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:00:03.783 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T15:00:03.783 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T15:00:03.783 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.00024523 s, 2.1 MB/s 2024-09-16T15:00:03.784 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:00:03.840 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-09-16T15:00:03.895 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 14:59:21.466407571 +0000 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 14:56:58.827747444 +0000 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 14:56:58.827747444 +0000 2024-09-16T15:00:03.896 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-16 14:56:58.827747444 +0000 2024-09-16T15:00:03.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:00:03.959 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T15:00:03.959 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T15:00:03.959 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000180296 s, 2.8 MB/s 2024-09-16T15:00:03.960 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:00:04.016 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:00:04.071 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 14:59:21.466407571 +0000 2024-09-16T15:00:04.072 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 14:56:59.143741158 +0000 2024-09-16T15:00:04.072 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 14:56:59.143741158 +0000 2024-09-16T15:00:04.072 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-16 14:56:59.143741158 +0000 2024-09-16T15:00:04.072 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:00:04.134 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T15:00:04.135 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T15:00:04.135 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000182927 s, 2.8 MB/s 2024-09-16T15:00:04.136 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:00:04.196 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:04.196 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-16T15:00:04.196 INFO:tasks.ceph:Generating config... 2024-09-16T15:00:04.197 INFO:tasks.ceph:[client] debug monc = 20 2024-09-16T15:00:04.197 INFO:tasks.ceph:[client] debug ms = 1 2024-09-16T15:00:04.197 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T15:00:04.197 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-16T15:00:04.197 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-16T15:00:04.198 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-16T15:00:04.198 INFO:tasks.ceph:[global] ms inject delay max = 1 2024-09-16T15:00:04.198 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2024-09-16T15:00:04.198 INFO:tasks.ceph:[global] ms inject delay type = mon 2024-09-16T15:00:04.198 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2024-09-16T15:00:04.199 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-16T15:00:04.199 INFO:tasks.ceph:[global] ms type = async 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mgr] debug monc = 10 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-16T15:00:04.199 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] paxos min = 10 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-16T15:00:04.200 INFO:tasks.ceph:[mon] paxos trim min = 10 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-16T15:00:04.200 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-16T15:00:04.201 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T15:00:04.202 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-16T15:00:04.202 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-16T15:00:04.202 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T15:00:04.202 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:04.248 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T15:00:04.251 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T15:00:04.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:00:04.360 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.134'), ('mon.b', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]'), ('mon.c', '[v2:172.21.15.134:3302,v1:172.21.15.134:6791]'), ('mon.d', '[v2:172.21.15.134:3303,v1:172.21.15.134:6792]'), ('mon.e', '[v2:172.21.15.134:3304,v1:172.21.15.134:6793]'), ('mon.f', '172.21.15.162'), ('mon.g', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]'), ('mon.h', '[v2:172.21.15.162:3302,v1:172.21.15.162:6791]'), ('mon.i', '[v2:172.21.15.162:3303,v1:172.21.15.162:6792]')] 2024-09-16T15:00:04.360 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790],[v2:172.21.15.134:3302,v1:172.21.15.134:6791],[v2:172.21.15.134:3303,v1:172.21.15.134:6792],[v2:172.21.15.134:3304,v1:172.21.15.134:6793],172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],[v2:172.21.15.162:3302,v1:172.21.15.162:6791],[v2:172.21.15.162:3303,v1:172.21.15.162:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-09-16T15:00:04.361 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:04.361 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:00:04.416 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.134 --addv b '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --addv c '[v2:172.21.15.134:3302,v1:172.21.15.134:6791]' --addv d '[v2:172.21.15.134:3303,v1:172.21.15.134:6792]' --addv e '[v2:172.21.15.134:3304,v1:172.21.15.134:6793]' --add f 172.21.15.162 --addv g '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --addv h '[v2:172.21.15.162:3302,v1:172.21.15.162:6791]' --addv i '[v2:172.21.15.162:3303,v1:172.21.15.162:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:04.519 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:04.519 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: generated fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:04.519 INFO:teuthology.orchestra.run.smithi134.stdout:setting min_mon_release = pacific 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:epoch 0 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:min_mon_release 16 (pacific) 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:election_strategy: 1 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:3: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:4: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:04.520 INFO:teuthology.orchestra.run.smithi134.stdout:5: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:04.521 INFO:teuthology.orchestra.run.smithi134.stdout:6: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:04.521 INFO:teuthology.orchestra.run.smithi134.stdout:7: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:04.521 INFO:teuthology.orchestra.run.smithi134.stdout:8: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:04.521 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-09-16T15:00:04.522 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:00:04.537 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9b832ad6-88ab-4e38-a956-73c2de3b6492... 2024-09-16T15:00:04.538 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:00:04.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:00:04.620 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-16T15:00:04.620 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-16T15:00:04.621 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.622 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.623 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:00:04.624 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:00:04.625 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.626 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790],[v2:172.21.15.134:3302,v1:172.21.15.134:6791],[v2:172.21.15.134:3303,v1:172.21.15.134:6792],[v2:172.21.15.134:3304,v1:172.21.15.134:6793],172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],[v2:172.21.15.162:3302,v1:172.21.15.162:6791],[v2:172.21.15.162:3303,v1:172.21.15.162:6792]" 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: mon client directed command retry = 5 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: mon election default strategy = 1 2024-09-16T15:00:04.627 INFO:teuthology.orchestra.run.smithi134.stdout: ms bind msgr1 = False 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms bind msgr2 = True 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay max = 1 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay probability = 0.005 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay type = mon 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject internal delays = 0.002 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject socket failures = 2500 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: ms type = async 2024-09-16T15:00:04.628 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-16T15:00:04.629 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-16T15:00:04.630 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:00:04.630 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.630 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-16T15:00:04.630 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-16T15:00:04.630 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.631 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-16T15:00:04.631 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-16T15:00:04.631 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-16T15:00:04.631 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-16T15:00:04.632 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-16T15:00:04.633 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore compression algorithm = zlib 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore compression mode = aggressive 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore zero block detection = True 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 20 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 20 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-16T15:00:04.634 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 10 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify cached snaps = True 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify missing on start = True 2024-09-16T15:00:04.635 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:00:04.636 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock override recovery settings = True 2024-09-16T15:00:04.636 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout: osd op queue = debug_random 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout: osd op queue cut off = debug_random 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-16T15:00:04.637 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: debug monc = 10 2024-09-16T15:00:04.638 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:00:04.639 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:00:04.641 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:00:04.641 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:00:04.641 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.641 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-16T15:00:04.641 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon min osdmap epochs = 25 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon osdmap full prune interval = 2 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon osdmap full prune min = 15 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon osdmap full prune txsize = 2 2024-09-16T15:00:04.642 INFO:teuthology.orchestra.run.smithi134.stdout: mon scrub inject crc mismatch = 0.01 2024-09-16T15:00:04.643 INFO:teuthology.orchestra.run.smithi134.stdout: mon scrub inject missing keys = 0.05 2024-09-16T15:00:04.643 INFO:teuthology.orchestra.run.smithi134.stdout: mon scrub interval = 300 2024-09-16T15:00:04.643 INFO:teuthology.orchestra.run.smithi134.stdout: paxos min = 10 2024-09-16T15:00:04.643 INFO:teuthology.orchestra.run.smithi134.stdout: paxos service trim min = 5 2024-09-16T15:00:04.643 INFO:teuthology.orchestra.run.smithi134.stdout: paxos trim min = 10 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-16T15:00:04.644 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout: debug monc = 20 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-16T15:00:04.645 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.d] 2024-09-16T15:00:04.646 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.e] 2024-09-16T15:00:04.646 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.f] 2024-09-16T15:00:04.646 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.g] 2024-09-16T15:00:04.646 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.h] 2024-09-16T15:00:04.646 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.i] 2024-09-16T15:00:04.649 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-09-16T15:00:04.649 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-09-16T15:00:04.649 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:00:04.650 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-09-16T15:00:04.650 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.650 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-09-16T15:00:04.650 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-09-16T15:00:04.651 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-09-16T15:00:04.652 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:00:04.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-09-16T15:00:04.652 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.652 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:00:04.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:00:04.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-09-16T15:00:04.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.653 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:00:04.654 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-09-16T15:00:04.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:00:04.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:00:04.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-09-16T15:00:04.655 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.655 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:00:04.656 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.656 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-09-16T15:00:04.656 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.657 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-09-16T15:00:04.657 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.657 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-09-16T15:00:04.657 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-09-16T15:00:04.657 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790],[v2:172.21.15.134:3302,v1:172.21.15.134:6791],[v2:172.21.15.134:3303,v1:172.21.15.134:6792],[v2:172.21.15.134:3304,v1:172.21.15.134:6793],172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],[v2:172.21.15.162:3302,v1:172.21.15.162:6791],[v2:172.21.15.162:3303,v1:172.21.15.162:6792]" 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: mon client directed command retry = 5 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: mon election default strategy = 1 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: ms bind msgr1 = False 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: ms bind msgr2 = True 2024-09-16T15:00:04.658 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject delay max = 1 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject delay probability = 0.005 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject delay type = mon 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject internal delays = 0.002 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject socket failures = 2500 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: ms type = async 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:04.659 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-09-16T15:00:04.660 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-09-16T15:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore compression algorithm = zlib 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore compression mode = aggressive 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore zero block detection = True 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-09-16T15:00:04.662 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify cached snaps = True 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify missing on start = True 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock override recovery settings = True 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd op queue = debug_random 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: osd op queue cut off = debug_random 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T15:00:04.663 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug monc = 10 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-09-16T15:00:04.664 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:00:04.665 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon min osdmap epochs = 25 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon osdmap full prune interval = 2 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon osdmap full prune min = 15 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon osdmap full prune txsize = 2 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon scrub inject crc mismatch = 0.01 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon scrub inject missing keys = 0.05 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: mon scrub interval = 300 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: paxos min = 10 2024-09-16T15:00:04.666 INFO:teuthology.orchestra.run.smithi162.stdout: paxos service trim min = 5 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: paxos trim min = 10 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: debug monc = 20 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-09-16T15:00:04.667 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.d] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.e] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.f] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.g] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.h] 2024-09-16T15:00:04.668 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.i] 2024-09-16T15:00:04.668 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T15:00:04.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T15:00:04.743 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T15:00:04.743 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:04.743 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T15:00:04.758 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:04.758 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T15:00:04.813 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:00:04.813 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:04.814 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:00:04.814 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:00:04.888 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:04.888 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:04.943 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:00:04.944 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:04.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:00:04.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:00:04.978 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:04.979 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:05.034 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T15:00:05.034 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T15:00:05.035 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:00:05.136 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:00:05.138 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T15:00:05.138 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T15:00:05.139 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T15:00:05.179 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T15:00:05.192 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T15:00:05.192 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-16T15:00:05.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T15:00:05.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:05.218 INFO:tasks.ceph:role: osd.0 2024-09-16T15:00:05.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:00:05.218 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:00:05.290 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:05.291 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:05.292 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:05.292 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:05.292 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:05.775 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-16T15:00:05.778 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-16T15:00:05.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T15:00:05.888 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T15:00:05.921 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T15:00:05.987 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:05.987 INFO:tasks.ceph:role: osd.1 2024-09-16T15:00:05.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:00:05.987 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:00:06.061 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:06.062 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:06.517 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-16T15:00:06.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-16T15:00:06.519 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T15:00:06.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T15:00:06.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T15:00:06.733 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:06.733 INFO:tasks.ceph:role: osd.2 2024-09-16T15:00:06.733 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:00:06.734 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:00:06.805 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:06.806 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:07.294 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-16T15:00:07.295 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-16T15:00:07.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T15:00:07.396 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T15:00:07.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:07.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.519+0000 7fdb5a1dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T15:00:07.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.524+0000 7fdb5a1dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T15:00:07.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.524+0000 7fdb5a1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:00:07.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.524+0000 7fdb5a1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:00:07.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.524+0000 7fdb5a1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:00:07.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:07.526+0000 7fdb5a1dd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T15:00:10.381 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T15:00:10.410 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:10.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.500+0000 7f0e3d633640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T15:00:10.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.500+0000 7f0e3d633640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T15:00:10.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.500+0000 7f0e3d633640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:00:10.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.500+0000 7f0e3d633640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:00:10.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.500+0000 7f0e3d633640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:00:10.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:10.501+0000 7f0e3d633640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T15:00:13.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T15:00:13.387 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:13.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.474+0000 7fb444102640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T15:00:13.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.474+0000 7fb444102640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T15:00:13.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.474+0000 7fb444102640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:00:13.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.474+0000 7fb444102640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:00:13.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.474+0000 7fb444102640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:00:13.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:13.475+0000 7fb444102640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T15:00:16.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T15:00:16.360 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T15:00:16.391 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:16.391 INFO:tasks.ceph:role: osd.3 2024-09-16T15:00:16.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:00:16.392 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:00:16.467 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:16.468 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:16.971 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-16T15:00:16.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T15:00:16.973 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-16T15:00:17.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T15:00:17.122 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T15:00:17.191 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:17.191 INFO:tasks.ceph:role: osd.4 2024-09-16T15:00:17.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:00:17.191 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:00:17.264 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:17.265 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:17.736 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-16T15:00:17.738 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T15:00:17.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-16T15:00:17.835 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T15:00:17.867 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T15:00:17.934 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-16T15:00:17.934 INFO:tasks.ceph:role: osd.5 2024-09-16T15:00:17.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:00:17.935 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:00:18.007 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:00:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:00:18.501 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-09-16T15:00:18.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T15:00:18.503 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-16T15:00:18.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T15:00:18.675 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:18.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.772+0000 7f4cb6644640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T15:00:18.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.773+0000 7f4cb6644640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T15:00:18.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.773+0000 7f4cb6644640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:00:18.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.773+0000 7f4cb6644640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:00:18.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.773+0000 7f4cb6644640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:00:18.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:18.781+0000 7f4cb6644640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T15:00:21.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T15:00:21.666 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:21.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.752+0000 7fcb5a41c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T15:00:21.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.752+0000 7fcb5a41c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T15:00:21.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.752+0000 7fcb5a41c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T15:00:21.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.753+0000 7fcb5a41c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T15:00:21.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.753+0000 7fcb5a41c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T15:00:21.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:21.753+0000 7fcb5a41c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T15:00:24.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T15:00:24.631 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:24.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.716+0000 7fb26e919640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T15:00:24.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.716+0000 7fb26e919640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T15:00:24.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.716+0000 7fb26e919640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T15:00:24.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.717+0000 7fb26e919640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T15:00:24.718 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.717+0000 7fb26e919640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T15:00:24.719 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:24.717+0000 7fb26e919640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T15:00:27.572 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T15:00:27.601 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T15:00:27.601 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:27.602 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T15:00:27.632 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:27.632 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T15:00:27.701 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:27.701 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T15:00:27.768 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:27.768 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T15:00:27.798 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:27.798 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T15:00:27.868 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:27.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T15:00:27.936 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:27.936 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T15:00:28.003 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:28.003 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T15:00:28.060 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T15:00:28.060 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:00:28.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:00:28.103 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-09-16T15:00:28.103 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQD3R+hmKRQHHxAABFVXVDAnO7ENSmvN5qLUyA== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQD6R+hm8C7iHRAAeeMGUH/2JOnUuJlMqA+ngg== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQD9R+hmOL5RHBAAhfQNoa+Rfo5L5XHABidMfQ== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQD1R+hmDa0VCBAAWDWmywoRrQudHznUMuPeiQ== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQACSOhmLxUeLhAA7Dh+OGPxN2tlcbj74XpO5Q== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAFSOhmnpvoLBAAL6uYNMx7XS43WqgTtdzazw== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAISOhmq6vDKhAArDw86foa3lehNFsjCbwyxA== 2024-09-16T15:00:28.104 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-09-16T15:00:28.105 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQD1R+hmTcGsChAAABdfbAK5MqW/i0HITL1BZA== 2024-09-16T15:00:28.124 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2024-09-16T15:00:28.124 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQD3R+hmKRQHHxAABFVXVDAnO7ENSmvN5qLUyA== 2024-09-16T15:00:28.124 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQD6R+hm8C7iHRAAeeMGUH/2JOnUuJlMqA+ngg== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQD9R+hmOL5RHBAAhfQNoa+Rfo5L5XHABidMfQ== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQD1R+hmDa0VCBAAWDWmywoRrQudHznUMuPeiQ== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQACSOhmLxUeLhAA7Dh+OGPxN2tlcbj74XpO5Q== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAFSOhmnpvoLBAAL6uYNMx7XS43WqgTtdzazw== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQAISOhmq6vDKhAArDw86foa3lehNFsjCbwyxA== 2024-09-16T15:00:28.125 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-09-16T15:00:28.126 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQD1R+hmTcGsChAAABdfbAK5MqW/i0HITL1BZA== 2024-09-16T15:00:28.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.221 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.275 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.324 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:00:28.326 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:00:28.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.379 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:00:28.534 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:00:28.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:00:28.589 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T15:00:28.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T15:00:28.614 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:28.709 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T15:00:28.733 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T15:00:28.798 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:28.893 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T15:00:28.917 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T15:00:28.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.076 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T15:00:29.100 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-09-16T15:00:29.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.260 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-09-16T15:00:29.284 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-09-16T15:00:29.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.444 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-09-16T15:00:29.468 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-09-16T15:00:29.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-09-16T15:00:29.618 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-09-16T15:00:29.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.781 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-09-16T15:00:29.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-09-16T15:00:29.869 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:29.965 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-09-16T15:00:29.990 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-09-16T15:00:30.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:00:30.153 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-09-16T15:00:30.178 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:30.180 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:00:30.233 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T15:00:30.233 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:00:30.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:00:30.236 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:00:30.236 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:00:30.236 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:00:30.238 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:00:30.239 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T15:00:30.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T15:00:30.241 INFO:tasks.ceph.mon.c:Started 2024-09-16T15:00:30.241 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-16T15:00:30.241 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-16T15:00:30.243 INFO:tasks.ceph.mon.d:Started 2024-09-16T15:00:30.243 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-16T15:00:30.243 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-16T15:00:30.245 INFO:tasks.ceph.mon.e:Started 2024-09-16T15:00:30.245 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-16T15:00:30.246 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-16T15:00:30.275 INFO:tasks.ceph.mon.f:Started 2024-09-16T15:00:30.275 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-16T15:00:30.275 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-16T15:00:30.277 INFO:tasks.ceph.mon.g:Started 2024-09-16T15:00:30.277 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-16T15:00:30.277 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-16T15:00:30.279 INFO:tasks.ceph.mon.h:Started 2024-09-16T15:00:30.279 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-16T15:00:30.279 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-16T15:00:30.281 INFO:tasks.ceph.mon.i:Started 2024-09-16T15:00:30.281 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T15:00:30.282 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T15:00:30.282 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T15:00:30.284 INFO:tasks.ceph.mgr.x:Started 2024-09-16T15:00:30.284 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T15:00:30.284 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.410+0000 7f66468b1640 10 monclient: get_monmap_and_config 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.410+0000 7f66468b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.410+0000 7f66468b1640 10 monclient: monmap: 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:30.411629+0000 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:30.411629+0000 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:30.412 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:30.413 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:30.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 Processor -- start 2024-09-16T15:00:30.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 -- start start 2024-09-16T15:00:30.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient: init 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient: _add_conns ranks=[7,1,10,5,3,2,8,0,4,6,9] 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-i con 0x7f66400a3940 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f664009ffb0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-f con 0x7f66400a0390 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6640073250 con 0x7f664009ffb0 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66400a6140 con 0x7f66400a0390 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f66400a3d20 con 0x7f66400a3940 2024-09-16T15:00:30.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.411+0000 7f66468b1640 10 monclient(hunting): authenticate will time out at 767.000978s 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:44110/0 (socket says 172.21.15.134:44110) 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:45228/0 (socket says 172.21.15.134:45228) 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 learned_addr learned my addr 172.21.15.134:0/3254892499 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 10 monclient(hunting): get_auth_request con 0x7f66400a3940 auth_method 0 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 10 monclient(hunting): get_auth_request con 0x7f66400a0390 auth_method 0 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:30.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.412+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:30.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.414+0000 7f663f7fe640 1 -- 172.21.15.134:0/3254892499 >> v1:172.21.15.162:6789/0 conn(0x7f664009ffb0 legacy=0x7f6640069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 10 monclient(hunting): get_auth_request con 0x7f66400a0390 auth_method 0 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 10 monclient(hunting): get_auth_request con 0x7f66400a3940 auth_method 0 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:30.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:30.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:30.613+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:00:31.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 10 monclient(hunting): get_auth_request con 0x7f66400a0390 auth_method 0 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 10 monclient(hunting): get_auth_request con 0x7f66400a3940 auth_method 0 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:31.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.014+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:00:31.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504172850 0 0) 0x7f6640073250 con 0x7f664009ffb0 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 10 monclient(hunting): _negotiate 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 10 monclient(hunting): my global_id is 4105 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.193+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6628003520 con 0x7f664009ffb0 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 1 --1- 172.21.15.134:0/3254892499 >> v1:172.21.15.162:6789/0 conn(0x7f664009ffb0 0x7f6640069e50 :-1 s=OPENED pgs=10 cs=1 l=1).queue_received will delay after 1.7265e+09 on 0x7f6634001be0 mon_map magic: 0 v1 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 973301317 0 0) 0x7f6628003520 con 0x7f664009ffb0 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 msgr2=0x7f6640134ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f66400a3940 0x7f6640134ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 msgr2=0x7f664006a560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 --2- 172.21.15.134:0/3254892499 >> v2:172.21.15.162:3300/0 conn(0x7f66400a0390 0x7f664006a560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66400a07d0 con 0x7f664009ffb0 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f66468b1640 5 monclient: authenticate success, global_id 4105 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f66468b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f6634001be0 con 0x7f664009ffb0 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: dump: 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.200 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 >> v1:172.21.15.162:6789/0 conn(0x7f664009ffb0 legacy=0x7f6640069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient: _add_conns ranks=[1,2,3,8,4,5,6,7,0] 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66280041f0 0x7f66280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): picked mon.f con 0x7f66280041f0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 0x7f6628009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): picked mon.b con 0x7f6628004e20 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6628009eb0 0x7f662800c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): picked mon.g con 0x7f6628009eb0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66400a6140 con 0x7f66280041f0 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66400a3d20 con 0x7f6628004e20 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662800c970 con 0x7f6628009eb0 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.1 v1:172.21.15.162:6789/0 5 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f6634003c30 con 0x7f664009ffb0 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663effd640 10 monclient(hunting): discarding stray monitor message mon_map magic: 0 v1 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f66468b1640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 0x7f6628009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66280041f0 0x7f66280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): get_auth_request con 0x7f6628004e20 auth_method 0 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): get_auth_request con 0x7f66280041f0 auth_method 0 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6628009eb0 0x7f662800c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): get_auth_request con 0x7f6628009eb0 auth_method 0 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.194+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 10 monclient(hunting): handle_auth_done global_id 4105 payload 293 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6628009eb0 msgr2=0x7f662800c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6628009eb0 0x7f662800c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66280041f0 msgr2=0x7f66280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66280041f0 0x7f66280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 monclient: found mon.b 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 1 -- 172.21.15.134:0/3254892499 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628003520 con 0x7f6628004e20 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f6644e4e640 1 -- 172.21.15.134:0/3254892499 reap_dead start 2024-09-16T15:00:31.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.195+0000 7f663ffff640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66280041f0 0x7f66280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:31.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.197+0000 7f663f7fe640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 msgr2=0x7f6628009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.197+0000 7f663f7fe640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 0x7f6628009970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6634003e10 tx=0x7f66280057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef151388651a4655 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6634008540 con 0x7f6628004e20 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: dump: 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.212 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66340057c0 con 0x7f6628004e20 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f66468b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f66468b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 1 -- 172.21.15.134:0/3254892499 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6634005da0 con 0x7f6628004e20 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f66468b1640 4 set_mon_vals no callback set 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.198+0000 7f663effd640 10 monclient: dump: 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.214 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 10 monclient: shutdown 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 -- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 msgr2=0x7f6628009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 0x7f6628009970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6634003e10 tx=0x7f66280057f0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 -- 172.21.15.134:0/3254892499 shutdown_connections 2024-09-16T15:00:31.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 --2- 172.21.15.134:0/3254892499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628004e20 0x7f6628009970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 -- 172.21.15.134:0/3254892499 >> 172.21.15.134:0/3254892499 conn(0x7f66400a2e20 msgr2=0x7f66400a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 -- 172.21.15.134:0/3254892499 shutdown_connections 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.199+0000 7f66468b1640 1 -- 172.21.15.134:0/3254892499 wait complete. 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.200+0000 7f66468b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.200+0000 7f66468b1640 10 monclient: monmap: 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:31.216 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 Processor -- start 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 -- start start 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient: init 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient: _add_conns ranks=[4,1,8,6,5,2,3,7,0] 2024-09-16T15:00:31.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 0x7f6640099450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-c con 0x7f6628004e20 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6640099990 0x7f664012b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-f con 0x7f6640099990 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 0x7f664012bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): picked mon.noname-e con 0x7f664012b920 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664012e090 con 0x7f6640099990 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f664009f790 con 0x7f6628004e20 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f664012d7f0 con 0x7f664012b920 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f66468b1640 10 monclient(hunting): authenticate will time out at 767.790747s 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 0x7f6640099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f6644e4e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 0x7f664012bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 0x7f6640099450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41914/0 (socket says 172.21.15.134:41914) 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f6644e4e640 10 monclient(hunting): get_auth_request con 0x7f664012b920 auth_method 0 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 1 -- 172.21.15.134:0/2666199676 learned_addr learned my addr 172.21.15.134:0/2666199676 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f6644e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f6644e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f6644e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): get_auth_request con 0x7f6628004e20 auth_method 0 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663f7fe640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6640099990 0x7f664012b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663f7fe640 10 monclient(hunting): get_auth_request con 0x7f6640099990 auth_method 0 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.201+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f663ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f663f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient(hunting): handle_auth_done global_id 4112 payload 824 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 -- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 msgr2=0x7f6640099450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 0x7f6640099450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 -- 172.21.15.134:0/2666199676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6640099990 msgr2=0x7f664012b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6640099990 0x7f664012b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 monclient: found mon.noname-e 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.202+0000 7f6644e4e640 1 -- 172.21.15.134:0/2666199676 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640132230 con 0x7f664012b920 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.204+0000 7f6644e4e640 1 -- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 msgr2=0x7f664012bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.204+0000 7f6644e4e640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 0x7f664012bda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66400a3e60 tx=0x7f6638003f70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6638007e60 con 0x7f664012b920 2024-09-16T15:00:31.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: dump: 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.222 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663800f040 con 0x7f664012b920 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 5 monclient: authenticate success, global_id 4112 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 10 monclient: _renew_subs 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6638018ba0 con 0x7f664012b920 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6640132700 con 0x7f664012b920 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:31.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663e7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 10 monclient: dump: 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.224 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 10 monclient: _renew_subs 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6640099d70 con 0x7f664012b920 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f663801e070 con 0x7f664012b920 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.208+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6638021d90 con 0x7f664012b920 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.209+0000 7f66468b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.209+0000 7f66468b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.209+0000 7f66468b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:31.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.209+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6640072550 con 0x7f664012b920 2024-09-16T15:00:31.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.212+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f663803fe30 con 0x7f664012b920 2024-09-16T15:00:31.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.212+0000 7f663cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.212+0000 7f663cff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:31.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.272+0000 7f66468b1640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-16T15:00:31.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.272+0000 7f66468b1640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-16T15:00:31.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.272+0000 7f66468b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:31.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.272+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f664009d1b0 con 0x7f664012b920 2024-09-16T15:00:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.280+0000 7f663cff9640 1 -- 172.21.15.134:0/2666199676 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f663802e020 con 0x7f664012b920 2024-09-16T15:00:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.280+0000 7f663cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-16T15:00:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.280+0000 7f663cff9640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-16T15:00:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:adjusted tunables profile to default 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 10 monclient: shutdown 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 msgr2=0x7f664012bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 0x7f664012bda0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66400a3e60 tx=0x7f6638003f70 comp rx=0 tx=0).stop 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 shutdown_connections 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f664012b920 0x7f664012bda0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6640099990 0x7f664012b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 --2- 172.21.15.134:0/2666199676 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6628004e20 0x7f6640099450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 >> 172.21.15.134:0/2666199676 conn(0x7f66400a2e20 msgr2=0x7f6640065c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 shutdown_connections 2024-09-16T15:00:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:31.281+0000 7f66468b1640 1 -- 172.21.15.134:0/2666199676 wait complete. 2024-09-16T15:00:31.293 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T15:00:31.293 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T15:00:31.293 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:31.293 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T15:00:31.317 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:31.317 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T15:00:31.381 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-16T15:00:31.382 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T15:00:31.446 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:31.446 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T15:00:31.474 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:31.474 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T15:00:31.540 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T15:00:31.541 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T15:00:31.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 8700f1de-9dc0-436a-a2f6-7e955103c5c1 0 2024-09-16T15:00:31.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.744+0000 7efdc2420640 10 monclient: get_monmap_and_config 2024-09-16T15:00:31.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.744+0000 7efdc2420640 10 monclient: build_initial_monmap 2024-09-16T15:00:31.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.744+0000 7efdc2420640 10 monclient: monmap: 2024-09-16T15:00:31.746 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:31.746 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:31.745712+0000 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:31.745712+0000 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:31.747 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:31.748 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:31.748 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:31.748 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.745+0000 7efdc2420640 1 Processor -- start 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.745+0000 7efdc2420640 1 -- start start 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.745+0000 7efdc2420640 10 monclient: init 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient: _add_conns ranks=[9,0,7,8,2,6,4,1,3,10,5] 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 0x7efdbc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-a con 0x7efdbc136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efdbc137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-i con 0x7efdbc13c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:31.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdbc073250 con 0x7efdbc137220 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdbc0a56e0 con 0x7efdbc136e40 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc09abc0 con 0x7efdbc13c370 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc2420640 10 monclient(hunting): authenticate will time out at 783.051209s 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.162:46180/0 (socket says 172.21.15.162:46180) 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 -- 172.21.15.162:0/233939192 learned_addr learned my addr 172.21.15.162:0/233939192 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): get_auth_request con 0x7efdbc13c370 auth_method 0 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdbbfff640 1 --2- 172.21.15.162:0/233939192 >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 0x7efdbc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient(hunting): handle_auth_done global_id 4111 payload 274 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 -- 172.21.15.162:0/233939192 >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 msgr2=0x7efdbc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 --2- 172.21.15.162:0/233939192 >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 0x7efdbc09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 -- 172.21.15.162:0/233939192 >> v1:172.21.15.134:6789/0 conn(0x7efdbc137220 legacy=0x7efdbc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 monclient: found mon.noname-i 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdc09bd640 1 -- 172.21.15.162:0/233939192 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc09ba50 con 0x7efdbc13c370 2024-09-16T15:00:31.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdbbfff640 10 monclient: get_auth_request con 0x7efdbc136e40 auth_method 0 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.746+0000 7efdbbfff640 1 --2- 172.21.15.162:0/233939192 >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 0x7efdbc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdc09bd640 1 -- 172.21.15.162:0/233939192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 msgr2=0x7efdbc13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdc09bd640 1 --2- 172.21.15.162:0/233939192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009e50 tx=0x7efdb402a5b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cf9461d7eef935da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 1 -- 172.21.15.162:0/233939192 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdb4007a70 con 0x7efdbc13c370 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: dump: 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.758 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 1 -- 172.21.15.162:0/233939192 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb402d050 con 0x7efdbc13c370 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 1 -- 172.21.15.162:0/233939192 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdb4007c30 con 0x7efdbc13c370 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdbaffd640 10 monclient: dump: 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.760 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdc2420640 5 monclient: authenticate success, global_id 4111 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdc2420640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:31.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.749+0000 7efdc2420640 4 set_mon_vals no callback set 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 10 monclient: shutdown 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 -- 172.21.15.162:0/233939192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 msgr2=0x7efdbc13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 --2- 172.21.15.162:0/233939192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009e50 tx=0x7efdb402a5b0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 -- 172.21.15.162:0/233939192 shutdown_connections 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 --2- 172.21.15.162:0/233939192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efdbc13c370 0x7efdbc13e760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 --2- 172.21.15.162:0/233939192 >> v2:172.21.15.134:3300/0 conn(0x7efdbc136e40 0x7efdbc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 -- 172.21.15.162:0/233939192 >> 172.21.15.162:0/233939192 conn(0x7efdbc0a6110 msgr2=0x7efdbc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 -- 172.21.15.162:0/233939192 shutdown_connections 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 1 -- 172.21.15.162:0/233939192 wait complete. 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 10 monclient: build_initial_monmap 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.750+0000 7efdc2420640 10 monclient: monmap: 2024-09-16T15:00:31.762 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 Processor -- start 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 -- start start 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient: init 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient: _add_conns ranks=[6,8,1,3,4,2,0,5,7] 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efdbc136e40 0x7efdbc11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-d con 0x7efdbc136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efdbc137220 0x7efdbc1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-e con 0x7efdbc137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): picked mon.noname-f con 0x7efdbc13c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc122440 con 0x7efdbc13c370 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc122690 con 0x7efdbc136e40 2024-09-16T15:00:31.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc13b410 con 0x7efdbc137220 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc2420640 10 monclient(hunting): authenticate will time out at 783.056794s 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59016/0 (socket says 172.21.15.162:59016) 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 1 -- 172.21.15.162:0/3698689588 learned_addr learned my addr 172.21.15.162:0/3698689588 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 10 monclient(hunting): get_auth_request con 0x7efdbc13c370 auth_method 0 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.751+0000 7efdc09bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efdbc137220 0x7efdbc1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efdbc136e40 0x7efdbc11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): get_auth_request con 0x7efdbc137220 auth_method 0 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): get_auth_request con 0x7efdbc136e40 auth_method 0 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient(hunting): handle_auth_done global_id 4123 payload 824 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 -- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efdbc137220 msgr2=0x7efdbc1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efdbc137220 0x7efdbc1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 -- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efdbc136e40 msgr2=0x7efdbc11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efdbc136e40 0x7efdbc11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 monclient: found mon.noname-f 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:31.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.752+0000 7efdc09bd640 1 -- 172.21.15.162:0/3698689588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009b30 con 0x7efdbc13c370 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc09bd640 1 -- 172.21.15.162:0/3698689588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 msgr2=0x7efdbc1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc09bd640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdb4034040 tx=0x7efdb4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdb4004710 con 0x7efdbc13c370 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: dump: 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.769 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.770 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb4007a70 con 0x7efdbc13c370 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efdb4007be0 con 0x7efdbc13c370 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdb8ff9640 10 monclient: dump: 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.771 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 5 monclient: authenticate success, global_id 4123 2024-09-16T15:00:31.772 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdba7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 10 monclient: _renew_subs 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc0a7960 con 0x7efdbc13c370 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 10 monclient: _renew_subs 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.754+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc121df0 con 0x7efdbc13c370 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7efdb4038070 con 0x7efdbc13c370 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7efdb40447a0 con 0x7efdbc13c370 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdc2420640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdc2420640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdc2420640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:31.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.755+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc09b490 con 0x7efdbc13c370 2024-09-16T15:00:31.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.757+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7efdb403f030 con 0x7efdbc13c370 2024-09-16T15:00:31.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.757+0000 7efdb8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.757+0000 7efdb8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.827+0000 7efdc2420640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8700f1de-9dc0-436a-a2f6-7e955103c5c1", "id": 0}] 2024-09-16T15:00:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.827+0000 7efdc2420640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8700f1de-9dc0-436a-a2f6-7e955103c5c1", "id": 0}] 2024-09-16T15:00:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.827+0000 7efdc2420640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:31.828 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.827+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8700f1de-9dc0-436a-a2f6-7e955103c5c1", "id": 0} v 0) v1 -- 0x7efdbc0008d0 con 0x7efdbc13c370 2024-09-16T15:00:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdb8ff9640 1 -- 172.21.15.162:0/3698689588 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8700f1de-9dc0-436a-a2f6-7e955103c5c1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efdb4036070 con 0x7efdbc13c370 2024-09-16T15:00:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdb8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8700f1de-9dc0-436a-a2f6-7e955103c5c1", "id": 0}] 2024-09-16T15:00:31.835 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdb8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:31.835 INFO:teuthology.orchestra.run.smithi162.stdout:0 2024-09-16T15:00:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdc2420640 10 monclient: shutdown 2024-09-16T15:00:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdc2420640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:31.836 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 msgr2=0x7efdbc1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.834+0000 7efdc2420640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdb4034040 tx=0x7efdb4004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 shutdown_connections 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efdbc13c370 0x7efdbc1206e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efdbc137220 0x7efdbc1201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 --2- 172.21.15.162:0/3698689588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efdbc136e40 0x7efdbc11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 >> 172.21.15.162:0/3698689588 conn(0x7efdbc0a6110 msgr2=0x7efdbc13b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 shutdown_connections 2024-09-16T15:00:31.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.835+0000 7efdc2420640 1 -- 172.21.15.162:0/3698689588 wait complete. 2024-09-16T15:00:31.846 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 92341517-5f3d-458a-904f-eae7c1d454a0 1 2024-09-16T15:00:31.892 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:31.891+0000 7f4033e38040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:00:31.935 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:31.933+0000 7f4033e38040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:00:31.943 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.942+0000 7ff04e4d7640 10 monclient: get_monmap_and_config 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.942+0000 7ff04e4d7640 10 monclient: build_initial_monmap 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.942+0000 7ff04e4d7640 10 monclient: monmap: 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:31.943413+0000 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:31.943413+0000 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:31.944 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:31.945 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 Processor -- start 2024-09-16T15:00:31.947 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 -- start start 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient: init 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient: _add_conns ranks=[6,1,3,9,10,8,4,7,0,5,2] 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff048069e50 0x7ff04806a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-d con 0x7ff048069e50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff04806a770 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff04813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-g con 0x7ff04813ccc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff048073250 con 0x7ff04806a770 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff048099d30 con 0x7ff04813ccc0 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.948 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480a7260 con 0x7ff048069e50 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.943+0000 7ff04e4d7640 10 monclient(hunting): authenticate will time out at 783.248920s 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff0477fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff04806a770 0x7ff04813b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57318/0 (socket says 172.21.15.162:57318) 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff0477fe640 1 -- 172.21.15.162:0/4140672211 learned_addr learned my addr 172.21.15.162:0/4140672211 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff04813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): get_auth_request con 0x7ff04813ccc0 auth_method 0 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff048069e50 0x7ff04806a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.949 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): get_auth_request con 0x7ff048069e50 auth_method 0 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient(hunting): handle_auth_done global_id 4107 payload 274 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 -- 172.21.15.162:0/4140672211 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff048069e50 msgr2=0x7ff04806a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff048069e50 0x7ff04806a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 -- 172.21.15.162:0/4140672211 >> v1:172.21.15.162:6789/0 conn(0x7ff04806a770 legacy=0x7ff04813b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.950 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 monclient: found mon.noname-g 2024-09-16T15:00:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:31.951 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.944+0000 7ff04ca74640 1 -- 172.21.15.162:0/4140672211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048067020 con 0x7ff04813ccc0 2024-09-16T15:00:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff04ca74640 1 -- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 msgr2=0x7ff04813f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff04ca74640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff04813f0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff040009dc0 tx=0x7ff04002a4d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3afff0690601f92b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 1 -- 172.21.15.162:0/4140672211 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff040007bc0 con 0x7ff04813ccc0 2024-09-16T15:00:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.956 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: dump: 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.957 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 1 -- 172.21.15.162:0/4140672211 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04002d050 con 0x7ff04813ccc0 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 1 -- 172.21.15.162:0/4140672211 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff040007d40 con 0x7ff04813ccc0 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff04e4d7640 5 monclient: authenticate success, global_id 4107 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff04e4d7640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff04e4d7640 4 set_mon_vals no callback set 2024-09-16T15:00:31.958 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.947+0000 7ff046ffd640 10 monclient: dump: 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.959 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 10 monclient: shutdown 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 -- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 msgr2=0x7ff04813f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff04813f0b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff040009dc0 tx=0x7ff04002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 -- 172.21.15.162:0/4140672211 shutdown_connections 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff04813f0b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/4140672211 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff048069e50 0x7ff04806a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 -- 172.21.15.162:0/4140672211 >> 172.21.15.162:0/4140672211 conn(0x7ff0480a2f00 msgr2=0x7ff0480a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 -- 172.21.15.162:0/4140672211 shutdown_connections 2024-09-16T15:00:31.960 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.948+0000 7ff04e4d7640 1 -- 172.21.15.162:0/4140672211 wait complete. 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.949+0000 7ff04e4d7640 10 monclient: build_initial_monmap 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.949+0000 7ff04e4d7640 10 monclient: monmap: 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:31.961 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.949+0000 7ff04e4d7640 1 Processor -- start 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.949+0000 7ff04e4d7640 1 -- start start 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.949+0000 7ff04e4d7640 10 monclient: init 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient: _add_conns ranks=[4,2,3,1,0,8,7,5,6] 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-c con 0x7ff048069e50 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 0x7ff048070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.962 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-b con 0x7ff04806a770 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): picked mon.noname-g con 0x7ff04813ccc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff048066ae0 con 0x7ff04806a770 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff048092640 con 0x7ff04813ccc0 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff048090eb0 con 0x7ff048069e50 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04e4d7640 10 monclient(hunting): authenticate will time out at 783.255244s 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.963 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 0x7ff048070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55932/0 (socket says 172.21.15.162:55932) 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 -- 172.21.15.162:0/159459937 learned_addr learned my addr 172.21.15.162:0/159459937 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.162:45952/0 (socket says 172.21.15.162:45952) 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): get_auth_request con 0x7ff04813ccc0 auth_method 0 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): get_auth_request con 0x7ff048069e50 auth_method 0 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): get_auth_request con 0x7ff04806a770 auth_method 0 2024-09-16T15:00:31.964 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient(hunting): handle_auth_done global_id 4116 payload 824 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 -- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 msgr2=0x7ff048098e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.965 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 -- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 msgr2=0x7ff048070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 0x7ff048070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient: _finish_hunting 0 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 monclient: found mon.noname-g 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff04ca74640 1 -- 172.21.15.162:0/159459937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040009a70 con 0x7ff04813ccc0 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff0477fe640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 0x7ff048070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.950+0000 7ff047fff640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.952+0000 7ff04ca74640 1 -- 172.21.15.162:0/159459937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 msgr2=0x7ff048070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04ca74640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff04002aac0 tx=0x7ff040004a40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff040007ea0 con 0x7ff04813ccc0 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.966 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: dump: 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.967 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0400044b0 con 0x7ff04813ccc0 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 5 monclient: authenticate success, global_id 4116 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 10 monclient: _renew_subs 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff048098330 con 0x7ff04813ccc0 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff040004620 con 0x7ff04813ccc0 2024-09-16T15:00:31.968 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 10 monclient: dump: 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:31.969 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff0467fc640 4 set_mon_vals no callback set 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 10 monclient: _renew_subs 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff048091060 con 0x7ff04813ccc0 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff04003b410 con 0x7ff04813ccc0 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.953+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff0400437b0 con 0x7ff04813ccc0 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.954+0000 7ff04e4d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.954+0000 7ff04e4d7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.954+0000 7ff04e4d7640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:31.970 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.954+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048090780 con 0x7ff04813ccc0 2024-09-16T15:00:31.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.956+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff0400047c0 con 0x7ff04813ccc0 2024-09-16T15:00:31.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.956+0000 7ff044ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:31.971 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:31.956+0000 7ff044ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.024+0000 7ff04e4d7640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "92341517-5f3d-458a-904f-eae7c1d454a0", "id": 1}] 2024-09-16T15:00:32.025 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.024+0000 7ff04e4d7640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "92341517-5f3d-458a-904f-eae7c1d454a0", "id": 1}] 2024-09-16T15:00:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.024+0000 7ff04e4d7640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:32.026 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.024+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "92341517-5f3d-458a-904f-eae7c1d454a0", "id": 1} v 0) v1 -- 0x7ff04809e780 con 0x7ff04813ccc0 2024-09-16T15:00:32.087 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.086+0000 7f4033e38040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:00:32.137 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.135+0000 7f4033e38040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:00:32.298 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.297+0000 7f4033e38040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:00:32.406 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.405+0000 7f4033e38040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:00:32.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.565+0000 7ff044ff9640 1 -- 172.21.15.162:0/159459937 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92341517-5f3d-458a-904f-eae7c1d454a0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff040043a90 con 0x7ff04813ccc0 2024-09-16T15:00:32.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.565+0000 7ff044ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "92341517-5f3d-458a-904f-eae7c1d454a0", "id": 1}] 2024-09-16T15:00:32.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.565+0000 7ff044ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:32.568 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 10 monclient: shutdown 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 msgr2=0x7ff048070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff04002aac0 tx=0x7ff040004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 shutdown_connections 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04813ccc0 0x7ff048070ca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff04806a770 0x7ff048070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 --2- 172.21.15.162:0/159459937 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff048069e50 0x7ff048098e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 >> 172.21.15.162:0/159459937 conn(0x7ff0480a2f00 msgr2=0x7ff04809cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 shutdown_connections 2024-09-16T15:00:32.569 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.566+0000 7ff04e4d7640 1 -- 172.21.15.162:0/159459937 wait complete. 2024-09-16T15:00:32.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 12bf62b9-5ecd-4476-8a55-c620754c3c85 2 2024-09-16T15:00:32.678 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.676+0000 7f858da06640 10 monclient: get_monmap_and_config 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.676+0000 7f858da06640 10 monclient: build_initial_monmap 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.676+0000 7f858da06640 10 monclient: monmap: 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:32.678262+0000 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:32.678262+0000 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:32.679 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:32.680 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 Processor -- start 2024-09-16T15:00:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 -- start start 2024-09-16T15:00:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient: init 2024-09-16T15:00:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:32.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient: _add_conns ranks=[8,0,9,10,6,5,3,4,7,1,2] 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 0x7f8588130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): picked mon.noname-e con 0x7f8588130ab0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f85881305f0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8588134200 0x7f858813be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): picked mon.noname-a con 0x7f8588134200 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8588073250 con 0x7f85881305f0 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858809f600 con 0x7f8588134200 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f85880673e0 con 0x7f8588130ab0 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:32.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f858da06640 10 monclient(hunting): authenticate will time out at 783.983664s 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8588134200 0x7f858813be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8588134200 0x7f858813be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.162:33686/0 (socket says 172.21.15.162:33686) 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 1 -- 172.21.15.162:0/983856703 learned_addr learned my addr 172.21.15.162:0/983856703 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f8586ffd640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 0x7f8588130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 10 monclient(hunting): get_auth_request con 0x7f8588134200 auth_method 0 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.677+0000 7f85877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): get_auth_request con 0x7f8588130ab0 auth_method 0 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:32.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient(hunting): handle_auth_done global_id 4113 payload 274 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 1 -- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 msgr2=0x7f8588130e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 0x7f8588130e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 1 -- 172.21.15.162:0/983856703 >> v1:172.21.15.134:6789/0 conn(0x7f85881305f0 legacy=0x7f8588133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 1 monclient: found mon.noname-a 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f85877fe640 1 -- 172.21.15.162:0/983856703 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858809e740 con 0x7f8588134200 2024-09-16T15:00:32.690 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.678+0000 7f8586ffd640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 0x7f8588130e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f85877fe640 1 -- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588134200 msgr2=0x7f858813be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f85877fe640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588134200 0x7f858813be30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f858000d150 tx=0x7f858002c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a1dc4eaabe49a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 1 -- 172.21.15.162:0/983856703 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8580030050 con 0x7f8588134200 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: dump: 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:32.691 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 1 -- 172.21.15.162:0/983856703 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8580009a70 con 0x7f8588134200 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 5 monclient: authenticate success, global_id 4113 2024-09-16T15:00:32.692 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 1 -- 172.21.15.162:0/983856703 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8580009c20 con 0x7f8588134200 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 4 set_mon_vals no callback set 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f8585ffb640 10 monclient: dump: 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.693 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 10 monclient: shutdown 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 1 -- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588134200 msgr2=0x7f858813be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.681+0000 7f858da06640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588134200 0x7f858813be30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f858000d150 tx=0x7f858002c840 comp rx=0 tx=0).stop 2024-09-16T15:00:32.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 -- 172.21.15.162:0/983856703 shutdown_connections 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8588134200 0x7f858813be30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 --2- 172.21.15.162:0/983856703 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8588130ab0 0x7f8588130e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 -- 172.21.15.162:0/983856703 >> 172.21.15.162:0/983856703 conn(0x7f85880a6400 msgr2=0x7f85880a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 -- 172.21.15.162:0/983856703 shutdown_connections 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 -- 172.21.15.162:0/983856703 wait complete. 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 10 monclient: build_initial_monmap 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 10 monclient: monmap: 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.695 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.682+0000 7f858da06640 1 Processor -- start 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 -- start start 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient: init 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:32.696 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient: _add_conns ranks=[1,7,6,0,3,4,2,5,8] 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): picked mon.noname-f con 0x7f85881305f0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 0x7f858806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): picked mon.noname-i con 0x7f8588130ab0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8588134200 0x7f85880701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): picked mon.noname-d con 0x7f8588134200 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588133d80 con 0x7f85881305f0 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8588072050 con 0x7f8588134200 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f858808c140 con 0x7f8588130ab0 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:32.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f858da06640 10 monclient(hunting): authenticate will time out at 783.989673s 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 0x7f858806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59036/0 (socket says 172.21.15.162:59036) 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 1 -- 172.21.15.162:0/517949278 learned_addr learned my addr 172.21.15.162:0/517949278 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): get_auth_request con 0x7f85881305f0 auth_method 0 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): get_auth_request con 0x7f8588130ab0 auth_method 0 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:32.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f8586ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.683+0000 7f85867fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 10 monclient(hunting): handle_auth_done global_id 4141 payload 824 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f85877fe640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8588134200 0x7f85880701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 -- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 msgr2=0x7f858806fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 0x7f858806fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 -- 172.21.15.162:0/517949278 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8588134200 msgr2=0x7f85880701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8588134200 0x7f85880701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 monclient: found mon.noname-f 2024-09-16T15:00:32.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:32.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:32.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f8586ffd640 1 -- 172.21.15.162:0/517949278 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8578009400 con 0x7f85881305f0 2024-09-16T15:00:32.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.684+0000 7f85867fc640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 0x7f858806fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:32.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f8586ffd640 1 -- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 msgr2=0x7f858806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:32.701 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f8586ffd640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f857800ebe0 tx=0x7f857800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8578009760 con 0x7f85881305f0 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: dump: 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:32.702 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800ce40 con 0x7f85881305f0 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 5 monclient: authenticate success, global_id 4141 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 10 monclient: _renew_subs 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:32.703 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858000bd40 con 0x7f85881305f0 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f857800eee0 con 0x7f85881305f0 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 10 monclient: dump: 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:32.704 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 10 monclient: _renew_subs 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f85857fa640 4 set_mon_vals no callback set 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8588131aa0 con 0x7f85881305f0 2024-09-16T15:00:32.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8578004320 con 0x7f85881305f0 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.686+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8578005990 con 0x7f85881305f0 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.687+0000 7f858da06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.687+0000 7f858da06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.687+0000 7f858da06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.687+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85880a0030 con 0x7f85881305f0 2024-09-16T15:00:32.706 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:32.689+0000 7f8586ffd640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f857800ebe0 tx=0x7f857800c520 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f8578017070 mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 2024-09-16T15:00:32.751 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.749+0000 7f4033e38040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:00:32.801 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.799+0000 7f4033e38040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:00:32.947 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:32.944+0000 7f4033e38040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:00:33.519 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.517+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f8578017070 con 0x7f85881305f0 2024-09-16T15:00:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.517+0000 7f856ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:33.520 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.517+0000 7f856ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:33.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.586+0000 7f858da06640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "12bf62b9-5ecd-4476-8a55-c620754c3c85", "id": 2}] 2024-09-16T15:00:33.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.586+0000 7f858da06640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "12bf62b9-5ecd-4476-8a55-c620754c3c85", "id": 2}] 2024-09-16T15:00:33.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.586+0000 7f858da06640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:33.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.586+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12bf62b9-5ecd-4476-8a55-c620754c3c85", "id": 2} v 0) v1 -- 0x7f858809ba10 con 0x7f85881305f0 2024-09-16T15:00:33.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.641+0000 7f856ffff640 1 -- 172.21.15.162:0/517949278 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12bf62b9-5ecd-4476-8a55-c620754c3c85", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8578006820 con 0x7f85881305f0 2024-09-16T15:00:33.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.641+0000 7f856ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "12bf62b9-5ecd-4476-8a55-c620754c3c85", "id": 2}] 2024-09-16T15:00:33.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.641+0000 7f856ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:33.644 INFO:teuthology.orchestra.run.smithi162.stdout:2 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 10 monclient: shutdown 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 msgr2=0x7f858806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f857800ebe0 tx=0x7f857800c520 comp rx=0 tx=0).stop 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 shutdown_connections 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8588134200 0x7f85880701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8588130ab0 0x7f858806fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 --2- 172.21.15.162:0/517949278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85881305f0 0x7f858806f730 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 >> 172.21.15.162:0/517949278 conn(0x7f85880a6400 msgr2=0x7f85880995a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 shutdown_connections 2024-09-16T15:00:33.645 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.642+0000 7f858da06640 1 -- 172.21.15.162:0/517949278 wait complete. 2024-09-16T15:00:33.658 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 91058726-d953-423c-8958-65f169388831 3 2024-09-16T15:00:33.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.753+0000 7fbfc5061640 10 monclient: get_monmap_and_config 2024-09-16T15:00:33.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.753+0000 7fbfc5061640 10 monclient: build_initial_monmap 2024-09-16T15:00:33.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.753+0000 7fbfc5061640 10 monclient: monmap: 2024-09-16T15:00:33.756 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:33.755670+0000 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:33.755670+0000 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:33.757 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 Processor -- start 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 -- start start 2024-09-16T15:00:33.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient: init 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient: _add_conns ranks=[1,5,4,3,7,2,8,0,6,9,10] 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbfc0136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-h con 0x7fbfc0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 0x7fbfc013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-c con 0x7fbfc013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc0073250 con 0x7fbfc0136e40 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00a56e0 con 0x7fbfc013c370 2024-09-16T15:00:33.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc009abc0 con 0x7fbfc0137220 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.754+0000 7fbfc5061640 10 monclient(hunting): authenticate will time out at 785.060983s 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbed9d640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbfc0136e40 0x7fbfc009c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:57326/0 (socket says 172.21.15.162:57326) 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbed9d640 1 -- 172.21.15.162:0/3030272562 learned_addr learned my addr 172.21.15.162:0/3030272562 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): get_auth_request con 0x7fbfc0137220 auth_method 0 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 0x7fbfc013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:33.763 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 10 monclient(hunting): get_auth_request con 0x7fbfc013c370 auth_method 0 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_done global_id 4109 payload 274 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 -- 172.21.15.162:0/3030272562 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 msgr2=0x7fbfc013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 0x7fbfc013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 -- 172.21.15.162:0/3030272562 >> v1:172.21.15.162:6789/0 conn(0x7fbfc0136e40 legacy=0x7fbfc009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 monclient: found mon.noname-h 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbe59c640 1 -- 172.21.15.162:0/3030272562 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc009ba50 con 0x7fbfc0137220 2024-09-16T15:00:33.764 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.755+0000 7fbfbf59e640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 0x7fbfc013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbe59c640 1 -- 172.21.15.162:0/3030272562 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 msgr2=0x7fbfc0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbe59c640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0137600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097d0 tx=0x7fbfa802a000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=acb305b987b19ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 1 -- 172.21.15.162:0/3030272562 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa802ad20 con 0x7fbfc0137220 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: dump: 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:33.765 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: _finish_auth 0 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 1 -- 172.21.15.162:0/3030272562 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80307b0 con 0x7fbfc0137220 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 1 -- 172.21.15.162:0/3030272562 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfa802aea0 con 0x7fbfc0137220 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfc5061640 5 monclient: authenticate success, global_id 4109 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfc5061640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:33.766 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfc5061640 4 set_mon_vals no callback set 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.758+0000 7fbfbdd9b640 10 monclient: dump: 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:33.767 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 10 monclient: shutdown 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 -- 172.21.15.162:0/3030272562 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 msgr2=0x7fbfc0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0137600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097d0 tx=0x7fbfa802a000 comp rx=0 tx=0).stop 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 -- 172.21.15.162:0/3030272562 shutdown_connections 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbfc013c370 0x7fbfc013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 --2- 172.21.15.162:0/3030272562 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0137600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.768 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 -- 172.21.15.162:0/3030272562 >> 172.21.15.162:0/3030272562 conn(0x7fbfc00a6110 msgr2=0x7fbfc00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:33.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 -- 172.21.15.162:0/3030272562 shutdown_connections 2024-09-16T15:00:33.769 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.759+0000 7fbfc5061640 1 -- 172.21.15.162:0/3030272562 wait complete. 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.760+0000 7fbfc5061640 10 monclient: build_initial_monmap 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.760+0000 7fbfc5061640 10 monclient: monmap: 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:33.773 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.761+0000 7fbfc5061640 1 Processor -- start 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.761+0000 7fbfc5061640 1 -- start start 2024-09-16T15:00:33.774 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.761+0000 7fbfc5061640 10 monclient: init 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient: _add_conns ranks=[7,5,0,8,6,2,4,3,1] 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-i con 0x7fbfc0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0136040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-h con 0x7fbfc0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 0x7fbfc011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): picked mon.noname-a con 0x7fbfc013c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00a75a0 con 0x7fbfc013c370 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0121000 con 0x7fbfc0137220 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0122440 con 0x7fbfc0136e40 2024-09-16T15:00:33.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfc5061640 10 monclient(hunting): authenticate will time out at 785.068177s 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0136040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.162:46220/0 (socket says 172.21.15.162:46220) 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 -- 172.21.15.162:0/509211887 learned_addr learned my addr 172.21.15.162:0/509211887 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0136040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.162:57312/0 (socket says 172.21.15.162:57312) 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): get_auth_request con 0x7fbfc0136e40 auth_method 0 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): get_auth_request con 0x7fbfc0137220 auth_method 0 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:33.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbe59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 0x7fbfc011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 10 monclient(hunting): get_auth_request con 0x7fbfc013c370 auth_method 0 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient(hunting): handle_auth_done global_id 4138 payload 824 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 -- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 msgr2=0x7fbfc0136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:33.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0136040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 -- 172.21.15.162:0/509211887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 msgr2=0x7fbfc011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 0x7fbfc011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 monclient: found mon.noname-i 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbed9d640 1 -- 172.21.15.162:0/509211887 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009400 con 0x7fbfc0136e40 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.762+0000 7fbfbf59e640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 0x7fbfc011fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.764+0000 7fbfbed9d640 1 -- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 msgr2=0x7fbfc0135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.764+0000 7fbfbed9d640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c460 tx=0x7fbfb000c920 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb0005cf0 con 0x7fbfc0136e40 2024-09-16T15:00:33.778 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: dump: 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:33.779 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0005e90 con 0x7fbfc0136e40 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 5 monclient: authenticate success, global_id 4138 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfb00074e0 con 0x7fbfc0136e40 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 10 monclient: _renew_subs 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:33.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc013b510 con 0x7fbfc0136e40 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfbd59a640 4 set_mon_vals no callback set 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 10 monclient: dump: 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:33.781 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 10 monclient: _renew_subs 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc00a3360 con 0x7fbfc0136e40 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fbfb0012400 con 0x7fbfc0136e40 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.765+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fbfb00076c0 con 0x7fbfc0136e40 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.766+0000 7fbfc5061640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.766+0000 7fbfc5061640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:33.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.766+0000 7fbfc5061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:33.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.766+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc009ad60 con 0x7fbfc0136e40 2024-09-16T15:00:33.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.768+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fbfb00126b0 con 0x7fbfc0136e40 2024-09-16T15:00:33.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.768+0000 7fbfb77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:33.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.768+0000 7fbfb77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:33.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.867+0000 7fbfc5061640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "91058726-d953-423c-8958-65f169388831", "id": 3}] 2024-09-16T15:00:33.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.867+0000 7fbfc5061640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "91058726-d953-423c-8958-65f169388831", "id": 3}] 2024-09-16T15:00:33.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.867+0000 7fbfc5061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:33.870 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:33.867+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "osd new", "uuid": "91058726-d953-423c-8958-65f169388831", "id": 3} v 0) v1 -- 0x7fbfc009b770 con 0x7fbfc0136e40 2024-09-16T15:00:33.945 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:33.942+0000 7f4033e38040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:00:33.994 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:33.992+0000 7f4033e38040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:00:34.354 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.352+0000 7f4033e38040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T15:00:34.437 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.435+0000 7f4033e38040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:00:34.542 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.540+0000 7f4033e38040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:00:34.588 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.586+0000 7f4033e38040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:00:34.667 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.665+0000 7fbfb77fe640 1 -- 172.21.15.162:0/509211887 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91058726-d953-423c-8958-65f169388831", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbfb0012a00 con 0x7fbfc0136e40 2024-09-16T15:00:34.667 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.665+0000 7fbfb77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "91058726-d953-423c-8958-65f169388831", "id": 3}] 2024-09-16T15:00:34.667 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.665+0000 7fbfb77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:34.668 INFO:teuthology.orchestra.run.smithi162.stdout:3 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 10 monclient: shutdown 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 msgr2=0x7fbfc0135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c460 tx=0x7fbfb000c920 comp rx=0 tx=0).stop 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 shutdown_connections 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc013c370 0x7fbfc011fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbfc0137220 0x7fbfc0136040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 --2- 172.21.15.162:0/509211887 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbfc0136e40 0x7fbfc0135b00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 >> 172.21.15.162:0/509211887 conn(0x7fbfc00a6110 msgr2=0x7fbfc013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 shutdown_connections 2024-09-16T15:00:34.669 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.666+0000 7fbfc5061640 1 -- 172.21.15.162:0/509211887 wait complete. 2024-09-16T15:00:34.678 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 8d7393b8-3f8f-41f7-ab05-4de36eef0dff 4 2024-09-16T15:00:34.707 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.705+0000 7f4033e38040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T15:00:34.763 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.760+0000 7f4033e38040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:00:34.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.773+0000 7f5b15301640 10 monclient: get_monmap_and_config 2024-09-16T15:00:34.775 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.773+0000 7f5b15301640 10 monclient: build_initial_monmap 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.773+0000 7f5b15301640 10 monclient: monmap: 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:34.775336+0000 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:34.775336+0000 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:34.776 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:34.777 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 Processor -- start 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 -- start start 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient: init 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient: _add_conns ranks=[4,6,5,7,1,3,8,2,10,9,0] 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b10136e40 0x7f5b1009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-c con 0x7f5b10136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b10137220 0x7f5b10137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-d con 0x7f5b10137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-h con 0x7f5b10135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009abc0 con 0x7f5b10136e40 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1009ba50 con 0x7f5b10135dc0 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100a53c0 con 0x7f5b10137220 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b15301640 10 monclient(hunting): authenticate will time out at 786.080817s 2024-09-16T15:00:34.780 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.162:57314/0 (socket says 172.21.15.162:57314) 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 1 -- 172.21.15.162:0/1970453833 learned_addr learned my addr 172.21.15.162:0/1970453833 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 10 monclient(hunting): get_auth_request con 0x7f5b10135dc0 auth_method 0 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.774+0000 7f5b0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient(hunting): handle_auth_done global_id 4127 payload 274 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 -- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b10137220 msgr2=0x7f5b10137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:34.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b07fff640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b10137220 0x7f5b10137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0effd640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b10136e40 0x7f5b1009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b10137220 0x7f5b10137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 -- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b10136e40 msgr2=0x7f5b1009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b10136e40 0x7f5b1009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 monclient: found mon.noname-h 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:00:34.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.775+0000 7f5b0f7fe640 1 -- 172.21.15.162:0/1970453833 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10073250 con 0x7f5b10135dc0 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0f7fe640 1 -- 172.21.15.162:0/1970453833 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 msgr2=0x7f5b10136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0f7fe640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009e80 tx=0x7f5b0802a530 comp rx=0 tx=0).ready entity=mon.5 client_cookie=9415a1f96025e210 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 1 -- 172.21.15.162:0/1970453833 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b08007a70 con 0x7f5b10135dc0 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: dump: 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.788 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:34.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 5 monclient: authenticate success, global_id 4127 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 1 -- 172.21.15.162:0/1970453833 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0802d050 con 0x7f5b10135dc0 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 4 set_mon_vals no callback set 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 1 -- 172.21.15.162:0/1970453833 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b08007bf0 con 0x7f5b10135dc0 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b0e7fc640 10 monclient: dump: 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:34.790 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 10 monclient: shutdown 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 -- 172.21.15.162:0/1970453833 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 msgr2=0x7f5b10136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:34.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009e80 tx=0x7f5b0802a530 comp rx=0 tx=0).stop 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 -- 172.21.15.162:0/1970453833 shutdown_connections 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10135dc0 0x7f5b10136220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b10137220 0x7f5b10137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.778+0000 7f5b15301640 1 --2- 172.21.15.162:0/1970453833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b10136e40 0x7f5b1009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 1 -- 172.21.15.162:0/1970453833 >> 172.21.15.162:0/1970453833 conn(0x7f5b100a6110 msgr2=0x7f5b100a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 1 -- 172.21.15.162:0/1970453833 shutdown_connections 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 1 -- 172.21.15.162:0/1970453833 wait complete. 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 10 monclient: build_initial_monmap 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 10 monclient: monmap: 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:34.792 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.779+0000 7f5b15301640 1 Processor -- start 2024-09-16T15:00:34.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 -- start start 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient: init 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient: _add_conns ranks=[7,5,8,0,2,4,6,1,3] 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-i con 0x7f5b10135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-h con 0x7f5b10136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 0x7f5b10135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): picked mon.noname-e con 0x7f5b10137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101cba70 con 0x7f5b10136e40 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100a8980 con 0x7f5b10135dc0 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5b101259f0 con 0x7f5b10137220 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b15301640 10 monclient(hunting): authenticate will time out at 786.086460s 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.162:57328/0 (socket says 172.21.15.162:57328) 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.162:46232/0 (socket says 172.21.15.162:46232) 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 1 -- 172.21.15.162:0/2287426441 learned_addr learned my addr 172.21.15.162:0/2287426441 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): get_auth_request con 0x7f5b10136e40 auth_method 0 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:34.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): get_auth_request con 0x7f5b10135dc0 auth_method 0 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 0x7f5b10135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 10 monclient(hunting): get_auth_request con 0x7f5b10137220 auth_method 0 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:34.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 -- 172.21.15.162:0/2287426441 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 msgr2=0x7f5b10135800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 0x7f5b10135800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 -- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 msgr2=0x7f5b101352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 monclient: found mon.noname-i 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0effd640 1 -- 172.21.15.162:0/2287426441 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b08009b30 con 0x7f5b10135dc0 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b07fff640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.780+0000 7f5b0f7fe640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 0x7f5b10135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b0effd640 1 -- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 msgr2=0x7f5b101cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:34.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b0effd640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c5e0 tx=0x7f5b0000caa0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b00011c80 con 0x7f5b10135dc0 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: dump: 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:34.798 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 5 monclient: authenticate success, global_id 4147 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b00011e20 con 0x7f5b10135dc0 2024-09-16T15:00:34.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b000107c0 con 0x7f5b10135dc0 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 10 monclient: _renew_subs 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b10125cd0 con 0x7f5b10135dc0 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b0dffb640 4 set_mon_vals no callback set 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 10 monclient: dump: 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:34.800 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 10 monclient: _renew_subs 2024-09-16T15:00:34.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b10124050 con 0x7f5b10135dc0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5b00010350 con 0x7f5b10135dc0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.783+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f5b00003610 con 0x7f5b10135dc0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.784+0000 7f5b15301640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.784+0000 7f5b15301640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.784+0000 7f5b15301640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.784+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1009fcc0 con 0x7f5b10135dc0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.786+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f5b00015070 con 0x7f5b10135dc0 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.786+0000 7f5b077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:34.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.786+0000 7f5b077fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:34.811 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.808+0000 7f4033e38040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:00:34.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.854+0000 7f5b15301640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8d7393b8-3f8f-41f7-ab05-4de36eef0dff", "id": 4}] 2024-09-16T15:00:34.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.854+0000 7f5b15301640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8d7393b8-3f8f-41f7-ab05-4de36eef0dff", "id": 4}] 2024-09-16T15:00:34.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.854+0000 7f5b15301640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:00:34.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:34.854+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "osd new", "uuid": "8d7393b8-3f8f-41f7-ab05-4de36eef0dff", "id": 4} v 0) v1 -- 0x7f5b1009ad60 con 0x7f5b10135dc0 2024-09-16T15:00:34.922 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.920+0000 7f4033e38040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:00:34.971 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:34.968+0000 7f4033e38040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:00:35.021 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:35.019+0000 7f4033e38040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:00:35.088 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:35.086+0000 7f4033e38040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:00:35.132 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:35.130+0000 7f4033e38040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:00:35.176 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:35.173+0000 7f4033e38040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:00:35.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.632+0000 7f5b077fe640 1 -- 172.21.15.162:0/2287426441 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d7393b8-3f8f-41f7-ab05-4de36eef0dff", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5b00004020 con 0x7f5b10135dc0 2024-09-16T15:00:35.634 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.632+0000 7f5b077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8d7393b8-3f8f-41f7-ab05-4de36eef0dff", "id": 4}] 2024-09-16T15:00:35.635 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.632+0000 7f5b077fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:35.635 INFO:teuthology.orchestra.run.smithi162.stdout:4 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 10 monclient: shutdown 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 msgr2=0x7f5b101cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c5e0 tx=0x7f5b0000caa0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 shutdown_connections 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b10137220 0x7f5b10135800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b10136e40 0x7f5b101352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 --2- 172.21.15.162:0/2287426441 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b10135dc0 0x7f5b101cb530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 >> 172.21.15.162:0/2287426441 conn(0x7f5b100a6110 msgr2=0x7f5b1009f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:35.636 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 shutdown_connections 2024-09-16T15:00:35.637 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.633+0000 7f5b15301640 1 -- 172.21.15.162:0/2287426441 wait complete. 2024-09-16T15:00:35.645 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 7c88ec95-db14-4be8-87a5-3151b85a4058 5 2024-09-16T15:00:35.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.779+0000 7ff1f37f4640 10 monclient: get_monmap_and_config 2024-09-16T15:00:35.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.779+0000 7ff1f37f4640 10 monclient: build_initial_monmap 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.779+0000 7ff1f37f4640 10 monclient: monmap: 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:35.781724+0000 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:35.781724+0000 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:35.783 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.780+0000 7ff1f37f4640 1 Processor -- start 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.780+0000 7ff1f37f4640 1 -- start start 2024-09-16T15:00:35.784 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.780+0000 7ff1f37f4640 10 monclient: init 2024-09-16T15:00:35.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:35.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient: _add_conns ranks=[10,1,0,5,2,3,7,6,8,9,4] 2024-09-16T15:00:35.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1ec132060 0x7ff1ec132440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:35.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-f con 0x7ff1ec132060 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:35.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff1ec071cb0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff1ec1354c0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec073250 con 0x7ff1ec1354c0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec070490 con 0x7ff1ec071cb0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec070770 con 0x7ff1ec132060 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f37f4640 10 monclient(hunting): authenticate will time out at 787.087255s 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1ec132060 0x7ff1ec132440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1ec132060 0x7ff1ec132440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:59046/0 (socket says 172.21.15.162:59046) 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 1 -- 172.21.15.162:0/844742054 learned_addr learned my addr 172.21.15.162:0/844742054 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): get_auth_request con 0x7ff1ec132060 auth_method 0 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:35.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.781+0000 7ff1f1590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/844742054 >> v1:172.21.15.162:6789/0 conn(0x7ff1ec071cb0 legacy=0x7ff1ec134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 10 monclient(hunting): handle_auth_done global_id 4150 payload 274 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 1 -- 172.21.15.162:0/844742054 >> v1:172.21.15.162:6789/0 conn(0x7ff1ec071cb0 legacy=0x7ff1ec134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 1 -- 172.21.15.162:0/844742054 >> v1:172.21.15.134:6789/0 conn(0x7ff1ec1354c0 legacy=0x7ff1ec13d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 10 monclient: _finish_hunting 0 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 1 monclient: found mon.noname-f 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:35.788 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.783+0000 7ff1f1590640 1 -- 172.21.15.162:0/844742054 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec070a50 con 0x7ff1ec132060 2024-09-16T15:00:35.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f1590640 1 -- 172.21.15.162:0/844742054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ec132060 msgr2=0x7ff1ec132440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:35.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f1590640 1 --2- 172.21.15.162:0/844742054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ec132060 0x7ff1ec132440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00098f0 tx=0x7ff1e002a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88df2bd8c3f4ce05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:35.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 1 -- 172.21.15.162:0/844742054 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1e002f920 con 0x7ff1ec132060 2024-09-16T15:00:35.790 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: dump: 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:35.791 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 1 -- 172.21.15.162:0/844742054 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e002faa0 con 0x7ff1ec132060 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f37f4640 5 monclient: authenticate success, global_id 4150 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f37f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f37f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:35.792 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 1 -- 172.21.15.162:0/844742054 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1e002fc10 con 0x7ff1ec132060 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1f37f4640 4 set_mon_vals no callback set 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.786+0000 7ff1dffff640 10 monclient: dump: 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:35.793 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 10 monclient: shutdown 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 -- 172.21.15.162:0/844742054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ec132060 msgr2=0x7ff1ec132440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/844742054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ec132060 0x7ff1ec132440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00098f0 tx=0x7ff1e002a320 comp rx=0 tx=0).stop 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 -- 172.21.15.162:0/844742054 shutdown_connections 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/844742054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ec132060 0x7ff1ec132440 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.794 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 -- 172.21.15.162:0/844742054 >> 172.21.15.162:0/844742054 conn(0x7ff1ec0a7480 msgr2=0x7ff1ec0a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 -- 172.21.15.162:0/844742054 shutdown_connections 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.787+0000 7ff1f37f4640 1 -- 172.21.15.162:0/844742054 wait complete. 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient: build_initial_monmap 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient: monmap: 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 0 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 0.000000 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:created 0.000000 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:35.795 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 Processor -- start 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 -- start start 2024-09-16T15:00:35.796 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient: init 2024-09-16T15:00:35.799 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient: _add_conns ranks=[0,3,4,2,5,6,7,8,1] 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 0x7ff1ec1cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-a con 0x7ff1ec071cb0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-g con 0x7ff1ec132060 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 0x7ff1ec1cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): picked mon.noname-c con 0x7ff1ec1354c0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec122360 con 0x7ff1ec071cb0 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:35.800 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec131b90 con 0x7ff1ec132060 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0a4b20 con 0x7ff1ec1354c0 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f37f4640 10 monclient(hunting): authenticate will time out at 787.094901s 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.788+0000 7ff1f0d8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:55940/0 (socket says 172.21.15.162:55940) 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/2279543090 learned_addr learned my addr 172.21.15.162:0/2279543090 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): get_auth_request con 0x7ff1ec132060 auth_method 0 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:35.801 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1d91640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 0x7ff1ec1cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1590640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 0x7ff1ec1cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient(hunting): handle_auth_done global_id 4125 payload 824 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 msgr2=0x7ff1ec1cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 0x7ff1ec1cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 msgr2=0x7ff1ec1cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 0x7ff1ec1cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient: _finish_hunting 0 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 monclient: found mon.noname-g 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/2279543090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00094d0 con 0x7ff1ec132060 2024-09-16T15:00:35.802 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1d91640 10 monclient: get_auth_request con 0x7ff1ec1354c0 auth_method 0 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1590640 10 monclient: get_auth_request con 0x7ff1ec071cb0 auth_method 0 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1d91640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 0x7ff1ec1cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.789+0000 7ff1f1590640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 0x7ff1ec1cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f0d8f640 1 -- 172.21.15.162:0/2279543090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 msgr2=0x7ff1ec1cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f0d8f640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c390 tx=0x7ff1e800c850 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1e80074f0 con 0x7ff1ec132060 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: dump: 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.803 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:35.804 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e8005c30 con 0x7ff1ec132060 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 5 monclient: authenticate success, global_id 4125 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 10 monclient: _renew_subs 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0a4660 con 0x7ff1ec132060 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1e8005da0 con 0x7ff1ec132060 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 10 monclient: dump: 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:epoch 1 2024-09-16T15:00:35.805 INFO:teuthology.orchestra.run.smithi162.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:election_strategy: 1 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:35.806 INFO:teuthology.orchestra.run.smithi162.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1df7fe640 4 set_mon_vals no callback set 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 10 monclient: _renew_subs 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff1e8012400 con 0x7ff1ec132060 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.791+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec123000 con 0x7ff1ec132060 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.792+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff1e8012c30 con 0x7ff1ec132060 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.792+0000 7ff1f37f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.792+0000 7ff1f37f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:35.807 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.792+0000 7ff1f37f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:35.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.792+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec1326f0 con 0x7ff1ec132060 2024-09-16T15:00:35.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.794+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff1e8020060 con 0x7ff1ec132060 2024-09-16T15:00:35.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.794+0000 7ff1ddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:35.808 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.794+0000 7ff1ddffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:35.825 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:35.822+0000 7f4033e38040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:00:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.863+0000 7ff1f37f4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "7c88ec95-db14-4be8-87a5-3151b85a4058", "id": 5}] 2024-09-16T15:00:35.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.863+0000 7ff1f37f4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "7c88ec95-db14-4be8-87a5-3151b85a4058", "id": 5}] 2024-09-16T15:00:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.863+0000 7ff1f37f4640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:00:35.866 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:35.863+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c88ec95-db14-4be8-87a5-3151b85a4058", "id": 5} v 0) v1 -- 0x7ff1ec071520 con 0x7ff1ec132060 2024-09-16T15:00:36.154 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.152+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mgrmap(e 2) v1 ==== 44528+0+0 (secure 0 0 0) 0x7ff1e80126b0 con 0x7ff1ec132060 2024-09-16T15:00:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.161+0000 7ff1ddffb640 1 -- 172.21.15.162:0/2279543090 <== mon.3 v2:172.21.15.162:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c88ec95-db14-4be8-87a5-3151b85a4058", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff1e8017020 con 0x7ff1ec132060 2024-09-16T15:00:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.161+0000 7ff1ddffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "7c88ec95-db14-4be8-87a5-3151b85a4058", "id": 5}] 2024-09-16T15:00:36.164 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.161+0000 7ff1ddffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:36.164 INFO:teuthology.orchestra.run.smithi162.stdout:5 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 10 monclient: shutdown 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 msgr2=0x7ff1ec1cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c390 tx=0x7ff1e800c850 comp rx=0 tx=0).stop 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 shutdown_connections 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff1ec1354c0 0x7ff1ec1cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ec132060 0x7ff1ec1cbee0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 --2- 172.21.15.162:0/2279543090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1ec071cb0 0x7ff1ec1cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 >> 172.21.15.162:0/2279543090 conn(0x7ff1ec0a7480 msgr2=0x7ff1ec1bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:36.166 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 shutdown_connections 2024-09-16T15:00:36.167 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T15:00:36.163+0000 7ff1f37f4640 1 -- 172.21.15.162:0/2279543090 wait complete. 2024-09-16T15:00:36.177 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T15:00:36.178 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T15:00:36.180 INFO:tasks.ceph.osd.0:Started 2024-09-16T15:00:36.180 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T15:00:36.180 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T15:00:36.182 INFO:tasks.ceph.osd.1:Started 2024-09-16T15:00:36.182 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T15:00:36.182 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T15:00:36.184 INFO:tasks.ceph.osd.2:Started 2024-09-16T15:00:36.184 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T15:00:36.184 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T15:00:36.186 INFO:tasks.ceph.osd.3:Started 2024-09-16T15:00:36.186 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T15:00:36.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T15:00:36.188 INFO:tasks.ceph.osd.4:Started 2024-09-16T15:00:36.188 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T15:00:36.189 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T15:00:36.191 INFO:tasks.ceph.osd.5:Started 2024-09-16T15:00:36.191 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.298+0000 7f395a745640 10 monclient: get_monmap_and_config 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.298+0000 7f395a745640 10 monclient: build_initial_monmap 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.298+0000 7f395a745640 10 monclient: monmap: 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:36.300689+0000 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:36.300689+0000 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:36.301 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:36.302 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.299+0000 7f395a745640 1 Processor -- start 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.299+0000 7f395a745640 1 -- start start 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.299+0000 7f395a745640 10 monclient: init 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient: _add_conns ranks=[8,2,10,3,4,0,7,9,6,5,1] 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 0x7f3954136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): picked mon.noname-e con 0x7f3954136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): picked mon.noname-b con 0x7f395409bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3954136c60 0x7f39541370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): picked mon.noname-f con 0x7f3954136c60 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:36.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395409b600 con 0x7f3954136c60 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540a5740 con 0x7f395409bfe0 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f39540a52c0 con 0x7f3954136030 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f395a745640 10 monclient(hunting): authenticate will time out at 772.890108s 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 0x7f3954136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:39118/0 (socket says 172.21.15.134:39118) 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 -- 172.21.15.134:0/381127029 learned_addr learned my addr 172.21.15.134:0/381127029 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): get_auth_request con 0x7f395409bfe0 auth_method 0 2024-09-16T15:00:36.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): get_auth_request con 0x7f3954136030 auth_method 0 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 1 --2- 172.21.15.134:0/381127029 >> v2:172.21.15.162:3300/0 conn(0x7f3954136c60 0x7f39541370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): get_auth_request con 0x7f3954136c60 auth_method 0 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient(hunting): handle_auth_done global_id 4160 payload 274 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 -- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 msgr2=0x7f3954136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 0x7f3954136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 -- 172.21.15.134:0/381127029 >> v2:172.21.15.162:3300/0 conn(0x7f3954136c60 msgr2=0x7f39541370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 --2- 172.21.15.134:0/381127029 >> v2:172.21.15.162:3300/0 conn(0x7f3954136c60 0x7f39541370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:36.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:36.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:36.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:36.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f39537fe640 1 -- 172.21.15.134:0/381127029 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954073250 con 0x7f395409bfe0 2024-09-16T15:00:36.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.300+0000 7f3953fff640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 0x7f3954136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f39537fe640 1 -- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 msgr2=0x7f395409c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f39537fe640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3948009870 tx=0x7f394802a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b385ebd36c7bf72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 1 -- 172.21.15.134:0/381127029 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f394802ae00 con 0x7f395409bfe0 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:36.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: dump: 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:36.316 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 1 -- 172.21.15.134:0/381127029 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39480307b0 con 0x7f395409bfe0 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 1 -- 172.21.15.134:0/381127029 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3948030dd0 con 0x7f395409bfe0 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f3952ffd640 10 monclient: dump: 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:36.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f395a745640 5 monclient: authenticate success, global_id 4160 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f395a745640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.303+0000 7f395a745640 4 set_mon_vals no callback set 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 10 monclient: shutdown 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 -- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 msgr2=0x7f395409c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3948009870 tx=0x7f394802a0a0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 -- 172.21.15.134:0/381127029 shutdown_connections 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 --2- 172.21.15.134:0/381127029 >> v2:172.21.15.162:3300/0 conn(0x7f3954136c60 0x7f39541370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f395409bfe0 0x7f395409c400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 --2- 172.21.15.134:0/381127029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3954136030 0x7f3954136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 -- 172.21.15.134:0/381127029 >> 172.21.15.134:0/381127029 conn(0x7f39540a9ea0 msgr2=0x7f39540a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 -- 172.21.15.134:0/381127029 shutdown_connections 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 -- 172.21.15.134:0/381127029 wait complete. 2024-09-16T15:00:36.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 10 monclient: build_initial_monmap 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 10 monclient: monmap: 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:36.320 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.304+0000 7f395a745640 1 Processor -- start 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 -- start start 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient: init 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient: _add_conns ranks=[0,7,2,5,6,4,8,1,3] 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395409bfe0 0x7f39541bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): picked mon.noname-a con 0x7f395409bfe0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3954136030 0x7f395411fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): picked mon.noname-i con 0x7f3954136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:36.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): picked mon.noname-b con 0x7f3954136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39541220d0 con 0x7f395409bfe0 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3954121df0 con 0x7f3954136c60 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f39541223b0 con 0x7f3954136030 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f395a745640 10 monclient(hunting): authenticate will time out at 772.895667s 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:39124/0 (socket says 172.21.15.134:39124) 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 1 -- 172.21.15.134:0/3325448366 learned_addr learned my addr 172.21.15.134:0/3325448366 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): get_auth_request con 0x7f3954136c60 auth_method 0 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395409bfe0 0x7f39541bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): get_auth_request con 0x7f395409bfe0 auth_method 0 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3954136030 0x7f395411fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): get_auth_request con 0x7f3954136030 auth_method 0 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:36.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3953fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f39537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.305+0000 7f3958ce2640 10 monclient(hunting): handle_auth_done global_id 4169 payload 824 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 -- 172.21.15.134:0/3325448366 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3954136030 msgr2=0x7f395411fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3954136030 0x7f395411fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 -- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395409bfe0 msgr2=0x7f39541bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395409bfe0 0x7f39541bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 10 monclient: _finish_hunting 0 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 monclient: found mon.noname-b 2024-09-16T15:00:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.306+0000 7f3958ce2640 1 -- 172.21.15.134:0/3325448366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009400 con 0x7f3954136c60 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3958ce2640 1 -- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 msgr2=0x7f3954134cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3958ce2640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f394c00efc0 tx=0x7f394c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f394c009ba0 con 0x7f3954136c60 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: dump: 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00f040 con 0x7f3954136c60 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f394c014d60 con 0x7f3954136c60 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f3950ff9640 10 monclient: dump: 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 5 monclient: authenticate success, global_id 4169 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f39527fc640 4 set_mon_vals no callback set 2024-09-16T15:00:36.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 10 monclient: _renew_subs 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480094d0 con 0x7f3954136c60 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 10 monclient: _renew_subs 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.308+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3954120050 con 0x7f3954136c60 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f3958ce2640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f394c00efc0 tx=0x7f394c00c490 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f394c014480 mgrmap(e 2) v1 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 2) v1 ==== 44528+0+0 (secure 0 0 0) 0x7f394c014480 con 0x7f3954136c60 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f395a745640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f395a745640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f395a745640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.309+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39541201e0 con 0x7f3954136c60 2024-09-16T15:00:36.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:36.311+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f394c049c10 con 0x7f3954136c60 2024-09-16T15:00:36.782 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-16T15:00:36.779+0000 7fc24a20b640 -1 Falling back to public interface 2024-09-16T15:00:36.785 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T15:00:36.783+0000 7f76227cc640 -1 Falling back to public interface 2024-09-16T15:00:36.786 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-16T15:00:36.783+0000 7fa86a4ef640 -1 Falling back to public interface 2024-09-16T15:00:36.786 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-16T15:00:36.784+0000 7f7150c1a640 -1 Falling back to public interface 2024-09-16T15:00:36.788 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-16T15:00:36.786+0000 7f62b2334640 -1 Falling back to public interface 2024-09-16T15:00:36.790 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-16T15:00:36.787+0000 7f7e65c10640 -1 Falling back to public interface 2024-09-16T15:00:37.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.138+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f394c018070 con 0x7f3954136c60 2024-09-16T15:00:37.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.138+0000 7f3950ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:37.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.138+0000 7f3950ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:37.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.194+0000 7f395a745640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:37.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.194+0000 7f395a745640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:37.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.194+0000 7f395a745640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:37.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.194+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f395409c400 con 0x7f3954136c60 2024-09-16T15:00:37.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.429+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mgrmap(e 3) v1 ==== 44620+0+0 (secure 0 0 0) 0x7f394c0133e0 con 0x7f3954136c60 2024-09-16T15:00:37.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.429+0000 7f3950ff9640 1 --2- 172.21.15.134:0/3325448366 >> v2:172.21.15.162:6800/980547263 conn(0x7f3940039ad0 0x7f394003bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:37.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.431+0000 7f3953fff640 1 --2- 172.21.15.134:0/3325448366 >> v2:172.21.15.162:6800/980547263 conn(0x7f3940039ad0 0x7f394003bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:37.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.432+0000 7f3953fff640 10 monclient: get_auth_request con 0x7f3940039ad0 auth_method 0 2024-09-16T15:00:37.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.433+0000 7f3950ff9640 1 -- 172.21.15.134:0/3325448366 <== mon.2 v2:172.21.15.134:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5604 (secure 0 0 0) 0x7f394c01a680 con 0x7f3954136c60 2024-09-16T15:00:37.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.433+0000 7f3950ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:37.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.433+0000 7f3950ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:37.435 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:37.436 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":8,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:36.156025+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 >> v2:172.21.15.162:6800/980547263 conn(0x7f3940039ad0 msgr2=0x7f394003bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> v2:172.21.15.162:6800/980547263 conn(0x7f3940039ad0 0x7f394003bec0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f3944009cd0 tx=0x7f3944009190 comp rx=0 tx=0).stop 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 10 monclient: shutdown 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 msgr2=0x7f3954134cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f394c00efc0 tx=0x7f394c00c490 comp rx=0 tx=0).stop 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 shutdown_connections 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> v2:172.21.15.162:6800/980547263 conn(0x7f3940039ad0 0x7f394003bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:37.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3954136c60 0x7f3954134cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:37.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3954136030 0x7f395411fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:37.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 --2- 172.21.15.134:0/3325448366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395409bfe0 0x7f39541bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:37.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 >> 172.21.15.134:0/3325448366 conn(0x7f39540a9ea0 msgr2=0x7f395406bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:37.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 shutdown_connections 2024-09-16T15:00:37.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:37.434+0000 7f395a745640 1 -- 172.21.15.134:0/3325448366 wait complete. 2024-09-16T15:00:37.438 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.435+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-09-16T15:00:37.441 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.439+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T15:00:37.442 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.440+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-09-16T15:00:37.442 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.440+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T15:00:37.442 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.440+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-09-16T15:00:37.444 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.442+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-09-16T15:00:37.447 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T15:00:37.447 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T15:00:37.449 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.446+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-09-16T15:00:37.449 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-09-16T15:00:37.447+0000 7f3fd3f93640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-09-16T15:00:38.053 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:38.145 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.142+0000 7fd3338b1640 10 monclient: get_monmap_and_config 2024-09-16T15:00:38.145 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.142+0000 7fd3338b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:38.145 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.142+0000 7fd3338b1640 10 monclient: monmap: 2024-09-16T15:00:38.145 INFO:teuthology.misc.health.smithi134.stderr:epoch 0 2024-09-16T15:00:38.145 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:38.144699+0000 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:38.144699+0000 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:38.146 INFO:teuthology.misc.health.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:38.147 INFO:teuthology.misc.health.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:38.147 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 Processor -- start 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 -- start start 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient: init 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient: _add_conns ranks=[1,3,8,5,9,0,7,10,4,6,2] 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd32c0a3940 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c06a560 0x7fd32c13c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-g con 0x7fd32c06a560 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-e con 0x7fd32c06a940 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:38.151 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c073250 con 0x7fd32c0a3940 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0a50f0 con 0x7fd32c06a560 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c09ab30 con 0x7fd32c06a940 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.143+0000 7fd3338b1640 10 monclient(hunting): authenticate will time out at 774.733985s 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33858/0 (socket says 172.21.15.134:33858) 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 -- 172.21.15.134:0/764090125 learned_addr learned my addr 172.21.15.134:0/764090125 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): get_auth_request con 0x7fd32c06a940 auth_method 0 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:38.152 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd330e4c640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c06a560 0x7fd32c13c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd330e4c640 10 monclient(hunting): get_auth_request con 0x7fd32c06a560 auth_method 0 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd330e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd330e4c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd330e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient(hunting): handle_auth_done global_id 4202 payload 274 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 -- 172.21.15.134:0/764090125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c06a560 msgr2=0x7fd32c13c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c06a560 0x7fd32c13c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 -- 172.21.15.134:0/764090125 >> v1:172.21.15.162:6789/0 conn(0x7fd32c0a3940 legacy=0x7fd32c069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:38.153 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient: _finish_hunting 0 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 monclient: found mon.noname-e 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.144+0000 7fd331e4e640 1 -- 172.21.15.134:0/764090125 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0a3d20 con 0x7fd32c06a940 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd331e4e640 1 -- 172.21.15.134:0/764090125 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 msgr2=0x7fd32c13ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd331e4e640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd324009e80 tx=0x7fd32402a580 comp rx=0 tx=0).ready entity=mon.8 client_cookie=80ae3460572a9349 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 1 -- 172.21.15.134:0/764090125 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd324007a70 con 0x7fd32c06a940 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: dump: 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:38.154 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.155 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 1 -- 172.21.15.134:0/764090125 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32402d050 con 0x7fd32c06a940 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 5 monclient: authenticate success, global_id 4202 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 1 -- 172.21.15.134:0/764090125 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd324007bf0 con 0x7fd32c06a940 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 4 set_mon_vals no callback set 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd323fff640 10 monclient: dump: 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:38.156 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:38.157 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 10 monclient: shutdown 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 1 -- 172.21.15.134:0/764090125 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 msgr2=0x7fd32c13ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.147+0000 7fd3338b1640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd324009e80 tx=0x7fd32402a580 comp rx=0 tx=0).stop 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 -- 172.21.15.134:0/764090125 shutdown_connections 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd32c06a940 0x7fd32c13ea20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 --2- 172.21.15.134:0/764090125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c06a560 0x7fd32c13c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 -- 172.21.15.134:0/764090125 >> 172.21.15.134:0/764090125 conn(0x7fd32c0a2e20 msgr2=0x7fd32c0a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 -- 172.21.15.134:0/764090125 shutdown_connections 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 -- 172.21.15.134:0/764090125 wait complete. 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 10 monclient: monmap: 2024-09-16T15:00:38.158 INFO:teuthology.misc.health.smithi134.stderr:epoch 0 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:created 0.000000 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:38.159 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.160 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.148+0000 7fd3338b1640 1 Processor -- start 2024-09-16T15:00:38.160 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 -- start start 2024-09-16T15:00:38.160 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient: init 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient: _add_conns ranks=[7,0,3,5,6,2,1,8,4] 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd32c06a560 0x7fd32c1cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-i con 0x7fd32c06a560 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-a con 0x7fd32c06a940 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c0a3940 0x7fd32c1cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): picked mon.noname-g con 0x7fd32c0a3940 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:38.165 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0a36c0 con 0x7fd32c06a940 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c092d60 con 0x7fd32c0a3940 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c093040 con 0x7fd32c06a560 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd3338b1640 10 monclient(hunting): authenticate will time out at 774.739510s 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58470/0 (socket says 172.21.15.134:58470) 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 -- 172.21.15.134:0/3335120017 learned_addr learned my addr 172.21.15.134:0/3335120017 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): get_auth_request con 0x7fd32c06a940 auth_method 0 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:38.166 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c0a3940 0x7fd32c1cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): get_auth_request con 0x7fd32c0a3940 auth_method 0 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd33164d640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd32c06a560 0x7fd32c1cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd33164d640 10 monclient(hunting): get_auth_request con 0x7fd32c06a560 auth_method 0 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd33164d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd33164d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd33164d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:38.167 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd331e4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient(hunting): handle_auth_done global_id 4203 payload 824 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 -- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd32c06a560 msgr2=0x7fd32c1cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd32c06a560 0x7fd32c1cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 -- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c0a3940 msgr2=0x7fd32c1cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c0a3940 0x7fd32c1cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 monclient: found mon.noname-a 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:38.168 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.149+0000 7fd330e4c640 1 -- 172.21.15.134:0/3335120017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd324009b30 con 0x7fd32c06a940 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd330e4c640 1 -- 172.21.15.134:0/3335120017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 msgr2=0x7fd32c1cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd330e4c640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00d7d0 tx=0x7fd31c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd31c018070 con 0x7fd32c06a940 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: dump: 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:38.169 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 5 monclient: authenticate success, global_id 4203 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c00bd40 con 0x7fd32c06a940 2024-09-16T15:00:38.170 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 10 monclient: _renew_subs 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c0a3200 con 0x7fd32c06a940 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd31c0148a0 con 0x7fd32c06a940 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd321ffb640 10 monclient: dump: 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:38.171 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3237fe640 4 set_mon_vals no callback set 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 10 monclient: _renew_subs 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:38.172 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.152+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c071b30 con 0x7fd32c06a940 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 3) v1 ==== 44620+0+0 (secure 0 0 0) 0x7fd31c014430 con 0x7fd32c06a940 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd3338b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd3338b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd3338b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c0679e0 con 0x7fd32c06a940 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd321ffb640 1 --2- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 0x7fd31003ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd31c049270 con 0x7fd32c06a940 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.153+0000 7fd33164d640 1 --2- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 0x7fd31003ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.154+0000 7fd33164d640 10 monclient: get_auth_request con 0x7fd3100386d0 auth_method 0 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.156+0000 7fd33164d640 1 --2- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 0x7fd31003ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd328006fd0 tx=0x7fd328008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.158+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd31c0146e0 con 0x7fd32c06a940 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.158+0000 7fd321ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:38.173 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.158+0000 7fd321ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:38.323 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.320+0000 7fd3338b1640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:38.323 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.320+0000 7fd3338b1640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:38.323 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.320+0000 7fd3338b1640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:38.323 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.320+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd32c09b0d0 con 0x7fd32c06a940 2024-09-16T15:00:38.324 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.321+0000 7fd321ffb640 1 -- 172.21.15.134:0/3335120017 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5604 (secure 0 0 0) 0x7fd31c011340 con 0x7fd32c06a940 2024-09-16T15:00:38.324 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.321+0000 7fd321ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:38.324 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.321+0000 7fd321ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:38.324 INFO:teuthology.misc.health.smithi134.stdout: 2024-09-16T15:00:38.324 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":8,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:36.156025+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 msgr2=0x7fd31003ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 0x7fd31003ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd328006fd0 tx=0x7fd328008040 comp rx=0 tx=0).stop 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 10 monclient: shutdown 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 msgr2=0x7fd32c1cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00d7d0 tx=0x7fd31c00dc90 comp rx=0 tx=0).stop 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 shutdown_connections 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> v2:172.21.15.162:6800/980547263 conn(0x7fd3100386d0 0x7fd31003ab90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd32c0a3940 0x7fd32c1cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd32c06a940 0x7fd32c1cbba0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.326 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 --2- 172.21.15.134:0/3335120017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd32c06a560 0x7fd32c1cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:38.327 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 >> 172.21.15.134:0/3335120017 conn(0x7fd32c0a2e20 msgr2=0x7fd32c09f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:38.327 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 shutdown_connections 2024-09-16T15:00:38.327 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:38.323+0000 7fd3338b1640 1 -- 172.21.15.134:0/3335120017 wait complete. 2024-09-16T15:00:38.335 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-16T15:00:40.209 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-16T15:00:40.206+0000 7fc24a20b640 -1 osd.4 0 log_to_monitors true 2024-09-16T15:00:40.255 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-16T15:00:40.253+0000 7fa86a4ef640 -1 osd.2 0 log_to_monitors true 2024-09-16T15:00:40.262 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T15:00:40.260+0000 7f76227cc640 -1 osd.3 0 log_to_monitors true 2024-09-16T15:00:40.281 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-16T15:00:40.279+0000 7f62b2334640 -1 osd.5 0 log_to_monitors true 2024-09-16T15:00:40.302 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-16T15:00:40.301+0000 7f7e65c10640 -1 osd.1 0 log_to_monitors true 2024-09-16T15:00:40.311 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-16T15:00:40.310+0000 7f7150c1a640 -1 osd.0 0 log_to_monitors true 2024-09-16T15:00:41.615 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-16T15:00:41.613+0000 7f714c5e5640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T15:00:41.622 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-16T15:00:41.621+0000 7f714840e640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:41.623 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-16T15:00:41.620+0000 7fc245c2f640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T15:00:41.626 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-09-16T15:00:41.624+0000 7fc241a58640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:44.415 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-16T15:00:44.414+0000 7f7e614fc640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T15:00:44.422 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-16T15:00:44.421+0000 7f7e5931d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:44.440 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-16T15:00:44.439+0000 7fa86647c640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T15:00:44.444 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-16T15:00:44.443+0000 7fa85e29d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:44.456 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T15:00:44.455+0000 7f761e759640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T15:00:44.456 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-16T15:00:44.455+0000 7f62ae2c1640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T15:00:44.459 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T15:00:44.458+0000 7f761657a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:44.462 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-09-16T15:00:44.461+0000 7f62a60e2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:00:44.941 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.034+0000 7fb77fa20640 10 monclient: get_monmap_and_config 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.034+0000 7fb77fa20640 10 monclient: build_initial_monmap 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.034+0000 7fb77fa20640 10 monclient: monmap: 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:epoch 0 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:45.035607+0000 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:45.035607+0000 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.036 INFO:teuthology.misc.health.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:45.037 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.041 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 Processor -- start 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 -- start start 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient: init 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient: _add_conns ranks=[0,5,8,7,3,2,4,10,6,9,1] 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb778136030 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77809bfe0 0x7fb77809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-h con 0x7fb77809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 0x7fb77813e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-e con 0x7fb77813bf80 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb778073250 con 0x7fb778136030 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.042 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb77809b600 con 0x7fb77809bfe0 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780a5740 con 0x7fb77813bf80 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.035+0000 7fb77fa20640 10 monclient(hunting): authenticate will time out at 781.624921s 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77d7bc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb778136030 0x7fb778136410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50684/0 (socket says 172.21.15.134:50684) 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77d7bc640 1 -- 172.21.15.134:0/403923277 learned_addr learned my addr 172.21.15.134:0/403923277 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 0x7fb77813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): get_auth_request con 0x7fb77813bf80 auth_method 0 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.043 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77809bfe0 0x7fb77809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): get_auth_request con 0x7fb77809bfe0 auth_method 0 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_done global_id 4238 payload 274 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 -- 172.21.15.134:0/403923277 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77809bfe0 msgr2=0x7fb77809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77809bfe0 0x7fb77809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 -- 172.21.15.134:0/403923277 >> v1:172.21.15.134:6789/0 conn(0x7fb778136030 legacy=0x7fb778136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:45.044 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 monclient: found mon.noname-e 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.036+0000 7fb77dfbd640 1 -- 172.21.15.134:0/403923277 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780a52c0 con 0x7fb77813bf80 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77dfbd640 1 -- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 msgr2=0x7fb77813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77dfbd640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 0x7fb77813e370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb770009dc0 tx=0x7fb77002a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f00199efb23fd60d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 1 -- 172.21.15.134:0/403923277 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb770007bc0 con 0x7fb77813bf80 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: dump: 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.045 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.046 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 1 -- 172.21.15.134:0/403923277 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002d050 con 0x7fb77813bf80 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 1 -- 172.21.15.134:0/403923277 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb770007d80 con 0x7fb77813bf80 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 5 monclient: authenticate success, global_id 4238 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 4 set_mon_vals no callback set 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb76ffff640 10 monclient: dump: 2024-09-16T15:00:45.047 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.048 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 10 monclient: shutdown 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 -- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 msgr2=0x7fb77813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 0x7fb77813e370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb770009dc0 tx=0x7fb77002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 -- 172.21.15.134:0/403923277 shutdown_connections 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb77813bf80 0x7fb77813e370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 --2- 172.21.15.134:0/403923277 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77809bfe0 0x7fb77809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 -- 172.21.15.134:0/403923277 >> 172.21.15.134:0/403923277 conn(0x7fb7780a9ea0 msgr2=0x7fb7780a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 -- 172.21.15.134:0/403923277 shutdown_connections 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.039+0000 7fb77fa20640 1 -- 172.21.15.134:0/403923277 wait complete. 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.040+0000 7fb77fa20640 10 monclient: build_initial_monmap 2024-09-16T15:00:45.049 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.040+0000 7fb77fa20640 10 monclient: monmap: 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:epoch 0 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:created 0.000000 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:45.050 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:45.051 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:45.051 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.051 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.040+0000 7fb77fa20640 1 Processor -- start 2024-09-16T15:00:45.051 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.040+0000 7fb77fa20640 1 -- start start 2024-09-16T15:00:45.051 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.040+0000 7fb77fa20640 10 monclient: init 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient: _add_conns ranks=[4,0,5,1,6,3,8,7,2] 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 0x7fb7781cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-c con 0x7fb77809bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-a con 0x7fb778136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:45.055 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77813bf80 0x7fb778138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): picked mon.noname-h con 0x7fb77813bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7781203f0 con 0x7fb778136030 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb77811fc10 con 0x7fb77809bfe0 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780a3020 con 0x7fb77813bf80 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77fa20640 10 monclient(hunting): authenticate will time out at 781.630199s 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 0x7fb7781cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58524/0 (socket says 172.21.15.134:58524) 2024-09-16T15:00:45.056 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 -- 172.21.15.134:0/2050887832 learned_addr learned my addr 172.21.15.134:0/2050887832 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): get_auth_request con 0x7fb778136030 auth_method 0 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): get_auth_request con 0x7fb77809bfe0 auth_method 0 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.057 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77813bf80 0x7fb778138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): get_auth_request con 0x7fb77813bf80 auth_method 0 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77dfbd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient(hunting): handle_auth_done global_id 4230 payload 824 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 -- 172.21.15.134:0/2050887832 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77813bf80 msgr2=0x7fb778138b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77813bf80 0x7fb778138b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.058 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 -- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 msgr2=0x7fb7781cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 0x7fb7781cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient: _finish_hunting 0 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 monclient: found mon.noname-a 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77cfbb640 1 -- 172.21.15.134:0/2050887832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770009a70 con 0x7fb778136030 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.041+0000 7fb77d7bc640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 0x7fb7781cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.043+0000 7fb77cfbb640 1 -- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 msgr2=0x7fb7781cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.043+0000 7fb77cfbb640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e980 tx=0x7fb76800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb768009670 con 0x7fb778136030 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.059 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: dump: 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.060 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800cc10 con 0x7fb778136030 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb76800cd80 con 0x7fb778136030 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76dffb640 10 monclient: dump: 2024-09-16T15:00:45.061 INFO:teuthology.misc.health.smithi134.stderr:epoch 1 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.062 INFO:teuthology.misc.health.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr: 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 5 monclient: authenticate success, global_id 4230 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb76f7fe640 4 set_mon_vals no callback set 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 10 monclient: _renew_subs 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7781208d0 con 0x7fb778136030 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 10 monclient: _renew_subs 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.044+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7781367c0 con 0x7fb778136030 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 4) v1 ==== 44726+0+0 (secure 0 0 0) 0x7fb768013470 con 0x7fb778136030 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77fa20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77fa20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77fa20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:45.063 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb76dffb640 1 --2- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 0x7fb75c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77809c400 con 0x7fb778136030 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3142+0+0 (secure 0 0 0) 0x7fb76804a040 con 0x7fb778136030 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77d7bc640 1 --2- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 0x7fb75c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.045+0000 7fb77d7bc640 10 monclient: get_auth_request con 0x7fb75c0387f0 auth_method 0 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.048+0000 7fb77d7bc640 1 --2- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 0x7fb75c03acb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb774007c90 tx=0x7fb774007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.050+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb76801bcf0 con 0x7fb778136030 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.050+0000 7fb76dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.064 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.050+0000 7fb76dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:45.211 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.210+0000 7fb77fa20640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:45.212 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.210+0000 7fb77fa20640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:45.212 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.210+0000 7fb77fa20640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:45.212 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.210+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb778134580 con 0x7fb778136030 2024-09-16T15:00:45.212 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.211+0000 7fb76dffb640 1 -- 172.21.15.134:0/2050887832 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8919 (secure 0 0 0) 0x7fb768017030 con 0x7fb778136030 2024-09-16T15:00:45.213 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.211+0000 7fb76dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:45.213 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.211+0000 7fb76dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:45.213 INFO:teuthology.misc.health.smithi134.stdout: 2024-09-16T15:00:45.213 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":13,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:44.667324+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:45.214 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 msgr2=0x7fb75c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 0x7fb75c03acb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb774007c90 tx=0x7fb774007be0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 10 monclient: shutdown 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 msgr2=0x7fb7781cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e980 tx=0x7fb76800ee40 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 shutdown_connections 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> v2:172.21.15.162:6800/980547263 conn(0x7fb75c0387f0 0x7fb75c03acb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb77813bf80 0x7fb778138b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778136030 0x7fb7781cbc30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 --2- 172.21.15.134:0/2050887832 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb77809bfe0 0x7fb7781cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 >> 172.21.15.134:0/2050887832 conn(0x7fb7780a9ea0 msgr2=0x7fb7780a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 shutdown_connections 2024-09-16T15:00:45.215 INFO:teuthology.misc.health.smithi134.stderr:2024-09-16T15:00:45.213+0000 7fb77fa20640 1 -- 172.21.15.134:0/2050887832 wait complete. 2024-09-16T15:00:45.223 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-16T15:00:45.223 INFO:tasks.ceph:Creating RBD pool 2024-09-16T15:00:45.224 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.318+0000 7f1a91945640 10 monclient: get_monmap_and_config 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.318+0000 7f1a91945640 10 monclient: build_initial_monmap 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.318+0000 7f1a91945640 10 monclient: monmap: 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:45.319710+0000 2024-09-16T15:00:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:45.319710+0000 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:45.321 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:45.322 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:45.322 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 1 Processor -- start 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 1 -- start start 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient: init 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient: _add_conns ranks=[7,5,6,3,10,2,1,4,8,9,0] 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1a8c136e40 0x7f1a8c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-i con 0x7f1a8c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1a8c137220 0x7f1a8c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-h con 0x7f1a8c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.319+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-d con 0x7f1a8c135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c09abc0 con 0x7f1a8c137220 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c09ba50 con 0x7f1a8c135dc0 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0a53c0 con 0x7f1a8c136e40 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a91945640 10 monclient(hunting): authenticate will time out at 781.909022s 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:53810/0 (socket says 172.21.15.134:53810) 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 -- 172.21.15.134:0/728583359 learned_addr learned my addr 172.21.15.134:0/728583359 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): get_auth_request con 0x7f1a8c135dc0 auth_method 0 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1a8c137220 0x7f1a8c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1a8c136e40 0x7f1a8c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): get_auth_request con 0x7f1a8c136e40 auth_method 0 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): get_auth_request con 0x7f1a8c137220 auth_method 0 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_done global_id 4155 payload 274 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 -- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1a8c136e40 msgr2=0x7f1a8c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1a8c136e40 0x7f1a8c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 -- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1a8c137220 msgr2=0x7f1a8c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1a8c137220 0x7f1a8c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 monclient: found mon.noname-d 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.320+0000 7f1a8b7fe640 1 -- 172.21.15.134:0/728583359 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c073250 con 0x7f1a8c135dc0 2024-09-16T15:00:45.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8b7fe640 1 -- 172.21.15.134:0/728583359 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 msgr2=0x7f1a8c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8b7fe640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a8402a530 comp rx=0 tx=0).ready entity=mon.6 client_cookie=52d8756736a741ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 1 -- 172.21.15.134:0/728583359 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a84007a70 con 0x7f1a8c135dc0 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: dump: 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 1 -- 172.21.15.134:0/728583359 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8402d050 con 0x7f1a8c135dc0 2024-09-16T15:00:45.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a91945640 5 monclient: authenticate success, global_id 4155 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a91945640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a91945640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 1 -- 172.21.15.134:0/728583359 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a84007c30 con 0x7f1a8c135dc0 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a91945640 4 set_mon_vals no callback set 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.323+0000 7f1a8a7fc640 10 monclient: dump: 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.335 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 10 monclient: shutdown 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 -- 172.21.15.134:0/728583359 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 msgr2=0x7f1a8c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e80 tx=0x7f1a8402a530 comp rx=0 tx=0).stop 2024-09-16T15:00:45.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 -- 172.21.15.134:0/728583359 shutdown_connections 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c135dc0 0x7f1a8c136220 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1a8c137220 0x7f1a8c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 --2- 172.21.15.134:0/728583359 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1a8c136e40 0x7f1a8c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 -- 172.21.15.134:0/728583359 >> 172.21.15.134:0/728583359 conn(0x7f1a8c0a6110 msgr2=0x7f1a8c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 -- 172.21.15.134:0/728583359 shutdown_connections 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 1 -- 172.21.15.134:0/728583359 wait complete. 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 10 monclient: build_initial_monmap 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.324+0000 7f1a91945640 10 monclient: monmap: 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:45.337 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 Processor -- start 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 -- start start 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient: init 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient: _add_conns ranks=[4,6,1,5,0,3,8,2,7] 2024-09-16T15:00:45.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-c con 0x7f1a8c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-d con 0x7f1a8c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a8c137220 0x7f1a8c135680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): picked mon.noname-f con 0x7f1a8c137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0a89e0 con 0x7f1a8c137220 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c123f20 con 0x7f1a8c135dc0 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c123760 con 0x7f1a8c136e40 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a91945640 10 monclient(hunting): authenticate will time out at 781.914595s 2024-09-16T15:00:45.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:53822/0 (socket says 172.21.15.134:53822) 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:49584/0 (socket says 172.21.15.134:49584) 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 -- 172.21.15.134:0/4080788829 learned_addr learned my addr 172.21.15.134:0/4080788829 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): get_auth_request con 0x7f1a8c136e40 auth_method 0 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): get_auth_request con 0x7f1a8c135dc0 auth_method 0 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a8c137220 0x7f1a8c135680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): get_auth_request con 0x7f1a8c137220 auth_method 0 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:45.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient(hunting): handle_auth_done global_id 4164 payload 824 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 -- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 msgr2=0x7f1a8c1cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 -- 172.21.15.134:0/4080788829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a8c137220 msgr2=0x7f1a8c135680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a8c137220 0x7f1a8c135680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 monclient: found mon.noname-d 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a83fff640 1 -- 172.21.15.134:0/4080788829 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009b30 con 0x7f1a8c136e40 2024-09-16T15:00:45.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.325+0000 7f1a8affd640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a83fff640 1 -- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 msgr2=0x7f1a8c135140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a83fff640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400e8a0 tx=0x7f1a7400ed60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a740094f0 con 0x7f1a8c136e40 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: dump: 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74013470 con 0x7f1a8c136e40 2024-09-16T15:00:45.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 5 monclient: authenticate success, global_id 4164 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 10 monclient: _renew_subs 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c1368a0 con 0x7f1a8c136e40 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1a74013a50 con 0x7f1a8c136e40 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a88ff9640 10 monclient: dump: 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:45.347 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 10 monclient: _renew_subs 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a89ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.328+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c125850 con 0x7f1a8c136e40 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 4) v1 ==== 44726+0+0 (secure 0 0 0) 0x7f1a74013bb0 con 0x7f1a8c136e40 2024-09-16T15:00:45.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a88ff9640 1 --2- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 0x7f1a6c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a91945640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a91945640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a91945640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3142+0+0 (secure 0 0 0) 0x7f1a74049e50 con 0x7f1a8c136e40 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c09fca0 con 0x7f1a8c136e40 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a8affd640 1 --2- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 0x7f1a6c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.329+0000 7f1a8affd640 10 monclient: get_auth_request con 0x7f1a6c0387f0 auth_method 0 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.332+0000 7f1a8affd640 1 --2- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 0x7f1a6c03acb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009eb0 tx=0x7f1a7c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.333+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1a74017070 con 0x7f1a8c136e40 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.333+0000 7f1a88ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:45.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.333+0000 7f1a88ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:45.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.494+0000 7f1a91945640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-16T15:00:45.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.494+0000 7f1a91945640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-16T15:00:45.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.494+0000 7f1a91945640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:45.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.494+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f1a8c09b770 con 0x7f1a8c136e40 2024-09-16T15:00:45.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.965+0000 7f1a88ff9640 1 -- 172.21.15.134:0/4080788829 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1a7401b7c0 con 0x7f1a8c136e40 2024-09-16T15:00:45.967 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'rbd' created 2024-09-16T15:00:45.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.965+0000 7f1a88ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-16T15:00:45.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.965+0000 7f1a88ff9640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 msgr2=0x7f1a6c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 0x7f1a6c03acb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009eb0 tx=0x7f1a7c009190 comp rx=0 tx=0).stop 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 10 monclient: shutdown 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 msgr2=0x7f1a8c135140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400e8a0 tx=0x7f1a7400ed60 comp rx=0 tx=0).stop 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 shutdown_connections 2024-09-16T15:00:45.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> v2:172.21.15.162:6800/980547263 conn(0x7f1a6c0387f0 0x7f1a6c03acb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a8c137220 0x7f1a8c135680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1a8c136e40 0x7f1a8c135140 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 --2- 172.21.15.134:0/4080788829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1a8c135dc0 0x7f1a8c1cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 >> 172.21.15.134:0/4080788829 conn(0x7f1a8c0a6110 msgr2=0x7f1a8c09f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 shutdown_connections 2024-09-16T15:00:45.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:45.968+0000 7f1a91945640 1 -- 172.21.15.134:0/4080788829 wait complete. 2024-09-16T15:00:45.985 DEBUG:teuthology.orchestra.run.smithi134:> rbd --cluster ceph pool init rbd 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.099+0000 7f00e1690c00 10 monclient: get_monmap_and_config 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.099+0000 7f00e1690c00 10 monclient: build_initial_monmap 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.099+0000 7f00e1690c00 10 monclient: monmap: 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:46.100494+0000 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:46.100494+0000 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:46.104 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:46.105 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 Processor -- start 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 -- start start 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient: init 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient: _reopen_session rank -1 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient: _add_conns ranks=[2,5,9,10,6,1,0,7,8,3,4] 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-b con 0x55e283c62f40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x55e283bcdba0 0x55e283bcc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-h con 0x55e283bcdba0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 --2- >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-a con 0x55e283c63860 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e283aa4640 con 0x55e283c63860 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e283aa1c10 con 0x55e283c62f40 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x55e283a9d500 con 0x55e283bcdba0 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): _renew_subs 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e1690c00 10 monclient(hunting): authenticate will time out at 782.689751s 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58532/0 (socket says 172.21.15.134:58532) 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 1 -- 172.21.15.134:0/2600896961 learned_addr learned my addr 172.21.15.134:0/2600896961 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:39162/0 (socket says 172.21.15.134:39162) 2024-09-16T15:00:46.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 10 monclient(hunting): get_auth_request con 0x55e283c63860 auth_method 0 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00e13f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 10 monclient(hunting): get_auth_request con 0x55e283c62f40 auth_method 0 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.100+0000 7f00dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00e13f1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00e13f1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x55e283bcdba0 0x55e283bcc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00e13f1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): get_auth_request con 0x55e283bcdba0 auth_method 0 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient(hunting): handle_auth_done global_id 4205 payload 274 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 -- 172.21.15.134:0/2600896961 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x55e283bcdba0 msgr2=0x55e283bcc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x55e283bcdba0 0x55e283bcc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 -- 172.21.15.134:0/2600896961 >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 msgr2=0x55e283bccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 --2- 172.21.15.134:0/2600896961 >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:46.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 monclient: found mon.noname-b 2024-09-16T15:00:46.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:46.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:46.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00dbfff640 1 -- 172.21.15.134:0/2600896961 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e283a61df0 con 0x55e283c62f40 2024-09-16T15:00:46.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.101+0000 7f00e13f1640 1 --2- 172.21.15.134:0/2600896961 >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.104+0000 7f00dbfff640 1 -- 172.21.15.134:0/2600896961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 msgr2=0x55e283c63320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00dbfff640 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00a520 tx=0x7f00cc045aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42fa1495d4d3bf03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 1 -- 172.21.15.134:0/2600896961 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00cc052890 con 0x55e283c62f40 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: dump: 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 5 monclient: authenticate success, global_id 4205 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 1 -- 172.21.15.134:0/2600896961 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc052a10 con 0x55e283c62f40 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 10 monclient: get_monmap_and_config success 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 1 -- 172.21.15.134:0/2600896961 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00cc051380 con 0x55e283c62f40 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 4 set_mon_vals no callback set 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:46.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00db7fe640 10 monclient: dump: 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:46.119 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 10 monclient: shutdown 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 -- 172.21.15.134:0/2600896961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 msgr2=0x55e283c63320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00a520 tx=0x7f00cc045aa0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 -- 172.21.15.134:0/2600896961 shutdown_connections 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 --2- 172.21.15.134:0/2600896961 >> v2:172.21.15.134:3300/0 conn(0x55e283c63860 0x55e283bccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x55e283bcdba0 0x55e283bcc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 --2- 172.21.15.134:0/2600896961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e283c62f40 0x55e283c63320 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 -- 172.21.15.134:0/2600896961 >> 172.21.15.134:0/2600896961 conn(0x55e283bafe90 msgr2=0x55e283bb02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 -- 172.21.15.134:0/2600896961 shutdown_connections 2024-09-16T15:00:46.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.105+0000 7f00e1690c00 1 -- 172.21.15.134:0/2600896961 wait complete. 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.106+0000 7f00e1690c00 10 monclient: build_initial_monmap 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.106+0000 7f00e1690c00 10 monclient: monmap: 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:46.121 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.106+0000 7f00e1690c00 1 Processor -- start 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.106+0000 7f00e1690c00 1 -- start start 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.106+0000 7f00e1690c00 10 monclient: init 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient: _reopen_session rank -1 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient: _add_conns ranks=[1,0,7,6,4,3,8,2,5] 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e283bcdba0 0x55e283c4b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-f con 0x55e283bcdba0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 0x55e283c4bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-a con 0x55e283c62f40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:46.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x55e283c4c470 0x55e283c77e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): picked mon.noname-i con 0x55e283c4c470 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e283a38b90 con 0x55e283c62f40 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e283a38830 con 0x55e283bcdba0 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x55e283a1b550 con 0x55e283c4c470 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): _renew_subs 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e1690c00 10 monclient(hunting): authenticate will time out at 782.696094s 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 0x55e283c4bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 0x55e283c4bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58544/0 (socket says 172.21.15.134:58544) 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 -- 172.21.15.134:0/1457831818 learned_addr learned my addr 172.21.15.134:0/1457831818 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:46.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): get_auth_request con 0x55e283c62f40 auth_method 0 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e283bcdba0 0x55e283c4b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): get_auth_request con 0x55e283bcdba0 auth_method 0 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e13f1640 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x55e283c4c470 0x55e283c77e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e13f1640 10 monclient(hunting): get_auth_request con 0x55e283c4c470 auth_method 0 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e13f1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e13f1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00e13f1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient(hunting): handle_auth_done global_id 4257 payload 824 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 -- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x55e283c4c470 msgr2=0x55e283c77e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x55e283c4c470 0x55e283c77e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 -- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e283bcdba0 msgr2=0x55e283c4b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e283bcdba0 0x55e283c4b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:46.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 monclient: found mon.noname-a 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.107+0000 7f00d37fe640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d400a1c0 con 0x55e283c62f40 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d37fe640 1 -- 172.21.15.134:0/1457831818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 msgr2=0x55e283c4bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d37fe640 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 0x55e283c4bf30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00c4008790 tx=0x7f00c4008880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00c400f040 con 0x55e283c62f40 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: dump: 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:46.126 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4012c10 con 0x55e283c62f40 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00e1690c00 5 monclient: authenticate success, global_id 4257 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00c4012d80 con 0x55e283c62f40 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00e1690c00 10 monclient: _renew_subs 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00e1690c00 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc005d20 con 0x55e283c62f40 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00d8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.109+0000 7f00daffd640 4 set_mon_vals no callback set 2024-09-16T15:00:46.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00d8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00d8ff9640 10 monclient: dump: 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:46.129 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00e1690c00 10 monclient: _renew_subs 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00e1690c00 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e283a3ad00 con 0x55e283c62f40 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 4) v1 ==== 44726+0+0 (secure 0 0 0) 0x7f00c40127a0 con 0x55e283c62f40 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00d8ff9640 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6800/980547263 conn(0x7f00bc0370e0 0x7f00bc0395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.110+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3495+0+0 (secure 0 0 0) 0x7f00c4048d10 con 0x55e283c62f40 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00e1690c00 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00e1690c00 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00e1690c00 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00dbfff640 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6800/980547263 conn(0x7f00bc0370e0 0x7f00bc0395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x55e283a47c50 con 0x55e283c62f40 2024-09-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.111+0000 7f00dbfff640 10 monclient: get_auth_request con 0x7f00bc0370e0 auth_method 0 2024-09-16T15:00:46.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.113+0000 7f00dbfff640 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6800/980547263 conn(0x7f00bc0370e0 0x7f00bc0395a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00a520 tx=0x7f00cc060040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:46.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.966+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 141+0+0 (secure 0 0 0) 0x7f00c4012a50 con 0x55e283c62f40 2024-09-16T15:00:46.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.966+0000 7f00d8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-16T15:00:46.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.966+0000 7f00d8ff9640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-16T15:00:46.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.966+0000 7f00e1690c00 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.966+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55e283a38620 con 0x55e283c62f40 2024-09-16T15:00:46.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00c4012a50 con 0x55e283c62f40 2024-09-16T15:00:46.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00d8ff9640 10 monclient: handle_get_version_reply finishing 1 version 15 2024-09-16T15:00:46.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00daffd640 10 monclient: _renew_subs 2024-09-16T15:00:46.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00daffd640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00daffd640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f00c0002c50 con 0x55e283c62f40 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 1028+0+0 (secure 0 0 0) 0x7f00c401d460 con 0x55e283c62f40 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00e1690c00 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6809/518841521 conn(0x55e283c78d80 0x55e283c7b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.967+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> v2:172.21.15.162:6809/518841521 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x55e283c7b6b0 con 0x55e283c78d80 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.968+0000 7f00e13f1640 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6809/518841521 conn(0x55e283c78d80 0x55e283c7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:46.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.968+0000 7f00e13f1640 10 monclient: get_auth_request con 0x55e283c78d80 auth_method 0 2024-09-16T15:00:46.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.972+0000 7f00e13f1640 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6809/518841521 conn(0x55e283c78d80 0x55e283c7b170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:46.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.979+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 <== osd.5 v2:172.21.15.162:6809/518841521 1 ==== osd_op_reply(1 rbd_trash [create] v15'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7f00d400a9f0 con 0x55e283c78d80 2024-09-16T15:00:46.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.982+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> v2:172.21.15.162:6809/518841521 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x55e283cca670 con 0x55e283c78d80 2024-09-16T15:00:46.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.982+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 <== osd.5 v2:172.21.15.162:6809/518841521 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f00d400a9f0 con 0x55e283c78d80 2024-09-16T15:00:46.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.982+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 --> v2:172.21.15.162:6809/518841521 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x55e283cc9780 con 0x55e283c78d80 2024-09-16T15:00:46.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.983+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 <== osd.5 v2:172.21.15.162:6809/518841521 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f00d400a9f0 con 0x55e283c78d80 2024-09-16T15:00:46.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.983+0000 7f00da7fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:46.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:46.983+0000 7f00da7fc640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7f00b4001f20 con 0x55e283c62f40 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.302+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v16) v1 ==== 55+0+0 (secure 0 0 0) 0x7f00c401bb10 con 0x55e283c62f40 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.302+0000 7f00d8ff9640 10 monclient: _renew_subs 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.302+0000 7f00d8ff9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.302+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7f00bc002830 con 0x55e283c62f40 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.303+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 10 ==== osd_map(16..16 src has 1..16) v4 ==== 626+0+0 (secure 0 0 0) 0x7f00c401dc10 con 0x55e283c62f40 2024-09-16T15:00:48.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.303+0000 7f00daffd640 1 -- 172.21.15.134:0/1457831818 --> v2:172.21.15.162:6809/518841521 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f00c0005770 con 0x55e283c78d80 2024-09-16T15:00:48.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.307+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 <== osd.5 v2:172.21.15.162:6809/518841521 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v16'1 uv1 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f00d400a9f0 con 0x55e283c78d80 2024-09-16T15:00:48.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.307+0000 7f00da7fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:48.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.307+0000 7f00da7fc640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v16) v4 -- 0x7f00b40033a0 con 0x55e283c62f40 2024-09-16T15:00:48.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:48.450+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 11 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f00c4005020 con 0x55e283c62f40 2024-09-16T15:00:49.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.107+0000 7f00d2ffd640 10 monclient: tick 2024-09-16T15:00:49.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.107+0000 7f00d2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:49.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.107+0000 7f00d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:49.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.107+0000 7f00d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:49.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 12 ==== pool_op_reply(tid 6 (0) Success v17) v1 ==== 43+0+0 (secure 0 0 0) 0x7f00c401bb10 con 0x55e283c62f40 2024-09-16T15:00:49.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00d8ff9640 10 monclient: _renew_subs 2024-09-16T15:00:49.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00d8ff9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:49.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7f00bc03d380 con 0x55e283c62f40 2024-09-16T15:00:49.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00d8ff9640 1 -- 172.21.15.134:0/1457831818 <== mon.0 v2:172.21.15.134:3300/0 13 ==== osd_map(17..17 src has 1..17) v4 ==== 654+0+0 (secure 0 0 0) 0x7f00c4042da0 con 0x55e283c62f40 2024-09-16T15:00:49.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.317+0000 7f00daffd640 1 -- 172.21.15.134:0/1457831818 --> v2:172.21.15.162:6809/518841521 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x7f00c0005b40 con 0x55e283c78d80 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 <== osd.5 v2:172.21.15.162:6809/518841521 5 ==== osd_op_reply(7 rbd_info [write 0~19] v17'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f00d400a9f0 con 0x55e283c78d80 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6809/518841521 conn(0x55e283c78d80 msgr2=0x55e283c7b170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6809/518841521 conn(0x55e283c78d80 0x55e283c7b170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6800/980547263 conn(0x7f00bc0370e0 msgr2=0x7f00bc0395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 --2- 172.21.15.134:0/1457831818 >> v2:172.21.15.162:6800/980547263 conn(0x7f00bc0370e0 0x7f00bc0395a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00a520 tx=0x7f00cc060040 comp rx=0 tx=0).stop 2024-09-16T15:00:49.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 10 monclient: shutdown 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 msgr2=0x55e283c4bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 --2- 172.21.15.134:0/1457831818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e283c62f40 0x55e283c4bf30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00c4008790 tx=0x7f00c4008880 comp rx=0 tx=0).stop 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e13f1640 1 -- 172.21.15.134:0/1457831818 reap_dead start 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 shutdown_connections 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 >> 172.21.15.134:0/1457831818 conn(0x55e283bafe90 msgr2=0x55e283bd3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 shutdown_connections 2024-09-16T15:00:49.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.329+0000 7f00e1690c00 1 -- 172.21.15.134:0/1457831818 wait complete. 2024-09-16T15:00:49.335 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T15:00:49.335 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T15:00:49.336 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T15:00:49.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T15:00:49.336 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T15:00:49.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 10 monclient: get_monmap_and_config 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 10 monclient: build_initial_monmap 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 10 monclient: monmap: 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:49.434048+0000 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:49.434048+0000 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:49.435 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:49.436 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 1 Processor -- start 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 1 -- start start 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.433+0000 7f0379632640 10 monclient: init 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient: _add_conns ranks=[4,1,10,3,5,0,6,2,8,7,9] 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): picked mon.noname-c con 0x7f0374133a10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0374133df0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 0x7f037413b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): picked mon.noname-f con 0x7f0374138f40 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:49.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0374073250 con 0x7f0374133df0 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03740a3ca0 con 0x7f0374138f40 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f037409bcf0 con 0x7f0374133a10 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0379632640 10 monclient(hunting): authenticate will time out at 786.023385s 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33318/0 (socket says 172.21.15.134:33318) 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 -- 172.21.15.134:0/2275554460 learned_addr learned my addr 172.21.15.134:0/2275554460 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): get_auth_request con 0x7f0374133a10 auth_method 0 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f03737fe640 1 --2- 172.21.15.134:0/2275554460 >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 0x7f037413b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient(hunting): handle_auth_done global_id 4189 payload 274 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 -- 172.21.15.134:0/2275554460 >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 msgr2=0x7f037413b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 --2- 172.21.15.134:0/2275554460 >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 0x7f037413b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 -- 172.21.15.134:0/2275554460 >> v1:172.21.15.162:6789/0 conn(0x7f0374133df0 legacy=0x7f03741341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 monclient: found mon.noname-c 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f0372ffd640 1 -- 172.21.15.134:0/2275554460 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037409bfd0 con 0x7f0374133a10 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f03737fe640 10 monclient: get_auth_request con 0x7f0374138f40 auth_method 0 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.434+0000 7f03737fe640 1 --2- 172.21.15.134:0/2275554460 >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 0x7f037413b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0372ffd640 1 -- 172.21.15.134:0/2275554460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 msgr2=0x7f037409cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0372ffd640 1 --2- 172.21.15.134:0/2275554460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0364009870 tx=0x7f036402a100 comp rx=0 tx=0).ready entity=mon.4 client_cookie=67a1d781461100d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 1 -- 172.21.15.134:0/2275554460 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03640307b0 con 0x7f0374133a10 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: dump: 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.450 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:49.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 1 -- 172.21.15.134:0/2275554460 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364030930 con 0x7f0374133a10 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0379632640 5 monclient: authenticate success, global_id 4189 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0379632640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0379632640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 1 -- 172.21.15.134:0/2275554460 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03640313a0 con 0x7f0374133a10 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f0379632640 4 set_mon_vals no callback set 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.437+0000 7f03727fc640 10 monclient: dump: 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.452 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 10 monclient: shutdown 2024-09-16T15:00:49.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 -- 172.21.15.134:0/2275554460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 msgr2=0x7f037409cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 --2- 172.21.15.134:0/2275554460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0364009870 tx=0x7f036402a100 comp rx=0 tx=0).stop 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 -- 172.21.15.134:0/2275554460 shutdown_connections 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 --2- 172.21.15.134:0/2275554460 >> v2:172.21.15.162:3300/0 conn(0x7f0374138f40 0x7f037413b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 --2- 172.21.15.134:0/2275554460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f037409cbb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 -- 172.21.15.134:0/2275554460 >> 172.21.15.134:0/2275554460 conn(0x7f03740a2e20 msgr2=0x7f03740a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 -- 172.21.15.134:0/2275554460 shutdown_connections 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 1 -- 172.21.15.134:0/2275554460 wait complete. 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 10 monclient: build_initial_monmap 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.438+0000 7f0379632640 10 monclient: monmap: 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:49.454 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:49.455 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 Processor -- start 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 -- start start 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient: init 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient: _add_conns ranks=[4,3,6,5,7,0,8,1,2] 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): picked mon.noname-c con 0x7f0374133a10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:49.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0374133df0 0x7f0374099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): picked mon.noname-g con 0x7f0374133df0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0374138f40 0x7f037413c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): picked mon.noname-d con 0x7f0374138f40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0374132f00 con 0x7f0374133df0 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f037406a300 con 0x7f0374133a10 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f037406a7c0 con 0x7f0374138f40 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.439+0000 7f0379632640 10 monclient(hunting): authenticate will time out at 786.028983s 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33328/0 (socket says 172.21.15.134:33328) 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 -- 172.21.15.134:0/879253270 learned_addr learned my addr 172.21.15.134:0/879253270 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): get_auth_request con 0x7f0374133a10 auth_method 0 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0374138f40 0x7f037413c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): get_auth_request con 0x7f0374138f40 auth_method 0 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0374133df0 0x7f0374099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): get_auth_request con 0x7f0374133df0 auth_method 0 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f03737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f036a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient(hunting): handle_auth_done global_id 4198 payload 824 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 -- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0374138f40 msgr2=0x7f037413c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0374138f40 0x7f037413c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 -- 172.21.15.134:0/879253270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0374133df0 msgr2=0x7f0374099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0374133df0 0x7f0374099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 monclient: found mon.noname-c 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.440+0000 7f0372ffd640 1 -- 172.21.15.134:0/879253270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640094d0 con 0x7f0374133a10 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0372ffd640 1 -- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 msgr2=0x7f03740992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0372ffd640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f03640046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0364004460 con 0x7f0374133a10 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: dump: 2024-09-16T15:00:49.460 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.461 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: _finish_auth 0 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364003a80 con 0x7f0374133a10 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 5 monclient: authenticate success, global_id 4198 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0364003bf0 con 0x7f0374133a10 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 10 monclient: _renew_subs 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0374071620 con 0x7f0374133a10 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0371ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f036bfff640 10 monclient: dump: 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.463 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 10 monclient: _renew_subs 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.442+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0374071900 con 0x7f0374133a10 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.443+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0364039040 con 0x7f0374133a10 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.443+0000 7f0379632640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.443+0000 7f0379632640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.443+0000 7f0379632640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.444+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037409cbb0 con 0x7f0374133a10 2024-09-16T15:00:49.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.444+0000 7f036bfff640 1 --2- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 0x7f035403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.444+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f036406a5a0 con 0x7f0374133a10 2024-09-16T15:00:49.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.444+0000 7f036a7fc640 1 --2- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 0x7f035403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.444+0000 7f036a7fc640 10 monclient: get_auth_request con 0x7f0354038880 auth_method 0 2024-09-16T15:00:49.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.446+0000 7f036a7fc640 1 --2- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 0x7f035403ad40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0374066eb0 tx=0x7f035c007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.448+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0364003610 con 0x7f0374133a10 2024-09-16T15:00:49.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.448+0000 7f036bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:49.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.448+0000 7f036bfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:49.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.652+0000 7f0379632640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-16T15:00:49.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.652+0000 7f0379632640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-16T15:00:49.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.652+0000 7f0379632640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.652+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0374066770 con 0x7f0374133a10 2024-09-16T15:00:49.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.654+0000 7f036bfff640 1 -- 172.21.15.134:0/879253270 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86085 (secure 0 0 0) 0x7f0364037020 con 0x7f0374133a10 2024-09-16T15:00:49.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.654+0000 7f036bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-16T15:00:49.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.654+0000 7f036bfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:49.657 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 msgr2=0x7f035403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 0x7f035403ad40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0374066eb0 tx=0x7f035c007e30 comp rx=0 tx=0).stop 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 10 monclient: shutdown 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 msgr2=0x7f03740992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f03640046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 shutdown_connections 2024-09-16T15:00:49.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0374138f40 0x7f037413c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> v2:172.21.15.162:6800/980547263 conn(0x7f0354038880 0x7f035403ad40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0374133df0 0x7f0374099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 --2- 172.21.15.134:0/879253270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0374133a10 0x7f03740992e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 >> 172.21.15.134:0/879253270 conn(0x7f03740a2e20 msgr2=0x7f03740a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 shutdown_connections 2024-09-16T15:00:49.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.656+0000 7f0379632640 1 -- 172.21.15.134:0/879253270 wait complete. 2024-09-16T15:00:49.668 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":5,"flags":0,"active_gid":4120,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":980547263}]},"active_addr":"172.21.15.162:6800/980547263","active_change":"2024-09-16T15:00:35.826636+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2732230041}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2224591047}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":2998553745}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3241348588}]}]} 2024-09-16T15:00:49.671 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T15:00:49.671 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T15:00:49.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.765+0000 7fe841d1f640 10 monclient: get_monmap_and_config 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.765+0000 7fe841d1f640 10 monclient: build_initial_monmap 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.765+0000 7fe841d1f640 10 monclient: monmap: 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:49.766409+0000 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:49.766409+0000 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:49.768 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 Processor -- start 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 -- start start 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient: init 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient: _add_conns ranks=[6,0,9,2,4,7,8,10,5,1,3] 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 0x7fe83c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-d con 0x7fe83c136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe83c09bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 0x7fe83c137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-a con 0x7fe83c09c400 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c073250 con 0x7fe83c09bfe0 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c09b600 con 0x7fe83c09c400 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0a5740 con 0x7fe83c136030 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.766+0000 7fe841d1f640 10 monclient(hunting): authenticate will time out at 786.355946s 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 0x7fe83c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 0x7fe83c137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe833fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe83c09bfe0 0x7fe83c136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38916/0 (socket says 172.21.15.134:38916) 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe833fff640 1 -- 172.21.15.134:0/1631948860 learned_addr learned my addr 172.21.15.134:0/1631948860 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request con 0x7fe83c136030 auth_method 0 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): get_auth_request con 0x7fe83c09c400 auth_method 0 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_done global_id 4209 payload 274 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1631948860 >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 msgr2=0x7fe83c137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1631948860 >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 0x7fe83c137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1631948860 >> v1:172.21.15.134:6789/0 conn(0x7fe83c09bfe0 legacy=0x7fe83c136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 monclient: found mon.noname-d 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1631948860 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0a52c0 con 0x7fe83c136030 2024-09-16T15:00:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.767+0000 7fe83bfff640 1 --2- 172.21.15.134:0/1631948860 >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 0x7fe83c137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1631948860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 msgr2=0x7fe83c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1631948860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 0x7fe83c136410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097a0 tx=0x7fe82c02a010 comp rx=0 tx=0).ready entity=mon.6 client_cookie=ebfefb67b25ff09 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 1 -- 172.21.15.134:0/1631948860 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c02ad30 con 0x7fe83c136030 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: dump: 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.780 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.781 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: _finish_auth 0 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 1 -- 172.21.15.134:0/1631948860 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c0307b0 con 0x7fe83c136030 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 5 monclient: authenticate success, global_id 4209 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 1 -- 172.21.15.134:0/1631948860 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c030d90 con 0x7fe83c136030 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 4 set_mon_vals no callback set 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:49.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe83affd640 10 monclient: dump: 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.783 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 10 monclient: shutdown 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 -- 172.21.15.134:0/1631948860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 msgr2=0x7fe83c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1631948860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 0x7fe83c136410 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097a0 tx=0x7fe82c02a010 comp rx=0 tx=0).stop 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 -- 172.21.15.134:0/1631948860 shutdown_connections 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1631948860 >> v2:172.21.15.134:3300/0 conn(0x7fe83c09c400 0x7fe83c137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1631948860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe83c136030 0x7fe83c136410 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 -- 172.21.15.134:0/1631948860 >> 172.21.15.134:0/1631948860 conn(0x7fe83c0a9ea0 msgr2=0x7fe83c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.770+0000 7fe841d1f640 1 -- 172.21.15.134:0/1631948860 shutdown_connections 2024-09-16T15:00:49.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 1 -- 172.21.15.134:0/1631948860 wait complete. 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 10 monclient: build_initial_monmap 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 10 monclient: monmap: 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:49.785 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 1 Processor -- start 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 1 -- start start 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.771+0000 7fe841d1f640 10 monclient: init 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient: _add_conns ranks=[4,2,5,0,3,7,1,8,6] 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-c con 0x7fe83c09bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 0x7fe83c1cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-b con 0x7fe83c09c400 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 0x7fe83c1235a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): picked mon.noname-h con 0x7fe83c136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c126fd0 con 0x7fe83c09c400 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0a8920 con 0x7fe83c09bfe0 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c127220 con 0x7fe83c136030 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe841d1f640 10 monclient(hunting): authenticate will time out at 786.361342s 2024-09-16T15:00:49.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 0x7fe83c1cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33336/0 (socket says 172.21.15.134:33336) 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1261953020 learned_addr learned my addr 172.21.15.134:0/1261953020 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request con 0x7fe83c09bfe0 auth_method 0 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): get_auth_request con 0x7fe83c09c400 auth_method 0 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 0x7fe83c1235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 10 monclient(hunting): get_auth_request con 0x7fe83c136030 auth_method 0 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient(hunting): handle_auth_done global_id 4207 payload 824 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1261953020 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 msgr2=0x7fe83c1235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 0x7fe83c1235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 msgr2=0x7fe83c1cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 0x7fe83c1cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 monclient: found mon.noname-c 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1261953020 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009a70 con 0x7fe83c09bfe0 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe83bfff640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 0x7fe83c1235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.772+0000 7fe833fff640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 0x7fe83c1cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe83b7fe640 1 -- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 msgr2=0x7fe83c1cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097a0 tx=0x7fe82c0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c030d40 con 0x7fe83c09bfe0 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: dump: 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.791 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c003a80 con 0x7fe83c09bfe0 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 5 monclient: authenticate success, global_id 4207 2024-09-16T15:00:49.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 10 monclient: _renew_subs 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c009400 con 0x7fe83c09bfe0 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe82c003bf0 con 0x7fe83c09bfe0 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe838ff9640 10 monclient: dump: 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:49.793 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe83a7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:49.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 10 monclient: _renew_subs 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c09f860 con 0x7fe83c09bfe0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.775+0000 7fe83b7fe640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097a0 tx=0x7fe82c0046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fe82c06a5f0 osd_map(17..17 src has 1..17) v4 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe82c039040 con 0x7fe83c09bfe0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe838ff9640 1 --2- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 0x7fe81c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe841d1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe841d1f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe841d1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c0a8d50 con 0x7fe83c09bfe0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe833fff640 1 --2- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 0x7fe81c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:49.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.776+0000 7fe833fff640 10 monclient: get_auth_request con 0x7fe81c038880 auth_method 0 2024-09-16T15:00:49.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.779+0000 7fe833fff640 1 --2- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 0x7fe81c03ad40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe824007c90 tx=0x7fe824007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:49.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:49.781+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fe82c06a5f0 con 0x7fe83c09bfe0 2024-09-16T15:00:50.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.604+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe82c037020 con 0x7fe83c09bfe0 2024-09-16T15:00:50.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.604+0000 7fe838ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:50.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.604+0000 7fe838ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe841d1f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:50.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe841d1f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe841d1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe83c136410 con 0x7fe83c09bfe0 2024-09-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe838ff9640 1 -- 172.21.15.134:0/1261953020 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+10854 (secure 0 0 0) 0x7fe82c041070 con 0x7fe83c09bfe0 2024-09-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe838ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:50.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.770+0000 7fe838ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:50.773 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":17,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:49.308510+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:00:45.513777+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.343686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.259639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.265227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.302963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 msgr2=0x7fe81c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 0x7fe81c03ad40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe824007c90 tx=0x7fe824007be0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 10 monclient: shutdown 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 msgr2=0x7fe83c1cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0097a0 tx=0x7fe82c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 shutdown_connections 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> v2:172.21.15.162:6800/980547263 conn(0x7fe81c038880 0x7fe81c03ad40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe83c136030 0x7fe83c1235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe83c09c400 0x7fe83c1cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 --2- 172.21.15.134:0/1261953020 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe83c09bfe0 0x7fe83c1cb6d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 >> 172.21.15.134:0/1261953020 conn(0x7fe83c0a9ea0 msgr2=0x7fe83c09e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:50.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 shutdown_connections 2024-09-16T15:00:50.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.773+0000 7fe841d1f640 1 -- 172.21.15.134:0/1261953020 wait complete. 2024-09-16T15:00:50.785 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T15:00:50.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:50.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.879+0000 7f48b585a640 10 monclient: get_monmap_and_config 2024-09-16T15:00:50.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.879+0000 7f48b585a640 10 monclient: build_initial_monmap 2024-09-16T15:00:50.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.879+0000 7f48b585a640 10 monclient: monmap: 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:50.880819+0000 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:50.880819+0000 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:50.882 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:50.883 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:50.884 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:50.884 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:50.884 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.880+0000 7f48b585a640 1 Processor -- start 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.880+0000 7f48b585a640 1 -- start start 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.880+0000 7f48b585a640 10 monclient: init 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient: _add_conns ranks=[8,1,7,10,5,0,6,2,3,4,9] 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-e con 0x7f48b0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f48b009bfe0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:50.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b0137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-i con 0x7f48b009c400 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b0073250 con 0x7f48b009bfe0 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48b009c880 con 0x7f48b009c400 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f48b009b600 con 0x7f48b0136030 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48b585a640 10 monclient(hunting): authenticate will time out at 787.470201s 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41686/0 (socket says 172.21.15.134:41686) 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 -- 172.21.15.134:0/483268393 learned_addr learned my addr 172.21.15.134:0/483268393 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:50.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): get_auth_request con 0x7f48b0136030 auth_method 0 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48af7fe640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b0137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48af7fe640 10 monclient(hunting): get_auth_request con 0x7f48b009c400 auth_method 0 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient(hunting): handle_auth_done global_id 4256 payload 274 2024-09-16T15:00:50.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 -- 172.21.15.134:0/483268393 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 msgr2=0x7f48b0137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b0137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 -- 172.21.15.134:0/483268393 >> v1:172.21.15.162:6789/0 conn(0x7f48b009bfe0 legacy=0x7f48b0136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 monclient: found mon.noname-e 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.881+0000 7f48aeffd640 1 -- 172.21.15.134:0/483268393 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b00a5740 con 0x7f48b0136030 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48aeffd640 1 -- 172.21.15.134:0/483268393 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 msgr2=0x7f48b0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48aeffd640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48a00098a0 tx=0x7f48a002a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=34785475be01853b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 1 -- 172.21.15.134:0/483268393 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48a002adf0 con 0x7f48b0136030 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:50.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: dump: 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:50.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 1 -- 172.21.15.134:0/483268393 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a00307b0 con 0x7f48b0136030 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 1 -- 172.21.15.134:0/483268393 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48a0030d90 con 0x7f48b0136030 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48b585a640 5 monclient: authenticate success, global_id 4256 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48b585a640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48b585a640 4 set_mon_vals no callback set 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.884+0000 7f48ae7fc640 10 monclient: dump: 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 10 monclient: shutdown 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:50.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 -- 172.21.15.134:0/483268393 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 msgr2=0x7f48b0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48a00098a0 tx=0x7f48a002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 -- 172.21.15.134:0/483268393 shutdown_connections 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b0137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 --2- 172.21.15.134:0/483268393 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b0136030 0x7f48b0136410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 -- 172.21.15.134:0/483268393 >> 172.21.15.134:0/483268393 conn(0x7f48b00a9ea0 msgr2=0x7f48b00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 -- 172.21.15.134:0/483268393 shutdown_connections 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 1 -- 172.21.15.134:0/483268393 wait complete. 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 10 monclient: build_initial_monmap 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.885+0000 7f48b585a640 10 monclient: monmap: 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:50.901 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 Processor -- start 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 -- start start 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient: init 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient: _add_conns ranks=[8,7,0,1,5,2,4,6,3] 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-e con 0x7f48b009bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-i con 0x7f48b009c400 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:50.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 0x7f48b011ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): picked mon.noname-a con 0x7f48b0136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b009f920 con 0x7f48b0136030 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00a8f10 con 0x7f48b009c400 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0121950 con 0x7f48b009bfe0 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48b585a640 10 monclient(hunting): authenticate will time out at 787.475770s 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 0x7f48b011ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41696/0 (socket says 172.21.15.134:41696) 2024-09-16T15:00:50.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 1 -- 172.21.15.134:0/2670154795 learned_addr learned my addr 172.21.15.134:0/2670154795 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): get_auth_request con 0x7f48b009bfe0 auth_method 0 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): get_auth_request con 0x7f48b0136030 auth_method 0 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:50.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.886+0000 7f48af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 10 monclient(hunting): get_auth_request con 0x7f48b009c400 auth_method 0 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 10 monclient(hunting): handle_auth_done global_id 4265 payload 824 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 -- 172.21.15.134:0/2670154795 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 msgr2=0x7f48b011fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b011fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 -- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 msgr2=0x7f48b011ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 0x7f48b011ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:50.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 monclient: found mon.noname-e 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48aeffd640 1 -- 172.21.15.134:0/2670154795 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00094d0 con 0x7f48b009bfe0 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48af7fe640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 0x7f48b011ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.887+0000 7f48a7fff640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48aeffd640 1 -- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 msgr2=0x7f48b01bec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48aeffd640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48a00090b0 tx=0x7f48a00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48a0004510 con 0x7f48b009bfe0 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: dump: 2024-09-16T15:00:50.907 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:50.908 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0003610 con 0x7f48b009bfe0 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 5 monclient: authenticate success, global_id 4265 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 10 monclient: _renew_subs 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b00a8a50 con 0x7f48b009bfe0 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48a0003bf0 con 0x7f48b009bfe0 2024-09-16T15:00:50.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48acff9640 10 monclient: dump: 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:50.910 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 10 monclient: _renew_subs 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b0120da0 con 0x7f48b009bfe0 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.889+0000 7f48adffb640 4 set_mon_vals no callback set 2024-09-16T15:00:50.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f48a0039040 con 0x7f48b009bfe0 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48b585a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48b585a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48b585a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0121d80 con 0x7f48b009bfe0 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48acff9640 1 --2- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 0x7f489003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.890+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f48a006a500 con 0x7f48b009bfe0 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.891+0000 7f48a7fff640 1 --2- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 0x7f489003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:50.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.891+0000 7f48a7fff640 10 monclient: get_auth_request con 0x7f4890038880 auth_method 0 2024-09-16T15:00:50.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.893+0000 7f48a7fff640 1 --2- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 0x7f489003ad40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f48b01bf370 tx=0x7f4898009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:50.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.895+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f48a003a7a0 con 0x7f48b009bfe0 2024-09-16T15:00:50.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.895+0000 7f48acff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:50.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:50.895+0000 7f48acff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.061+0000 7f48b585a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.061+0000 7f48b585a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.061+0000 7f48b585a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.061+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b00a6940 con 0x7f48b009bfe0 2024-09-16T15:00:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.062+0000 7f48acff9640 1 -- 172.21.15.134:0/2670154795 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+10854 (secure 0 0 0) 0x7f48a0041090 con 0x7f48b009bfe0 2024-09-16T15:00:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.062+0000 7f48acff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.062+0000 7f48acff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.064 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:51.064 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":17,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:49.308510+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:00:45.513777+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.343686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.259639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.265227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.302963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 msgr2=0x7f489003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 0x7f489003ad40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f48b01bf370 tx=0x7f4898009040 comp rx=0 tx=0).stop 2024-09-16T15:00:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 10 monclient: shutdown 2024-09-16T15:00:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 msgr2=0x7f48b01bec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f48a00090b0 tx=0x7f48a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 shutdown_connections 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> v2:172.21.15.162:6800/980547263 conn(0x7f4890038880 0x7f489003ad40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48b0136030 0x7f48b011ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f48b009c400 0x7f48b011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 --2- 172.21.15.134:0/2670154795 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f48b009bfe0 0x7f48b01bec20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 >> 172.21.15.134:0/2670154795 conn(0x7f48b00a9ea0 msgr2=0x7f48b009e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 shutdown_connections 2024-09-16T15:00:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.064+0000 7f48b585a640 1 -- 172.21.15.134:0/2670154795 wait complete. 2024-09-16T15:00:51.077 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T15:00:51.077 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T15:00:51.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T15:00:51.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T15:00:51.079 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T15:00:51.079 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T15:00:51.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f6a6a1ae640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f32d811f640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f32d811f640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f32d811f640 10 monclient: monmap: 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.188021+0000 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.188021+0000 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.189 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.190 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f6a6a1ae640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f6a6a1ae640 10 monclient: monmap: 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.188019+0000 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.188019+0000 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f32d811f640 1 Processor -- start 2024-09-16T15:00:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f6a6a1ae640 1 Processor -- start 2024-09-16T15:00:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.187+0000 7f32d811f640 1 -- start start 2024-09-16T15:00:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient: init 2024-09-16T15:00:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 1 -- start start 2024-09-16T15:00:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 10 monclient: init 2024-09-16T15:00:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient: _add_conns ranks=[4,0,9,1,3,5,2,6,8,10,7] 2024-09-16T15:00:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 0x7f32d0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-c con 0x7f32d0136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f32d009bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:51.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d56ba640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f32d009bfe0 0x7f32d0136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38920/0 (socket says 172.21.15.134:38920) 2024-09-16T15:00:51.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 10 monclient: _add_conns ranks=[8,7,3,4,9,2,5,6,1,10,0] 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f32d009c400 0x7f32d0137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d56ba640 1 -- 172.21.15.134:0/940526564 learned_addr learned my addr 172.21.15.134:0/940526564 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-a con 0x7f32d009c400 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 0x7f32d0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d0073250 con 0x7f32d009bfe0 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d009b600 con 0x7f32d009c400 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00a5740 con 0x7f32d0136030 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f32d811f640 10 monclient(hunting): authenticate will time out at 787.777979s 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): get_auth_request con 0x7f32d0136030 auth_method 0 2024-09-16T15:00:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 0x7f6a64136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-e con 0x7f6a64136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a6409bfe0 0x7f6a6409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-i con 0x7f6a6409bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.188+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 0x7f6a641370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-g con 0x7f6a64136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 0x7f6a641370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a62ffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a6409bfe0 0x7f6a6409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_done global_id 4216 payload 274 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 1 -- 172.21.15.134:0/940526564 >> v2:172.21.15.134:3300/0 conn(0x7f32d009c400 msgr2=0x7f32d0137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/940526564 >> v2:172.21.15.134:3300/0 conn(0x7f32d009c400 0x7f32d0137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 1 -- 172.21.15.134:0/940526564 >> v1:172.21.15.134:6789/0 conn(0x7f32d009bfe0 legacy=0x7f32d0136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 1 monclient: found mon.noname-c 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:51.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f32d5ebb640 1 -- 172.21.15.134:0/940526564 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00a52c0 con 0x7f32d0136030 2024-09-16T15:00:51.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 0x7f6a641370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:60974/0 (socket says 172.21.15.134:60974) 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 1 -- 172.21.15.134:0/2570158357 learned_addr learned my addr 172.21.15.134:0/2570158357 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6409b600 con 0x7f6a64136c60 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a5740 con 0x7f6a6409bfe0 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a52c0 con 0x7f6a64136030 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 0x7f6a64136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a6a1ae640 10 monclient(hunting): authenticate will time out at 787.778654s 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 10 monclient(hunting): get_auth_request con 0x7f6a64136c60 auth_method 0 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): get_auth_request con 0x7f6a64136030 auth_method 0 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient(hunting): handle_auth_done global_id 4274 payload 274 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 -- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a6409bfe0 msgr2=0x7f6a6409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a6409bfe0 0x7f6a6409c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 -- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 msgr2=0x7f6a641370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 0x7f6a641370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 monclient: found mon.noname-e 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.189+0000 7f6a637fe640 1 -- 172.21.15.134:0/2570158357 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64073250 con 0x7f6a64136030 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.192+0000 7f32d5ebb640 1 -- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 msgr2=0x7f32d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.192+0000 7f6a637fe640 1 -- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 msgr2=0x7f6a64136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.192+0000 7f6a637fe640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 0x7f6a64136410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a540097d0 tx=0x7f6a54029fa0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=56c1dea5c8faad93 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.192+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 0x7f32d0136410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02a010 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a8b787892f6b02da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.192+0000 7f32d4eb9640 1 -- 172.21.15.134:0/940526564 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc02ae10 con 0x7f32d0136030 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: dump: 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.208 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.209 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d811f640 5 monclient: authenticate success, global_id 4216 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d811f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 1 -- 172.21.15.134:0/940526564 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc030340 con 0x7f32d0136030 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d4eb9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d811f640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f32d811f640 4 set_mon_vals no callback set 2024-09-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 1 -- 172.21.15.134:0/2570158357 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a5402ad00 con 0x7f6a64136030 2024-09-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: dump: 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 1 -- 172.21.15.134:0/2570158357 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5402ae80 con 0x7f6a64136030 2024-09-16T15:00:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 1 -- 172.21.15.134:0/2570158357 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a540307b0 con 0x7f6a64136030 2024-09-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a627fc640 10 monclient: dump: 2024-09-16T15:00:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.220 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.220 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.220 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.220 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.220 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a6a1ae640 5 monclient: authenticate success, global_id 4274 2024-09-16T15:00:51.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a6a1ae640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.193+0000 7f6a6a1ae640 4 set_mon_vals no callback set 2024-09-16T15:00:51.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d4eb9640 1 -- 172.21.15.134:0/940526564 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc030960 con 0x7f32d0136030 2024-09-16T15:00:51.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 10 monclient: shutdown 2024-09-16T15:00:51.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d4eb9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 msgr2=0x7f6a64136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 0x7f6a64136410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a540097d0 tx=0x7f6a54029fa0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d4eb9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d4eb9640 10 monclient: dump: 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 10 monclient: shutdown 2024-09-16T15:00:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 msgr2=0x7f32d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 --2- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 0x7f32d0136410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0097a0 tx=0x7f32cc02a010 comp rx=0 tx=0).stop 2024-09-16T15:00:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 shutdown_connections 2024-09-16T15:00:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 --2- 172.21.15.134:0/940526564 >> v2:172.21.15.134:3300/0 conn(0x7f32d009c400 0x7f32d0137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 --2- 172.21.15.134:0/940526564 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32d0136030 0x7f32d0136410 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 >> 172.21.15.134:0/940526564 conn(0x7f32d00a9ea0 msgr2=0x7f32d00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 shutdown_connections 2024-09-16T15:00:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64136c60 0x7f6a641370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 shutdown_connections 2024-09-16T15:00:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a6409bfe0 0x7f6a6409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/2570158357 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a64136030 0x7f6a64136410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 >> 172.21.15.134:0/2570158357 conn(0x7f6a640a9ea0 msgr2=0x7f6a640a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 shutdown_connections 2024-09-16T15:00:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/2570158357 wait complete. 2024-09-16T15:00:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.194+0000 7f32d811f640 1 -- 172.21.15.134:0/940526564 wait complete. 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f32d811f640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f32d811f640 10 monclient: monmap: 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.236 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.236 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.236 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.236 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.236 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.238 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f6a6a1ae640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f6a6a1ae640 10 monclient: monmap: 2024-09-16T15:00:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.241 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.241 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.241 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.241 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.241 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.242 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f32d811f640 1 Processor -- start 2024-09-16T15:00:51.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f32d811f640 1 -- start start 2024-09-16T15:00:51.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f32d811f640 10 monclient: init 2024-09-16T15:00:51.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.195+0000 7f6a6a1ae640 1 Processor -- start 2024-09-16T15:00:51.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 -- start start 2024-09-16T15:00:51.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient: init 2024-09-16T15:00:51.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient: _add_conns ranks=[2,1,5,0,7,4,3,8,6] 2024-09-16T15:00:51.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 0x7f32d01cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-b con 0x7f32d009bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d009c400 0x7f32d01cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-f con 0x7f32d009c400 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32d0136030 0x7f32d01201c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): picked mon.noname-h con 0x7f32d0136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 0x7f32d01cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00a8d70 con 0x7f32d009c400 2024-09-16T15:00:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 0x7f32d01cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58006/0 (socket says 172.21.15.134:58006) 2024-09-16T15:00:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 -- 172.21.15.134:0/2341031433 learned_addr learned my addr 172.21.15.134:0/2341031433 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00a8860 con 0x7f32d009bfe0 2024-09-16T15:00:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0123880 con 0x7f32d0136030 2024-09-16T15:00:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d811f640 10 monclient(hunting): authenticate will time out at 787.785385s 2024-09-16T15:00:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): get_auth_request con 0x7f32d009bfe0 auth_method 0 2024-09-16T15:00:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32d0136030 0x7f32d01201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient: _add_conns ranks=[3,5,4,1,7,8,6,2,0] 2024-09-16T15:00:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6409bfe0 0x7f6a640a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-g con 0x7f6a6409bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a64136030 0x7f6a64134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): get_auth_request con 0x7f32d0136030 auth_method 0 2024-09-16T15:00:51.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-h con 0x7f6a64136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 0x7f6a64135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): picked mon.noname-c con 0x7f6a64136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a641cb850 con 0x7f6a6409bfe0 2024-09-16T15:00:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a641227c0 con 0x7f6a64136c60 2024-09-16T15:00:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a7ce0 con 0x7f6a64136030 2024-09-16T15:00:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a6a1ae640 10 monclient(hunting): authenticate will time out at 787.785889s 2024-09-16T15:00:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6409bfe0 0x7f6a640a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d009c400 0x7f32d01cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 0x7f6a64135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): get_auth_request con 0x7f32d009c400 auth_method 0 2024-09-16T15:00:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d66bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d56ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient(hunting): handle_auth_done global_id 4232 payload 824 2024-09-16T15:00:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 -- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32d0136030 msgr2=0x7f32d01201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32d0136030 0x7f32d01201c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 -- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d009c400 msgr2=0x7f32d01cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32d009c400 0x7f32d01cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.196+0000 7f32d5ebb640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc009400 con 0x7f32d009bfe0 2024-09-16T15:00:51.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 0x7f6a64135340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33344/0 (socket says 172.21.15.134:33344) 2024-09-16T15:00:51.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 learned_addr learned my addr 172.21.15.134:0/1084812753 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): get_auth_request con 0x7f6a64136c60 auth_method 0 2024-09-16T15:00:51.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): get_auth_request con 0x7f6a6409bfe0 auth_method 0 2024-09-16T15:00:51.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a62ffd640 1 --2- 172.21.15.134:0/1084812753 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a64136030 0x7f6a64134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a62ffd640 10 monclient(hunting): get_auth_request con 0x7f6a64136030 auth_method 0 2024-09-16T15:00:51.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a62ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a62ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient(hunting): handle_auth_done global_id 4225 payload 824 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a64136030 msgr2=0x7f6a64134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 --2- 172.21.15.134:0/1084812753 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a64136030 0x7f6a64134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6409bfe0 msgr2=0x7f6a640a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 --2- 172.21.15.134:0/1084812753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6409bfe0 0x7f6a640a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 monclient: found mon.noname-c 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.197+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009400 con 0x7f6a64136c60 2024-09-16T15:00:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d5ebb640 1 -- 172.21.15.134:0/2341031433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 msgr2=0x7f32d01cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d5ebb640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 0x7f32d01cb980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32cc02a530 tx=0x7f32cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc003d30 con 0x7f32d009bfe0 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: dump: 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc045070 con 0x7f32d009bfe0 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 5 monclient: authenticate success, global_id 4232 2024-09-16T15:00:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 10 monclient: _renew_subs 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d0123b60 con 0x7f32d009bfe0 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc0304b0 con 0x7f32d009bfe0 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c7fff640 4 set_mon_vals no callback set 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32c67fc640 10 monclient: dump: 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 10 monclient: _renew_subs 2024-09-16T15:00:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 msgr2=0x7f6a64135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f6a63fff640 1 --2- 172.21.15.134:0/1084812753 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 0x7f6a64135340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007a70 tx=0x7f6a5c007f30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.199+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01cc400 con 0x7f32d009bfe0 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a5c017070 con 0x7f6a64136c60 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: dump: 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00f040 con 0x7f6a64136c60 2024-09-16T15:00:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a5c013cd0 con 0x7f6a64136c60 2024-09-16T15:00:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f32cc0038c0 con 0x7f32d009bfe0 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a4bfff640 10 monclient: dump: 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.285 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.200+0000 7f6a61ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:51.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 5 monclient: authenticate success, global_id 4225 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 10 monclient: _renew_subs 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640a8180 con 0x7f6a64136c60 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32d811f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 10 monclient: _renew_subs 2024-09-16T15:00:51.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:51.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64122200 con 0x7f6a64136c60 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32d0067a30 con 0x7f32d009bfe0 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6800/980547263 conn(0x7f32b40388b0 0x7f32b403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f32cc047070 con 0x7f32d009bfe0 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 --> v2:172.21.15.162:6801/1098590432 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32b4040e30 con 0x7f32b403e360 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32cc030610 con 0x7f32d009bfe0 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32c67fc640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32d56ba640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6800/980547263 conn(0x7f32b40388b0 0x7f32b403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.201+0000 7f32d56ba640 10 monclient: get_auth_request con 0x7f32b40388b0 auth_method 0 2024-09-16T15:00:51.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f6a6a1ae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:51.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 10 monclient: get_auth_request con 0x7f32b403e360 auth_method 0 2024-09-16T15:00:51.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a64061ae0 con 0x7f6a64136c60 2024-09-16T15:00:51.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 1 -- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 msgr2=0x7f32b40407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:00:51.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 1 -- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 msgr2=0x7f32b40407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:00:51.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:51.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.202+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6a5c00dc50 con 0x7f6a64136c60 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f6a5c0498b0 con 0x7f6a64136c60 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.134:6804/4141541690 conn(0x7f6a5003e3b0 0x7f6a500407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 --> v2:172.21.15.134:6804/4141541690 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a50040e80 con 0x7f6a5003e3b0 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a5c04bdc0 con 0x7f6a64136c60 2024-09-16T15:00:51.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a4bfff640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-16T15:00:51.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a637fe640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a637fe640 10 monclient: get_auth_request con 0x7f6a50038880 auth_method 0 2024-09-16T15:00:51.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a62ffd640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.134:6804/4141541690 conn(0x7f6a5003e3b0 0x7f6a500407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.203+0000 7f6a62ffd640 10 monclient: get_auth_request con 0x7f6a5003e3b0 auth_method 0 2024-09-16T15:00:51.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.204+0000 7f32d56ba640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6800/980547263 conn(0x7f32b40388b0 0x7f32b403ad70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32c00092a0 tx=0x7f32c0009230 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.205+0000 7f6a637fe640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a540097a0 tx=0x7f6a5402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.208+0000 7f6a62ffd640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.134:6804/4141541690 conn(0x7f6a5003e3b0 0x7f6a500407f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.208+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== osd.0 v2:172.21.15.134:6804/4141541690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6a50040e80 con 0x7f6a5003e3b0 2024-09-16T15:00:51.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.222+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 --> v2:172.21.15.134:6804/4141541690 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a6407a680 con 0x7f6a5003e3b0 2024-09-16T15:00:51.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a4bfff640 1 -- 172.21.15.134:0/1084812753 <== osd.0 v2:172.21.15.134:6804/4141541690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a5800c040 con 0x7f6a5003e3b0 2024-09-16T15:00:51.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 >> v2:172.21.15.134:6804/4141541690 conn(0x7f6a5003e3b0 msgr2=0x7f6a500407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.134:6804/4141541690 conn(0x7f6a5003e3b0 0x7f6a500407f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 msgr2=0x7f6a5003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/1084812753 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a540097a0 tx=0x7f6a5402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 10 monclient: shutdown 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 msgr2=0x7f6a64135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 --2- 172.21.15.134:0/1084812753 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6a64136c60 0x7f6a64135340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007a70 tx=0x7f6a5c007f30 comp rx=0 tx=0).stop 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a63fff640 1 -- 172.21.15.134:0/1084812753 reap_dead start 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 shutdown_connections 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.223+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 >> 172.21.15.134:0/1084812753 conn(0x7f6a640a9ea0 msgr2=0x7f6a641bee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.224+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 shutdown_connections 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.224+0000 7f6a6a1ae640 1 -- 172.21.15.134:0/1084812753 wait complete. 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7fc68a78a640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7fc68a78a640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7ff6c393e640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7ff6c393e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7ff6c393e640 10 monclient: monmap: 2024-09-16T15:00:51.297 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.229277+0000 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.229277+0000 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.298 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7fc68a78a640 10 monclient: monmap: 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.229270+0000 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.229270+0000 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.299 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.300 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.228+0000 7ff6c393e640 1 Processor -- start 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 -- start start 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient: init 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7fc68a78a640 1 Processor -- start 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7fc68a78a640 1 -- start start 2024-09-16T15:00:51.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7fc68a78a640 10 monclient: init 2024-09-16T15:00:51.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient: _add_conns ranks=[3,0,5,6,8,10,9,4,2,7,1] 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-g con 0x7ff6bc069e50 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff6bc0664f0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc13ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-h con 0x7ff6bc0668d0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc073250 con 0x7ff6bc0664f0 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc06a4a0 con 0x7ff6bc069e50 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc09ac50 con 0x7ff6bc0668d0 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c393e640 10 monclient(hunting): authenticate will time out at 787.818851s 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c0ed9640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff6bc0664f0 0x7ff6bc13b4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38922/0 (socket says 172.21.15.134:38922) 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.229+0000 7ff6c0ed9640 1 -- 172.21.15.134:0/1918635855 learned_addr learned my addr 172.21.15.134:0/1918635855 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc13ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): get_auth_request con 0x7ff6bc069e50 auth_method 0 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): get_auth_request con 0x7ff6bc0668d0 auth_method 0 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient: _add_conns ranks=[10,1,6,7,9,5,4,3,0,2,8] 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc68409c8d0 0x7fc684130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-f con 0x7fc68409c8d0 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc684130d90 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 0x7fc6841340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient(hunting): handle_auth_done global_id 4253 payload 274 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 -- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 msgr2=0x7ff6bc065fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc065fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-d con 0x7fc684131190 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 -- 172.21.15.134:0/1918635855 >> v1:172.21.15.134:6789/0 conn(0x7ff6bc0664f0 legacy=0x7ff6bc13b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 monclient: found mon.noname-h 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:00:51.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c1edb640 1 -- 172.21.15.134:0/1918635855 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc09bc40 con 0x7ff6bc0668d0 2024-09-16T15:00:51.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7ff6c16da640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc684073250 con 0x7fc684130d90 2024-09-16T15:00:51.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68409aa70 con 0x7fc68409c8d0 2024-09-16T15:00:51.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc68409ccb0 con 0x7fc684131190 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc68a78a640 10 monclient(hunting): authenticate will time out at 787.819493s 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc68409c8d0 0x7fc684130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc6837fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fc684130d90 0x7fc6841339a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:56526/0 (socket says 172.21.15.134:56526) 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc6837fe640 1 -- 172.21.15.134:0/1572115705 learned_addr learned my addr 172.21.15.134:0/1572115705 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 0x7fc6841340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): get_auth_request con 0x7fc68409c8d0 auth_method 0 2024-09-16T15:00:51.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): get_auth_request con 0x7fc684131190 auth_method 0 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient(hunting): handle_auth_done global_id 4276 payload 274 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 -- 172.21.15.134:0/1572115705 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 msgr2=0x7fc6841340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 0x7fc6841340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 -- 172.21.15.134:0/1572115705 >> v1:172.21.15.162:6789/0 conn(0x7fc684130d90 legacy=0x7fc6841339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 monclient: found mon.noname-f 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc683fff640 1 -- 172.21.15.134:0/1572115705 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68409a5f0 con 0x7fc68409c8d0 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.230+0000 7fc688d27640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 0x7fc6841340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6c1edb640 1 -- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 msgr2=0x7ff6bc13ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc13ebf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009dc0 tx=0x7ff6b402a4d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=320245b71e55827e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc683fff640 1 -- 172.21.15.134:0/1572115705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc68409c8d0 msgr2=0x7fc684130850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc683fff640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc68409c8d0 0x7fc684130850 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc67400f9b0 tx=0x7fc674032710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925ad5d7c9b1ba1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 1 -- 172.21.15.134:0/1918635855 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b4007bc0 con 0x7ff6bc0668d0 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: dump: 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 1 -- 172.21.15.134:0/1918635855 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b402d050 con 0x7ff6bc0668d0 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 1 -- 172.21.15.134:0/1918635855 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b4007d80 con 0x7ff6bc0668d0 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6b3fff640 10 monclient: dump: 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6c393e640 5 monclient: authenticate success, global_id 4253 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6c393e640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7ff6c393e640 4 set_mon_vals no callback set 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc682ffd640 1 -- 172.21.15.134:0/1572115705 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc674037b90 con 0x7fc68409c8d0 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc682ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.233+0000 7fc682ffd640 10 monclient: dump: 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.320 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 1 -- 172.21.15.134:0/1572115705 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc674035050 con 0x7fc68409c8d0 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 5 monclient: authenticate success, global_id 4276 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 1 -- 172.21.15.134:0/1572115705 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc674037d10 con 0x7fc68409c8d0 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc682ffd640 10 monclient: dump: 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.322 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 4 set_mon_vals no callback set 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 10 monclient: shutdown 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 -- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 msgr2=0x7ff6bc13ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc13ebf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009dc0 tx=0x7ff6b402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 -- 172.21.15.134:0/1918635855 shutdown_connections 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc13ebf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1918635855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 -- 172.21.15.134:0/1918635855 >> 172.21.15.134:0/1918635855 conn(0x7ff6bc0a6320 msgr2=0x7ff6bc0a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 -- 172.21.15.134:0/1918635855 shutdown_connections 2024-09-16T15:00:51.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7ff6c393e640 1 -- 172.21.15.134:0/1918635855 wait complete. 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 10 monclient: shutdown 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 1 -- 172.21.15.134:0/1572115705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc68409c8d0 msgr2=0x7fc684130850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.234+0000 7fc68a78a640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc68409c8d0 0x7fc684130850 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc67400f9b0 tx=0x7fc674032710 comp rx=0 tx=0).stop 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 -- 172.21.15.134:0/1572115705 shutdown_connections 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc684131190 0x7fc6841340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 --2- 172.21.15.134:0/1572115705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc68409c8d0 0x7fc684130850 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 -- 172.21.15.134:0/1572115705 >> 172.21.15.134:0/1572115705 conn(0x7fc6840a6320 msgr2=0x7fc6840a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 -- 172.21.15.134:0/1572115705 shutdown_connections 2024-09-16T15:00:51.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 1 -- 172.21.15.134:0/1572115705 wait complete. 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7ff6c393e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7ff6c393e640 10 monclient: monmap: 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.327 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7ff6c393e640 1 Processor -- start 2024-09-16T15:00:51.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7ff6c393e640 1 -- start start 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7fc68a78a640 10 monclient: monmap: 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.235+0000 7ff6c393e640 10 monclient: init 2024-09-16T15:00:51.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7fc68a78a640 1 Processor -- start 2024-09-16T15:00:51.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient: _add_conns ranks=[2,4,3,5,6,0,8,7,1] 2024-09-16T15:00:51.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 0x7ff6bc0903b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-b con 0x7ff6bc0664f0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7fc68a78a640 1 -- start start 2024-09-16T15:00:51.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7fc68a78a640 10 monclient: init 2024-09-16T15:00:51.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc0908f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7f487bac9640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7f487bac9640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7f487bac9640 10 monclient: monmap: 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.237585+0000 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.237585+0000 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc0908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc0908f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33358/0 (socket says 172.21.15.134:33358) 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c16da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 0x7ff6bc0903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 1 -- 172.21.15.134:0/1009282031 learned_addr learned my addr 172.21.15.134:0/1009282031 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7f806d15e640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7f806d15e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-c con 0x7ff6bc0668d0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7f806d15e640 10 monclient: monmap: 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.237914+0000 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.237914+0000 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.337 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc090e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): picked mon.noname-g con 0x7ff6bc069e50 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc092b30 con 0x7ff6bc0664f0 2024-09-16T15:00:51.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.339 INFO:teuthology.orchestra.run.smithi134.stdout:47244640259 2024-09-16T15:00:51.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:00:51.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc1cc690 con 0x7ff6bc069e50 2024-09-16T15:00:51.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc093b00 con 0x7ff6bc0668d0 2024-09-16T15:00:51.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c393e640 10 monclient(hunting): authenticate will time out at 787.825904s 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient: _add_conns ranks=[6,1,4,0,5,3,2,8,7] 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 10 monclient(hunting): get_auth_request con 0x7ff6bc0668d0 auth_method 0 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c0ed9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c16da640 10 monclient(hunting): get_auth_request con 0x7ff6bc0664f0 auth_method 0 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c16da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c16da640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.236+0000 7ff6c16da640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c0ed9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c0ed9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc090e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c0ed9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 0x7fc6841c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7f806d15e640 1 Processor -- start 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c1edb640 10 monclient(hunting): get_auth_request con 0x7ff6bc069e50 auth_method 0 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c1edb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c1edb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c1edb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient(hunting): handle_auth_done global_id 4241 payload 824 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 -- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6bc0668d0 msgr2=0x7ff6bc0908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-d con 0x7fc68409c8d0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc684130d90 0x7fc6841c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-f con 0x7fc684130d90 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc684131190 0x7fc68412c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): picked mon.noname-c con 0x7fc684131190 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc684098e40 con 0x7fc684130d90 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840a3bc0 con 0x7fc684131190 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6bc0668d0 0x7ff6bc0908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 -- 172.21.15.134:0/1009282031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 msgr2=0x7ff6bc090e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6bc069e50 0x7ff6bc090e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840a0d20 con 0x7fc68409c8d0 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc68a78a640 10 monclient(hunting): authenticate will time out at 787.826574s 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7f487bac9640 1 Processor -- start 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7ff6c16da640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009400 con 0x7ff6bc0664f0 2024-09-16T15:00:51.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7f806d15e640 1 -- start start 2024-09-16T15:00:51.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc684131190 0x7fc68412c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc684131190 0x7fc68412c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33362/0 (socket says 172.21.15.134:33362) 2024-09-16T15:00:51.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 1 -- 172.21.15.134:0/3406754029 learned_addr learned my addr 172.21.15.134:0/3406754029 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 10 monclient(hunting): get_auth_request con 0x7fc684131190 auth_method 0 2024-09-16T15:00:51.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc688d27640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc683fff640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 0x7fc6841c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc683fff640 10 monclient(hunting): get_auth_request con 0x7fc68409c8d0 auth_method 0 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc683fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7fc683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.237+0000 7f806d15e640 10 monclient: init 2024-09-16T15:00:51.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 1 -- start start 2024-09-16T15:00:51.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient: init 2024-09-16T15:00:51.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc688d27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc684130d90 0x7fc6841c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 10 monclient(hunting): get_auth_request con 0x7fc684130d90 auth_method 0 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient(hunting): handle_auth_done global_id 4227 payload 824 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 -- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc684131190 msgr2=0x7fc68412c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc684131190 0x7fc68412c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 -- 172.21.15.134:0/3406754029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc684130d90 msgr2=0x7fc6841c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc684130d90 0x7fc6841c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 monclient: found mon.noname-d 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc683fff640 1 -- 172.21.15.134:0/3406754029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c00ba90 con 0x7fc68409c8d0 2024-09-16T15:00:51.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7fc6837fe640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc684130d90 0x7fc6841c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:51.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient: _add_conns ranks=[5,1,3,10,0,9,7,8,4,6,2] 2024-09-16T15:00:51.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-h con 0x7f8068066060 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8068071920 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f806813b5c0 0x7f80680a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-g con 0x7f806813b5c0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8068073250 con 0x7f8068071920 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806813c020 con 0x7f806813b5c0 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8068066440 con 0x7f8068066060 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806d15e640 10 monclient(hunting): authenticate will time out at 787.827654s 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:59842/0 (socket says 172.21.15.134:59842) 2024-09-16T15:00:51.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 1 -- 172.21.15.134:0/1525791320 learned_addr learned my addr 172.21.15.134:0/1525791320 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): get_auth_request con 0x7f8068066060 auth_method 0 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f806813b5c0 0x7f80680a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient: _add_conns ranks=[1,9,6,0,10,5,8,4,3,2,7] 2024-09-16T15:00:51.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4874136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:51.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): get_auth_request con 0x7f806813b5c0 auth_method 0 2024-09-16T15:00:51.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 0x7f487409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-a con 0x7f487409bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 0x7f487413e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-d con 0x7f487413bf80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f487bac9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4874073250 con 0x7f4874136030 2024-09-16T15:00:51.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 10 monclient(hunting): handle_auth_done global_id 4262 payload 274 2024-09-16T15:00:51.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 1 -- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f806813b5c0 msgr2=0x7f80680a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.238+0000 7f8066d9d640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f806813b5c0 0x7f80680a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 1 -- 172.21.15.134:0/1525791320 >> v1:172.21.15.162:6789/0 conn(0x7f8068071920 legacy=0x7f80680a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 1 monclient: found mon.noname-h 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:00:51.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f8066d9d640 1 -- 172.21.15.134:0/1525791320 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806806e600 con 0x7f8068066060 2024-09-16T15:00:51.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487409b600 con 0x7f487409bfe0 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48740a5740 con 0x7f487413bf80 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487bac9640 10 monclient(hunting): authenticate will time out at 787.828051s 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 0x7f487409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 0x7f487409c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51824/0 (socket says 172.21.15.134:51824) 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 1 -- 172.21.15.134:0/4277655372 learned_addr learned my addr 172.21.15.134:0/4277655372 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): get_auth_request con 0x7f487409bfe0 auth_method 0 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 --2- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 0x7f487413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): get_auth_request con 0x7f487413bf80 auth_method 0 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient(hunting): handle_auth_done global_id 4236 payload 274 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 -- 172.21.15.134:0/4277655372 >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 msgr2=0x7f487409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 --2- 172.21.15.134:0/4277655372 >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 0x7f487409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 -- 172.21.15.134:0/4277655372 >> v1:172.21.15.162:6789/0 conn(0x7f4874136030 legacy=0x7f4874136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 monclient: found mon.noname-d 2024-09-16T15:00:51.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7f487a066640 1 -- 172.21.15.134:0/4277655372 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740a52c0 con 0x7f487413bf80 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.239+0000 7ff6c16da640 1 -- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 msgr2=0x7ff6bc0903b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c16da640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 0x7ff6bc0903b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc093cd0 tx=0x7ff6b800b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b8019070 con 0x7ff6bc0664f0 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: dump: 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.364 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.365 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc683fff640 1 -- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 msgr2=0x7fc6841c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc683fff640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 0x7fc6841c7860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc674039000 tx=0x7fc6740046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc674003a80 con 0x7fc68409c8d0 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: dump: 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.367 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b801b990 con 0x7ff6bc0664f0 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 5 monclient: authenticate success, global_id 4241 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 10 monclient: _renew_subs 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b4009a70 con 0x7ff6bc0664f0 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6b801bb00 con 0x7ff6bc0664f0 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b1ffb640 10 monclient: dump: 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6b37fe640 4 set_mon_vals no callback set 2024-09-16T15:00:51.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 10 monclient: _renew_subs 2024-09-16T15:00:51.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc09ece0 con 0x7ff6bc0664f0 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc68a78a640 5 monclient: authenticate success, global_id 4227 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc68a78a640 10 monclient: _renew_subs 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc68a78a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc68a78a640 1 -- 172.21.15.134:0/3406754029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67400f610 con 0x7fc68409c8d0 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc674003c40 con 0x7fc68409c8d0 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc674003db0 con 0x7fc68409c8d0 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc68a78a640 10 monclient: _renew_subs 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc68a78a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc68a78a640 1 -- 172.21.15.134:0/3406754029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68412fa60 con 0x7fc68409c8d0 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc680ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.240+0000 7fc6827fc640 4 set_mon_vals no callback set 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc680ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc680ff9640 10 monclient: dump: 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.375 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7ff6c393e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6bc071bf0 con 0x7ff6bc0664f0 2024-09-16T15:00:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc66a7fc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.241+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc65c000f10 con 0x7fc68409c8d0 2024-09-16T15:00:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8066d9d640 1 -- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 msgr2=0x7f80680713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8066d9d640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f80580098d0 tx=0x7f805802a100 comp rx=0 tx=0).ready entity=mon.5 client_cookie=107e19984b8cc227 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f487a066640 1 -- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 msgr2=0x7f487413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f487a066640 1 --2- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 0x7f487413e370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02a4e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2ff85fadf6c266cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff6b801bc80 con 0x7ff6bc0664f0 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 1 -- 172.21.15.134:0/4277655372 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f486c007bc0 con 0x7f487413bf80 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: dump: 2024-09-16T15:00:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc674003610 con 0x7fc68409c8d0 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 1 -- 172.21.15.134:0/1525791320 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f805802ae20 con 0x7f8068066060 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: dump: 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 1 -- 172.21.15.134:0/1525791320 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80580307b0 con 0x7f8068066060 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 1 -- 172.21.15.134:0/1525791320 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8058030d90 con 0x7f8068066060 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f806d15e640 5 monclient: authenticate success, global_id 4262 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f806d15e640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f806d15e640 4 set_mon_vals no callback set 2024-09-16T15:00:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 1 -- 172.21.15.134:0/4277655372 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c02d050 con 0x7f487413bf80 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 1 -- 172.21.15.134:0/4277655372 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f486c007d20 con 0x7f487413bf80 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f4878863640 10 monclient: dump: 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.382 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f487bac9640 5 monclient: authenticate success, global_id 4236 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f487bac9640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f487bac9640 4 set_mon_vals no callback set 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7f8065d9b640 10 monclient: dump: 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.384 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7ff6b1ffb640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6a0038880 0x7ff6a003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7ff6b8051de0 con 0x7ff6bc0664f0 2024-09-16T15:00:51.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.242+0000 7fc680ff9640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6800/980547263 conn(0x7fc670038880 0x7fc67003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fc674073110 con 0x7fc68409c8d0 2024-09-16T15:00:51.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6b1ffb640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.134:6800/1264438958 conn(0x7ff6a003e330 0x7ff6a0040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 --> v2:172.21.15.134:6800/1264438958 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6a0040e00 con 0x7ff6a003e330 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6b8052080 con 0x7ff6bc0664f0 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6b1ffb640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc680ff9640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc67003e330 0x7fc670040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 --> v2:172.21.15.162:6805/3922313654 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc670040e00 con 0x7fc67003e330 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6740733b0 con 0x7fc68409c8d0 2024-09-16T15:00:51.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc6837fe640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6800/980547263 conn(0x7fc670038880 0x7fc67003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6c0ed9640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6a0038880 0x7ff6a003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc680ff9640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-16T15:00:51.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.134:6800/1264438958 conn(0x7ff6a003e330 0x7ff6a0040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 10 monclient: shutdown 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 -- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 msgr2=0x7f80680713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f80580098d0 tx=0x7f805802a100 comp rx=0 tx=0).stop 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6c0ed9640 10 monclient: get_auth_request con 0x7ff6a0038880 auth_method 0 2024-09-16T15:00:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc6837fe640 10 monclient: get_auth_request con 0x7fc670038880 auth_method 0 2024-09-16T15:00:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 -- 172.21.15.134:0/1525791320 shutdown_connections 2024-09-16T15:00:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7ff6c1edb640 10 monclient: get_auth_request con 0x7ff6a003e330 auth_method 0 2024-09-16T15:00:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f806813b5c0 0x7f80680a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 --2- 172.21.15.134:0/1525791320 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8068066060 0x7f80680713e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 -- 172.21.15.134:0/1525791320 >> 172.21.15.134:0/1525791320 conn(0x7f80680a6260 msgr2=0x7f80680a2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 -- 172.21.15.134:0/1525791320 shutdown_connections 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f806d15e640 1 -- 172.21.15.134:0/1525791320 wait complete. 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc688d27640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc67003e330 0x7fc670040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7fc688d27640 10 monclient: get_auth_request con 0x7fc67003e330 auth_method 0 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 10 monclient: shutdown 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 -- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 msgr2=0x7f487413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 --2- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 0x7f487413e370 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02a4e0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 -- 172.21.15.134:0/4277655372 shutdown_connections 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 --2- 172.21.15.134:0/4277655372 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f487413bf80 0x7f487413e370 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 --2- 172.21.15.134:0/4277655372 >> v2:172.21.15.134:3300/0 conn(0x7f487409bfe0 0x7f487409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.243+0000 7f487bac9640 1 -- 172.21.15.134:0/4277655372 >> 172.21.15.134:0/4277655372 conn(0x7f48740a9ea0 msgr2=0x7f48740a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f487bac9640 1 -- 172.21.15.134:0/4277655372 shutdown_connections 2024-09-16T15:00:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f487bac9640 1 -- 172.21.15.134:0/4277655372 wait complete. 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f806d15e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f806d15e640 10 monclient: monmap: 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f487bac9640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f487bac9640 10 monclient: monmap: 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f806d15e640 1 Processor -- start 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f806d15e640 1 -- start start 2024-09-16T15:00:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f806d15e640 10 monclient: init 2024-09-16T15:00:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.244+0000 7f487bac9640 1 Processor -- start 2024-09-16T15:00:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 1 -- start start 2024-09-16T15:00:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 10 monclient: init 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient: _add_conns ranks=[8,3,0,7,4,1,6,2,5] 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 0x7f80681cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-e con 0x7f8068066060 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8068071920 0x7f8068137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-g con 0x7f8068071920 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f806813b5c0 0x7f80681375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): picked mon.noname-a con 0x7f806813b5c0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 0x7f80681cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806809eb60 con 0x7f806813b5c0 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80681cfe00 con 0x7f8068071920 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 0x7f80681cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41716/0 (socket says 172.21.15.134:41716) 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 -- 172.21.15.134:0/1324793857 learned_addr learned my addr 172.21.15.134:0/1324793857 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f80680a1f90 con 0x7f8068066060 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806d15e640 10 monclient(hunting): authenticate will time out at 787.834449s 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): get_auth_request con 0x7f8068066060 auth_method 0 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f806813b5c0 0x7f80681375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): get_auth_request con 0x7f806813b5c0 auth_method 0 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8068071920 0x7f8068137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): get_auth_request con 0x7f8068071920 auth_method 0 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient(hunting): handle_auth_done global_id 4283 payload 824 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 -- 172.21.15.134:0/1324793857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8068071920 msgr2=0x7f8068137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8068071920 0x7f8068137080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 -- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f806813b5c0 msgr2=0x7f80681375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f806813b5c0 0x7f80681375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 monclient: found mon.noname-e 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f8066d9d640 1 -- 172.21.15.134:0/1324793857 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580094d0 con 0x7f8068066060 2024-09-16T15:00:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f806759e640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f806813b5c0 0x7f80681375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7fc6837fe640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6800/980547263 conn(0x7fc670038880 0x7fc67003ad40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6840a1f10 tx=0x7fc678009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 10 monclient: _add_conns ranks=[7,1,3,0,6,5,4,8,2] 2024-09-16T15:00:51.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f487409bfe0 0x7f48741cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7ff6c0ed9640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6a0038880 0x7ff6a003ad40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009370 tx=0x7ff6ac009300 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-i con 0x7f487409bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.245+0000 7f487bac9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-f con 0x7f4874136030 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f487413bf80 0x7f4874138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): picked mon.noname-g con 0x7f487413bf80 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487409fa70 con 0x7f4874136030 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48740a7f00 con 0x7f487413bf80 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f48740a79b0 con 0x7f487409bfe0 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487bac9640 10 monclient(hunting): authenticate will time out at 787.835081s 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f487409bfe0 0x7f48741cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:55680/0 (socket says 172.21.15.134:55680) 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 -- 172.21.15.134:0/2861648137 learned_addr learned my addr 172.21.15.134:0/2861648137 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): get_auth_request con 0x7f4874136030 auth_method 0 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): get_auth_request con 0x7f487409bfe0 auth_method 0 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f487413bf80 0x7f4874138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): get_auth_request con 0x7f487413bf80 auth_method 0 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f487a066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient(hunting): handle_auth_done global_id 4294 payload 824 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 -- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f487409bfe0 msgr2=0x7f48741cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f487409bfe0 0x7f48741cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 -- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f487413bf80 msgr2=0x7f4874138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f487413bf80 0x7f4874138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 monclient: found mon.noname-f 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.246+0000 7f4879064640 1 -- 172.21.15.134:0/2861648137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009a70 con 0x7f4874136030 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.247+0000 7ff6c1edb640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.134:6800/1264438958 conn(0x7ff6a003e330 0x7ff6a0040770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f8066d9d640 1 -- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 msgr2=0x7f80681cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f8066d9d640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 0x7f80681cf8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80580090b0 tx=0x7f80580046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7fc688d27640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc67003e330 0x7fc670040770 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f80580044c0 con 0x7f8068066060 2024-09-16T15:00:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: dump: 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.413 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8058030d40 con 0x7f8068066060 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f805803d440 con 0x7f8068066060 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:51.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f804f7fe640 10 monclient: dump: 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.415 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806559a640 4 set_mon_vals no callback set 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 5 monclient: authenticate success, global_id 4283 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 10 monclient: _renew_subs 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80680a1bf0 con 0x7f8068066060 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 10 monclient: _renew_subs 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806809f9a0 con 0x7f8068066060 2024-09-16T15:00:51.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f4879064640 1 -- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 msgr2=0x7f4874133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7f4879064640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4864009fd0 tx=0x7f486400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== osd.2 v2:172.21.15.134:6800/1264438958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff6a0040e00 con 0x7ff6a003e330 2024-09-16T15:00:51.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.248+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== osd.3 v2:172.21.15.162:6805/3922313654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc670040e00 con 0x7fc67003e330 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4864013470 con 0x7f4874136030 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: dump: 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864013610 con 0x7f4874136030 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4864013bf0 con 0x7f4874136030 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486a7fc640 10 monclient: dump: 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f486bfff640 4 set_mon_vals no callback set 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 5 monclient: authenticate success, global_id 4294 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 10 monclient: _renew_subs 2024-09-16T15:00:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f806d15e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8068061ae0 con 0x7f8068066060 2024-09-16T15:00:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48741cc0e0 con 0x7f4874136030 2024-09-16T15:00:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 10 monclient: _renew_subs 2024-09-16T15:00:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8058039040 con 0x7f8068066060 2024-09-16T15:00:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487411fed0 con 0x7f4874136030 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.249+0000 7f804f7fe640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6800/980547263 conn(0x7f80540388b0 0x7f805403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f805806a450 con 0x7f8068066060 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f804f7fe640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6809/518841521 conn(0x7f805403e3e0 0x7f8054040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 --> v2:172.21.15.162:6809/518841521 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8054040eb0 con 0x7f805403e3e0 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f805806c9e0 con 0x7f8068066060 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f804f7fe640 10 monclient: handle_get_version_reply finishing 1 version 17 2024-09-16T15:00:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f487bac9640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f806759e640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6809/518841521 conn(0x7f805403e3e0 0x7f8054040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4874061ae0 con 0x7f4874136030 2024-09-16T15:00:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f806759e640 10 monclient: get_auth_request con 0x7f805403e3e0 auth_method 0 2024-09-16T15:00:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f806659c640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6800/980547263 conn(0x7f80540388b0 0x7f805403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f806659c640 10 monclient: get_auth_request con 0x7f80540388b0 auth_method 0 2024-09-16T15:00:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.250+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4864013780 con 0x7f4874136030 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f4879064640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4864009fd0 tx=0x7f486400ed40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f486404c8c0 mon_get_version_reply(handle=1 version=17) v2 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f486a7fc640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.162:6800/980547263 conn(0x7f4858038880 0x7f485803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f486404a330 con 0x7f4874136030 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f486a7fc640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.134:6808/443532050 conn(0x7f485803e330 0x7f4858040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 --> v2:172.21.15.134:6808/443532050 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4858040e00 con 0x7f485803e330 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f4879865640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.162:6800/980547263 conn(0x7f4858038880 0x7f485803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f4879865640 10 monclient: get_auth_request con 0x7f4858038880 auth_method 0 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f487a066640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.134:6808/443532050 conn(0x7f485803e330 0x7f4858040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.251+0000 7f487a066640 10 monclient: get_auth_request con 0x7f485803e330 auth_method 0 2024-09-16T15:00:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.252+0000 7f806659c640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6800/980547263 conn(0x7f80540388b0 0x7f805403ad70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f806809d450 tx=0x7f805c007e50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.253+0000 7f4879865640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.162:6800/980547263 conn(0x7f4858038880 0x7f485803ad40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48700017f0 tx=0x7f4870009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.254+0000 7f806759e640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6809/518841521 conn(0x7f805403e3e0 0x7f8054040820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.255+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== osd.5 v2:172.21.15.162:6809/518841521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8054040eb0 con 0x7f805403e3e0 2024-09-16T15:00:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.256+0000 7f487a066640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.134:6808/443532050 conn(0x7f485803e330 0x7f4858040770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.256+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== osd.1 v2:172.21.15.134:6808/443532050 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4858040e00 con 0x7f485803e330 2024-09-16T15:00:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.263+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 --> v2:172.21.15.162:6805/3922313654 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc65c002b50 con 0x7fc67003e330 2024-09-16T15:00:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.263+0000 7fc680ff9640 1 -- 172.21.15.134:0/3406754029 <== osd.3 v2:172.21.15.162:6805/3922313654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc65c002b50 con 0x7fc67003e330 2024-09-16T15:00:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc67003e330 msgr2=0x7fc670040770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc67003e330 0x7fc670040770 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6800/980547263 conn(0x7fc670038880 msgr2=0x7fc67003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 --2- 172.21.15.134:0/3406754029 >> v2:172.21.15.162:6800/980547263 conn(0x7fc670038880 0x7fc67003ad40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6840a1f10 tx=0x7fc678009040 comp rx=0 tx=0).stop 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 10 monclient: shutdown 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 msgr2=0x7fc6841c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc66a7fc640 1 --2- 172.21.15.134:0/3406754029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc68409c8d0 0x7fc6841c7860 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc674039000 tx=0x7fc6740046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.264+0000 7fc688d27640 1 -- 172.21.15.134:0/3406754029 reap_dead start 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.266+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 shutdown_connections 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.266+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 >> 172.21.15.134:0/3406754029 conn(0x7fc6840a6320 msgr2=0x7fc68409f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.266+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 shutdown_connections 2024-09-16T15:00:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.266+0000 7fc66a7fc640 1 -- 172.21.15.134:0/3406754029 wait complete. 2024-09-16T15:00:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.269+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 --> v2:172.21.15.134:6800/1264438958 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6bc061ae0 con 0x7ff6a003e330 2024-09-16T15:00:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.269+0000 7ff6b1ffb640 1 -- 172.21.15.134:0/1009282031 <== osd.2 v2:172.21.15.134:6800/1264438958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff6b4003c10 con 0x7ff6a003e330 2024-09-16T15:00:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.269+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 >> v2:172.21.15.134:6800/1264438958 conn(0x7ff6a003e330 msgr2=0x7ff6a0040770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.269+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.134:6800/1264438958 conn(0x7ff6a003e330 0x7ff6a0040770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6a0038880 msgr2=0x7ff6a003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1009282031 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6a0038880 0x7ff6a003ad40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009370 tx=0x7ff6ac009300 comp rx=0 tx=0).stop 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 10 monclient: shutdown 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 msgr2=0x7ff6bc0903b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 --2- 172.21.15.134:0/1009282031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6bc0664f0 0x7ff6bc0903b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc093cd0 tx=0x7ff6b800b540 comp rx=0 tx=0).stop 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c1edb640 1 -- 172.21.15.134:0/1009282031 reap_dead start 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 shutdown_connections 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 >> 172.21.15.134:0/1009282031 conn(0x7ff6bc0a6320 msgr2=0x7ff6bc1bb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 shutdown_connections 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.270+0000 7ff6c393e640 1 -- 172.21.15.134:0/1009282031 wait complete. 2024-09-16T15:00:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.271+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 --> v2:172.21.15.162:6809/518841521 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f806807a680 con 0x7f805403e3e0 2024-09-16T15:00:51.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f804f7fe640 1 -- 172.21.15.134:0/1324793857 <== osd.5 v2:172.21.15.162:6809/518841521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f806000f040 con 0x7f805403e3e0 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6809/518841521 conn(0x7f805403e3e0 msgr2=0x7f8054040820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6809/518841521 conn(0x7f805403e3e0 0x7f8054040820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6800/980547263 conn(0x7f80540388b0 msgr2=0x7f805403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 --2- 172.21.15.134:0/1324793857 >> v2:172.21.15.162:6800/980547263 conn(0x7f80540388b0 0x7f805403ad70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f806809d450 tx=0x7f805c007e50 comp rx=0 tx=0).stop 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 10 monclient: shutdown 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 msgr2=0x7f80681cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 --2- 172.21.15.134:0/1324793857 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8068066060 0x7f80681cf8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f80580090b0 tx=0x7f80580046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806759e640 1 -- 172.21.15.134:0/1324793857 reap_dead start 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 shutdown_connections 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 >> 172.21.15.134:0/1324793857 conn(0x7f80680a6260 msgr2=0x7f80681393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 shutdown_connections 2024-09-16T15:00:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.272+0000 7f806d15e640 1 -- 172.21.15.134:0/1324793857 wait complete. 2024-09-16T15:00:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.277+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 --> v2:172.21.15.134:6808/443532050 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f487407a680 con 0x7f485803e330 2024-09-16T15:00:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.277+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== osd.1 v2:172.21.15.134:6808/443532050 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f486c003c90 con 0x7f485803e330 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 >> v2:172.21.15.134:6808/443532050 conn(0x7f485803e330 msgr2=0x7f4858040770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.134:6808/443532050 conn(0x7f485803e330 0x7f4858040770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 >> v2:172.21.15.162:6800/980547263 conn(0x7f4858038880 msgr2=0x7f485803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 --2- 172.21.15.134:0/2861648137 >> v2:172.21.15.162:6800/980547263 conn(0x7f4858038880 0x7f485803ad40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48700017f0 tx=0x7f4870009040 comp rx=0 tx=0).stop 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 10 monclient: shutdown 2024-09-16T15:00:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 20 monclient: shutdown canceling and discarding version request 1 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 msgr2=0x7f4874133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 --2- 172.21.15.134:0/2861648137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4874136030 0x7f4874133dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4864009fd0 tx=0x7f486400ed40 comp rx=0 tx=0).stop 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487a066640 1 -- 172.21.15.134:0/2861648137 reap_dead start 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 shutdown_connections 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 >> 172.21.15.134:0/2861648137 conn(0x7f48740a9ea0 msgr2=0x7f4874134de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f486a7fc640 1 -- 172.21.15.134:0/2861648137 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (unknown 0 0 0) 0x7f486404c8c0 con 0x7f4874136030 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f486a7fc640 10 monclient: discarding stray monitor message mon_get_version_reply(handle=1 version=17) v2 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 shutdown_connections 2024-09-16T15:00:51.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.278+0000 7f487bac9640 1 -- 172.21.15.134:0/2861648137 wait complete. 2024-09-16T15:00:51.441 INFO:teuthology.orchestra.run.smithi134.stdout:55834574854 2024-09-16T15:00:51.441 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:00:51.442 INFO:teuthology.orchestra.run.smithi134.stdout:55834574853 2024-09-16T15:00:51.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:00:51.443 INFO:teuthology.orchestra.run.smithi134.stdout:55834574852 2024-09-16T15:00:51.443 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:00:51.444 INFO:teuthology.orchestra.run.smithi134.stdout:55834574853 2024-09-16T15:00:51.445 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:00:51.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.403+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.403+0000 7f32d66bc640 10 monclient: get_auth_request con 0x7f32b403e360 auth_method 0 2024-09-16T15:00:51.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.408+0000 7f32d66bc640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.409+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== osd.4 v2:172.21.15.162:6801/1098590432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f32b4040e30 con 0x7f32b403e360 2024-09-16T15:00:51.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.422+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 --> v2:172.21.15.162:6801/1098590432 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32d007a680 con 0x7f32b403e360 2024-09-16T15:00:51.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.422+0000 7f32c67fc640 1 -- 172.21.15.134:0/2341031433 <== osd.4 v2:172.21.15.162:6801/1098590432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32c8011040 con 0x7f32b403e360 2024-09-16T15:00:51.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 msgr2=0x7f32b40407a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6801/1098590432 conn(0x7f32b403e360 0x7f32b40407a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6800/980547263 conn(0x7f32b40388b0 msgr2=0x7f32b403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 --2- 172.21.15.134:0/2341031433 >> v2:172.21.15.162:6800/980547263 conn(0x7f32b40388b0 0x7f32b403ad70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f32c00092a0 tx=0x7f32c0009230 comp rx=0 tx=0).stop 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 10 monclient: shutdown 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 msgr2=0x7f32d01cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 --2- 172.21.15.134:0/2341031433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32d009bfe0 0x7f32d01cb980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32cc02a530 tx=0x7f32cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d66bc640 1 -- 172.21.15.134:0/2341031433 reap_dead start 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 shutdown_connections 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 >> 172.21.15.134:0/2341031433 conn(0x7f32d00a9ea0 msgr2=0x7f32d00a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 shutdown_connections 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.423+0000 7f32d811f640 1 -- 172.21.15.134:0/2341031433 wait complete. 2024-09-16T15:00:51.448 INFO:teuthology.orchestra.run.smithi134.stdout:47244640259 2024-09-16T15:00:51.448 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.591+0000 7f6120490640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.591+0000 7f6120490640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.591+0000 7f6120490640 10 monclient: monmap: 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.592721+0000 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.592721+0000 2024-09-16T15:00:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.594 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.592+0000 7f6120490640 1 Processor -- start 2024-09-16T15:00:51.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.592+0000 7f6120490640 1 -- start start 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.592+0000 7f6120490640 10 monclient: init 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient: _add_conns ranks=[3,4,9,2,7,5,6,8,1,0,10] 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f611809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): picked mon.noname-g con 0x7f6118136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): picked mon.noname-c con 0x7f6118137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6118135dc0 0x7f6118136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): picked mon.noname-a con 0x7f6118135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611809abc0 con 0x7f6118135dc0 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611809ba50 con 0x7f6118136e40 2024-09-16T15:00:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61180a53c0 con 0x7f6118137220 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f6120490640 10 monclient(hunting): authenticate will time out at 788.182319s 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33376/0 (socket says 172.21.15.134:33376) 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 1 -- 172.21.15.134:0/1560932829 learned_addr learned my addr 172.21.15.134:0/1560932829 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): get_auth_request con 0x7f6118137220 auth_method 0 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 1 --2- 172.21.15.134:0/1560932829 >> v2:172.21.15.134:3300/0 conn(0x7f6118135dc0 0x7f6118136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): get_auth_request con 0x7f6118135dc0 auth_method 0 2024-09-16T15:00:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611e22c640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f611809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611e22c640 10 monclient(hunting): get_auth_request con 0x7f6118136e40 auth_method 0 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611e22c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611e22c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611e22c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.593+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 10 monclient(hunting): handle_auth_done global_id 4311 payload 274 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 -- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 msgr2=0x7f6118137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 -- 172.21.15.134:0/1560932829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 msgr2=0x7f611809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f611809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 monclient: found mon.noname-a 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611ea2d640 1 -- 172.21.15.134:0/1560932829 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118073250 con 0x7f6118135dc0 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611da2b640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.594+0000 7f611e22c640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f611809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.596+0000 7f8fa8e0d640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.596+0000 7f8fa8e0d640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f8fa8e0d640 10 monclient: monmap: 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.597919+0000 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.597919+0000 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.605 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611ea2d640 1 -- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6118135dc0 msgr2=0x7f6118136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611ea2d640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6118135dc0 0x7f6118136220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f611000b050 tx=0x7f611002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5459e5c4631ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 1 -- 172.21.15.134:0/1560932829 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61100079b0 con 0x7f6118135dc0 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: dump: 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.606 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.607 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f8fa8e0d640 1 Processor -- start 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 1 -- 172.21.15.134:0/1560932829 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611002d050 con 0x7f6118135dc0 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 1 -- 172.21.15.134:0/1560932829 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6110031400 con 0x7f6118135dc0 2024-09-16T15:00:51.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f6120490640 5 monclient: authenticate success, global_id 4311 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f8fa8e0d640 1 -- start start 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f6120490640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f6120490640 4 set_mon_vals no callback set 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f611d22a640 10 monclient: dump: 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.609 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.610 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.597+0000 7f8fa8e0d640 10 monclient: init 2024-09-16T15:00:51.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 10 monclient: shutdown 2024-09-16T15:00:51.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 -- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6118135dc0 msgr2=0x7f6118136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6118135dc0 0x7f6118136220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f611000b050 tx=0x7f611002a710 comp rx=0 tx=0).stop 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient: _add_conns ranks=[3,0,1,2,10,8,6,7,9,5,4] 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 0x7f8fa409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-g con 0x7f8fa4136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8fa4137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:51.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8fa413c370 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa4073250 con 0x7f8fa4137220 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa409ba50 con 0x7f8fa413c370 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 -- 172.21.15.134:0/1560932829 shutdown_connections 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6118135dc0 0x7f6118136220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6118137220 0x7f6118137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 --2- 172.21.15.134:0/1560932829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f611809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40a56e0 con 0x7f8fa4136e40 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa8e0d640 10 monclient(hunting): authenticate will time out at 788.187651s 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 -- 172.21.15.134:0/1560932829 >> 172.21.15.134:0/1560932829 conn(0x7f61180a6110 msgr2=0x7f61180a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 -- 172.21.15.134:0/1560932829 shutdown_connections 2024-09-16T15:00:51.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa1d9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8fa4137220 0x7f8fa4137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38934/0 (socket says 172.21.15.134:38934) 2024-09-16T15:00:51.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f6120490640 1 -- 172.21.15.134:0/1560932829 wait complete. 2024-09-16T15:00:51.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa1d9b640 1 -- 172.21.15.134:0/3156013568 learned_addr learned my addr 172.21.15.134:0/3156013568 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.598+0000 7f8fa259c640 1 --2- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 0x7f8fa409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): get_auth_request con 0x7f8fa4136e40 auth_method 0 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient(hunting): handle_auth_done global_id 4314 payload 274 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 1 -- 172.21.15.134:0/3156013568 >> v1:172.21.15.162:6789/0 conn(0x7f8fa413c370 legacy=0x7f8fa413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 1 -- 172.21.15.134:0/3156013568 >> v1:172.21.15.134:6789/0 conn(0x7f8fa4137220 legacy=0x7f8fa4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 1 monclient: found mon.noname-g 2024-09-16T15:00:51.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:51.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f8fa259c640 1 -- 172.21.15.134:0/3156013568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40a53c0 con 0x7f8fa4136e40 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f6120490640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.599+0000 7f6120490640 10 monclient: monmap: 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.621 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 Processor -- start 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 -- start start 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient: init 2024-09-16T15:00:51.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient: _add_conns ranks=[5,3,6,8,2,0,1,7,4] 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6118135dc0 0x7f61181cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): picked mon.noname-h con 0x7f6118135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f6118134fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): picked mon.noname-g con 0x7f6118136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): picked mon.noname-d con 0x7f6118137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6118123710 con 0x7f6118136e40 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61181cb7f0 con 0x7f6118135dc0 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6118136350 con 0x7f6118137220 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f6120490640 10 monclient(hunting): authenticate will time out at 788.189791s 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611da2b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f6118134fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611e22c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6118135dc0 0x7f61181cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42244/0 (socket says 172.21.15.134:42244) 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611da2b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f6118134fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:32820/0 (socket says 172.21.15.134:32820) 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 1 -- 172.21.15.134:0/914279103 learned_addr learned my addr 172.21.15.134:0/914279103 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 10 monclient(hunting): get_auth_request con 0x7f6118137220 auth_method 0 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611ea2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611e22c640 10 monclient(hunting): get_auth_request con 0x7f6118135dc0 auth_method 0 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611e22c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611e22c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.600+0000 7f611e22c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): get_auth_request con 0x7f6118136e40 auth_method 0 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611e22c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611e22c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611e22c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611da2b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient(hunting): handle_auth_done global_id 4245 payload 824 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 -- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6118135dc0 msgr2=0x7f61181cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6118135dc0 0x7f61181cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 -- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 msgr2=0x7f6118134fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f6118134fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 monclient: found mon.noname-d 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.601+0000 7f611ea2d640 1 -- 172.21.15.134:0/914279103 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080094d0 con 0x7f6118137220 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa259c640 1 -- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 msgr2=0x7f8fa409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa259c640 1 --2- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 0x7f8fa409c150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8f940097d0 tx=0x7f8f9402a040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=82ad806f0ee23eac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 1 -- 172.21.15.134:0/3156013568 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f9402ae40 con 0x7f8fa4136e40 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:51.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: dump: 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 1 -- 172.21.15.134:0/3156013568 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f940307b0 con 0x7f8fa4136e40 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 1 -- 172.21.15.134:0/3156013568 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f94030d90 con 0x7f8fa4136e40 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa159a640 10 monclient: dump: 2024-09-16T15:00:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.631 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa8e0d640 5 monclient: authenticate success, global_id 4314 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa8e0d640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.602+0000 7f8fa8e0d640 4 set_mon_vals no callback set 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 10 monclient: shutdown 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 msgr2=0x7f8fa409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 0x7f8fa409c150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8f940097d0 tx=0x7f8f9402a040 comp rx=0 tx=0).stop 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/3156013568 shutdown_connections 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/3156013568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fa4136e40 0x7f8fa409c150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/3156013568 >> 172.21.15.134:0/3156013568 conn(0x7f8fa40a6110 msgr2=0x7f8fa40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/3156013568 shutdown_connections 2024-09-16T15:00:51.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/3156013568 wait complete. 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f611ea2d640 1 -- 172.21.15.134:0/914279103 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 msgr2=0x7f6118135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f611ea2d640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f611002ac30 tx=0x7f6110004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6110004850 con 0x7f6118137220 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 10 monclient: dump: 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.634 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.635 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f610effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f6120490640 5 monclient: authenticate success, global_id 4245 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f6120490640 10 monclient: _renew_subs 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f6120490640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.603+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110009a70 con 0x7f6118137220 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.604+0000 7f6120490640 10 monclient: _renew_subs 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.604+0000 7f6120490640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.604+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61180a86a0 con 0x7f6118137220 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.604+0000 7f8fa8e0d640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.604+0000 7f8fa8e0d640 10 monclient: monmap: 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.637 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 Processor -- start 2024-09-16T15:00:51.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f6120490640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f6120490640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f6120490640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6118125130 con 0x7f6118137220 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6110007540 con 0x7f6118137220 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6110007b20 con 0x7f6118137220 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 10 monclient: dump: 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.640 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6110007c80 con 0x7f6118137220 2024-09-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 -- start start 2024-09-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient: init 2024-09-16T15:00:51.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient: _add_conns ranks=[5,2,8,4,7,0,1,3,6] 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8fa4136e40 0x7f8fa41cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-h con 0x7f8fa4136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 0x7f8fa4070780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-b con 0x7f8fa4137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 0x7f8fa4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): picked mon.noname-e con 0x7f8fa413c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa406ed20 con 0x7f8fa4137220 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa406f250 con 0x7f8fa4136e40 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa406dc90 con 0x7f8fa413c370 2024-09-16T15:00:51.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f8fa8e0d640 10 monclient(hunting): authenticate will time out at 788.194848s 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 0x7f8fa4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 0x7f8fa4070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 0x7f8fa4070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58014/0 (socket says 172.21.15.134:58014) 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 1 -- 172.21.15.134:0/2003896803 learned_addr learned my addr 172.21.15.134:0/2003896803 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): get_auth_request con 0x7f8fa4137220 auth_method 0 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): get_auth_request con 0x7f8fa413c370 auth_method 0 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient(hunting): handle_auth_done global_id 4292 payload 824 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 -- 172.21.15.134:0/2003896803 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8fa4136e40 msgr2=0x7f8fa41cfa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8fa4136e40 0x7f8fa41cfa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 -- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 msgr2=0x7f8fa4070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 0x7f8fa4070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 monclient: found mon.noname-e 2024-09-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f8fa2d9d640 1 -- 172.21.15.134:0/2003896803 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94009400 con 0x7f8fa413c370 2024-09-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 1 --2- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 0x7f60fc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.605+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f611006b580 con 0x7f6118137220 2024-09-16T15:00:51.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.606+0000 7f611ca29640 4 set_mon_vals no callback set 2024-09-16T15:00:51.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.607+0000 7f611e22c640 1 --2- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 0x7f60fc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.607+0000 7f611e22c640 10 monclient: get_auth_request con 0x7f60fc038880 auth_method 0 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa2d9d640 1 -- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 msgr2=0x7f8fa4070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa2d9d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 0x7f8fa4070cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406de50 tx=0x7f8f9c00c410 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f9c00ce50 con 0x7f8fa413c370 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 10 monclient: dump: 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.648 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8f92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f611e22c640 1 --2- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 0x7f60fc03ad40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6114006fd0 tx=0x7f6114008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 5 monclient: authenticate success, global_id 4292 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 10 monclient: _renew_subs 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa406da50 con 0x7f8fa413c370 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 10 monclient: _renew_subs 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.609+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa40a39e0 con 0x7f8fa413c370 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.611+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c0127b0 con 0x7f8fa413c370 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.611+0000 7f8f92ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.611+0000 7f8f93fff640 4 set_mon_vals no callback set 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.611+0000 7f8fa8e0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8f9c012e10 con 0x7f8fa413c370 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8fa8e0d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8fa8e0d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f88005040 con 0x7f8fa413c370 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 10 monclient: dump: 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.653 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f611003c420 con 0x7f6118137220 2024-09-16T15:00:51.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f610effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f610effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8f9c01c360 con 0x7f8fa413c370 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 1 --2- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 0x7f8f8403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.612+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f8f9c04a2a0 con 0x7f8fa413c370 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.613+0000 7f8fa259c640 1 --2- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 0x7f8f8403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.613+0000 7f8fa259c640 10 monclient: get_auth_request con 0x7f8f84038810 auth_method 0 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.615+0000 7f8fa259c640 1 --2- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 0x7f8f8403acd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406cb00 tx=0x7f8f9402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.617+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8f9c0133a0 con 0x7f8fa413c370 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.617+0000 7f8f92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.617+0000 7f8f92ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.628+0000 7ff016133640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.628+0000 7ff016133640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.628+0000 7ff016133640 10 monclient: monmap: 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.629374+0000 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.629374+0000 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.662 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 Processor -- start 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 -- start start 2024-09-16T15:00:51.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient: init 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient: _add_conns ranks=[2,3,10,5,7,8,9,1,6,0,4] 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): picked mon.noname-b con 0x7ff010136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff010137220 0x7ff010137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): picked mon.noname-g con 0x7ff010137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff010135dc0 0x7ff010136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): picked mon.noname-f con 0x7ff010135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01009abc0 con 0x7ff010135dc0 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01009ba50 con 0x7ff010136e40 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100a53c0 con 0x7ff010137220 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff016133640 10 monclient(hunting): authenticate will time out at 788.218829s 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58026/0 (socket says 172.21.15.134:58026) 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 1 -- 172.21.15.134:0/861181090 learned_addr learned my addr 172.21.15.134:0/861181090 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 10 monclient(hunting): get_auth_request con 0x7ff010136e40 auth_method 0 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.629+0000 7ff00f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 1 --2- 172.21.15.134:0/861181090 >> v2:172.21.15.162:3300/0 conn(0x7ff010135dc0 0x7ff010136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff010137220 0x7ff010137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): get_auth_request con 0x7ff010135dc0 auth_method 0 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): get_auth_request con 0x7ff010137220 auth_method 0 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_done global_id 4259 payload 274 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 -- 172.21.15.134:0/861181090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff010137220 msgr2=0x7ff010137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff010137220 0x7ff010137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 -- 172.21.15.134:0/861181090 >> v2:172.21.15.162:3300/0 conn(0x7ff010135dc0 msgr2=0x7ff010136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/861181090 >> v2:172.21.15.162:3300/0 conn(0x7ff010135dc0 0x7ff010136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.630+0000 7ff00f7fe640 1 -- 172.21.15.134:0/861181090 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010073250 con 0x7ff010136e40 2024-09-16T15:00:51.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00f7fe640 1 -- 172.21.15.134:0/861181090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 msgr2=0x7ff01009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0000098a0 tx=0x7ff00002a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c57e59eec0fb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7f2088244640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7f2088244640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 1 -- 172.21.15.134:0/861181090 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff00002adf0 con 0x7ff010136e40 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: dump: 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff016133640 5 monclient: authenticate success, global_id 4259 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff016133640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7f2088244640 10 monclient: monmap: 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.634332+0000 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.634332+0000 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.671 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 1 -- 172.21.15.134:0/861181090 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000030340 con 0x7ff010136e40 2024-09-16T15:00:51.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 1 -- 172.21.15.134:0/861181090 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff000030960 con 0x7ff010136e40 2024-09-16T15:00:51.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff00e7fc640 10 monclient: dump: 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff016133640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.633+0000 7ff016133640 4 set_mon_vals no callback set 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f2088244640 1 Processor -- start 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f908655e640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f908655e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f908655e640 10 monclient: monmap: 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.635480+0000 2024-09-16T15:00:51.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.635480+0000 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.677 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 10 monclient: shutdown 2024-09-16T15:00:51.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 -- 172.21.15.134:0/861181090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 msgr2=0x7ff01009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0000098a0 tx=0x7ff00002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 -- 172.21.15.134:0/861181090 shutdown_connections 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 --2- 172.21.15.134:0/861181090 >> v2:172.21.15.162:3300/0 conn(0x7ff010135dc0 0x7ff010136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff010137220 0x7ff010137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 --2- 172.21.15.134:0/861181090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010136e40 0x7ff01009c150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 -- 172.21.15.134:0/861181090 >> 172.21.15.134:0/861181090 conn(0x7ff0100a6110 msgr2=0x7ff0100a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f2088244640 1 -- start start 2024-09-16T15:00:51.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7f2088244640 10 monclient: init 2024-09-16T15:00:51.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.634+0000 7ff016133640 1 -- 172.21.15.134:0/861181090 shutdown_connections 2024-09-16T15:00:51.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7ff016133640 1 -- 172.21.15.134:0/861181090 wait complete. 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient: _add_conns ranks=[3,9,2,10,4,0,7,8,5,6,1] 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 0x7f208009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): picked mon.noname-g con 0x7f2080136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): picked mon.noname-a con 0x7f2080137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 0x7f2080136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): picked mon.noname-b con 0x7f2080135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208009abc0 con 0x7f2080137220 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51858/0 (socket says 172.21.15.134:51858) 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 1 -- 172.21.15.134:0/3829854226 learned_addr learned my addr 172.21.15.134:0/3829854226 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 0x7f208009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208009ba50 con 0x7f2080135dc0 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20800a53c0 con 0x7f2080136e40 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2088244640 10 monclient(hunting): authenticate will time out at 788.224671s 2024-09-16T15:00:51.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): get_auth_request con 0x7f2080137220 auth_method 0 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): get_auth_request con 0x7f2080136e40 auth_method 0 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 0x7f2080136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 Processor -- start 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 -- start start 2024-09-16T15:00:51.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient: init 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): get_auth_request con 0x7f2080135dc0 auth_method 0 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.635+0000 7f20867e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 10 monclient(hunting): handle_auth_done global_id 4341 payload 274 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 -- 172.21.15.134:0/3829854226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 msgr2=0x7f2080136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 0x7f2080136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 -- 172.21.15.134:0/3829854226 >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 msgr2=0x7f2080137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 --2- 172.21.15.134:0/3829854226 >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 monclient: found mon.noname-g 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:51.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f2085fe0640 1 -- 172.21.15.134:0/3829854226 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080073250 con 0x7f2080136e40 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7ff016133640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7ff016133640 10 monclient: monmap: 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.687 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f20857df640 1 --2- 172.21.15.134:0/3829854226 >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f20867e1640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 0x7f2080136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient: _add_conns ranks=[7,2,4,6,0,9,10,5,8,1,3] 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 0x7f9080130dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): picked mon.noname-i con 0x7f90801309e0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f9080133b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): picked mon.noname-b con 0x7f9080131300 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 0x7f9080134520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): picked mon.noname-c con 0x7f90801340a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908009e050 con 0x7f9080131300 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90800a0930 con 0x7f90801340a0 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f908009e8f0 con 0x7f90801309e0 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f908655e640 10 monclient(hunting): authenticate will time out at 788.225780s 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.636+0000 7f907ffff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 0x7f9080130dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 0x7f9080134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f9080133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 0x7f9080130dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:59362/0 (socket says 172.21.15.134:59362) 2024-09-16T15:00:51.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 0x7f9080134520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33380/0 (socket says 172.21.15.134:33380) 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 1 -- 172.21.15.134:0/3908322812 learned_addr learned my addr 172.21.15.134:0/3908322812 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): get_auth_request con 0x7f90801309e0 auth_method 0 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): get_auth_request con 0x7f90801340a0 auth_method 0 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): get_auth_request con 0x7f9080131300 auth_method 0 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7ff016133640 1 Processor -- start 2024-09-16T15:00:51.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f9084afb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient(hunting): handle_auth_done global_id 4277 payload 274 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7ff016133640 1 -- start start 2024-09-16T15:00:51.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7ff016133640 10 monclient: init 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 -- 172.21.15.134:0/3908322812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 msgr2=0x7f9080130dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 0x7f9080130dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 -- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 msgr2=0x7f9080134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 0x7f9080134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.637+0000 7f907f7fe640 1 -- 172.21.15.134:0/3908322812 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080073250 con 0x7f9080131300 2024-09-16T15:00:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient: _add_conns ranks=[2,8,6,7,5,1,0,4,3] 2024-09-16T15:00:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): picked mon.noname-b con 0x7ff010135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 0x7ff010135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): picked mon.noname-e con 0x7ff010136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 0x7ff010121850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): picked mon.noname-d con 0x7ff010137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100a8a10 con 0x7ff010135dc0 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff010136390 con 0x7ff010137220 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff010123990 con 0x7ff010136e40 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff016133640 10 monclient(hunting): authenticate will time out at 788.227218s 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58058/0 (socket says 172.21.15.134:58058) 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 1 -- 172.21.15.134:0/428838094 learned_addr learned my addr 172.21.15.134:0/428838094 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): get_auth_request con 0x7ff010135dc0 auth_method 0 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 0x7ff010135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): get_auth_request con 0x7ff010136e40 auth_method 0 2024-09-16T15:00:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00ffff640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 0x7ff010121850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00ffff640 10 monclient(hunting): get_auth_request con 0x7ff010137220 auth_method 0 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 10 monclient(hunting): handle_auth_done global_id 4286 payload 824 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.638+0000 7ff00f7fe640 1 -- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 msgr2=0x7ff010135330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 0x7ff010135330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 1 -- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 msgr2=0x7ff010121850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 0x7ff010121850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00f7fe640 1 -- 172.21.15.134:0/428838094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000094d0 con 0x7ff010135dc0 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00ffff640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 0x7ff010121850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.639+0000 7ff00effd640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 0x7ff010135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2085fe0640 1 -- 172.21.15.134:0/3829854226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 msgr2=0x7f208009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2085fe0640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 0x7f208009c150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f207c0097d0 tx=0x7f207c032800 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f3eb0fcfcb0dce76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 1 -- 172.21.15.134:0/3829854226 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f207c037b90 con 0x7f2080136e40 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: dump: 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 1 -- 172.21.15.134:0/3829854226 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c035050 con 0x7f2080136e40 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2088244640 5 monclient: authenticate success, global_id 4341 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2088244640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f907f7fe640 1 -- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 msgr2=0x7f9080133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f907f7fe640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f9080133b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90740097d0 tx=0x7f907402e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2377ca2b8d155ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 1 -- 172.21.15.134:0/3829854226 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f207c037d10 con 0x7f2080136e40 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2084fde640 10 monclient: dump: 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2088244640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.640+0000 7f2088244640 4 set_mon_vals no callback set 2024-09-16T15:00:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 1 -- 172.21.15.134:0/3908322812 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9074033920 con 0x7f9080131300 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: dump: 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.711 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 1 -- 172.21.15.134:0/3908322812 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074033aa0 con 0x7f9080131300 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f908655e640 5 monclient: authenticate success, global_id 4277 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f908655e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:51.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 1 -- 172.21.15.134:0/3908322812 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9074033c10 con 0x7f9080131300 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f907effd640 10 monclient: dump: 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f908655e640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f908655e640 4 set_mon_vals no callback set 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff00f7fe640 1 -- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 msgr2=0x7ff010134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff00f7fe640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff0000097c0 tx=0x7ff0000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0000304b0 con 0x7ff010135dc0 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 10 monclient: shutdown 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 msgr2=0x7f208009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 0x7f208009c150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f207c0097d0 tx=0x7f207c032800 comp rx=0 tx=0).stop 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 shutdown_connections 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2080135dc0 0x7f2080136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 --2- 172.21.15.134:0/3829854226 >> v2:172.21.15.134:3300/0 conn(0x7f2080137220 0x7f2080137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 --2- 172.21.15.134:0/3829854226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2080136e40 0x7f208009c150 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f207c0097d0 tx=0x7f207c032800 comp rx=0 tx=0).stop 2024-09-16T15:00:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 >> 172.21.15.134:0/3829854226 conn(0x7f20800a6110 msgr2=0x7f20800a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: dump: 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 shutdown_connections 2024-09-16T15:00:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7f2088244640 1 -- 172.21.15.134:0/3829854226 wait complete. 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000003af0 con 0x7ff010135dc0 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff000003c60 con 0x7ff010135dc0 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7feff7fff640 10 monclient: dump: 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 10 monclient: shutdown 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 -- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 msgr2=0x7f9080133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f9080133b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90740097d0 tx=0x7f907402e3c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 5 monclient: authenticate success, global_id 4286 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 10 monclient: _renew_subs 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff00dffb640 4 set_mon_vals no callback set 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 -- 172.21.15.134:0/3908322812 shutdown_connections 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90801340a0 0x7f9080134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f9080133b60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 --2- 172.21.15.134:0/3908322812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90801309e0 0x7f9080130dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 -- 172.21.15.134:0/3908322812 >> 172.21.15.134:0/3908322812 conn(0x7f90800a6470 msgr2=0x7f90800a6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0101368c0 con 0x7ff010135dc0 2024-09-16T15:00:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 -- 172.21.15.134:0/3908322812 shutdown_connections 2024-09-16T15:00:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f4278b33640 10 monclient: get_monmap_and_config 2024-09-16T15:00:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f4278b33640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 10 monclient: _renew_subs 2024-09-16T15:00:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.641+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0101257e0 con 0x7ff010135dc0 2024-09-16T15:00:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f4278b33640 10 monclient: monmap: 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:51.643507+0000 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:51.643507+0000 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f908655e640 1 -- 172.21.15.134:0/3908322812 wait complete. 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.642+0000 7f2088244640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f2088244640 10 monclient: monmap: 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f908655e640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f908655e640 10 monclient: monmap: 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.730 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f2088244640 1 Processor -- start 2024-09-16T15:00:51.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f2088244640 1 -- start start 2024-09-16T15:00:51.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f2088244640 10 monclient: init 2024-09-16T15:00:51.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7ff016133640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f4278b33640 1 Processor -- start 2024-09-16T15:00:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7ff016133640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f908655e640 1 Processor -- start 2024-09-16T15:00:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f4278b33640 1 -- start start 2024-09-16T15:00:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7f4278b33640 10 monclient: init 2024-09-16T15:00:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7ff016133640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0101250d0 con 0x7ff010135dc0 2024-09-16T15:00:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.643+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff000039070 con 0x7ff010135dc0 2024-09-16T15:00:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient: _add_conns ranks=[7,0,8,4,6,2,3,5,1] 2024-09-16T15:00:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 1 -- start start 2024-09-16T15:00:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 0x7f2080134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 10 monclient: init 2024-09-16T15:00:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7feff7fff640 1 --2- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 0x7feffc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): picked mon.noname-i con 0x7f2080135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7ff00006a8f0 con 0x7ff010135dc0 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient: _add_conns ranks=[5,9,6,0,7,4,10,3,2,8,1] 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 0x7f426c028650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-h con 0x7f426c02a870 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f20857df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2085fe0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 0x7f2080134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2085fe0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 0x7f2080134f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:59364/0 (socket says 172.21.15.134:59364) 2024-09-16T15:00:51.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f20857df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51872/0 (socket says 172.21.15.134:51872) 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f426c028b90 0x7f426c022d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-a con 0x7f426c028b90 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 0x7f426c023700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-d con 0x7f426c023290 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c024300 con 0x7f426c028b90 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f426c02ac50 con 0x7f426c02a870 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2085fe0640 1 -- 172.21.15.134:0/230859543 learned_addr learned my addr 172.21.15.134:0/230859543 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): picked mon.noname-a con 0x7f2080137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0245d0 con 0x7f426c023290 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f4278b33640 10 monclient(hunting): authenticate will time out at 788.233662s 2024-09-16T15:00:51.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 0x7f208006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 10 monclient: _add_conns ranks=[1,2,6,8,0,3,5,4,7] 2024-09-16T15:00:51.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): picked mon.noname-e con 0x7f20801c74b0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801c81b0 con 0x7f2080137220 2024-09-16T15:00:51.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f42737fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 0x7f426c028650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 0x7f90801c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 10 monclient(hunting): picked mon.noname-f con 0x7f90801309e0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:51.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2080121940 con 0x7f2080135dc0 2024-09-16T15:00:51.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f908655e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f908012fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.644+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2080136270 con 0x7f20801c74b0 2024-09-16T15:00:51.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): picked mon.noname-b con 0x7f9080131300 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2088244640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 0x7f208006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f90801340a0 0x7f90801c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2088244640 10 monclient(hunting): authenticate will time out at 788.234050s 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 0x7f426c023700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): picked mon.noname-d con 0x7f90801340a0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 0x7f426c028650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:59870/0 (socket says 172.21.15.134:59870) 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 1 -- 172.21.15.134:0/1454679711 learned_addr learned my addr 172.21.15.134:0/1454679711 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): get_auth_request con 0x7f426c02a870 auth_method 0 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90801c7940 con 0x7f90801309e0 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 1 --2- 172.21.15.134:0/1454679711 >> v2:172.21.15.134:3300/0 conn(0x7f426c028b90 0x7f426c022d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): get_auth_request con 0x7f426c028b90 auth_method 0 2024-09-16T15:00:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 0x7f90801c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908012c680 con 0x7f9080131300 2024-09-16T15:00:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9080133a10 con 0x7f90801340a0 2024-09-16T15:00:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): get_auth_request con 0x7f426c023290 auth_method 0 2024-09-16T15:00:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f908655e640 10 monclient(hunting): authenticate will time out at 788.234294s 2024-09-16T15:00:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f908012fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f908012fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58062/0 (socket says 172.21.15.134:58062) 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 1 -- 172.21.15.134:0/1199618089 learned_addr learned my addr 172.21.15.134:0/1199618089 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): get_auth_request con 0x7f9080131300 auth_method 0 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): get_auth_request con 0x7f90801309e0 auth_method 0 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): get_auth_request con 0x7f2080135dc0 auth_method 0 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): get_auth_request con 0x7f2080137220 auth_method 0 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 10 monclient(hunting): get_auth_request con 0x7f20801c74b0 auth_method 0 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f2085fe0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient(hunting): handle_auth_done global_id 4329 payload 824 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 -- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 msgr2=0x7f208006efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 0x7f208006efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 -- 172.21.15.134:0/230859543 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 msgr2=0x7f2080134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 0x7f2080134f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 monclient: found mon.noname-a 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20857df640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009a70 con 0x7f2080137220 2024-09-16T15:00:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f20867e1640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 0x7f208006efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient(hunting): handle_auth_done global_id 4312 payload 824 2024-09-16T15:00:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 -- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f90801340a0 msgr2=0x7f90801c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f90801340a0 0x7f90801c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient(hunting): handle_auth_done global_id 4263 payload 274 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 -- 172.21.15.134:0/1454679711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 msgr2=0x7f426c028650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 --2- 172.21.15.134:0/1454679711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 0x7f426c028650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 -- 172.21.15.134:0/1454679711 >> v2:172.21.15.134:3300/0 conn(0x7f426c028b90 msgr2=0x7f426c022d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 --2- 172.21.15.134:0/1454679711 >> v2:172.21.15.134:3300/0 conn(0x7f426c028b90 0x7f426c022d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 monclient: found mon.noname-d 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f4273fff640 1 -- 172.21.15.134:0/1454679711 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0042e0 con 0x7f426c023290 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 -- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 msgr2=0x7f908012fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f908012fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 monclient: found mon.noname-f 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.645+0000 7f907ffff640 1 -- 172.21.15.134:0/1199618089 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9074009400 con 0x7f90801309e0 2024-09-16T15:00:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.646+0000 7ff00effd640 1 --2- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 0x7feffc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.646+0000 7ff00effd640 10 monclient: get_auth_request con 0x7feffc038880 auth_method 0 2024-09-16T15:00:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20857df640 1 -- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 msgr2=0x7f2080135440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20857df640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f207000ebe0 tx=0x7f207000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20700044a0 con 0x7f2080137220 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907ffff640 1 -- 172.21.15.134:0/1199618089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 msgr2=0x7f90801c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907ffff640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 0x7f90801c6ee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f907800efc0 tx=0x7f907800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: dump: 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.759 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207000cc30 con 0x7f2080137220 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20777fe640 4 set_mon_vals no callback set 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f907800cdb0 con 0x7f90801309e0 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: dump: 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907800f040 con 0x7f90801309e0 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9078014d60 con 0x7f90801309e0 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907cff9640 10 monclient: dump: 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f908655e640 5 monclient: authenticate success, global_id 4312 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f908655e640 10 monclient: _renew_subs 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f908655e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9080130560 con 0x7f90801309e0 2024-09-16T15:00:51.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7ff00effd640 1 --2- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 0x7feffc03ad40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff010122fb0 tx=0x7ff004009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f907e7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:51.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f908655e640 10 monclient: _renew_subs 2024-09-16T15:00:51.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f908655e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908012c0c0 con 0x7f90801309e0 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f4273fff640 1 -- 172.21.15.134:0/1454679711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 msgr2=0x7f426c023700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f4273fff640 1 --2- 172.21.15.134:0/1454679711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 0x7f426c023700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4260009a00 tx=0x7f426002a230 comp rx=0 tx=0).ready entity=mon.6 client_cookie=628277783cda9dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f207000cda0 con 0x7f2080137220 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f20767fc640 10 monclient: dump: 2024-09-16T15:00:51.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f2088244640 5 monclient: authenticate success, global_id 4329 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f2088244640 10 monclient: _renew_subs 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f2088244640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.648+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20700094d0 con 0x7f2080137220 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f2088244640 10 monclient: _renew_subs 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f2088244640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c009400 con 0x7f2080137220 2024-09-16T15:00:51.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 1 -- 172.21.15.134:0/1454679711 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f426002ad80 con 0x7f426c023290 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 10 monclient: dump: 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.769 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f42727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f4278b33640 5 monclient: authenticate success, global_id 4263 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.649+0000 7f4278b33640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 1 -- 172.21.15.134:0/1454679711 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260030340 con 0x7f426c023290 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:51.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 4 set_mon_vals no callback set 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2070004030 con 0x7f2080137220 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f20767fc640 1 --2- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 0x7f206403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f2088244640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f2088244640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f2088244640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f2070049fd0 con 0x7f2080137220 2024-09-16T15:00:51.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208009cbc0 con 0x7f2080137220 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 1 -- 172.21.15.134:0/1454679711 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f426002ad80 con 0x7f426c023290 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f42727fc640 10 monclient: dump: 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.772 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 10 monclient: shutdown 2024-09-16T15:00:51.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 1 -- 172.21.15.134:0/1454679711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 msgr2=0x7f426c023700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.650+0000 7f4278b33640 1 --2- 172.21.15.134:0/1454679711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 0x7f426c023700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4260009a00 tx=0x7f426002a230 comp rx=0 tx=0).stop 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 -- 172.21.15.134:0/1454679711 shutdown_connections 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 --2- 172.21.15.134:0/1454679711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f426c023290 0x7f426c023700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 --2- 172.21.15.134:0/1454679711 >> v2:172.21.15.134:3300/0 conn(0x7f426c028b90 0x7f426c022d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 --2- 172.21.15.134:0/1454679711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f426c02a870 0x7f426c028650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 -- 172.21.15.134:0/1454679711 >> 172.21.15.134:0/1454679711 conn(0x7f426c007be0 msgr2=0x7f426c007ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9078014480 con 0x7f90801309e0 2024-09-16T15:00:51.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 -- 172.21.15.134:0/1454679711 shutdown_connections 2024-09-16T15:00:51.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f4278b33640 1 -- 172.21.15.134:0/1454679711 wait complete. 2024-09-16T15:00:51.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff00003cd40 con 0x7ff010135dc0 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7feff7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7feff7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f907cff9640 1 --2- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 0x7f906c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f907804a3d0 con 0x7f90801309e0 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f2085fe0640 1 --2- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 0x7f206403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f2085fe0640 10 monclient: get_auth_request con 0x7f2064038880 auth_method 0 2024-09-16T15:00:51.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f907f7fe640 1 --2- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 0x7f906c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f907f7fe640 10 monclient: get_auth_request con 0x7f906c038880 auth_method 0 2024-09-16T15:00:51.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.651+0000 7f908655e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f908655e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f908655e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f4278b33640 10 monclient: build_initial_monmap 2024-09-16T15:00:51.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908012c250 con 0x7f90801309e0 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f4278b33640 10 monclient: monmap: 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.780 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f4278b33640 1 Processor -- start 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f4278b33640 1 -- start start 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.652+0000 7f4278b33640 10 monclient: init 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:51.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient: _add_conns ranks=[2,8,3,4,1,6,0,7,5] 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-b con 0x7f426c023290 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f426c028b90 0x7f426c0a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-e con 0x7f426c028b90 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 0x7f426c0a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): picked mon.noname-g con 0x7f426c02a870 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0082e0 con 0x7f426c023290 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c028900 con 0x7f426c02a870 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0aa430 con 0x7f426c028b90 2024-09-16T15:00:51.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4278b33640 10 monclient(hunting): authenticate will time out at 788.242303s 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58076/0 (socket says 172.21.15.134:58076) 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 -- 172.21.15.134:0/926629671 learned_addr learned my addr 172.21.15.134:0/926629671 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): get_auth_request con 0x7f426c023290 auth_method 0 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f426c028b90 0x7f426c0a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): get_auth_request con 0x7f426c028b90 auth_method 0 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4272ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient(hunting): handle_auth_done global_id 4304 payload 824 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 -- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f426c028b90 msgr2=0x7f426c0a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f426c028b90 0x7f426c0a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 -- 172.21.15.134:0/926629671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 msgr2=0x7f426c0a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4273fff640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 0x7f426c0a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 0x7f426c0a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:51.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f42737fe640 1 -- 172.21.15.134:0/926629671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009400 con 0x7f426c023290 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.653+0000 7f4273fff640 10 monclient: get_auth_request con 0x7f426c02a870 auth_method 0 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.654+0000 7f4273fff640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 0x7f426c0a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.654+0000 7f2085fe0640 1 --2- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 0x7f206403ad40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c008030 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.654+0000 7f907f7fe640 1 --2- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 0x7f906c03ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f9074032000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f42737fe640 1 -- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 msgr2=0x7f426c0a6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f42737fe640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f427404efc0 tx=0x7f4274072540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4274067450 con 0x7f426c023290 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:51.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: dump: 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4274092390 con 0x7f426c023290 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4274092970 con 0x7f426c023290 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f425ffff640 10 monclient: dump: 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:51.788 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:51.789 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f20700042e0 con 0x7f2080137220 2024-09-16T15:00:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f20767fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f20767fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 5 monclient: authenticate success, global_id 4304 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 10 monclient: _renew_subs 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0a9410 con 0x7f426c023290 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4271ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 10 monclient: _renew_subs 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.656+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c0a8620 con 0x7f426c023290 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4274092500 con 0x7f426c023290 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f425ffff640 1 --2- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 0x7f425803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f907801d410 con 0x7f90801309e0 2024-09-16T15:00:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f907cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f42740c3ae0 con 0x7f426c023290 2024-09-16T15:00:51.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.657+0000 7f907cff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4272ffd640 1 --2- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 0x7f425803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:51.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4272ffd640 10 monclient: get_auth_request con 0x7f42580388b0 auth_method 0 2024-09-16T15:00:51.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4278b33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4278b33640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4278b33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.658+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c0bc7c0 con 0x7f426c023290 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.660+0000 7f4272ffd640 1 --2- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 0x7f425803ad70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f426c0a6b30 tx=0x7f4268009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.668+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4274098070 con 0x7f426c023290 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.668+0000 7f425ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:51.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.668+0000 7f425ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:51.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f6120490640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:51.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f6120490640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:51.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f6120490640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:51.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f611809b770 con 0x7f6118137220 2024-09-16T15:00:51.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f610effd640 1 -- 172.21.15.134:0/914279103 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f611003a3d0 con 0x7f6118137220 2024-09-16T15:00:51.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f610effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:51.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.802+0000 7f610effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.804 INFO:teuthology.orchestra.run.smithi134.stdout:55834574851 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.804+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 msgr2=0x7f60fc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.804+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 0x7f60fc03ad40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6114006fd0 tx=0x7f6114008040 comp rx=0 tx=0).stop 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 10 monclient: shutdown 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 msgr2=0x7f6118135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f611002ac30 tx=0x7f6110004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 shutdown_connections 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6118137220 0x7f6118135520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> v2:172.21.15.162:6800/980547263 conn(0x7f60fc038880 0x7f60fc03ad40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6118136e40 0x7f6118134fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 --2- 172.21.15.134:0/914279103 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6118135dc0 0x7f61181cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 >> 172.21.15.134:0/914279103 conn(0x7f61180a6110 msgr2=0x7f611809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 shutdown_connections 2024-09-16T15:00:51.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.805+0000 7f6120490640 1 -- 172.21.15.134:0/914279103 wait complete. 2024-09-16T15:00:51.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.2 2024-09-16T15:00:51.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f4278b33640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:51.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f4278b33640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f4278b33640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f426c0aa670 con 0x7f426c023290 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f425ffff640 1 -- 172.21.15.134:0/926629671 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4274066fe0 con 0x7f426c023290 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f425ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.840+0000 7f425ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.842 INFO:teuthology.orchestra.run.smithi134.stdout:47244640258 2024-09-16T15:00:51.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 msgr2=0x7f425803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 0x7f425803ad70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f426c0a6b30 tx=0x7f4268009190 comp rx=0 tx=0).stop 2024-09-16T15:00:51.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 10 monclient: shutdown 2024-09-16T15:00:51.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 msgr2=0x7f426c0a6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f427404efc0 tx=0x7f4274072540 comp rx=0 tx=0).stop 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 shutdown_connections 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> v2:172.21.15.162:6800/980547263 conn(0x7f42580388b0 0x7f425803ad70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f426c02a870 0x7f426c0a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f426c028b90 0x7f426c0a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 --2- 172.21.15.134:0/926629671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c023290 0x7f426c0a6dd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 >> 172.21.15.134:0/926629671 conn(0x7f426c007be0 msgr2=0x7f426c027da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 shutdown_connections 2024-09-16T15:00:51.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.842+0000 7f4278b33640 1 -- 172.21.15.134:0/926629671 wait complete. 2024-09-16T15:00:51.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.4 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8fa8e0d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8fa8e0d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8fa8e0d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f880052f0 con 0x7f8fa413c370 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8f92ffd640 1 -- 172.21.15.134:0/2003896803 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f9c019020 con 0x7f8fa413c370 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8f92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:51.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.960+0000 7f8f92ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.963 INFO:teuthology.orchestra.run.smithi134.stdout:55834574851 2024-09-16T15:00:51.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 msgr2=0x7f8f8403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 0x7f8f8403acd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406cb00 tx=0x7f8f9402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 10 monclient: shutdown 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 msgr2=0x7f8fa4070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 0x7f8fa4070cc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406de50 tx=0x7f8f9c00c410 comp rx=0 tx=0).stop 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 shutdown_connections 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> v2:172.21.15.162:6800/980547263 conn(0x7f8f84038810 0x7f8f8403acd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8fa413c370 0x7f8fa4070cc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fa4137220 0x7f8fa4070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 --2- 172.21.15.134:0/2003896803 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8fa4136e40 0x7f8fa41cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 >> 172.21.15.134:0/2003896803 conn(0x7f8fa40a6110 msgr2=0x7f8fa413a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 shutdown_connections 2024-09-16T15:00:51.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.963+0000 7f8fa8e0d640 1 -- 172.21.15.134:0/2003896803 wait complete. 2024-09-16T15:00:51.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.1 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7ff016133640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7ff016133640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7ff016133640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff01009b770 con 0x7ff010135dc0 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7feff7fff640 1 -- 172.21.15.134:0/428838094 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff000041020 con 0x7ff010135dc0 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7feff7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:51.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.984+0000 7feff7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.987 INFO:teuthology.orchestra.run.smithi134.stdout:55834574850 2024-09-16T15:00:51.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 msgr2=0x7feffc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 0x7feffc03ad40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff010122fb0 tx=0x7ff004009190 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 10 monclient: shutdown 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 msgr2=0x7ff010134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.986+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff0000097c0 tx=0x7ff0000046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 shutdown_connections 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> v2:172.21.15.162:6800/980547263 conn(0x7feffc038880 0x7feffc03ad40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff010137220 0x7ff010121850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff010136e40 0x7ff010135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 --2- 172.21.15.134:0/428838094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff010135dc0 0x7ff010134df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 >> 172.21.15.134:0/428838094 conn(0x7ff0100a6110 msgr2=0x7ff01006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 shutdown_connections 2024-09-16T15:00:51.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.987+0000 7ff016133640 1 -- 172.21.15.134:0/428838094 wait complete. 2024-09-16T15:00:51.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f2088244640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:51.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f2088244640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:51.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f2088244640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:51.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f208009b770 con 0x7f2080137220 2024-09-16T15:00:51.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f20767fc640 1 -- 172.21.15.134:0/230859543 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f207003d090 con 0x7f2080137220 2024-09-16T15:00:51.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f20767fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:51.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.992+0000 7f20767fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.994 INFO:teuthology.orchestra.run.smithi134.stdout:55834574852 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.993+0000 7f908655e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.993+0000 7f908655e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.993+0000 7f908655e640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.993+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90800a01b0 con 0x7f90801309e0 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f907cff9640 1 -- 172.21.15.134:0/1199618089 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9078012b30 con 0x7f90801309e0 2024-09-16T15:00:51.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f907cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:51.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f907cff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:51.996 INFO:teuthology.orchestra.run.smithi134.stdout:47244640258 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 msgr2=0x7f206403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 0x7f206403ad40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c008030 comp rx=0 tx=0).stop 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 10 monclient: shutdown 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 msgr2=0x7f2080135440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f207000ebe0 tx=0x7f207000c520 comp rx=0 tx=0).stop 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 shutdown_connections 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> v2:172.21.15.162:6800/980547263 conn(0x7f2064038880 0x7f206403ad40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f20801c74b0 0x7f208006efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080137220 0x7f2080135440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 --2- 172.21.15.134:0/230859543 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2080135dc0 0x7f2080134f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 >> 172.21.15.134:0/230859543 conn(0x7f20800a6110 msgr2=0x7f208006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 shutdown_connections 2024-09-16T15:00:51.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.994+0000 7f2088244640 1 -- 172.21.15.134:0/230859543 wait complete. 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 msgr2=0x7f906c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 0x7f906c03ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f9074032000 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 10 monclient: shutdown 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 msgr2=0x7f90801c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 0x7f90801c6ee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f907800efc0 tx=0x7f907800c490 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 shutdown_connections 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> v2:172.21.15.162:6800/980547263 conn(0x7f906c038880 0x7f906c03ad40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f90801340a0 0x7f90801c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9080131300 0x7f908012fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 --2- 172.21.15.134:0/1199618089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90801309e0 0x7f90801c6ee0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f907800efc0 tx=0x7f907800c490 comp rx=0 tx=0).stop 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 >> 172.21.15.134:0/1199618089 conn(0x7f90800a6470 msgr2=0x7f9080099640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:51.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 shutdown_connections 2024-09-16T15:00:52.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:51.996+0000 7f908655e640 1 -- 172.21.15.134:0/1199618089 wait complete. 2024-09-16T15:00:52.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.5 2024-09-16T15:00:52.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2024-09-16T15:00:52.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640258 for osd.0 2024-09-16T15:00:52.818 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:00:52.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:00:52.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.924+0000 7f066f984640 10 monclient: get_monmap_and_config 2024-09-16T15:00:52.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.924+0000 7f066f984640 10 monclient: build_initial_monmap 2024-09-16T15:00:52.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.924+0000 7f066f984640 10 monclient: monmap: 2024-09-16T15:00:52.929 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:52.929 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:52.925528+0000 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:52.925528+0000 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:52.930 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:52.931 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 Processor -- start 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 -- start start 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient: init 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient: _add_conns ranks=[8,10,0,4,2,6,9,3,5,7,1] 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 0x7f066809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): picked mon.noname-e con 0x7f0668136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0668137220 0x7f0668137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): picked mon.noname-f con 0x7f0668137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0668135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:52.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0668073250 con 0x7f0668135dc0 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066809abc0 con 0x7f0668137220 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f066809ba50 con 0x7f0668136e40 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.925+0000 7f066f984640 10 monclient(hunting): authenticate will time out at 789.514925s 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 0x7f066809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066df21640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0668135dc0 0x7f0668136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38938/0 (socket says 172.21.15.134:38938) 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066df21640 1 -- 172.21.15.134:0/3853088319 learned_addr learned my addr 172.21.15.134:0/3853088319 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): get_auth_request con 0x7f0668136e40 auth_method 0 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066cf1f640 1 --2- 172.21.15.134:0/3853088319 >> v2:172.21.15.162:3300/0 conn(0x7f0668137220 0x7f0668137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066cf1f640 10 monclient(hunting): get_auth_request con 0x7f0668137220 auth_method 0 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066cf1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066cf1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066cf1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient(hunting): handle_auth_done global_id 4328 payload 274 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 -- 172.21.15.134:0/3853088319 >> v2:172.21.15.162:3300/0 conn(0x7f0668137220 msgr2=0x7f0668137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 --2- 172.21.15.134:0/3853088319 >> v2:172.21.15.162:3300/0 conn(0x7f0668137220 0x7f0668137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 -- 172.21.15.134:0/3853088319 >> v1:172.21.15.134:6789/0 conn(0x7f0668135dc0 legacy=0x7f0668136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient: _finish_hunting 0 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 monclient: found mon.noname-e 2024-09-16T15:00:52.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:52.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:52.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.926+0000 7f066d720640 1 -- 172.21.15.134:0/3853088319 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680a53c0 con 0x7f0668136e40 2024-09-16T15:00:52.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066d720640 1 -- 172.21.15.134:0/3853088319 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 msgr2=0x7f066809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:52.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066d720640 1 --2- 172.21.15.134:0/3853088319 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 0x7f066809c150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06640097d0 tx=0x7f066402a040 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cc2aab825a1cbd23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 1 -- 172.21.15.134:0/3853088319 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f066402ad60 con 0x7f0668136e40 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: dump: 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.941 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 1 -- 172.21.15.134:0/3853088319 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06640307b0 con 0x7f0668136e40 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066f984640 5 monclient: authenticate success, global_id 4328 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066f984640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066f984640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 1 -- 172.21.15.134:0/3853088319 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0664030d90 con 0x7f0668136e40 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f066f984640 4 set_mon_vals no callback set 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.929+0000 7f065ffff640 10 monclient: dump: 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.944 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 10 monclient: shutdown 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:52.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 -- 172.21.15.134:0/3853088319 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 msgr2=0x7f066809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 --2- 172.21.15.134:0/3853088319 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 0x7f066809c150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06640097d0 tx=0x7f066402a040 comp rx=0 tx=0).stop 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 -- 172.21.15.134:0/3853088319 shutdown_connections 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 --2- 172.21.15.134:0/3853088319 >> v2:172.21.15.162:3300/0 conn(0x7f0668137220 0x7f0668137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 --2- 172.21.15.134:0/3853088319 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0668136e40 0x7f066809c150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 -- 172.21.15.134:0/3853088319 >> 172.21.15.134:0/3853088319 conn(0x7f06680a6110 msgr2=0x7f06680a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 -- 172.21.15.134:0/3853088319 shutdown_connections 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 1 -- 172.21.15.134:0/3853088319 wait complete. 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 10 monclient: build_initial_monmap 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.930+0000 7f066f984640 10 monclient: monmap: 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 Processor -- start 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 -- start start 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient: init 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient: _add_conns ranks=[1,0,2,4,3,5,8,7,6] 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0668135dc0 0x7f0668134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): picked mon.noname-f con 0x7f0668135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668136e40 0x7f0668135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): picked mon.noname-a con 0x7f0668136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): picked mon.noname-b con 0x7f0668137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0668120d20 con 0x7f0668136e40 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681225f0 con 0x7f0668135dc0 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06681364e0 con 0x7f0668137220 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066f984640 10 monclient(hunting): authenticate will time out at 789.520601s 2024-09-16T15:00:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668136e40 0x7f0668135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58078/0 (socket says 172.21.15.134:58078) 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 1 -- 172.21.15.134:0/3653927706 learned_addr learned my addr 172.21.15.134:0/3653927706 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): get_auth_request con 0x7f0668137220 auth_method 0 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): get_auth_request con 0x7f0668136e40 auth_method 0 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066cf1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066df21640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066d720640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0668135dc0 0x7f0668134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066d720640 10 monclient(hunting): get_auth_request con 0x7f0668135dc0 auth_method 0 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066d720640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066d720640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.931+0000 7f066d720640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 10 monclient(hunting): handle_auth_done global_id 4313 payload 824 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 -- 172.21.15.134:0/3653927706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0668135dc0 msgr2=0x7f0668134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0668135dc0 0x7f0668134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 -- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668136e40 msgr2=0x7f0668135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668136e40 0x7f0668135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 10 monclient: _finish_hunting 0 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 monclient: found mon.noname-b 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.932+0000 7f066df21640 1 -- 172.21.15.134:0/3653927706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664009400 con 0x7f0668137220 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066df21640 1 -- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 msgr2=0x7f06681cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066df21640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f066000caf0 tx=0x7f066000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f066000acb0 con 0x7f0668137220 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: dump: 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.953 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000ae50 con 0x7f0668137220 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 5 monclient: authenticate success, global_id 4313 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 10 monclient: _renew_subs 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680a3560 con 0x7f0668137220 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0660012ce0 con 0x7f0668137220 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065dffb640 10 monclient: dump: 2024-09-16T15:00:52.954 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.955 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f065f7fe640 4 set_mon_vals no callback set 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 10 monclient: _renew_subs 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.934+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0668122310 con 0x7f0668137220 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.935+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0660012400 con 0x7f0668137220 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.935+0000 7f065dffb640 1 --2- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 0x7f064c03ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.935+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f066004a3c0 con 0x7f0668137220 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.935+0000 7f066f984640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.935+0000 7f066f984640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.936+0000 7f066f984640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.936+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066809fc30 con 0x7f0668137220 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.936+0000 7f066d720640 1 --2- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 0x7f064c03ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.936+0000 7f066d720640 10 monclient: get_auth_request con 0x7f064c0388e0 auth_method 0 2024-09-16T15:00:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.938+0000 7f066d720640 1 --2- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 0x7f064c03ada0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f066806b4b0 tx=0x7f066402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.940+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0660012e40 con 0x7f0668137220 2024-09-16T15:00:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.940+0000 7f065dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.940+0000 7f065dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.964+0000 7f92054b1640 10 monclient: get_monmap_and_config 2024-09-16T15:00:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.964+0000 7f92054b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.964+0000 7f92054b1640 10 monclient: monmap: 2024-09-16T15:00:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:52.965301+0000 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:52.965301+0000 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:52.968 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.964+0000 7f92054b1640 1 Processor -- start 2024-09-16T15:00:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 -- start start 2024-09-16T15:00:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient: init 2024-09-16T15:00:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient: _add_conns ranks=[0,4,10,9,2,7,8,6,3,1,5] 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9200136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 0x7f9200137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-c con 0x7f9200137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f920013c370 0x7f920013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-f con 0x7f920013c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9200073250 con 0x7f9200136e40 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000a56e0 con 0x7f920013c370 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f920009abc0 con 0x7f9200137220 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f92054b1640 10 monclient(hunting): authenticate will time out at 789.554512s 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 0x7f9200137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91feffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9200136e40 0x7f920009c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38944/0 (socket says 172.21.15.134:38944) 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91feffd640 1 -- 172.21.15.134:0/3534149992 learned_addr learned my addr 172.21.15.134:0/3534149992 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): get_auth_request con 0x7f9200137220 auth_method 0 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91ff7fe640 1 --2- 172.21.15.134:0/3534149992 >> v2:172.21.15.162:3300/0 conn(0x7f920013c370 0x7f920013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request con 0x7f920013c370 auth_method 0 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_done global_id 4270 payload 274 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 -- 172.21.15.134:0/3534149992 >> v2:172.21.15.162:3300/0 conn(0x7f920013c370 msgr2=0x7f920013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 --2- 172.21.15.134:0/3534149992 >> v2:172.21.15.162:3300/0 conn(0x7f920013c370 0x7f920013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 -- 172.21.15.134:0/3534149992 >> v1:172.21.15.134:6789/0 conn(0x7f9200136e40 legacy=0x7f920009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 monclient: found mon.noname-c 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.965+0000 7f91fe7fc640 1 -- 172.21.15.134:0/3534149992 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920009ba50 con 0x7f9200137220 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fe7fc640 1 -- 172.21.15.134:0/3534149992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 msgr2=0x7f9200137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fe7fc640 1 --2- 172.21.15.134:0/3534149992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 0x7f9200137600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91f00097a0 tx=0x7f91f002a010 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7b370aef48837e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 1 -- 172.21.15.134:0/3534149992 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91f002ad30 con 0x7f9200137220 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: dump: 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 1 -- 172.21.15.134:0/3534149992 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00307b0 con 0x7f9200137220 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f92054b1640 5 monclient: authenticate success, global_id 4270 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f92054b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f92054b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 1 -- 172.21.15.134:0/3534149992 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91f0030d90 con 0x7f9200137220 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f92054b1640 4 set_mon_vals no callback set 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.968+0000 7f91fdffb640 10 monclient: dump: 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 10 monclient: shutdown 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 -- 172.21.15.134:0/3534149992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 msgr2=0x7f9200137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 --2- 172.21.15.134:0/3534149992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 0x7f9200137600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91f00097a0 tx=0x7f91f002a010 comp rx=0 tx=0).stop 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 -- 172.21.15.134:0/3534149992 shutdown_connections 2024-09-16T15:00:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 --2- 172.21.15.134:0/3534149992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9200137220 0x7f9200137600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 --2- 172.21.15.134:0/3534149992 >> v2:172.21.15.162:3300/0 conn(0x7f920013c370 0x7f920013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 -- 172.21.15.134:0/3534149992 >> 172.21.15.134:0/3534149992 conn(0x7f92000a6110 msgr2=0x7f92000a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 -- 172.21.15.134:0/3534149992 shutdown_connections 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.969+0000 7f92054b1640 1 -- 172.21.15.134:0/3534149992 wait complete. 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.970+0000 7f92054b1640 10 monclient: build_initial_monmap 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.970+0000 7f92054b1640 10 monclient: monmap: 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.970+0000 7f92054b1640 1 Processor -- start 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.970+0000 7f92054b1640 1 -- start start 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.970+0000 7f92054b1640 10 monclient: init 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient: _add_conns ranks=[2,8,7,5,1,6,3,4,0] 2024-09-16T15:00:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 0x7f92000a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-b con 0x7f9200136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-e con 0x7f9200137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f920013c370 0x7f9200136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): picked mon.noname-i con 0x7f920013c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9200122430 con 0x7f9200136e40 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9200122960 con 0x7f920013c370 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f92000a7af0 con 0x7f9200137220 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f92054b1640 10 monclient(hunting): authenticate will time out at 789.560205s 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 0x7f92000a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41782/0 (socket says 172.21.15.134:41782) 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 -- 172.21.15.134:0/1817998224 learned_addr learned my addr 172.21.15.134:0/1817998224 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): get_auth_request con 0x7f9200137220 auth_method 0 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): get_auth_request con 0x7f9200136e40 auth_method 0 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f920013c370 0x7f9200136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request con 0x7f920013c370 auth_method 0 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient(hunting): handle_auth_done global_id 4337 payload 824 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 -- 172.21.15.134:0/1817998224 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f920013c370 msgr2=0x7f9200136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f920013c370 0x7f9200136480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 -- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 msgr2=0x7f92000a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 0x7f92000a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 monclient: found mon.noname-e 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91fe7fc640 1 -- 172.21.15.134:0/1817998224 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0009400 con 0x7f9200137220 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.971+0000 7f91feffd640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 0x7f92000a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.973+0000 7f91fe7fc640 1 -- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 msgr2=0x7f9200135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.973+0000 7f91fe7fc640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f00096f0 tx=0x7f91f00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91f00044b0 con 0x7f9200137220 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: dump: 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 5 monclient: authenticate success, global_id 4337 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f0030d40 con 0x7f9200137220 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f91f003d440 con 0x7f9200137220 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91fd7fa640 4 set_mon_vals no callback set 2024-09-16T15:00:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f91e77fe640 10 monclient: dump: 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 10 monclient: _renew_subs 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92000a75a0 con 0x7f9200137220 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 10 monclient: _renew_subs 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.974+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9200122050 con 0x7f9200137220 2024-09-16T15:00:52.992 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f91f0039040 con 0x7f9200137220 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f92054b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f92054b1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f92054b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920009cba0 con 0x7f9200137220 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f91e77fe640 1 --2- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 0x7f91ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f91f006a460 con 0x7f9200137220 2024-09-16T15:00:52.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f91feffd640 1 --2- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 0x7f91ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:52.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.975+0000 7f91feffd640 10 monclient: get_auth_request con 0x7f91ec038880 auth_method 0 2024-09-16T15:00:52.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.978+0000 7f91feffd640 1 --2- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 0x7f91ec03ad40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91f4007c90 tx=0x7f91f4007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:52.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.980+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f91f003a980 con 0x7f9200137220 2024-09-16T15:00:52.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.980+0000 7f91e77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:52.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:52.980+0000 7f91e77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.001 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:00:53.008 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:00:53.009 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:00:53.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 10 monclient: monmap: 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.101006+0000 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.101006+0000 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 1 Processor -- start 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 1 -- start start 2024-09-16T15:00:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.100+0000 7f89103b5640 10 monclient: init 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient: _add_conns ranks=[10,0,7,4,3,8,2,1,5,9,6] 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8908136e40 0x7f890809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-f con 0x7f8908136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8908137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 0x7f890813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-i con 0x7f890813c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8908073250 con 0x7f8908137220 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89080a56e0 con 0x7f8908136e40 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f890809abc0 con 0x7f890813c370 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f89103b5640 10 monclient(hunting): authenticate will time out at 789.690655s 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890d950640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8908137220 0x7f8908137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38958/0 (socket says 172.21.15.134:38958) 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890d950640 1 -- 172.21.15.134:0/2421171812 learned_addr learned my addr 172.21.15.134:0/2421171812 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e151640 1 --2- 172.21.15.134:0/2421171812 >> v2:172.21.15.162:3300/0 conn(0x7f8908136e40 0x7f890809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e952640 1 --2- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 0x7f890813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e952640 10 monclient(hunting): get_auth_request con 0x7f890813c370 auth_method 0 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e952640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.101+0000 7f890e952640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): get_auth_request con 0x7f8908136e40 auth_method 0 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient(hunting): handle_auth_done global_id 4282 payload 274 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 1 -- 172.21.15.134:0/2421171812 >> v2:172.21.15.162:3300/0 conn(0x7f8908136e40 msgr2=0x7f890809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 1 --2- 172.21.15.134:0/2421171812 >> v2:172.21.15.162:3300/0 conn(0x7f8908136e40 0x7f890809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 1 -- 172.21.15.134:0/2421171812 >> v1:172.21.15.134:6789/0 conn(0x7f8908137220 legacy=0x7f8908137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 1 monclient: found mon.noname-i 2024-09-16T15:00:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:53.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.102+0000 7f890e952640 1 -- 172.21.15.134:0/2421171812 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890809ba50 con 0x7f890813c370 2024-09-16T15:00:53.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890e952640 1 -- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 msgr2=0x7f890813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890e952640 1 --2- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 0x7f890813e760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8900009d90 tx=0x7f890002a4e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7731f80f066323d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 1 -- 172.21.15.134:0/2421171812 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8900007bc0 con 0x7f890813c370 2024-09-16T15:00:53.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: dump: 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.114 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 1 -- 172.21.15.134:0/2421171812 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890002d050 con 0x7f890813c370 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 1 -- 172.21.15.134:0/2421171812 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8900007d40 con 0x7f890813c370 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f890d14f640 10 monclient: dump: 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f89103b5640 5 monclient: authenticate success, global_id 4282 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f89103b5640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.105+0000 7f89103b5640 4 set_mon_vals no callback set 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 10 monclient: shutdown 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 -- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 msgr2=0x7f890813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 --2- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 0x7f890813e760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8900009d90 tx=0x7f890002a4e0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 -- 172.21.15.134:0/2421171812 shutdown_connections 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 --2- 172.21.15.134:0/2421171812 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f890813c370 0x7f890813e760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 --2- 172.21.15.134:0/2421171812 >> v2:172.21.15.162:3300/0 conn(0x7f8908136e40 0x7f890809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 -- 172.21.15.134:0/2421171812 >> 172.21.15.134:0/2421171812 conn(0x7f89080a6110 msgr2=0x7f89080a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 -- 172.21.15.134:0/2421171812 shutdown_connections 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.106+0000 7f89103b5640 1 -- 172.21.15.134:0/2421171812 wait complete. 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.107+0000 7f89103b5640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.107+0000 7f89103b5640 10 monclient: monmap: 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.107+0000 7f89103b5640 1 Processor -- start 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.107+0000 7f89103b5640 1 -- start start 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.107+0000 7f89103b5640 10 monclient: init 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient: _add_conns ranks=[6,1,8,4,0,7,3,2,5] 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-d con 0x7f8908136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 0x7f89081201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-f con 0x7f8908137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 0x7f89081206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): picked mon.noname-e con 0x7f890813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8908121dd0 con 0x7f8908137220 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89080a37b0 con 0x7f8908136e40 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f89080a7410 con 0x7f890813c370 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f89103b5640 10 monclient(hunting): authenticate will time out at 789.697362s 2024-09-16T15:00:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42294/0 (socket says 172.21.15.134:42294) 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 1 -- 172.21.15.134:0/1732064025 learned_addr learned my addr 172.21.15.134:0/1732064025 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): get_auth_request con 0x7f8908136e40 auth_method 0 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 0x7f89081201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 0x7f89081206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): get_auth_request con 0x7f8908137220 auth_method 0 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): get_auth_request con 0x7f890813c370 auth_method 0 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient(hunting): handle_auth_done global_id 4346 payload 824 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 -- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 msgr2=0x7f890811fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 -- 172.21.15.134:0/1732064025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 msgr2=0x7f89081201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 0x7f89081201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 monclient: found mon.noname-e 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e952640 1 -- 172.21.15.134:0/1732064025 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8900009a70 con 0x7f890813c370 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890d950640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 0x7f89081201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.108+0000 7f890e151640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f890e952640 1 -- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 msgr2=0x7f89081206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f890e952640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 0x7f89081206e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8900031000 tx=0x7f8900004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8900042e80 con 0x7f890813c370 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 10 monclient: dump: 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f88feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f89103b5640 5 monclient: authenticate success, global_id 4346 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f89103b5640 10 monclient: _renew_subs 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f89103b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8908122670 con 0x7f890813c370 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.111+0000 7f89103b5640 10 monclient: _renew_subs 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f89103b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89080a78f0 con 0x7f890813c370 2024-09-16T15:00:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900007c10 con 0x7f890813c370 2024-09-16T15:00:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8900007d80 con 0x7f890813c370 2024-09-16T15:00:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f88feffd640 10 monclient: dump: 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.112+0000 7f890c94e640 4 set_mon_vals no callback set 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.113+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8900048070 con 0x7f890813c370 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.113+0000 7f88feffd640 1 --2- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 0x7f88ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.113+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f8900073570 con 0x7f890813c370 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f89103b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f890e151640 1 --2- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 0x7f88ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f89103b5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f89103b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8908121f80 con 0x7f890813c370 2024-09-16T15:00:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.114+0000 7f890e151640 10 monclient: get_auth_request con 0x7f88ec038880 auth_method 0 2024-09-16T15:00:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.116+0000 7f890e151640 1 --2- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 0x7f88ec03ad40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89040112c0 tx=0x7f8904009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.119+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8900052990 con 0x7f890813c370 2024-09-16T15:00:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.119+0000 7f88feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.119+0000 7f88feffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.163+0000 7fc5480b8640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.163+0000 7fc5480b8640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.163+0000 7fc5480b8640 10 monclient: monmap: 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.164628+0000 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.164628+0000 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.165 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.166 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.164+0000 7fc5480b8640 1 Processor -- start 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.164+0000 7fc5480b8640 1 -- start start 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.164+0000 7fc5480b8640 10 monclient: init 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient: _add_conns ranks=[8,3,1,9,7,0,2,10,5,4,6] 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 0x7fc54009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-e con 0x7fc540136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-g con 0x7fc540137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc540135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc540073250 con 0x7fc540135dc0 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54009abc0 con 0x7fc540137220 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc54009ba50 con 0x7fc540136e40 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc5480b8640 10 monclient(hunting): authenticate will time out at 789.754499s 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 0x7fc54009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:32854/0 (socket says 172.21.15.134:32854) 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 0x7fc54009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41798/0 (socket says 172.21.15.134:41798) 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 1 -- 172.21.15.134:0/1844656888 learned_addr learned my addr 172.21.15.134:0/1844656888 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): get_auth_request con 0x7fc540137220 auth_method 0 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 10 monclient(hunting): get_auth_request con 0x7fc540136e40 auth_method 0 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545e54640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.165+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 10 monclient(hunting): handle_auth_done global_id 4350 payload 274 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 1 -- 172.21.15.134:0/1844656888 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 msgr2=0x7fc54009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 1 --2- 172.21.15.134:0/1844656888 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 0x7fc54009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 1 -- 172.21.15.134:0/1844656888 >> v1:172.21.15.162:6789/0 conn(0x7fc540135dc0 legacy=0x7fc540136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 1 monclient: found mon.noname-g 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.166+0000 7fc545653640 1 -- 172.21.15.134:0/1844656888 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400a53c0 con 0x7fc540137220 2024-09-16T15:00:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc545653640 1 -- 172.21.15.134:0/1844656888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 msgr2=0x7fc540137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc545653640 1 --2- 172.21.15.134:0/1844656888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098a0 tx=0x7fc53002a0d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3ccf21bc9224394f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 1 -- 172.21.15.134:0/1844656888 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc53002adf0 con 0x7fc540137220 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: dump: 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 1 -- 172.21.15.134:0/1844656888 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5300307b0 con 0x7fc540137220 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc5480b8640 5 monclient: authenticate success, global_id 4350 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc5480b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc5480b8640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 1 -- 172.21.15.134:0/1844656888 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc530030d90 con 0x7fc540137220 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc5480b8640 4 set_mon_vals no callback set 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.169+0000 7fc544e52640 10 monclient: dump: 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 10 monclient: shutdown 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 -- 172.21.15.134:0/1844656888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 msgr2=0x7fc540137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 --2- 172.21.15.134:0/1844656888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5300098a0 tx=0x7fc53002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 -- 172.21.15.134:0/1844656888 shutdown_connections 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 --2- 172.21.15.134:0/1844656888 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc540137220 0x7fc540137600 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 --2- 172.21.15.134:0/1844656888 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540136e40 0x7fc54009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 -- 172.21.15.134:0/1844656888 >> 172.21.15.134:0/1844656888 conn(0x7fc5400a6110 msgr2=0x7fc5400a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 -- 172.21.15.134:0/1844656888 shutdown_connections 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.170+0000 7fc5480b8640 1 -- 172.21.15.134:0/1844656888 wait complete. 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.171+0000 7fc5480b8640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.171+0000 7fc5480b8640 10 monclient: monmap: 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.171+0000 7fc5480b8640 1 Processor -- start 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 1 -- start start 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 10 monclient: init 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 10 monclient: _add_conns ranks=[1,6,8,5,0,4,7,3,2] 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-f con 0x7fc540135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-d con 0x7fc540136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc5480b8640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc545653640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.172+0000 7fc545653640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42308/0 (socket says 172.21.15.134:42308) 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:55732/0 (socket says 172.21.15.134:55732) 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41806/0 (socket says 172.21.15.134:41806) 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 -- 172.21.15.134:0/573625442 learned_addr learned my addr 172.21.15.134:0/573625442 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): picked mon.noname-e con 0x7fc540137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5401254c0 con 0x7fc540135dc0 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc540136980 con 0x7fc540136e40 2024-09-16T15:00:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc540136450 con 0x7fc540137220 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc5480b8640 10 monclient(hunting): authenticate will time out at 789.762184s 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): get_auth_request con 0x7fc540136e40 auth_method 0 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): get_auth_request con 0x7fc540135dc0 auth_method 0 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): get_auth_request con 0x7fc540137220 auth_method 0 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545e54640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient(hunting): handle_auth_done global_id 4281 payload 824 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 -- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 msgr2=0x7fc540121970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 -- 172.21.15.134:0/573625442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 msgr2=0x7fc540134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 monclient: found mon.noname-d 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc545653640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300094d0 con 0x7fc540136e40 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.173+0000 7fc546655640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.174+0000 7fc545e54640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc545653640 1 -- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 msgr2=0x7fc5401354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc545653640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5300090b0 tx=0x7fc5300046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc530004490 con 0x7fc540136e40 2024-09-16T15:00:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: dump: 2024-09-16T15:00:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530003610 con 0x7fc540136e40 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5480b8640 5 monclient: authenticate success, global_id 4281 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc530003bf0 con 0x7fc540136e40 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc5367fc640 10 monclient: dump: 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.176+0000 7fc537fff640 4 set_mon_vals no callback set 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 10 monclient: _renew_subs 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5401cc1d0 con 0x7fc540136e40 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 10 monclient: _renew_subs 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.177+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5400a8aa0 con 0x7fc540136e40 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.178+0000 7fc5480b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.178+0000 7fc5480b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.178+0000 7fc5480b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.178+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc540123b50 con 0x7fc540136e40 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.178+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc530039070 con 0x7fc540136e40 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.179+0000 7fc5367fc640 1 --2- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 0x7fc52403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.179+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fc53006b100 con 0x7fc540136e40 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.179+0000 7fc545e54640 1 --2- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 0x7fc52403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.179+0000 7fc545e54640 10 monclient: get_auth_request con 0x7fc524038880 auth_method 0 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f066f984640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f066f984640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f066f984640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f066809b770 con 0x7f0668137220 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7fc545e54640 1 --2- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 0x7fc52403ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009f30 tx=0x7fc53c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f065dffb640 1 -- 172.21.15.134:0/3653927706 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0660019300 con 0x7f0668137220 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f065dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-16T15:00:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.182+0000 7f065dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.197 INFO:teuthology.orchestra.run.smithi134.stdout:55834574853 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 msgr2=0x7f064c03ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 0x7f064c03ada0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f066806b4b0 tx=0x7f066402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 10 monclient: shutdown 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 msgr2=0x7f06681cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f066000caf0 tx=0x7f066000a060 comp rx=0 tx=0).stop 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 shutdown_connections 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> v2:172.21.15.162:6800/980547263 conn(0x7f064c0388e0 0x7f064c03ada0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0668137220 0x7f06681cb3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0668136e40 0x7f0668135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 --2- 172.21.15.134:0/3653927706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0668135dc0 0x7f0668134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 >> 172.21.15.134:0/3653927706 conn(0x7f06680a6110 msgr2=0x7f066809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 shutdown_connections 2024-09-16T15:00:53.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.185+0000 7f066f984640 1 -- 172.21.15.134:0/3653927706 wait complete. 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.187+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc53003b420 con 0x7fc540136e40 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.187+0000 7fc5367fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.187+0000 7fc5367fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.192+0000 7f7c0759e640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.192+0000 7f7c0759e640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.193+0000 7f7c0759e640 10 monclient: monmap: 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.193952+0000 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.193952+0000 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.201 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.193+0000 7f7c0759e640 1 Processor -- start 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.194+0000 7fba35bbc640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.194+0000 7fba35bbc640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.194+0000 7fba35bbc640 10 monclient: monmap: 2024-09-16T15:00:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.195073+0000 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.195073+0000 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.204 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.194+0000 7f7c0759e640 1 -- start start 2024-09-16T15:00:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.194+0000 7f7c0759e640 10 monclient: init 2024-09-16T15:00:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 1 Processor -- start 2024-09-16T15:00:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 1 -- start start 2024-09-16T15:00:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 10 monclient: init 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 10 monclient: _add_conns ranks=[2,8,10,6,0,4,5,7,3,9,1] 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-b con 0x7f7c0002a880 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 0x7f7c00022d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-e con 0x7f7c00028ba0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c000232a0 0x7f7c00023750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-f con 0x7f7c000232a0 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c00024310 con 0x7f7c000232a0 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58102/0 (socket says 172.21.15.134:58102) 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 1 -- 172.21.15.134:0/3153675557 learned_addr learned my addr 172.21.15.134:0/3153675557 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0002ac60 con 0x7f7c0002a880 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 0x7f7c00022d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 10 monclient: _add_conns ranks=[2,3,6,9,8,10,1,4,0,7,5] 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba30065f90 0x7fba30066370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.195+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-b con 0x7fba30065f90 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 0x7fba3013b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-g con 0x7fba300668b0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 0x7fba3013df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-d con 0x7fba3013bb50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300a0900 con 0x7fba30065f90 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3009e580 con 0x7fba300668b0 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fba30067260 con 0x7fba3013bb50 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba35bbc640 10 monclient(hunting): authenticate will time out at 789.785090s 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 0x7fba3013df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 0x7fba3013df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42316/0 (socket says 172.21.15.134:42316) 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 1 -- 172.21.15.134:0/1124830213 learned_addr learned my addr 172.21.15.134:0/1124830213 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): get_auth_request con 0x7fba3013bb50 auth_method 0 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 0x7fba3013b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): get_auth_request con 0x7fba300668b0 auth_method 0 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c06d9d640 1 --2- 172.21.15.134:0/3153675557 >> v2:172.21.15.162:3300/0 conn(0x7f7c000232a0 0x7f7c00023750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba30065f90 0x7fba30066370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): get_auth_request con 0x7fba30065f90 auth_method 0 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000245e0 con 0x7f7c00028ba0 2024-09-16T15:00:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient(hunting): handle_auth_done global_id 4359 payload 274 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 -- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 msgr2=0x7fba3013df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 0x7fba3013df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 -- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba30065f90 msgr2=0x7fba30066370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba30065f90 0x7fba30066370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0759e640 10 monclient(hunting): authenticate will time out at 789.785791s 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 monclient: found mon.noname-g 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7fba2effd640 1 -- 172.21.15.134:0/1124830213 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30073250 con 0x7fba300668b0 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 10 monclient(hunting): get_auth_request con 0x7f7c0002a880 auth_method 0 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c0659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c05d9b640 10 monclient(hunting): get_auth_request con 0x7f7c00028ba0 auth_method 0 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c05d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c05d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c05d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c06d9d640 10 monclient(hunting): get_auth_request con 0x7f7c000232a0 auth_method 0 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c06d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c06d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.196+0000 7f7c06d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_done global_id 4373 payload 274 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 -- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 msgr2=0x7f7c00028660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 -- 172.21.15.134:0/3153675557 >> v2:172.21.15.162:3300/0 conn(0x7f7c000232a0 msgr2=0x7f7c00023750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/3153675557 >> v2:172.21.15.162:3300/0 conn(0x7f7c000232a0 0x7f7c00023750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 monclient: found mon.noname-e 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c05d9b640 1 -- 172.21.15.134:0/3153675557 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000042f0 con 0x7f7c00028ba0 2024-09-16T15:00:53.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.197+0000 7f7c0659c640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-09-16T15:00:53.218 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2effd640 1 -- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 msgr2=0x7fba3013b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2effd640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 0x7fba3013b610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba240097d0 tx=0x7fba24029f90 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3b006791c31e6606 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 1 -- 172.21.15.134:0/1124830213 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba2402acf0 con 0x7fba300668b0 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: dump: 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 1 -- 172.21.15.134:0/1124830213 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2402ae70 con 0x7fba300668b0 2024-09-16T15:00:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 1 -- 172.21.15.134:0/1124830213 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba24030c20 con 0x7fba300668b0 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7fba2e7fc640 10 monclient: dump: 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.222 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7f7c05d9b640 1 -- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 msgr2=0x7f7c00022d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.200+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 0x7f7c00022d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097a0 tx=0x7f7bfc029fd0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=277bed8d54627d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 1 -- 172.21.15.134:0/3153675557 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc02ad30 con 0x7f7c00028ba0 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: dump: 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.224 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.225 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 1 -- 172.21.15.134:0/3153675557 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc0307b0 con 0x7f7c00028ba0 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 1 -- 172.21.15.134:0/3153675557 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc030dd0 con 0x7f7c00028ba0 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0559a640 10 monclient: dump: 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.226 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0759e640 5 monclient: authenticate success, global_id 4373 2024-09-16T15:00:53.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0759e640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.201+0000 7f7c0759e640 4 set_mon_vals no callback set 2024-09-16T15:00:53.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 5 monclient: authenticate success, global_id 4359 2024-09-16T15:00:53.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 4 set_mon_vals no callback set 2024-09-16T15:00:53.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 10 monclient: shutdown 2024-09-16T15:00:53.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 msgr2=0x7f7c00022d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 0x7f7c00022d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097a0 tx=0x7f7bfc029fd0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 10 monclient: shutdown 2024-09-16T15:00:53.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 shutdown_connections 2024-09-16T15:00:53.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 --2- 172.21.15.134:0/3153675557 >> v2:172.21.15.162:3300/0 conn(0x7f7c000232a0 0x7f7c00023750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c00028ba0 0x7f7c00022d60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 --2- 172.21.15.134:0/3153675557 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c0002a880 0x7f7c00028660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 >> 172.21.15.134:0/3153675557 conn(0x7f7c00007bf0 msgr2=0x7f7c00008000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 shutdown_connections 2024-09-16T15:00:53.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7f7c0759e640 1 -- 172.21.15.134:0/3153675557 wait complete. 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 1 -- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 msgr2=0x7fba3013b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.202+0000 7fba35bbc640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 0x7fba3013b610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fba240097d0 tx=0x7fba24029f90 comp rx=0 tx=0).stop 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 -- 172.21.15.134:0/1124830213 shutdown_connections 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fba3013bb50 0x7fba3013df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba300668b0 0x7fba3013b610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 --2- 172.21.15.134:0/1124830213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba30065f90 0x7fba30066370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 -- 172.21.15.134:0/1124830213 >> 172.21.15.134:0/1124830213 conn(0x7fba300a6470 msgr2=0x7fba300a6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 -- 172.21.15.134:0/1124830213 shutdown_connections 2024-09-16T15:00:53.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.203+0000 7fba35bbc640 1 -- 172.21.15.134:0/1124830213 wait complete. 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7f7c0759e640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7f7c0759e640 10 monclient: monmap: 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.233 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7fba35bbc640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7fba35bbc640 10 monclient: monmap: 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.235 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7f7c0759e640 1 Processor -- start 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7f7c0759e640 1 -- start start 2024-09-16T15:00:53.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7f7c0759e640 10 monclient: init 2024-09-16T15:00:53.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.204+0000 7fba35bbc640 1 Processor -- start 2024-09-16T15:00:53.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 1 -- start start 2024-09-16T15:00:53.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 10 monclient: init 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient: _add_conns ranks=[8,2,3,5,7,6,4,1,0] 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-e con 0x7f7c000232a0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41828/0 (socket says 172.21.15.134:41828) 2024-09-16T15:00:53.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c05d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 10 monclient: _add_conns ranks=[4,2,8,7,1,3,0,6,5] 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-c con 0x7fba30065f90 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba300668b0 0x7fba300985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-b con 0x7fba300668b0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba35bbc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fba3013bb50 0x7fba30098ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7fba2f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): picked mon.noname-e con 0x7fba3013bb50 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba300668b0 0x7fba300985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33394/0 (socket says 172.21.15.134:33394) 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 -- 172.21.15.134:0/4144854629 learned_addr learned my addr 172.21.15.134:0/4144854629 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3013e170 con 0x7fba300668b0 2024-09-16T15:00:53.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba30071bd0 con 0x7fba30065f90 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fba30137180 con 0x7fba3013bb50 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba35bbc640 10 monclient(hunting): authenticate will time out at 789.795074s 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): get_auth_request con 0x7fba30065f90 auth_method 0 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2effd640 10 monclient(hunting): get_auth_request con 0x7fba300668b0 auth_method 0 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient(hunting): handle_auth_done global_id 4279 payload 824 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 -- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fba3013bb50 msgr2=0x7fba30098ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fba3013bb50 0x7fba30098ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 -- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba300668b0 msgr2=0x7fba300985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba300668b0 0x7fba300985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 monclient: found mon.noname-c 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7fba2f7fe640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009400 con 0x7fba30065f90 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c05d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58110/0 (socket says 172.21.15.134:58110) 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 1 -- 172.21.15.134:0/2793324971 learned_addr learned my addr 172.21.15.134:0/2793324971 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-b con 0x7f7c00028ba0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0002a880 0x7f7c000b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): picked mon.noname-g con 0x7f7c0002a880 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001521b0 con 0x7f7c00028ba0 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000a71c0 con 0x7f7c0002a880 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000a7410 con 0x7f7c000232a0 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0759e640 10 monclient(hunting): authenticate will time out at 789.794940s 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 10 monclient(hunting): get_auth_request con 0x7f7c000232a0 auth_method 0 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.205+0000 7f7c0659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): get_auth_request con 0x7f7c00028ba0 auth_method 0 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c06d9d640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0002a880 0x7f7c000b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c06d9d640 10 monclient(hunting): get_auth_request con 0x7f7c0002a880 auth_method 0 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c06d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c06d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c06d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient(hunting): handle_auth_done global_id 4382 payload 824 2024-09-16T15:00:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 -- 172.21.15.134:0/2793324971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0002a880 msgr2=0x7f7c000b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0002a880 0x7f7c000b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 -- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 msgr2=0x7f7c000b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 monclient: found mon.noname-e 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c0659c640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80094d0 con 0x7f7c000232a0 2024-09-16T15:00:53.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.206+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.208+0000 7fba2f7fe640 1 -- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 msgr2=0x7fba30098060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.208+0000 7fba2f7fe640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba2000e930 tx=0x7fba2000edf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba20009690 con 0x7fba30065f90 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: dump: 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.247 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2000cc10 con 0x7fba30065f90 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fba2000cdc0 con 0x7fba30065f90 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba17fff640 10 monclient: dump: 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.249 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.250 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7c0659c640 1 -- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 msgr2=0x7f7c00151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7c0659c640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9e0 tx=0x7f7bf800eea0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf800cbe0 con 0x7f7c000232a0 2024-09-16T15:00:53.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: dump: 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.252 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf800cd60 con 0x7f7c000232a0 2024-09-16T15:00:53.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba35bbc640 5 monclient: authenticate success, global_id 4279 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba35bbc640 10 monclient: _renew_subs 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba35bbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba30137460 con 0x7fba30065f90 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7fba2dffb640 4 set_mon_vals no callback set 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7fba35bbc640 10 monclient: _renew_subs 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7fba35bbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba30099290 con 0x7fba30065f90 2024-09-16T15:00:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf800ced0 con 0x7f7c000232a0 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7bf6ffd640 10 monclient: dump: 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.209+0000 7f7c04d99640 4 set_mon_vals no callback set 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 5 monclient: authenticate success, global_id 4382 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 10 monclient: _renew_subs 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc009400 con 0x7f7c000232a0 2024-09-16T15:00:53.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 10 monclient: _renew_subs 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.210+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c00007fd0 con 0x7f7c000232a0 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.211+0000 7fba35bbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.211+0000 7fba35bbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.211+0000 7fba35bbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.211+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3009e2a0 con 0x7fba30065f90 2024-09-16T15:00:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c0759e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fba20013470 con 0x7fba30065f90 2024-09-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c0759e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c0759e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c000a7550 con 0x7f7c000232a0 2024-09-16T15:00:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7fba17fff640 1 --2- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 0x7fba1c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fba2004a2e0 con 0x7fba30065f90 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7fba2effd640 1 --2- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 0x7fba1c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7bf8024480 con 0x7f7c000232a0 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7bf6ffd640 1 --2- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f7bf805b200 con 0x7f7c000232a0 2024-09-16T15:00:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7f7c05d9b640 10 monclient: get_auth_request con 0x7f7be8038880 auth_method 0 2024-09-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.212+0000 7fba2effd640 10 monclient: get_auth_request con 0x7fba1c038880 auth_method 0 2024-09-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.215+0000 7f7c05d9b640 1 --2- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.215+0000 7fba2effd640 1 --2- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 0x7fba1c03ad40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba24031000 tx=0x7fba2402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.217+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f7c000a7550 con 0x7f7c000232a0 2024-09-16T15:00:53.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.217+0000 7f7bf6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.217+0000 7f7bf6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.220+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fba2001bd40 con 0x7fba30065f90 2024-09-16T15:00:53.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.220+0000 7fba17fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.220+0000 7fba17fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.233+0000 7f92054b1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.233+0000 7f92054b1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.233+0000 7f92054b1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.233+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f920009b490 con 0x7f9200137220 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.234+0000 7f91e77fe640 1 -- 172.21.15.134:0/1817998224 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91f0041090 con 0x7f9200137220 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.234+0000 7f91e77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.234+0000 7f91e77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.266 INFO:teuthology.orchestra.run.smithi134.stdout:47244640259 2024-09-16T15:00:53.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 msgr2=0x7f91ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 0x7f91ec03ad40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91f4007c90 tx=0x7f91f4007be0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 10 monclient: shutdown 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 msgr2=0x7f9200135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91f00096f0 tx=0x7f91f00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 shutdown_connections 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> v2:172.21.15.162:6800/980547263 conn(0x7f91ec038880 0x7f91ec03ad40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f920013c370 0x7f9200136480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9200137220 0x7f9200135f40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 --2- 172.21.15.134:0/1817998224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9200136e40 0x7f92000a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 >> 172.21.15.134:0/1817998224 conn(0x7f92000a6110 msgr2=0x7f920013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 shutdown_connections 2024-09-16T15:00:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.237+0000 7f92054b1640 1 -- 172.21.15.134:0/1817998224 wait complete. 2024-09-16T15:00:53.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.4 2024-09-16T15:00:53.270 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f89103b5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f89103b5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f89103b5640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f890813b410 con 0x7f890813c370 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f88feffd640 1 -- 172.21.15.134:0/1732064025 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f89000523a0 con 0x7f890813c370 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f88feffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.328+0000 7f88feffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.330 INFO:teuthology.orchestra.run.smithi134.stdout:55834574853 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 msgr2=0x7f88ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 0x7f88ec03ad40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89040112c0 tx=0x7f8904009190 comp rx=0 tx=0).stop 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 10 monclient: shutdown 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 msgr2=0x7f89081206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 0x7f89081206e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8900031000 tx=0x7f8900004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 shutdown_connections 2024-09-16T15:00:53.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> v2:172.21.15.162:6800/980547263 conn(0x7f88ec038880 0x7f88ec03ad40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f890813c370 0x7f89081206e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8908137220 0x7f89081201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 --2- 172.21.15.134:0/1732064025 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8908136e40 0x7f890811fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.330+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 >> 172.21.15.134:0/1732064025 conn(0x7f89080a6110 msgr2=0x7f890813b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.331+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 shutdown_connections 2024-09-16T15:00:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.331+0000 7f89103b5640 1 -- 172.21.15.134:0/1732064025 wait complete. 2024-09-16T15:00:53.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-09-16T15:00:53.347 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5480b8640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5480b8640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5480b8640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc540123b50 con 0x7fc540136e40 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5367fc640 1 -- 172.21.15.134:0/573625442 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc530037020 con 0x7fc540136e40 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5367fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.372+0000 7fc5367fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.374 INFO:teuthology.orchestra.run.smithi134.stdout:47244640259 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 msgr2=0x7fc52403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 0x7fc52403ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009f30 tx=0x7fc53c009190 comp rx=0 tx=0).stop 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 10 monclient: shutdown 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 msgr2=0x7fc5401354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc5300090b0 tx=0x7fc5300046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 shutdown_connections 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> v2:172.21.15.162:6800/980547263 conn(0x7fc524038880 0x7fc52403ad40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc540137220 0x7fc540121970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc540136e40 0x7fc5401354f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 --2- 172.21.15.134:0/573625442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc540135dc0 0x7fc540134fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 >> 172.21.15.134:0/573625442 conn(0x7fc5400a6110 msgr2=0x7fc5400a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 shutdown_connections 2024-09-16T15:00:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.374+0000 7fc5480b8640 1 -- 172.21.15.134:0/573625442 wait complete. 2024-09-16T15:00:53.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.0 2024-09-16T15:00:53.386 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7c0759e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7c0759e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7c0759e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7c000a7550 con 0x7f7c000232a0 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7bf6ffd640 1 -- 172.21.15.134:0/2793324971 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c000a7550 con 0x7f7c000232a0 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7bf6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.391+0000 7f7bf6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.393 INFO:teuthology.orchestra.run.smithi134.stdout:55834574852 2024-09-16T15:00:53.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 msgr2=0x7f7be803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02e000 comp rx=0 tx=0).stop 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 10 monclient: shutdown 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 msgr2=0x7f7c00151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800e9e0 tx=0x7f7bf800eea0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 shutdown_connections 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c0002a880 0x7f7c000b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c00028ba0 0x7f7c000b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 --2- 172.21.15.134:0/2793324971 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c000232a0 0x7f7c00151460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 >> 172.21.15.134:0/2793324971 conn(0x7f7c00007bf0 msgr2=0x7f7c0002e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 shutdown_connections 2024-09-16T15:00:53.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.393+0000 7f7c0759e640 1 -- 172.21.15.134:0/2793324971 wait complete. 2024-09-16T15:00:53.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-09-16T15:00:53.405 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.411+0000 7fba35bbc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.411+0000 7fba35bbc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.411+0000 7fba35bbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.411+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fba3006bb20 con 0x7fba30065f90 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.412+0000 7fba17fff640 1 -- 172.21.15.134:0/4144854629 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba20017070 con 0x7fba30065f90 2024-09-16T15:00:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.412+0000 7fba17fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-16T15:00:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.412+0000 7fba17fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:53.415 INFO:teuthology.orchestra.run.smithi134.stdout:55834574854 2024-09-16T15:00:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 msgr2=0x7fba1c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 0x7fba1c03ad40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fba24031000 tx=0x7fba2402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 10 monclient: shutdown 2024-09-16T15:00:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 msgr2=0x7fba30098060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba2000e930 tx=0x7fba2000edf0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 shutdown_connections 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> v2:172.21.15.162:6800/980547263 conn(0x7fba1c038880 0x7fba1c03ad40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fba3013bb50 0x7fba30098ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba300668b0 0x7fba300985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 --2- 172.21.15.134:0/4144854629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fba30065f90 0x7fba30098060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 >> 172.21.15.134:0/4144854629 conn(0x7fba300a6470 msgr2=0x7fba3009f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 shutdown_connections 2024-09-16T15:00:53.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.414+0000 7fba35bbc640 1 -- 172.21.15.134:0/4144854629 wait complete. 2024-09-16T15:00:53.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-09-16T15:00:53.426 DEBUG:teuthology.parallel:result is None 2024-09-16T15:00:53.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:00:53.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.560+0000 7f8ab50e9640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.560+0000 7f8ab50e9640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.560+0000 7f8ab50e9640 10 monclient: monmap: 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.562 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.561873+0000 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.561873+0000 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.563 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.564 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 1 Processor -- start 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 1 -- start start 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 10 monclient: init 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 10 monclient: _add_conns ranks=[0,5,6,2,4,7,8,10,1,9,3] 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.561+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ab009fda0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 0x7f8ab0139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-h con 0x7f8ab009c630 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 0x7f8ab013c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-d con 0x7f8ab0135060 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab0073250 con 0x7f8ab009fda0 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00a0890 con 0x7f8ab009c630 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab009d2e0 con 0x7f8ab0135060 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8ab50e9640 10 monclient(hunting): authenticate will time out at 790.151085s 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaed9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8ab009fda0 0x7f8ab00a0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38970/0 (socket says 172.21.15.134:38970) 2024-09-16T15:00:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaed9d640 1 -- 172.21.15.134:0/934837218 learned_addr learned my addr 172.21.15.134:0/934837218 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 0x7f8ab013c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): get_auth_request con 0x7f8ab0135060 auth_method 0 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aae59c640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 0x7f8ab0139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient(hunting): handle_auth_done global_id 4299 payload 274 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 -- 172.21.15.134:0/934837218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 msgr2=0x7f8ab0139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 0x7f8ab0139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 -- 172.21.15.134:0/934837218 >> v1:172.21.15.134:6789/0 conn(0x7f8ab009fda0 legacy=0x7f8ab00a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 monclient: found mon.noname-d 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aaf59e640 1 -- 172.21.15.134:0/934837218 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0135a80 con 0x7f8ab0135060 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aae59c640 10 monclient: get_auth_request con 0x7f8ab009c630 auth_method 0 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.562+0000 7f8aae59c640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 0x7f8ab0139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aaf59e640 1 -- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 msgr2=0x7f8ab013c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aaf59e640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 0x7f8ab013c3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009dc0 tx=0x7f8aa802a4d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=eb5fff3ea9608723 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 1 -- 172.21.15.134:0/934837218 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa8007ca0 con 0x7f8ab0135060 2024-09-16T15:00:53.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: dump: 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.574 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 5 monclient: authenticate success, global_id 4299 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 1 -- 172.21.15.134:0/934837218 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa802d050 con 0x7f8ab0135060 2024-09-16T15:00:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 1 -- 172.21.15.134:0/934837218 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa8007e60 con 0x7f8ab0135060 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 4 set_mon_vals no callback set 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8aadd9b640 10 monclient: dump: 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 10 monclient: shutdown 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 -- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 msgr2=0x7f8ab013c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 0x7f8ab013c3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009dc0 tx=0x7f8aa802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 -- 172.21.15.134:0/934837218 shutdown_connections 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab0135060 0x7f8ab013c3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/934837218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009c630 0x7f8ab0139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 -- 172.21.15.134:0/934837218 >> 172.21.15.134:0/934837218 conn(0x7f8ab00a2e20 msgr2=0x7f8ab00a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 -- 172.21.15.134:0/934837218 shutdown_connections 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.565+0000 7f8ab50e9640 1 -- 172.21.15.134:0/934837218 wait complete. 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.566+0000 7f8ab50e9640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.566+0000 7f8ab50e9640 10 monclient: monmap: 2024-09-16T15:00:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 Processor -- start 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 -- start start 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient: init 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient: _add_conns ranks=[6,5,7,2,0,3,4,1,8] 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-d con 0x7f8ab009c630 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009fda0 0x7f8ab0133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-h con 0x7f8ab009fda0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8ab0135060 0x7f8ab012b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): picked mon.noname-i con 0x7f8ab0135060 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab006a9a0 con 0x7f8ab009fda0 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab009fac0 con 0x7f8ab009c630 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab012f110 con 0x7f8ab0135060 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8ab50e9640 10 monclient(hunting): authenticate will time out at 790.156629s 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42342/0 (socket says 172.21.15.134:42342) 2024-09-16T15:00:53.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 1 -- 172.21.15.134:0/1135529024 learned_addr learned my addr 172.21.15.134:0/1135529024 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): get_auth_request con 0x7f8ab009c630 auth_method 0 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aae59c640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009fda0 0x7f8ab0133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaf59e640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8ab0135060 0x7f8ab012b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aae59c640 10 monclient(hunting): get_auth_request con 0x7f8ab009fda0 auth_method 0 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aae59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaf59e640 10 monclient(hunting): get_auth_request con 0x7f8ab0135060 auth_method 0 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.567+0000 7f8aaed9d640 10 monclient(hunting): handle_auth_done global_id 4308 payload 824 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 -- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8ab0135060 msgr2=0x7f8ab012b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8ab0135060 0x7f8ab012b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 -- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009fda0 msgr2=0x7f8ab0133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009fda0 0x7f8ab0133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 monclient: found mon.noname-d 2024-09-16T15:00:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.568+0000 7f8aaed9d640 1 -- 172.21.15.134:0/1135529024 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8009a70 con 0x7f8ab009c630 2024-09-16T15:00:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8aaed9d640 1 -- 172.21.15.134:0/1135529024 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 msgr2=0x7f8ab0133950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8aaed9d640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c460 tx=0x7f8aa000c920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa0005cf0 con 0x7f8ab009c630 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: dump: 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0005e90 con 0x7f8ab009c630 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 5 monclient: authenticate success, global_id 4308 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 10 monclient: _renew_subs 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab01ca110 con 0x7f8ab009c630 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8aa00074e0 con 0x7f8ab009c630 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8a977fe640 10 monclient: dump: 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 10 monclient: _renew_subs 2024-09-16T15:00:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8aad59a640 4 set_mon_vals no callback set 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.570+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab012d0b0 con 0x7f8ab009c630 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8aa0007640 con 0x7f8ab009c630 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8ab50e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8ab50e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8ab50e9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab012ece0 con 0x7f8ab009c630 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8a977fe640 1 --2- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 0x7f8a9c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f8aa0049a70 con 0x7f8ab009c630 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8aae59c640 1 --2- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 0x7f8a9c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.571+0000 7f8aae59c640 10 monclient: get_auth_request con 0x7f8a9c038880 auth_method 0 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.574+0000 7f8aae59c640 1 --2- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 0x7f8a9c03ad40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0072280 tx=0x7f8aa4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.575+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8aa0019cd0 con 0x7f8ab009c630 2024-09-16T15:00:53.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.576+0000 7f8a977fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.576+0000 7f8a977fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:53.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.738+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ab0061ae0 con 0x7f8a9c038880 2024-09-16T15:00:53.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.741+0000 7f8a977fe640 1 -- 172.21.15.134:0/1135529024 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41040 (secure 0 0 0) 0x7f8aa400a1f0 con 0x7f8a9c038880 2024-09-16T15:00:53.743 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:53.743 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 msgr2=0x7f8a9c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 0x7f8a9c03ad40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0072280 tx=0x7f8aa4008040 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 10 monclient: shutdown 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 msgr2=0x7f8ab0133950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c460 tx=0x7f8aa000c920 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 shutdown_connections 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> v2:172.21.15.162:6800/980547263 conn(0x7f8a9c038880 0x7f8a9c03ad40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8ab0135060 0x7f8ab012b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8ab009fda0 0x7f8ab0133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 --2- 172.21.15.134:0/1135529024 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8ab009c630 0x7f8ab0133950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 >> 172.21.15.134:0/1135529024 conn(0x7f8ab00a2e20 msgr2=0x7f8ab01bca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 shutdown_connections 2024-09-16T15:00:53.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.743+0000 7f8ab50e9640 1 -- 172.21.15.134:0/1135529024 wait complete. 2024-09-16T15:00:53.755 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-16T15:00:52.160501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163084,"kb_used_data":972,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067956,"statfs":{"total":579820584960,"available":579653586944,"internally_reserved":0,"allocated":995328,"data_stored":1322734,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319854+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:00:49.319854+0000","last_peered":"2024-09-16T15:00:49.319854+0000","last_clean":"2024-09-16T15:00:49.319854+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:00:49.319854+0000","last_undegraded":"2024-09-16T15:00:49.319854+0000","last_fullsized":"2024-09-16T15:00:49.319854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320197+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:00:49.320197+0000","last_peered":"2024-09-16T15:00:49.320197+0000","last_clean":"2024-09-16T15:00:49.320197+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:00:49.320197+0000","last_undegraded":"2024-09-16T15:00:49.320197+0000","last_fullsized":"2024-09-16T15:00:49.320197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319591+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:00:49.319591+0000","last_peered":"2024-09-16T15:00:49.319591+0000","last_clean":"2024-09-16T15:00:49.319591+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:00:49.319591+0000","last_undegraded":"2024-09-16T15:00:49.319591+0000","last_fullsized":"2024-09-16T15:00:49.319591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320254+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:00:49.320254+0000","last_peered":"2024-09-16T15:00:49.320254+0000","last_clean":"2024-09-16T15:00:49.320254+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:00:49.320254+0000","last_undegraded":"2024-09-16T15:00:49.320254+0000","last_fullsized":"2024-09-16T15:00:49.320254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.330191+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:00:49.330191+0000","last_peered":"2024-09-16T15:00:49.330191+0000","last_clean":"2024-09-16T15:00:49.330191+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:00:49.330191+0000","last_undegraded":"2024-09-16T15:00:49.330191+0000","last_fullsized":"2024-09-16T15:00:49.330191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.566752+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:00:49.566752+0000","last_peered":"2024-09-16T15:00:49.566752+0000","last_clean":"2024-09-16T15:00:49.566752+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:00:49.566752+0000","last_undegraded":"2024-09-16T15:00:49.566752+0000","last_fullsized":"2024-09-16T15:00:49.566752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319607+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:00:49.319607+0000","last_peered":"2024-09-16T15:00:49.319607+0000","last_clean":"2024-09-16T15:00:49.319607+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:00:49.319607+0000","last_undegraded":"2024-09-16T15:00:49.319607+0000","last_fullsized":"2024-09-16T15:00:49.319607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.323958+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:00:49.323958+0000","last_peered":"2024-09-16T15:00:49.323958+0000","last_clean":"2024-09-16T15:00:49.323958+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:00:49.323958+0000","last_undegraded":"2024-09-16T15:00:49.323958+0000","last_fullsized":"2024-09-16T15:00:49.323958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-16T15:00:48.539257+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:00:48.539257+0000","last_peered":"2024-09-16T15:00:48.539257+0000","last_clean":"2024-09-16T15:00:48.539257+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:00:48.539257+0000","last_undegraded":"2024-09-16T15:00:48.539257+0000","last_fullsized":"2024-09-16T15:00:48.539257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:00:53.756 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:00:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.850+0000 7fef526b8640 10 monclient: get_monmap_and_config 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.850+0000 7fef526b8640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.850+0000 7fef526b8640 10 monclient: monmap: 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:53.851264+0000 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:53.851264+0000 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:53.853 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 Processor -- start 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 -- start start 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient: init 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient: _add_conns ranks=[0,9,4,6,1,8,5,2,3,7,10] 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fef4c136030 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-a con 0x7fef4c09bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-c con 0x7fef4c13bf80 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c073250 con 0x7fef4c136030 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c09b600 con 0x7fef4c09bfe0 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0a5740 con 0x7fef4c13bf80 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef526b8640 10 monclient(hunting): authenticate will time out at 790.440724s 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fef4c136030 0x7fef4c136410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38978/0 (socket says 172.21.15.134:38978) 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51888/0 (socket says 172.21.15.134:51888) 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33396/0 (socket says 172.21.15.134:33396) 2024-09-16T15:00:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4bfff640 1 -- 172.21.15.134:0/2750064747 learned_addr learned my addr 172.21.15.134:0/2750064747 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): get_auth_request con 0x7fef4c09bfe0 auth_method 0 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): get_auth_request con 0x7fef4c13bf80 auth_method 0 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.851+0000 7fef50c55640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 10 monclient(hunting): handle_auth_done global_id 4288 payload 274 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 1 -- 172.21.15.134:0/2750064747 >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 msgr2=0x7fef4c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 1 --2- 172.21.15.134:0/2750064747 >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 1 -- 172.21.15.134:0/2750064747 >> v1:172.21.15.134:6789/0 conn(0x7fef4c136030 legacy=0x7fef4c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 1 monclient: found mon.noname-c 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef50c55640 1 -- 172.21.15.134:0/2750064747 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0a52c0 con 0x7fef4c13bf80 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.852+0000 7fef4b7fe640 1 --2- 172.21.15.134:0/2750064747 >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.854+0000 7fef50c55640 1 -- 172.21.15.134:0/2750064747 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 msgr2=0x7fef4c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.854+0000 7fef50c55640 1 --2- 172.21.15.134:0/2750064747 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d90 tx=0x7fef4402a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=305478e0b2468744 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 1 -- 172.21.15.134:0/2750064747 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef44007bc0 con 0x7fef4c13bf80 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: dump: 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 1 -- 172.21.15.134:0/2750064747 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4402d050 con 0x7fef4c13bf80 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 5 monclient: authenticate success, global_id 4288 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 1 -- 172.21.15.134:0/2750064747 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef44007d40 con 0x7fef4c13bf80 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 4 set_mon_vals no callback set 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef4affd640 10 monclient: dump: 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 10 monclient: shutdown 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 -- 172.21.15.134:0/2750064747 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 msgr2=0x7fef4c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 --2- 172.21.15.134:0/2750064747 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef44009d90 tx=0x7fef4402a4e0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 -- 172.21.15.134:0/2750064747 shutdown_connections 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 --2- 172.21.15.134:0/2750064747 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c13bf80 0x7fef4c13e370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 --2- 172.21.15.134:0/2750064747 >> v2:172.21.15.134:3300/0 conn(0x7fef4c09bfe0 0x7fef4c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 -- 172.21.15.134:0/2750064747 >> 172.21.15.134:0/2750064747 conn(0x7fef4c0a9ea0 msgr2=0x7fef4c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 -- 172.21.15.134:0/2750064747 shutdown_connections 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.855+0000 7fef526b8640 1 -- 172.21.15.134:0/2750064747 wait complete. 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.856+0000 7fef526b8640 10 monclient: build_initial_monmap 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.856+0000 7fef526b8640 10 monclient: monmap: 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.856+0000 7fef526b8640 1 Processor -- start 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.856+0000 7fef526b8640 1 -- start start 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.856+0000 7fef526b8640 10 monclient: init 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient: _add_conns ranks=[4,0,5,8,2,6,3,7,1] 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-c con 0x7fef4c09bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-a con 0x7fef4c136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 0x7fef4c1353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): picked mon.noname-h con 0x7fef4c13bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1cb730 con 0x7fef4c136030 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c120f80 con 0x7fef4c09bfe0 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c13bcf0 con 0x7fef4c13bf80 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef526b8640 10 monclient(hunting): authenticate will time out at 790.446374s 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51898/0 (socket says 172.21.15.134:51898) 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33406/0 (socket says 172.21.15.134:33406) 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 -- 172.21.15.134:0/1928980640 learned_addr learned my addr 172.21.15.134:0/1928980640 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): get_auth_request con 0x7fef4c136030 auth_method 0 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): get_auth_request con 0x7fef4c09bfe0 auth_method 0 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient(hunting): handle_auth_done global_id 4365 payload 824 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef50c55640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 0x7fef4c1353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 -- 172.21.15.134:0/1928980640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 msgr2=0x7fef4c1353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 0x7fef4c1353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 -- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 msgr2=0x7fef4c1cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 monclient: found mon.noname-a 2024-09-16T15:00:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4b7fe640 1 -- 172.21.15.134:0/1928980640 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009400 con 0x7fef4c136030 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef50c55640 10 monclient: get_auth_request con 0x7fef4c13bf80 auth_method 0 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef4bfff640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.857+0000 7fef50c55640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 0x7fef4c1353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.859+0000 7fef4b7fe640 1 -- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 msgr2=0x7fef4c134e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.859+0000 7fef4b7fe640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef4000ec10 tx=0x7fef4000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef40009760 con 0x7fef4c136030 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: dump: 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 5 monclient: authenticate success, global_id 4365 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4000ce40 con 0x7fef4c136030 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 10 monclient: _renew_subs 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef44009a70 con 0x7fef4c136030 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef40013d50 con 0x7fef4c136030 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef48ff9640 10 monclient: dump: 2024-09-16T15:00:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 10 monclient: _renew_subs 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0a7af0 con 0x7fef4c136030 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.860+0000 7fef4a7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fef40013470 con 0x7fef4c136030 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef526b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef526b8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef526b8640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c09c400 con 0x7fef4c136030 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef48ff9640 1 --2- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 0x7fef3803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:53.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fef4004a330 con 0x7fef4c136030 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef4bfff640 1 --2- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 0x7fef3803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.861+0000 7fef4bfff640 10 monclient: get_auth_request con 0x7fef38038880 auth_method 0 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.864+0000 7fef4bfff640 1 --2- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 0x7fef3803ad40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007c90 tx=0x7fef3c007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.865+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fef4001c450 con 0x7fef4c136030 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.865+0000 7fef48ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:53.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:53.865+0000 7fef48ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:54.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.023+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef4c061ae0 con 0x7fef38038880 2024-09-16T15:00:54.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.025+0000 7fef48ff9640 1 -- 172.21.15.134:0/1928980640 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41040 (secure 0 0 0) 0x7fef3c007360 con 0x7fef38038880 2024-09-16T15:00:54.027 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:54.028 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:00:54.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 msgr2=0x7fef3803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 0x7fef3803ad40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007c90 tx=0x7fef3c007be0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 10 monclient: shutdown 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 msgr2=0x7fef4c134e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef4000ec10 tx=0x7fef4000c520 comp rx=0 tx=0).stop 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 shutdown_connections 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> v2:172.21.15.162:6800/980547263 conn(0x7fef38038880 0x7fef3803ad40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fef4c13bf80 0x7fef4c1353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef4c136030 0x7fef4c134e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 --2- 172.21.15.134:0/1928980640 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef4c09bfe0 0x7fef4c1cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.027+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 >> 172.21.15.134:0/1928980640 conn(0x7fef4c0a9ea0 msgr2=0x7fef4c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.028+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 shutdown_connections 2024-09-16T15:00:54.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.028+0000 7fef526b8640 1 -- 172.21.15.134:0/1928980640 wait complete. 2024-09-16T15:00:54.039 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-16T15:00:52.160501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163084,"kb_used_data":972,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067956,"statfs":{"total":579820584960,"available":579653586944,"internally_reserved":0,"allocated":995328,"data_stored":1322734,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319854+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:00:49.319854+0000","last_peered":"2024-09-16T15:00:49.319854+0000","last_clean":"2024-09-16T15:00:49.319854+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:00:49.319854+0000","last_undegraded":"2024-09-16T15:00:49.319854+0000","last_fullsized":"2024-09-16T15:00:49.319854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320197+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:00:49.320197+0000","last_peered":"2024-09-16T15:00:49.320197+0000","last_clean":"2024-09-16T15:00:49.320197+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:00:49.320197+0000","last_undegraded":"2024-09-16T15:00:49.320197+0000","last_fullsized":"2024-09-16T15:00:49.320197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319591+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:00:49.319591+0000","last_peered":"2024-09-16T15:00:49.319591+0000","last_clean":"2024-09-16T15:00:49.319591+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:00:49.319591+0000","last_undegraded":"2024-09-16T15:00:49.319591+0000","last_fullsized":"2024-09-16T15:00:49.319591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320254+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:00:49.320254+0000","last_peered":"2024-09-16T15:00:49.320254+0000","last_clean":"2024-09-16T15:00:49.320254+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:00:49.320254+0000","last_undegraded":"2024-09-16T15:00:49.320254+0000","last_fullsized":"2024-09-16T15:00:49.320254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.330191+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:00:49.330191+0000","last_peered":"2024-09-16T15:00:49.330191+0000","last_clean":"2024-09-16T15:00:49.330191+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:00:49.330191+0000","last_undegraded":"2024-09-16T15:00:49.330191+0000","last_fullsized":"2024-09-16T15:00:49.330191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.566752+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:00:49.566752+0000","last_peered":"2024-09-16T15:00:49.566752+0000","last_clean":"2024-09-16T15:00:49.566752+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:00:49.566752+0000","last_undegraded":"2024-09-16T15:00:49.566752+0000","last_fullsized":"2024-09-16T15:00:49.566752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319607+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:00:49.319607+0000","last_peered":"2024-09-16T15:00:49.319607+0000","last_clean":"2024-09-16T15:00:49.319607+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:00:49.319607+0000","last_undegraded":"2024-09-16T15:00:49.319607+0000","last_fullsized":"2024-09-16T15:00:49.319607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.323958+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:00:49.323958+0000","last_peered":"2024-09-16T15:00:49.323958+0000","last_clean":"2024-09-16T15:00:49.323958+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:00:49.323958+0000","last_undegraded":"2024-09-16T15:00:49.323958+0000","last_fullsized":"2024-09-16T15:00:49.323958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-16T15:00:48.539257+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:00:48.539257+0000","last_peered":"2024-09-16T15:00:48.539257+0000","last_clean":"2024-09-16T15:00:48.539257+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:00:48.539257+0000","last_undegraded":"2024-09-16T15:00:48.539257+0000","last_fullsized":"2024-09-16T15:00:48.539257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:00:54.040 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:00:54.040 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T15:00:54.040 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T15:00:54.041 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.135+0000 7fc2c7296640 10 monclient: get_monmap_and_config 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.135+0000 7fc2c7296640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.135+0000 7fc2c7296640 10 monclient: monmap: 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:54.136584+0000 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:54.136584+0000 2024-09-16T15:00:54.153 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 Processor -- start 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 -- start start 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient: init 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient: _add_conns ranks=[1,0,6,9,10,7,4,3,5,8,2] 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc2c009ca30 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc2c01350c0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c013da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-d con 0x7fc2c013b640 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c004e730 con 0x7fc2c01350c0 2024-09-16T15:00:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c00a0920 con 0x7fc2c009ca30 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c006c1b0 con 0x7fc2c013b640 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.136+0000 7fc2c7296640 10 monclient(hunting): authenticate will time out at 790.725931s 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c4831640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc2c01350c0 0x7fc2c0139f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38982/0 (socket says 172.21.15.134:38982) 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c4831640 1 -- 172.21.15.134:0/3553016404 learned_addr learned my addr 172.21.15.134:0/3553016404 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): get_auth_request con 0x7fc2c013b640 auth_method 0 2024-09-16T15:00:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient(hunting): handle_auth_done global_id 4317 payload 274 2024-09-16T15:00:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 1 -- 172.21.15.134:0/3553016404 >> v1:172.21.15.162:6789/0 conn(0x7fc2c009ca30 legacy=0x7fc2c009ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 1 -- 172.21.15.134:0/3553016404 >> v1:172.21.15.134:6789/0 conn(0x7fc2c01350c0 legacy=0x7fc2c0139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 1 monclient: found mon.noname-d 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.137+0000 7fc2c5833640 1 -- 172.21.15.134:0/3553016404 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c009fba0 con 0x7fc2c013b640 2024-09-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c5833640 1 -- 172.21.15.134:0/3553016404 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 msgr2=0x7fc2c013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c5833640 1 --2- 172.21.15.134:0/3553016404 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c013da30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009dc0 tx=0x7fc2b802a510 comp rx=0 tx=0).ready entity=mon.6 client_cookie=fefcccca11e2a9ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 1 -- 172.21.15.134:0/3553016404 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2b8007ca0 con 0x7fc2c013b640 2024-09-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: dump: 2024-09-16T15:00:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 1 -- 172.21.15.134:0/3553016404 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b802d050 con 0x7fc2c013b640 2024-09-16T15:00:54.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 5 monclient: authenticate success, global_id 4317 2024-09-16T15:00:54.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:54.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:54.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 4 set_mon_vals no callback set 2024-09-16T15:00:54.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 1 -- 172.21.15.134:0/3553016404 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2b8007e20 con 0x7fc2c013b640 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2b7fff640 10 monclient: dump: 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.166 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.167 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.168 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.168 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.168 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 10 monclient: shutdown 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 -- 172.21.15.134:0/3553016404 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 msgr2=0x7fc2c013da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 --2- 172.21.15.134:0/3553016404 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c013da30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009dc0 tx=0x7fc2b802a510 comp rx=0 tx=0).stop 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 -- 172.21.15.134:0/3553016404 shutdown_connections 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 --2- 172.21.15.134:0/3553016404 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c013da30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 -- 172.21.15.134:0/3553016404 >> 172.21.15.134:0/3553016404 conn(0x7fc2c00a2f00 msgr2=0x7fc2c00a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 -- 172.21.15.134:0/3553016404 shutdown_connections 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.140+0000 7fc2c7296640 1 -- 172.21.15.134:0/3553016404 wait complete. 2024-09-16T15:00:54.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.141+0000 7fc2c7296640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.141+0000 7fc2c7296640 10 monclient: monmap: 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.170 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.141+0000 7fc2c7296640 1 Processor -- start 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.141+0000 7fc2c7296640 1 -- start start 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.141+0000 7fc2c7296640 10 monclient: init 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient: _add_conns ranks=[1,8,6,2,4,3,5,0,7] 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c009ca30 0x7fc2c012d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-f con 0x7fc2c009ca30 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 0x7fc2c012d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-e con 0x7fc2c01350c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:54.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): picked mon.noname-d con 0x7fc2c013b640 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c006eab0 con 0x7fc2c009ca30 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c006f1f0 con 0x7fc2c013b640 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c006f4d0 con 0x7fc2c01350c0 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c7296640 10 monclient(hunting): authenticate will time out at 790.731396s 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42358/0 (socket says 172.21.15.134:42358) 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 -- 172.21.15.134:0/804401228 learned_addr learned my addr 172.21.15.134:0/804401228 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 0x7fc2c012d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): get_auth_request con 0x7fc2c013b640 auth_method 0 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): get_auth_request con 0x7fc2c01350c0 auth_method 0 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c009ca30 0x7fc2c012d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): get_auth_request con 0x7fc2c009ca30 auth_method 0 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5032640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient(hunting): handle_auth_done global_id 4326 payload 824 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 -- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 msgr2=0x7fc2c012d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 0x7fc2c012d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 -- 172.21.15.134:0/804401228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c009ca30 msgr2=0x7fc2c012d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c009ca30 0x7fc2c012d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 monclient: found mon.noname-d 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c5833640 1 -- 172.21.15.134:0/804401228 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8009a70 con 0x7fc2c013b640 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.142+0000 7fc2c4831640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 0x7fc2c012d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c5833640 1 -- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 msgr2=0x7fc2c012de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c5833640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009ee0 tx=0x7fc2b8004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2b8007e30 con 0x7fc2c013b640 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: dump: 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 5 monclient: authenticate success, global_id 4326 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b80046c0 con 0x7fc2c013b640 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc2b8004830 con 0x7fc2c013b640 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 10 monclient: _renew_subs 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c006f7b0 con 0x7fc2c013b640 2024-09-16T15:00:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b77fe640 4 set_mon_vals no callback set 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2b5ffb640 10 monclient: dump: 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 10 monclient: _renew_subs 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.145+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c0072190 con 0x7fc2c013b640 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc2b8004250 con 0x7fc2c013b640 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c7296640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c7296640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c7296640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c00a3cf0 con 0x7fc2c013b640 2024-09-16T15:00:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2b5ffb640 1 --2- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 0x7fc2a403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7fc2b806b030 con 0x7fc2c013b640 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c5032640 1 --2- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 0x7fc2a403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.146+0000 7fc2c5032640 10 monclient: get_auth_request con 0x7fc2a40388b0 auth_method 0 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.149+0000 7fc2c5032640 1 --2- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 0x7fc2a403ad70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004f60 tx=0x7fc2bc007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.151+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc2b809e9c0 con 0x7fc2c013b640 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.151+0000 7fc2b5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.151+0000 7fc2b5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.347+0000 7fc2c7296640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-16T15:00:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.347+0000 7fc2c7296640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.347+0000 7fc2c7296640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.347+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc2c006f330 con 0x7fc2c013b640 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.348+0000 7fc2b5ffb640 1 -- 172.21.15.134:0/804401228 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc2c006f330 con 0x7fc2c013b640 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.348+0000 7fc2b5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.348+0000 7fc2b5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:54.350 INFO:teuthology.orchestra.run.smithi134.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 msgr2=0x7fc2a403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 0x7fc2a403ad70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004f60 tx=0x7fc2bc007e30 comp rx=0 tx=0).stop 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 10 monclient: shutdown 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 msgr2=0x7fc2c012de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009ee0 tx=0x7fc2b8004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 shutdown_connections 2024-09-16T15:00:54.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> v2:172.21.15.162:6800/980547263 conn(0x7fc2a40388b0 0x7fc2a403ad70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc2c013b640 0x7fc2c012de50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2c01350c0 0x7fc2c012d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 --2- 172.21.15.134:0/804401228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2c009ca30 0x7fc2c012d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 >> 172.21.15.134:0/804401228 conn(0x7fc2c00a2f00 msgr2=0x7fc2c00a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 shutdown_connections 2024-09-16T15:00:54.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.350+0000 7fc2c7296640 1 -- 172.21.15.134:0/804401228 wait complete. 2024-09-16T15:00:54.362 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T15:00:54.362 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-16T15:00:54.375 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-16T15:00:54.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.471+0000 7f767ff11640 10 monclient: get_monmap_and_config 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.471+0000 7f767ff11640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.471+0000 7f767ff11640 10 monclient: monmap: 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:54.472500+0000 2024-09-16T15:00:54.473 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:54.472500+0000 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:54.474 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:54.475 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 Processor -- start 2024-09-16T15:00:54.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 -- start start 2024-09-16T15:00:54.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient: init 2024-09-16T15:00:54.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient: _add_conns ranks=[4,5,1,0,8,9,10,3,7,6,2] 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-c con 0x7f7678136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-h con 0x7f767809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7678136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7678073250 con 0x7f7678136c60 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f767809b600 con 0x7f7678136030 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76780a5740 con 0x7f767809bfe0 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767ff11640 10 monclient(hunting): authenticate will time out at 791.061821s 2024-09-16T15:00:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33420/0 (socket says 172.21.15.134:33420) 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 1 -- 172.21.15.134:0/3898373550 learned_addr learned my addr 172.21.15.134:0/3898373550 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 10 monclient(hunting): get_auth_request con 0x7f7678136030 auth_method 0 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.472+0000 7f767dcad640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient(hunting): handle_auth_done global_id 4306 payload 274 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 1 -- 172.21.15.134:0/3898373550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 msgr2=0x7f767809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 1 --2- 172.21.15.134:0/3898373550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 1 -- 172.21.15.134:0/3898373550 >> v1:172.21.15.162:6789/0 conn(0x7f7678136c60 legacy=0x7f76781370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 1 monclient: found mon.noname-c 2024-09-16T15:00:54.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.473+0000 7f767dcad640 1 -- 172.21.15.134:0/3898373550 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780a52c0 con 0x7f7678136030 2024-09-16T15:00:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767dcad640 1 -- 172.21.15.134:0/3898373550 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 msgr2=0x7f7678136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767dcad640 1 --2- 172.21.15.134:0/3898373550 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f76740098a0 tx=0x7f767402a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3793df0f3be4254a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 1 -- 172.21.15.134:0/3898373550 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76740307b0 con 0x7f7678136030 2024-09-16T15:00:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: dump: 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 1 -- 172.21.15.134:0/3898373550 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674030930 con 0x7f7678136030 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 5 monclient: authenticate success, global_id 4306 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 1 -- 172.21.15.134:0/3898373550 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76740313a0 con 0x7f7678136030 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 4 set_mon_vals no callback set 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ccab640 10 monclient: dump: 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.487 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 10 monclient: shutdown 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 -- 172.21.15.134:0/3898373550 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 msgr2=0x7f7678136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 --2- 172.21.15.134:0/3898373550 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f76740098a0 tx=0x7f767402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 -- 172.21.15.134:0/3898373550 shutdown_connections 2024-09-16T15:00:54.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 --2- 172.21.15.134:0/3898373550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 --2- 172.21.15.134:0/3898373550 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7678136030 0x7f7678136410 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.476+0000 7f767ff11640 1 -- 172.21.15.134:0/3898373550 >> 172.21.15.134:0/3898373550 conn(0x7f76780a9ea0 msgr2=0x7f76780a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.477+0000 7f767ff11640 1 -- 172.21.15.134:0/3898373550 shutdown_connections 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.477+0000 7f767ff11640 1 -- 172.21.15.134:0/3898373550 wait complete. 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.477+0000 7f767ff11640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.477+0000 7f767ff11640 10 monclient: monmap: 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:54.489 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.477+0000 7f767ff11640 1 Processor -- start 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 -- start start 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient: init 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient: _add_conns ranks=[5,8,2,1,3,0,7,4,6] 2024-09-16T15:00:54.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-h con 0x7f767809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 0x7f767811ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-e con 0x7f7678136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): picked mon.noname-b con 0x7f7678136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76780a80f0 con 0x7f7678136c60 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7678135940 con 0x7f767809bfe0 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f76781220f0 con 0x7f7678136030 2024-09-16T15:00:54.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767ff11640 10 monclient(hunting): authenticate will time out at 791.067484s 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 0x7f767811ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 0x7f767811ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41846/0 (socket says 172.21.15.134:41846) 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 1 -- 172.21.15.134:0/2661812215 learned_addr learned my addr 172.21.15.134:0/2661812215 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58138/0 (socket says 172.21.15.134:58138) 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): get_auth_request con 0x7f7678136030 auth_method 0 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): get_auth_request con 0x7f7678136c60 auth_method 0 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767d4ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): get_auth_request con 0x7f767809bfe0 auth_method 0 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767dcad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient(hunting): handle_auth_done global_id 4367 payload 824 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 -- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 msgr2=0x7f767811ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 0x7f767811ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 -- 172.21.15.134:0/2661812215 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 msgr2=0x7f767811fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767811fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 monclient: found mon.noname-b 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.478+0000 7f767e4ae640 1 -- 172.21.15.134:0/2661812215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76740094d0 con 0x7f7678136c60 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767e4ae640 1 -- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 msgr2=0x7f7678120500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767e4ae640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f767000be00 tx=0x7f767000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f767000ce80 con 0x7f7678136c60 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: dump: 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.495 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76700148b0 con 0x7f7678136c60 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7670014e90 con 0x7f7678136c60 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766e7fc640 10 monclient: dump: 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.497 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 5 monclient: authenticate success, global_id 4367 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f766ffff640 4 set_mon_vals no callback set 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 10 monclient: _renew_subs 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76781226b0 con 0x7f7678136c60 2024-09-16T15:00:54.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 10 monclient: _renew_subs 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.481+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7678121200 con 0x7f7678136c60 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7670014a20 con 0x7f7678136c60 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767ff11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767ff11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767ff11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76781222a0 con 0x7f7678136c60 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f766e7fc640 1 --2- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 0x7f765c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f7670049ee0 con 0x7f7678136c60 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767dcad640 1 --2- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 0x7f765c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.482+0000 7f767dcad640 10 monclient: get_auth_request con 0x7f765c038880 auth_method 0 2024-09-16T15:00:54.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.485+0000 7f767dcad640 1 --2- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 0x7f765c03ad40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7674034000 tx=0x7f767402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.486+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f76781222a0 con 0x7f7678136c60 2024-09-16T15:00:54.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.486+0000 7f766e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.486+0000 7f766e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:54.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.648+0000 7f767ff11640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:54.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.648+0000 7f767ff11640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:54.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.648+0000 7f767ff11640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:54.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.648+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76781222a0 con 0x7f7678136c60 2024-09-16T15:00:54.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.649+0000 7f766e7fc640 1 -- 172.21.15.134:0/2661812215 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+10854 (secure 0 0 0) 0x7f76781222a0 con 0x7f7678136c60 2024-09-16T15:00:54.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.649+0000 7f766e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:00:54.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.649+0000 7f766e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:54.651 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:54.651 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":17,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:00:49.308510+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:00:45.513777+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.343686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.259639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.265227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.302963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:00:54.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 msgr2=0x7f765c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 0x7f765c03ad40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7674034000 tx=0x7f767402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:54.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 10 monclient: shutdown 2024-09-16T15:00:54.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 msgr2=0x7f7678120500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f767000be00 tx=0x7f767000bef0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 shutdown_connections 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> v2:172.21.15.162:6800/980547263 conn(0x7f765c038880 0x7f765c03ad40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7678136c60 0x7f7678120500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7678136030 0x7f767811ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 --2- 172.21.15.134:0/2661812215 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f767809bfe0 0x7f767811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 >> 172.21.15.134:0/2661812215 conn(0x7f76780a9ea0 msgr2=0x7f76781bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 shutdown_connections 2024-09-16T15:00:54.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.651+0000 7f767ff11640 1 -- 172.21.15.134:0/2661812215 wait complete. 2024-09-16T15:00:54.664 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:00:44.203747+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T15:00:45.513777+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:00:54.664 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:00:54.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 10 monclient: get_monmap_and_config 2024-09-16T15:00:54.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 10 monclient: monmap: 2024-09-16T15:00:54.763 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.763 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:54.762971+0000 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:54.762971+0000 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.764 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:54.765 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:54.765 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 1 Processor -- start 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 1 -- start start 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.762+0000 7f4e7fb76640 10 monclient: init 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient: _add_conns ranks=[0,10,7,8,6,1,9,5,2,4,3] 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4e78136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 0x7f4e78137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-f con 0x7f4e78137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 0x7f4e7813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-i con 0x7f4e7813c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:54.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e78073250 con 0x7f4e78136e40 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780a56e0 con 0x7f4e78137220 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7809abc0 con 0x7f4e7813c370 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7fb76640 10 monclient(hunting): authenticate will time out at 791.352325s 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d912640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4e78136e40 0x7f4e7809c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:38990/0 (socket says 172.21.15.134:38990) 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d912640 1 -- 172.21.15.134:0/1658309232 learned_addr learned my addr 172.21.15.134:0/1658309232 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 1 --2- 172.21.15.134:0/1658309232 >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 0x7f4e78137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): get_auth_request con 0x7f4e78137220 auth_method 0 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 1 --2- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 0x7f4e7813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): get_auth_request con 0x7f4e7813c370 auth_method 0 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7d111640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 10 monclient(hunting): handle_auth_done global_id 4300 payload 274 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.763+0000 7f4e7e113640 1 -- 172.21.15.134:0/1658309232 >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 msgr2=0x7f4e78137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 1 --2- 172.21.15.134:0/1658309232 >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 0x7f4e78137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 1 -- 172.21.15.134:0/1658309232 >> v1:172.21.15.134:6789/0 conn(0x7f4e78136e40 legacy=0x7f4e7809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 1 monclient: found mon.noname-i 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7e113640 1 -- 172.21.15.134:0/1658309232 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7809ba50 con 0x7f4e7813c370 2024-09-16T15:00:54.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.764+0000 7f4e7d111640 1 --2- 172.21.15.134:0/1658309232 >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 0x7f4e78137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.766+0000 7f4e7e113640 1 -- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 msgr2=0x7f4e7813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.766+0000 7f4e7e113640 1 --2- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 0x7f4e7813e760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009d90 tx=0x7f4e7002a4e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=a98c7e34673413c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 1 -- 172.21.15.134:0/1658309232 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e70007bc0 con 0x7f4e7813c370 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: dump: 2024-09-16T15:00:54.775 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.776 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 1 -- 172.21.15.134:0/1658309232 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7002d050 con 0x7f4e7813c370 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 1 -- 172.21.15.134:0/1658309232 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e70007d40 con 0x7f4e7813c370 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7c910640 10 monclient: dump: 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.777 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 5 monclient: authenticate success, global_id 4300 2024-09-16T15:00:54.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 4 set_mon_vals no callback set 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 10 monclient: shutdown 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 1 -- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 msgr2=0x7f4e7813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.767+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 0x7f4e7813e760 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009d90 tx=0x7f4e7002a4e0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 -- 172.21.15.134:0/1658309232 shutdown_connections 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/1658309232 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4e7813c370 0x7f4e7813e760 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/1658309232 >> v2:172.21.15.162:3300/0 conn(0x7f4e78137220 0x7f4e78137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 -- 172.21.15.134:0/1658309232 >> 172.21.15.134:0/1658309232 conn(0x7f4e780a6110 msgr2=0x7f4e780a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 -- 172.21.15.134:0/1658309232 shutdown_connections 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 -- 172.21.15.134:0/1658309232 wait complete. 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 10 monclient: build_initial_monmap 2024-09-16T15:00:54.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 10 monclient: monmap: 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:54.780 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 Processor -- start 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 1 -- start start 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.768+0000 7f4e7fb76640 10 monclient: init 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient: _add_conns ranks=[4,3,1,7,5,6,8,0,2] 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-c con 0x7f4e78136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e78137220 0x7f4e78136610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-g con 0x7f4e78137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e7813c370 0x7f4e781cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): picked mon.noname-f con 0x7f4e7813c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78120ed0 con 0x7f4e7813c370 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781210d0 con 0x7f4e78137220 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781221e0 con 0x7f4e78136e40 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7fb76640 10 monclient(hunting): authenticate will time out at 791.358421s 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33434/0 (socket says 172.21.15.134:33434) 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 -- 172.21.15.134:0/451569165 learned_addr learned my addr 172.21.15.134:0/451569165 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): get_auth_request con 0x7f4e78136e40 auth_method 0 2024-09-16T15:00:54.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient(hunting): handle_auth_done global_id 4315 payload 824 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 -- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e78137220 msgr2=0x7f4e78136610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7e113640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e7813c370 0x7f4e781cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d111640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e78137220 0x7f4e78136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e78137220 0x7f4e78136610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 -- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e7813c370 msgr2=0x7f4e781cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e7813c370 0x7f4e781cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient: _finish_hunting 0 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 monclient: found mon.noname-c 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.769+0000 7f4e7d912640 1 -- 172.21.15.134:0/451569165 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e74009400 con 0x7f4e78136e40 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7d912640 1 -- 172.21.15.134:0/451569165 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 msgr2=0x7f4e781360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7d912640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c460 tx=0x7f4e6800c920 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e68005cf0 con 0x7f4e78136e40 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: dump: 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68005e90 con 0x7f4e78136e40 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 5 monclient: authenticate success, global_id 4315 2024-09-16T15:00:54.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 10 monclient: _renew_subs 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e70009a70 con 0x7f4e78136e40 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4e680074e0 con 0x7f4e78136e40 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6e7fc640 10 monclient: dump: 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e6ffff640 4 set_mon_vals no callback set 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 10 monclient: _renew_subs 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.772+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7813b380 con 0x7f4e78136e40 2024-09-16T15:00:54.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4e68007070 con 0x7f4e78136e40 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7fb76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7fb76640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7fb76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e781223e0 con 0x7f4e78136e40 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e6e7fc640 1 --2- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 0x7f4e5c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f4e68049bf0 con 0x7f4e78136e40 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7d111640 1 --2- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 0x7f4e5c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.773+0000 7f4e7d111640 10 monclient: get_auth_request con 0x7f4e5c038880 auth_method 0 2024-09-16T15:00:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.776+0000 7f4e7d111640 1 --2- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 0x7f4e5c03ad40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e780a2c60 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.778+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4e6801b920 con 0x7f4e78136e40 2024-09-16T15:00:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.778+0000 7f4e6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.778+0000 7f4e6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:54.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e7fb76640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e7fb76640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e7fb76640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e780a7410 con 0x7f4e78136e40 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e6e7fc640 1 -- 172.21.15.134:0/451569165 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e68010300 con 0x7f4e78136e40 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e6e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.937+0000 7f4e6e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:54.939 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 1 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 msgr2=0x7f4e5c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 0x7f4e5c03ad40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e780a2c60 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 10 monclient: shutdown 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 msgr2=0x7f4e781360d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c460 tx=0x7f4e6800c920 comp rx=0 tx=0).stop 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 shutdown_connections 2024-09-16T15:00:54.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e7813c370 0x7f4e781cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> v2:172.21.15.162:6800/980547263 conn(0x7f4e5c038880 0x7f4e5c03ad40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e78137220 0x7f4e78136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 --2- 172.21.15.134:0/451569165 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4e78136e40 0x7f4e781360d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 >> 172.21.15.134:0/451569165 conn(0x7f4e780a6110 msgr2=0x7f4e7813a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 shutdown_connections 2024-09-16T15:00:54.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:54.939+0000 7f4e7fb76640 1 -- 172.21.15.134:0/451569165 wait complete. 2024-09-16T15:00:54.952 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 10 monclient: get_monmap_and_config 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 10 monclient: monmap: 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.051 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:55.050263+0000 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:55.050263+0000 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.052 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:55.053 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:55.053 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 1 Processor -- start 2024-09-16T15:00:55.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 1 -- start start 2024-09-16T15:00:55.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.049+0000 7f89b0ed7640 10 monclient: init 2024-09-16T15:00:55.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient: _add_conns ranks=[1,0,5,10,6,8,3,2,7,4,9] 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f89ac136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f89ac137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 0x7f89ac13f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-h con 0x7f89ac13d380 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ac073250 con 0x7f89ac137220 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ac09ba50 con 0x7f89ac136e40 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09c860 con 0x7f89ac13d380 2024-09-16T15:00:55.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89b0ed7640 10 monclient(hunting): authenticate will time out at 791.639604s 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89a9d9b640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f89ac137220 0x7f89ac137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:39000/0 (socket says 172.21.15.134:39000) 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89a9d9b640 1 -- 172.21.15.134:0/1256729779 learned_addr learned my addr 172.21.15.134:0/1256729779 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89aad9d640 1 --2- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 0x7f89ac13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89aad9d640 10 monclient(hunting): get_auth_request con 0x7f89ac13d380 auth_method 0 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.050+0000 7f89aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient(hunting): handle_auth_done global_id 4307 payload 274 2024-09-16T15:00:55.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 1 -- 172.21.15.134:0/1256729779 >> v1:172.21.15.162:6789/0 conn(0x7f89ac136e40 legacy=0x7f89ac09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 1 -- 172.21.15.134:0/1256729779 >> v1:172.21.15.134:6789/0 conn(0x7f89ac137220 legacy=0x7f89ac137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 1 monclient: found mon.noname-h 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:00:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.051+0000 7f89aad9d640 1 -- 172.21.15.134:0/1256729779 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0a53c0 con 0x7f89ac13d380 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89aad9d640 1 -- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 msgr2=0x7f89ac13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89aad9d640 1 --2- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 0x7f89ac13f770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009dc0 tx=0x7f89a402a510 comp rx=0 tx=0).ready entity=mon.5 client_cookie=72bcccd94768414f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 1 -- 172.21.15.134:0/1256729779 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89a4007ca0 con 0x7f89ac13d380 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: dump: 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 1 -- 172.21.15.134:0/1256729779 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a402d050 con 0x7f89ac13d380 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 1 -- 172.21.15.134:0/1256729779 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89a4007e20 con 0x7f89ac13d380 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 5 monclient: authenticate success, global_id 4307 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 4 set_mon_vals no callback set 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89a959a640 10 monclient: dump: 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.065 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 10 monclient: shutdown 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 -- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 msgr2=0x7f89ac13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 0x7f89ac13f770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009dc0 tx=0x7f89a402a510 comp rx=0 tx=0).stop 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 -- 172.21.15.134:0/1256729779 shutdown_connections 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/1256729779 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ac13d380 0x7f89ac13f770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 -- 172.21.15.134:0/1256729779 >> 172.21.15.134:0/1256729779 conn(0x7f89ac0a6110 msgr2=0x7f89ac0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 -- 172.21.15.134:0/1256729779 shutdown_connections 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.054+0000 7f89b0ed7640 1 -- 172.21.15.134:0/1256729779 wait complete. 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.055+0000 7f89b0ed7640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.055+0000 7f89b0ed7640 10 monclient: monmap: 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.067 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 Processor -- start 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 -- start start 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient: init 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient: _add_conns ranks=[0,3,6,4,5,2,8,1,7] 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-a con 0x7f89ac136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac137220 0x7f89ac135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-g con 0x7f89ac137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ac13d380 0x7f89ac13a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): picked mon.noname-d con 0x7f89ac13d380 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:55.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac121860 con 0x7f89ac136e40 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac120fc0 con 0x7f89ac137220 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac1212a0 con 0x7f89ac13d380 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89b0ed7640 10 monclient(hunting): authenticate will time out at 791.645558s 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51904/0 (socket says 172.21.15.134:51904) 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 -- 172.21.15.134:0/4284043893 learned_addr learned my addr 172.21.15.134:0/4284043893 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): get_auth_request con 0x7f89ac136e40 auth_method 0 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ac13d380 0x7f89ac13a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): get_auth_request con 0x7f89ac13d380 auth_method 0 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac137220 0x7f89ac135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): get_auth_request con 0x7f89ac137220 auth_method 0 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89a9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient(hunting): handle_auth_done global_id 4374 payload 824 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 -- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ac13d380 msgr2=0x7f89ac13a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ac13d380 0x7f89ac13a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 -- 172.21.15.134:0/4284043893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac137220 msgr2=0x7f89ac135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac137220 0x7f89ac135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 monclient: found mon.noname-a 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.056+0000 7f89aa59c640 1 -- 172.21.15.134:0/4284043893 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a4009a70 con 0x7f89ac136e40 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89aa59c640 1 -- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 msgr2=0x7f89ac1cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89aa59c640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c430 tx=0x7f899c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f899c011c10 con 0x7f89ac136e40 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: dump: 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.072 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c011d90 con 0x7f89ac136e40 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 5 monclient: authenticate success, global_id 4374 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 10 monclient: _renew_subs 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac121580 con 0x7f89ac136e40 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f899c012890 con 0x7f89ac136e40 2024-09-16T15:00:55.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f899affd640 10 monclient: dump: 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.074 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89a8d99640 4 set_mon_vals no callback set 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 10 monclient: _renew_subs 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.059+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac11fe40 con 0x7f89ac136e40 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f899c012420 con 0x7f89ac136e40 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89b0ed7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89b0ed7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89b0ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac09b490 con 0x7f89ac136e40 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f899affd640 1 --2- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 0x7f898c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3562+0+0 (secure 0 0 0) 0x7f899c049e70 con 0x7f89ac136e40 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89a9d9b640 1 --2- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 0x7f898c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.060+0000 7f89a9d9b640 10 monclient: get_auth_request con 0x7f898c038880 auth_method 0 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.063+0000 7f89a9d9b640 1 --2- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 0x7f898c03ad40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89a0007c90 tx=0x7f89a0007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.064+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f899c0126d0 con 0x7f89ac136e40 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.064+0000 7f899affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.064+0000 7f899affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:55.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f89b0ed7640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-16T15:00:55.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f89b0ed7640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-16T15:00:55.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f89b0ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89ac072000 con 0x7f89ac136e40 2024-09-16T15:00:55.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f899affd640 1 -- 172.21.15.134:0/4284043893 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f899c0102e0 con 0x7f89ac136e40 2024-09-16T15:00:55.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f899affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-16T15:00:55.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.224+0000 7f899affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:55.226 INFO:teuthology.orchestra.run.smithi134.stdout:pg_num: 8 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 msgr2=0x7f898c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 0x7f898c03ad40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f89a0007c90 tx=0x7f89a0007be0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 10 monclient: shutdown 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 msgr2=0x7f89ac1cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c430 tx=0x7f899c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 shutdown_connections 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ac13d380 0x7f89ac13a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> v2:172.21.15.162:6800/980547263 conn(0x7f898c038880 0x7f898c03ad40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89ac137220 0x7f89ac135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 --2- 172.21.15.134:0/4284043893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89ac136e40 0x7f89ac1cb570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 >> 172.21.15.134:0/4284043893 conn(0x7f89ac0a6110 msgr2=0x7f89ac0a4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 shutdown_connections 2024-09-16T15:00:55.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.226+0000 7f89b0ed7640 1 -- 172.21.15.134:0/4284043893 wait complete. 2024-09-16T15:00:55.238 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-16T15:00:55.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:00:55.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.333+0000 7fe7d6273640 10 monclient: get_monmap_and_config 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.333+0000 7fe7d6273640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.333+0000 7fe7d6273640 10 monclient: monmap: 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:55.334103+0000 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:55.334103+0000 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:55.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.333+0000 7fe7d6273640 1 Processor -- start 2024-09-16T15:00:55.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 -- start start 2024-09-16T15:00:55.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient: init 2024-09-16T15:00:55.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient: _add_conns ranks=[8,0,3,1,10,4,6,7,9,2,5] 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 0x7fe7d0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-e con 0x7fe7d0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe7d009bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d009c400 0x7fe7d0137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-g con 0x7fe7d009c400 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d0073250 con 0x7fe7d009bfe0 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d009c880 con 0x7fe7d009c400 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d009b600 con 0x7fe7d0136030 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7d6273640 10 monclient(hunting): authenticate will time out at 791.923644s 2024-09-16T15:00:55.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 0x7fe7d0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cf7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe7d009bfe0 0x7fe7d0136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:39004/0 (socket says 172.21.15.134:39004) 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cf7fe640 1 -- 172.21.15.134:0/779188840 learned_addr learned my addr 172.21.15.134:0/779188840 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): get_auth_request con 0x7fe7d0136030 auth_method 0 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 10 monclient(hunting): handle_auth_done global_id 4409 payload 274 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.334+0000 7fe7cffff640 1 -- 172.21.15.134:0/779188840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d009c400 msgr2=0x7fe7d0137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 1 --2- 172.21.15.134:0/779188840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d009c400 0x7fe7d0137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 1 -- 172.21.15.134:0/779188840 >> v1:172.21.15.134:6789/0 conn(0x7fe7d009bfe0 legacy=0x7fe7d0136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 1 monclient: found mon.noname-e 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.335+0000 7fe7cffff640 1 -- 172.21.15.134:0/779188840 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d00a5740 con 0x7fe7d0136030 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7cffff640 1 -- 172.21.15.134:0/779188840 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 msgr2=0x7fe7d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7cffff640 1 --2- 172.21.15.134:0/779188840 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 0x7fe7d0136410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097d0 tx=0x7fe7c002a000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=927df167caf261df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 1 -- 172.21.15.134:0/779188840 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7c002ae00 con 0x7fe7d0136030 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 10 monclient: dump: 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.337+0000 7fe7ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 5 monclient: authenticate success, global_id 4409 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 1 -- 172.21.15.134:0/779188840 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c00307b0 con 0x7fe7d0136030 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 4 set_mon_vals no callback set 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 1 -- 172.21.15.134:0/779188840 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7c0030dd0 con 0x7fe7d0136030 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7ceffd640 10 monclient: dump: 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 10 monclient: shutdown 2024-09-16T15:00:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 -- 172.21.15.134:0/779188840 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 msgr2=0x7fe7d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 --2- 172.21.15.134:0/779188840 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 0x7fe7d0136410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097d0 tx=0x7fe7c002a000 comp rx=0 tx=0).stop 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 -- 172.21.15.134:0/779188840 shutdown_connections 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 --2- 172.21.15.134:0/779188840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7d009c400 0x7fe7d0137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 --2- 172.21.15.134:0/779188840 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe7d0136030 0x7fe7d0136410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 -- 172.21.15.134:0/779188840 >> 172.21.15.134:0/779188840 conn(0x7fe7d00a9ea0 msgr2=0x7fe7d00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 -- 172.21.15.134:0/779188840 shutdown_connections 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.338+0000 7fe7d6273640 1 -- 172.21.15.134:0/779188840 wait complete. 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.339+0000 7fe7d6273640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.339+0000 7fe7d6273640 10 monclient: monmap: 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.339+0000 7fe7d6273640 1 Processor -- start 2024-09-16T15:00:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.339+0000 7fe7d6273640 1 -- start start 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.339+0000 7fe7d6273640 10 monclient: init 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient: _add_conns ranks=[5,7,2,8,6,0,1,3,4] 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe7d009bfe0 0x7fe7d01bec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-h con 0x7fe7d009bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe7d009c400 0x7fe7d0134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-i con 0x7fe7d009c400 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): picked mon.noname-b con 0x7fe7d0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d009faa0 con 0x7fe7d0136030 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00a9000 con 0x7fe7d009bfe0 2024-09-16T15:00:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00a8b40 con 0x7fe7d009c400 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d6273640 10 monclient(hunting): authenticate will time out at 791.929132s 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58154/0 (socket says 172.21.15.134:58154) 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 -- 172.21.15.134:0/2646039651 learned_addr learned my addr 172.21.15.134:0/2646039651 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): get_auth_request con 0x7fe7d0136030 auth_method 0 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe7d009bfe0 0x7fe7d01bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe7d009c400 0x7fe7d0134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): get_auth_request con 0x7fe7d009bfe0 auth_method 0 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): get_auth_request con 0x7fe7d009c400 auth_method 0 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient(hunting): handle_auth_done global_id 4376 payload 824 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 -- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe7d009c400 msgr2=0x7fe7d0134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe7d009c400 0x7fe7d0134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 -- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe7d009bfe0 msgr2=0x7fe7d01bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe7d009bfe0 0x7fe7d01bec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 monclient: found mon.noname-b 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.340+0000 7fe7d4810640 1 -- 172.21.15.134:0/2646039651 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0009400 con 0x7fe7d0136030 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.342+0000 7fe7d4810640 1 -- 172.21.15.134:0/2646039651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 msgr2=0x7fe7d0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.342+0000 7fe7d4810640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b9a0 tx=0x7fe7c800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7c800cbb0 con 0x7fe7d0136030 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: dump: 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c800cd50 con 0x7fe7d0136030 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe7c8007c50 con 0x7fe7d0136030 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 5 monclient: authenticate success, global_id 4376 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 10 monclient: _renew_subs 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d01222a0 con 0x7fe7d0136030 2024-09-16T15:00:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ce7fc640 4 set_mon_vals no callback set 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7ccff9640 10 monclient: dump: 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 10 monclient: _renew_subs 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.348+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d01bf8b0 con 0x7fe7d0136030 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.349+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe7c8007370 con 0x7fe7d0136030 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.349+0000 7fe7ccff9640 1 --2- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 0x7fe7bc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.349+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7fe7c8049ab0 con 0x7fe7d0136030 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.349+0000 7fe7cffff640 1 --2- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 0x7fe7bc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.349+0000 7fe7cffff640 10 monclient: get_auth_request con 0x7fe7bc038880 auth_method 0 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.350+0000 7fe7d6273640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.350+0000 7fe7d6273640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.350+0000 7fe7d6273640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.350+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d00a6940 con 0x7fe7d0136030 2024-09-16T15:00:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.352+0000 7fe7cffff640 1 --2- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 0x7fe7bc03ad40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.356+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe7c8019080 con 0x7fe7d0136030 2024-09-16T15:00:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.356+0000 7fe7ccff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.356+0000 7fe7ccff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.516+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7d0061ae0 con 0x7fe7bc038880 2024-09-16T15:00:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.519+0000 7fe7ccff9640 1 -- 172.21.15.134:0/2646039651 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41030 (secure 0 0 0) 0x7fe7c0004040 con 0x7fe7bc038880 2024-09-16T15:00:55.521 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:00:55.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 msgr2=0x7fe7bc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 0x7fe7bc03ad40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0034000 tx=0x7fe7c002e000 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 10 monclient: shutdown 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 msgr2=0x7fe7d0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b9a0 tx=0x7fe7c800be60 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 shutdown_connections 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7bc038880 0x7fe7bc03ad40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7d0136030 0x7fe7d0135100 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe7d009c400 0x7fe7d0134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 --2- 172.21.15.134:0/2646039651 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe7d009bfe0 0x7fe7d01bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 >> 172.21.15.134:0/2646039651 conn(0x7fe7d00a9ea0 msgr2=0x7fe7d009e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 shutdown_connections 2024-09-16T15:00:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.522+0000 7fe7d6273640 1 -- 172.21.15.134:0/2646039651 wait complete. 2024-09-16T15:00:55.534 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-16T15:00:55.343686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163084,"kb_used_data":972,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067956,"statfs":{"total":579820584960,"available":579653586944,"internally_reserved":0,"allocated":995328,"data_stored":1322734,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.183185"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319854+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:00:49.319854+0000","last_peered":"2024-09-16T15:00:49.319854+0000","last_clean":"2024-09-16T15:00:49.319854+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:00:49.319854+0000","last_undegraded":"2024-09-16T15:00:49.319854+0000","last_fullsized":"2024-09-16T15:00:49.319854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320197+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:00:49.320197+0000","last_peered":"2024-09-16T15:00:49.320197+0000","last_clean":"2024-09-16T15:00:49.320197+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:00:49.320197+0000","last_undegraded":"2024-09-16T15:00:49.320197+0000","last_fullsized":"2024-09-16T15:00:49.320197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319591+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:00:49.319591+0000","last_peered":"2024-09-16T15:00:49.319591+0000","last_clean":"2024-09-16T15:00:49.319591+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:00:49.319591+0000","last_undegraded":"2024-09-16T15:00:49.319591+0000","last_fullsized":"2024-09-16T15:00:49.319591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320254+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:00:49.320254+0000","last_peered":"2024-09-16T15:00:49.320254+0000","last_clean":"2024-09-16T15:00:49.320254+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:00:49.320254+0000","last_undegraded":"2024-09-16T15:00:49.320254+0000","last_fullsized":"2024-09-16T15:00:49.320254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.330191+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:00:49.330191+0000","last_peered":"2024-09-16T15:00:49.330191+0000","last_clean":"2024-09-16T15:00:49.330191+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:00:49.330191+0000","last_undegraded":"2024-09-16T15:00:49.330191+0000","last_fullsized":"2024-09-16T15:00:49.330191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.566752+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:00:49.566752+0000","last_peered":"2024-09-16T15:00:49.566752+0000","last_clean":"2024-09-16T15:00:49.566752+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:00:49.566752+0000","last_undegraded":"2024-09-16T15:00:49.566752+0000","last_fullsized":"2024-09-16T15:00:49.566752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319607+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:00:49.319607+0000","last_peered":"2024-09-16T15:00:49.319607+0000","last_clean":"2024-09-16T15:00:49.319607+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:00:49.319607+0000","last_undegraded":"2024-09-16T15:00:49.319607+0000","last_fullsized":"2024-09-16T15:00:49.319607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.323958+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:00:49.323958+0000","last_peered":"2024-09-16T15:00:49.323958+0000","last_clean":"2024-09-16T15:00:49.323958+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:00:49.323958+0000","last_undegraded":"2024-09-16T15:00:49.323958+0000","last_fullsized":"2024-09-16T15:00:49.323958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-16T15:00:48.539257+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:00:48.539257+0000","last_peered":"2024-09-16T15:00:48.539257+0000","last_clean":"2024-09-16T15:00:48.539257+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:00:48.539257+0000","last_undegraded":"2024-09-16T15:00:48.539257+0000","last_fullsized":"2024-09-16T15:00:48.539257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:00:55.535 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:00:55.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.630+0000 7fa9ee965640 10 monclient: get_monmap_and_config 2024-09-16T15:00:55.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.630+0000 7fa9ee965640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.630+0000 7fa9ee965640 10 monclient: monmap: 2024-09-16T15:00:55.631 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:55.631073+0000 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:55.631073+0000 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.632 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.633 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.633 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:55.633 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:55.633 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 Processor -- start 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 -- start start 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient: init 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient: _add_conns ranks=[7,6,10,1,8,9,4,0,2,5,3] 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a1d30 0x7fa9e806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-i con 0x7fa9e80a1d30 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-d con 0x7fa9e8070e90 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9e80a2650 0x7fa9e8134b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-f con 0x7fa9e80a2650 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e80a3880 con 0x7fa9e80a2650 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8070540 con 0x7fa9e8070e90 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e80a3a80 con 0x7fa9e80a1d30 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ee965640 10 monclient(hunting): authenticate will time out at 792.220543s 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42374/0 (socket says 172.21.15.134:42374) 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 -- 172.21.15.134:0/3938128849 learned_addr learned my addr 172.21.15.134:0/3938128849 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): get_auth_request con 0x7fa9e8070e90 auth_method 0 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9e7fff640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a1d30 0x7fa9e806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9ecf02640 1 --2- 172.21.15.134:0/3938128849 >> v2:172.21.15.162:3300/0 conn(0x7fa9e80a2650 0x7fa9e8134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient(hunting): handle_auth_done global_id 4344 payload 274 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 -- 172.21.15.134:0/3938128849 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a1d30 msgr2=0x7fa9e806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a1d30 0x7fa9e806f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 -- 172.21.15.134:0/3938128849 >> v2:172.21.15.162:3300/0 conn(0x7fa9e80a2650 msgr2=0x7fa9e8134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 --2- 172.21.15.134:0/3938128849 >> v2:172.21.15.162:3300/0 conn(0x7fa9e80a2650 0x7fa9e8134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 monclient: found mon.noname-d 2024-09-16T15:00:55.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:55.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.631+0000 7fa9dffff640 1 -- 172.21.15.134:0/3938128849 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e8073250 con 0x7fa9e8070e90 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.634+0000 7fa9dffff640 1 -- 172.21.15.134:0/3938128849 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 msgr2=0x7fa9e80a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.634+0000 7fa9dffff640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009870 tx=0x7fa9d002a130 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7f788cfe890109b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.634+0000 7fa9e77fe640 1 -- 172.21.15.134:0/3938128849 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa9d00307b0 con 0x7fa9e8070e90 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.634+0000 7fa9e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: dump: 2024-09-16T15:00:55.643 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.644 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 1 -- 172.21.15.134:0/3938128849 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0030930 con 0x7fa9e8070e90 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 1 -- 172.21.15.134:0/3938128849 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa9d00313a0 con 0x7fa9e8070e90 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 5 monclient: authenticate success, global_id 4344 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 4 set_mon_vals no callback set 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9e77fe640 10 monclient: dump: 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.646 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 10 monclient: shutdown 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 -- 172.21.15.134:0/3938128849 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 msgr2=0x7fa9e80a2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009870 tx=0x7fa9d002a130 comp rx=0 tx=0).stop 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 -- 172.21.15.134:0/3938128849 shutdown_connections 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 --2- 172.21.15.134:0/3938128849 >> v2:172.21.15.162:3300/0 conn(0x7fa9e80a2650 0x7fa9e8134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa9e8070e90 0x7fa9e80a2110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 --2- 172.21.15.134:0/3938128849 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a1d30 0x7fa9e806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 -- 172.21.15.134:0/3938128849 >> 172.21.15.134:0/3938128849 conn(0x7fa9e809abf0 msgr2=0x7fa9e80a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 -- 172.21.15.134:0/3938128849 shutdown_connections 2024-09-16T15:00:55.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.635+0000 7fa9ee965640 1 -- 172.21.15.134:0/3938128849 wait complete. 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.636+0000 7fa9ee965640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.636+0000 7fa9ee965640 10 monclient: monmap: 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.648 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.636+0000 7fa9ee965640 1 Processor -- start 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.636+0000 7fa9ee965640 1 -- start start 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.636+0000 7fa9ee965640 10 monclient: init 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient: _add_conns ranks=[1,0,7,6,2,8,5,3,4] 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e8070e90 0x7fa9e81cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-f con 0x7fa9e8070e90 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:55.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-a con 0x7fa9e80a1d30 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a2650 0x7fa9e8135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): picked mon.noname-i con 0x7fa9e80a2650 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e81cb990 con 0x7fa9e80a1d30 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e8124450 con 0x7fa9e8070e90 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e80a73f0 con 0x7fa9e80a2650 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ee965640 10 monclient(hunting): authenticate will time out at 792.226074s 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51914/0 (socket says 172.21.15.134:51914) 2024-09-16T15:00:55.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 -- 172.21.15.134:0/1657382941 learned_addr learned my addr 172.21.15.134:0/1657382941 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): get_auth_request con 0x7fa9e80a1d30 auth_method 0 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9e7fff640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e8070e90 0x7fa9e81cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9e7fff640 10 monclient(hunting): get_auth_request con 0x7fa9e8070e90 auth_method 0 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient(hunting): handle_auth_done global_id 4383 payload 824 2024-09-16T15:00:55.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9ecf02640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a2650 0x7fa9e8135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 -- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a2650 msgr2=0x7fa9e8135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a2650 0x7fa9e8135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 -- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e8070e90 msgr2=0x7fa9e81cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e8070e90 0x7fa9e81cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 monclient: found mon.noname-a 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.637+0000 7fa9dffff640 1 -- 172.21.15.134:0/1657382941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00094d0 con 0x7fa9e80a1d30 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9dffff640 1 -- 172.21.15.134:0/1657382941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 msgr2=0x7fa9e8134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9dffff640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00090b0 tx=0x7fa9d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa9d0004460 con 0x7fa9e80a1d30 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: dump: 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.653 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0003a80 con 0x7fa9e80a1d30 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 5 monclient: authenticate success, global_id 4383 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 10 monclient: _renew_subs 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e8124170 con 0x7fa9e80a1d30 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa9d0003bf0 con 0x7fa9e80a1d30 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e57fa640 10 monclient: dump: 2024-09-16T15:00:55.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9e6ffd640 4 set_mon_vals no callback set 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 10 monclient: _renew_subs 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.639+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e8123e90 con 0x7fa9e80a1d30 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa9d0039070 con 0x7fa9e80a1d30 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9e57fa640 1 --2- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 0x7fa9c803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9ee965640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9ee965640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9ee965640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7fa9d006a9d0 con 0x7fa9e80a1d30 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.640+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8071270 con 0x7fa9e80a1d30 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.641+0000 7fa9e7fff640 1 --2- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 0x7fa9c803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.641+0000 7fa9e7fff640 10 monclient: get_auth_request con 0x7fa9c8038880 auth_method 0 2024-09-16T15:00:55.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.643+0000 7fa9e7fff640 1 --2- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 0x7fa9c803ad40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800a970 tx=0x7fa9d8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.645+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa9d0041020 con 0x7fa9e80a1d30 2024-09-16T15:00:55.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.645+0000 7fa9e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:55.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.645+0000 7fa9e57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:55.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.805+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9e8067a30 con 0x7fa9c8038880 2024-09-16T15:00:55.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.808+0000 7fa9e57fa640 1 -- 172.21.15.134:0/1657382941 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41030 (secure 0 0 0) 0x7fa9e8067a30 con 0x7fa9c8038880 2024-09-16T15:00:55.809 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:00:55.810 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 msgr2=0x7fa9c803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 0x7fa9c803ad40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800a970 tx=0x7fa9d8008040 comp rx=0 tx=0).stop 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 10 monclient: shutdown 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 msgr2=0x7fa9e8134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00090b0 tx=0x7fa9d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 shutdown_connections 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> v2:172.21.15.162:6800/980547263 conn(0x7fa9c8038880 0x7fa9c803ad40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa9e80a2650 0x7fa9e8135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e80a1d30 0x7fa9e8134d50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 --2- 172.21.15.134:0/1657382941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e8070e90 0x7fa9e81cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 >> 172.21.15.134:0/1657382941 conn(0x7fa9e809abf0 msgr2=0x7fa9e80a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 shutdown_connections 2024-09-16T15:00:55.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.809+0000 7fa9ee965640 1 -- 172.21.15.134:0/1657382941 wait complete. 2024-09-16T15:00:55.821 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-16T15:00:55.343686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163084,"kb_used_data":972,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067956,"statfs":{"total":579820584960,"available":579653586944,"internally_reserved":0,"allocated":995328,"data_stored":1322734,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.183185"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319854+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:00:49.319854+0000","last_peered":"2024-09-16T15:00:49.319854+0000","last_clean":"2024-09-16T15:00:49.319854+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:00:49.319854+0000","last_undegraded":"2024-09-16T15:00:49.319854+0000","last_fullsized":"2024-09-16T15:00:49.319854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320197+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:00:49.320197+0000","last_peered":"2024-09-16T15:00:49.320197+0000","last_clean":"2024-09-16T15:00:49.320197+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:00:49.320197+0000","last_undegraded":"2024-09-16T15:00:49.320197+0000","last_fullsized":"2024-09-16T15:00:49.320197+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319591+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:00:49.319591+0000","last_peered":"2024-09-16T15:00:49.319591+0000","last_clean":"2024-09-16T15:00:49.319591+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:00:49.319591+0000","last_undegraded":"2024-09-16T15:00:49.319591+0000","last_fullsized":"2024-09-16T15:00:49.319591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.320254+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:00:49.320254+0000","last_peered":"2024-09-16T15:00:49.320254+0000","last_clean":"2024-09-16T15:00:49.320254+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:00:49.320254+0000","last_undegraded":"2024-09-16T15:00:49.320254+0000","last_fullsized":"2024-09-16T15:00:49.320254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.330191+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:00:49.330191+0000","last_peered":"2024-09-16T15:00:49.330191+0000","last_clean":"2024-09-16T15:00:49.330191+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:00:49.330191+0000","last_undegraded":"2024-09-16T15:00:49.330191+0000","last_fullsized":"2024-09-16T15:00:49.330191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.566752+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:00:49.566752+0000","last_peered":"2024-09-16T15:00:49.566752+0000","last_clean":"2024-09-16T15:00:49.566752+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:00:49.566752+0000","last_undegraded":"2024-09-16T15:00:49.566752+0000","last_fullsized":"2024-09-16T15:00:49.566752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.319607+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:00:49.319607+0000","last_peered":"2024-09-16T15:00:49.319607+0000","last_clean":"2024-09-16T15:00:49.319607+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:00:49.319607+0000","last_undegraded":"2024-09-16T15:00:49.319607+0000","last_fullsized":"2024-09-16T15:00:49.319607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-16T15:00:49.323958+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:00:49.323958+0000","last_peered":"2024-09-16T15:00:49.323958+0000","last_clean":"2024-09-16T15:00:49.323958+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:00:49.323958+0000","last_undegraded":"2024-09-16T15:00:49.323958+0000","last_fullsized":"2024-09-16T15:00:49.323958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-16T15:00:48.539257+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:00:48.539257+0000","last_peered":"2024-09-16T15:00:48.539257+0000","last_clean":"2024-09-16T15:00:48.539257+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:00:48.539257+0000","last_undegraded":"2024-09-16T15:00:48.539257+0000","last_fullsized":"2024-09-16T15:00:48.539257+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":44353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.218}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":573262,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:00:55.822 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-16T15:00:55.822 DEBUG:tasks.mon_thrash:Yielding 2024-09-16T15:00:55.822 INFO:teuthology.run_tasks:Running task rados... 2024-09-16T15:00:55.834 INFO:tasks.rados:Beginning rados... 2024-09-16T15:00:55.834 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-16T15:00:55.834 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}} 2024-09-16T15:00:55.834 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-16T15:00:55.834 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-16T15:00:55.857 INFO:tasks.rados:joining rados 2024-09-16T15:00:55.857 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-16T15:00:55.858 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726498855, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-16T15:00:55.858 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:00:55.858 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:00:55.858 INFO:tasks.rados:clients are ['client.0'] 2024-09-16T15:00:55.858 INFO:tasks.rados:starting run 0 out of 1 2024-09-16T15:00:55.859 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-16T15:00:55.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-16T15:00:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.954+0000 7fd0a6b53640 10 monclient: get_monmap_and_config 2024-09-16T15:00:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.954+0000 7fd0a6b53640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.955+0000 7fd0a6b53640 10 monclient: monmap: 2024-09-16T15:00:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:55.955954+0000 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:55.955954+0000 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.958 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.958 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:55.958 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:55.958 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.955+0000 7fd0a6b53640 1 Processor -- start 2024-09-16T15:00:55.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.955+0000 7fd0a6b53640 1 -- start start 2024-09-16T15:00:55.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.955+0000 7fd0a6b53640 10 monclient: init 2024-09-16T15:00:55.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient: _add_conns ranks=[10,3,6,2,8,4,0,1,7,5,9] 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0a0136030 0x7fd0a0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-f con 0x7fd0a0136030 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a009bfe0 0x7fd0a009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-g con 0x7fd0a009bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-d con 0x7fd0a0136c60 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a009b600 con 0x7fd0a0136030 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00a5740 con 0x7fd0a009bfe0 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00a52c0 con 0x7fd0a0136c60 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a6b53640 10 monclient(hunting): authenticate will time out at 792.545435s 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:42384/0 (socket says 172.21.15.134:42384) 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 -- 172.21.15.134:0/3015130314 learned_addr learned my addr 172.21.15.134:0/3015130314 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): get_auth_request con 0x7fd0a0136c60 auth_method 0 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a48ef640 1 --2- 172.21.15.134:0/3015130314 >> v2:172.21.15.162:3300/0 conn(0x7fd0a0136030 0x7fd0a0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd09ffff640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a009bfe0 0x7fd0a009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a48ef640 10 monclient(hunting): get_auth_request con 0x7fd0a0136030 auth_method 0 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a48ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a48ef640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a48ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd09ffff640 10 monclient(hunting): get_auth_request con 0x7fd0a009bfe0 auth_method 0 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd09ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd09ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd09ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_done global_id 4353 payload 274 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 -- 172.21.15.134:0/3015130314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a009bfe0 msgr2=0x7fd0a009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a009bfe0 0x7fd0a009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 -- 172.21.15.134:0/3015130314 >> v2:172.21.15.162:3300/0 conn(0x7fd0a0136030 msgr2=0x7fd0a0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 --2- 172.21.15.134:0/3015130314 >> v2:172.21.15.162:3300/0 conn(0x7fd0a0136030 0x7fd0a0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 monclient: found mon.noname-d 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:55.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.956+0000 7fd0a50f0640 1 -- 172.21.15.134:0/3015130314 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0073250 con 0x7fd0a0136c60 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.959+0000 7f2e37496640 10 monclient: get_monmap_and_config 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.959+0000 7f2e37496640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.959+0000 7f2e37496640 10 monclient: monmap: 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:55.960561+0000 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:55.960561+0000 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:55.966 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:55.967 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.959+0000 7fd0a50f0640 1 -- 172.21.15.134:0/3015130314 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 msgr2=0x7fd0a01370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.959+0000 7fd0a50f0640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd098009e50 tx=0x7fd09802a560 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e5019891038c09f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 1 -- 172.21.15.134:0/3015130314 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd098007a70 con 0x7fd0a0136c60 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: dump: 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.969 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 1 -- 172.21.15.134:0/3015130314 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09802d050 con 0x7fd0a0136c60 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 5 monclient: authenticate success, global_id 4353 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 4 set_mon_vals no callback set 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 1 -- 172.21.15.134:0/3015130314 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd098007bf0 con 0x7fd0a0136c60 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 Processor -- start 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd09f7fe640 10 monclient: dump: 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.971 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.972 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.973 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 -- start start 2024-09-16T15:00:55.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient: init 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 10 monclient: shutdown 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 -- 172.21.15.134:0/3015130314 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 msgr2=0x7fd0a01370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd098009e50 tx=0x7fd09802a560 comp rx=0 tx=0).stop 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 -- 172.21.15.134:0/3015130314 shutdown_connections 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd0a0136c60 0x7fd0a01370e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/3015130314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a009bfe0 0x7fd0a009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/3015130314 >> v2:172.21.15.162:3300/0 conn(0x7fd0a0136030 0x7fd0a0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 -- 172.21.15.134:0/3015130314 >> 172.21.15.134:0/3015130314 conn(0x7fd0a00a9ea0 msgr2=0x7fd0a00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:55.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient: _add_conns ranks=[3,10,7,0,8,6,4,2,9,1,5] 2024-09-16T15:00:55.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 -- 172.21.15.134:0/3015130314 shutdown_connections 2024-09-16T15:00:55.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7fd0a6b53640 1 -- 172.21.15.134:0/3015130314 wait complete. 2024-09-16T15:00:55.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-g con 0x7f2e30069e50 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:55.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e3009d890 0x7f2e3006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-f con 0x7f2e3009d890 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 0x7f2e301379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-i con 0x7f2e30137510 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3006cf00 con 0x7f2e3009d890 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300a6200 con 0x7f2e30069e50 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3006acb0 con 0x7f2e30137510 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.960+0000 7f2e37496640 10 monclient(hunting): authenticate will time out at 792.549972s 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:32900/0 (socket says 172.21.15.134:32900) 2024-09-16T15:00:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 -- 172.21.15.134:0/3241620213 learned_addr learned my addr 172.21.15.134:0/3241620213 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): get_auth_request con 0x7f2e30069e50 auth_method 0 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 1 --2- 172.21.15.134:0/3241620213 >> v2:172.21.15.162:3300/0 conn(0x7f2e3009d890 0x7f2e3006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 0x7f2e301379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): get_auth_request con 0x7f2e30137510 auth_method 0 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): get_auth_request con 0x7f2e3009d890 auth_method 0 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient(hunting): handle_auth_done global_id 4395 payload 274 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 -- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 msgr2=0x7f2e301379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 0x7f2e301379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 -- 172.21.15.134:0/3241620213 >> v2:172.21.15.162:3300/0 conn(0x7f2e3009d890 msgr2=0x7f2e3006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:55.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 --2- 172.21.15.134:0/3241620213 >> v2:172.21.15.162:3300/0 conn(0x7f2e3009d890 0x7f2e3006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 monclient: found mon.noname-g 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35232640 1 -- 172.21.15.134:0/3241620213 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e30073250 con 0x7f2e30069e50 2024-09-16T15:00:55.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7f2e35a33640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 0x7f2e301379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7fd0a6b53640 10 monclient: build_initial_monmap 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7fd0a6b53640 10 monclient: monmap: 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:55.987 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7fd0a6b53640 1 Processor -- start 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7fd0a6b53640 1 -- start start 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.961+0000 7fd0a6b53640 10 monclient: init 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient: _add_conns ranks=[0,4,3,2,7,5,6,1,8] 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-a con 0x7fd0a009bfe0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:55.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-c con 0x7fd0a0136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 0x7fd0a01cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): picked mon.noname-g con 0x7fd0a0136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0120fe0 con 0x7fd0a009bfe0 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01226d0 con 0x7fd0a0136c60 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0120a20 con 0x7fd0a0136030 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a6b53640 10 monclient(hunting): authenticate will time out at 792.551597s 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51930/0 (socket says 172.21.15.134:51930) 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33446/0 (socket says 172.21.15.134:33446) 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 1 -- 172.21.15.134:0/1747898111 learned_addr learned my addr 172.21.15.134:0/1747898111 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): get_auth_request con 0x7fd0a009bfe0 auth_method 0 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 0x7fd0a01cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): get_auth_request con 0x7fd0a0136030 auth_method 0 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): get_auth_request con 0x7fd0a0136c60 auth_method 0 2024-09-16T15:00:55.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd09ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a50f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.962+0000 7fd0a48ef640 10 monclient(hunting): handle_auth_done global_id 4392 payload 824 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 -- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 msgr2=0x7fd0a01354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 -- 172.21.15.134:0/1747898111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 msgr2=0x7fd0a01cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 0x7fd0a01cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 10 monclient: _finish_hunting 0 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 monclient: found mon.noname-a 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a48ef640 1 -- 172.21.15.134:0/1747898111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098009b30 con 0x7fd0a009bfe0 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd09ffff640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:55.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.963+0000 7fd0a50f0640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 0x7fd0a01cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:55.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e35232640 1 -- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 msgr2=0x7f2e3006a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e35232640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02a0a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=5d68f33034c983ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 1 -- 172.21.15.134:0/3241620213 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e2c02ae00 con 0x7f2e30069e50 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: dump: 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.995 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:55.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 1 -- 172.21.15.134:0/3241620213 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0307b0 con 0x7f2e30069e50 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e37496640 5 monclient: authenticate success, global_id 4395 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e37496640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e37496640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 1 -- 172.21.15.134:0/3241620213 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e2c030d90 con 0x7f2e30069e50 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e37496640 4 set_mon_vals no callback set 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.964+0000 7f2e27fff640 10 monclient: dump: 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:55.997 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 10 monclient: shutdown 2024-09-16T15:00:55.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:55.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a48ef640 1 -- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 msgr2=0x7fd0a0134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:55.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 -- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 msgr2=0x7f2e3006a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:55.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a48ef640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd09400d860 tx=0x7fd09400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009870 tx=0x7f2e2c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 -- 172.21.15.134:0/3241620213 shutdown_connections 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2e30137510 0x7f2e301379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 --2- 172.21.15.134:0/3241620213 >> v2:172.21.15.162:3300/0 conn(0x7f2e3009d890 0x7f2e3006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 --2- 172.21.15.134:0/3241620213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e30069e50 0x7f2e3006a230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 -- 172.21.15.134:0/3241620213 >> 172.21.15.134:0/3241620213 conn(0x7f2e300a2e90 msgr2=0x7f2e300a32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 -- 172.21.15.134:0/3241620213 shutdown_connections 2024-09-16T15:00:56.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd09400de50 con 0x7fd0a009bfe0 2024-09-16T15:00:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7f2e37496640 1 -- 172.21.15.134:0/3241620213 wait complete. 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: dump: 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 5 monclient: authenticate success, global_id 4392 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094014430 con 0x7fd0a009bfe0 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 10 monclient: _renew_subs 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a00a7d20 con 0x7fd0a009bfe0 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd094014a10 con 0x7fd0a009bfe0 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09d7fa640 10 monclient: dump: 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 10 monclient: _renew_subs 2024-09-16T15:00:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a009fc00 con 0x7fd0a009bfe0 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.965+0000 7fd09effd640 4 set_mon_vals no callback set 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd094014bb0 con 0x7fd0a009bfe0 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 10 monclient: monmap: 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:56.006 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.007 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd0a6b53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd0a6b53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd0a6b53640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a009c400 con 0x7fd0a009bfe0 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 1 Processor -- start 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 1 -- start start 2024-09-16T15:00:56.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 10 monclient: init 2024-09-16T15:00:56.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd09d7fa640 1 --2- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 0x7fd08403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7fd094049f40 con 0x7fd0a009bfe0 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.966+0000 7f2e37496640 10 monclient: _add_conns ranks=[8,3,2,1,0,7,5,4,6] 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-e con 0x7f2e30069e50 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e3009d890 0x7f2e3008f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-g con 0x7f2e3009d890 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 0x7f2e301cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7fd09ffff640 1 --2- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 0x7fd08403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7fd09ffff640 10 monclient: get_auth_request con 0x7fd084038880 auth_method 0 2024-09-16T15:00:56.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41860/0 (socket says 172.21.15.134:41860) 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 -- 172.21.15.134:0/3077246710 learned_addr learned my addr 172.21.15.134:0/3077246710 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): picked mon.noname-b con 0x7f2e30137510 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3008c5b0 con 0x7f2e30137510 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 0x7f2e301cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e3009d890 0x7f2e3008f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e30136950 con 0x7f2e3009d890 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300a3430 con 0x7f2e30069e50 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e37496640 10 monclient(hunting): authenticate will time out at 792.556311s 2024-09-16T15:00:56.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): get_auth_request con 0x7f2e30069e50 auth_method 0 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): get_auth_request con 0x7f2e30137510 auth_method 0 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): get_auth_request con 0x7f2e3009d890 auth_method 0 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e34a31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient(hunting): handle_auth_done global_id 4418 payload 824 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 -- 172.21.15.134:0/3077246710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e3009d890 msgr2=0x7f2e3008f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e3009d890 0x7f2e3008f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 -- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 msgr2=0x7f2e301cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 0x7f2e301cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 monclient: found mon.noname-e 2024-09-16T15:00:56.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35232640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009a70 con 0x7f2e30069e50 2024-09-16T15:00:56.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.967+0000 7f2e35a33640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 0x7f2e301cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:56.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.969+0000 7fd09ffff640 1 --2- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 0x7fd08403ad40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00a32b0 tx=0x7fd08c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.969+0000 7f2e35232640 1 -- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 msgr2=0x7f2e301cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.969+0000 7f2e35232640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097c0 tx=0x7f2e2c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e2c003610 con 0x7f2e30069e50 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: dump: 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.019 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0037b0 con 0x7f2e30069e50 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 5 monclient: authenticate success, global_id 4418 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 10 monclient: _renew_subs 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c0094d0 con 0x7f2e30069e50 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e2c003920 con 0x7f2e30069e50 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e277fe640 4 set_mon_vals no callback set 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e25ffb640 10 monclient: dump: 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.021 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 10 monclient: _renew_subs 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.970+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3008c330 con 0x7f2e30069e50 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2e2c039070 con 0x7f2e30069e50 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e25ffb640 1 --2- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 0x7f2e1403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7f2e2c06aa90 con 0x7f2e30069e50 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e37496640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e37496640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e37496640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e300a5130 con 0x7f2e30069e50 2024-09-16T15:00:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd09401d410 con 0x7fd0a009bfe0 2024-09-16T15:00:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7fd09d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7fd09d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e34a31640 1 --2- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 0x7f2e1403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.971+0000 7f2e34a31640 10 monclient: get_auth_request con 0x7f2e14038880 auth_method 0 2024-09-16T15:00:56.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.974+0000 7f2e34a31640 1 --2- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 0x7f2e1403ad40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e30068b30 tx=0x7f2e20009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.976+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2e2c041020 con 0x7f2e30069e50 2024-09-16T15:00:56.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.976+0000 7f2e25ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:55.976+0000 7f2e25ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.142+0000 7f2e37496640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-16T15:00:56.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.142+0000 7f2e37496640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-16T15:00:56.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.142+0000 7f2e37496640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.142+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} v 0) v1 -- 0x7f2e3009e2a0 con 0x7f2e30069e50 2024-09-16T15:00:56.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.203+0000 7fd0a6b53640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.203+0000 7fd0a6b53640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.203+0000 7fd0a6b53640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.203+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0a00a81e0 con 0x7fd0a009bfe0 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.204+0000 7fd09d7fa640 1 -- 172.21.15.134:0/1747898111 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7fd094014e60 con 0x7fd0a009bfe0 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.204+0000 7fd09d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:00:56.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.204+0000 7fd09d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:56.206 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":4,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:00:56.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 msgr2=0x7fd08403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 0x7fd08403ad40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00a32b0 tx=0x7fd08c008040 comp rx=0 tx=0).stop 2024-09-16T15:00:56.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 10 monclient: shutdown 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 msgr2=0x7fd0a0134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd09400d860 tx=0x7fd09400dd20 comp rx=0 tx=0).stop 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 shutdown_connections 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> v2:172.21.15.162:6800/980547263 conn(0x7fd084038880 0x7fd08403ad40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0136c60 0x7fd0a01cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd0a0136030 0x7fd0a01354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 --2- 172.21.15.134:0/1747898111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a009bfe0 0x7fd0a0134f60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 >> 172.21.15.134:0/1747898111 conn(0x7fd0a00a9ea0 msgr2=0x7fd0a01bedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 shutdown_connections 2024-09-16T15:00:56.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.206+0000 7fd0a6b53640 1 -- 172.21.15.134:0/1747898111 wait complete. 2024-09-16T15:00:56.218 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:00:56.218 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:00:56.218 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:00:56.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:00:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.315+0000 7fef837f4640 10 monclient: get_monmap_and_config 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.315+0000 7fef837f4640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.315+0000 7fef837f4640 10 monclient: monmap: 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:56.316194+0000 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:56.316194+0000 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:56.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.315+0000 7fef837f4640 1 Processor -- start 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 -- start start 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient: init 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient: _add_conns ranks=[10,3,7,0,2,4,5,9,1,8,6] 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef7c136e40 0x7fef7c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-f con 0x7fef7c136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-g con 0x7fef7c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c135dc0 0x7fef7c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-i con 0x7fef7c135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c09abc0 con 0x7fef7c136e40 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c09ba50 con 0x7fef7c137220 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0a53c0 con 0x7fef7c135dc0 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef837f4640 10 monclient(hunting): authenticate will time out at 792.905606s 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef7c136e40 0x7fef7c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81d91640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c135dc0 0x7fef7c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:38232/0 (socket says 172.21.15.134:38232) 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 1 -- 172.21.15.134:0/4156855385 learned_addr learned my addr 172.21.15.134:0/4156855385 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81d91640 10 monclient(hunting): get_auth_request con 0x7fef7c135dc0 auth_method 0 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81d91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81d91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81d91640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): get_auth_request con 0x7fef7c136e40 auth_method 0 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): get_auth_request con 0x7fef7c137220 auth_method 0 2024-09-16T15:00:56.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.316+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef81d91640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef81d91640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef81d91640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 10 monclient(hunting): handle_auth_done global_id 4422 payload 274 2024-09-16T15:00:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 -- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c135dc0 msgr2=0x7fef7c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 --2- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c135dc0 0x7fef7c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 -- 172.21.15.134:0/4156855385 >> v2:172.21.15.162:3300/0 conn(0x7fef7c136e40 msgr2=0x7fef7c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 --2- 172.21.15.134:0/4156855385 >> v2:172.21.15.162:3300/0 conn(0x7fef7c136e40 0x7fef7c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 monclient: found mon.noname-g 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.317+0000 7fef80d8f640 1 -- 172.21.15.134:0/4156855385 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c073250 con 0x7fef7c137220 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef80d8f640 1 -- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 msgr2=0x7fef7c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef80d8f640 1 --2- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef700098a0 tx=0x7fef7002a0d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c50a1b7420fba622 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 1 -- 172.21.15.134:0/4156855385 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef7002adf0 con 0x7fef7c137220 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: dump: 2024-09-16T15:00:56.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 5 monclient: authenticate success, global_id 4422 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 1 -- 172.21.15.134:0/4156855385 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef700307b0 con 0x7fef7c137220 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 4 set_mon_vals no callback set 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 1 -- 172.21.15.134:0/4156855385 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef70030d90 con 0x7fef7c137220 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef6ffff640 10 monclient: dump: 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 10 monclient: shutdown 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 -- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 msgr2=0x7fef7c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 --2- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef700098a0 tx=0x7fef7002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 -- 172.21.15.134:0/4156855385 shutdown_connections 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 --2- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c135dc0 0x7fef7c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 --2- 172.21.15.134:0/4156855385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef7c137220 0x7fef7c137600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 --2- 172.21.15.134:0/4156855385 >> v2:172.21.15.162:3300/0 conn(0x7fef7c136e40 0x7fef7c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 -- 172.21.15.134:0/4156855385 >> 172.21.15.134:0/4156855385 conn(0x7fef7c0a6110 msgr2=0x7fef7c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 -- 172.21.15.134:0/4156855385 shutdown_connections 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.320+0000 7fef837f4640 1 -- 172.21.15.134:0/4156855385 wait complete. 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.321+0000 7fef837f4640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.321+0000 7fef837f4640 10 monclient: monmap: 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.321+0000 7fef837f4640 1 Processor -- start 2024-09-16T15:00:56.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.321+0000 7fef837f4640 1 -- start start 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.321+0000 7fef837f4640 10 monclient: init 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient: _add_conns ranks=[4,6,7,2,0,3,8,5,1] 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 0x7fef7c134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-c con 0x7fef7c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 0x7fef7c1352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-d con 0x7fef7c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c137220 0x7fef7c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): picked mon.noname-i con 0x7fef7c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c120d70 con 0x7fef7c135dc0 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1c07b0 con 0x7fef7c136e40 2024-09-16T15:00:56.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c136390 con 0x7fef7c137220 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef837f4640 10 monclient(hunting): authenticate will time out at 792.911306s 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 0x7fef7c1352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 0x7fef7c134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 0x7fef7c1352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57420/0 (socket says 172.21.15.134:57420) 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 0x7fef7c134d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:58286/0 (socket says 172.21.15.134:58286) 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 -- 172.21.15.134:0/454937191 learned_addr learned my addr 172.21.15.134:0/454937191 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): get_auth_request con 0x7fef7c136e40 auth_method 0 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): get_auth_request con 0x7fef7c135dc0 auth_method 0 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81d91640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c137220 0x7fef7c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81d91640 10 monclient(hunting): get_auth_request con 0x7fef7c137220 auth_method 0 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81d91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81d91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81d91640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient(hunting): handle_auth_done global_id 4362 payload 824 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c137220 msgr2=0x7fef7c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fef7c137220 0x7fef7c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 msgr2=0x7fef7c134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 0x7fef7c134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 monclient: found mon.noname-d 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef80d8f640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700094d0 con 0x7fef7c136e40 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.322+0000 7fef81590640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fef7c135dc0 0x7fef7c134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.324+0000 7fef80d8f640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 msgr2=0x7fef7c1352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef80d8f640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 0x7fef7c1352a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef70035040 tx=0x7fef700046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef70030d90 con 0x7fef7c136e40 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: dump: 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.344 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70003610 con 0x7fef7c136e40 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 5 monclient: authenticate success, global_id 4362 2024-09-16T15:00:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 10 monclient: _renew_subs 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c121e80 con 0x7fef7c136e40 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef70003bf0 con 0x7fef7c136e40 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6dffb640 10 monclient: dump: 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.346 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef6effd640 4 set_mon_vals no callback set 2024-09-16T15:00:56.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 10 monclient: _renew_subs 2024-09-16T15:00:56.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.325+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c122440 con 0x7fef7c136e40 2024-09-16T15:00:56.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef837f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef837f4640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 0x7fef7c02b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c071b20 con 0x7fef7c02b260 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fef70039040 con 0x7fef7c136e40 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 0x7fef7c02b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient: get_auth_request con 0x7fef7c02b260 auth_method 0 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef6dffb640 1 --2- 172.21.15.134:0/454937191 >> v2:172.21.15.162:6800/980547263 conn(0x7fef60038880 0x7fef6003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7fef7006a290 con 0x7fef7c136e40 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 10 monclient(hunting): handle_auth_done global_id 4401 payload 824 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81590640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef7c061ae0 con 0x7fef7c02b260 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81d91640 1 --2- 172.21.15.134:0/454937191 >> v2:172.21.15.162:6800/980547263 conn(0x7fef60038880 0x7fef6003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.326+0000 7fef81d91640 10 monclient: get_auth_request con 0x7fef60038880 auth_method 0 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.328+0000 7fef81590640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 msgr2=0x7fef7c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.328+0000 7fef81590640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 0x7fef7c02b640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef78011d20 tx=0x7fef7800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef78004240 con 0x7fef7c02b260 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 10 monclient: dump: 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fef780043a0 con 0x7fef7c02b260 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 msgr2=0x7fef7c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 0x7fef7c02b640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef78011d20 tx=0x7fef7800f610 comp rx=0 tx=0).stop 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 msgr2=0x7fef7c02b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef6dffb640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02b260 0x7fef7c02b640 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef78011d20 tx=0x7fef7800f610 comp rx=0 tx=0).stop 2024-09-16T15:00:56.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.329+0000 7fef81d91640 1 --2- 172.21.15.134:0/454937191 >> v2:172.21.15.162:6800/980547263 conn(0x7fef60038880 0x7fef6003ad40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0a30c0 tx=0x7fef7400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef837f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:56.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef837f4640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 0x7fef7c02bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef837f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c122010 con 0x7fef7c02bb80 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 0x7fef7c02bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient: get_auth_request con 0x7fef7c02bb80 auth_method 0 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 10 monclient(hunting): handle_auth_done global_id 4410 payload 824 2024-09-16T15:00:56.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.335+0000 7fef81590640 1 -- 172.21.15.134:0/454937191 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef7c07a680 con 0x7fef7c02bb80 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.337+0000 7fef81590640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 msgr2=0x7fef7c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.337+0000 7fef81590640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 0x7fef7c02bf90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef7800f650 tx=0x7fef780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fef7801edc0 con 0x7fef7c02bb80 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 10 monclient: dump: 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.356 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7fef7801e950 con 0x7fef7c02bb80 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:56.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 msgr2=0x7fef7c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 0x7fef7c02bf90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef7800f650 tx=0x7fef780046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 msgr2=0x7fef7c02bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef6dffb640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef7c02bb80 0x7fef7c02bf90 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef7800f650 tx=0x7fef780046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:00:56.358 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.359 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 25, 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.360 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.361 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.362 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.363 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.364 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:56.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:56.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:56.373 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:56.375 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.381 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:56.382 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 >> v2:172.21.15.162:6800/980547263 conn(0x7fef60038880 msgr2=0x7fef6003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 --2- 172.21.15.134:0/454937191 >> v2:172.21.15.162:6800/980547263 conn(0x7fef60038880 0x7fef6003ad40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0a30c0 tx=0x7fef7400b040 comp rx=0 tx=0).stop 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 10 monclient: shutdown 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 msgr2=0x7fef7c1352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 --2- 172.21.15.134:0/454937191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fef7c136e40 0x7fef7c1352a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef70035040 tx=0x7fef700046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef81d91640 1 -- 172.21.15.134:0/454937191 reap_dead start 2024-09-16T15:00:56.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef81d91640 1 -- 172.21.15.134:0/454937191 reap_dead start 2024-09-16T15:00:56.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 shutdown_connections 2024-09-16T15:00:56.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 >> 172.21.15.134:0/454937191 conn(0x7fef7c0a6110 msgr2=0x7fef7c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 shutdown_connections 2024-09-16T15:00:56.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.338+0000 7fef837f4640 1 -- 172.21.15.134:0/454937191 wait complete. 2024-09-16T15:00:56.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:56.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:00:56.385 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:00:56.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 10 monclient: get_monmap_and_config 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 10 monclient: monmap: 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:56.483135+0000 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:56.483135+0000 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.484 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:56.485 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 1 Processor -- start 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 1 -- start start 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.482+0000 7fafd3ad0640 10 monclient: init 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient: _add_conns ranks=[1,8,10,0,6,3,7,9,2,5,4] 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fafcc136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-e con 0x7fafcc137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 0x7fafcc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-f con 0x7fafcc13c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:56.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc073250 con 0x7fafcc136e40 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0a56e0 con 0x7fafcc13c370 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc09abc0 con 0x7fafcc137220 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd3ad0640 10 monclient(hunting): authenticate will time out at 793.072470s 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52992/0 (socket says 172.21.15.134:52992) 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 -- 172.21.15.134:0/4233180551 learned_addr learned my addr 172.21.15.134:0/4233180551 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): get_auth_request con 0x7fafcc137220 auth_method 0 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient(hunting): handle_auth_done global_id 4427 payload 274 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd206d640 1 --2- 172.21.15.134:0/4233180551 >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 0x7fafcc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 -- 172.21.15.134:0/4233180551 >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 msgr2=0x7fafcc13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 --2- 172.21.15.134:0/4233180551 >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 0x7fafcc13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 -- 172.21.15.134:0/4233180551 >> v1:172.21.15.162:6789/0 conn(0x7fafcc136e40 legacy=0x7fafcc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 monclient: found mon.noname-e 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd106b640 1 -- 172.21.15.134:0/4233180551 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc09ba50 con 0x7fafcc137220 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd206d640 10 monclient: get_auth_request con 0x7fafcc13c370 auth_method 0 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.483+0000 7fafd206d640 1 --2- 172.21.15.134:0/4233180551 >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 0x7fafcc13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd106b640 1 -- 172.21.15.134:0/4233180551 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 msgr2=0x7fafcc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd106b640 1 --2- 172.21.15.134:0/4233180551 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009870 tx=0x7fafbc02a100 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4da42d68f4b33197 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 1 -- 172.21.15.134:0/4233180551 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafbc0307b0 con 0x7fafcc137220 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: dump: 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd3ad0640 5 monclient: authenticate success, global_id 4427 2024-09-16T15:00:56.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd3ad0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 1 -- 172.21.15.134:0/4233180551 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc030930 con 0x7fafcc137220 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd3ad0640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd3ad0640 4 set_mon_vals no callback set 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 1 -- 172.21.15.134:0/4233180551 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafbc0313a0 con 0x7fafcc137220 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.486+0000 7fafd086a640 10 monclient: dump: 2024-09-16T15:00:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 10 monclient: shutdown 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 -- 172.21.15.134:0/4233180551 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 msgr2=0x7fafcc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/4233180551 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009870 tx=0x7fafbc02a100 comp rx=0 tx=0).stop 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 -- 172.21.15.134:0/4233180551 shutdown_connections 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/4233180551 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fafcc137220 0x7fafcc137600 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/4233180551 >> v2:172.21.15.162:3300/0 conn(0x7fafcc13c370 0x7fafcc13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 -- 172.21.15.134:0/4233180551 >> 172.21.15.134:0/4233180551 conn(0x7fafcc0a6110 msgr2=0x7fafcc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 -- 172.21.15.134:0/4233180551 shutdown_connections 2024-09-16T15:00:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.487+0000 7fafd3ad0640 1 -- 172.21.15.134:0/4233180551 wait complete. 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient: monmap: 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 Processor -- start 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 -- start start 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient: init 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient: _add_conns ranks=[7,2,6,5,3,0,8,4,1] 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fafcc136e40 0x7fafcc139f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-i con 0x7fafcc136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 0x7fafcc135d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-b con 0x7fafcc137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fafcc13c370 0x7fafcc1362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): picked mon.noname-d con 0x7fafcc13c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc122230 con 0x7fafcc137220 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc122480 con 0x7fafcc13c370 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc13b460 con 0x7fafcc136e40 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd3ad0640 10 monclient(hunting): authenticate will time out at 793.077833s 2024-09-16T15:00:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 0x7fafcc135d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd206d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fafcc13c370 0x7fafcc1362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 0x7fafcc135d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:55128/0 (socket says 172.21.15.134:55128) 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 1 -- 172.21.15.134:0/173601345 learned_addr learned my addr 172.21.15.134:0/173601345 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 10 monclient(hunting): get_auth_request con 0x7fafcc137220 auth_method 0 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd106b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd206d640 10 monclient(hunting): get_auth_request con 0x7fafcc13c370 auth_method 0 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd206d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd206d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.488+0000 7fafd206d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fafcc136e40 0x7fafcc139f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd206d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd206d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd206d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): get_auth_request con 0x7fafcc136e40 auth_method 0 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient(hunting): handle_auth_done global_id 4394 payload 824 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fafcc136e40 msgr2=0x7fafcc139f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fafcc136e40 0x7fafcc139f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fafcc13c370 msgr2=0x7fafcc1362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fafcc13c370 0x7fafcc1362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 monclient: found mon.noname-b 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd106b640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0094d0 con 0x7fafcc137220 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.489+0000 7fafd206d640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fafcc13c370 0x7fafcc1362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd106b640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 msgr2=0x7fafcc135d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd106b640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 0x7fafcc135d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097c0 tx=0x7fafbc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafbc004250 con 0x7fafcc137220 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: dump: 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc004410 con 0x7fafcc137220 2024-09-16T15:00:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 5 monclient: authenticate success, global_id 4394 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 10 monclient: _renew_subs 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc0a7ab0 con 0x7fafcc137220 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafbc032440 con 0x7fafcc137220 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc27fc640 10 monclient: dump: 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 10 monclient: _renew_subs 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafc3fff640 4 set_mon_vals no callback set 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.491+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc120ce0 con 0x7fafcc137220 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd3ad0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fafbc039040 con 0x7fafcc137220 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 0x7fafcc120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc122610 con 0x7fafcc11fc60 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 0x7fafcc120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient: get_auth_request con 0x7fafcc11fc60 auth_method 0 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafc27fc640 1 --2- 172.21.15.134:0/173601345 >> v2:172.21.15.162:6800/980547263 conn(0x7fafb00388b0 0x7fafb003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3534+0+0 (secure 0 0 0) 0x7fafbc06a730 con 0x7fafcc137220 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 10 monclient(hunting): handle_auth_done global_id 4403 payload 824 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.492+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafcc061ae0 con 0x7fafcc11fc60 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.493+0000 7fafd206d640 1 --2- 172.21.15.134:0/173601345 >> v2:172.21.15.162:6800/980547263 conn(0x7fafb00388b0 0x7fafb003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.493+0000 7fafd206d640 10 monclient: get_auth_request con 0x7fafb00388b0 auth_method 0 2024-09-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 msgr2=0x7fafcc120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 0x7fafcc120040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc800fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafc800de20 con 0x7fafcc11fc60 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 10 monclient: dump: 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fafc800d9b0 con 0x7fafcc11fc60 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 msgr2=0x7fafcc120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 0x7fafcc120040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc800fc60 comp rx=0 tx=0).stop 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 msgr2=0x7fafcc120040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafc27fc640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc11fc60 0x7fafcc120040 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc800fc60 comp rx=0 tx=0).stop 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.495+0000 7fafd206d640 1 --2- 172.21.15.134:0/173601345 >> v2:172.21.15.162:6800/980547263 conn(0x7fafb00388b0 0x7fafb003ad70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007fd0 tx=0x7fafc40076e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd3ad0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd3ad0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc071b20 con 0x7fafcc120580 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient: get_auth_request con 0x7fafcc120580 auth_method 0 2024-09-16T15:00:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 msgr2=0x7fafcc09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 msgr2=0x7fafcc09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.501+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.681+0000 7f2e25ffb640 1 -- 172.21.15.134:0/3077246710 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]=0 pool 'unique_pool_0' created v19) v1 ==== 130+0+0 (secure 0 0 0) 0x7f2e2c037070 con 0x7f2e30069e50 2024-09-16T15:00:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.681+0000 7f2e25ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-16T15:00:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.681+0000 7f2e25ffb640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' created 2024-09-16T15:00:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'unique_pool_0' created 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 msgr2=0x7f2e1403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 0x7f2e1403ad40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e30068b30 tx=0x7f2e20009190 comp rx=0 tx=0).stop 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 10 monclient: shutdown 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 msgr2=0x7f2e301cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097c0 tx=0x7f2e2c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 shutdown_connections 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e30137510 0x7f2e301cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> v2:172.21.15.162:6800/980547263 conn(0x7f2e14038880 0x7f2e1403ad40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e3009d890 0x7f2e3008f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 --2- 172.21.15.134:0/3077246710 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2e30069e50 0x7f2e301cfb10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 >> 172.21.15.134:0/3077246710 conn(0x7f2e300a2e90 msgr2=0x7f2e30136060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 shutdown_connections 2024-09-16T15:00:56.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.684+0000 7f2e37496640 1 -- 172.21.15.134:0/3077246710 wait complete. 2024-09-16T15:00:56.696 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient: get_auth_request con 0x7fafcc120580 auth_method 0 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 10 monclient(hunting): handle_auth_done global_id 4421 payload 824 2024-09-16T15:00:56.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.702+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fafcc07a680 con 0x7fafcc120580 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.704+0000 7fafd186c640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 msgr2=0x7fafcc09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.704+0000 7fafd186c640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc80036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fafc802b070 con 0x7fafcc120580 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 10 monclient: dump: 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.707 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.708 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fafc8023040 con 0x7fafcc120580 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 msgr2=0x7fafcc09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc80036f0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 msgr2=0x7fafcc09d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafc27fc640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc120580 0x7fafcc09d4f0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafc800f7a0 tx=0x7fafc80036f0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 26, 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.714 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:56.715 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:56.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:56.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.722 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:56.724 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.731 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.732 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:56.733 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:56.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 >> v2:172.21.15.162:6800/980547263 conn(0x7fafb00388b0 msgr2=0x7fafb003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/173601345 >> v2:172.21.15.162:6800/980547263 conn(0x7fafb00388b0 0x7fafb003ad70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafc4007fd0 tx=0x7fafc40076e0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 10 monclient: shutdown 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 msgr2=0x7fafcc135d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 --2- 172.21.15.134:0/173601345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafcc137220 0x7fafcc135d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0097c0 tx=0x7fafbc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd206d640 1 -- 172.21.15.134:0/173601345 reap_dead start 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd206d640 1 -- 172.21.15.134:0/173601345 reap_dead start 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 shutdown_connections 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 >> 172.21.15.134:0/173601345 conn(0x7fafcc0a6110 msgr2=0x7fafcc0a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 shutdown_connections 2024-09-16T15:00:56.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.705+0000 7fafd3ad0640 1 -- 172.21.15.134:0/173601345 wait complete. 2024-09-16T15:00:56.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:56.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:00:56.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:00:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.798+0000 7f31f1ad7640 10 monclient: get_monmap_and_config 2024-09-16T15:00:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.798+0000 7f31f1ad7640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.798+0000 7f31f1ad7640 10 monclient: monmap: 2024-09-16T15:00:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:56.799153+0000 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:56.799153+0000 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:56.800 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:56.801 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.798+0000 7f31f1ad7640 1 Processor -- start 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 -- start start 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient: init 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient: _add_conns ranks=[5,8,2,4,6,9,1,10,7,0,3] 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec134e20 0x7f31ec135200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-h con 0x7f31ec134e20 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-e con 0x7f31ec135740 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-b con 0x7f31ec06d7b0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec134a60 con 0x7f31ec06d7b0 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec066490 con 0x7f31ec134e20 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0709f0 con 0x7f31ec135740 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31f1ad7640 10 monclient(hunting): authenticate will time out at 793.388523s 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:55190/0 (socket says 172.21.15.134:55190) 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53016/0 (socket says 172.21.15.134:53016) 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 1 -- 172.21.15.134:0/892339935 learned_addr learned my addr 172.21.15.134:0/892339935 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): get_auth_request con 0x7f31ec06d7b0 auth_method 0 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): get_auth_request con 0x7f31ec135740 auth_method 0 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec134e20 0x7f31ec135200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): get_auth_request con 0x7f31ec134e20 auth_method 0 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 10 monclient(hunting): handle_auth_done global_id 4436 payload 274 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 1 -- 172.21.15.134:0/892339935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec134e20 msgr2=0x7f31ec135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.799+0000 7f31eaffd640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec134e20 0x7f31ec135200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 1 -- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 msgr2=0x7f31ec0a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 1 monclient: found mon.noname-e 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31eaffd640 1 -- 172.21.15.134:0/892339935 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec04e730 con 0x7f31ec135740 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.800+0000 7f31ebfff640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.802+0000 7f31eaffd640 1 -- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 msgr2=0x7f31ec0a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.802+0000 7f31eaffd640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31e000dc20 tx=0x7f31e002e2e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=597734385c2f6433 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.802+0000 7f31ea7fc640 1 -- 172.21.15.134:0/892339935 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e000d2e0 con 0x7f31ec135740 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: dump: 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.818 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 5 monclient: authenticate success, global_id 4436 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 1 -- 172.21.15.134:0/892339935 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e002edf0 con 0x7f31ec135740 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 4 set_mon_vals no callback set 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 1 -- 172.21.15.134:0/892339935 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e000bd30 con 0x7f31ec135740 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31ea7fc640 10 monclient: dump: 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 10 monclient: shutdown 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 -- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 msgr2=0x7f31ec0a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31e000dc20 tx=0x7f31e002e2e0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 -- 172.21.15.134:0/892339935 shutdown_connections 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31ec06d7b0 0x7f31ec0a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f31ec135740 0x7f31ec0a15c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/892339935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec134e20 0x7f31ec135200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 -- 172.21.15.134:0/892339935 >> 172.21.15.134:0/892339935 conn(0x7f31ec0a26b0 msgr2=0x7f31ec0a2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 -- 172.21.15.134:0/892339935 shutdown_connections 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.803+0000 7f31f1ad7640 1 -- 172.21.15.134:0/892339935 wait complete. 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.804+0000 7f31f1ad7640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.804+0000 7f31f1ad7640 10 monclient: monmap: 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.804+0000 7f31f1ad7640 1 Processor -- start 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.804+0000 7f31f1ad7640 1 -- start start 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.804+0000 7f31f1ad7640 10 monclient: init 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient: _add_conns ranks=[5,7,6,8,0,3,1,4,2] 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec06d7b0 0x7f31ec1c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-h con 0x7f31ec06d7b0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 0x7f31ec12f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-i con 0x7f31ec134e20 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): picked mon.noname-d con 0x7f31ec135740 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1c7c90 con 0x7f31ec06d7b0 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0a4ac0 con 0x7f31ec135740 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0a2000 con 0x7f31ec134e20 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31f1ad7640 10 monclient(hunting): authenticate will time out at 793.394463s 2024-09-16T15:00:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57436/0 (socket says 172.21.15.134:57436) 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 -- 172.21.15.134:0/3252783813 learned_addr learned my addr 172.21.15.134:0/3252783813 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): get_auth_request con 0x7f31ec135740 auth_method 0 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eaffd640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 0x7f31ec12f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eb7fe640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec06d7b0 0x7f31ec1c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eb7fe640 10 monclient(hunting): get_auth_request con 0x7f31ec06d7b0 auth_method 0 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient(hunting): handle_auth_done global_id 4380 payload 824 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 -- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 msgr2=0x7f31ec12f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 0x7f31ec12f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 -- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec06d7b0 msgr2=0x7f31ec1c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec06d7b0 0x7f31ec1c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 monclient: found mon.noname-d 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31ebfff640 1 -- 172.21.15.134:0/3252783813 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4009b30 con 0x7f31ec135740 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eaffd640 10 monclient: get_auth_request con 0x7f31ec134e20 auth_method 0 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.805+0000 7f31eaffd640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 0x7f31ec12f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31ebfff640 1 -- 172.21.15.134:0/3252783813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 msgr2=0x7f31ec09c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31ebfff640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31e400efc0 tx=0x7f31e400c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e400cdd0 con 0x7f31ec135740 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: dump: 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e400f040 con 0x7f31ec135740 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 5 monclient: authenticate success, global_id 4380 2024-09-16T15:00:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 10 monclient: _renew_subs 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e000d8a0 con 0x7f31ec135740 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31e4014d60 con 0x7f31ec135740 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31d3fff640 10 monclient: dump: 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 10 monclient: _renew_subs 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31e9ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.808+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec11c6c0 con 0x7f31ec135740 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f31e4014480 con 0x7f31ec135740 2024-09-16T15:00:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31f1ad7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31f1ad7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31f1ad7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec070fb0 con 0x7f31ec135740 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31d3fff640 1 --2- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 0x7f31d803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7f31e404a980 con 0x7f31ec135740 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31eb7fe640 1 --2- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 0x7f31d803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.809+0000 7f31eb7fe640 10 monclient: get_auth_request con 0x7f31d80388b0 auth_method 0 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.812+0000 7f31eb7fe640 1 --2- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 0x7f31d803ad70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009a50 tx=0x7f31dc008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.818+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f31e401b310 con 0x7f31ec135740 2024-09-16T15:00:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.818+0000 7f31d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.818+0000 7f31d3fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.876+0000 7f324a487640 10 monclient: get_monmap_and_config 2024-09-16T15:00:56.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.876+0000 7f324a487640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.876+0000 7f324a487640 10 monclient: monmap: 2024-09-16T15:00:56.877 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:56.877188+0000 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:56.877188+0000 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:56.878 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:56.879 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 Processor -- start 2024-09-16T15:00:56.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 -- start start 2024-09-16T15:00:56.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient: init 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient: _add_conns ranks=[10,4,3,0,1,8,6,5,9,7,2] 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 0x7f324409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): picked mon.noname-f con 0x7f3244136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): picked mon.noname-c con 0x7f3244137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244135dc0 0x7f3244136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): picked mon.noname-g con 0x7f3244135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324409abc0 con 0x7f3244136e40 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324409ba50 con 0x7f3244135dc0 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32440a53c0 con 0x7f3244137220 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f324a487640 10 monclient(hunting): authenticate will time out at 793.466753s 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:58298/0 (socket says 172.21.15.134:58298) 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 1 -- 172.21.15.134:0/969007699 learned_addr learned my addr 172.21.15.134:0/969007699 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 10 monclient(hunting): get_auth_request con 0x7f3244137220 auth_method 0 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.877+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 1 --2- 172.21.15.134:0/969007699 >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 0x7f324409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 10 monclient(hunting): get_auth_request con 0x7f3244136e40 auth_method 0 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3248a24640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244135dc0 0x7f3244136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3248a24640 10 monclient(hunting): get_auth_request con 0x7f3244135dc0 auth_method 0 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3248a24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3248a24640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3248a24640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 10 monclient(hunting): handle_auth_done global_id 4342 payload 274 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 -- 172.21.15.134:0/969007699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244135dc0 msgr2=0x7f3244136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244135dc0 0x7f3244136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 -- 172.21.15.134:0/969007699 >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 msgr2=0x7f324409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 --2- 172.21.15.134:0/969007699 >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 0x7f324409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 monclient: found mon.noname-c 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f323bfff640 1 -- 172.21.15.134:0/969007699 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244073250 con 0x7f3244137220 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.878+0000 7f3243fff640 1 --2- 172.21.15.134:0/969007699 >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 0x7f324409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.880+0000 7f323bfff640 1 -- 172.21.15.134:0/969007699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 msgr2=0x7f3244137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.880+0000 7f323bfff640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f322c0098a0 tx=0x7f322c02a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=4b4f843875a675ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 1 -- 172.21.15.134:0/969007699 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f322c02adf0 con 0x7f3244137220 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:56.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: dump: 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.894 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 5 monclient: authenticate success, global_id 4342 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 1 -- 172.21.15.134:0/969007699 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c0307b0 con 0x7f3244137220 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:56.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 4 set_mon_vals no callback set 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 1 -- 172.21.15.134:0/969007699 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f322c030d90 con 0x7f3244137220 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f32437fe640 10 monclient: dump: 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.896 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 10 monclient: shutdown 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 -- 172.21.15.134:0/969007699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 msgr2=0x7f3244137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f322c0098a0 tx=0x7f322c02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 -- 172.21.15.134:0/969007699 shutdown_connections 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244135dc0 0x7f3244136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 --2- 172.21.15.134:0/969007699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3244137220 0x7f3244137600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 --2- 172.21.15.134:0/969007699 >> v2:172.21.15.162:3300/0 conn(0x7f3244136e40 0x7f324409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 -- 172.21.15.134:0/969007699 >> 172.21.15.134:0/969007699 conn(0x7f32440a6110 msgr2=0x7f32440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 -- 172.21.15.134:0/969007699 shutdown_connections 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.881+0000 7f324a487640 1 -- 172.21.15.134:0/969007699 wait complete. 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.882+0000 7f324a487640 10 monclient: build_initial_monmap 2024-09-16T15:00:56.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.882+0000 7f324a487640 10 monclient: monmap: 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:56.899 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.882+0000 7f324a487640 1 Processor -- start 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.882+0000 7f324a487640 1 -- start start 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.882+0000 7f324a487640 10 monclient: init 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient: _add_conns ranks=[8,0,3,4,5,6,7,2,1] 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3244135dc0 0x7f32441cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): picked mon.noname-e con 0x7f3244135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:56.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 0x7f3244134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): picked mon.noname-a con 0x7f3244136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244137220 0x7f3244135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): picked mon.noname-g con 0x7f3244137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441cb760 con 0x7f3244136e40 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3244123cc0 con 0x7f3244137220 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f32440a8a30 con 0x7f3244135dc0 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f324a487640 10 monclient(hunting): authenticate will time out at 793.472229s 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3244135dc0 0x7f32441cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3244135dc0 0x7f32441cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53018/0 (socket says 172.21.15.134:53018) 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 0x7f3244134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 1 -- 172.21.15.134:0/1839872630 learned_addr learned my addr 172.21.15.134:0/1839872630 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): get_auth_request con 0x7f3244135dc0 auth_method 0 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): get_auth_request con 0x7f3244136e40 auth_method 0 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244137220 0x7f3244135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): get_auth_request con 0x7f3244137220 auth_method 0 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f3248a24640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient(hunting): handle_auth_done global_id 4419 payload 824 2024-09-16T15:00:56.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3244135dc0 msgr2=0x7f32441cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3244135dc0 0x7f32441cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244137220 msgr2=0x7f3244135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3244137220 0x7f3244135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 monclient: found mon.noname-a 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.883+0000 7f323bfff640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0094d0 con 0x7f3244136e40 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.885+0000 7f323bfff640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 msgr2=0x7f3244134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.885+0000 7f323bfff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 0x7f3244134f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f322c031000 tx=0x7f322c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f322c003610 con 0x7f3244136e40 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: dump: 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.905 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: _finish_auth 0 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: _check_auth_tickets 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c003790 con 0x7f3244136e40 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 5 monclient: authenticate success, global_id 4419 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 monclient: _renew_subs 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32441368e0 con 0x7f3244136e40 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f322c003d70 con 0x7f3244136e40 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f32417fa640 10 monclient: dump: 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.907 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f3242ffd640 4 set_mon_vals no callback set 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 monclient: _renew_subs 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:56.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3244125660 con 0x7f3244136e40 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 0x7f324402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.886+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3244071b20 con 0x7f32441221f0 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 0x7f324402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient: get_auth_request con 0x7f32441221f0 auth_method 0 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f322c039040 con 0x7f3244136e40 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 10 monclient(hunting): handle_auth_done global_id 4351 payload 824 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f32417fa640 1 --2- 172.21.15.134:0/1839872630 >> v2:172.21.15.162:6800/980547263 conn(0x7f32240388b0 0x7f322403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3243fff640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3244061ae0 con 0x7f32441221f0 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7f322c06a3a0 con 0x7f3244136e40 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3248a24640 1 --2- 172.21.15.134:0/1839872630 >> v2:172.21.15.162:6800/980547263 conn(0x7f32240388b0 0x7f322403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.887+0000 7f3248a24640 10 monclient: get_auth_request con 0x7f32240388b0 auth_method 0 2024-09-16T15:00:56.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f3243fff640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 msgr2=0x7f324402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f3243fff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 0x7f324402b260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3234006d40 tx=0x7f3234011c90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f323400fca0 con 0x7f32441221f0 2024-09-16T15:00:56.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f32417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f32417fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f32417fa640 10 monclient: dump: 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.914 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.889+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f323400fe00 con 0x7f32441221f0 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 msgr2=0x7f324402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 0x7f324402b260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3234006d40 tx=0x7f3234011c90 comp rx=0 tx=0).stop 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 msgr2=0x7f324402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f32417fa640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441221f0 0x7f324402b260 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3234006d40 tx=0x7f3234011c90 comp rx=0 tx=0).stop 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.890+0000 7f3248a24640 1 --2- 172.21.15.134:0/1839872630 >> v2:172.21.15.162:6800/980547263 conn(0x7f32240388b0 0x7f322403ad70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3244073780 tx=0x7f323c009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f324a487640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f324a487640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 0x7f324402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f324a487640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:56.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f324407a680 con 0x7f32441225d0 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 0x7f324402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient: get_auth_request con 0x7f32441225d0 auth_method 0 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 10 monclient(hunting): handle_auth_done global_id 4360 payload 824 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.896+0000 7f3243fff640 1 -- 172.21.15.134:0/1839872630 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f324409b490 con 0x7f32441225d0 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.898+0000 7f3243fff640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 msgr2=0x7f324402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.898+0000 7f3243fff640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 0x7f324402b7a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3234008010 tx=0x7f3234004730 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.898+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32340202e0 con 0x7f32441225d0 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.898+0000 7f32417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 10 monclient: dump: 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:56.917 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f3234020d80 con 0x7f32441225d0 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 msgr2=0x7f324402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 0x7f324402b7a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3234008010 tx=0x7f3234004730 comp rx=0 tx=0).stop 2024-09-16T15:00:56.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 msgr2=0x7f324402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f32417fa640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f32441225d0 0x7f324402b7a0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3234008010 tx=0x7f3234004730 comp rx=0 tx=0).stop 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:56.919 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 26, 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:56.920 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:56.921 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.922 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:56.923 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:56.924 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:56.925 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.926 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:56.928 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.929 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.931 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:56.932 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:56.935 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:56.937 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.938 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:56.939 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:56.940 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:56.941 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:56.942 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 >> v2:172.21.15.162:6800/980547263 conn(0x7f32240388b0 msgr2=0x7f322403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 --2- 172.21.15.134:0/1839872630 >> v2:172.21.15.162:6800/980547263 conn(0x7f32240388b0 0x7f322403ad70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3244073780 tx=0x7f323c009820 comp rx=0 tx=0).stop 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 10 monclient: shutdown 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 msgr2=0x7f3244134f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 --2- 172.21.15.134:0/1839872630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3244136e40 0x7f3244134f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f322c031000 tx=0x7f322c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f3248a24640 1 -- 172.21.15.134:0/1839872630 reap_dead start 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f3248a24640 1 -- 172.21.15.134:0/1839872630 reap_dead start 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 shutdown_connections 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 >> 172.21.15.134:0/1839872630 conn(0x7f32440a6110 msgr2=0x7f324409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:56.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 shutdown_connections 2024-09-16T15:00:56.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.899+0000 7f324a487640 1 -- 172.21.15.134:0/1839872630 wait complete. 2024-09-16T15:00:56.945 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:56.945 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:00:56.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:00:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.987+0000 7f31f1ad7640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-16T15:00:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.987+0000 7f31f1ad7640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-16T15:00:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.987+0000 7f31f1ad7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:00:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:56.987+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31ec134210 con 0x7f31ec135740 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.079+0000 7f474383a640 10 monclient: get_monmap_and_config 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.079+0000 7f474383a640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.079+0000 7f474383a640 10 monclient: monmap: 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:57.080825+0000 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:57.080825+0000 2024-09-16T15:00:57.084 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.085 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.080+0000 7f474383a640 1 Processor -- start 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.080+0000 7f474383a640 1 -- start start 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.080+0000 7f474383a640 10 monclient: init 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient: _add_conns ranks=[3,8,2,9,1,7,4,6,5,10,0] 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f473c136eb0 0x7f473c09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): picked mon.noname-g con 0x7f473c136eb0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): picked mon.noname-e con 0x7f473c137290 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): picked mon.noname-b con 0x7f473c133d10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0a5560 con 0x7f473c133d10 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c09b240 con 0x7f473c136eb0 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f473c09b520 con 0x7f473c137290 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f474383a640 10 monclient(hunting): authenticate will time out at 793.670148s 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53030/0 (socket says 172.21.15.134:53030) 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:55194/0 (socket says 172.21.15.134:55194) 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 1 -- 172.21.15.134:0/623334019 learned_addr learned my addr 172.21.15.134:0/623334019 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): get_auth_request con 0x7f473c137290 auth_method 0 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): get_auth_request con 0x7f473c133d10 auth_method 0 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f47415d6640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f473c136eb0 0x7f473c09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f47415d6640 10 monclient(hunting): get_auth_request con 0x7f473c136eb0 auth_method 0 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f47415d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f47415d6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f47415d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient(hunting): handle_auth_done global_id 4439 payload 274 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 -- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 msgr2=0x7f473c137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 -- 172.21.15.134:0/623334019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f473c136eb0 msgr2=0x7f473c09bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f473c136eb0 0x7f473c09bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 monclient: found mon.noname-b 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:57.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4741dd7640 1 -- 172.21.15.134:0/623334019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c073250 con 0x7f473c133d10 2024-09-16T15:00:57.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.081+0000 7f4740dd5640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f4741dd7640 1 -- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 msgr2=0x7f473c134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f4741dd7640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f472c009dc0 tx=0x7f472c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446912ff8b198920 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 1 -- 172.21.15.134:0/623334019 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c007bc0 con 0x7f473c133d10 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: dump: 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.098 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.099 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 1 -- 172.21.15.134:0/623334019 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c02d050 con 0x7f473c133d10 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f474383a640 5 monclient: authenticate success, global_id 4439 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f474383a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f474383a640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 1 -- 172.21.15.134:0/623334019 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c007d40 con 0x7f473c133d10 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f474383a640 4 set_mon_vals no callback set 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:57.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.084+0000 7f473bfff640 10 monclient: dump: 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.101 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 10 monclient: shutdown 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 -- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 msgr2=0x7f473c134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f472c009dc0 tx=0x7f472c02a480 comp rx=0 tx=0).stop 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 -- 172.21.15.134:0/623334019 shutdown_connections 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c133d10 0x7f473c134190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 --2- 172.21.15.134:0/623334019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f473c136eb0 0x7f473c09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 -- 172.21.15.134:0/623334019 >> 172.21.15.134:0/623334019 conn(0x7f473c0a99a0 msgr2=0x7f473c0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 -- 172.21.15.134:0/623334019 shutdown_connections 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 1 -- 172.21.15.134:0/623334019 wait complete. 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.085+0000 7f474383a640 10 monclient: monmap: 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:57.103 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 Processor -- start 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 -- start start 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient: init 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient: _add_conns ranks=[4,6,8,0,5,3,2,7,1] 2024-09-16T15:00:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 0x7f473c132ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): picked mon.noname-c con 0x7f473c133d10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 0x7f473c11fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): picked mon.noname-d con 0x7f473c136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c120110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): picked mon.noname-e con 0x7f473c137290 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1cc120 con 0x7f473c133d10 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0a7610 con 0x7f473c136eb0 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f473c121b30 con 0x7f473c137290 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f474383a640 10 monclient(hunting): authenticate will time out at 793.675730s 2024-09-16T15:00:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 0x7f473c132ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 0x7f473c11fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c120110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 0x7f473c11fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57446/0 (socket says 172.21.15.134:57446) 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 0x7f473c132ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:58326/0 (socket says 172.21.15.134:58326) 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 1 -- 172.21.15.134:0/1086658173 learned_addr learned my addr 172.21.15.134:0/1086658173 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): get_auth_request con 0x7f473c136eb0 auth_method 0 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): get_auth_request con 0x7f473c133d10 auth_method 0 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): get_auth_request con 0x7f473c137290 auth_method 0 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4740dd5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f47415d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.086+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 10 monclient(hunting): handle_auth_done global_id 4369 payload 824 2024-09-16T15:00:57.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 msgr2=0x7f473c120110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f473c137290 0x7f473c120110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 msgr2=0x7f473c11fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 0x7f473c11fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 monclient: found mon.noname-c 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f47415d6640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300094d0 con 0x7f473c133d10 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.087+0000 7f4740dd5640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c136eb0 0x7f473c11fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f47415d6640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 msgr2=0x7f473c132ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f47415d6640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 0x7f473c132ff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f473400d5f0 tx=0x7f473400dab0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4734012c70 con 0x7f473c133d10 2024-09-16T15:00:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: dump: 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4734012e30 con 0x7f473c133d10 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 5 monclient: authenticate success, global_id 4369 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 10 monclient: _renew_subs 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c009a70 con 0x7f473c133d10 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47340148a0 con 0x7f473c133d10 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f4739ffb640 10 monclient: dump: 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.111 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 10 monclient: _renew_subs 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f473b7fe640 4 set_mon_vals no callback set 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.089+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c1363e0 con 0x7f473c133d10 2024-09-16T15:00:57.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4734014430 con 0x7f473c133d10 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f474383a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4739ffb640 1 --2- 172.21.15.134:0/1086658173 >> v2:172.21.15.162:6800/980547263 conn(0x7f47280388b0 0x7f472803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f474383a640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 0x7f473c120a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f473c073880 con 0x7f473c120650 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7f473404a660 con 0x7f473c133d10 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 0x7f473c120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient: get_auth_request con 0x7f473c120650 auth_method 0 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4740dd5640 1 --2- 172.21.15.134:0/1086658173 >> v2:172.21.15.162:6800/980547263 conn(0x7f47280388b0 0x7f472803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.090+0000 7f4740dd5640 10 monclient: get_auth_request con 0x7f47280388b0 auth_method 0 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.091+0000 7f4741dd7640 10 monclient(hunting): handle_auth_done global_id 4398 payload 824 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.091+0000 7f4741dd7640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f473c122ea0 con 0x7f473c120650 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4741dd7640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 msgr2=0x7f473c120a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4741dd7640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 0x7f473c120a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4740dd5640 1 --2- 172.21.15.134:0/1086658173 >> v2:172.21.15.162:6800/980547263 conn(0x7f47280388b0 0x7f472803ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f473c04e4b0 tx=0x7f4730009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c007da0 con 0x7f473c120650 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 10 monclient: dump: 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.114 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f472c003fd0 con 0x7f473c120650 2024-09-16T15:00:57.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 msgr2=0x7f473c120a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 0x7f473c120a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 msgr2=0x7f473c120a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.093+0000 7f4739ffb640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c120650 0x7f473c120a30 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c004a40 comp rx=0 tx=0).stop 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f474383a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f474383a640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 0x7f473c134500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f474383a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f473c071b20 con 0x7f473c1340f0 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 0x7f473c134500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient: get_auth_request con 0x7f473c1340f0 auth_method 0 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.099+0000 7f4741dd7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.100+0000 7f4741dd7640 10 monclient(hunting): handle_auth_done global_id 4407 payload 824 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.100+0000 7f4741dd7640 1 -- 172.21.15.134:0/1086658173 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f473c061ae0 con 0x7f473c1340f0 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4741dd7640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 msgr2=0x7f473c134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4741dd7640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 0x7f473c134500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f472c02a9a0 tx=0x7f472c030000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472c04d070 con 0x7f473c1340f0 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 10 monclient: dump: 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f472c0482e0 con 0x7f473c1340f0 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 msgr2=0x7f473c134500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 0x7f473c134500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f472c02a9a0 tx=0x7f472c030000 comp rx=0 tx=0).stop 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 msgr2=0x7f473c134500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.102+0000 7f4739ffb640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f473c1340f0 0x7f473c134500 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f472c02a9a0 tx=0x7f472c030000 comp rx=0 tx=0).stop 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:57.121 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 26, 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:57.122 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:57.123 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:57.124 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.125 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.126 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:57.127 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.129 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.131 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.133 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.134 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.136 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.139 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.141 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.143 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:57.143 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 >> v2:172.21.15.162:6800/980547263 conn(0x7f47280388b0 msgr2=0x7f472803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 --2- 172.21.15.134:0/1086658173 >> v2:172.21.15.162:6800/980547263 conn(0x7f47280388b0 0x7f472803ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f473c04e4b0 tx=0x7f4730009240 comp rx=0 tx=0).stop 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 10 monclient: shutdown 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 msgr2=0x7f473c132ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 --2- 172.21.15.134:0/1086658173 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f473c133d10 0x7f473c132ff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f473400d5f0 tx=0x7f473400dab0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f4741dd7640 1 -- 172.21.15.134:0/1086658173 reap_dead start 2024-09-16T15:00:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 shutdown_connections 2024-09-16T15:00:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 >> 172.21.15.134:0/1086658173 conn(0x7f473c0a99a0 msgr2=0x7f473c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 shutdown_connections 2024-09-16T15:00:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.103+0000 7f474383a640 1 -- 172.21.15.134:0/1086658173 wait complete. 2024-09-16T15:00:57.145 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:57.145 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:00:57.145 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.280+0000 7f57cd853640 10 monclient: get_monmap_and_config 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.280+0000 7f57cd853640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.280+0000 7f57cd853640 10 monclient: monmap: 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:57.281769+0000 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:57.281769+0000 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.282 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:57.283 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.281+0000 7f57cd853640 1 Processor -- start 2024-09-16T15:00:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.281+0000 7f57cd853640 1 -- start start 2024-09-16T15:00:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.281+0000 7f57cd853640 10 monclient: init 2024-09-16T15:00:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient: _add_conns ranks=[3,7,1,0,9,10,8,2,5,6,4] 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8136e40 0x7f57c809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-g con 0x7f57c8136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 0x7f57c8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-i con 0x7f57c8137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f57c8135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c8073250 con 0x7f57c8135dc0 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c809abc0 con 0x7f57c8136e40 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57c809ba50 con 0x7f57c8137220 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57cd853640 10 monclient(hunting): authenticate will time out at 793.871158s 2024-09-16T15:00:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 0x7f57c8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c77fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f57c8135dc0 0x7f57c8136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:51570/0 (socket says 172.21.15.134:51570) 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c77fe640 1 -- 172.21.15.134:0/3320982586 learned_addr learned my addr 172.21.15.134:0/3320982586 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8136e40 0x7f57c809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): get_auth_request con 0x7f57c8137220 auth_method 0 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): get_auth_request con 0x7f57c8136e40 auth_method 0 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient(hunting): handle_auth_done global_id 4354 payload 274 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 -- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8136e40 msgr2=0x7f57c809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8136e40 0x7f57c809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 -- 172.21.15.134:0/3320982586 >> v1:172.21.15.162:6789/0 conn(0x7f57c8135dc0 legacy=0x7f57c8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 monclient: found mon.noname-i 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:00:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.282+0000 7f57be7fc640 1 -- 172.21.15.134:0/3320982586 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80a53c0 con 0x7f57c8137220 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.285+0000 7f57be7fc640 1 -- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 msgr2=0x7f57c8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.285+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 0x7f57c8137600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009800 tx=0x7f57b002a030 comp rx=0 tx=0).ready entity=mon.7 client_cookie=5c72f1810ad037b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 1 -- 172.21.15.134:0/3320982586 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b002ad50 con 0x7f57c8137220 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: dump: 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 1 -- 172.21.15.134:0/3320982586 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b00307b0 con 0x7f57c8137220 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 1 -- 172.21.15.134:0/3320982586 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b0030d90 con 0x7f57c8137220 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57c67fc640 10 monclient: dump: 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.299 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 5 monclient: authenticate success, global_id 4354 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 4 set_mon_vals no callback set 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 10 monclient: shutdown 2024-09-16T15:00:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 -- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 msgr2=0x7f57c8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 0x7f57c8137600 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009800 tx=0x7f57b002a030 comp rx=0 tx=0).stop 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 -- 172.21.15.134:0/3320982586 shutdown_connections 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f57c8137220 0x7f57c8137600 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 --2- 172.21.15.134:0/3320982586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8136e40 0x7f57c809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 -- 172.21.15.134:0/3320982586 >> 172.21.15.134:0/3320982586 conn(0x7f57c80a6110 msgr2=0x7f57c80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.286+0000 7f57cd853640 1 -- 172.21.15.134:0/3320982586 shutdown_connections 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.287+0000 7f57cd853640 1 -- 172.21.15.134:0/3320982586 wait complete. 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.287+0000 7f57cd853640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.287+0000 7f57cd853640 10 monclient: monmap: 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.287+0000 7f57cd853640 1 Processor -- start 2024-09-16T15:00:57.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 -- start start 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient: init 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient: _add_conns ranks=[3,5,2,6,4,8,0,1,7] 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8135dc0 0x7f57c81cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-g con 0x7f57c8135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f57c8136e40 0x7f57c8135350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-h con 0x7f57c8136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 0x7f57c8121eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): picked mon.noname-b con 0x7f57c8137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8125780 con 0x7f57c8137220 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81cbb50 con 0x7f57c8135dc0 2024-09-16T15:00:57.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81364e0 con 0x7f57c8136e40 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57cd853640 10 monclient(hunting): authenticate will time out at 793.877460s 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 0x7f57c8121eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 0x7f57c8121eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:55210/0 (socket says 172.21.15.134:55210) 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 learned_addr learned my addr 172.21.15.134:0/3980966675 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): get_auth_request con 0x7f57c8137220 auth_method 0 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8135dc0 0x7f57c81cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f57c8136e40 0x7f57c8135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): get_auth_request con 0x7f57c8135dc0 auth_method 0 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 10 monclient(hunting): get_auth_request con 0x7f57c8136e40 auth_method 0 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient(hunting): handle_auth_done global_id 4448 payload 824 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f57c8136e40 msgr2=0x7f57c8135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f57c8136e40 0x7f57c8135350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8135dc0 msgr2=0x7f57c81cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c8135dc0 0x7f57c81cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 monclient: found mon.noname-b 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009400 con 0x7f57c8137220 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.288+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f57c8136e40 0x7f57c8135350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 msgr2=0x7f57c8121eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c77fe640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 0x7f57c8121eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57c000caf0 tx=0x7f57c000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57c000acb0 con 0x7f57c8137220 2024-09-16T15:00:57.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: dump: 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.307 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c000ae50 con 0x7f57c8137220 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 5 monclient: authenticate success, global_id 4448 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 10 monclient: _renew_subs 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c80a8bc0 con 0x7f57c8137220 2024-09-16T15:00:57.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57c0012ce0 con 0x7f57c8137220 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c4ff9640 10 monclient: dump: 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 10 monclient: _renew_subs 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57c5ffb640 4 set_mon_vals no callback set 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.291+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81254a0 con 0x7f57c8137220 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f57c0012400 con 0x7f57c8137220 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57c4ff9640 1 --2- 172.21.15.134:0/3980966675 >> v2:172.21.15.162:6800/980547263 conn(0x7f57a8038880 0x7f57a803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57cd853640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7f57c004a400 con 0x7f57c8137220 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57cd853640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 0x7f57c802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8071b20 con 0x7f57c8135890 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 0x7f57c802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient: get_auth_request con 0x7f57c8135890 auth_method 0 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57c6ffd640 1 --2- 172.21.15.134:0/3980966675 >> v2:172.21.15.162:6800/980547263 conn(0x7f57a8038880 0x7f57a803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57c6ffd640 10 monclient: get_auth_request con 0x7f57a8038880 auth_method 0 2024-09-16T15:00:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 10 monclient(hunting): handle_auth_done global_id 4472 payload 824 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.292+0000 7f57be7fc640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57c8061ae0 con 0x7f57c8135890 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.294+0000 7f57be7fc640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 msgr2=0x7f57c802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.294+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 0x7f57c802b260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f57b0034000 tx=0x7f57b00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b0030d40 con 0x7f57c8135890 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 10 monclient: dump: 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c6ffd640 1 --2- 172.21.15.134:0/3980966675 >> v2:172.21.15.162:6800/980547263 conn(0x7f57a8038880 0x7f57a803ad40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009b40 tx=0x7f57b8009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f57b0003610 con 0x7f57c8135890 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 msgr2=0x7f57c802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 0x7f57c802b260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f57b0034000 tx=0x7f57b00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 msgr2=0x7f57c802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.295+0000 7f57c4ff9640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c8135890 0x7f57c802b260 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f57b0034000 tx=0x7f57b00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57cd853640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57cd853640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 0x7f57c802bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57cd853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8135c70 con 0x7f57c802b7a0 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 0x7f57c802bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient: get_auth_request con 0x7f57c802b7a0 auth_method 0 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 10 monclient(hunting): handle_auth_done global_id 4481 payload 824 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.301+0000 7f57be7fc640 1 -- 172.21.15.134:0/3980966675 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f57c807a680 con 0x7f57c802b7a0 2024-09-16T15:00:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.303+0000 7f57be7fc640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 msgr2=0x7f57c802bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.303+0000 7f57be7fc640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 0x7f57c802bb80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b0003ee0 con 0x7f57c802b7a0 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 10 monclient: dump: 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f57b00482c0 con 0x7f57c802b7a0 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 msgr2=0x7f57c802bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 0x7f57c802bb80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0008030 comp rx=0 tx=0).stop 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 msgr2=0x7f57c802bb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c4ff9640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57c802b7a0 0x7f57c802bb80 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000c00 tx=0x7f57b0008030 comp rx=0 tx=0).stop 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:57.319 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 26, 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.320 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.321 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.323 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.324 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.325 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.326 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:57.328 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:57.331 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.332 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.334 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.337 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.339 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:00:57.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:57.341 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 >> v2:172.21.15.162:6800/980547263 conn(0x7f57a8038880 msgr2=0x7f57a803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 --2- 172.21.15.134:0/3980966675 >> v2:172.21.15.162:6800/980547263 conn(0x7f57a8038880 0x7f57a803ad40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009b40 tx=0x7f57b8009190 comp rx=0 tx=0).stop 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 10 monclient: shutdown 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 msgr2=0x7f57c8121eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 --2- 172.21.15.134:0/3980966675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57c8137220 0x7f57c8121eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57c000caf0 tx=0x7f57c000a060 comp rx=0 tx=0).stop 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 reap_dead start 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57c77fe640 1 -- 172.21.15.134:0/3980966675 reap_dead start 2024-09-16T15:00:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 shutdown_connections 2024-09-16T15:00:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 >> 172.21.15.134:0/3980966675 conn(0x7f57c80a6110 msgr2=0x7f57c809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 shutdown_connections 2024-09-16T15:00:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.304+0000 7f57cd853640 1 -- 172.21.15.134:0/3980966675 wait complete. 2024-09-16T15:00:57.344 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:57.344 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:00:57.344 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:00:57.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.440+0000 7fb5fb99c640 10 monclient: get_monmap_and_config 2024-09-16T15:00:57.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.440+0000 7fb5fb99c640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.440+0000 7fb5fb99c640 10 monclient: monmap: 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:57.441595+0000 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:57.441595+0000 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.443 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 Processor -- start 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 -- start start 2024-09-16T15:00:57.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient: init 2024-09-16T15:00:57.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient: _add_conns ranks=[3,8,10,2,6,0,5,9,1,4,7] 2024-09-16T15:00:57.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5f4136e40 0x7fb5f409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-g con 0x7fb5f4136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-e con 0x7fb5f4137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb5f4135dc0 0x7fb5f4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-f con 0x7fb5f4135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f409abc0 con 0x7fb5f4135dc0 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f409ba50 con 0x7fb5f4136e40 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40a53c0 con 0x7fb5f4137220 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.441+0000 7fb5fb99c640 10 monclient(hunting): authenticate will time out at 794.030918s 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53084/0 (socket says 172.21.15.134:53084) 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 -- 172.21.15.134:0/2034733726 learned_addr learned my addr 172.21.15.134:0/2034733726 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): get_auth_request con 0x7fb5f4137220 auth_method 0 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 1 --2- 172.21.15.134:0/2034733726 >> v2:172.21.15.162:3300/0 conn(0x7fb5f4135dc0 0x7fb5f4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5f4136e40 0x7fb5f409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): get_auth_request con 0x7fb5f4135dc0 auth_method 0 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): get_auth_request con 0x7fb5f4136e40 auth_method 0 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient(hunting): handle_auth_done global_id 4490 payload 274 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 -- 172.21.15.134:0/2034733726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5f4136e40 msgr2=0x7fb5f409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5f4136e40 0x7fb5f409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 -- 172.21.15.134:0/2034733726 >> v2:172.21.15.162:3300/0 conn(0x7fb5f4135dc0 msgr2=0x7fb5f4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2034733726 >> v2:172.21.15.162:3300/0 conn(0x7fb5f4135dc0 0x7fb5f4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 monclient: found mon.noname-e 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.442+0000 7fb5f8f37640 1 -- 172.21.15.134:0/2034733726 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4073250 con 0x7fb5f4137220 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5f8f37640 1 -- 172.21.15.134:0/2034733726 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 msgr2=0x7fb5f4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40098a0 tx=0x7fb5e402a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a752f730527bbcb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 1 -- 172.21.15.134:0/2034733726 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5e402adf0 con 0x7fb5f4137220 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: dump: 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.459 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 1 -- 172.21.15.134:0/2034733726 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e40307b0 con 0x7fb5f4137220 2024-09-16T15:00:57.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 5 monclient: authenticate success, global_id 4490 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 1 -- 172.21.15.134:0/2034733726 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5e4030d90 con 0x7fb5f4137220 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 4 set_mon_vals no callback set 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5ebfff640 10 monclient: dump: 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.461 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.462 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 10 monclient: shutdown 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2034733726 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 msgr2=0x7fb5f4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40098a0 tx=0x7fb5e402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2034733726 shutdown_connections 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2034733726 >> v2:172.21.15.162:3300/0 conn(0x7fb5f4135dc0 0x7fb5f4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5f4137220 0x7fb5f4137600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2034733726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5f4136e40 0x7fb5f409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2034733726 >> 172.21.15.134:0/2034733726 conn(0x7fb5f40a6110 msgr2=0x7fb5f40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2034733726 shutdown_connections 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.445+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2034733726 wait complete. 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.446+0000 7fb5fb99c640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.446+0000 7fb5fb99c640 10 monclient: monmap: 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.464 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.446+0000 7fb5fb99c640 1 Processor -- start 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.446+0000 7fb5fb99c640 1 -- start start 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.446+0000 7fb5fb99c640 10 monclient: init 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient: _add_conns ranks=[7,1,0,2,5,6,8,3,4] 2024-09-16T15:00:57.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb5f4135dc0 0x7fb5f4134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-i con 0x7fb5f4135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4136e40 0x7fb5f4135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-f con 0x7fb5f4136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): picked mon.noname-a con 0x7fb5f4137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4136850 con 0x7fb5f4137220 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4136300 con 0x7fb5f4136e40 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4125840 con 0x7fb5f4135dc0 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5fb99c640 10 monclient(hunting): authenticate will time out at 794.036250s 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35946/0 (socket says 172.21.15.134:35946) 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 learned_addr learned my addr 172.21.15.134:0/2895912827 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): get_auth_request con 0x7fb5f4137220 auth_method 0 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb5f4135dc0 0x7fb5f4134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 10 monclient(hunting): get_auth_request con 0x7fb5f4135dc0 auth_method 0 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4136e40 0x7fb5f4135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f8f37640 10 monclient(hunting): get_auth_request con 0x7fb5f4136e40 auth_method 0 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f8f37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f8f37640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f8f37640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient(hunting): handle_auth_done global_id 4428 payload 824 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb5f4135dc0 msgr2=0x7fb5f4134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb5f4135dc0 0x7fb5f4134e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4136e40 msgr2=0x7fb5f4135390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4136e40 0x7fb5f4135390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 monclient: found mon.noname-a 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40094d0 con 0x7fb5f4137220 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.447+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb5f4135dc0 0x7fb5f4134e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.449+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 msgr2=0x7fb5f4121890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.449+0000 7fb5f9f39640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00efc0 tx=0x7fb5ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5ec0148f0 con 0x7fb5f4137220 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: dump: 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.469 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec00f040 con 0x7fb5f4137220 2024-09-16T15:00:57.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 5 monclient: authenticate success, global_id 4428 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5ec0133e0 con 0x7fb5f4137220 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 10 monclient: _renew_subs 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40a8980 con 0x7fb5f4137220 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5eb7fe640 4 set_mon_vals no callback set 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5e9ffb640 10 monclient: dump: 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.471 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 10 monclient: _renew_subs 2024-09-16T15:00:57.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.450+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f4125560 con 0x7fb5f4137220 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5fb99c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 0x7fb5f41221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f4071b20 con 0x7fb5f4121dd0 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb5ec014480 con 0x7fb5f4137220 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 0x7fb5f41221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5e9ffb640 1 --2- 172.21.15.134:0/2895912827 >> v2:172.21.15.162:6800/980547263 conn(0x7fb5d8038880 0x7fb5d803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient: get_auth_request con 0x7fb5f4121dd0 auth_method 0 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7fb5ec04ac20 con 0x7fb5f4137220 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2895912827 >> v2:172.21.15.162:6800/980547263 conn(0x7fb5d8038880 0x7fb5d803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f8f37640 10 monclient: get_auth_request con 0x7fb5d8038880 auth_method 0 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 10 monclient(hunting): handle_auth_done global_id 4465 payload 824 2024-09-16T15:00:57.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.451+0000 7fb5f9738640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5f4061ae0 con 0x7fb5f4121dd0 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.453+0000 7fb5f9738640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 msgr2=0x7fb5f41221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.453+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 0x7fb5f41221b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00115d0 tx=0x7fb5f0011a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.453+0000 7fb5f8f37640 1 --2- 172.21.15.134:0/2895912827 >> v2:172.21.15.162:6800/980547263 conn(0x7fb5d8038880 0x7fb5d803ad40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f406b550 tx=0x7fb5e402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5f000fec0 con 0x7fb5f4121dd0 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 10 monclient: dump: 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.477 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.478 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb5f000fa50 con 0x7fb5f4121dd0 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 msgr2=0x7fb5f41221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 0x7fb5f41221b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00115d0 tx=0x7fb5f0011a90 comp rx=0 tx=0).stop 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 msgr2=0x7fb5f41221b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.454+0000 7fb5e9ffb640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f4121dd0 0x7fb5f41221b0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00115d0 tx=0x7fb5f0011a90 comp rx=0 tx=0).stop 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5fb99c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 0x7fb5f40a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5fb99c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f407a680 con 0x7fb5f41226f0 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 0x7fb5f40a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient: get_auth_request con 0x7fb5f41226f0 auth_method 0 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.460+0000 7fb5f9738640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.461+0000 7fb5f9738640 10 monclient(hunting): handle_auth_done global_id 4474 payload 824 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.461+0000 7fb5f9738640 1 -- 172.21.15.134:0/2895912827 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5f409b490 con 0x7fb5f41226f0 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5f9738640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 msgr2=0x7fb5f40a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5f9738640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 0x7fb5f40a1a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0000c00 tx=0x7fb5f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb5f00068d0 con 0x7fb5f41226f0 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 10 monclient: dump: 2024-09-16T15:00:57.480 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.481 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9527 (secure 0 0 0) 0x7fb5f0006460 con 0x7fb5f41226f0 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 msgr2=0x7fb5f40a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 0x7fb5f40a1a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0000c00 tx=0x7fb5f00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 msgr2=0x7fb5f40a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.463+0000 7fb5e9ffb640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5f41226f0 0x7fb5f40a1a20 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0000c00 tx=0x7fb5f00046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:57.482 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:57.483 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 26, 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.484 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.485 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.487 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.489 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:57.490 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:57.492 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:57.493 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.495 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:57.497 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:57.500 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.501 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.503 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.504 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.505 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:57.506 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 >> v2:172.21.15.162:6800/980547263 conn(0x7fb5d8038880 msgr2=0x7fb5d803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2895912827 >> v2:172.21.15.162:6800/980547263 conn(0x7fb5d8038880 0x7fb5d803ad40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5f406b550 tx=0x7fb5e402e000 comp rx=0 tx=0).stop 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 10 monclient: shutdown 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 msgr2=0x7fb5f4121890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5f9f39640 1 -- 172.21.15.134:0/2895912827 reap_dead start 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00efc0 tx=0x7fb5ec00c490 comp rx=0 tx=0).stop 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 shutdown_connections 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 --2- 172.21.15.134:0/2895912827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f4137220 0x7fb5f4121890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 >> 172.21.15.134:0/2895912827 conn(0x7fb5f40a6110 msgr2=0x7fb5f406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 shutdown_connections 2024-09-16T15:00:57.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.464+0000 7fb5fb99c640 1 -- 172.21.15.134:0/2895912827 wait complete. 2024-09-16T15:00:57.509 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:57.509 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:00:57.509 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.605+0000 7f9720925640 10 monclient: get_monmap_and_config 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.605+0000 7f9720925640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.605+0000 7f9720925640 10 monclient: monmap: 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:57.606451+0000 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:57.606451+0000 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:57.607 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:57.608 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 Processor -- start 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 -- start start 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient: init 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient: _add_conns ranks=[2,5,9,4,7,3,0,10,1,6,8] 2024-09-16T15:00:57.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): picked mon.noname-b con 0x7f971c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): picked mon.noname-h con 0x7f971c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): picked mon.noname-a con 0x7f971c135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c09abc0 con 0x7f971c135dc0 2024-09-16T15:00:57.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c09ba50 con 0x7f971c136e40 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0a53c0 con 0x7f971c137220 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f9720925640 10 monclient(hunting): authenticate will time out at 794.195786s 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:55212/0 (socket says 172.21.15.134:55212) 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35958/0 (socket says 172.21.15.134:35958) 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 1 -- 172.21.15.134:0/761900330 learned_addr learned my addr 172.21.15.134:0/761900330 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 10 monclient(hunting): get_auth_request con 0x7f971c136e40 auth_method 0 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 10 monclient(hunting): get_auth_request con 0x7f971c135dc0 auth_method 0 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.606+0000 7f971ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f9719d9b640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient(hunting): handle_auth_done global_id 4457 payload 274 2024-09-16T15:00:57.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 -- 172.21.15.134:0/761900330 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 msgr2=0x7f971c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 -- 172.21.15.134:0/761900330 >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 msgr2=0x7f971c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 --2- 172.21.15.134:0/761900330 >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 monclient: found mon.noname-b 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971a59c640 1 -- 172.21.15.134:0/761900330 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c073250 con 0x7f971c136e40 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f9719d9b640 10 monclient: get_auth_request con 0x7f971c137220 auth_method 0 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f9719d9b640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.607+0000 7f971ad9d640 1 --2- 172.21.15.134:0/761900330 >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.609+0000 7f971a59c640 1 -- 172.21.15.134:0/761900330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 msgr2=0x7f971c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.609+0000 7f971a59c640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098a0 tx=0x7f970c02a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d0bac57fe9c503 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 1 -- 172.21.15.134:0/761900330 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f970c02adf0 con 0x7f971c136e40 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: dump: 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.617 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 1 -- 172.21.15.134:0/761900330 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c0307b0 con 0x7f971c136e40 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 5 monclient: authenticate success, global_id 4457 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 1 -- 172.21.15.134:0/761900330 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f970c030dd0 con 0x7f971c136e40 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 4 set_mon_vals no callback set 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f971959a640 10 monclient: dump: 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.619 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 10 monclient: shutdown 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 -- 172.21.15.134:0/761900330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 msgr2=0x7f971c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098a0 tx=0x7f970c02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 -- 172.21.15.134:0/761900330 shutdown_connections 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 --2- 172.21.15.134:0/761900330 >> v2:172.21.15.134:3300/0 conn(0x7f971c135dc0 0x7f971c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 --2- 172.21.15.134:0/761900330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f971c136e40 0x7f971c09c150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 -- 172.21.15.134:0/761900330 >> 172.21.15.134:0/761900330 conn(0x7f971c0a6110 msgr2=0x7f971c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 -- 172.21.15.134:0/761900330 shutdown_connections 2024-09-16T15:00:57.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.610+0000 7f9720925640 1 -- 172.21.15.134:0/761900330 wait complete. 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.611+0000 7f9720925640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.611+0000 7f9720925640 10 monclient: monmap: 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.626 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.611+0000 7f9720925640 1 Processor -- start 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.611+0000 7f9720925640 1 -- start start 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.611+0000 7f9720925640 10 monclient: init 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient: _add_conns ranks=[0,6,5,8,7,1,3,4,2] 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 0x7f971c1cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): picked mon.noname-a con 0x7f971c135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f971c136e40 0x7f971c1352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): picked mon.noname-d con 0x7f971c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c1357f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): picked mon.noname-h con 0x7f971c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1cba70 con 0x7f971c135dc0 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1363b0 con 0x7f971c137220 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1cc490 con 0x7f971c136e40 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9720925640 10 monclient(hunting): authenticate will time out at 794.201405s 2024-09-16T15:00:57.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 0x7f971c1cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 0x7f971c1cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35966/0 (socket says 172.21.15.134:35966) 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f971c136e40 0x7f971c1352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 -- 172.21.15.134:0/1233829910 learned_addr learned my addr 172.21.15.134:0/1233829910 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): get_auth_request con 0x7f971c135dc0 auth_method 0 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): get_auth_request con 0x7f971c136e40 auth_method 0 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient(hunting): handle_auth_done global_id 4446 payload 824 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f971c136e40 msgr2=0x7f971c1352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f971c136e40 0x7f971c1352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 msgr2=0x7f971c1357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f971c137220 0x7f971c1357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 monclient: found mon.noname-a 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.612+0000 7f971a59c640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714009a70 con 0x7f971c135dc0 2024-09-16T15:00:57.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.614+0000 7f971a59c640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 msgr2=0x7f971c1cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.614+0000 7f971a59c640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 0x7f971c1cb530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f970c0097c0 tx=0x7f970c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f970c004490 con 0x7f971c135dc0 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: dump: 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.631 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c003610 con 0x7f971c135dc0 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 5 monclient: authenticate success, global_id 4446 2024-09-16T15:00:57.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 10 monclient: _renew_subs 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c0094d0 con 0x7f971c135dc0 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f970c003bf0 con 0x7f971c135dc0 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f970affd640 10 monclient: dump: 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.633 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 10 monclient: _renew_subs 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c0a8a80 con 0x7f971c135dc0 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9718d99640 4 set_mon_vals no callback set 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.615+0000 7f9720925640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c071b20 con 0x7f971c1224b0 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f970c039070 con 0x7f971c135dc0 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient: get_auth_request con 0x7f971c1224b0 auth_method 0 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f970affd640 1 --2- 172.21.15.134:0/1233829910 >> v2:172.21.15.162:6800/980547263 conn(0x7f96fc0388b0 0x7f96fc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3858+0+0 (secure 0 0 0) 0x7f970c06a710 con 0x7f971c135dc0 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f971ad9d640 1 --2- 172.21.15.134:0/1233829910 >> v2:172.21.15.162:6800/980547263 conn(0x7f96fc0388b0 0x7f96fc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f971ad9d640 10 monclient: get_auth_request con 0x7f96fc0388b0 auth_method 0 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 msgr2=0x7f971c02b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 msgr2=0x7f971c02b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.616+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:57.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.618+0000 7f971ad9d640 1 --2- 172.21.15.134:0/1233829910 >> v2:172.21.15.162:6800/980547263 conn(0x7f96fc0388b0 0x7f96fc03ad70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f971c126190 tx=0x7f971400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.688+0000 7f31d3fff640 1 -- 172.21.15.134:0/3252783813 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]=0 enabled application 'rados' on pool 'unique_pool_0' v20) v1 ==== 197+0+0 (secure 0 0 0) 0x7f31e4012b30 con 0x7f31ec135740 2024-09-16T15:00:57.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.688+0000 7f31d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-16T15:00:57.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.688+0000 7f31d3fff640 10 monclient: _finish_command 2 = system:0 enabled application 'rados' on pool 'unique_pool_0' 2024-09-16T15:00:57.690 INFO:teuthology.orchestra.run.smithi134.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 msgr2=0x7f31d803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 0x7f31d803ad70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009a50 tx=0x7f31dc008040 comp rx=0 tx=0).stop 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 10 monclient: shutdown 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 msgr2=0x7f31ec09c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.691+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31e400efc0 tx=0x7f31e400c490 comp rx=0 tx=0).stop 2024-09-16T15:00:57.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 shutdown_connections 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> v2:172.21.15.162:6800/980547263 conn(0x7f31d80388b0 0x7f31d803ad70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f31ec135740 0x7f31ec09c940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f31ec134e20 0x7f31ec12f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 --2- 172.21.15.134:0/3252783813 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31ec06d7b0 0x7f31ec1c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 >> 172.21.15.134:0/3252783813 conn(0x7f31ec0a26b0 msgr2=0x7f31ec132b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 shutdown_connections 2024-09-16T15:00:57.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.692+0000 7f31f1ad7640 1 -- 172.21.15.134:0/3252783813 wait complete. 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient: get_auth_request con 0x7f971c1224b0 auth_method 0 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 10 monclient(hunting): handle_auth_done global_id 4494 payload 824 2024-09-16T15:00:57.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.817+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f971c061ae0 con 0x7f971c1224b0 2024-09-16T15:00:57.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 msgr2=0x7f971c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f971c123450 tx=0x7f9710012cb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9710010d20 con 0x7f971c1224b0 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f970affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f970affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.820+0000 7f970affd640 10 monclient: dump: 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.822 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f971000c040 con 0x7f971c1224b0 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 msgr2=0x7f971c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f971c123450 tx=0x7f9710012cb0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 msgr2=0x7f971c02b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.821+0000 7f970affd640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c1224b0 0x7f971c02b260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9720925640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9720925640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 0x7f971c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9720925640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c07a680 con 0x7f971c122890 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 0x7f971c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9719d9b640 10 monclient: get_auth_request con 0x7f971c122890 auth_method 0 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9719d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9719d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.827+0000 7f9719d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.828+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.828+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.828+0000 7f9719d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.828+0000 7f9719d9b640 10 monclient(hunting): handle_auth_done global_id 4503 payload 824 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.828+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f971c09b490 con 0x7f971c122890 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f9719d9b640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 msgr2=0x7f971c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f9719d9b640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 0x7f971c02b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f971c123450 tx=0x7f9710004730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f971001ccb0 con 0x7f971c122890 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f970affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f970affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.830+0000 7f970affd640 10 monclient: dump: 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.835 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f971001ce10 con 0x7f971c122890 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:57.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 msgr2=0x7f971c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 0x7f971c02b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f971c123450 tx=0x7f9710004730 comp rx=0 tx=0).stop 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 msgr2=0x7f971c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f970affd640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f971c122890 0x7f971c02b7a0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f971c123450 tx=0x7f9710004730 comp rx=0 tx=0).stop 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:57.837 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 27, 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:57.838 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.839 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:57.840 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:57.841 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:57.842 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.844 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:57.846 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.848 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:57.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:57.855 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.856 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:57.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:57.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:57.860 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 >> v2:172.21.15.162:6800/980547263 conn(0x7f96fc0388b0 msgr2=0x7f96fc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 --2- 172.21.15.134:0/1233829910 >> v2:172.21.15.162:6800/980547263 conn(0x7f96fc0388b0 0x7f96fc03ad70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f971c126190 tx=0x7f971400b040 comp rx=0 tx=0).stop 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 10 monclient: shutdown 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 msgr2=0x7f971c1cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 --2- 172.21.15.134:0/1233829910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f971c135dc0 0x7f971c1cb530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f970c0097c0 tx=0x7f970c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f971ad9d640 1 -- 172.21.15.134:0/1233829910 reap_dead start 2024-09-16T15:00:57.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f971ad9d640 1 -- 172.21.15.134:0/1233829910 reap_dead start 2024-09-16T15:00:57.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 shutdown_connections 2024-09-16T15:00:57.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 >> 172.21.15.134:0/1233829910 conn(0x7f971c0a6110 msgr2=0x7f971c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 shutdown_connections 2024-09-16T15:00:57.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.831+0000 7f9720925640 1 -- 172.21.15.134:0/1233829910 wait complete. 2024-09-16T15:00:57.863 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:57.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:00:57.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:00:57.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 10 monclient: get_monmap_and_config 2024-09-16T15:00:57.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 10 monclient: monmap: 2024-09-16T15:00:57.964 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.964 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:57.961105+0000 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:57.961105+0000 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.965 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 1 Processor -- start 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 1 -- start start 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.960+0000 7f2d9d2f0640 10 monclient: init 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient: _add_conns ranks=[0,5,8,10,9,2,4,7,1,3,6] 2024-09-16T15:00:57.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d980a1f60 addr v1:172.21.15.134:6789/0 2024-09-16T15:00:57.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 0x7f2d9813c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-h con 0x7f2d9806cbd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:57.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 0x7f2d9813eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-e con 0x7f2d980a2a50 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:57.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d98073250 con 0x7f2d980a1f60 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9806bfa0 con 0x7f2d9806cbd0 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9806c820 con 0x7f2d980a2a50 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d9d2f0640 10 monclient(hunting): authenticate will time out at 794.550364s 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 0x7f2d9813eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d96ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f2d980a1f60 0x7f2d980a2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:34526/0 (socket says 172.21.15.134:34526) 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d96ffd640 1 -- 172.21.15.134:0/2306476399 learned_addr learned my addr 172.21.15.134:0/2306476399 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): get_auth_request con 0x7f2d980a2a50 auth_method 0 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient(hunting): handle_auth_done global_id 4499 payload 274 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d967fc640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 0x7f2d9813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 -- 172.21.15.134:0/2306476399 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 msgr2=0x7f2d9813c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 0x7f2d9813c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 -- 172.21.15.134:0/2306476399 >> v1:172.21.15.134:6789/0 conn(0x7f2d980a1f60 legacy=0x7f2d980a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 monclient: found mon.noname-e 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:57.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d977fe640 1 -- 172.21.15.134:0/2306476399 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98072450 con 0x7f2d980a2a50 2024-09-16T15:00:57.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d967fc640 10 monclient: get_auth_request con 0x7f2d9806cbd0 auth_method 0 2024-09-16T15:00:57.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.961+0000 7f2d967fc640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 0x7f2d9813c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d977fe640 1 -- 172.21.15.134:0/2306476399 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 msgr2=0x7f2d9813eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d977fe640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 0x7f2d9813eab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9002a5f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2849ad1cd8e24c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 1 -- 172.21.15.134:0/2306476399 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d90007ca0 con 0x7f2d980a2a50 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: dump: 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 1 -- 172.21.15.134:0/2306476399 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9002d050 con 0x7f2d980a2a50 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 1 -- 172.21.15.134:0/2306476399 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d90007e20 con 0x7f2d980a2a50 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d9d2f0640 5 monclient: authenticate success, global_id 4499 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d9d2f0640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d9d2f0640 4 set_mon_vals no callback set 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.964+0000 7f2d95ffb640 10 monclient: dump: 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.982 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 10 monclient: shutdown 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/2306476399 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 msgr2=0x7f2d9813eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 0x7f2d9813eab0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/2306476399 shutdown_connections 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a2a50 0x7f2d9813eab0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/2306476399 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d9806cbd0 0x7f2d9813c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/2306476399 >> 172.21.15.134:0/2306476399 conn(0x7f2d980a7560 msgr2=0x7f2d980a7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/2306476399 shutdown_connections 2024-09-16T15:00:57.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/2306476399 wait complete. 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.965+0000 7f2d9d2f0640 10 monclient: build_initial_monmap 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient: monmap: 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:57.984 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 Processor -- start 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 -- start start 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient: init 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient: _add_conns ranks=[0,8,5,3,7,2,6,4,1] 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 0x7f2d981cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-a con 0x7f2d9806cbd0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a1f60 0x7f2d981374c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-e con 0x7f2d980a1f60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d980a2a50 0x7f2d98137a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): picked mon.noname-h con 0x7f2d980a2a50 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98125bf0 con 0x7f2d9806cbd0 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d981cbaf0 con 0x7f2d980a2a50 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98127eb0 con 0x7f2d980a1f60 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.966+0000 7f2d9d2f0640 10 monclient(hunting): authenticate will time out at 794.555991s 2024-09-16T15:00:57.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 0x7f2d981cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a1f60 0x7f2d981374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 0x7f2d981cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35976/0 (socket says 172.21.15.134:35976) 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 -- 172.21.15.134:0/3850130711 learned_addr learned my addr 172.21.15.134:0/3850130711 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request con 0x7f2d9806cbd0 auth_method 0 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): get_auth_request con 0x7f2d980a1f60 auth_method 0 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d977fe640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d980a2a50 0x7f2d98137a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d977fe640 10 monclient(hunting): get_auth_request con 0x7f2d980a2a50 auth_method 0 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d977fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient(hunting): handle_auth_done global_id 4455 payload 824 2024-09-16T15:00:57.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a1f60 msgr2=0x7f2d981374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a1f60 0x7f2d981374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d980a2a50 msgr2=0x7f2d98137a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d980a2a50 0x7f2d98137a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 monclient: found mon.noname-a 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d96ffd640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90009a70 con 0x7f2d9806cbd0 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.967+0000 7f2d967fc640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2d980a1f60 0x7f2d981374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d96ffd640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 msgr2=0x7f2d981cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d96ffd640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 0x7f2d981cb5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c490 tx=0x7f2d8800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d88005cf0 con 0x7f2d9806cbd0 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: dump: 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.991 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:57.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d88005e90 con 0x7f2d9806cbd0 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 5 monclient: authenticate success, global_id 4455 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 10 monclient: _renew_subs 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d98070b60 con 0x7f2d9806cbd0 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d880074e0 con 0x7f2d9806cbd0 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d7f7fe640 10 monclient: dump: 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:57.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 10 monclient: _renew_subs 2024-09-16T15:00:57.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d957fa640 4 set_mon_vals no callback set 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.969+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d980a4d20 con 0x7f2d9806cbd0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2d88007640 con 0x7f2d9806cbd0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d9d2f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 0x7f2d981243f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9806a3f0 con 0x7f2d98124010 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d7f7fe640 1 --2- 172.21.15.134:0/3850130711 >> v2:172.21.15.162:6800/980547263 conn(0x7f2d840387b0 0x7f2d8403ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3871+0+0 (secure 0 0 0) 0x7f2d88049b70 con 0x7f2d9806cbd0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.970+0000 7f2d967fc640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 0x7f2d981243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient: get_auth_request con 0x7f2d98124010 auth_method 0 2024-09-16T15:00:57.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:57.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:57.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:57.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d977fe640 1 --2- 172.21.15.134:0/3850130711 >> v2:172.21.15.162:6800/980547263 conn(0x7f2d840387b0 0x7f2d8403ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:57.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:57.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:57.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:57.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d977fe640 10 monclient: get_auth_request con 0x7f2d840387b0 auth_method 0 2024-09-16T15:00:57.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 10 monclient(hunting): handle_auth_done global_id 4361 payload 824 2024-09-16T15:00:57.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.971+0000 7f2d967fc640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d98061ae0 con 0x7f2d98124010 2024-09-16T15:00:57.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d967fc640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 msgr2=0x7f2d981243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d967fc640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 0x7f2d981243f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fc70 tx=0x7f2d8c00d590 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d977fe640 1 --2- 172.21.15.134:0/3850130711 >> v2:172.21.15.162:6800/980547263 conn(0x7f2d840387b0 0x7f2d8403ac70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9002f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d8c00fe80 con 0x7f2d98124010 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d7f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.973+0000 7f2d7f7fe640 10 monclient: dump: 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:57.999 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.000 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2d8c00c040 con 0x7f2d98124010 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 msgr2=0x7f2d981243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 0x7f2d981243f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fc70 tx=0x7f2d8c00d590 comp rx=0 tx=0).stop 2024-09-16T15:00:58.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 msgr2=0x7f2d981243f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:58.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.974+0000 7f2d7f7fe640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124010 0x7f2d981243f0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fc70 tx=0x7f2d8c00d590 comp rx=0 tx=0).stop 2024-09-16T15:00:58.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d9d2f0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 0x7f2d9809e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d9d2f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9807a680 con 0x7f2d98124930 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 0x7f2d9809e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient: get_auth_request con 0x7f2d98124930 auth_method 0 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 10 monclient(hunting): handle_auth_done global_id 4370 payload 824 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.980+0000 7f2d967fc640 1 -- 172.21.15.134:0/3850130711 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2d9806c4d0 con 0x7f2d98124930 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.982+0000 7f2d967fc640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 msgr2=0x7f2d9809e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d967fc640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 0x7f2d9809e960 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fc70 tx=0x7f2d8c004730 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d8c020040 con 0x7f2d98124930 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 10 monclient: dump: 2024-09-16T15:00:58.006 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.007 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f2d8c0263f0 con 0x7f2d98124930 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 msgr2=0x7f2d9809e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 0x7f2d9809e960 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00fc70 tx=0x7f2d8c004730 comp rx=0 tx=0).stop 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 msgr2=0x7f2d9809e960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:58.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.983+0000 7f2d7f7fe640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2d98124930 0x7f2d9809e960 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 27, 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:00:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:58.038 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:58.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 >> v2:172.21.15.162:6800/980547263 conn(0x7f2d840387b0 msgr2=0x7f2d8403ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/3850130711 >> v2:172.21.15.162:6800/980547263 conn(0x7f2d840387b0 0x7f2d8403ac70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009dc0 tx=0x7f2d9002f000 comp rx=0 tx=0).stop 2024-09-16T15:00:58.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 10 monclient: shutdown 2024-09-16T15:00:58.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 msgr2=0x7f2d981cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 --2- 172.21.15.134:0/3850130711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9806cbd0 0x7f2d981cb5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c490 tx=0x7f2d8800c950 comp rx=0 tx=0).stop 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d977fe640 1 -- 172.21.15.134:0/3850130711 reap_dead start 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d977fe640 1 -- 172.21.15.134:0/3850130711 reap_dead start 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 shutdown_connections 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 >> 172.21.15.134:0/3850130711 conn(0x7f2d980a7560 msgr2=0x7f2d9809d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 shutdown_connections 2024-09-16T15:00:58.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:57.984+0000 7f2d9d2f0640 1 -- 172.21.15.134:0/3850130711 wait complete. 2024-09-16T15:00:58.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:58.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:00:58.041 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:00:58.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 10 monclient: get_monmap_and_config 2024-09-16T15:00:58.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 10 monclient: build_initial_monmap 2024-09-16T15:00:58.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 10 monclient: monmap: 2024-09-16T15:00:58.138 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:58.138019+0000 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:58.138019+0000 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:58.139 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:00:58.140 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:00:58.140 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 1 Processor -- start 2024-09-16T15:00:58.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 1 -- start start 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.137+0000 7fb6e05ae640 10 monclient: init 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient: _add_conns ranks=[3,10,6,5,4,1,8,2,9,0,7] 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-g con 0x7fb6d8136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-f con 0x7fb6d8137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 0x7fb6d8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-d con 0x7fb6d8135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:00:58.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d809abc0 con 0x7fb6d8137220 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d809ba50 con 0x7fb6d8136e40 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80a53c0 con 0x7fb6d8135dc0 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6e05ae640 10 monclient(hunting): authenticate will time out at 794.727524s 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6deb4b640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 msgr2=0x7fb6d8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6deb4b640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 msgr2=0x7fb6d8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6deb4b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 0x7fb6d8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6deb4b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 0x7fb6d8136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:38296/0 (socket says 172.21.15.134:38296) 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:53938/0 (socket says 172.21.15.134:53938) 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 1 -- 172.21.15.134:0/3022562644 learned_addr learned my addr 172.21.15.134:0/3022562644 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): get_auth_request con 0x7fb6d8136e40 auth_method 0 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request con 0x7fb6d8137220 auth_method 0 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.138+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 10 monclient(hunting): handle_auth_done global_id 4512 payload 274 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 -- 172.21.15.134:0/3022562644 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 msgr2=0x7fb6d8136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 --2- 172.21.15.134:0/3022562644 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 0x7fb6d8136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 -- 172.21.15.134:0/3022562644 >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 msgr2=0x7fb6d8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 --2- 172.21.15.134:0/3022562644 >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 10 monclient: _finish_hunting 0 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 monclient: found mon.noname-g 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6de34a640 1 -- 172.21.15.134:0/3022562644 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8073250 con 0x7fb6d8136e40 2024-09-16T15:00:58.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.139+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/3022562644 >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:58.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.141+0000 7fb6de34a640 1 -- 172.21.15.134:0/3022562644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 msgr2=0x7fb6d809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:58.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.141+0000 7fb6de34a640 1 --2- 172.21.15.134:0/3022562644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098a0 tx=0x7fb6d402a0d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3850fdfbb9621be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 1 -- 172.21.15.134:0/3022562644 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d402adf0 con 0x7fb6d8136e40 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: dump: 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.154 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: _finish_auth 0 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: _check_auth_tickets 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 5 monclient: authenticate success, global_id 4512 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 1 -- 172.21.15.134:0/3022562644 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40307b0 con 0x7fb6d8136e40 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 10 monclient: get_monmap_and_config success 2024-09-16T15:00:58.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 1 -- 172.21.15.134:0/3022562644 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d4030d90 con 0x7fb6d8136e40 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 4 set_mon_vals no callback set 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6dd348640 10 monclient: dump: 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.156 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 10 monclient: shutdown 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 -- 172.21.15.134:0/3022562644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 msgr2=0x7fb6d809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/3022562644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098a0 tx=0x7fb6d402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 -- 172.21.15.134:0/3022562644 shutdown_connections 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/3022562644 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6d8135dc0 0x7fb6d8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/3022562644 >> v2:172.21.15.162:3300/0 conn(0x7fb6d8137220 0x7fb6d8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/3022562644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d8136e40 0x7fb6d809c150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 -- 172.21.15.134:0/3022562644 >> 172.21.15.134:0/3022562644 conn(0x7fb6d80a6110 msgr2=0x7fb6d80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 -- 172.21.15.134:0/3022562644 shutdown_connections 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.142+0000 7fb6e05ae640 1 -- 172.21.15.134:0/3022562644 wait complete. 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.143+0000 7fb6e05ae640 10 monclient: build_initial_monmap 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.143+0000 7fb6e05ae640 10 monclient: monmap: 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:00:58.158 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.143+0000 7fb6e05ae640 1 Processor -- start 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.143+0000 7fb6e05ae640 1 -- start start 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.143+0000 7fb6e05ae640 10 monclient: init 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient: _add_conns ranks=[8,1,0,4,5,3,2,6,7] 2024-09-16T15:00:58.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-e con 0x7fb6d8135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8136e40 0x7fb6d81cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-f con 0x7fb6d8136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d8137220 0x7fb6d81cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): picked mon.noname-a con 0x7fb6d8137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8122030 con 0x7fb6d8137220 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8120e30 con 0x7fb6d8136e40 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8136550 con 0x7fb6d8135dc0 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6e05ae640 10 monclient(hunting): authenticate will time out at 794.733121s 2024-09-16T15:00:58.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d8137220 0x7fb6d81cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53116/0 (socket says 172.21.15.134:53116) 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 -- 172.21.15.134:0/4167534731 learned_addr learned my addr 172.21.15.134:0/4167534731 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): get_auth_request con 0x7fb6d8135dc0 auth_method 0 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): get_auth_request con 0x7fb6d8137220 auth_method 0 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8136e40 0x7fb6d81cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request con 0x7fb6d8136e40 auth_method 0 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient(hunting): handle_auth_done global_id 4517 payload 824 2024-09-16T15:00:58.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8136e40 msgr2=0x7fb6d81cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8136e40 0x7fb6d81cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d8137220 msgr2=0x7fb6d81cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d8137220 0x7fb6d81cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient: _finish_hunting 0 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 monclient: found mon.noname-e 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6de34a640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009400 con 0x7fb6d8135dc0 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.144+0000 7fb6deb4b640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6d8137220 0x7fb6d81cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6de34a640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 msgr2=0x7fb6d81cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6de34a640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40090b0 tx=0x7fb6d40046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d4004510 con 0x7fb6d8135dc0 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: dump: 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.164 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: _finish_auth 0 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: _check_auth_tickets 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6e05ae640 5 monclient: authenticate success, global_id 4517 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4003610 con 0x7fb6d8135dc0 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6e05ae640 10 monclient: _renew_subs 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6e05ae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d40094d0 con 0x7fb6d8135dc0 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.146+0000 7fb6ceffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d4003bf0 con 0x7fb6d8135dc0 2024-09-16T15:00:58.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ceffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ceffd640 10 monclient: dump: 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.166 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6dcb47640 4 set_mon_vals no callback set 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 10 monclient: _renew_subs 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d8135880 con 0x7fb6d8135dc0 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 0x7fb6d802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8071b20 con 0x7fb6d8134fd0 2024-09-16T15:00:58.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6de34a640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40090b0 tx=0x7fb6d40046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fb6d406a970 osd_map(20..20 src has 1..20) v4 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb6d4039070 con 0x7fb6d8135dc0 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.147+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 0x7fb6d802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient: get_auth_request con 0x7fb6d8134fd0 auth_method 0 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ceffd640 1 --2- 172.21.15.134:0/4167534731 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6bc038880 0x7fb6bc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6deb4b640 1 --2- 172.21.15.134:0/4167534731 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6bc038880 0x7fb6bc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6deb4b640 10 monclient: get_auth_request con 0x7fb6bc038880 auth_method 0 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_done global_id 4372 payload 824 2024-09-16T15:00:58.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.148+0000 7fb6ddb49640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6d8061ae0 con 0x7fb6d8134fd0 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ddb49640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 msgr2=0x7fb6d802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 0x7fb6d802b260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80a3110 tx=0x7fb6c800f610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6c8011ea0 con 0x7fb6d8134fd0 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ceffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6ceffd640 10 monclient: dump: 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.169 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.150+0000 7fb6deb4b640 1 --2- 172.21.15.134:0/4167534731 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6bc038880 0x7fb6bc03ad40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005d00 tx=0x7fb6d00095d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb6c800fe40 con 0x7fb6d8134fd0 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 msgr2=0x7fb6d802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 0x7fb6d802b260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80a3110 tx=0x7fb6c800f610 comp rx=0 tx=0).stop 2024-09-16T15:00:58.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 msgr2=0x7fb6d802b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:58.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.151+0000 7fb6ceffd640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d8134fd0 0x7fb6d802b260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6e05ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:00:58.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 0x7fb6d802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:00:58.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6e05ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:00:58.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d807a680 con 0x7fb6d81353b0 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 0x7fb6d802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient: get_auth_request con 0x7fb6d81353b0 auth_method 0 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.157+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.158+0000 7fb6ddb49640 10 monclient(hunting): handle_auth_done global_id 4381 payload 824 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.158+0000 7fb6ddb49640 1 -- 172.21.15.134:0/4167534731 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb6d809b490 con 0x7fb6d81353b0 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ddb49640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 msgr2=0x7fb6d802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ddb49640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 0x7fb6d802b7a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8010010 tx=0x7fb6c80046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:00:58.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6c802c070 con 0x7fb6d81353b0 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ceffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ceffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.160+0000 7fb6ceffd640 10 monclient: dump: 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:00:58.174 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fb6c8024040 con 0x7fb6d81353b0 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 msgr2=0x7fb6d802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 0x7fb6d802b7a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8010010 tx=0x7fb6c80046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 msgr2=0x7fb6d802b7a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6d81353b0 0x7fb6d802b7a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.175 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 4, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:00:58.176 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 27, 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.177 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.178 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:00:58.179 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:00:58.180 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.182 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:00:58.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.187 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.189 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.190 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.191 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.192 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.193 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:00:58.194 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:58.196 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:00:58.197 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6bc038880 msgr2=0x7fb6bc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/4167534731 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6bc038880 0x7fb6bc03ad40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005d00 tx=0x7fb6d00095d0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 10 monclient: shutdown 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 msgr2=0x7fb6d81cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 --2- 172.21.15.134:0/4167534731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6d8135dc0 0x7fb6d81cb530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40090b0 tx=0x7fb6d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6deb4b640 1 -- 172.21.15.134:0/4167534731 reap_dead start 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6ceffd640 1 -- 172.21.15.134:0/4167534731 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3871+0+0 (unknown 0 0 0) 0x7fb6d406a970 con 0x7fb6d8135dc0 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6deb4b640 1 -- 172.21.15.134:0/4167534731 reap_dead start 2024-09-16T15:00:58.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 shutdown_connections 2024-09-16T15:00:58.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 >> 172.21.15.134:0/4167534731 conn(0x7fb6d80a6110 msgr2=0x7fb6d809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:00:58.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 shutdown_connections 2024-09-16T15:00:58.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:00:58.161+0000 7fb6e05ae640 1 -- 172.21.15.134:0/4167534731 wait complete. 2024-09-16T15:00:58.200 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:00:58.200 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'd', 'g', 'b'] 2024-09-16T15:00:58.200 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-16T15:00:58.200 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-16T15:00:58.201 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:00:58.201 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-16T15:00:58.201 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:00:58.216 INFO:tasks.ceph.mon.f:Stopped 2024-09-16T15:00:58.216 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-16T15:00:58.216 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-16T15:00:58.217 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-16T15:00:58.217 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:00:58.226 INFO:tasks.ceph.mon.d:Stopped 2024-09-16T15:00:58.226 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-16T15:00:58.226 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:00:58.226 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-16T15:00:58.227 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:00:58.270 INFO:tasks.ceph.mon.g:Stopped 2024-09-16T15:00:58.270 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-16T15:00:58.270 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:00:58.271 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:00:58.271 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:00:58.282 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:00:58.282 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-16T15:00:58.283 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-16T15:00:58.283 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-16T15:00:58.708 DEBUG:teuthology.orchestra.run.smithi162:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0 2024-09-16T15:00:58.742 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight read -> 100 2024-09-16T15:00:58.742 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight write -> 50 2024-09-16T15:00:58.742 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight delete -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight snap_create -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight snap_remove -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight rollback -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight copy_from -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:adding op weight write_excl -> 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout:Configuration: 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Number of operations: 4000 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Number of objects: 50 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Max in flight operations: 16 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Object size (in bytes): 4000000 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Write stride min: 400000 2024-09-16T15:00:58.743 INFO:tasks.rados.rados.0.smithi162.stdout: Write stride max: 800000 2024-09-16T15:01:13.284 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-16T15:01:13.285 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-16T15:01:13.285 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-16T15:01:13.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.392+0000 7fb35ef9c640 10 monclient: get_monmap_and_config 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.392+0000 7fb35ef9c640 10 monclient: build_initial_monmap 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.392+0000 7fb35ef9c640 10 monclient: monmap: 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:13.393657+0000 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:13.393657+0000 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:13.394 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:13.395 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.393+0000 7fb35ef9c640 1 Processor -- start 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.393+0000 7fb35ef9c640 1 -- start start 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.393+0000 7fb35ef9c640 10 monclient: init 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient: _add_conns ranks=[0,9,10,1,5,6,3,2,8,4,7] 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb358136030 addr v1:172.21.15.134:6789/0 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:13.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-a con 0x7fb35809bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-f con 0x7fb35813bf80 addr v2:172.21.15.162:3300/0 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb358073250 con 0x7fb358136030 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35809b600 con 0x7fb35809bfe0 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3580a5740 con 0x7fb35813bf80 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35ef9c640 10 monclient(hunting): authenticate will time out at 809.983138s 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35cd38640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb358136030 0x7fb358136410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:40990/0 (socket says 172.21.15.134:40990) 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 learned_addr learned my addr 172.21.15.134:0/63309286 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35d539640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 msgr2=0x7fb35813e370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:13.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb35809bfe0 auth_method 0 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.394+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:13.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.396+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.134:6789/0 conn(0x7fb358136030 legacy=0x7fb358136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 msgr2=0x7fb35813e370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb35809bfe0 auth_method 0 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:13.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:13.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:01:13.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:13.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:13.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.595+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:13.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 msgr2=0x7fb35813e370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:13.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:13.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:13.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb35809bfe0 auth_method 0 2024-09-16T15:01:13.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:13.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:13.996+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 msgr2=0x7fb35813e370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb35809bfe0 auth_method 0 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:14.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:14.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:01:14.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:14.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:14.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:14.797+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): tick 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 msgr2=0x7fb35813e370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.162:3300/0 conn(0x7fb35813bf80 0x7fb35813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 msgr2=0x7fb35809c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> v2:172.21.15.134:3300/0 conn(0x7fb35809bfe0 0x7fb35809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.134:6789/0 conn(0x7fb358136030 legacy=0x7fb358136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient: _add_conns ranks=[3,5,1,8,6,7,2,4] 2024-09-16T15:01:16.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-g con 0x7fb344003860 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-h con 0x7fb344006380 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb344008ec0 addr v1:172.21.15.162:6789/0 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3580a52c0 con 0x7fb344008ec0 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35809b600 con 0x7fb344003860 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580a5740 con 0x7fb344006380 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.162:6789/0 conn(0x7fb344008ec0 legacy=0x7fb34400b380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:01:16.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34e7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344006380 auth_method 0 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.394+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:16.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.395+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:16.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.595+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:16.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.595+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.162:6789/0 conn(0x7fb344008ec0 legacy=0x7fb34400b380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:01:16.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.595+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344006380 auth_method 0 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:16.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.996+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.996+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.162:6789/0 conn(0x7fb344008ec0 legacy=0x7fb34400b380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:01:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.996+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344006380 auth_method 0 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:16.997+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.797+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.162:6789/0 conn(0x7fb344008ec0 legacy=0x7fb34400b380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:01:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.797+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.797+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.798+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.798+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344006380 auth_method 0 2024-09-16T15:01:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.798+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.798+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.798+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.799+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.799+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.799+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:17.799+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:19.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): tick 2024-09-16T15:01:19.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:01:19.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:01:19.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 msgr2=0x7fb3440087f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb344006380 0x7fb3440087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344003860 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> v1:172.21.15.162:6789/0 conn(0x7fb344008ec0 legacy=0x7fb34400b380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient: _add_conns ranks=[2,6,7,8,4] 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 reap_dead start 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-b con 0x7fb34400cd50 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 reap_dead start 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:19.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-d con 0x7fb34400d710 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 msgr2=0x7fb34400d130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-i con 0x7fb34400bb50 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35809b600 con 0x7fb34400cd50 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580a5740 con 0x7fb34400d710 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb344008a00 con 0x7fb34400bb50 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34e7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:19.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 10 monclient(hunting): get_auth_request con 0x7fb34400bb50 auth_method 0 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:19.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.394+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 msgr2=0x7fb34400d130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 10 monclient(hunting): get_auth_request con 0x7fb34400bb50 auth_method 0 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:19.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:19.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.595+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:19.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 msgr2=0x7fb34400d130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:19.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:19.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:19.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 10 monclient(hunting): get_auth_request con 0x7fb34400bb50 auth_method 0 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:19.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:19.996+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:01:20.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 msgr2=0x7fb34400d130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:20.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:20.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:20.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:20.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:20.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35d539640 10 monclient(hunting): get_auth_request con 0x7fb34400bb50 auth_method 0 2024-09-16T15:01:20.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35d539640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:20.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35d539640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:20.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.797+0000 7fb35d539640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:20.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.798+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:20.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.798+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:20.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.798+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:20.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:20.798+0000 7fb35d539640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:01:22.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): tick 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 msgr2=0x7fb34400bf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb34400bb50 0x7fb34400bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 msgr2=0x7fb344005c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb34400d710 0x7fb344005c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 msgr2=0x7fb34400d130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb34400cd50 0x7fb34400d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient: _add_conns ranks=[8,4] 2024-09-16T15:01:22.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-e con 0x7fb34400c470 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): picked mon.noname-c con 0x7fb344010170 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb35809b600 con 0x7fb344010170 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580a5740 con 0x7fb34400c470 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34e7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 10 monclient(hunting): get_auth_request con 0x7fb34400c470 auth_method 0 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:22.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344010170 auth_method 0 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 msgr2=0x7fb34400b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 msgr2=0x7fb34400b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:22.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.395+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 10 monclient(hunting): get_auth_request con 0x7fb34400c470 auth_method 0 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344010170 auth_method 0 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:22.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 msgr2=0x7fb34400b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 msgr2=0x7fb34400b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:22.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.596+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:01:23.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb344010170 auth_method 0 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): get_auth_request con 0x7fb34400c470 auth_method 0 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient(hunting): handle_auth_done global_id 4378 payload 274 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 msgr2=0x7fb34400b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 1 monclient: found mon.noname-c 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34400dd70 con 0x7fb344010170 2024-09-16T15:01:23.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:22.998+0000 7fb35cd38640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb34400c470 0x7fb34400b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:01:23.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34ffff640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34ffff640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb348017a80 tx=0x7fb34802c4b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c36c8ea38a623eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 1 -- 172.21.15.134:0/63309286 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3480044a0 con 0x7fb344010170 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: dump: 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 1 -- 172.21.15.134:0/63309286 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34802f030 con 0x7fb344010170 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb35ef9c640 5 monclient: authenticate success, global_id 4378 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb35ef9c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb35ef9c640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb35ef9c640 4 set_mon_vals no callback set 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 1 -- 172.21.15.134:0/63309286 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb348003a80 con 0x7fb344010170 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.001+0000 7fb34f7fe640 10 monclient: dump: 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.008 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 10 monclient: shutdown 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 -- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 msgr2=0x7fb344012560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/63309286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb344010170 0x7fb344012560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb348017a80 tx=0x7fb34802c4b0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35d539640 1 -- 172.21.15.134:0/63309286 reap_dead start 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 -- 172.21.15.134:0/63309286 shutdown_connections 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 -- 172.21.15.134:0/63309286 >> 172.21.15.134:0/63309286 conn(0x7fb3580a9ea0 msgr2=0x7fb3580a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 -- 172.21.15.134:0/63309286 shutdown_connections 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.002+0000 7fb35ef9c640 1 -- 172.21.15.134:0/63309286 wait complete. 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.003+0000 7fb35ef9c640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.003+0000 7fb35ef9c640 10 monclient: monmap: 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.010 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.011 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 Processor -- start 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 -- start start 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient: init 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient: _add_conns ranks=[3,0,6,7,1,2,4,5,8] 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 0x7fb358133ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-g con 0x7fb344010170 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-a con 0x7fb3581340e0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 0x7fb358139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): picked mon.noname-d con 0x7fb358138e20 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580a4ad0 con 0x7fb3581340e0 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3581344e0 con 0x7fb344010170 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb35809faa0 con 0x7fb358138e20 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35ef9c640 10 monclient(hunting): authenticate will time out at 819.593960s 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.004+0000 7fb35d539640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 msgr2=0x7fb358139260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:23.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb35d539640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 0x7fb358139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb35cd38640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 msgr2=0x7fb358133ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb35cd38640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 0x7fb358133ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55346/0 (socket says 172.21.15.134:55346) 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 -- 172.21.15.134:0/1911076375 learned_addr learned my addr 172.21.15.134:0/1911076375 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): get_auth_request con 0x7fb3581340e0 auth_method 0 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient(hunting): handle_auth_done global_id 4491 payload 824 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 -- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 msgr2=0x7fb358139260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 0x7fb358139260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 -- 172.21.15.134:0/1911076375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 msgr2=0x7fb358133ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 0x7fb358133ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 monclient: found mon.noname-a 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.005+0000 7fb34ffff640 1 -- 172.21.15.134:0/1911076375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34800e040 con 0x7fb3581340e0 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.007+0000 7fb34ffff640 1 -- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 msgr2=0x7fb3581388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.007+0000 7fb34ffff640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb348000c00 tx=0x7fb3480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb348003ce0 con 0x7fb3581340e0 2024-09-16T15:01:23.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: dump: 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.020 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb348003e40 con 0x7fb3581340e0 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 5 monclient: authenticate success, global_id 4491 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 10 monclient: _renew_subs 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3580a4580 con 0x7fb3581340e0 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb348032d60 con 0x7fb3581340e0 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34d7fa640 10 monclient: dump: 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 10 monclient: _renew_subs 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb358120270 con 0x7fb3581340e0 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.008+0000 7fb34effd640 4 set_mon_vals no callback set 2024-09-16T15:01:23.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb348032480 con 0x7fb3581340e0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb35ef9c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb35ef9c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb35ef9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35809c400 con 0x7fb3581340e0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb34d7fa640 1 --2- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 0x7fb34403c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.009+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3871+0+0 (secure 0 0 0) 0x7fb348000e40 con 0x7fb3581340e0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.010+0000 7fb35cd38640 1 --2- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 0x7fb34403c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.010+0000 7fb35cd38640 10 monclient: get_auth_request con 0x7fb34403a280 auth_method 0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.012+0000 7fb35cd38640 1 --2- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 0x7fb34403c740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c00 tx=0x7fb354009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.014+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb348048720 con 0x7fb3581340e0 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.014+0000 7fb34d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.014+0000 7fb34d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb35ef9c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:01:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb35ef9c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:01:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb35ef9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb35811fd70 con 0x7fb3581340e0 2024-09-16T15:01:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb34d7fa640 1 -- 172.21.15.134:0/1911076375 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3171 (secure 0 0 0) 0x7fb348048360 con 0x7fb3581340e0 2024-09-16T15:01:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb34d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:01:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.242+0000 7fb34d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:23.244 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":12,"quorum":[0,4,5,7,8],"quorum_names":["a","c","h","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 msgr2=0x7fb34403c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 0x7fb34403c740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c00 tx=0x7fb354009040 comp rx=0 tx=0).stop 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 10 monclient: shutdown 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 msgr2=0x7fb3581388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb348000c00 tx=0x7fb3480046c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 shutdown_connections 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb358138e20 0x7fb358139260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3581340e0 0x7fb3581388e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> v2:172.21.15.162:6800/980547263 conn(0x7fb34403a280 0x7fb34403c740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 --2- 172.21.15.134:0/1911076375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb344010170 0x7fb358133ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 >> 172.21.15.134:0/1911076375 conn(0x7fb3580a9ea0 msgr2=0x7fb358134de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 shutdown_connections 2024-09-16T15:01:23.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.244+0000 7fb35ef9c640 1 -- 172.21.15.134:0/1911076375 wait complete. 2024-09-16T15:01:23.257 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-16T15:01:23.257 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-16T15:01:23.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:01:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.353+0000 7fca4999c640 10 monclient: get_monmap_and_config 2024-09-16T15:01:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.353+0000 7fca4999c640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.353+0000 7fca4999c640 10 monclient: monmap: 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:23.354746+0000 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:23.354746+0000 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 1 Processor -- start 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 1 -- start start 2024-09-16T15:01:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient: init 2024-09-16T15:01:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient: _add_conns ranks=[2,8,1,0,10,4,3,5,6,7,9] 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 0x7fca4409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-b con 0x7fca44136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-e con 0x7fca44137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.354+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fca44135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca44073250 con 0x7fca44135dc0 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4409abc0 con 0x7fca44136e40 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fca4409ba50 con 0x7fca44137220 2024-09-16T15:01:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca4999c640 10 monclient(hunting): authenticate will time out at 819.944023s 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca42ffd640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 msgr2=0x7fca4409c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 0x7fca4409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca437fe640 1 -- >> v1:172.21.15.162:6789/0 conn(0x7fca44135dc0 legacy=0x7fca44136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43702/0 (socket says 172.21.15.134:43702) 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 -- 172.21.15.134:0/1667400071 learned_addr learned my addr 172.21.15.134:0/1667400071 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): get_auth_request con 0x7fca44137220 auth_method 0 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient(hunting): handle_auth_done global_id 4544 payload 274 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 -- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 msgr2=0x7fca4409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 --2- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 0x7fca4409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 -- 172.21.15.134:0/1667400071 >> v1:172.21.15.162:6789/0 conn(0x7fca44135dc0 legacy=0x7fca44136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 monclient: found mon.noname-e 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.355+0000 7fca427fc640 1 -- 172.21.15.134:0/1667400071 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440a53c0 con 0x7fca44137220 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca427fc640 1 -- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 msgr2=0x7fca44137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca427fc640 1 --2- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca380098d0 tx=0x7fca3802a110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=cb8ba277ca837e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 1 -- 172.21.15.134:0/1667400071 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca380307b0 con 0x7fca44137220 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: dump: 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 5 monclient: authenticate success, global_id 4544 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 1 -- 172.21.15.134:0/1667400071 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38030930 con 0x7fca44137220 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 4 set_mon_vals no callback set 2024-09-16T15:01:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 1 -- 172.21.15.134:0/1667400071 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca380313a0 con 0x7fca44137220 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca41ffb640 10 monclient: dump: 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 10 monclient: shutdown 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 -- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 msgr2=0x7fca44137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 --2- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca380098d0 tx=0x7fca3802a110 comp rx=0 tx=0).stop 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 -- 172.21.15.134:0/1667400071 shutdown_connections 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 --2- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca44137220 0x7fca44137600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 --2- 172.21.15.134:0/1667400071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca44136e40 0x7fca4409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 -- 172.21.15.134:0/1667400071 >> 172.21.15.134:0/1667400071 conn(0x7fca440a6110 msgr2=0x7fca440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 -- 172.21.15.134:0/1667400071 shutdown_connections 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.358+0000 7fca4999c640 1 -- 172.21.15.134:0/1667400071 wait complete. 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.359+0000 7fca4999c640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.359+0000 7fca4999c640 10 monclient: monmap: 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.359+0000 7fca4999c640 1 Processor -- start 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 -- start start 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient: init 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient: _add_conns ranks=[5,3,1,6,4,8,7,2,0] 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 0x7fca440a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-h con 0x7fca44135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca44136e40 0x7fca44135130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-g con 0x7fca44136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44137220 0x7fca44135670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): picked mon.noname-f con 0x7fca44137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44120110 con 0x7fca44137220 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca441218b0 con 0x7fca44136e40 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca44122250 con 0x7fca44135dc0 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca4999c640 10 monclient(hunting): authenticate will time out at 819.949525s 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca427fc640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca44136e40 msgr2=0x7fca44135130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:01:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca437fe640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44137220 msgr2=0x7fca44135670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca427fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca44136e40 0x7fca44135130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca437fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44137220 0x7fca44135670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 0x7fca440a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 0x7fca440a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:52982/0 (socket says 172.21.15.134:52982) 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 1 -- 172.21.15.134:0/783328244 learned_addr learned my addr 172.21.15.134:0/783328244 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 10 monclient(hunting): get_auth_request con 0x7fca44135dc0 auth_method 0 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.360+0000 7fca42ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient(hunting): handle_auth_done global_id 4379 payload 824 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca44136e40 msgr2=0x7fca44135130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca44136e40 0x7fca44135130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44137220 msgr2=0x7fca44135670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca44137220 0x7fca44135670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 monclient: found mon.noname-h 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:01:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.361+0000 7fca42ffd640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380094d0 con 0x7fca44135dc0 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca42ffd640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 msgr2=0x7fca440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca42ffd640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 0x7fca440a2c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b550 tx=0x7fca3400ba10 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca34003dd0 con 0x7fca44135dc0 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:01:23.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: dump: 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.381 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 5 monclient: authenticate success, global_id 4379 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca340054e0 con 0x7fca44135dc0 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca34005690 con 0x7fca44135dc0 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 10 monclient: _renew_subs 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44122530 con 0x7fca44135dc0 2024-09-16T15:01:23.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca2bfff640 10 monclient: dump: 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.383 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca417fa640 4 set_mon_vals no callback set 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 10 monclient: _renew_subs 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.363+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441212f0 con 0x7fca44135dc0 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca4999c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca4999c640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 0x7fca4402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44071b20 con 0x7fca441cbe90 2024-09-16T15:01:23.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 0x7fca4402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient: get_auth_request con 0x7fca441cbe90 auth_method 0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fca34005070 con 0x7fca44135dc0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 10 monclient(hunting): handle_auth_done global_id 4500 payload 824 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca427fc640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca44061ae0 con 0x7fca441cbe90 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.364+0000 7fca2bfff640 1 --2- 172.21.15.134:0/783328244 >> v2:172.21.15.162:6800/980547263 conn(0x7fca30038880 0x7fca3003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.365+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3871+0+0 (secure 0 0 0) 0x7fca34049d20 con 0x7fca44135dc0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.365+0000 7fca437fe640 1 --2- 172.21.15.134:0/783328244 >> v2:172.21.15.162:6800/980547263 conn(0x7fca30038880 0x7fca3003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.365+0000 7fca437fe640 10 monclient: get_auth_request con 0x7fca30038880 auth_method 0 2024-09-16T15:01:23.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca427fc640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 msgr2=0x7fca4402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca427fc640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 0x7fca4402b260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca4406b550 tx=0x7fca380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca380313a0 con 0x7fca441cbe90 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 10 monclient: dump: 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.386 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fca38004250 con 0x7fca441cbe90 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 msgr2=0x7fca4402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 0x7fca4402b260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca4406b550 tx=0x7fca380046c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 msgr2=0x7fca4402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca2bfff640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cbe90 0x7fca4402b260 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca4406b550 tx=0x7fca380046c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.367+0000 7fca437fe640 1 --2- 172.21.15.134:0/783328244 >> v2:172.21.15.162:6800/980547263 conn(0x7fca30038880 0x7fca3003ad40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003cb0 tx=0x7fca3c009270 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca4999c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca4999c640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 0x7fca4402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca4999c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440a3240 con 0x7fca441cc270 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 0x7fca4402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient: get_auth_request con 0x7fca441cc270 auth_method 0 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 10 monclient(hunting): handle_auth_done global_id 4509 payload 824 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.373+0000 7fca427fc640 1 -- 172.21.15.134:0/783328244 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fca4407a680 con 0x7fca441cc270 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca427fc640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 msgr2=0x7fca4402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca427fc640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 0x7fca4402b7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca3803f000 tx=0x7fca38008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca38031c00 con 0x7fca441cc270 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 10 monclient: dump: 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.389 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9485 (secure 0 0 0) 0x7fca38031d60 con 0x7fca441cc270 2024-09-16T15:01:23.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 msgr2=0x7fca4402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 0x7fca4402b7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca3803f000 tx=0x7fca38008030 comp rx=0 tx=0).stop 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 msgr2=0x7fca4402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca2bfff640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca441cc270 0x7fca4402b7a0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca3803f000 tx=0x7fca38008030 comp rx=0 tx=0).stop 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:23.391 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 12, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:01:23.392 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:01:23.393 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:01:23.394 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.395 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:01:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.397 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.398 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.399 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:01:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.403 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:23.404 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.405 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.408 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.409 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:01:23.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:01:23.413 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.414 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.415 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:01:23.416 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 >> v2:172.21.15.162:6800/980547263 conn(0x7fca30038880 msgr2=0x7fca3003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 --2- 172.21.15.134:0/783328244 >> v2:172.21.15.162:6800/980547263 conn(0x7fca30038880 0x7fca3003ad40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca3c003cb0 tx=0x7fca3c009270 comp rx=0 tx=0).stop 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 10 monclient: shutdown 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 msgr2=0x7fca440a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 --2- 172.21.15.134:0/783328244 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fca44135dc0 0x7fca440a2c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca3400b550 tx=0x7fca3400ba10 comp rx=0 tx=0).stop 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca437fe640 1 -- 172.21.15.134:0/783328244 reap_dead start 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca437fe640 1 -- 172.21.15.134:0/783328244 reap_dead start 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 shutdown_connections 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.376+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 >> 172.21.15.134:0/783328244 conn(0x7fca440a6110 msgr2=0x7fca4409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.377+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 shutdown_connections 2024-09-16T15:01:23.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.377+0000 7fca4999c640 1 -- 172.21.15.134:0/783328244 wait complete. 2024-09-16T15:01:23.418 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:01:23.418 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-09-16T15:01:23.419 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:01:23.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.515+0000 7fdfef75e640 10 monclient: get_monmap_and_config 2024-09-16T15:01:23.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.515+0000 7fdfef75e640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.515+0000 7fdfef75e640 10 monclient: monmap: 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:23.516761+0000 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:23.516761+0000 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.518 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.516+0000 7fdfef75e640 1 Processor -- start 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.516+0000 7fdfef75e640 1 -- start start 2024-09-16T15:01:23.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.516+0000 7fdfef75e640 10 monclient: init 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient: _add_conns ranks=[7,2,8,9,5,1,6,4,0,10,3] 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfe8136e40 0x7fdfe809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-i con 0x7fdfe8136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-b con 0x7fdfe8137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-e con 0x7fdfe8135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe809abc0 con 0x7fdfe8137220 2024-09-16T15:01:23.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe809ba50 con 0x7fdfe8136e40 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80a53c0 con 0x7fdfe8135dc0 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfef75e640 10 monclient(hunting): authenticate will time out at 820.106231s 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfeccf9640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 msgr2=0x7fdfe8137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfeccf9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43714/0 (socket says 172.21.15.134:43714) 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 -- 172.21.15.134:0/3211452177 learned_addr learned my addr 172.21.15.134:0/3211452177 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): get_auth_request con 0x7fdfe8135dc0 auth_method 0 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfed4fa640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfe8136e40 0x7fdfe809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfed4fa640 10 monclient(hunting): get_auth_request con 0x7fdfe8136e40 auth_method 0 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfed4fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfed4fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfed4fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient(hunting): handle_auth_done global_id 4553 payload 274 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 -- 172.21.15.134:0/3211452177 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfe8136e40 msgr2=0x7fdfe809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfe8136e40 0x7fdfe809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 -- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 msgr2=0x7fdfe8137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 monclient: found mon.noname-e 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.517+0000 7fdfedcfb640 1 -- 172.21.15.134:0/3211452177 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8073250 con 0x7fdfe8135dc0 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfedcfb640 1 -- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 msgr2=0x7fdfe8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfedcfb640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e80 tx=0x7fdfe002a560 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c2d7fae457d48a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 1 -- 172.21.15.134:0/3211452177 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe0007a70 con 0x7fdfe8135dc0 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: dump: 2024-09-16T15:01:23.529 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.530 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfef75e640 5 monclient: authenticate success, global_id 4553 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfef75e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 1 -- 172.21.15.134:0/3211452177 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe002d050 con 0x7fdfe8135dc0 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 1 -- 172.21.15.134:0/3211452177 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe0007c30 con 0x7fdfe8135dc0 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfef75e640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfef75e640 4 set_mon_vals no callback set 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.520+0000 7fdfdffff640 10 monclient: dump: 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.532 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 10 monclient: shutdown 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 -- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 msgr2=0x7fdfe8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e80 tx=0x7fdfe002a560 comp rx=0 tx=0).stop 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 -- 172.21.15.134:0/3211452177 shutdown_connections 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdfe8135dc0 0x7fdfe8136220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 --2- 172.21.15.134:0/3211452177 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfe8136e40 0x7fdfe809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 -- 172.21.15.134:0/3211452177 >> 172.21.15.134:0/3211452177 conn(0x7fdfe80a6110 msgr2=0x7fdfe80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 -- 172.21.15.134:0/3211452177 shutdown_connections 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 1 -- 172.21.15.134:0/3211452177 wait complete. 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.521+0000 7fdfef75e640 10 monclient: monmap: 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.534 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 Processor -- start 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 -- start start 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient: init 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient: _add_conns ranks=[1,0,2,6,8,3,7,4,5] 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfe8135dc0 0x7fdfe8134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-f con 0x7fdfe8135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 0x7fdfe81353a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-a con 0x7fdfe8136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe81218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): picked mon.noname-b con 0x7fdfe8137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80a8a50 con 0x7fdfe8136e40 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8136330 con 0x7fdfe8135dc0 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8123480 con 0x7fdfe8137220 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfef75e640 10 monclient(hunting): authenticate will time out at 820.111740s 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfedcfb640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 msgr2=0x7fdfe81218a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfedcfb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe81218a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfed4fa640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfe8135dc0 msgr2=0x7fdfe8134e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 0x7fdfe81353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfed4fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfe8135dc0 0x7fdfe8134e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 0x7fdfe81353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55374/0 (socket says 172.21.15.134:55374) 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 1 -- 172.21.15.134:0/2771365750 learned_addr learned my addr 172.21.15.134:0/2771365750 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): get_auth_request con 0x7fdfe8136e40 auth_method 0 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.522+0000 7fdfeccf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 msgr2=0x7fdfe81218a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfe8137220 0x7fdfe81218a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfe8135dc0 msgr2=0x7fdfe8134e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfe8135dc0 0x7fdfe8134e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 monclient: found mon.noname-a 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.523+0000 7fdfeccf9640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009b30 con 0x7fdfe8136e40 2024-09-16T15:01:23.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfeccf9640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 msgr2=0x7fdfe81353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfeccf9640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 0x7fdfe81353a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800b550 tx=0x7fdfd800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfd8003dd0 con 0x7fdfe8136e40 2024-09-16T15:01:23.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: dump: 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.544 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd80054e0 con 0x7fdfe8136e40 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 5 monclient: authenticate success, global_id 4518 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 10 monclient: _renew_subs 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe8136880 con 0x7fdfe8136e40 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfd8005650 con 0x7fdfe8136e40 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfddffb640 10 monclient: dump: 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.546 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfdf7fe640 4 set_mon_vals no callback set 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 10 monclient: _renew_subs 2024-09-16T15:01:23.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.525+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe8125570 con 0x7fdfe8136e40 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdfd8005070 con 0x7fdfe8136e40 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfef75e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfef75e640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 0x7fdfe802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe8071b20 con 0x7fdfe81225a0 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 0x7fdfe802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient: get_auth_request con 0x7fdfe81225a0 auth_method 0 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfddffb640 1 --2- 172.21.15.134:0/2771365750 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfcc038880 0x7fdfcc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.526+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3871+0+0 (secure 0 0 0) 0x7fdfd8049d60 con 0x7fdfe8136e40 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.527+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_done global_id 4387 payload 824 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.527+0000 7fdfed4fa640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfe8061ae0 con 0x7fdfe81225a0 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.527+0000 7fdfedcfb640 1 --2- 172.21.15.134:0/2771365750 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfcc038880 0x7fdfcc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.527+0000 7fdfedcfb640 10 monclient: get_auth_request con 0x7fdfcc038880 auth_method 0 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfed4fa640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 msgr2=0x7fdfe802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfed4fa640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 0x7fdfe802b260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e500 tx=0x7fdfe400e9c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe40044a0 con 0x7fdfe81225a0 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 10 monclient: dump: 2024-09-16T15:01:23.549 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.550 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdfe4004030 con 0x7fdfe81225a0 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 msgr2=0x7fdfe802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 0x7fdfe802b260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e500 tx=0x7fdfe400e9c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 msgr2=0x7fdfe802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfddffb640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe81225a0 0x7fdfe802b260 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e500 tx=0x7fdfe400e9c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.529+0000 7fdfedcfb640 1 --2- 172.21.15.134:0/2771365750 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfcc038880 0x7fdfcc03ad40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e50 tx=0x7fdfe0009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfef75e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfef75e640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 0x7fdfe802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfef75e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe807a680 con 0x7fdfe8122980 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfed4fa640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 0x7fdfe802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfed4fa640 10 monclient: get_auth_request con 0x7fdfe8122980 auth_method 0 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfed4fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfed4fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.535+0000 7fdfed4fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.536+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.536+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.536+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.536+0000 7fdfed4fa640 10 monclient(hunting): handle_auth_done global_id 4396 payload 824 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.536+0000 7fdfed4fa640 1 -- 172.21.15.134:0/2771365750 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdfe809b490 con 0x7fdfe8122980 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfed4fa640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 msgr2=0x7fdfe802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfed4fa640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 0x7fdfe802b7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4017000 tx=0x7fdfe4004730 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe4023070 con 0x7fdfe8122980 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 10 monclient: dump: 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.553 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7fdfe4013b40 con 0x7fdfe8122980 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 msgr2=0x7fdfe802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 0x7fdfe802b7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4017000 tx=0x7fdfe4004730 comp rx=0 tx=0).stop 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 msgr2=0x7fdfe802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.538+0000 7fdfddffb640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdfe8122980 0x7fdfe802b7a0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4017000 tx=0x7fdfe4004730 comp rx=0 tx=0).stop 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 12, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:01:23.555 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.556 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.557 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.558 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.559 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:01:23.562 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:01:23.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.566 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:01:23.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:01:23.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.575 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.576 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:01:23.577 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:01:23.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfcc038880 msgr2=0x7fdfcc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 --2- 172.21.15.134:0/2771365750 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfcc038880 0x7fdfcc03ad40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009e50 tx=0x7fdfe0009000 comp rx=0 tx=0).stop 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 10 monclient: shutdown 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 msgr2=0x7fdfe81353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 --2- 172.21.15.134:0/2771365750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe8136e40 0x7fdfe81353a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800b550 tx=0x7fdfd800ba10 comp rx=0 tx=0).stop 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfedcfb640 1 -- 172.21.15.134:0/2771365750 reap_dead start 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfedcfb640 1 -- 172.21.15.134:0/2771365750 reap_dead start 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 shutdown_connections 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 >> 172.21.15.134:0/2771365750 conn(0x7fdfe80a6110 msgr2=0x7fdfe80a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 shutdown_connections 2024-09-16T15:01:23.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.539+0000 7fdfef75e640 1 -- 172.21.15.134:0/2771365750 wait complete. 2024-09-16T15:01:23.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:01:23.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-09-16T15:01:23.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:01:23.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.677+0000 7fbda93aa640 10 monclient: get_monmap_and_config 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.677+0000 7fbda93aa640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.677+0000 7fbda93aa640 10 monclient: monmap: 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:23.678439+0000 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:23.678439+0000 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:23.679 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:23.680 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 Processor -- start 2024-09-16T15:01:23.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 -- start start 2024-09-16T15:01:23.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient: init 2024-09-16T15:01:23.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient: _add_conns ranks=[6,2,8,10,3,1,0,5,9,4,7] 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-d con 0x7fbda4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-b con 0x7fbda4137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-e con 0x7fbda4135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda409abc0 con 0x7fbda4137220 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbda409ba50 con 0x7fbda4136e40 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a53c0 con 0x7fbda4135dc0 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda93aa640 10 monclient(hunting): authenticate will time out at 820.267689s 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda2ffd640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 msgr2=0x7fbda409c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda27fc640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 msgr2=0x7fbda4137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43720/0 (socket says 172.21.15.134:43720) 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 1 -- 172.21.15.134:0/1492973435 learned_addr learned my addr 172.21.15.134:0/1492973435 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): get_auth_request con 0x7fbda4135dc0 auth_method 0 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 10 monclient(hunting): handle_auth_done global_id 4562 payload 274 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.678+0000 7fbda37fe640 1 -- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 msgr2=0x7fbda409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 1 -- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 msgr2=0x7fbda4137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 1 monclient: found mon.noname-e 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.679+0000 7fbda37fe640 1 -- 172.21.15.134:0/1492973435 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4073250 con 0x7fbda4135dc0 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.681+0000 7fbda37fe640 1 -- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.681+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c02a570 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b1b5083b018dd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.681+0000 7fbda1ffb640 1 -- 172.21.15.134:0/1492973435 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c007a70 con 0x7fbda4135dc0 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.681+0000 7fbda1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: dump: 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 5 monclient: authenticate success, global_id 4562 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 1 -- 172.21.15.134:0/1492973435 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c02d050 con 0x7fbda4135dc0 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 4 set_mon_vals no callback set 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 1 -- 172.21.15.134:0/1492973435 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c007bf0 con 0x7fbda4135dc0 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda1ffb640 10 monclient: dump: 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 10 monclient: shutdown 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 -- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c02a570 comp rx=0 tx=0).stop 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 -- 172.21.15.134:0/1492973435 shutdown_connections 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1492973435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 -- 172.21.15.134:0/1492973435 >> 172.21.15.134:0/1492973435 conn(0x7fbda40a6110 msgr2=0x7fbda40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 -- 172.21.15.134:0/1492973435 shutdown_connections 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.682+0000 7fbda93aa640 1 -- 172.21.15.134:0/1492973435 wait complete. 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.683+0000 7fbda93aa640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.683+0000 7fbda93aa640 10 monclient: monmap: 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.692 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.693 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.683+0000 7fbda93aa640 1 Processor -- start 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.683+0000 7fbda93aa640 1 -- start start 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.683+0000 7fbda93aa640 10 monclient: init 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient: _add_conns ranks=[0,2,4,8,6,7,5,3,1] 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 0x7fbda4135070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-a con 0x7fbda4135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4136e40 0x7fbda4121850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-b con 0x7fbda4136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 0x7fbda4121d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): picked mon.noname-c con 0x7fbda4137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4123c10 con 0x7fbda4135dc0 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a8730 con 0x7fbda4136e40 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4136460 con 0x7fbda4137220 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda93aa640 10 monclient(hunting): authenticate will time out at 820.273208s 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda27fc640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4136e40 msgr2=0x7fbda4121850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda27fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4136e40 0x7fbda4121850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 0x7fbda4135070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 0x7fbda4121d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 0x7fbda4121d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41708/0 (socket says 172.21.15.134:41708) 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 0x7fbda4135070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55382/0 (socket says 172.21.15.134:55382) 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 1 -- 172.21.15.134:0/1773278777 learned_addr learned my addr 172.21.15.134:0/1773278777 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): get_auth_request con 0x7fbda4137220 auth_method 0 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): get_auth_request con 0x7fbda4135dc0 auth_method 0 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient(hunting): handle_auth_done global_id 4527 payload 824 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 msgr2=0x7fbda4121d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 0x7fbda4121d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4136e40 msgr2=0x7fbda4121850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbda4136e40 0x7fbda4121850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 monclient: found mon.noname-a 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda2ffd640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009b30 con 0x7fbda4135dc0 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.684+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbda4137220 0x7fbda4121d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.686+0000 7fbda2ffd640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4135070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.686+0000 7fbda2ffd640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 0x7fbda4135070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400b430 tx=0x7fbd9400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd94003dd0 con 0x7fbda4135dc0 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.704 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd940054e0 con 0x7fbda4135dc0 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 5 monclient: authenticate success, global_id 4527 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 monclient: _renew_subs 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda4125530 con 0x7fbda4135dc0 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd94005650 con 0x7fbda4135dc0 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.706 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.707 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 monclient: _renew_subs 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda41233e0 con 0x7fbda4135dc0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda17fa640 4 set_mon_vals no callback set 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 0x7fbda41226b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.687+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4071b20 con 0x7fbda41222d0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbd94005070 con 0x7fbda4135dc0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 0x7fbda41226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient: get_auth_request con 0x7fbda41222d0 auth_method 0 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbd8b7fe640 1 --2- 172.21.15.134:0/1773278777 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd900388b0 0x7fbd9003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 10 monclient(hunting): handle_auth_done global_id 4571 payload 824 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda27fc640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbda4061ae0 con 0x7fbda41222d0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3871+0+0 (secure 0 0 0) 0x7fbd94049110 con 0x7fbda4135dc0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1773278777 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd900388b0 0x7fbd9003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.688+0000 7fbda37fe640 10 monclient: get_auth_request con 0x7fbd900388b0 auth_method 0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbda27fc640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 msgr2=0x7fbda41226b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbda27fc640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 0x7fbda41226b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a3ca0 tx=0x7fbd9800b830 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd98003de0 con 0x7fbda41222d0 2024-09-16T15:01:23.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.710 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbd98005070 con 0x7fbda41222d0 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 msgr2=0x7fbda41226b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 0x7fbda41226b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a3ca0 tx=0x7fbd9800b830 comp rx=0 tx=0).stop 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 msgr2=0x7fbda41226b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbd8b7fe640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41222d0 0x7fbda41226b0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbda40a3ca0 tx=0x7fbd9800b830 comp rx=0 tx=0).stop 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.690+0000 7fbda37fe640 1 --2- 172.21.15.134:0/1773278777 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd900388b0 0x7fbd9003ad70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.696+0000 7fbda93aa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:01:23.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.696+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 0x7fbda40a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.696+0000 7fbda93aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.696+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda407a680 con 0x7fbda40a1a20 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 0x7fbda40a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient: get_auth_request con 0x7fbda40a1a20 auth_method 0 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 10 monclient(hunting): handle_auth_done global_id 4580 payload 824 2024-09-16T15:01:23.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.697+0000 7fbda27fc640 1 -- 172.21.15.134:0/1773278777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbda409b490 con 0x7fbda40a1a20 2024-09-16T15:01:23.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbda27fc640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 msgr2=0x7fbda40a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbda27fc640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 0x7fbda40a1e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd98013000 tx=0x7fbd98005940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9801e070 con 0x7fbda40a1a20 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 10 monclient: dump: 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.714 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7fbd98023470 con 0x7fbda40a1a20 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 msgr2=0x7fbda40a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 0x7fbda40a1e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd98013000 tx=0x7fbd98005940 comp rx=0 tx=0).stop 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 msgr2=0x7fbda40a1e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.699+0000 7fbd8b7fe640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda40a1a20 0x7fbda40a1e30 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbd98013000 tx=0x7fbd98005940 comp rx=0 tx=0).stop 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 12, 2024-09-16T15:01:23.716 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.717 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.718 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:01:23.719 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:01:23.720 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:01:23.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:01:23.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:01:23.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.734 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.737 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.738 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:01:23.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:01:23.740 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:01:23.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd900388b0 msgr2=0x7fbd9003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1773278777 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd900388b0 0x7fbd9003ad70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c009000 comp rx=0 tx=0).stop 2024-09-16T15:01:23.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 10 monclient: shutdown 2024-09-16T15:01:23.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4135070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 --2- 172.21.15.134:0/1773278777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4135dc0 0x7fbda4135070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400b430 tx=0x7fbd9400b8f0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda37fe640 1 -- 172.21.15.134:0/1773278777 reap_dead start 2024-09-16T15:01:23.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda37fe640 1 -- 172.21.15.134:0/1773278777 reap_dead start 2024-09-16T15:01:23.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 shutdown_connections 2024-09-16T15:01:23.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 >> 172.21.15.134:0/1773278777 conn(0x7fbda40a6110 msgr2=0x7fbda406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 shutdown_connections 2024-09-16T15:01:23.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.700+0000 7fbda93aa640 1 -- 172.21.15.134:0/1773278777 wait complete. 2024-09-16T15:01:23.744 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:01:23.744 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-09-16T15:01:23.744 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:01:23.745 INFO:tasks.rados.rados.0.smithi162.stdout:1: write initial oid 1 2024-09-16T15:01:23.745 INFO:tasks.rados.rados.0.smithi162.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2024-09-16T15:01:23.746 INFO:tasks.rados.rados.0.smithi162.stdout:1: writing smithi16234619-1 from 416807 to 892056 tid 1 2024-09-16T15:01:23.746 INFO:tasks.rados.rados.0.smithi162.stdout:1: writing smithi16234619-1 from 1542129 to 2016807 tid 2 2024-09-16T15:01:23.746 INFO:tasks.rados.rados.0.smithi162.stdout:2: write initial oid 2 2024-09-16T15:01:23.746 INFO:tasks.rados.rados.0.smithi162.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2024-09-16T15:01:23.746 INFO:tasks.rados.rados.0.smithi162.stdout:2: writing smithi16234619-2 from 416807 to 892056 tid 1 2024-09-16T15:01:23.749 INFO:tasks.rados.rados.0.smithi162.stdout:2: writing smithi16234619-2 from 1542129 to 2016807 tid 2 2024-09-16T15:01:23.750 INFO:tasks.rados.rados.0.smithi162.stdout:3: write initial oid 3 2024-09-16T15:01:23.750 INFO:tasks.rados.rados.0.smithi162.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2024-09-16T15:01:23.756 INFO:tasks.rados.rados.0.smithi162.stdout:3: writing smithi16234619-3 from 433614 to 984112 tid 1 2024-09-16T15:01:23.763 INFO:tasks.rados.rados.0.smithi162.stdout:3: writing smithi16234619-3 from 1600611 to 2033614 tid 2 2024-09-16T15:01:23.763 INFO:tasks.rados.rados.0.smithi162.stdout:4: write initial oid 4 2024-09-16T15:01:23.763 INFO:tasks.rados.rados.0.smithi162.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2024-09-16T15:01:23.769 INFO:tasks.rados.rados.0.smithi162.stdout:4: writing smithi16234619-4 from 450421 to 1076168 tid 1 2024-09-16T15:01:23.775 INFO:tasks.rados.rados.0.smithi162.stdout:4: writing smithi16234619-4 from 1659093 to 2050421 tid 2 2024-09-16T15:01:23.775 INFO:tasks.rados.rados.0.smithi162.stdout:5: write initial oid 5 2024-09-16T15:01:23.775 INFO:tasks.rados.rados.0.smithi162.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2024-09-16T15:01:23.782 INFO:tasks.rados.rados.0.smithi162.stdout:5: writing smithi16234619-5 from 467228 to 1168224 tid 1 2024-09-16T15:01:23.788 INFO:tasks.rados.rados.0.smithi162.stdout:5: writing smithi16234619-5 from 1717575 to 2067228 tid 2 2024-09-16T15:01:23.788 INFO:tasks.rados.rados.0.smithi162.stdout:6: write initial oid 6 2024-09-16T15:01:23.788 INFO:tasks.rados.rados.0.smithi162.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2024-09-16T15:01:23.796 INFO:tasks.rados.rados.0.smithi162.stdout:6: writing smithi16234619-6 from 484035 to 1260280 tid 1 2024-09-16T15:01:23.801 INFO:tasks.rados.rados.0.smithi162.stdout:6: writing smithi16234619-6 from 2059704 to 2084035 tid 2 2024-09-16T15:01:23.801 INFO:tasks.rados.rados.0.smithi162.stdout:7: write initial oid 7 2024-09-16T15:01:23.801 INFO:tasks.rados.rados.0.smithi162.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2024-09-16T15:01:23.806 INFO:tasks.rados.rados.0.smithi162.stdout:7: writing smithi16234619-7 from 500842 to 952336 tid 1 2024-09-16T15:01:23.812 INFO:tasks.rados.rados.0.smithi162.stdout:7: writing smithi16234619-7 from 1718186 to 2100842 tid 2 2024-09-16T15:01:23.812 INFO:tasks.rados.rados.0.smithi162.stdout:8: write initial oid 8 2024-09-16T15:01:23.812 INFO:tasks.rados.rados.0.smithi162.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2024-09-16T15:01:23.818 INFO:tasks.rados.rados.0.smithi162.stdout:8: writing smithi16234619-8 from 517649 to 1044392 tid 1 2024-09-16T15:01:23.824 INFO:tasks.rados.rados.0.smithi162.stdout:8: writing smithi16234619-8 from 1776668 to 2117649 tid 2 2024-09-16T15:01:23.824 INFO:tasks.rados.rados.0.smithi162.stdout:9: write initial oid 9 2024-09-16T15:01:23.824 INFO:tasks.rados.rados.0.smithi162.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2024-09-16T15:01:23.831 INFO:tasks.rados.rados.0.smithi162.stdout:9: writing smithi16234619-9 from 534456 to 1252801 tid 1 2024-09-16T15:01:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:9: writing smithi16234619-9 from 1951503 to 2134456 tid 2 2024-09-16T15:01:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:10: write initial oid 10 2024-09-16T15:01:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2024-09-16T15:01:23.843 INFO:tasks.rados.rados.0.smithi162.stdout:10: writing smithi16234619-10 from 551263 to 1344857 tid 1 2024-09-16T15:01:23.848 INFO:tasks.rados.rados.0.smithi162.stdout:10: writing smithi16234619-10 from 1893632 to 2151263 tid 2 2024-09-16T15:01:23.848 INFO:tasks.rados.rados.0.smithi162.stdout:11: write initial oid 11 2024-09-16T15:01:23.848 INFO:tasks.rados.rados.0.smithi162.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2024-09-16T15:01:23.853 INFO:tasks.rados.rados.0.smithi162.stdout:11: writing smithi16234619-11 from 568070 to 1036913 tid 1 2024-09-16T15:01:23.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.855+0000 7fc21022b640 10 monclient: get_monmap_and_config 2024-09-16T15:01:23.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.855+0000 7fc21022b640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.855+0000 7fc21022b640 10 monclient: monmap: 2024-09-16T15:01:23.856 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:23.856354+0000 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:23.856354+0000 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.857 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.858 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.858 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:23.858 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:23.858 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.855+0000 7fc21022b640 1 Processor -- start 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 -- start start 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient: init 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient: _add_conns ranks=[4,2,5,1,10,9,0,6,7,8,3] 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-c con 0x7fc208130ab0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 0x7fc208133af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-b con 0x7fc2081305f0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc2081344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-h con 0x7fc208134030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20809f600 con 0x7fc2081305f0 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2080673e0 con 0x7fc208130ab0 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc20809e740 con 0x7fc208134030 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc21022b640 10 monclient(hunting): authenticate will time out at 820.445725s 2024-09-16T15:01:23.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20d7c6640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 msgr2=0x7fc208133af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20d7c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 0x7fc208133af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41720/0 (socket says 172.21.15.134:41720) 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 1 -- 172.21.15.134:0/23701950 learned_addr learned my addr 172.21.15.134:0/23701950 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): get_auth_request con 0x7fc208130ab0 auth_method 0 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20e7c8640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc2081344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20e7c8640 10 monclient(hunting): get_auth_request con 0x7fc208134030 auth_method 0 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.856+0000 7fc20e7c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_done global_id 4414 payload 274 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 -- 172.21.15.134:0/23701950 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 msgr2=0x7fc2081344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc2081344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 -- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 msgr2=0x7fc208133af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 0x7fc208133af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 monclient: found mon.noname-c 2024-09-16T15:01:23.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.857+0000 7fc20dfc7640 1 -- 172.21.15.134:0/23701950 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208073250 con 0x7fc208130ab0 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.859+0000 7fc20dfc7640 1 -- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 msgr2=0x7fc208130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.859+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098a0 tx=0x7fc20402a130 comp rx=0 tx=0).ready entity=mon.4 client_cookie=98c4a808cad795d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 1 -- 172.21.15.134:0/23701950 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc20402ae90 con 0x7fc208130ab0 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: dump: 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.866 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 5 monclient: authenticate success, global_id 4414 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 1 -- 172.21.15.134:0/23701950 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2040307b0 con 0x7fc208130ab0 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 4 set_mon_vals no callback set 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 1 -- 172.21.15.134:0/23701950 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc204030dd0 con 0x7fc208130ab0 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc20cfc5640 10 monclient: dump: 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.868 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.869 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 10 monclient: shutdown 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 -- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 msgr2=0x7fc208130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098a0 tx=0x7fc20402a130 comp rx=0 tx=0).stop 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 -- 172.21.15.134:0/23701950 shutdown_connections 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc2081344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2081305f0 0x7fc208133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 --2- 172.21.15.134:0/23701950 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc208130ab0 0x7fc208130e90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 -- 172.21.15.134:0/23701950 >> 172.21.15.134:0/23701950 conn(0x7fc2080a6400 msgr2=0x7fc2080a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 -- 172.21.15.134:0/23701950 shutdown_connections 2024-09-16T15:01:23.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.860+0000 7fc21022b640 1 -- 172.21.15.134:0/23701950 wait complete. 2024-09-16T15:01:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:11: writing smithi16234619-11 from 1552114 to 2168070 tid 2 2024-09-16T15:01:23.872 INFO:tasks.rados.rados.0.smithi162.stdout:12: write initial oid 12 2024-09-16T15:01:23.872 INFO:tasks.rados.rados.0.smithi162.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2024-09-16T15:01:23.872 INFO:tasks.rados.rados.0.smithi162.stdout:12: writing smithi16234619-12 from 584877 to 1128969 tid 1 2024-09-16T15:01:23.877 INFO:tasks.rados.rados.0.smithi162.stdout:12: writing smithi16234619-12 from 1610596 to 2172599 tid 2 2024-09-16T15:01:23.877 INFO:tasks.rados.rados.0.smithi162.stdout:13: write initial oid 13 2024-09-16T15:01:23.877 INFO:tasks.rados.rados.0.smithi162.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2024-09-16T15:01:23.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.861+0000 7fc21022b640 10 monclient: build_initial_monmap 2024-09-16T15:01:23.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.861+0000 7fc21022b640 10 monclient: monmap: 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:23.879 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.861+0000 7fc21022b640 1 Processor -- start 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.861+0000 7fc21022b640 1 -- start start 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.861+0000 7fc21022b640 10 monclient: init 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient: _add_conns ranks=[8,2,5,4,0,6,1,7,3] 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-e con 0x7fc2081305f0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208130ab0 0x7fc2081c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-b con 0x7fc208130ab0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc20806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): picked mon.noname-h con 0x7fc208134030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20812b4c0 con 0x7fc208130ab0 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc20812caf0 con 0x7fc208134030 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc208133b10 con 0x7fc2081305f0 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc21022b640 10 monclient(hunting): authenticate will time out at 820.451455s 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20d7c6640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208130ab0 msgr2=0x7fc2081c8180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20d7c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208130ab0 0x7fc2081c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:23.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43746/0 (socket says 172.21.15.134:43746) 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 1 -- 172.21.15.134:0/1066885510 learned_addr learned my addr 172.21.15.134:0/1066885510 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): get_auth_request con 0x7fc2081305f0 auth_method 0 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc20806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): get_auth_request con 0x7fc208134030 auth_method 0 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20e7c8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 10 monclient(hunting): handle_auth_done global_id 4589 payload 824 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 msgr2=0x7fc20806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.862+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208134030 0x7fc20806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208130ab0 msgr2=0x7fc2081c8180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc208130ab0 0x7fc2081c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 10 monclient: _finish_hunting 0 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 1 monclient: found mon.noname-e 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:23.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.863+0000 7fc20dfc7640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040094d0 con 0x7fc2081305f0 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.865+0000 7fc20dfc7640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 msgr2=0x7fc2081c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.865+0000 7fc20dfc7640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2040097c0 tx=0x7fc2040046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.865+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc204030d40 con 0x7fc2081305f0 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: dump: 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.884 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: _finish_auth 0 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:01:23.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20402ae90 con 0x7fc2081305f0 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 5 monclient: authenticate success, global_id 4589 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 10 monclient: _renew_subs 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20812bb80 con 0x7fc2081305f0 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc20403d450 con 0x7fc2081305f0 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fffff640 4 set_mon_vals no callback set 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc1fe7fc640 10 monclient: dump: 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.886 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 10 monclient: _renew_subs 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.867+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20812c530 con 0x7fc2081305f0 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc21022b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc21022b640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 0x7fc20802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc208073880 con 0x7fc208070880 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 0x7fc20802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient: get_auth_request con 0x7fc208070880 auth_method 0 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc204039070 con 0x7fc2081305f0 2024-09-16T15:01:23.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_done global_id 4406 payload 824 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.868+0000 7fc20d7c6640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc208061ae0 con 0x7fc208070880 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.869+0000 7fc1fe7fc640 1 --2- 172.21.15.134:0/1066885510 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1ec038880 0x7fc1ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.869+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3871+0+0 (secure 0 0 0) 0x7fc20406a900 con 0x7fc2081305f0 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.869+0000 7fc20e7c8640 1 --2- 172.21.15.134:0/1066885510 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1ec038880 0x7fc1ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.869+0000 7fc20e7c8640 10 monclient: get_auth_request con 0x7fc1ec038880 auth_method 0 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc20d7c6640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 msgr2=0x7fc20802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc20d7c6640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 0x7fc20802b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2080713c0 tx=0x7fc1f800b810 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1f8003e60 con 0x7fc208070880 2024-09-16T15:01:23.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 10 monclient: dump: 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.890 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc1f8005070 con 0x7fc208070880 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc20e7c8640 1 --2- 172.21.15.134:0/1066885510 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1ec038880 0x7fc1ec03ad40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc20806df60 tx=0x7fc20000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 msgr2=0x7fc20802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 0x7fc20802b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2080713c0 tx=0x7fc1f800b810 comp rx=0 tx=0).stop 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 msgr2=0x7fc20802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.871+0000 7fc1fe7fc640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070880 0x7fc20802b260 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2080713c0 tx=0x7fc1f800b810 comp rx=0 tx=0).stop 2024-09-16T15:01:23.893 INFO:tasks.rados.rados.0.smithi162.stdout:13: writing smithi16234619-13 from 601684 to 1221025 tid 1 2024-09-16T15:01:23.893 INFO:tasks.rados.rados.0.smithi162.stdout:13: writing smithi16234619-13 from 1669078 to 2201684 tid 2 2024-09-16T15:01:23.893 INFO:tasks.rados.rados.0.smithi162.stdout:14: write initial oid 14 2024-09-16T15:01:23.893 INFO:tasks.rados.rados.0.smithi162.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.883+0000 7fc21022b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.883+0000 7fc21022b640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 0x7fc20802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.883+0000 7fc21022b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.883+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc20806b5d0 con 0x7fc208070c60 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 0x7fc20802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient: get_auth_request con 0x7fc208070c60 auth_method 0 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:23.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 10 monclient(hunting): handle_auth_done global_id 4415 payload 824 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.884+0000 7fc20d7c6640 1 -- 172.21.15.134:0/1066885510 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc20807a680 con 0x7fc208070c60 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.886+0000 7fc20d7c6640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 msgr2=0x7fc20802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.886+0000 7fc20d7c6640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 0x7fc20802b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8013000 tx=0x7fc1f80037f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1f801e070 con 0x7fc208070c60 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 10 monclient: dump: 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:23.896 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7fc1f8011400 con 0x7fc208070c60 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:01:23.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 msgr2=0x7fc20802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 0x7fc20802b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8013000 tx=0x7fc1f80037f0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 msgr2=0x7fc20802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc1fe7fc640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc208070c60 0x7fc20802b7a0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8013000 tx=0x7fc1f80037f0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 12, 2024-09-16T15:01:23.898 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.899 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.900 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:01:23.901 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:23.902 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:01:23.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.904 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:01:23.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:01:23.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.910 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:01:23.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:01:23.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.916 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.917 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.919 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.920 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:01:23.921 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:01:23.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1ec038880 msgr2=0x7fc1ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 1 --2- 172.21.15.134:0/1066885510 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1ec038880 0x7fc1ec03ad40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc20806df60 tx=0x7fc20000b040 comp rx=0 tx=0).stop 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 10 monclient: shutdown 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 msgr2=0x7fc2081c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc21022b640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2040097c0 tx=0x7fc2040046c0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.887+0000 7fc20e7c8640 1 -- 172.21.15.134:0/1066885510 reap_dead start 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.888+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 shutdown_connections 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.888+0000 7fc21022b640 1 --2- 172.21.15.134:0/1066885510 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2081305f0 0x7fc2081c7c40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.888+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 >> 172.21.15.134:0/1066885510 conn(0x7fc2080a6400 msgr2=0x7fc208099740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.888+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 shutdown_connections 2024-09-16T15:01:23.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:23.888+0000 7fc21022b640 1 -- 172.21.15.134:0/1066885510 wait complete. 2024-09-16T15:01:23.924 INFO:tasks.rados.rados.0.smithi162.stdout:14: writing smithi16234619-14 from 618491 to 1313081 tid 1 2024-09-16T15:01:23.924 INFO:tasks.rados.rados.0.smithi162.stdout:14: writing smithi16234619-14 from 2011207 to 2218491 tid 2 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:15: write initial oid 15 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:15: writing smithi16234619-15 from 635298 to 1405137 tid 1 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:15: writing smithi16234619-15 from 2069689 to 2235298 tid 2 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:16: write initial oid 16 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2024-09-16T15:01:23.925 INFO:tasks.rados.rados.0.smithi162.stdout:16: writing smithi16234619-16 from 652105 to 1097193 tid 1 2024-09-16T15:01:23.925 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:01:23.925 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon i 2024-09-16T15:01:23.926 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:16: writing smithi16234619-16 from 1728171 to 2181159 tid 2 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: finishing write tid 1 to smithi16234619-1 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: finishing write tid 2 to smithi16234619-1 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: finishing write tid 3 to smithi16234619-1 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: finishing write tid 4 to smithi16234619-1 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: oid 1 updating version 0 to 2 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: oid 1 updating version 2 to 3 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: oid 1 version 3 is already newer than 1 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-16T15:01:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: finishing write tid 1 to smithi16234619-2 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: finishing write tid 2 to smithi16234619-2 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: finishing write tid 3 to smithi16234619-2 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: finishing write tid 4 to smithi16234619-2 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: oid 2 updating version 0 to 1 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: oid 2 updating version 1 to 2 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: oid 2 updating version 2 to 3 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:3: finishing write tid 1 to smithi16234619-3 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:3: finishing write tid 2 to smithi16234619-3 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:3: finishing write tid 3 to smithi16234619-3 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:3: finishing write tid 4 to smithi16234619-3 2024-09-16T15:01:23.929 INFO:tasks.rados.rados.0.smithi162.stdout:3: oid 3 updating version 0 to 1 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:3: oid 3 updating version 1 to 2 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:3: oid 3 updating version 2 to 3 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: finishing write tid 1 to smithi16234619-4 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: finishing write tid 2 to smithi16234619-4 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: finishing write tid 3 to smithi16234619-4 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: finishing write tid 4 to smithi16234619-4 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: oid 4 updating version 0 to 5 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: oid 4 version 5 is already newer than 4 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: oid 4 updating version 5 to 6 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 6 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T15:01:23.930 INFO:tasks.rados.rados.0.smithi162.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: finishing write tid 1 to smithi16234619-5 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: finishing write tid 2 to smithi16234619-5 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:6: finishing write tid 1 to smithi16234619-6 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: finishing write tid 3 to smithi16234619-5 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:6: finishing write tid 2 to smithi16234619-6 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: finishing write tid 4 to smithi16234619-5 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: oid 5 updating version 0 to 3 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: oid 5 version 3 is already newer than 1 2024-09-16T15:01:23.931 INFO:tasks.rados.rados.0.smithi162.stdout:5: oid 5 version 3 is already newer than 2 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:6: finishing write tid 3 to smithi16234619-6 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:1: done (15 left) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:2: done (14 left) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:3: done (13 left) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:4: done (12 left) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:5: done (11 left) 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:17: write initial oid 17 2024-09-16T15:01:23.932 INFO:tasks.rados.rados.0.smithi162.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2024-09-16T15:01:23.934 INFO:tasks.rados.rados.0.smithi162.stdout:17: writing smithi16234619-17 from 668912 to 1305602 tid 1 2024-09-16T15:01:23.939 INFO:tasks.rados.rados.0.smithi162.stdout:17: writing smithi16234619-17 from 1903006 to 2268912 tid 2 2024-09-16T15:01:23.939 INFO:tasks.rados.rados.0.smithi162.stdout:18: write initial oid 18 2024-09-16T15:01:23.939 INFO:tasks.rados.rados.0.smithi162.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2024-09-16T15:01:23.947 INFO:tasks.rados.rados.0.smithi162.stdout:18: writing smithi16234619-18 from 685719 to 1397658 tid 1 2024-09-16T15:01:23.952 INFO:tasks.rados.rados.0.smithi162.stdout:18: writing smithi16234619-18 from 1845135 to 2285719 tid 2 2024-09-16T15:01:23.952 INFO:tasks.rados.rados.0.smithi162.stdout:19: write initial oid 19 2024-09-16T15:01:23.952 INFO:tasks.rados.rados.0.smithi162.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2024-09-16T15:01:23.960 INFO:tasks.rados.rados.0.smithi162.stdout:19: writing smithi16234619-19 from 702526 to 1489714 tid 1 2024-09-16T15:01:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:19: writing smithi16234619-19 from 1903617 to 2302526 tid 2 2024-09-16T15:01:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:20: write initial oid 20 2024-09-16T15:01:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2024-09-16T15:01:23.970 INFO:tasks.rados.rados.0.smithi162.stdout:20: writing smithi16234619-20 from 719333 to 1181770 tid 1 2024-09-16T15:01:23.977 INFO:tasks.rados.rados.0.smithi162.stdout:20: writing smithi16234619-20 from 1962099 to 2319333 tid 2 2024-09-16T15:01:23.977 INFO:tasks.rados.rados.0.smithi162.stdout:21: write initial oid 21 2024-09-16T15:01:23.977 INFO:tasks.rados.rados.0.smithi162.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2024-09-16T15:01:23.985 INFO:tasks.rados.rados.0.smithi162.stdout:21: writing smithi16234619-21 from 736140 to 1273826 tid 1 2024-09-16T15:01:23.991 INFO:tasks.rados.rados.0.smithi162.stdout:21: writing smithi16234619-21 from 2020581 to 2336140 tid 2 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: finishing write tid 4 to smithi16234619-6 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: oid 6 updating version 0 to 3 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: oid 6 version 3 is already newer than 2 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: oid 6 version 3 is already newer than 1 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:6: done (15 left) 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:22: write initial oid 22 2024-09-16T15:01:23.992 INFO:tasks.rados.rados.0.smithi162.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2024-09-16T15:01:23.999 INFO:tasks.rados.rados.0.smithi162.stdout:22: writing smithi16234619-22 from 752947 to 1365882 tid 1 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:22: writing smithi16234619-22 from 1962710 to 2352947 tid 2 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:7: finishing write tid 1 to smithi16234619-7 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:7: finishing write tid 2 to smithi16234619-7 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:7: finishing write tid 3 to smithi16234619-7 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:7: finishing write tid 4 to smithi16234619-7 2024-09-16T15:01:24.005 INFO:tasks.rados.rados.0.smithi162.stdout:7: oid 7 updating version 0 to 1 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:7: oid 7 updating version 1 to 2 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:7: oid 7 updating version 2 to 3 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: finishing write tid 1 to smithi16234619-8 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: finishing write tid 2 to smithi16234619-8 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: finishing write tid 3 to smithi16234619-8 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: finishing write tid 4 to smithi16234619-8 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: oid 8 updating version 0 to 5 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: oid 8 updating version 5 to 6 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: oid 8 version 6 is already newer than 4 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T15:01:24.006 INFO:tasks.rados.rados.0.smithi162.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: finishing write tid 1 to smithi16234619-9 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: finishing write tid 2 to smithi16234619-9 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: finishing write tid 3 to smithi16234619-9 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: finishing write tid 4 to smithi16234619-9 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: oid 9 updating version 0 to 8 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: oid 9 version 8 is already newer than 7 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: oid 9 updating version 8 to 9 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 9 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:10: finishing write tid 1 to smithi16234619-10 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:10: finishing write tid 2 to smithi16234619-10 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:10: finishing write tid 3 to smithi16234619-10 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:10: finishing write tid 4 to smithi16234619-10 2024-09-16T15:01:24.007 INFO:tasks.rados.rados.0.smithi162.stdout:10: oid 10 updating version 0 to 4 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:10: oid 10 updating version 4 to 6 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:10: oid 10 version 6 is already newer than 5 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 6 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:11: finishing write tid 1 to smithi16234619-11 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:7: done (15 left) 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:8: done (14 left) 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:9: done (13 left) 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:10: done (12 left) 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:23: write initial oid 23 2024-09-16T15:01:24.008 INFO:tasks.rados.rados.0.smithi162.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2024-09-16T15:01:24.012 INFO:tasks.rados.rados.0.smithi162.stdout:23: writing smithi16234619-23 from 769754 to 1457938 tid 1 2024-09-16T15:01:24.017 INFO:tasks.rados.rados.0.smithi162.stdout:23: writing smithi16234619-23 from 2021192 to 2369754 tid 2 2024-09-16T15:01:24.017 INFO:tasks.rados.rados.0.smithi162.stdout:24: write initial oid 24 2024-09-16T15:01:24.017 INFO:tasks.rados.rados.0.smithi162.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2024-09-16T15:01:24.024 INFO:tasks.rados.rados.0.smithi162.stdout:24: writing smithi16234619-24 from 786561 to 1266347 tid 1 2024-09-16T15:01:24.029 INFO:tasks.rados.rados.0.smithi162.stdout:24: writing smithi16234619-24 from 1796027 to 2263691 tid 2 2024-09-16T15:01:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:25: write initial oid 25 2024-09-16T15:01:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2024-09-16T15:01:24.035 INFO:tasks.rados.rados.0.smithi162.stdout:25: writing smithi16234619-25 from 403368 to 958403 tid 1 2024-09-16T15:01:24.041 INFO:tasks.rados.rados.0.smithi162.stdout:25: writing smithi16234619-25 from 1454509 to 2182184 tid 2 2024-09-16T15:01:24.041 INFO:tasks.rados.rados.0.smithi162.stdout:26: write initial oid 26 2024-09-16T15:01:24.042 INFO:tasks.rados.rados.0.smithi162.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 10 monclient: get_monmap_and_config 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 10 monclient: build_initial_monmap 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 10 monclient: monmap: 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.046 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:01:24.046025+0000 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:01:24.046025+0000 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:01:24.047 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:01:24.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.048 INFO:tasks.rados.rados.0.smithi162.stdout:26: writing smithi16234619-26 from 420175 to 1050459 tid 1 2024-09-16T15:01:24.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 1 Processor -- start 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 1 -- start start 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.045+0000 7f228a6e9640 10 monclient: init 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient: _add_conns ranks=[6,2,9,3,10,1,7,5,4,0,8] 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 0x7f2284071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-d con 0x7f2284070c70 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 0x7f228406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-b con 0x7f228409fb80 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2284071590 0x7f228406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-a con 0x7f2284071590 addr v2:172.21.15.134:3300/0 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228406f4f0 con 0x7f2284071590 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228406e360 con 0x7f228409fb80 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f228406eea0 con 0x7f2284070c70 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f228a6e9640 10 monclient(hunting): authenticate will time out at 820.635486s 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f22837fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 msgr2=0x7f228406ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2283fff640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 msgr2=0x7f2284071050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f22837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 0x7f228406ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2283fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 0x7f2284071050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2284071590 0x7f228406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2284071590 0x7f228406d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:55392/0 (socket says 172.21.15.134:55392) 2024-09-16T15:01:24.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 1 -- 172.21.15.134:0/3122000384 learned_addr learned my addr 172.21.15.134:0/3122000384 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): get_auth_request con 0x7f2284071590 auth_method 0 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.046+0000 7f2288c86640 10 monclient(hunting): handle_auth_done global_id 4536 payload 274 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 -- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 msgr2=0x7f2284071050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 0x7f2284071050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 -- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 msgr2=0x7f228406ce50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 0x7f228406ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 10 monclient: _finish_hunting 0 2024-09-16T15:01:24.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 monclient: found mon.noname-a 2024-09-16T15:01:24.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:24.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:01:24.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.047+0000 7f2288c86640 1 -- 172.21.15.134:0/3122000384 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284073250 con 0x7f2284071590 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:26: writing smithi16234619-26 from 1796638 to 2267971 tid 2 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: finishing write tid 2 to smithi16234619-11 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: finishing write tid 3 to smithi16234619-11 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: finishing write tid 4 to smithi16234619-11 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: oid 11 updating version 0 to 1 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: oid 11 updating version 1 to 2 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: oid 11 updating version 2 to 3 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:12: finishing write tid 1 to smithi16234619-12 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:12: finishing write tid 2 to smithi16234619-12 2024-09-16T15:01:24.058 INFO:tasks.rados.rados.0.smithi162.stdout:12: finishing write tid 3 to smithi16234619-12 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:12: finishing write tid 4 to smithi16234619-12 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:12: oid 12 updating version 0 to 1 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:12: oid 12 updating version 1 to 2 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:12: oid 12 updating version 2 to 3 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: finishing write tid 1 to smithi16234619-13 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: finishing write tid 2 to smithi16234619-13 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: finishing write tid 3 to smithi16234619-13 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: finishing write tid 4 to smithi16234619-13 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: oid 13 updating version 0 to 5 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: oid 13 version 5 is already newer than 4 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:13: oid 13 updating version 5 to 6 2024-09-16T15:01:24.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 6 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: finishing write tid 1 to smithi16234619-14 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: finishing write tid 2 to smithi16234619-14 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: finishing write tid 3 to smithi16234619-14 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: finishing write tid 4 to smithi16234619-14 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: oid 14 updating version 0 to 1 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: oid 14 updating version 1 to 3 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: oid 14 version 3 is already newer than 2 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 3 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:11: done (15 left) 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:12: done (14 left) 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:13: done (13 left) 2024-09-16T15:01:24.060 INFO:tasks.rados.rados.0.smithi162.stdout:14: done (12 left) 2024-09-16T15:01:24.061 INFO:tasks.rados.rados.0.smithi162.stdout:27: write initial oid 27 2024-09-16T15:01:24.061 INFO:tasks.rados.rados.0.smithi162.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2024-09-16T15:01:24.062 INFO:tasks.rados.rados.0.smithi162.stdout:27: writing smithi16234619-27 from 436982 to 1142515 tid 1 2024-09-16T15:01:24.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2288c86640 1 -- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2284071590 msgr2=0x7f228406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:24.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2288c86640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2284071590 0x7f228406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f227c009e50 tx=0x7f227c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fb51b21577b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:24.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 1 -- 172.21.15.134:0/3122000384 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f227c007a70 con 0x7f2284071590 2024-09-16T15:01:24.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: dump: 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.067 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: _finish_auth 0 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 1 -- 172.21.15.134:0/3122000384 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c02d050 con 0x7f2284071590 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f228a6e9640 5 monclient: authenticate success, global_id 4536 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f228a6e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 1 -- 172.21.15.134:0/3122000384 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f227c007bf0 con 0x7f2284071590 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f228a6e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:01:24.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f228a6e9640 4 set_mon_vals no callback set 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.050+0000 7f2282ffd640 10 monclient: dump: 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.069 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 10 monclient: shutdown 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 -- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2284071590 msgr2=0x7f228406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2284071590 0x7f228406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f227c009e50 tx=0x7f227c02a6c0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 -- 172.21.15.134:0/3122000384 shutdown_connections 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f228409fb80 0x7f228406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2284071590 0x7f228406d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3122000384 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2284070c70 0x7f2284071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 -- 172.21.15.134:0/3122000384 >> 172.21.15.134:0/3122000384 conn(0x7f2284066010 msgr2=0x7f2284066420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 -- 172.21.15.134:0/3122000384 shutdown_connections 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.051+0000 7f228a6e9640 1 -- 172.21.15.134:0/3122000384 wait complete. 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.052+0000 7f228a6e9640 10 monclient: build_initial_monmap 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.052+0000 7f228a6e9640 10 monclient: monmap: 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:01:24.071 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.052+0000 7f228a6e9640 1 Processor -- start 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.052+0000 7f228a6e9640 1 -- start start 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.052+0000 7f228a6e9640 10 monclient: init 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient: _add_conns ranks=[8,1,6,3,0,2,7,5,4] 2024-09-16T15:01:24.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-e con 0x7f2284070c70 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2284071590 0x7f22841c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-f con 0x7f2284071590 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f228409fb80 0x7f228411d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): picked mon.noname-d con 0x7f228409fb80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284120f00 con 0x7f2284071590 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2284066480 con 0x7f228409fb80 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2284120c20 con 0x7f2284070c70 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:01:24.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f228a6e9640 10 monclient(hunting): authenticate will time out at 820.642502s 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2288c86640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f228409fb80 msgr2=0x7f228411d5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f22837fe640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2284071590 msgr2=0x7f22841c8b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2288c86640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f228409fb80 0x7f228411d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f22837fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2284071590 0x7f22841c8b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43758/0 (socket says 172.21.15.134:43758) 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 -- 172.21.15.134:0/3848082216 learned_addr learned my addr 172.21.15.134:0/3848082216 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): get_auth_request con 0x7f2284070c70 auth_method 0 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:24.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient(hunting): handle_auth_done global_id 4598 payload 824 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f228409fb80 msgr2=0x7f228411d5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f228409fb80 0x7f228411d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2284071590 msgr2=0x7f22841c8b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2284071590 0x7f22841c8b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient: _finish_hunting 0 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 monclient: found mon.noname-e 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.053+0000 7f2283fff640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009b30 con 0x7f2284070c70 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2283fff640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 msgr2=0x7f22841c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:24.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2283fff640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f227400b580 tx=0x7f227400ba40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2274003dd0 con 0x7f2284070c70 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: dump: 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: _finish_auth 0 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 5 monclient: authenticate success, global_id 4598 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274005560 con 0x7f2284070c70 2024-09-16T15:01:24.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 10 monclient: _renew_subs 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22740056d0 con 0x7f2284070c70 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228411f0f0 con 0x7f2284070c70 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f2280ff9640 10 monclient: dump: 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.078 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f22827fc640 4 set_mon_vals no callback set 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 10 monclient: _renew_subs 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.056+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22841c9090 con 0x7f2284070c70 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f2283fff640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f227400b580 tx=0x7f227400ba40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f22740050f0 mgrmap(e 5) v1 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f22740050f0 con 0x7f2284070c70 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f2280ff9640 1 --2- 172.21.15.134:0/3848082216 >> v2:172.21.15.162:6800/980547263 conn(0x7f2270038880 0x7f227003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f228a6e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 0x7f228411ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f22840a4650 con 0x7f228411daf0 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f22837fe640 1 --2- 172.21.15.134:0/3848082216 >> v2:172.21.15.162:6800/980547263 conn(0x7f2270038880 0x7f227003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.057+0000 7f22837fe640 10 monclient: get_auth_request con 0x7f2270038880 auth_method 0 2024-09-16T15:01:24.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 0x7f228411ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient: get_auth_request con 0x7f228411daf0 auth_method 0 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 10 monclient(hunting): handle_auth_done global_id 4399 payload 824 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.058+0000 7f2288c86640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2284061ae0 con 0x7f228411daf0 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.060+0000 7f22837fe640 1 --2- 172.21.15.134:0/3848082216 >> v2:172.21.15.162:6800/980547263 conn(0x7f2270038880 0x7f227003ad40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006d00 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.060+0000 7f2288c86640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 msgr2=0x7f228411ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.060+0000 7f2288c86640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 0x7f228411ded0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f227c033000 tx=0x7f227c004a40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:24.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f227c003db0 con 0x7f228411daf0 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 10 monclient: dump: 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.082 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f227c0099b0 con 0x7f228411daf0 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 msgr2=0x7f228411ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 0x7f228411ded0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f227c033000 tx=0x7f227c004a40 comp rx=0 tx=0).stop 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 msgr2=0x7f228411ded0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:24.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.061+0000 7f2280ff9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411daf0 0x7f228411ded0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f227c033000 tx=0x7f227c004a40 comp rx=0 tx=0).stop 2024-09-16T15:01:24.085 INFO:tasks.rados.rados.0.smithi162.stdout:27: writing smithi16234619-27 from 1855120 to 2436982 tid 2 2024-09-16T15:01:24.085 INFO:tasks.rados.rados.0.smithi162.stdout:28: write initial oid 28 2024-09-16T15:01:24.086 INFO:tasks.rados.rados.0.smithi162.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2024-09-16T15:01:24.086 INFO:tasks.rados.rados.0.smithi162.stdout:28: writing smithi16234619-28 from 453789 to 1234571 tid 1 2024-09-16T15:01:24.086 INFO:tasks.rados.rados.0.smithi162.stdout:28: writing smithi16234619-28 from 1913602 to 2388604 tid 2 2024-09-16T15:01:24.086 INFO:tasks.rados.rados.0.smithi162.stdout:29: write initial oid 29 2024-09-16T15:01:24.086 INFO:tasks.rados.rados.0.smithi162.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.072+0000 7f228a6e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 0x7f228406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f228a6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f228406a870 con 0x7f228411e410 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 0x7f228406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient: get_auth_request con 0x7f228411e410 auth_method 0 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:01:24.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 10 monclient(hunting): handle_auth_done global_id 4408 payload 824 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.073+0000 7f2288c86640 1 -- 172.21.15.134:0/3848082216 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f228406f740 con 0x7f228411e410 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.075+0000 7f2288c86640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 msgr2=0x7f228406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.075+0000 7f2288c86640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 0x7f228406ce50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f227c0093e0 tx=0x7f227c009810 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f227c007c40 con 0x7f228411e410 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 10 monclient: dump: 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.088 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7f227c004450 con 0x7f228411e410 2024-09-16T15:01:24.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 msgr2=0x7f228406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 0x7f228406ce50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f227c0093e0 tx=0x7f227c009810 comp rx=0 tx=0).stop 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 msgr2=0x7f228406ce50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.076+0000 7f2280ff9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f228411e410 0x7f228406ce50 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f227c0093e0 tx=0x7f227c009810 comp rx=0 tx=0).stop 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:24.090 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 12, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:01:24.091 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:01:24.092 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:01:24.094 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:01:24.095 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:01:24.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:01:24.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.102 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.105 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:01:24.107 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.110 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:01:24.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:24.112 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:01:24.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:01:24.114 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:01:24.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:01:24.114 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:01:24.114 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 >> v2:172.21.15.162:6800/980547263 conn(0x7f2270038880 msgr2=0x7f227003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3848082216 >> v2:172.21.15.162:6800/980547263 conn(0x7f2270038880 0x7f227003ad40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006d00 comp rx=0 tx=0).stop 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 10 monclient: shutdown 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 msgr2=0x7f22841c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:01:24.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 --2- 172.21.15.134:0/3848082216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2284070c70 0x7f22841c8610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f227400b580 tx=0x7f227400ba40 comp rx=0 tx=0).stop 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f2288c86640 1 -- 172.21.15.134:0/3848082216 reap_dead start 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f2280ff9640 1 -- 172.21.15.134:0/3848082216 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3871+0+0 (unknown 0 0 0) 0x7f2274049e00 con 0x7f2284070c70 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 shutdown_connections 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 >> 172.21.15.134:0/3848082216 conn(0x7f2284066010 msgr2=0x7f22841bab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 shutdown_connections 2024-09-16T15:01:24.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:01:24.077+0000 7f228a6e9640 1 -- 172.21.15.134:0/3848082216 wait complete. 2024-09-16T15:01:24.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:01:24.119 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:29: writing smithi16234619-29 from 470596 to 926627 tid 1 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:29: writing smithi16234619-29 from 1572084 to 2190744 tid 2 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:30: write initial oid 30 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:30: writing smithi16234619-30 from 487403 to 1018683 tid 1 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout:30: writing smithi16234619-30 from 1514213 to 1992884 tid 2 2024-09-16T15:01:24.121 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: finishing write tid 1 to smithi16234619-15 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: finishing write tid 2 to smithi16234619-15 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: finishing write tid 3 to smithi16234619-15 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: finishing write tid 4 to smithi16234619-15 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: oid 15 updating version 0 to 2 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: oid 15 updating version 2 to 3 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: oid 15 version 3 is already newer than 1 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:16: finishing write tid 1 to smithi16234619-16 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:16: finishing write tid 2 to smithi16234619-16 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:16: finishing write tid 3 to smithi16234619-16 2024-09-16T15:01:24.122 INFO:tasks.rados.rados.0.smithi162.stdout:16: finishing write tid 4 to smithi16234619-16 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:16: oid 16 updating version 0 to 6 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:16: oid 16 version 6 is already newer than 4 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:16: oid 16 version 6 is already newer than 5 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: finishing write tid 1 to smithi16234619-17 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: finishing write tid 2 to smithi16234619-17 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: finishing write tid 3 to smithi16234619-17 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: finishing write tid 4 to smithi16234619-17 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: oid 17 updating version 0 to 7 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: oid 17 updating version 7 to 8 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: oid 17 updating version 8 to 9 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 9 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T15:01:24.123 INFO:tasks.rados.rados.0.smithi162.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: finishing write tid 1 to smithi16234619-18 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: finishing write tid 2 to smithi16234619-18 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: finishing write tid 3 to smithi16234619-18 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: finishing write tid 4 to smithi16234619-18 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: oid 18 updating version 0 to 9 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: oid 18 version 9 is already newer than 7 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: oid 18 version 9 is already newer than 8 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 9 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:20: finishing write tid 1 to smithi16234619-20 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:20: finishing write tid 2 to smithi16234619-20 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:20: finishing write tid 3 to smithi16234619-20 2024-09-16T15:01:24.124 INFO:tasks.rados.rados.0.smithi162.stdout:20: finishing write tid 4 to smithi16234619-20 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:20: oid 20 updating version 0 to 1 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:20: oid 20 updating version 1 to 2 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:20: oid 20 updating version 2 to 3 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: finishing write tid 1 to smithi16234619-19 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: finishing write tid 2 to smithi16234619-19 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: finishing write tid 3 to smithi16234619-19 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: finishing write tid 4 to smithi16234619-19 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: oid 19 updating version 0 to 5 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: oid 19 version 5 is already newer than 4 2024-09-16T15:01:24.125 INFO:tasks.rados.rados.0.smithi162.stdout:19: oid 19 updating version 5 to 6 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: finishing write tid 1 to smithi16234619-21 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: finishing write tid 2 to smithi16234619-21 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: finishing write tid 3 to smithi16234619-21 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: finishing write tid 4 to smithi16234619-21 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: oid 21 updating version 0 to 11 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: oid 21 updating version 11 to 12 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: oid 21 version 12 is already newer than 10 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 12 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:22: finishing write tid 1 to smithi16234619-22 2024-09-16T15:01:24.126 INFO:tasks.rados.rados.0.smithi162.stdout:22: finishing write tid 2 to smithi16234619-22 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: finishing write tid 3 to smithi16234619-22 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: finishing write tid 4 to smithi16234619-22 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: oid 22 updating version 0 to 3 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: oid 22 version 3 is already newer than 1 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: oid 22 version 3 is already newer than 2 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 3 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:23: finishing write tid 1 to smithi16234619-23 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:23: finishing write tid 2 to smithi16234619-23 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:23: finishing write tid 3 to smithi16234619-23 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:23: finishing write tid 4 to smithi16234619-23 2024-09-16T15:01:24.127 INFO:tasks.rados.rados.0.smithi162.stdout:23: oid 23 updating version 0 to 2 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:23: oid 23 updating version 2 to 3 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:23: oid 23 version 3 is already newer than 1 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:15: done (15 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:16: done (14 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:17: done (13 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:18: done (12 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:19: done (11 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:20: done (10 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:21: done (9 left) 2024-09-16T15:01:24.128 INFO:tasks.rados.rados.0.smithi162.stdout:22: done (8 left) 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:23: done (7 left) 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:31: write initial oid 31 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:31: writing smithi16234619-31 from 504210 to 1110739 tid 1 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:31: writing smithi16234619-31 from 1572695 to 2195024 tid 2 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:32: write initial oid 32 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2024-09-16T15:01:24.129 INFO:tasks.rados.rados.0.smithi162.stdout:32: writing smithi16234619-32 from 521017 to 1319148 tid 1 2024-09-16T15:01:24.130 INFO:tasks.rados.rados.0.smithi162.stdout:32: writing smithi16234619-32 from 1747530 to 2229870 tid 2 2024-09-16T15:01:24.130 INFO:tasks.rados.rados.0.smithi162.stdout:33: write initial oid 33 2024-09-16T15:01:24.130 INFO:tasks.rados.rados.0.smithi162.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2024-09-16T15:01:24.135 INFO:tasks.rados.rados.0.smithi162.stdout:33: writing smithi16234619-33 from 537824 to 1011204 tid 1 2024-09-16T15:01:24.143 INFO:tasks.rados.rados.0.smithi162.stdout:33: writing smithi16234619-33 from 1806012 to 2432010 tid 2 2024-09-16T15:01:24.143 INFO:tasks.rados.rados.0.smithi162.stdout:34: write initial oid 34 2024-09-16T15:01:24.143 INFO:tasks.rados.rados.0.smithi162.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2024-09-16T15:01:24.149 INFO:tasks.rados.rados.0.smithi162.stdout:34: writing smithi16234619-34 from 554631 to 1103260 tid 1 2024-09-16T15:01:24.155 INFO:tasks.rados.rados.0.smithi162.stdout:34: writing smithi16234619-34 from 1748141 to 2234150 tid 2 2024-09-16T15:01:24.156 INFO:tasks.rados.rados.0.smithi162.stdout:35: write initial oid 35 2024-09-16T15:01:24.156 INFO:tasks.rados.rados.0.smithi162.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2024-09-16T15:01:24.162 INFO:tasks.rados.rados.0.smithi162.stdout:35: writing smithi16234619-35 from 571438 to 1195316 tid 1 2024-09-16T15:01:24.169 INFO:tasks.rados.rados.0.smithi162.stdout:35: writing smithi16234619-35 from 1806623 to 2436290 tid 2 2024-09-16T15:01:24.169 INFO:tasks.rados.rados.0.smithi162.stdout:36: write initial oid 36 2024-09-16T15:01:24.169 INFO:tasks.rados.rados.0.smithi162.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2024-09-16T15:01:24.176 INFO:tasks.rados.rados.0.smithi162.stdout:36: writing smithi16234619-36 from 588245 to 1287372 tid 1 2024-09-16T15:01:24.183 INFO:tasks.rados.rados.0.smithi162.stdout:36: writing smithi16234619-36 from 1865105 to 2354783 tid 2 2024-09-16T15:01:24.183 INFO:tasks.rados.rados.0.smithi162.stdout:37: write initial oid 37 2024-09-16T15:01:24.183 INFO:tasks.rados.rados.0.smithi162.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2024-09-16T15:01:24.192 INFO:tasks.rados.rados.0.smithi162.stdout:37: writing smithi16234619-37 from 605052 to 1379428 tid 1 2024-09-16T15:01:24.198 INFO:tasks.rados.rados.0.smithi162.stdout:37: writing smithi16234619-37 from 1923587 to 2556923 tid 2 2024-09-16T15:01:24.199 INFO:tasks.rados.rados.0.smithi162.stdout:38: write initial oid 38 2024-09-16T15:01:24.199 INFO:tasks.rados.rados.0.smithi162.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2024-09-16T15:01:24.204 INFO:tasks.rados.rados.0.smithi162.stdout:38: writing smithi16234619-38 from 621859 to 1071484 tid 1 2024-09-16T15:01:24.213 INFO:tasks.rados.rados.0.smithi162.stdout:38: writing smithi16234619-38 from 1865716 to 2621859 tid 2 2024-09-16T15:01:24.213 INFO:tasks.rados.rados.0.smithi162.stdout:39: write initial oid 39 2024-09-16T15:01:24.213 INFO:tasks.rados.rados.0.smithi162.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2024-09-16T15:01:24.220 INFO:tasks.rados.rados.0.smithi162.stdout:39: writing smithi16234619-39 from 638666 to 1163540 tid 1 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:39: writing smithi16234619-39 from 1924198 to 2561203 tid 2 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: finishing write tid 1 to smithi16234619-24 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: finishing write tid 2 to smithi16234619-24 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: finishing write tid 3 to smithi16234619-24 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: finishing write tid 4 to smithi16234619-24 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: oid 24 updating version 0 to 13 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: oid 24 updating version 13 to 14 2024-09-16T15:01:24.230 INFO:tasks.rados.rados.0.smithi162.stdout:24: oid 24 updating version 14 to 15 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 15 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:25: finishing write tid 1 to smithi16234619-25 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:25: finishing write tid 2 to smithi16234619-25 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:24: done (15 left) 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:40: write initial oid 40 2024-09-16T15:01:24.231 INFO:tasks.rados.rados.0.smithi162.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2024-09-16T15:01:24.237 INFO:tasks.rados.rados.0.smithi162.stdout:40: writing smithi16234619-40 from 655473 to 1371949 tid 1 2024-09-16T15:01:24.245 INFO:tasks.rados.rados.0.smithi162.stdout:40: writing smithi16234619-40 from 2099033 to 2655473 tid 2 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: finishing write tid 1 to smithi16234619-26 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: finishing write tid 2 to smithi16234619-26 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: finishing write tid 3 to smithi16234619-26 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: finishing write tid 4 to smithi16234619-26 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: oid 26 updating version 0 to 18 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: oid 26 version 18 is already newer than 16 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: oid 26 version 18 is already newer than 17 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 18 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:25: finishing write tid 3 to smithi16234619-25 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:27: finishing write tid 1 to smithi16234619-27 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:25: finishing write tid 4 to smithi16234619-25 2024-09-16T15:01:24.246 INFO:tasks.rados.rados.0.smithi162.stdout:25: oid 25 updating version 0 to 7 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:25: oid 25 updating version 7 to 8 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:25: oid 25 updating version 8 to 9 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 9 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:27: finishing write tid 2 to smithi16234619-27 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:25: done (15 left) 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:26: done (14 left) 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:41: write initial oid 41 2024-09-16T15:01:24.247 INFO:tasks.rados.rados.0.smithi162.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2024-09-16T15:01:24.254 INFO:tasks.rados.rados.0.smithi162.stdout:41: writing smithi16234619-41 from 672280 to 1464005 tid 1 2024-09-16T15:01:24.262 INFO:tasks.rados.rados.0.smithi162.stdout:41: writing smithi16234619-41 from 2157515 to 2672280 tid 2 2024-09-16T15:01:24.262 INFO:tasks.rados.rados.0.smithi162.stdout:42: write initial oid 42 2024-09-16T15:01:24.262 INFO:tasks.rados.rados.0.smithi162.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2024-09-16T15:01:24.268 INFO:tasks.rados.rados.0.smithi162.stdout:42: writing smithi16234619-42 from 689087 to 1156061 tid 1 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:42: writing smithi16234619-42 from 1699644 to 2483976 tid 2 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: finishing write tid 3 to smithi16234619-27 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: finishing write tid 4 to smithi16234619-27 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: oid 27 updating version 0 to 10 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: oid 27 updating version 10 to 11 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: oid 27 updating version 11 to 12 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:29: finishing write tid 1 to smithi16234619-29 2024-09-16T15:01:24.276 INFO:tasks.rados.rados.0.smithi162.stdout:29: finishing write tid 2 to smithi16234619-29 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: finishing write tid 3 to smithi16234619-29 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: finishing write tid 4 to smithi16234619-29 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: oid 29 updating version 0 to 7 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: oid 29 updating version 7 to 9 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: oid 29 version 9 is already newer than 8 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 9 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:27: done (15 left) 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:29: done (14 left) 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:43: write initial oid 43 2024-09-16T15:01:24.277 INFO:tasks.rados.rados.0.smithi162.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2024-09-16T15:01:24.282 INFO:tasks.rados.rados.0.smithi162.stdout:43: writing smithi16234619-43 from 705894 to 1248117 tid 1 2024-09-16T15:01:24.288 INFO:tasks.rados.rados.0.smithi162.stdout:43: writing smithi16234619-43 from 1758126 to 2402469 tid 2 2024-09-16T15:01:24.289 INFO:tasks.rados.rados.0.smithi162.stdout:44: write initial oid 44 2024-09-16T15:01:24.289 INFO:tasks.rados.rados.0.smithi162.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2024-09-16T15:01:24.295 INFO:tasks.rados.rados.0.smithi162.stdout:44: writing smithi16234619-44 from 722701 to 1340173 tid 1 2024-09-16T15:01:24.306 INFO:tasks.rados.rados.0.smithi162.stdout:44: writing smithi16234619-44 from 1816608 to 2604609 tid 2 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 16 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: finishing write tid 1 to smithi16234619-28 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: finishing write tid 2 to smithi16234619-28 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: finishing write tid 3 to smithi16234619-28 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: finishing write tid 4 to smithi16234619-28 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: oid 28 updating version 0 to 5 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: oid 28 version 5 is already newer than 4 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: oid 28 updating version 5 to 6 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:30: finishing write tid 1 to smithi16234619-30 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:30: finishing write tid 2 to smithi16234619-30 2024-09-16T15:01:24.307 INFO:tasks.rados.rados.0.smithi162.stdout:30: finishing write tid 3 to smithi16234619-30 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:30: finishing write tid 4 to smithi16234619-30 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:30: oid 30 updating version 0 to 8 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:30: oid 30 updating version 8 to 9 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:30: oid 30 version 9 is already newer than 7 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: finishing write tid 1 to smithi16234619-31 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: finishing write tid 2 to smithi16234619-31 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: finishing write tid 3 to smithi16234619-31 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: finishing write tid 4 to smithi16234619-31 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: oid 31 updating version 0 to 11 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: oid 31 updating version 11 to 12 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:31: oid 31 version 12 is already newer than 10 2024-09-16T15:01:24.308 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: finishing write tid 1 to smithi16234619-33 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: finishing write tid 2 to smithi16234619-33 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: finishing write tid 3 to smithi16234619-33 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: finishing write tid 4 to smithi16234619-33 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: oid 33 updating version 0 to 6 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: oid 33 version 6 is already newer than 4 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: oid 33 version 6 is already newer than 5 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:32: finishing write tid 1 to smithi16234619-32 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:32: finishing write tid 2 to smithi16234619-32 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:32: finishing write tid 3 to smithi16234619-32 2024-09-16T15:01:24.309 INFO:tasks.rados.rados.0.smithi162.stdout:32: finishing write tid 4 to smithi16234619-32 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:32: oid 32 updating version 0 to 4 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:32: oid 32 updating version 4 to 5 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:32: oid 32 updating version 5 to 6 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: finishing write tid 1 to smithi16234619-34 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: finishing write tid 2 to smithi16234619-34 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: finishing write tid 3 to smithi16234619-34 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: finishing write tid 4 to smithi16234619-34 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: oid 34 updating version 0 to 10 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: oid 34 updating version 10 to 11 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: oid 34 updating version 11 to 12 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 12 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T15:01:24.310 INFO:tasks.rados.rados.0.smithi162.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: finishing write tid 1 to smithi16234619-35 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: finishing write tid 2 to smithi16234619-35 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: finishing write tid 3 to smithi16234619-35 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: finishing write tid 4 to smithi16234619-35 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: oid 35 updating version 0 to 9 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: oid 35 version 9 is already newer than 8 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: oid 35 version 9 is already newer than 7 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 9 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:36: finishing write tid 1 to smithi16234619-36 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:36: finishing write tid 2 to smithi16234619-36 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:36: finishing write tid 3 to smithi16234619-36 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:36: finishing write tid 4 to smithi16234619-36 2024-09-16T15:01:24.311 INFO:tasks.rados.rados.0.smithi162.stdout:36: oid 36 updating version 0 to 15 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:36: oid 36 version 15 is already newer than 14 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:36: oid 36 version 15 is already newer than 13 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 15 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:37: finishing write tid 1 to smithi16234619-37 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:37: finishing write tid 2 to smithi16234619-37 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: finishing write tid 1 to smithi16234619-38 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: finishing write tid 2 to smithi16234619-38 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: finishing write tid 3 to smithi16234619-38 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: finishing write tid 4 to smithi16234619-38 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: oid 38 updating version 0 to 11 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: oid 38 version 11 is already newer than 10 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:38: oid 38 updating version 11 to 12 2024-09-16T15:01:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: finishing write tid 3 to smithi16234619-37 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: finishing write tid 4 to smithi16234619-37 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: oid 37 updating version 0 to 10 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: oid 37 updating version 10 to 11 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: oid 37 updating version 11 to 12 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 12 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:39: finishing write tid 1 to smithi16234619-39 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:39: finishing write tid 2 to smithi16234619-39 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:39: finishing write tid 3 to smithi16234619-39 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:39: finishing write tid 4 to smithi16234619-39 2024-09-16T15:01:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:39: oid 39 updating version 0 to 13 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:39: oid 39 updating version 13 to 15 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:39: oid 39 version 15 is already newer than 14 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 15 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: finishing write tid 1 to smithi16234619-40 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: finishing write tid 2 to smithi16234619-40 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: finishing write tid 3 to smithi16234619-40 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: finishing write tid 4 to smithi16234619-40 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: oid 40 updating version 0 to 7 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: oid 40 updating version 7 to 9 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: oid 40 version 9 is already newer than 8 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 9 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-16T15:01:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:41: finishing write tid 1 to smithi16234619-41 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: finishing write tid 2 to smithi16234619-41 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: finishing write tid 3 to smithi16234619-41 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: finishing write tid 4 to smithi16234619-41 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: oid 41 updating version 0 to 6 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: oid 41 version 6 is already newer than 5 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: oid 41 version 6 is already newer than 4 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: finishing write tid 1 to smithi16234619-42 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: finishing write tid 2 to smithi16234619-42 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: finishing write tid 3 to smithi16234619-42 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: finishing write tid 4 to smithi16234619-42 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: oid 42 updating version 0 to 17 2024-09-16T15:01:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:42: oid 42 updating version 17 to 18 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:42: oid 42 version 18 is already newer than 16 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 18 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:28: done (15 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:30: done (14 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:31: done (13 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:32: done (12 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:33: done (11 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:34: done (10 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:35: done (9 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:36: done (8 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:37: done (7 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:38: done (6 left) 2024-09-16T15:01:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:39: done (5 left) 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:40: done (4 left) 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:41: done (3 left) 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:42: done (2 left) 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:45: write initial oid 45 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2024-09-16T15:01:24.317 INFO:tasks.rados.rados.0.smithi162.stdout:45: writing smithi16234619-45 from 739508 to 1432229 tid 1 2024-09-16T15:01:24.320 INFO:tasks.rados.rados.0.smithi162.stdout:45: writing smithi16234619-45 from 1875090 to 2523102 tid 2 2024-09-16T15:01:24.320 INFO:tasks.rados.rados.0.smithi162.stdout:46: write initial oid 46 2024-09-16T15:01:24.320 INFO:tasks.rados.rados.0.smithi162.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2024-09-16T15:01:24.328 INFO:tasks.rados.rados.0.smithi162.stdout:46: writing smithi16234619-46 from 756315 to 1524285 tid 1 2024-09-16T15:01:24.335 INFO:tasks.rados.rados.0.smithi162.stdout:46: writing smithi16234619-46 from 1933572 to 2725242 tid 2 2024-09-16T15:01:24.335 INFO:tasks.rados.rados.0.smithi162.stdout:47: write initial oid 47 2024-09-16T15:01:24.335 INFO:tasks.rados.rados.0.smithi162.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2024-09-16T15:01:24.342 INFO:tasks.rados.rados.0.smithi162.stdout:47: writing smithi16234619-47 from 773122 to 1332694 tid 1 2024-09-16T15:01:24.350 INFO:tasks.rados.rados.0.smithi162.stdout:47: writing smithi16234619-47 from 1992054 to 2643735 tid 2 2024-09-16T15:01:24.350 INFO:tasks.rados.rados.0.smithi162.stdout:48: write initial oid 48 2024-09-16T15:01:24.350 INFO:tasks.rados.rados.0.smithi162.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2024-09-16T15:01:24.358 INFO:tasks.rados.rados.0.smithi162.stdout:48: writing smithi16234619-48 from 789929 to 1424750 tid 1 2024-09-16T15:01:24.367 INFO:tasks.rados.rados.0.smithi162.stdout:48: writing smithi16234619-48 from 2050536 to 2789929 tid 2 2024-09-16T15:01:24.367 INFO:tasks.rados.rados.0.smithi162.stdout:49: write initial oid 49 2024-09-16T15:01:24.367 INFO:tasks.rados.rados.0.smithi162.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2024-09-16T15:01:24.374 INFO:tasks.rados.rados.0.smithi162.stdout:49: writing smithi16234619-49 from 406736 to 1116806 tid 1 2024-09-16T15:01:24.382 INFO:tasks.rados.rados.0.smithi162.stdout:49: writing smithi16234619-49 from 1709018 to 2364368 tid 2 2024-09-16T15:01:24.382 INFO:tasks.rados.rados.0.smithi162.stdout:50: write initial oid 50 2024-09-16T15:01:24.382 INFO:tasks.rados.rados.0.smithi162.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2024-09-16T15:01:24.388 INFO:tasks.rados.rados.0.smithi162.stdout:50: writing smithi16234619-50 from 423543 to 1208862 tid 1 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:50: writing smithi16234619-50 from 1767500 to 2566508 tid 2 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:51: rollback oid 35 current snap is 0 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:51: done (8 left) 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:52: snap_create 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:43: finishing write tid 1 to smithi16234619-43 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:43: finishing write tid 2 to smithi16234619-43 2024-09-16T15:01:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:43: finishing write tid 3 to smithi16234619-43 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:43: finishing write tid 4 to smithi16234619-43 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:43: oid 43 updating version 0 to 1 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:43: oid 43 updating version 1 to 2 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:43: oid 43 updating version 2 to 3 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 3 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: finishing write tid 1 to smithi16234619-44 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: finishing write tid 2 to smithi16234619-44 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: finishing write tid 3 to smithi16234619-44 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: finishing write tid 4 to smithi16234619-44 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: oid 44 updating version 0 to 15 2024-09-16T15:01:24.398 INFO:tasks.rados.rados.0.smithi162.stdout:44: oid 44 version 15 is already newer than 14 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:44: oid 44 version 15 is already newer than 13 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 15 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: finishing write tid 1 to smithi16234619-45 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: finishing write tid 2 to smithi16234619-45 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: finishing write tid 3 to smithi16234619-45 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: finishing write tid 4 to smithi16234619-45 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: oid 45 updating version 0 to 20 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: oid 45 updating version 20 to 21 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: oid 45 version 21 is already newer than 19 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 21 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:46: finishing write tid 1 to smithi16234619-46 2024-09-16T15:01:24.399 INFO:tasks.rados.rados.0.smithi162.stdout:46: finishing write tid 2 to smithi16234619-46 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: finishing write tid 3 to smithi16234619-46 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: finishing write tid 4 to smithi16234619-46 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: oid 46 updating version 0 to 2 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: oid 46 version 2 is already newer than 1 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: oid 46 updating version 2 to 3 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 3 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: finishing write tid 1 to smithi16234619-47 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: finishing write tid 2 to smithi16234619-47 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: finishing write tid 3 to smithi16234619-47 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: finishing write tid 4 to smithi16234619-47 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: oid 47 updating version 0 to 6 2024-09-16T15:01:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:47: oid 47 version 6 is already newer than 4 2024-09-16T15:01:24.401 INFO:tasks.rados.rados.0.smithi162.stdout:47: oid 47 version 6 is already newer than 5 2024-09-16T15:01:24.401 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 6 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T15:01:24.401 INFO:tasks.rados.rados.0.smithi162.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-16T15:01:24.401 INFO:tasks.rados.rados.0.smithi162.stdout:48: finishing write tid 1 to smithi16234619-48 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: finishing write tid 2 to smithi16234619-48 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: finishing write tid 3 to smithi16234619-48 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: finishing write tid 4 to smithi16234619-48 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: oid 48 updating version 0 to 13 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: oid 48 updating version 13 to 15 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: oid 48 version 15 is already newer than 14 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-16T15:01:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: finishing write tid 1 to smithi16234619-49 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: finishing write tid 2 to smithi16234619-49 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: finishing write tid 3 to smithi16234619-49 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: finishing write tid 4 to smithi16234619-49 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: oid 49 updating version 0 to 8 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: oid 49 version 8 is already newer than 7 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: oid 49 updating version 8 to 9 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-16T15:01:24.414 INFO:tasks.rados.rados.0.smithi162.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-16T15:01:24.424 INFO:tasks.rados.rados.0.smithi162.stdout:50: finishing write tid 1 to smithi16234619-50 2024-09-16T15:01:24.427 INFO:tasks.rados.rados.0.smithi162.stdout:50: finishing write tid 2 to smithi16234619-50 2024-09-16T15:01:24.430 INFO:tasks.rados.rados.0.smithi162.stdout:50: finishing write tid 3 to smithi16234619-50 2024-09-16T15:01:24.430 INFO:tasks.rados.rados.0.smithi162.stdout:50: finishing write tid 4 to smithi16234619-50 2024-09-16T15:01:24.430 INFO:tasks.rados.rados.0.smithi162.stdout:50: oid 50 updating version 0 to 17 2024-09-16T15:01:24.431 INFO:tasks.rados.rados.0.smithi162.stdout:50: oid 50 version 17 is already newer than 16 2024-09-16T15:01:24.431 INFO:tasks.rados.rados.0.smithi162.stdout:50: oid 50 updating version 17 to 18 2024-09-16T15:01:24.431 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 18 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T15:01:24.431 INFO:tasks.rados.rados.0.smithi162.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:43: done (8 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:44: done (7 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:45: done (6 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:46: done (5 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:47: done (4 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:48: done (3 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:49: done (2 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:50: done (1 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:52: done (0 left) 2024-09-16T15:01:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:53: snap_remove snap 0 2024-09-16T15:01:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:53: done (0 left) 2024-09-16T15:01:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:54: read oid 37 snap -1 2024-09-16T15:01:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:54: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T15:01:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:55: write oid 12 current snap is 1 2024-09-16T15:01:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:55: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2024-09-16T15:01:26.701 INFO:tasks.rados.rados.0.smithi162.stdout:55: writing smithi16234619-12 from 440350 to 900918 tid 1 2024-09-16T15:01:26.706 INFO:tasks.rados.rados.0.smithi162.stdout:55: writing smithi16234619-12 from 1309629 to 1852295 tid 2 2024-09-16T15:01:26.711 INFO:tasks.rados.rados.0.smithi162.stdout:55: writing smithi16234619-12 from 2323973 to 2840350 tid 3 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:56: rollback oid 10 current snap is 1 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:56: done (2 left) 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:57: rollback oid 27 current snap is 1 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:57: done (2 left) 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:58: write (excl) oid 23 current snap is 1 2024-09-16T15:01:26.712 INFO:tasks.rados.rados.0.smithi162.stdout:58: seq_num 51 ranges {457157=535817,1768111=402677} 2024-09-16T15:01:26.717 INFO:tasks.rados.rados.0.smithi162.stdout:58: writing smithi16234619-23 from 457157 to 992974 tid 1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:58: writing smithi16234619-23 from 1768111 to 2170788 tid 2 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:59: read oid 24 snap -1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:59: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:60: read oid 10 snap -1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:60: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:61: read oid 40 snap -1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:61: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:62: rollback oid 9 current snap is 1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:62: done (6 left) 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:63: copy_from oid 7 from oid 31 current snap is 1 2024-09-16T15:01:26.724 INFO:tasks.rados.rados.0.smithi162.stdout:54: expect (ObjNum 36 snap 0 seq_num 36) 2024-09-16T15:01:26.807 INFO:tasks.rados.rados.0.smithi162.stdout:59: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-16T15:01:26.884 INFO:tasks.rados.rados.0.smithi162.stdout:60: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T15:01:26.951 INFO:tasks.rados.rados.0.smithi162.stdout:55: finishing write tid 1 to smithi16234619-12 2024-09-16T15:01:26.951 INFO:tasks.rados.rados.0.smithi162.stdout:61: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T15:01:27.035 INFO:tasks.rados.rados.0.smithi162.stdout:55: finishing write tid 2 to smithi16234619-12 2024-09-16T15:01:27.035 INFO:tasks.rados.rados.0.smithi162.stdout:55: finishing write tid 3 to smithi16234619-12 2024-09-16T15:01:27.035 INFO:tasks.rados.rados.0.smithi162.stdout:55: finishing write tid 4 to smithi16234619-12 2024-09-16T15:01:27.035 INFO:tasks.rados.rados.0.smithi162.stdout:55: finishing write tid 5 to smithi16234619-12 2024-09-16T15:01:27.035 INFO:tasks.rados.rados.0.smithi162.stdout:55: oid 12 updating version 0 to 10 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:55: oid 12 updating version 10 to 12 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:55: oid 12 version 12 is already newer than 11 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:55: oid 12 updating version 12 to 13 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 13 (ObjNum 50 snap 1 seq_num 50) dirty exists 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:55: left oid 12 (ObjNum 50 snap 1 seq_num 50) 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: finishing write tid 1 to smithi16234619-23 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: finishing write tid 2 to smithi16234619-23 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: finishing write tid 3 to smithi16234619-23 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: finishing write tid 4 to smithi16234619-23 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: oid 23 updating version 0 to 5 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: oid 23 updating version 5 to 6 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:58: oid 23 version 6 is already newer than 4 2024-09-16T15:01:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 6 (ObjNum 51 snap 1 seq_num 51) dirty exists 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:58: left oid 23 (ObjNum 51 snap 1 seq_num 51) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:54: done (6 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:55: done (5 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:58: done (4 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:59: done (3 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:60: done (2 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:61: done (1 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:64: copy_from oid 37 from oid 25 current snap is 1 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:63: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:63: finishing copy_from to smithi16234619-7 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 12 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:63: done (1 left) 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:65: copy_from oid 3 from oid 12 current snap is 1 2024-09-16T15:01:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:66: write (excl) oid 34 current snap is 1 2024-09-16T15:01:27.038 INFO:tasks.rados.rados.0.smithi162.stdout:66: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2024-09-16T15:01:27.040 INFO:tasks.rados.rados.0.smithi162.stdout:66: writing smithi16234619-34 from 473964 to 1085030 tid 1 2024-09-16T15:01:27.048 INFO:tasks.rados.rados.0.smithi162.stdout:66: writing smithi16234619-34 from 1826593 to 2372928 tid 2 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:66: writing smithi16234619-34 from 2778819 to 2873964 tid 3 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:67: read oid 21 snap -1 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:67: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:68: read oid 42 snap -1 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:68: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:69: rollback oid 28 current snap is 1 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:69: done (5 left) 2024-09-16T15:01:27.051 INFO:tasks.rados.rados.0.smithi162.stdout:70: rollback oid 22 current snap is 1 2024-09-16T15:01:27.052 INFO:tasks.rados.rados.0.smithi162.stdout:70: done (5 left) 2024-09-16T15:01:27.052 INFO:tasks.rados.rados.0.smithi162.stdout:71: read oid 32 snap -1 2024-09-16T15:01:27.052 INFO:tasks.rados.rados.0.smithi162.stdout:71: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T15:01:27.052 INFO:tasks.rados.rados.0.smithi162.stdout:72: write (excl) oid 44 current snap is 1 2024-09-16T15:01:27.052 INFO:tasks.rados.rados.0.smithi162.stdout:72: seq_num 53 ranges {490771=686315,1885075=406346} 2024-09-16T15:01:27.057 INFO:tasks.rados.rados.0.smithi162.stdout:72: writing smithi16234619-44 from 490771 to 1177086 tid 1 2024-09-16T15:01:27.065 INFO:tasks.rados.rados.0.smithi162.stdout:72: writing smithi16234619-44 from 1885075 to 2291421 tid 2 2024-09-16T15:01:27.065 INFO:tasks.rados.rados.0.smithi162.stdout:73: write (excl) oid 48 current snap is 1 2024-09-16T15:01:27.065 INFO:tasks.rados.rados.0.smithi162.stdout:73: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2024-09-16T15:01:27.071 INFO:tasks.rados.rados.0.smithi162.stdout:73: writing smithi16234619-48 from 507578 to 985495 tid 1 2024-09-16T15:01:27.078 INFO:tasks.rados.rados.0.smithi162.stdout:73: writing smithi16234619-48 from 1543557 to 2093561 tid 2 2024-09-16T15:01:27.083 INFO:tasks.rados.rados.0.smithi162.stdout:73: writing smithi16234619-48 from 2833665 to 2907578 tid 3 2024-09-16T15:01:27.083 INFO:tasks.rados.rados.0.smithi162.stdout:74: write (excl) oid 31 current snap is 1 2024-09-16T15:01:27.083 INFO:tasks.rados.rados.0.smithi162.stdout:74: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2024-09-16T15:01:27.089 INFO:tasks.rados.rados.0.smithi162.stdout:74: writing smithi16234619-31 from 524385 to 1077551 tid 1 2024-09-16T15:01:27.095 INFO:tasks.rados.rados.0.smithi162.stdout:74: writing smithi16234619-31 from 1602039 to 2012054 tid 2 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:74: writing smithi16234619-31 from 2577441 to 2924385 tid 3 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:75: read oid 2 snap -1 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:75: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:76: read oid 35 snap -1 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:76: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:77: write oid 23 current snap is 1 2024-09-16T15:01:27.101 INFO:tasks.rados.rados.0.smithi162.stdout:77: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2024-09-16T15:01:27.108 INFO:tasks.rados.rados.0.smithi162.stdout:77: writing smithi16234619-23 from 541192 to 1169607 tid 1 2024-09-16T15:01:27.114 INFO:tasks.rados.rados.0.smithi162.stdout:77: writing smithi16234619-23 from 1660521 to 2214194 tid 2 2024-09-16T15:01:27.118 INFO:tasks.rados.rados.0.smithi162.stdout:77: writing smithi16234619-23 from 2888511 to 2941192 tid 3 2024-09-16T15:01:27.118 INFO:tasks.rados.rados.0.smithi162.stdout:78: delete oid 29 current snap is 1 2024-09-16T15:01:27.118 INFO:tasks.rados.rados.0.smithi162.stdout:71: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T15:01:27.198 INFO:tasks.rados.rados.0.smithi162.stdout:67: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T15:01:27.275 INFO:tasks.rados.rados.0.smithi162.stdout:68: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T15:01:27.360 INFO:tasks.rados.rados.0.smithi162.stdout:66: finishing write tid 1 to smithi16234619-34 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: finishing write tid 2 to smithi16234619-34 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: finishing write tid 3 to smithi16234619-34 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: finishing write tid 4 to smithi16234619-34 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: finishing write tid 5 to smithi16234619-34 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: oid 34 updating version 0 to 19 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: oid 34 updating version 19 to 22 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: oid 34 version 22 is already newer than 20 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: oid 34 version 22 is already newer than 21 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 22 (ObjNum 52 snap 1 seq_num 52) dirty exists 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:66: left oid 34 (ObjNum 52 snap 1 seq_num 52) 2024-09-16T15:01:27.361 INFO:tasks.rados.rados.0.smithi162.stdout:65: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:72: finishing write tid 1 to smithi16234619-44 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:66: done (12 left) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:67: done (11 left) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:68: done (10 left) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:71: done (9 left) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:78: done (8 left) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:79: read oid 24 snap -1 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:79: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:80: snap_create 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:64: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:01:27.362 INFO:tasks.rados.rados.0.smithi162.stdout:75: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:01:27.425 INFO:tasks.rados.rados.0.smithi162.stdout:76: expect (ObjNum 34 snap 0 seq_num 34) 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: finishing write tid 2 to smithi16234619-44 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: finishing write tid 3 to smithi16234619-44 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: finishing write tid 4 to smithi16234619-44 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: oid 44 updating version 0 to 17 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: oid 44 version 17 is already newer than 16 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: oid 44 updating version 17 to 18 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 18 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:72: left oid 44 (ObjNum 53 snap 1 seq_num 53) 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:64: finishing copy_from to smithi16234619-37 2024-09-16T15:01:27.506 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 9 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: finishing write tid 1 to smithi16234619-48 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:65: finishing copy_from to smithi16234619-3 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 13 (ObjNum 50 snap 1 seq_num 50) dirty exists 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: finishing write tid 2 to smithi16234619-48 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: finishing write tid 3 to smithi16234619-48 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: finishing write tid 4 to smithi16234619-48 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: finishing write tid 5 to smithi16234619-48 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: oid 48 updating version 0 to 16 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: oid 48 updating version 16 to 18 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: oid 48 updating version 18 to 19 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:73: oid 48 version 19 is already newer than 17 2024-09-16T15:01:27.507 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 19 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:73: left oid 48 (ObjNum 54 snap 1 seq_num 54) 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: finishing write tid 1 to smithi16234619-31 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: finishing write tid 2 to smithi16234619-31 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: finishing write tid 3 to smithi16234619-31 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: finishing write tid 4 to smithi16234619-31 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: finishing write tid 5 to smithi16234619-31 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: oid 31 updating version 0 to 14 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: oid 31 updating version 14 to 15 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: oid 31 updating version 15 to 17 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:74: oid 31 version 17 is already newer than 16 2024-09-16T15:01:27.508 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 17 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:74: left oid 31 (ObjNum 55 snap 1 seq_num 55) 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: finishing write tid 1 to smithi16234619-23 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: finishing write tid 2 to smithi16234619-23 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: finishing write tid 3 to smithi16234619-23 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: finishing write tid 4 to smithi16234619-23 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: finishing write tid 5 to smithi16234619-23 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: oid 23 updating version 0 to 9 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: oid 23 version 9 is already newer than 7 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: oid 23 updating version 9 to 10 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: oid 23 version 10 is already newer than 8 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 10 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-09-16T15:01:27.509 INFO:tasks.rados.rados.0.smithi162.stdout:77: left oid 23 (ObjNum 56 snap 1 seq_num 56) 2024-09-16T15:01:27.510 INFO:tasks.rados.rados.0.smithi162.stdout:79: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:64: done (9 left) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:65: done (8 left) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:72: done (7 left) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:73: done (6 left) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:74: done (5 left) 2024-09-16T15:01:27.697 INFO:tasks.rados.rados.0.smithi162.stdout:75: done (4 left) 2024-09-16T15:01:27.698 INFO:tasks.rados.rados.0.smithi162.stdout:76: done (3 left) 2024-09-16T15:01:27.698 INFO:tasks.rados.rados.0.smithi162.stdout:77: done (2 left) 2024-09-16T15:01:27.698 INFO:tasks.rados.rados.0.smithi162.stdout:79: done (1 left) 2024-09-16T15:01:27.698 INFO:tasks.rados.rados.0.smithi162.stdout:80: done (0 left) 2024-09-16T15:01:27.698 INFO:tasks.rados.rados.0.smithi162.stdout:81: snap_remove snap 1 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:81: done (0 left) 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:82: read oid 23 snap -1 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:82: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:83: read oid 27 snap -1 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:83: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-16T15:01:28.737 INFO:tasks.rados.rados.0.smithi162.stdout:84: write (excl) oid 22 current snap is 2 2024-09-16T15:01:28.738 INFO:tasks.rados.rados.0.smithi162.stdout:84: seq_num 58 ranges {574806=778913,2061132=557342} 2024-09-16T15:01:28.744 INFO:tasks.rados.rados.0.smithi162.stdout:84: writing smithi16234619-22 from 574806 to 1353719 tid 1 2024-09-16T15:01:28.751 INFO:tasks.rados.rados.0.smithi162.stdout:84: writing smithi16234619-22 from 2061132 to 2618474 tid 2 2024-09-16T15:01:28.751 INFO:tasks.rados.rados.0.smithi162.stdout:85: snap_create 2024-09-16T15:01:28.751 INFO:tasks.rados.rados.0.smithi162.stdout:83: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-16T15:01:28.827 INFO:tasks.rados.rados.0.smithi162.stdout:82: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: finishing write tid 1 to smithi16234619-22 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: finishing write tid 2 to smithi16234619-22 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: finishing write tid 3 to smithi16234619-22 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: finishing write tid 4 to smithi16234619-22 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: oid 22 updating version 0 to 11 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: oid 22 updating version 11 to 12 2024-09-16T15:01:28.997 INFO:tasks.rados.rados.0.smithi162.stdout:84: oid 22 version 12 is already newer than 10 2024-09-16T15:01:28.998 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 12 (ObjNum 58 snap 2 seq_num 58) dirty exists 2024-09-16T15:01:28.998 INFO:tasks.rados.rados.0.smithi162.stdout:84: left oid 22 (ObjNum 58 snap 2 seq_num 58) 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:82: done (3 left) 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:83: done (2 left) 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:84: done (1 left) 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:85: done (0 left) 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:86: write oid 31 current snap is 3 2024-09-16T15:01:29.738 INFO:tasks.rados.rados.0.smithi162.stdout:86: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2024-09-16T15:01:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:86: writing smithi16234619-31 from 591613 to 1045775 tid 1 2024-09-16T15:01:29.749 INFO:tasks.rados.rados.0.smithi162.stdout:86: writing smithi16234619-31 from 1719614 to 2136967 tid 2 2024-09-16T15:01:29.754 INFO:tasks.rados.rados.0.smithi162.stdout:86: writing smithi16234619-31 from 2570780 to 2991613 tid 3 2024-09-16T15:01:29.754 INFO:tasks.rados.rados.0.smithi162.stdout:87: snap_remove snap 2 2024-09-16T15:01:30.750 INFO:tasks.rados.rados.0.smithi162.stdout:87: done (1 left) 2024-09-16T15:01:30.751 INFO:tasks.rados.rados.0.smithi162.stdout:88: write (excl) oid 15 current snap is 3 2024-09-16T15:01:30.751 INFO:tasks.rados.rados.0.smithi162.stdout:86: finishing write tid 1 to smithi16234619-31 2024-09-16T15:01:30.751 INFO:tasks.rados.rados.0.smithi162.stdout:88: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2024-09-16T15:01:30.756 INFO:tasks.rados.rados.0.smithi162.stdout:88: writing smithi16234619-15 from 608420 to 1137831 tid 1 2024-09-16T15:01:30.763 INFO:tasks.rados.rados.0.smithi162.stdout:88: writing smithi16234619-15 from 1778096 to 2339107 tid 2 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:88: writing smithi16234619-15 from 2881850 to 3008420 tid 3 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:89: read oid 47 snap -1 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:89: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:90: copy_from oid 10 from oid 27 current snap is 3 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:86: finishing write tid 2 to smithi16234619-31 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:86: finishing write tid 3 to smithi16234619-31 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:86: finishing write tid 4 to smithi16234619-31 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:86: finishing write tid 5 to smithi16234619-31 2024-09-16T15:01:30.768 INFO:tasks.rados.rados.0.smithi162.stdout:86: oid 31 updating version 0 to 20 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:86: oid 31 updating version 20 to 21 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:86: oid 31 version 21 is already newer than 19 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:86: oid 31 updating version 21 to 22 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 22 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:86: left oid 31 (ObjNum 59 snap 3 seq_num 59) 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:86: done (3 left) 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:91: rollback oid 13 current snap is 3 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:91: done (3 left) 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:92: read oid 6 snap -1 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:92: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:93: rollback oid 42 current snap is 3 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:93: done (4 left) 2024-09-16T15:01:30.769 INFO:tasks.rados.rados.0.smithi162.stdout:94: copy_from oid 7 from oid 41 current snap is 3 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:95: read oid 32 snap -1 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:95: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:96: read oid 45 snap -1 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:97: write (excl) oid 22 current snap is 3 2024-09-16T15:01:30.770 INFO:tasks.rados.rados.0.smithi162.stdout:97: seq_num 61 ranges {625227=721013,1952931=704669} 2024-09-16T15:01:30.776 INFO:tasks.rados.rados.0.smithi162.stdout:97: writing smithi16234619-22 from 625227 to 1346240 tid 1 2024-09-16T15:01:30.783 INFO:tasks.rados.rados.0.smithi162.stdout:97: writing smithi16234619-22 from 1952931 to 2657600 tid 2 2024-09-16T15:01:30.783 INFO:tasks.rados.rados.0.smithi162.stdout:98: write oid 19 current snap is 3 2024-09-16T15:01:30.783 INFO:tasks.rados.rados.0.smithi162.stdout:98: seq_num 62 ranges {642034=796262,1895060=564680} 2024-09-16T15:01:30.791 INFO:tasks.rados.rados.0.smithi162.stdout:98: writing smithi16234619-19 from 642034 to 1438296 tid 1 2024-09-16T15:01:30.796 INFO:tasks.rados.rados.0.smithi162.stdout:98: writing smithi16234619-19 from 1895060 to 2459740 tid 2 2024-09-16T15:01:30.797 INFO:tasks.rados.rados.0.smithi162.stdout:99: copy_from oid 31 from oid 33 current snap is 3 2024-09-16T15:01:30.797 INFO:tasks.rados.rados.0.smithi162.stdout:95: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T15:01:30.878 INFO:tasks.rados.rados.0.smithi162.stdout:89: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-16T15:01:30.969 INFO:tasks.rados.rados.0.smithi162.stdout:96: expect (ObjNum 44 snap 0 seq_num 44) 2024-09-16T15:01:31.054 INFO:tasks.rados.rados.0.smithi162.stdout:92: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-16T15:01:31.124 INFO:tasks.rados.rados.0.smithi162.stdout:90: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:01:31.124 INFO:tasks.rados.rados.0.smithi162.stdout:88: finishing write tid 1 to smithi16234619-15 2024-09-16T15:01:31.124 INFO:tasks.rados.rados.0.smithi162.stdout:88: finishing write tid 2 to smithi16234619-15 2024-09-16T15:01:31.124 INFO:tasks.rados.rados.0.smithi162.stdout:88: finishing write tid 3 to smithi16234619-15 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:94: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: finishing write tid 4 to smithi16234619-15 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: finishing write tid 5 to smithi16234619-15 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: oid 15 updating version 0 to 12 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: oid 15 version 12 is already newer than 11 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: oid 15 updating version 12 to 13 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: oid 15 version 13 is already newer than 10 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 13 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:88: left oid 15 (ObjNum 60 snap 3 seq_num 60) 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:90: finishing copy_from to smithi16234619-10 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:94: finishing copy_from to smithi16234619-7 2024-09-16T15:01:31.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:97: finishing write tid 1 to smithi16234619-22 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:88: done (9 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:89: done (8 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:90: done (7 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:92: done (6 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:94: done (5 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:95: done (4 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:96: done (3 left) 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:100: delete oid 14 current snap is 3 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:97: finishing write tid 2 to smithi16234619-22 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:97: finishing write tid 3 to smithi16234619-22 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:97: finishing write tid 4 to smithi16234619-22 2024-09-16T15:01:31.126 INFO:tasks.rados.rados.0.smithi162.stdout:97: oid 22 updating version 0 to 14 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:97: oid 22 updating version 14 to 15 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:97: oid 22 version 15 is already newer than 13 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 15 (ObjNum 61 snap 3 seq_num 61) dirty exists 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:97: left oid 22 (ObjNum 61 snap 3 seq_num 61) 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: finishing write tid 1 to smithi16234619-19 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: finishing write tid 2 to smithi16234619-19 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: finishing write tid 3 to smithi16234619-19 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: finishing write tid 4 to smithi16234619-19 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: oid 19 updating version 0 to 24 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: oid 19 updating version 24 to 25 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: oid 19 version 25 is already newer than 23 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 25 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:98: left oid 19 (ObjNum 62 snap 3 seq_num 62) 2024-09-16T15:01:31.127 INFO:tasks.rados.rados.0.smithi162.stdout:99: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:99: finishing copy_from to smithi16234619-31 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:97: done (3 left) 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:98: done (2 left) 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:99: done (1 left) 2024-09-16T15:01:31.128 INFO:tasks.rados.rados.0.smithi162.stdout:100: done (0 left) 2024-09-16T15:01:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:101: write (excl) oid 10 current snap is 3 2024-09-16T15:01:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:101: seq_num 64 ranges {675648=546760,2012024=568349} 2024-09-16T15:01:31.131 INFO:tasks.rados.rados.0.smithi162.stdout:101: writing smithi16234619-10 from 675648 to 1222408 tid 1 2024-09-16T15:01:31.138 INFO:tasks.rados.rados.0.smithi162.stdout:101: writing smithi16234619-10 from 2012024 to 2580373 tid 2 2024-09-16T15:01:31.139 INFO:tasks.rados.rados.0.smithi162.stdout:102: snap_create 2024-09-16T15:01:31.156 INFO:tasks.rados.rados.0.smithi162.stdout:101: finishing write tid 1 to smithi16234619-10 2024-09-16T15:01:31.156 INFO:tasks.rados.rados.0.smithi162.stdout:101: finishing write tid 2 to smithi16234619-10 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:101: finishing write tid 3 to smithi16234619-10 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:101: finishing write tid 4 to smithi16234619-10 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:101: oid 10 updating version 0 to 24 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:101: oid 10 version 24 is already newer than 23 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:101: oid 10 updating version 24 to 25 2024-09-16T15:01:31.158 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 25 (ObjNum 64 snap 3 seq_num 64) dirty exists 2024-09-16T15:01:31.159 INFO:tasks.rados.rados.0.smithi162.stdout:101: left oid 10 (ObjNum 64 snap 3 seq_num 64) 2024-09-16T15:01:31.760 INFO:tasks.rados.rados.0.smithi162.stdout:101: done (1 left) 2024-09-16T15:01:31.760 INFO:tasks.rados.rados.0.smithi162.stdout:102: done (0 left) 2024-09-16T15:01:31.760 INFO:tasks.rados.rados.0.smithi162.stdout:103: read oid 6 snap 3 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:103: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:104: read oid 20 snap 3 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:104: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:105: copy_from oid 33 from oid 7 current snap is 4 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:106: write oid 17 current snap is 4 2024-09-16T15:01:31.761 INFO:tasks.rados.rados.0.smithi162.stdout:106: seq_num 65 ranges {692455=622009,2070506=712007} 2024-09-16T15:01:31.769 INFO:tasks.rados.rados.0.smithi162.stdout:106: writing smithi16234619-17 from 692455 to 1314464 tid 1 2024-09-16T15:01:31.776 INFO:tasks.rados.rados.0.smithi162.stdout:106: writing smithi16234619-17 from 2070506 to 2782513 tid 2 2024-09-16T15:01:31.776 INFO:tasks.rados.rados.0.smithi162.stdout:107: snap_create 2024-09-16T15:01:31.776 INFO:tasks.rados.rados.0.smithi162.stdout:103: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-16T15:01:31.847 INFO:tasks.rados.rados.0.smithi162.stdout:104: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-16T15:01:31.923 INFO:tasks.rados.rados.0.smithi162.stdout:106: finishing write tid 1 to smithi16234619-17 2024-09-16T15:01:31.923 INFO:tasks.rados.rados.0.smithi162.stdout:106: finishing write tid 2 to smithi16234619-17 2024-09-16T15:01:31.924 INFO:tasks.rados.rados.0.smithi162.stdout:106: finishing write tid 3 to smithi16234619-17 2024-09-16T15:01:31.924 INFO:tasks.rados.rados.0.smithi162.stdout:106: finishing write tid 4 to smithi16234619-17 2024-09-16T15:01:31.924 INFO:tasks.rados.rados.0.smithi162.stdout:106: oid 17 updating version 0 to 27 2024-09-16T15:01:31.924 INFO:tasks.rados.rados.0.smithi162.stdout:106: oid 17 updating version 27 to 29 2024-09-16T15:01:31.924 INFO:tasks.rados.rados.0.smithi162.stdout:106: oid 17 version 29 is already newer than 28 2024-09-16T15:01:31.925 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 29 (ObjNum 65 snap 4 seq_num 65) dirty exists 2024-09-16T15:01:31.925 INFO:tasks.rados.rados.0.smithi162.stdout:106: left oid 17 (ObjNum 65 snap 4 seq_num 65) 2024-09-16T15:01:31.925 INFO:tasks.rados.rados.0.smithi162.stdout:105: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:01:31.925 INFO:tasks.rados.rados.0.smithi162.stdout:105: finishing copy_from to smithi16234619-33 2024-09-16T15:01:31.925 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:103: done (4 left) 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:104: done (3 left) 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:105: done (2 left) 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:106: done (1 left) 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:107: done (0 left) 2024-09-16T15:01:33.374 INFO:tasks.rados.rados.0.smithi162.stdout:108: snap_remove snap 4 2024-09-16T15:01:34.381 INFO:tasks.rados.rados.0.smithi162.stdout:108: done (0 left) 2024-09-16T15:01:34.381 INFO:tasks.rados.rados.0.smithi162.stdout:109: read oid 22 snap -1 2024-09-16T15:01:34.381 INFO:tasks.rados.rados.0.smithi162.stdout:109: expect (ObjNum 61 snap 3 seq_num 61) 2024-09-16T15:01:34.381 INFO:tasks.rados.rados.0.smithi162.stdout:110: snap_remove snap 3 2024-09-16T15:01:35.352 INFO:tasks.rados.rados.0.smithi162.stdout:110: done (1 left) 2024-09-16T15:01:35.352 INFO:tasks.rados.rados.0.smithi162.stdout:111: snap_create 2024-09-16T15:01:35.352 INFO:tasks.rados.rados.0.smithi162.stdout:109: expect (ObjNum 61 snap 3 seq_num 61) 2024-09-16T15:01:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:109: done (1 left) 2024-09-16T15:01:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:111: done (0 left) 2024-09-16T15:01:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:112: snap_remove snap 5 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:112: done (0 left) 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:113: copy_from oid 43 from oid 32 current snap is 6 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:114: copy_from oid 36 from oid 17 current snap is 6 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:115: copy_from oid 27 from oid 22 current snap is 6 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:116: write (excl) oid 37 current snap is 6 2024-09-16T15:01:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:116: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2024-09-16T15:01:37.362 INFO:tasks.rados.rados.0.smithi162.stdout:116: writing smithi16234619-37 from 709262 to 1406520 tid 1 2024-09-16T15:01:37.370 INFO:tasks.rados.rados.0.smithi162.stdout:116: writing smithi16234619-37 from 2012635 to 2584653 tid 2 2024-09-16T15:01:37.373 INFO:tasks.rados.rados.0.smithi162.stdout:116: writing smithi16234619-37 from 3046388 to 3109262 tid 3 2024-09-16T15:01:37.373 INFO:tasks.rados.rados.0.smithi162.stdout:117: snap_create 2024-09-16T15:01:37.405 INFO:tasks.rados.rados.0.smithi162.stdout:113: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:01:37.405 INFO:tasks.rados.rados.0.smithi162.stdout:113: finishing copy_from to smithi16234619-43 2024-09-16T15:01:37.405 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T15:01:37.406 INFO:tasks.rados.rados.0.smithi162.stdout:114: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:01:37.412 INFO:tasks.rados.rados.0.smithi162.stdout:115: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:01:37.413 INFO:tasks.rados.rados.0.smithi162.stdout:115: finishing copy_from to smithi16234619-27 2024-09-16T15:01:37.413 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 15 (ObjNum 61 snap 3 seq_num 61) dirty exists 2024-09-16T15:01:37.413 INFO:tasks.rados.rados.0.smithi162.stdout:116: finishing write tid 1 to smithi16234619-37 2024-09-16T15:01:37.413 INFO:tasks.rados.rados.0.smithi162.stdout:116: finishing write tid 2 to smithi16234619-37 2024-09-16T15:01:37.414 INFO:tasks.rados.rados.0.smithi162.stdout:116: finishing write tid 3 to smithi16234619-37 2024-09-16T15:01:37.414 INFO:tasks.rados.rados.0.smithi162.stdout:116: finishing write tid 4 to smithi16234619-37 2024-09-16T15:01:37.414 INFO:tasks.rados.rados.0.smithi162.stdout:116: finishing write tid 5 to smithi16234619-37 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:116: oid 37 updating version 0 to 24 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:116: oid 37 version 24 is already newer than 21 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:116: oid 37 version 24 is already newer than 22 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:116: oid 37 version 24 is already newer than 23 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 24 (ObjNum 66 snap 6 seq_num 66) dirty exists 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:116: left oid 37 (ObjNum 66 snap 6 seq_num 66) 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:114: finishing copy_from to smithi16234619-36 2024-09-16T15:01:37.415 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 29 (ObjNum 65 snap 4 seq_num 65) dirty exists 2024-09-16T15:01:38.368 INFO:tasks.rados.rados.0.smithi162.stdout:113: done (4 left) 2024-09-16T15:01:38.368 INFO:tasks.rados.rados.0.smithi162.stdout:114: done (3 left) 2024-09-16T15:01:38.368 INFO:tasks.rados.rados.0.smithi162.stdout:115: done (2 left) 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:116: done (1 left) 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:117: done (0 left) 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:118: read oid 7 snap -1 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:118: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:119: read oid 4 snap -1 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:119: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T15:01:38.369 INFO:tasks.rados.rados.0.smithi162.stdout:120: snap_remove snap 6 2024-09-16T15:01:39.372 INFO:tasks.rados.rados.0.smithi162.stdout:120: done (2 left) 2024-09-16T15:01:39.372 INFO:tasks.rados.rados.0.smithi162.stdout:121: snap_create 2024-09-16T15:01:39.372 INFO:tasks.rados.rados.0.smithi162.stdout:119: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T15:01:39.438 INFO:tasks.rados.rados.0.smithi162.stdout:118: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:40.356 INFO:tasks.rados.rados.0.smithi162.stdout:118: done (2 left) 2024-09-16T15:01:40.357 INFO:tasks.rados.rados.0.smithi162.stdout:119: done (1 left) 2024-09-16T15:01:40.357 INFO:tasks.rados.rados.0.smithi162.stdout:121: done (0 left) 2024-09-16T15:01:40.357 INFO:tasks.rados.rados.0.smithi162.stdout:122: snap_create 2024-09-16T15:01:41.363 INFO:tasks.rados.rados.0.smithi162.stdout:122: done (0 left) 2024-09-16T15:01:41.364 INFO:tasks.rados.rados.0.smithi162.stdout:123: rollback oid 22 current snap is 9 2024-09-16T15:01:41.364 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 22 to 7 2024-09-16T15:01:41.364 INFO:tasks.rados.rados.0.smithi162.stdout:124: read oid 14 snap -1 2024-09-16T15:01:41.364 INFO:tasks.rados.rados.0.smithi162.stdout:124: expect deleted 2024-09-16T15:01:41.364 INFO:tasks.rados.rados.0.smithi162.stdout:125: snap_create 2024-09-16T15:01:41.369 INFO:tasks.rados.rados.0.smithi162.stdout:123: finishing rollback tid 0 to smithi16234619-22 2024-09-16T15:01:41.369 INFO:tasks.rados.rados.0.smithi162.stdout:123: finishing rollback tid 1 to smithi16234619-22 2024-09-16T15:01:41.372 INFO:tasks.rados.rados.0.smithi162.stdout:123: finishing rollback tid 2 to smithi16234619-22 2024-09-16T15:01:41.372 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 19 (ObjNum 61 snap 3 seq_num 61) dirty exists 2024-09-16T15:01:42.369 INFO:tasks.rados.rados.0.smithi162.stdout:123: done (2 left) 2024-09-16T15:01:42.369 INFO:tasks.rados.rados.0.smithi162.stdout:124: done (1 left) 2024-09-16T15:01:42.369 INFO:tasks.rados.rados.0.smithi162.stdout:125: done (0 left) 2024-09-16T15:01:42.369 INFO:tasks.rados.rados.0.smithi162.stdout:126: read oid 33 snap -1 2024-09-16T15:01:42.369 INFO:tasks.rados.rados.0.smithi162.stdout:126: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:42.370 INFO:tasks.rados.rados.0.smithi162.stdout:127: snap_remove snap 9 2024-09-16T15:01:43.368 INFO:tasks.rados.rados.0.smithi162.stdout:127: done (1 left) 2024-09-16T15:01:43.368 INFO:tasks.rados.rados.0.smithi162.stdout:128: snap_remove snap 8 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:128: done (1 left) 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:129: read oid 40 snap -1 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:129: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:130: read oid 19 snap -1 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:130: expect (ObjNum 62 snap 3 seq_num 62) 2024-09-16T15:01:44.387 INFO:tasks.rados.rados.0.smithi162.stdout:131: copy_from oid 17 from oid 41 current snap is 10 2024-09-16T15:01:44.388 INFO:tasks.rados.rados.0.smithi162.stdout:126: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:44.476 INFO:tasks.rados.rados.0.smithi162.stdout:129: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T15:01:44.557 INFO:tasks.rados.rados.0.smithi162.stdout:130: expect (ObjNum 62 snap 3 seq_num 62) 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:131: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:131: finishing copy_from to smithi16234619-17 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:126: done (3 left) 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:129: done (2 left) 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:130: done (1 left) 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:131: done (0 left) 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:132: write oid 29 current snap is 10 2024-09-16T15:01:44.693 INFO:tasks.rados.rados.0.smithi162.stdout:132: seq_num 67 ranges {726069=772507,2071117=715676} 2024-09-16T15:01:44.700 INFO:tasks.rados.rados.0.smithi162.stdout:132: writing smithi16234619-29 from 726069 to 1498576 tid 1 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:132: writing smithi16234619-29 from 2071117 to 2786793 tid 2 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:133: read oid 3 snap -1 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:133: expect (ObjNum 50 snap 1 seq_num 50) 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:134: read oid 7 snap -1 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:134: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:135: read oid 1 snap -1 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:135: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:136: read oid 34 snap -1 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:136: expect (ObjNum 52 snap 1 seq_num 52) 2024-09-16T15:01:44.708 INFO:tasks.rados.rados.0.smithi162.stdout:137: delete oid 49 current snap is 10 2024-09-16T15:01:44.711 INFO:tasks.rados.rados.0.smithi162.stdout:135: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T15:01:44.776 INFO:tasks.rados.rados.0.smithi162.stdout:133: expect (ObjNum 50 snap 1 seq_num 50) 2024-09-16T15:01:44.890 INFO:tasks.rados.rados.0.smithi162.stdout:134: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:44.974 INFO:tasks.rados.rados.0.smithi162.stdout:136: expect (ObjNum 52 snap 1 seq_num 52) 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: finishing write tid 1 to smithi16234619-29 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: finishing write tid 2 to smithi16234619-29 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: finishing write tid 3 to smithi16234619-29 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: finishing write tid 4 to smithi16234619-29 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: oid 29 updating version 0 to 25 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: oid 29 updating version 25 to 26 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: oid 29 updating version 26 to 27 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 27 (ObjNum 67 snap 10 seq_num 67) dirty exists 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: left oid 29 (ObjNum 67 snap 10 seq_num 67) 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:132: done (5 left) 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:133: done (4 left) 2024-09-16T15:01:45.106 INFO:tasks.rados.rados.0.smithi162.stdout:134: done (3 left) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:135: done (2 left) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:136: done (1 left) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:137: done (0 left) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:138: read oid 19 snap 7 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:138: expect (ObjNum 62 snap 3 seq_num 62) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:139: delete oid 17 current snap is 10 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:139: done (1 left) 2024-09-16T15:01:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:140: delete oid 27 current snap is 10 2024-09-16T15:01:45.110 INFO:tasks.rados.rados.0.smithi162.stdout:140: done (1 left) 2024-09-16T15:01:45.110 INFO:tasks.rados.rados.0.smithi162.stdout:141: copy_from oid 11 from oid 12 current snap is 10 2024-09-16T15:01:45.110 INFO:tasks.rados.rados.0.smithi162.stdout:142: write (excl) oid 17 current snap is 10 2024-09-16T15:01:45.110 INFO:tasks.rados.rados.0.smithi162.stdout:142: seq_num 71 ranges {793297=789856,2305045=723014} 2024-09-16T15:01:45.118 INFO:tasks.rados.rados.0.smithi162.stdout:142: writing smithi16234619-17 from 793297 to 1583153 tid 1 2024-09-16T15:01:45.128 INFO:tasks.rados.rados.0.smithi162.stdout:142: writing smithi16234619-17 from 2305045 to 3028059 tid 2 2024-09-16T15:01:45.128 INFO:tasks.rados.rados.0.smithi162.stdout:143: snap_create 2024-09-16T15:01:45.128 INFO:tasks.rados.rados.0.smithi162.stdout:138: expect (ObjNum 62 snap 3 seq_num 62) 2024-09-16T15:01:45.280 INFO:tasks.rados.rados.0.smithi162.stdout:142: finishing write tid 1 to smithi16234619-17 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: finishing write tid 2 to smithi16234619-17 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: finishing write tid 3 to smithi16234619-17 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: finishing write tid 4 to smithi16234619-17 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: oid 17 updating version 0 to 37 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: oid 17 version 37 is already newer than 36 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: oid 17 version 37 is already newer than 35 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 37 (ObjNum 71 snap 10 seq_num 71) dirty exists 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:142: left oid 17 (ObjNum 71 snap 10 seq_num 71) 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:141: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:141: finishing copy_from to smithi16234619-11 2024-09-16T15:01:45.281 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 13 (ObjNum 50 snap 1 seq_num 50) dirty exists 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:138: done (3 left) 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:141: done (2 left) 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:142: done (1 left) 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:143: done (0 left) 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:144: copy_from oid 43 from oid 48 current snap is 11 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:145: write (excl) oid 12 current snap is 11 2024-09-16T15:01:45.358 INFO:tasks.rados.rados.0.smithi162.stdout:145: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2024-09-16T15:01:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:145: writing smithi16234619-12 from 410104 to 875209 tid 1 2024-09-16T15:01:45.369 INFO:tasks.rados.rados.0.smithi162.stdout:145: writing smithi16234619-12 from 1563527 to 2146552 tid 2 2024-09-16T15:01:45.375 INFO:tasks.rados.rados.0.smithi162.stdout:145: writing smithi16234619-12 from 2810926 to 3210104 tid 3 2024-09-16T15:01:45.375 INFO:tasks.rados.rados.0.smithi162.stdout:146: delete oid 34 current snap is 11 2024-09-16T15:01:45.379 INFO:tasks.rados.rados.0.smithi162.stdout:146: done (2 left) 2024-09-16T15:01:45.379 INFO:tasks.rados.rados.0.smithi162.stdout:147: copy_from oid 40 from oid 10 current snap is 11 2024-09-16T15:01:45.379 INFO:tasks.rados.rados.0.smithi162.stdout:148: delete oid 49 current snap is 11 2024-09-16T15:01:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:148: done (3 left) 2024-09-16T15:01:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:149: snap_create 2024-09-16T15:01:45.399 INFO:tasks.rados.rados.0.smithi162.stdout:145: finishing write tid 1 to smithi16234619-12 2024-09-16T15:01:45.399 INFO:tasks.rados.rados.0.smithi162.stdout:145: finishing write tid 2 to smithi16234619-12 2024-09-16T15:01:45.399 INFO:tasks.rados.rados.0.smithi162.stdout:145: finishing write tid 3 to smithi16234619-12 2024-09-16T15:01:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:145: finishing write tid 4 to smithi16234619-12 2024-09-16T15:01:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:145: finishing write tid 5 to smithi16234619-12 2024-09-16T15:01:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:145: oid 12 updating version 0 to 16 2024-09-16T15:01:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:145: oid 12 updating version 16 to 18 2024-09-16T15:01:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:145: oid 12 version 18 is already newer than 17 2024-09-16T15:01:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:145: oid 12 version 18 is already newer than 15 2024-09-16T15:01:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 18 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-09-16T15:01:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:145: left oid 12 (ObjNum 72 snap 11 seq_num 72) 2024-09-16T15:01:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:144: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:01:45.420 INFO:tasks.rados.rados.0.smithi162.stdout:144: finishing copy_from to smithi16234619-43 2024-09-16T15:01:45.420 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 19 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T15:01:45.439 INFO:tasks.rados.rados.0.smithi162.stdout:147: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:01:45.442 INFO:tasks.rados.rados.0.smithi162.stdout:147: finishing copy_from to smithi16234619-40 2024-09-16T15:01:45.443 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 25 (ObjNum 64 snap 3 seq_num 64) dirty exists 2024-09-16T15:01:46.396 INFO:tasks.rados.rados.0.smithi162.stdout:144: done (3 left) 2024-09-16T15:01:46.396 INFO:tasks.rados.rados.0.smithi162.stdout:145: done (2 left) 2024-09-16T15:01:46.396 INFO:tasks.rados.rados.0.smithi162.stdout:147: done (1 left) 2024-09-16T15:01:46.396 INFO:tasks.rados.rados.0.smithi162.stdout:149: done (0 left) 2024-09-16T15:01:46.397 INFO:tasks.rados.rados.0.smithi162.stdout:150: rollback oid 37 current snap is 12 2024-09-16T15:01:46.397 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 10 2024-09-16T15:01:46.397 INFO:tasks.rados.rados.0.smithi162.stdout:151: write (excl) oid 44 current snap is 12 2024-09-16T15:01:46.397 INFO:tasks.rados.rados.0.smithi162.stdout:151: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2024-09-16T15:01:46.402 INFO:tasks.rados.rados.0.smithi162.stdout:151: writing smithi16234619-44 from 460525 to 1151377 tid 1 2024-09-16T15:01:46.408 INFO:tasks.rados.rados.0.smithi162.stdout:151: writing smithi16234619-44 from 1622620 to 2352972 tid 2 2024-09-16T15:01:46.414 INFO:tasks.rados.rados.0.smithi162.stdout:151: writing smithi16234619-44 from 3060489 to 3260525 tid 3 2024-09-16T15:01:46.414 INFO:tasks.rados.rados.0.smithi162.stdout:152: write oid 49 current snap is 12 2024-09-16T15:01:46.414 INFO:tasks.rados.rados.0.smithi162.stdout:152: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2024-09-16T15:01:46.421 INFO:tasks.rados.rados.0.smithi162.stdout:152: writing smithi16234619-49 from 477332 to 1243433 tid 1 2024-09-16T15:01:46.426 INFO:tasks.rados.rados.0.smithi162.stdout:152: writing smithi16234619-49 from 1681102 to 2155112 tid 2 2024-09-16T15:01:46.432 INFO:tasks.rados.rados.0.smithi162.stdout:152: writing smithi16234619-49 from 2687912 to 3277332 tid 3 2024-09-16T15:01:46.432 INFO:tasks.rados.rados.0.smithi162.stdout:153: write oid 30 current snap is 12 2024-09-16T15:01:46.433 INFO:tasks.rados.rados.0.smithi162.stdout:150: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:01:46.433 INFO:tasks.rados.rados.0.smithi162.stdout:150: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:01:46.433 INFO:tasks.rados.rados.0.smithi162.stdout:153: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2024-09-16T15:01:46.438 INFO:tasks.rados.rados.0.smithi162.stdout:153: writing smithi16234619-30 from 494139 to 1051842 tid 1 2024-09-16T15:01:46.445 INFO:tasks.rados.rados.0.smithi162.stdout:153: writing smithi16234619-30 from 1455937 to 2189958 tid 2 2024-09-16T15:01:46.453 INFO:tasks.rados.rados.0.smithi162.stdout:153: writing smithi16234619-30 from 2948041 to 3294139 tid 3 2024-09-16T15:01:46.453 INFO:tasks.rados.rados.0.smithi162.stdout:154: write oid 27 current snap is 12 2024-09-16T15:01:46.453 INFO:tasks.rados.rados.0.smithi162.stdout:150: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:01:46.453 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 31 (ObjNum 66 snap 6 seq_num 66) dirty exists 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: finishing write tid 1 to smithi16234619-44 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: finishing write tid 2 to smithi16234619-44 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: finishing write tid 3 to smithi16234619-44 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: finishing write tid 4 to smithi16234619-44 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: finishing write tid 5 to smithi16234619-44 2024-09-16T15:01:46.454 INFO:tasks.rados.rados.0.smithi162.stdout:151: oid 44 updating version 0 to 22 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:151: oid 44 updating version 22 to 23 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:151: oid 44 version 23 is already newer than 21 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:151: oid 44 version 23 is already newer than 20 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 23 (ObjNum 75 snap 12 seq_num 75) dirty exists 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:151: left oid 44 (ObjNum 75 snap 12 seq_num 75) 2024-09-16T15:01:46.455 INFO:tasks.rados.rados.0.smithi162.stdout:154: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2024-09-16T15:01:46.457 INFO:tasks.rados.rados.0.smithi162.stdout:154: writing smithi16234619-27 from 510946 to 1143898 tid 1 2024-09-16T15:01:46.463 INFO:tasks.rados.rados.0.smithi162.stdout:154: writing smithi16234619-27 from 1798066 to 2275745 tid 2 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:154: writing smithi16234619-27 from 2742758 to 3199975 tid 3 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:150: done (4 left) 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:151: done (3 left) 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:155: rollback oid 13 current snap is 12 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 7 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:156: write (excl) oid 37 current snap is 12 2024-09-16T15:01:46.469 INFO:tasks.rados.rados.0.smithi162.stdout:156: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2024-09-16T15:01:46.475 INFO:tasks.rados.rados.0.smithi162.stdout:156: writing smithi16234619-37 from 527753 to 1235954 tid 1 2024-09-16T15:01:46.483 INFO:tasks.rados.rados.0.smithi162.stdout:156: writing smithi16234619-37 from 1856548 to 2594238 tid 2 2024-09-16T15:01:46.487 INFO:tasks.rados.rados.0.smithi162.stdout:156: writing smithi16234619-37 from 3286534 to 3327753 tid 3 2024-09-16T15:01:46.487 INFO:tasks.rados.rados.0.smithi162.stdout:157: delete oid 39 current snap is 12 2024-09-16T15:01:46.487 INFO:tasks.rados.rados.0.smithi162.stdout:155: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:155: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:155: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 35 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: finishing write tid 1 to smithi16234619-49 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: finishing write tid 2 to smithi16234619-49 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: finishing write tid 3 to smithi16234619-49 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: finishing write tid 4 to smithi16234619-49 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: finishing write tid 5 to smithi16234619-49 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: oid 49 updating version 0 to 25 2024-09-16T15:01:46.488 INFO:tasks.rados.rados.0.smithi162.stdout:152: oid 49 updating version 25 to 28 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:152: oid 49 version 28 is already newer than 26 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:152: oid 49 version 28 is already newer than 27 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 28 (ObjNum 76 snap 12 seq_num 76) dirty exists 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:152: left oid 49 (ObjNum 76 snap 12 seq_num 76) 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: finishing write tid 1 to smithi16234619-30 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: finishing write tid 2 to smithi16234619-30 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: finishing write tid 3 to smithi16234619-30 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: finishing write tid 4 to smithi16234619-30 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: finishing write tid 5 to smithi16234619-30 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: oid 30 updating version 0 to 16 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: oid 30 updating version 16 to 17 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: oid 30 updating version 17 to 18 2024-09-16T15:01:46.489 INFO:tasks.rados.rados.0.smithi162.stdout:153: oid 30 version 18 is already newer than 15 2024-09-16T15:01:46.490 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 18 (ObjNum 77 snap 12 seq_num 77) dirty exists 2024-09-16T15:01:46.490 INFO:tasks.rados.rados.0.smithi162.stdout:153: left oid 30 (ObjNum 77 snap 12 seq_num 77) 2024-09-16T15:01:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:154: finishing write tid 1 to smithi16234619-27 2024-09-16T15:01:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:154: finishing write tid 2 to smithi16234619-27 2024-09-16T15:01:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:154: finishing write tid 3 to smithi16234619-27 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: finishing write tid 4 to smithi16234619-27 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: finishing write tid 5 to smithi16234619-27 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: oid 27 updating version 0 to 24 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: oid 27 updating version 24 to 26 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: oid 27 version 26 is already newer than 23 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: oid 27 version 26 is already newer than 25 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 26 (ObjNum 78 snap 12 seq_num 78) dirty exists 2024-09-16T15:01:46.500 INFO:tasks.rados.rados.0.smithi162.stdout:154: left oid 27 (ObjNum 78 snap 12 seq_num 78) 2024-09-16T15:01:46.512 INFO:tasks.rados.rados.0.smithi162.stdout:156: finishing write tid 1 to smithi16234619-37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: finishing write tid 2 to smithi16234619-37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: finishing write tid 3 to smithi16234619-37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: finishing write tid 4 to smithi16234619-37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: finishing write tid 5 to smithi16234619-37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: oid 37 updating version 0 to 36 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: oid 37 updating version 36 to 39 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: oid 37 version 39 is already newer than 38 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: oid 37 version 39 is already newer than 37 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 39 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:156: left oid 37 (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:152: done (5 left) 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:153: done (4 left) 2024-09-16T15:01:46.517 INFO:tasks.rados.rados.0.smithi162.stdout:154: done (3 left) 2024-09-16T15:01:46.518 INFO:tasks.rados.rados.0.smithi162.stdout:155: done (2 left) 2024-09-16T15:01:46.518 INFO:tasks.rados.rados.0.smithi162.stdout:156: done (1 left) 2024-09-16T15:01:46.518 INFO:tasks.rados.rados.0.smithi162.stdout:157: done (0 left) 2024-09-16T15:01:46.518 INFO:tasks.rados.rados.0.smithi162.stdout:158: write (excl) oid 6 current snap is 12 2024-09-16T15:01:46.518 INFO:tasks.rados.rados.0.smithi162.stdout:158: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2024-09-16T15:01:46.522 INFO:tasks.rados.rados.0.smithi162.stdout:158: writing smithi16234619-6 from 561367 to 1020066 tid 1 2024-09-16T15:01:46.529 INFO:tasks.rados.rados.0.smithi162.stdout:158: writing smithi16234619-6 from 1573512 to 2314871 tid 2 2024-09-16T15:01:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:158: writing smithi16234619-6 from 2941380 to 3361367 tid 3 2024-09-16T15:01:46.535 INFO:tasks.rados.rados.0.smithi162.stdout:159: snap_remove snap 11 2024-09-16T15:01:47.405 INFO:tasks.rados.rados.0.smithi162.stdout:159: done (1 left) 2024-09-16T15:01:47.405 INFO:tasks.rados.rados.0.smithi162.stdout:160: write (excl) oid 34 current snap is 12 2024-09-16T15:01:47.405 INFO:tasks.rados.rados.0.smithi162.stdout:160: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2024-09-16T15:01:47.411 INFO:tasks.rados.rados.0.smithi162.stdout:160: writing smithi16234619-34 from 578174 to 1112122 tid 1 2024-09-16T15:01:47.416 INFO:tasks.rados.rados.0.smithi162.stdout:160: writing smithi16234619-34 from 1515641 to 2000658 tid 2 2024-09-16T15:01:47.423 INFO:tasks.rados.rados.0.smithi162.stdout:160: writing smithi16234619-34 from 2736097 to 3238402 tid 3 2024-09-16T15:01:47.423 INFO:tasks.rados.rados.0.smithi162.stdout:161: read oid 37 snap 10 2024-09-16T15:01:47.423 INFO:tasks.rados.rados.0.smithi162.stdout:161: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-16T15:01:47.423 INFO:tasks.rados.rados.0.smithi162.stdout:162: write oid 9 current snap is 12 2024-09-16T15:01:47.423 INFO:tasks.rados.rados.0.smithi162.stdout:158: finishing write tid 1 to smithi16234619-6 2024-09-16T15:01:47.424 INFO:tasks.rados.rados.0.smithi162.stdout:162: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2024-09-16T15:01:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:162: writing smithi16234619-9 from 594981 to 1204178 tid 1 2024-09-16T15:01:47.438 INFO:tasks.rados.rados.0.smithi162.stdout:162: writing smithi16234619-9 from 1974123 to 2719151 tid 2 2024-09-16T15:01:47.442 INFO:tasks.rados.rados.0.smithi162.stdout:162: writing smithi16234619-9 from 3279873 to 3394981 tid 3 2024-09-16T15:01:47.442 INFO:tasks.rados.rados.0.smithi162.stdout:163: snap_remove snap 7 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:163: done (4 left) 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:164: read oid 36 snap 10 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:164: expect (ObjNum 65 snap 4 seq_num 65) 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:165: rollback oid 17 current snap is 12 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 10 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:158: finishing write tid 2 to smithi16234619-6 2024-09-16T15:01:49.380 INFO:tasks.rados.rados.0.smithi162.stdout:158: finishing write tid 3 to smithi16234619-6 2024-09-16T15:01:49.381 INFO:tasks.rados.rados.0.smithi162.stdout:158: finishing write tid 4 to smithi16234619-6 2024-09-16T15:01:49.381 INFO:tasks.rados.rados.0.smithi162.stdout:158: finishing write tid 5 to smithi16234619-6 2024-09-16T15:01:49.381 INFO:tasks.rados.rados.0.smithi162.stdout:158: oid 6 updating version 0 to 28 2024-09-16T15:01:49.383 INFO:tasks.rados.rados.0.smithi162.stdout:158: oid 6 version 28 is already newer than 25 2024-09-16T15:01:49.384 INFO:tasks.rados.rados.0.smithi162.stdout:158: oid 6 version 28 is already newer than 27 2024-09-16T15:01:49.384 INFO:tasks.rados.rados.0.smithi162.stdout:158: oid 6 version 28 is already newer than 26 2024-09-16T15:01:49.384 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 28 (ObjNum 81 snap 12 seq_num 81) dirty exists 2024-09-16T15:01:49.384 INFO:tasks.rados.rados.0.smithi162.stdout:158: left oid 6 (ObjNum 81 snap 12 seq_num 81) 2024-09-16T15:01:49.384 INFO:tasks.rados.rados.0.smithi162.stdout:161: expect (ObjNum 66 snap 6 seq_num 66) 2024-09-16T15:01:49.512 INFO:tasks.rados.rados.0.smithi162.stdout:160: finishing write tid 1 to smithi16234619-34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: finishing write tid 2 to smithi16234619-34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: finishing write tid 3 to smithi16234619-34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:162: finishing write tid 1 to smithi16234619-9 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:162: finishing write tid 2 to smithi16234619-9 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: finishing write tid 4 to smithi16234619-34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: finishing write tid 5 to smithi16234619-34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: oid 34 updating version 0 to 32 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: oid 34 version 32 is already newer than 31 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: oid 34 updating version 32 to 34 2024-09-16T15:01:49.513 INFO:tasks.rados.rados.0.smithi162.stdout:160: oid 34 version 34 is already newer than 33 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 34 (ObjNum 82 snap 12 seq_num 82) dirty exists 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:160: left oid 34 (ObjNum 82 snap 12 seq_num 82) 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: finishing write tid 3 to smithi16234619-9 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: finishing write tid 4 to smithi16234619-9 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: finishing write tid 5 to smithi16234619-9 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: oid 9 updating version 0 to 28 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: oid 9 updating version 28 to 30 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: oid 9 updating version 30 to 31 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: oid 9 version 31 is already newer than 29 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 31 (ObjNum 83 snap 12 seq_num 83) dirty exists 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:162: left oid 9 (ObjNum 83 snap 12 seq_num 83) 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:158: done (5 left) 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:160: done (4 left) 2024-09-16T15:01:49.514 INFO:tasks.rados.rados.0.smithi162.stdout:161: done (3 left) 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:162: done (2 left) 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:166: read oid 7 snap 10 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:166: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:167: write (excl) oid 45 current snap is 12 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:165: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:165: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:165: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 41 (ObjNum 71 snap 10 seq_num 71) dirty exists 2024-09-16T15:01:49.515 INFO:tasks.rados.rados.0.smithi162.stdout:164: expect (ObjNum 65 snap 4 seq_num 65) 2024-09-16T15:01:49.653 INFO:tasks.rados.rados.0.smithi162.stdout:166: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:49.737 INFO:tasks.rados.rados.0.smithi162.stdout:167: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2024-09-16T15:01:49.742 INFO:tasks.rados.rados.0.smithi162.stdout:167: writing smithi16234619-45 from 611788 to 1012587 tid 1 2024-09-16T15:01:49.749 INFO:tasks.rados.rados.0.smithi162.stdout:167: writing smithi16234619-45 from 1748958 to 2237644 tid 2 2024-09-16T15:01:49.756 INFO:tasks.rados.rados.0.smithi162.stdout:167: writing smithi16234619-45 from 2907296 to 3411788 tid 3 2024-09-16T15:01:49.756 INFO:tasks.rados.rados.0.smithi162.stdout:164: done (3 left) 2024-09-16T15:01:49.757 INFO:tasks.rados.rados.0.smithi162.stdout:165: done (2 left) 2024-09-16T15:01:49.757 INFO:tasks.rados.rados.0.smithi162.stdout:166: done (1 left) 2024-09-16T15:01:49.757 INFO:tasks.rados.rados.0.smithi162.stdout:168: copy_from oid 46 from oid 29 current snap is 12 2024-09-16T15:01:49.757 INFO:tasks.rados.rados.0.smithi162.stdout:169: snap_remove snap 10 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:169: done (2 left) 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:170: read oid 22 snap -1 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:170: expect (ObjNum 61 snap 3 seq_num 61) 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:171: snap_create 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:167: finishing write tid 1 to smithi16234619-45 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:167: finishing write tid 2 to smithi16234619-45 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:167: finishing write tid 3 to smithi16234619-45 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:167: finishing write tid 4 to smithi16234619-45 2024-09-16T15:01:50.358 INFO:tasks.rados.rados.0.smithi162.stdout:167: finishing write tid 5 to smithi16234619-45 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:167: oid 45 updating version 0 to 46 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:167: oid 45 updating version 46 to 47 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:167: oid 45 updating version 47 to 48 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:167: oid 45 version 48 is already newer than 45 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 48 (ObjNum 84 snap 12 seq_num 84) dirty exists 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:167: left oid 45 (ObjNum 84 snap 12 seq_num 84) 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:168: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:168: finishing copy_from to smithi16234619-46 2024-09-16T15:01:50.359 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 27 (ObjNum 67 snap 10 seq_num 67) dirty exists 2024-09-16T15:01:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:170: expect (ObjNum 61 snap 3 seq_num 61) 2024-09-16T15:01:51.368 INFO:tasks.rados.rados.0.smithi162.stdout:167: done (3 left) 2024-09-16T15:01:51.368 INFO:tasks.rados.rados.0.smithi162.stdout:168: done (2 left) 2024-09-16T15:01:51.368 INFO:tasks.rados.rados.0.smithi162.stdout:170: done (1 left) 2024-09-16T15:01:51.368 INFO:tasks.rados.rados.0.smithi162.stdout:171: done (0 left) 2024-09-16T15:01:51.368 INFO:tasks.rados.rados.0.smithi162.stdout:172: snap_remove snap 12 2024-09-16T15:01:52.368 INFO:tasks.rados.rados.0.smithi162.stdout:172: done (0 left) 2024-09-16T15:01:52.368 INFO:tasks.rados.rados.0.smithi162.stdout:173: snap_create 2024-09-16T15:01:53.372 INFO:tasks.rados.rados.0.smithi162.stdout:173: done (0 left) 2024-09-16T15:01:53.372 INFO:tasks.rados.rados.0.smithi162.stdout:174: snap_remove snap 13 2024-09-16T15:01:54.379 INFO:tasks.rados.rados.0.smithi162.stdout:174: done (0 left) 2024-09-16T15:01:54.379 INFO:tasks.rados.rados.0.smithi162.stdout:175: read oid 1 snap -1 2024-09-16T15:01:54.379 INFO:tasks.rados.rados.0.smithi162.stdout:175: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T15:01:54.379 INFO:tasks.rados.rados.0.smithi162.stdout:176: copy_from oid 44 from oid 48 current snap is 14 2024-09-16T15:01:54.380 INFO:tasks.rados.rados.0.smithi162.stdout:177: delete oid 34 current snap is 14 2024-09-16T15:01:54.383 INFO:tasks.rados.rados.0.smithi162.stdout:177: done (2 left) 2024-09-16T15:01:54.383 INFO:tasks.rados.rados.0.smithi162.stdout:178: read oid 46 snap -1 2024-09-16T15:01:54.383 INFO:tasks.rados.rados.0.smithi162.stdout:178: expect (ObjNum 67 snap 10 seq_num 67) 2024-09-16T15:01:54.383 INFO:tasks.rados.rados.0.smithi162.stdout:179: write oid 31 current snap is 14 2024-09-16T15:01:54.383 INFO:tasks.rados.rados.0.smithi162.stdout:179: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2024-09-16T15:01:54.390 INFO:tasks.rados.rados.0.smithi162.stdout:179: writing smithi16234619-31 from 645402 to 1196699 tid 1 2024-09-16T15:01:54.395 INFO:tasks.rados.rados.0.smithi162.stdout:179: writing smithi16234619-31 from 1749569 to 2241924 tid 2 2024-09-16T15:01:54.402 INFO:tasks.rados.rados.0.smithi162.stdout:179: writing smithi16234619-31 from 2845789 to 3445402 tid 3 2024-09-16T15:01:54.402 INFO:tasks.rados.rados.0.smithi162.stdout:180: write (excl) oid 28 current snap is 14 2024-09-16T15:01:54.402 INFO:tasks.rados.rados.0.smithi162.stdout:180: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2024-09-16T15:01:54.409 INFO:tasks.rados.rados.0.smithi162.stdout:180: writing smithi16234619-28 from 662209 to 1288755 tid 1 2024-09-16T15:01:54.415 INFO:tasks.rados.rados.0.smithi162.stdout:180: writing smithi16234619-28 from 1808051 to 2444064 tid 2 2024-09-16T15:01:54.421 INFO:tasks.rados.rados.0.smithi162.stdout:180: writing smithi16234619-28 from 2873212 to 3462209 tid 3 2024-09-16T15:01:54.421 INFO:tasks.rados.rados.0.smithi162.stdout:181: write (excl) oid 36 current snap is 14 2024-09-16T15:01:54.421 INFO:tasks.rados.rados.0.smithi162.stdout:175: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T15:01:54.495 INFO:tasks.rados.rados.0.smithi162.stdout:178: expect (ObjNum 67 snap 10 seq_num 67) 2024-09-16T15:01:54.593 INFO:tasks.rados.rados.0.smithi162.stdout:179: finishing write tid 1 to smithi16234619-31 2024-09-16T15:01:54.593 INFO:tasks.rados.rados.0.smithi162.stdout:176: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:01:54.593 INFO:tasks.rados.rados.0.smithi162.stdout:179: finishing write tid 2 to smithi16234619-31 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: finishing write tid 3 to smithi16234619-31 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: finishing write tid 4 to smithi16234619-31 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: finishing write tid 5 to smithi16234619-31 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: oid 31 updating version 0 to 29 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: oid 31 version 29 is already newer than 28 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: oid 31 updating version 29 to 30 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: oid 31 version 30 is already newer than 27 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 30 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:179: left oid 31 (ObjNum 86 snap 14 seq_num 86) 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:176: finishing copy_from to smithi16234619-44 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 19 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:180: finishing write tid 1 to smithi16234619-28 2024-09-16T15:01:54.594 INFO:tasks.rados.rados.0.smithi162.stdout:181: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2024-09-16T15:01:54.600 INFO:tasks.rados.rados.0.smithi162.stdout:181: writing smithi16234619-36 from 679016 to 1380811 tid 1 2024-09-16T15:01:54.606 INFO:tasks.rados.rados.0.smithi162.stdout:181: writing smithi16234619-36 from 1866533 to 2362557 tid 2 2024-09-16T15:01:54.611 INFO:tasks.rados.rados.0.smithi162.stdout:181: writing smithi16234619-36 from 2900635 to 3303278 tid 3 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:175: done (5 left) 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:176: done (4 left) 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:178: done (3 left) 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:179: done (2 left) 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:182: rollback oid 48 current snap is 14 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:182: done (2 left) 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:183: write oid 35 current snap is 14 2024-09-16T15:01:54.612 INFO:tasks.rados.rados.0.smithi162.stdout:183: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2024-09-16T15:01:54.619 INFO:tasks.rados.rados.0.smithi162.stdout:183: writing smithi16234619-35 from 695823 to 1472867 tid 1 2024-09-16T15:01:54.626 INFO:tasks.rados.rados.0.smithi162.stdout:183: writing smithi16234619-35 from 1925015 to 2564697 tid 2 2024-09-16T15:01:54.632 INFO:tasks.rados.rados.0.smithi162.stdout:183: writing smithi16234619-35 from 3328058 to 3495823 tid 3 2024-09-16T15:01:54.632 INFO:tasks.rados.rados.0.smithi162.stdout:184: snap_create 2024-09-16T15:01:54.632 INFO:tasks.rados.rados.0.smithi162.stdout:180: finishing write tid 2 to smithi16234619-28 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: finishing write tid 3 to smithi16234619-28 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: finishing write tid 4 to smithi16234619-28 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: finishing write tid 5 to smithi16234619-28 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: oid 28 updating version 0 to 25 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: oid 28 updating version 25 to 26 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: oid 28 updating version 26 to 27 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: oid 28 updating version 27 to 28 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 28 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-16T15:01:54.633 INFO:tasks.rados.rados.0.smithi162.stdout:180: left oid 28 (ObjNum 87 snap 14 seq_num 87) 2024-09-16T15:01:54.637 INFO:tasks.rados.rados.0.smithi162.stdout:181: finishing write tid 1 to smithi16234619-36 2024-09-16T15:01:54.637 INFO:tasks.rados.rados.0.smithi162.stdout:181: finishing write tid 2 to smithi16234619-36 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: finishing write tid 3 to smithi16234619-36 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: finishing write tid 4 to smithi16234619-36 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: finishing write tid 5 to smithi16234619-36 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: oid 36 updating version 0 to 39 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: oid 36 updating version 39 to 41 2024-09-16T15:01:54.638 INFO:tasks.rados.rados.0.smithi162.stdout:181: oid 36 version 41 is already newer than 40 2024-09-16T15:01:54.639 INFO:tasks.rados.rados.0.smithi162.stdout:181: oid 36 updating version 41 to 42 2024-09-16T15:01:54.639 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 42 (ObjNum 88 snap 14 seq_num 88) dirty exists 2024-09-16T15:01:54.639 INFO:tasks.rados.rados.0.smithi162.stdout:181: left oid 36 (ObjNum 88 snap 14 seq_num 88) 2024-09-16T15:01:54.654 INFO:tasks.rados.rados.0.smithi162.stdout:183: finishing write tid 1 to smithi16234619-35 2024-09-16T15:01:54.658 INFO:tasks.rados.rados.0.smithi162.stdout:183: finishing write tid 2 to smithi16234619-35 2024-09-16T15:01:54.659 INFO:tasks.rados.rados.0.smithi162.stdout:183: finishing write tid 3 to smithi16234619-35 2024-09-16T15:01:54.659 INFO:tasks.rados.rados.0.smithi162.stdout:183: finishing write tid 4 to smithi16234619-35 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: finishing write tid 5 to smithi16234619-35 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: oid 35 updating version 0 to 44 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: oid 35 version 44 is already newer than 43 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: oid 35 updating version 44 to 45 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: oid 35 version 45 is already newer than 42 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 45 (ObjNum 89 snap 14 seq_num 89) dirty exists 2024-09-16T15:01:54.660 INFO:tasks.rados.rados.0.smithi162.stdout:183: left oid 35 (ObjNum 89 snap 14 seq_num 89) 2024-09-16T15:01:55.361 INFO:tasks.rados.rados.0.smithi162.stdout:180: done (3 left) 2024-09-16T15:01:55.361 INFO:tasks.rados.rados.0.smithi162.stdout:181: done (2 left) 2024-09-16T15:01:55.361 INFO:tasks.rados.rados.0.smithi162.stdout:183: done (1 left) 2024-09-16T15:01:55.361 INFO:tasks.rados.rados.0.smithi162.stdout:184: done (0 left) 2024-09-16T15:01:55.362 INFO:tasks.rados.rados.0.smithi162.stdout:185: snap_create 2024-09-16T15:01:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:185: done (0 left) 2024-09-16T15:01:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:186: copy_from oid 12 from oid 45 current snap is 16 2024-09-16T15:01:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:187: write oid 32 current snap is 16 2024-09-16T15:01:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:187: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2024-09-16T15:01:56.366 INFO:tasks.rados.rados.0.smithi162.stdout:187: writing smithi16234619-32 from 712630 to 1164923 tid 1 2024-09-16T15:01:56.371 INFO:tasks.rados.rados.0.smithi162.stdout:187: writing smithi16234619-32 from 1583497 to 2083190 tid 2 2024-09-16T15:01:56.376 INFO:tasks.rados.rados.0.smithi162.stdout:187: writing smithi16234619-32 from 2555481 to 2980668 tid 3 2024-09-16T15:01:56.376 INFO:tasks.rados.rados.0.smithi162.stdout:188: snap_create 2024-09-16T15:01:56.418 INFO:tasks.rados.rados.0.smithi162.stdout:186: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:01:56.446 INFO:tasks.rados.rados.0.smithi162.stdout:186: finishing copy_from to smithi16234619-12 2024-09-16T15:01:56.447 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 48 (ObjNum 84 snap 12 seq_num 84) dirty exists 2024-09-16T15:01:56.455 INFO:tasks.rados.rados.0.smithi162.stdout:187: finishing write tid 1 to smithi16234619-32 2024-09-16T15:01:56.456 INFO:tasks.rados.rados.0.smithi162.stdout:187: finishing write tid 2 to smithi16234619-32 2024-09-16T15:01:56.456 INFO:tasks.rados.rados.0.smithi162.stdout:187: finishing write tid 3 to smithi16234619-32 2024-09-16T15:01:56.456 INFO:tasks.rados.rados.0.smithi162.stdout:187: finishing write tid 4 to smithi16234619-32 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: finishing write tid 5 to smithi16234619-32 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: oid 32 updating version 0 to 49 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: oid 32 updating version 49 to 52 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: oid 32 version 52 is already newer than 51 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: oid 32 version 52 is already newer than 50 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 52 (ObjNum 90 snap 16 seq_num 90) dirty exists 2024-09-16T15:01:56.457 INFO:tasks.rados.rados.0.smithi162.stdout:187: left oid 32 (ObjNum 90 snap 16 seq_num 90) 2024-09-16T15:01:57.371 INFO:tasks.rados.rados.0.smithi162.stdout:186: done (2 left) 2024-09-16T15:01:57.371 INFO:tasks.rados.rados.0.smithi162.stdout:187: done (1 left) 2024-09-16T15:01:57.371 INFO:tasks.rados.rados.0.smithi162.stdout:188: done (0 left) 2024-09-16T15:01:57.371 INFO:tasks.rados.rados.0.smithi162.stdout:189: delete oid 17 current snap is 17 2024-09-16T15:01:57.373 INFO:tasks.rados.rados.0.smithi162.stdout:189: done (0 left) 2024-09-16T15:01:57.374 INFO:tasks.rados.rados.0.smithi162.stdout:190: write (excl) oid 25 current snap is 17 2024-09-16T15:01:57.374 INFO:tasks.rados.rados.0.smithi162.stdout:190: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2024-09-16T15:01:57.381 INFO:tasks.rados.rados.0.smithi162.stdout:190: writing smithi16234619-25 from 746244 to 1465388 tid 1 2024-09-16T15:01:57.387 INFO:tasks.rados.rados.0.smithi162.stdout:190: writing smithi16234619-25 from 2100461 to 2603823 tid 2 2024-09-16T15:01:57.392 INFO:tasks.rados.rados.0.smithi162.stdout:190: writing smithi16234619-25 from 3126680 to 3546244 tid 3 2024-09-16T15:01:57.393 INFO:tasks.rados.rados.0.smithi162.stdout:191: read oid 46 snap -1 2024-09-16T15:01:57.393 INFO:tasks.rados.rados.0.smithi162.stdout:191: expect (ObjNum 67 snap 10 seq_num 67) 2024-09-16T15:01:57.393 INFO:tasks.rados.rados.0.smithi162.stdout:192: delete oid 28 current snap is 17 2024-09-16T15:01:57.396 INFO:tasks.rados.rados.0.smithi162.stdout:192: done (2 left) 2024-09-16T15:01:57.397 INFO:tasks.rados.rados.0.smithi162.stdout:193: rollback oid 49 current snap is 17 2024-09-16T15:01:57.397 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 14 2024-09-16T15:01:57.397 INFO:tasks.rados.rados.0.smithi162.stdout:194: write oid 42 current snap is 17 2024-09-16T15:01:57.397 INFO:tasks.rados.rados.0.smithi162.stdout:194: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2024-09-16T15:01:57.403 INFO:tasks.rados.rados.0.smithi162.stdout:194: writing smithi16234619-42 from 779858 to 1249500 tid 1 2024-09-16T15:01:57.409 INFO:tasks.rados.rados.0.smithi162.stdout:194: writing smithi16234619-42 from 1817425 to 2324456 tid 2 2024-09-16T15:01:57.415 INFO:tasks.rados.rados.0.smithi162.stdout:194: writing smithi16234619-42 from 2781526 to 3368154 tid 3 2024-09-16T15:01:57.415 INFO:tasks.rados.rados.0.smithi162.stdout:195: write oid 3 current snap is 17 2024-09-16T15:01:57.415 INFO:tasks.rados.rados.0.smithi162.stdout:195: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2024-09-16T15:01:57.422 INFO:tasks.rados.rados.0.smithi162.stdout:195: writing smithi16234619-3 from 796665 to 1341556 tid 1 2024-09-16T15:01:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:195: writing smithi16234619-3 from 1759554 to 2410243 tid 2 2024-09-16T15:01:57.435 INFO:tasks.rados.rados.0.smithi162.stdout:195: writing smithi16234619-3 from 2976243 to 3596665 tid 3 2024-09-16T15:01:57.435 INFO:tasks.rados.rados.0.smithi162.stdout:196: read oid 6 snap -1 2024-09-16T15:01:57.435 INFO:tasks.rados.rados.0.smithi162.stdout:196: expect (ObjNum 81 snap 12 seq_num 81) 2024-09-16T15:01:57.435 INFO:tasks.rados.rados.0.smithi162.stdout:197: snap_remove snap 16 2024-09-16T15:01:58.380 INFO:tasks.rados.rados.0.smithi162.stdout:197: done (6 left) 2024-09-16T15:01:58.381 INFO:tasks.rados.rados.0.smithi162.stdout:198: rollback oid 23 current snap is 17 2024-09-16T15:01:58.381 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 14 2024-09-16T15:01:58.381 INFO:tasks.rados.rados.0.smithi162.stdout:193: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:01:58.381 INFO:tasks.rados.rados.0.smithi162.stdout:191: expect (ObjNum 67 snap 10 seq_num 67) 2024-09-16T15:01:58.481 INFO:tasks.rados.rados.0.smithi162.stdout:193: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:01:58.481 INFO:tasks.rados.rados.0.smithi162.stdout:193: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:01:58.481 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 36 (ObjNum 76 snap 12 seq_num 76) dirty exists 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:191: done (6 left) 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:193: done (5 left) 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:199: read oid 41 snap -1 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:199: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:200: write oid 39 current snap is 17 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:190: finishing write tid 1 to smithi16234619-25 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:190: finishing write tid 2 to smithi16234619-25 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:190: finishing write tid 3 to smithi16234619-25 2024-09-16T15:01:58.482 INFO:tasks.rados.rados.0.smithi162.stdout:200: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2024-09-16T15:01:58.486 INFO:tasks.rados.rados.0.smithi162.stdout:200: writing smithi16234619-39 from 413472 to 1033612 tid 1 2024-09-16T15:01:58.493 INFO:tasks.rados.rados.0.smithi162.stdout:200: writing smithi16234619-39 from 1818036 to 2328736 tid 2 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:200: writing smithi16234619-39 from 3120019 to 3613472 tid 3 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:201: rollback oid 44 current snap is 17 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 15 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:190: finishing write tid 4 to smithi16234619-25 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:190: finishing write tid 5 to smithi16234619-25 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:190: oid 25 updating version 0 to 44 2024-09-16T15:01:58.501 INFO:tasks.rados.rados.0.smithi162.stdout:190: oid 25 updating version 44 to 46 2024-09-16T15:01:58.502 INFO:tasks.rados.rados.0.smithi162.stdout:190: oid 25 version 46 is already newer than 45 2024-09-16T15:01:58.502 INFO:tasks.rados.rados.0.smithi162.stdout:190: oid 25 updating version 46 to 47 2024-09-16T15:01:58.502 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 47 (ObjNum 92 snap 17 seq_num 92) dirty exists 2024-09-16T15:01:58.502 INFO:tasks.rados.rados.0.smithi162.stdout:190: left oid 25 (ObjNum 92 snap 17 seq_num 92) 2024-09-16T15:01:58.502 INFO:tasks.rados.rados.0.smithi162.stdout:196: expect (ObjNum 81 snap 12 seq_num 81) 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: finishing write tid 1 to smithi16234619-42 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: finishing write tid 2 to smithi16234619-42 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: finishing write tid 3 to smithi16234619-42 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: finishing write tid 4 to smithi16234619-42 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: finishing write tid 5 to smithi16234619-42 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: oid 42 updating version 0 to 61 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: oid 42 updating version 61 to 63 2024-09-16T15:01:58.644 INFO:tasks.rados.rados.0.smithi162.stdout:194: oid 42 version 63 is already newer than 62 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:194: oid 42 updating version 63 to 64 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 64 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:194: left oid 42 (ObjNum 94 snap 17 seq_num 94) 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:195: finishing write tid 1 to smithi16234619-3 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:190: done (7 left) 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:194: done (6 left) 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:196: done (5 left) 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:202: write oid 50 current snap is 17 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:195: finishing write tid 2 to smithi16234619-3 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:195: finishing write tid 3 to smithi16234619-3 2024-09-16T15:01:58.645 INFO:tasks.rados.rados.0.smithi162.stdout:202: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2024-09-16T15:01:58.649 INFO:tasks.rados.rados.0.smithi162.stdout:202: writing smithi16234619-50 from 430279 to 1125668 tid 1 2024-09-16T15:01:58.657 INFO:tasks.rados.rados.0.smithi162.stdout:202: writing smithi16234619-50 from 1876518 to 2530876 tid 2 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:202: writing smithi16234619-50 from 3031089 to 3567886 tid 3 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:203: snap_create 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: finishing write tid 4 to smithi16234619-3 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: finishing write tid 5 to smithi16234619-3 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: oid 3 updating version 0 to 17 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: oid 3 version 17 is already newer than 14 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: oid 3 version 17 is already newer than 16 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: oid 3 version 17 is already newer than 15 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 17 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:195: left oid 3 (ObjNum 95 snap 17 seq_num 95) 2024-09-16T15:01:58.663 INFO:tasks.rados.rados.0.smithi162.stdout:198: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:01:58.664 INFO:tasks.rados.rados.0.smithi162.stdout:198: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:01:58.664 INFO:tasks.rados.rados.0.smithi162.stdout:198: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:01:58.664 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 14 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-09-16T15:01:58.664 INFO:tasks.rados.rados.0.smithi162.stdout:199: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:01:58.747 INFO:tasks.rados.rados.0.smithi162.stdout:201: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:01:58.747 INFO:tasks.rados.rados.0.smithi162.stdout:201: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:201: finishing rollback tid 2 to smithi16234619-44 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 49 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: finishing write tid 1 to smithi16234619-39 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: finishing write tid 2 to smithi16234619-39 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: finishing write tid 3 to smithi16234619-39 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: finishing write tid 4 to smithi16234619-39 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: finishing write tid 5 to smithi16234619-39 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: oid 39 updating version 0 to 67 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: oid 39 updating version 67 to 70 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: oid 39 version 70 is already newer than 68 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: oid 39 version 70 is already newer than 69 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 70 (ObjNum 96 snap 17 seq_num 96) dirty exists 2024-09-16T15:01:58.748 INFO:tasks.rados.rados.0.smithi162.stdout:200: left oid 39 (ObjNum 96 snap 17 seq_num 96) 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: finishing write tid 1 to smithi16234619-50 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: finishing write tid 2 to smithi16234619-50 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: finishing write tid 3 to smithi16234619-50 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: finishing write tid 4 to smithi16234619-50 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: finishing write tid 5 to smithi16234619-50 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: oid 50 updating version 0 to 54 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: oid 50 version 54 is already newer than 51 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: oid 50 version 54 is already newer than 52 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:202: oid 50 version 54 is already newer than 53 2024-09-16T15:01:58.749 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 54 (ObjNum 97 snap 17 seq_num 97) dirty exists 2024-09-16T15:01:58.750 INFO:tasks.rados.rados.0.smithi162.stdout:202: left oid 50 (ObjNum 97 snap 17 seq_num 97) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:195: done (6 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:198: done (5 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:199: done (4 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:200: done (3 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:201: done (2 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:202: done (1 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:203: done (0 left) 2024-09-16T15:01:59.389 INFO:tasks.rados.rados.0.smithi162.stdout:204: snap_create 2024-09-16T15:02:00.362 INFO:tasks.rados.rados.0.smithi162.stdout:204: done (0 left) 2024-09-16T15:02:00.362 INFO:tasks.rados.rados.0.smithi162.stdout:205: rollback oid 48 current snap is 19 2024-09-16T15:02:00.362 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 48 to 14 2024-09-16T15:02:00.362 INFO:tasks.rados.rados.0.smithi162.stdout:206: delete oid 43 current snap is 19 2024-09-16T15:02:00.369 INFO:tasks.rados.rados.0.smithi162.stdout:206: done (1 left) 2024-09-16T15:02:00.369 INFO:tasks.rados.rados.0.smithi162.stdout:207: snap_remove snap 18 2024-09-16T15:02:01.378 INFO:tasks.rados.rados.0.smithi162.stdout:207: done (1 left) 2024-09-16T15:02:01.378 INFO:tasks.rados.rados.0.smithi162.stdout:205: finishing rollback tid 0 to smithi16234619-48 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:208: read oid 13 snap -1 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:208: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:205: finishing rollback tid 1 to smithi16234619-48 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:205: finishing rollback tid 2 to smithi16234619-48 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 41 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:205: done (1 left) 2024-09-16T15:02:01.379 INFO:tasks.rados.rados.0.smithi162.stdout:209: snap_remove snap 15 2024-09-16T15:02:02.689 INFO:tasks.rados.rados.0.smithi162.stdout:209: done (1 left) 2024-09-16T15:02:02.689 INFO:tasks.rados.rados.0.smithi162.stdout:210: read oid 32 snap -1 2024-09-16T15:02:02.689 INFO:tasks.rados.rados.0.smithi162.stdout:210: expect (ObjNum 90 snap 16 seq_num 90) 2024-09-16T15:02:02.689 INFO:tasks.rados.rados.0.smithi162.stdout:211: write (excl) oid 18 current snap is 19 2024-09-16T15:02:02.689 INFO:tasks.rados.rados.0.smithi162.stdout:211: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2024-09-16T15:02:02.694 INFO:tasks.rados.rados.0.smithi162.stdout:211: writing smithi16234619-18 from 463893 to 1026133 tid 1 2024-09-16T15:02:02.701 INFO:tasks.rados.rados.0.smithi162.stdout:211: writing smithi16234619-18 from 1593482 to 2251509 tid 2 2024-09-16T15:02:02.706 INFO:tasks.rados.rados.0.smithi162.stdout:211: writing smithi16234619-18 from 2685935 to 3245276 tid 3 2024-09-16T15:02:02.706 INFO:tasks.rados.rados.0.smithi162.stdout:212: write oid 1 current snap is 19 2024-09-16T15:02:02.706 INFO:tasks.rados.rados.0.smithi162.stdout:208: expect (ObjNum 12 snap 0 seq_num 12) 2024-09-16T15:02:02.775 INFO:tasks.rados.rados.0.smithi162.stdout:210: expect (ObjNum 90 snap 16 seq_num 90) 2024-09-16T15:02:02.929 INFO:tasks.rados.rados.0.smithi162.stdout:211: finishing write tid 1 to smithi16234619-18 2024-09-16T15:02:02.929 INFO:tasks.rados.rados.0.smithi162.stdout:211: finishing write tid 2 to smithi16234619-18 2024-09-16T15:02:02.929 INFO:tasks.rados.rados.0.smithi162.stdout:212: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2024-09-16T15:02:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:212: writing smithi16234619-1 from 480700 to 1118189 tid 1 2024-09-16T15:02:02.940 INFO:tasks.rados.rados.0.smithi162.stdout:212: writing smithi16234619-1 from 1651964 to 2053649 tid 2 2024-09-16T15:02:02.948 INFO:tasks.rados.rados.0.smithi162.stdout:212: writing smithi16234619-1 from 2713358 to 3483971 tid 3 2024-09-16T15:02:02.948 INFO:tasks.rados.rados.0.smithi162.stdout:208: done (3 left) 2024-09-16T15:02:02.948 INFO:tasks.rados.rados.0.smithi162.stdout:210: done (2 left) 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:213: copy_from oid 27 from oid 50 current snap is 19 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: finishing write tid 3 to smithi16234619-18 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: finishing write tid 4 to smithi16234619-18 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: finishing write tid 5 to smithi16234619-18 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: oid 18 updating version 0 to 34 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: oid 18 version 34 is already newer than 33 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: oid 18 version 34 is already newer than 32 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: oid 18 updating version 34 to 35 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 35 (ObjNum 99 snap 19 seq_num 99) dirty exists 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: left oid 18 (ObjNum 99 snap 19 seq_num 99) 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:211: done (2 left) 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:214: read oid 37 snap -1 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:214: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:02.949 INFO:tasks.rados.rados.0.smithi162.stdout:215: snap_remove snap 17 2024-09-16T15:02:04.177 INFO:tasks.rados.rados.0.smithi162.stdout:215: done (3 left) 2024-09-16T15:02:04.177 INFO:tasks.rados.rados.0.smithi162.stdout:216: delete oid 14 current snap is 19 2024-09-16T15:02:04.177 INFO:tasks.rados.rados.0.smithi162.stdout:214: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:04.364 INFO:tasks.rados.rados.0.smithi162.stdout:212: finishing write tid 1 to smithi16234619-1 2024-09-16T15:02:04.364 INFO:tasks.rados.rados.0.smithi162.stdout:212: finishing write tid 2 to smithi16234619-1 2024-09-16T15:02:04.364 INFO:tasks.rados.rados.0.smithi162.stdout:212: finishing write tid 3 to smithi16234619-1 2024-09-16T15:02:04.364 INFO:tasks.rados.rados.0.smithi162.stdout:214: done (3 left) 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:216: done (2 left) 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:217: read oid 28 snap -1 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:217: expect deleted 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:218: delete oid 11 current snap is 19 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: finishing write tid 4 to smithi16234619-1 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: finishing write tid 5 to smithi16234619-1 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: oid 1 updating version 0 to 48 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: oid 1 version 48 is already newer than 47 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: oid 1 updating version 48 to 50 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: oid 1 version 50 is already newer than 49 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 50 (ObjNum 100 snap 19 seq_num 100) dirty exists 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:212: left oid 1 (ObjNum 100 snap 19 seq_num 100) 2024-09-16T15:02:04.365 INFO:tasks.rados.rados.0.smithi162.stdout:213: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:213: finishing copy_from to smithi16234619-27 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 54 (ObjNum 97 snap 17 seq_num 97) dirty exists 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:212: done (3 left) 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:213: done (2 left) 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:217: done (1 left) 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:218: done (0 left) 2024-09-16T15:02:04.366 INFO:tasks.rados.rados.0.smithi162.stdout:219: delete oid 46 current snap is 19 2024-09-16T15:02:04.369 INFO:tasks.rados.rados.0.smithi162.stdout:219: done (0 left) 2024-09-16T15:02:04.369 INFO:tasks.rados.rados.0.smithi162.stdout:220: delete oid 4 current snap is 19 2024-09-16T15:02:04.371 INFO:tasks.rados.rados.0.smithi162.stdout:220: done (0 left) 2024-09-16T15:02:04.371 INFO:tasks.rados.rados.0.smithi162.stdout:221: write (excl) oid 10 current snap is 19 2024-09-16T15:02:04.371 INFO:tasks.rados.rados.0.smithi162.stdout:221: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2024-09-16T15:02:04.377 INFO:tasks.rados.rados.0.smithi162.stdout:221: writing smithi16234619-10 from 564735 to 1178469 tid 1 2024-09-16T15:02:04.385 INFO:tasks.rados.rados.0.smithi162.stdout:221: writing smithi16234619-10 from 1828021 to 2497055 tid 2 2024-09-16T15:02:04.392 INFO:tasks.rados.rados.0.smithi162.stdout:221: writing smithi16234619-10 from 3134120 to 3764735 tid 3 2024-09-16T15:02:04.392 INFO:tasks.rados.rados.0.smithi162.stdout:222: delete oid 11 current snap is 19 2024-09-16T15:02:04.393 INFO:tasks.rados.rados.0.smithi162.stdout:222: done (1 left) 2024-09-16T15:02:04.393 INFO:tasks.rados.rados.0.smithi162.stdout:223: write (excl) oid 11 current snap is 19 2024-09-16T15:02:04.394 INFO:tasks.rados.rados.0.smithi162.stdout:223: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2024-09-16T15:02:04.399 INFO:tasks.rados.rados.0.smithi162.stdout:223: writing smithi16234619-11 from 598349 to 1078934 tid 1 2024-09-16T15:02:04.405 INFO:tasks.rados.rados.0.smithi162.stdout:223: writing smithi16234619-11 from 1544985 to 2217688 tid 2 2024-09-16T15:02:04.412 INFO:tasks.rados.rados.0.smithi162.stdout:223: writing smithi16234619-11 from 2905319 to 3387542 tid 3 2024-09-16T15:02:04.412 INFO:tasks.rados.rados.0.smithi162.stdout:224: write oid 6 current snap is 19 2024-09-16T15:02:04.412 INFO:tasks.rados.rados.0.smithi162.stdout:224: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2024-09-16T15:02:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:224: writing smithi16234619-6 from 615156 to 1170990 tid 1 2024-09-16T15:02:04.423 INFO:tasks.rados.rados.0.smithi162.stdout:224: writing smithi16234619-6 from 1603467 to 2019828 tid 2 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:224: writing smithi16234619-6 from 2816389 to 3509884 tid 3 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:225: write oid 14 current snap is 19 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: finishing write tid 1 to smithi16234619-10 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: finishing write tid 2 to smithi16234619-10 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: finishing write tid 3 to smithi16234619-10 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: finishing write tid 4 to smithi16234619-10 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: finishing write tid 5 to smithi16234619-10 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: oid 10 updating version 0 to 63 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: oid 10 version 63 is already newer than 60 2024-09-16T15:02:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:221: oid 10 version 63 is already newer than 61 2024-09-16T15:02:04.433 INFO:tasks.rados.rados.0.smithi162.stdout:221: oid 10 version 63 is already newer than 62 2024-09-16T15:02:04.433 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 63 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:04.433 INFO:tasks.rados.rados.0.smithi162.stdout:221: left oid 10 (ObjNum 105 snap 19 seq_num 105) 2024-09-16T15:02:04.433 INFO:tasks.rados.rados.0.smithi162.stdout:225: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2024-09-16T15:02:04.439 INFO:tasks.rados.rados.0.smithi162.stdout:225: writing smithi16234619-14 from 631963 to 1263046 tid 1 2024-09-16T15:02:04.446 INFO:tasks.rados.rados.0.smithi162.stdout:225: writing smithi16234619-14 from 2061949 to 2621968 tid 2 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:225: writing smithi16234619-14 from 3243812 to 3748579 tid 3 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:221: done (3 left) 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:226: read oid 21 snap 14 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:226: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:227: snap_create 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:223: finishing write tid 1 to smithi16234619-11 2024-09-16T15:02:04.454 INFO:tasks.rados.rados.0.smithi162.stdout:223: finishing write tid 2 to smithi16234619-11 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: finishing write tid 3 to smithi16234619-11 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: finishing write tid 4 to smithi16234619-11 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: finishing write tid 5 to smithi16234619-11 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: oid 11 updating version 0 to 15 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: oid 11 updating version 15 to 16 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: oid 11 updating version 16 to 17 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: oid 11 updating version 17 to 18 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 18 (ObjNum 107 snap 19 seq_num 107) dirty exists 2024-09-16T15:02:04.455 INFO:tasks.rados.rados.0.smithi162.stdout:223: left oid 11 (ObjNum 107 snap 19 seq_num 107) 2024-09-16T15:02:04.457 INFO:tasks.rados.rados.0.smithi162.stdout:226: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T15:02:04.535 INFO:tasks.rados.rados.0.smithi162.stdout:224: finishing write tid 1 to smithi16234619-6 2024-09-16T15:02:04.535 INFO:tasks.rados.rados.0.smithi162.stdout:224: finishing write tid 2 to smithi16234619-6 2024-09-16T15:02:04.535 INFO:tasks.rados.rados.0.smithi162.stdout:224: finishing write tid 3 to smithi16234619-6 2024-09-16T15:02:04.535 INFO:tasks.rados.rados.0.smithi162.stdout:224: finishing write tid 4 to smithi16234619-6 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: finishing write tid 5 to smithi16234619-6 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: oid 6 updating version 0 to 54 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: oid 6 updating version 54 to 56 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: oid 6 version 56 is already newer than 55 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: oid 6 version 56 is already newer than 53 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 56 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:224: left oid 6 (ObjNum 108 snap 19 seq_num 108) 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:225: finishing write tid 1 to smithi16234619-14 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:225: finishing write tid 2 to smithi16234619-14 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:225: finishing write tid 3 to smithi16234619-14 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:225: finishing write tid 4 to smithi16234619-14 2024-09-16T15:02:04.536 INFO:tasks.rados.rados.0.smithi162.stdout:225: finishing write tid 5 to smithi16234619-14 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:225: oid 14 updating version 0 to 63 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:225: oid 14 version 63 is already newer than 60 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:225: oid 14 version 63 is already newer than 62 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:225: oid 14 version 63 is already newer than 61 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 63 (ObjNum 109 snap 19 seq_num 109) dirty exists 2024-09-16T15:02:04.537 INFO:tasks.rados.rados.0.smithi162.stdout:225: left oid 14 (ObjNum 109 snap 19 seq_num 109) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:223: done (4 left) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:224: done (3 left) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:225: done (2 left) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:226: done (1 left) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:227: done (0 left) 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:228: rollback oid 31 current snap is 20 2024-09-16T15:02:05.184 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 19 2024-09-16T15:02:05.185 INFO:tasks.rados.rados.0.smithi162.stdout:229: delete oid 18 current snap is 20 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:228: finishing rollback tid 0 to smithi16234619-31 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:228: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:228: finishing rollback tid 2 to smithi16234619-31 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 41 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:228: done (1 left) 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:229: done (0 left) 2024-09-16T15:02:05.207 INFO:tasks.rados.rados.0.smithi162.stdout:230: write (excl) oid 34 current snap is 20 2024-09-16T15:02:05.208 INFO:tasks.rados.rados.0.smithi162.stdout:230: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2024-09-16T15:02:05.214 INFO:tasks.rados.rados.0.smithi162.stdout:230: writing smithi16234619-34 from 665577 to 1447158 tid 1 2024-09-16T15:02:05.220 INFO:tasks.rados.rados.0.smithi162.stdout:230: writing smithi16234619-34 from 2062560 to 2626248 tid 2 2024-09-16T15:02:05.227 INFO:tasks.rados.rados.0.smithi162.stdout:230: writing smithi16234619-34 from 3182305 to 3825969 tid 3 2024-09-16T15:02:05.227 INFO:tasks.rados.rados.0.smithi162.stdout:231: write (excl) oid 1 current snap is 20 2024-09-16T15:02:05.227 INFO:tasks.rados.rados.0.smithi162.stdout:231: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2024-09-16T15:02:05.233 INFO:tasks.rados.rados.0.smithi162.stdout:231: writing smithi16234619-1 from 682384 to 1139214 tid 1 2024-09-16T15:02:05.238 INFO:tasks.rados.rados.0.smithi162.stdout:231: writing smithi16234619-1 from 1721042 to 2144741 tid 2 2024-09-16T15:02:05.245 INFO:tasks.rados.rados.0.smithi162.stdout:231: writing smithi16234619-1 from 2809728 to 3264664 tid 3 2024-09-16T15:02:05.248 INFO:tasks.rados.rados.0.smithi162.stdout:231: writing smithi16234619-1 from 3731357 to 3882384 tid 4 2024-09-16T15:02:05.248 INFO:tasks.rados.rados.0.smithi162.stdout:232: snap_create 2024-09-16T15:02:05.265 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 1 to smithi16234619-1 2024-09-16T15:02:05.270 INFO:tasks.rados.rados.0.smithi162.stdout:230: finishing write tid 1 to smithi16234619-34 2024-09-16T15:02:05.276 INFO:tasks.rados.rados.0.smithi162.stdout:230: finishing write tid 2 to smithi16234619-34 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: finishing write tid 3 to smithi16234619-34 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: finishing write tid 4 to smithi16234619-34 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 2 to smithi16234619-1 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: finishing write tid 5 to smithi16234619-34 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: oid 34 updating version 0 to 64 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: oid 34 updating version 64 to 67 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: oid 34 version 67 is already newer than 66 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: oid 34 version 67 is already newer than 65 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 67 (ObjNum 111 snap 20 seq_num 111) dirty exists 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:230: left oid 34 (ObjNum 111 snap 20 seq_num 111) 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 3 to smithi16234619-1 2024-09-16T15:02:05.281 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 4 to smithi16234619-1 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 5 to smithi16234619-1 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: finishing write tid 6 to smithi16234619-1 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: oid 1 updating version 0 to 66 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: oid 1 version 66 is already newer than 64 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: oid 1 version 66 is already newer than 65 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: oid 1 version 66 is already newer than 62 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: oid 1 version 66 is already newer than 63 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 66 (ObjNum 112 snap 20 seq_num 112) dirty exists 2024-09-16T15:02:05.283 INFO:tasks.rados.rados.0.smithi162.stdout:231: left oid 1 (ObjNum 112 snap 20 seq_num 112) 2024-09-16T15:02:05.358 INFO:tasks.rados.rados.0.smithi162.stdout:230: done (2 left) 2024-09-16T15:02:05.358 INFO:tasks.rados.rados.0.smithi162.stdout:231: done (1 left) 2024-09-16T15:02:05.358 INFO:tasks.rados.rados.0.smithi162.stdout:232: done (0 left) 2024-09-16T15:02:05.358 INFO:tasks.rados.rados.0.smithi162.stdout:233: delete oid 47 current snap is 21 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:233: done (0 left) 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:234: copy_from oid 41 from oid 18 current snap is 21 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:235: read oid 1 snap -1 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:235: expect (ObjNum 112 snap 20 seq_num 112) 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:236: write oid 28 current snap is 21 2024-09-16T15:02:05.362 INFO:tasks.rados.rados.0.smithi162.stdout:236: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2024-09-16T15:02:05.369 INFO:tasks.rados.rados.0.smithi162.stdout:236: writing smithi16234619-28 from 715998 to 1323326 tid 1 2024-09-16T15:02:05.374 INFO:tasks.rados.rados.0.smithi162.stdout:236: writing smithi16234619-28 from 1838006 to 2265374 tid 2 2024-09-16T15:02:05.380 INFO:tasks.rados.rados.0.smithi162.stdout:236: writing smithi16234619-28 from 2864574 to 3342054 tid 3 2024-09-16T15:02:05.383 INFO:tasks.rados.rados.0.smithi162.stdout:236: writing smithi16234619-28 from 3863835 to 3915998 tid 4 2024-09-16T15:02:05.383 INFO:tasks.rados.rados.0.smithi162.stdout:237: write oid 31 current snap is 21 2024-09-16T15:02:05.383 INFO:tasks.rados.rados.0.smithi162.stdout:237: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2024-09-16T15:02:05.391 INFO:tasks.rados.rados.0.smithi162.stdout:237: writing smithi16234619-31 from 732805 to 1531735 tid 1 2024-09-16T15:02:05.398 INFO:tasks.rados.rados.0.smithi162.stdout:237: writing smithi16234619-31 from 2296488 to 2867514 tid 2 2024-09-16T15:02:05.403 INFO:tasks.rados.rados.0.smithi162.stdout:237: writing smithi16234619-31 from 3291997 to 3697102 tid 3 2024-09-16T15:02:05.403 INFO:tasks.rados.rados.0.smithi162.stdout:238: read oid 14 snap -1 2024-09-16T15:02:05.403 INFO:tasks.rados.rados.0.smithi162.stdout:238: expect (ObjNum 109 snap 19 seq_num 109) 2024-09-16T15:02:05.403 INFO:tasks.rados.rados.0.smithi162.stdout:239: write (excl) oid 48 current snap is 21 2024-09-16T15:02:05.403 INFO:tasks.rados.rados.0.smithi162.stdout:239: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2024-09-16T15:02:05.409 INFO:tasks.rados.rados.0.smithi162.stdout:239: writing smithi16234619-48 from 749612 to 1223791 tid 1 2024-09-16T15:02:05.415 INFO:tasks.rados.rados.0.smithi162.stdout:239: writing smithi16234619-48 from 1954970 to 2386007 tid 2 2024-09-16T15:02:05.422 INFO:tasks.rados.rados.0.smithi162.stdout:239: writing smithi16234619-48 from 2919420 to 3535797 tid 3 2024-09-16T15:02:05.422 INFO:tasks.rados.rados.0.smithi162.stdout:240: write (excl) oid 25 current snap is 21 2024-09-16T15:02:05.422 INFO:tasks.rados.rados.0.smithi162.stdout:234: finishing copy_from to smithi16234619-41 2024-09-16T15:02:05.422 INFO:tasks.rados.rados.0.smithi162.stdout:234: got expected ENOENT (src dne) 2024-09-16T15:02:05.422 INFO:tasks.rados.rados.0.smithi162.stdout:234: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:02:05.423 INFO:tasks.rados.rados.0.smithi162.stdout:235: expect (ObjNum 112 snap 20 seq_num 112) 2024-09-16T15:02:05.652 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 1 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 2 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 3 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 4 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 5 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: finishing write tid 6 to smithi16234619-28 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: oid 28 updating version 0 to 38 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: oid 28 version 38 is already newer than 35 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: oid 28 version 38 is already newer than 36 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: oid 28 version 38 is already newer than 37 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: oid 28 updating version 38 to 39 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 39 (ObjNum 114 snap 21 seq_num 114) dirty exists 2024-09-16T15:02:05.653 INFO:tasks.rados.rados.0.smithi162.stdout:236: left oid 28 (ObjNum 114 snap 21 seq_num 114) 2024-09-16T15:02:05.654 INFO:tasks.rados.rados.0.smithi162.stdout:238: expect (ObjNum 109 snap 19 seq_num 109) 2024-09-16T15:02:05.776 INFO:tasks.rados.rados.0.smithi162.stdout:239: finishing write tid 1 to smithi16234619-48 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: finishing write tid 2 to smithi16234619-48 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: finishing write tid 3 to smithi16234619-48 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: finishing write tid 4 to smithi16234619-48 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: finishing write tid 5 to smithi16234619-48 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: oid 48 updating version 0 to 70 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: oid 48 version 70 is already newer than 68 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: oid 48 updating version 70 to 71 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: oid 48 version 71 is already newer than 69 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 71 (ObjNum 116 snap 21 seq_num 116) dirty exists 2024-09-16T15:02:05.777 INFO:tasks.rados.rados.0.smithi162.stdout:239: left oid 48 (ObjNum 116 snap 21 seq_num 116) 2024-09-16T15:02:05.778 INFO:tasks.rados.rados.0.smithi162.stdout:237: finishing write tid 1 to smithi16234619-31 2024-09-16T15:02:05.778 INFO:tasks.rados.rados.0.smithi162.stdout:240: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2024-09-16T15:02:05.782 INFO:tasks.rados.rados.0.smithi162.stdout:240: writing smithi16234619-25 from 766419 to 1315847 tid 1 2024-09-16T15:02:05.789 INFO:tasks.rados.rados.0.smithi162.stdout:240: writing smithi16234619-25 from 2013452 to 2588147 tid 2 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:240: writing smithi16234619-25 from 3346843 to 3774492 tid 3 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:234: done (6 left) 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:235: done (5 left) 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:236: done (4 left) 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:238: done (3 left) 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:239: done (2 left) 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:241: copy_from oid 5 from oid 38 current snap is 21 2024-09-16T15:02:05.796 INFO:tasks.rados.rados.0.smithi162.stdout:237: finishing write tid 2 to smithi16234619-31 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: finishing write tid 3 to smithi16234619-31 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: finishing write tid 4 to smithi16234619-31 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: finishing write tid 5 to smithi16234619-31 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: oid 31 updating version 0 to 47 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: oid 31 updating version 47 to 48 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: oid 31 version 48 is already newer than 46 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: oid 31 version 48 is already newer than 45 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 48 (ObjNum 115 snap 21 seq_num 115) dirty exists 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: left oid 31 (ObjNum 115 snap 21 seq_num 115) 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:237: done (2 left) 2024-09-16T15:02:05.797 INFO:tasks.rados.rados.0.smithi162.stdout:242: snap_create 2024-09-16T15:02:05.818 INFO:tasks.rados.rados.0.smithi162.stdout:240: finishing write tid 1 to smithi16234619-25 2024-09-16T15:02:05.819 INFO:tasks.rados.rados.0.smithi162.stdout:240: finishing write tid 2 to smithi16234619-25 2024-09-16T15:02:05.819 INFO:tasks.rados.rados.0.smithi162.stdout:240: finishing write tid 3 to smithi16234619-25 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: finishing write tid 4 to smithi16234619-25 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: finishing write tid 5 to smithi16234619-25 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: oid 25 updating version 0 to 71 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: oid 25 updating version 71 to 72 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: oid 25 version 72 is already newer than 70 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:240: oid 25 version 72 is already newer than 69 2024-09-16T15:02:05.820 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 72 (ObjNum 117 snap 21 seq_num 117) dirty exists 2024-09-16T15:02:05.821 INFO:tasks.rados.rados.0.smithi162.stdout:240: left oid 25 (ObjNum 117 snap 21 seq_num 117) 2024-09-16T15:02:05.831 INFO:tasks.rados.rados.0.smithi162.stdout:241: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:02:05.836 INFO:tasks.rados.rados.0.smithi162.stdout:241: finishing copy_from to smithi16234619-5 2024-09-16T15:02:05.836 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T15:02:06.367 INFO:tasks.rados.rados.0.smithi162.stdout:240: done (2 left) 2024-09-16T15:02:06.367 INFO:tasks.rados.rados.0.smithi162.stdout:241: done (1 left) 2024-09-16T15:02:06.367 INFO:tasks.rados.rados.0.smithi162.stdout:242: done (0 left) 2024-09-16T15:02:06.367 INFO:tasks.rados.rados.0.smithi162.stdout:243: write oid 40 current snap is 22 2024-09-16T15:02:06.367 INFO:tasks.rados.rados.0.smithi162.stdout:243: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2024-09-16T15:02:06.374 INFO:tasks.rados.rados.0.smithi162.stdout:243: writing smithi16234619-40 from 783226 to 1407903 tid 1 2024-09-16T15:02:06.380 INFO:tasks.rados.rados.0.smithi162.stdout:243: writing smithi16234619-40 from 2071934 to 2506640 tid 2 2024-09-16T15:02:06.386 INFO:tasks.rados.rados.0.smithi162.stdout:243: writing smithi16234619-40 from 2974266 to 3613187 tid 3 2024-09-16T15:02:06.386 INFO:tasks.rados.rados.0.smithi162.stdout:244: write oid 46 current snap is 22 2024-09-16T15:02:06.386 INFO:tasks.rados.rados.0.smithi162.stdout:244: seq_num 119 ranges {400033=699926,1614063=385970} 2024-09-16T15:02:06.392 INFO:tasks.rados.rados.0.smithi162.stdout:244: writing smithi16234619-46 from 400033 to 1099959 tid 1 2024-09-16T15:02:06.397 INFO:tasks.rados.rados.0.smithi162.stdout:244: writing smithi16234619-46 from 1614063 to 2000033 tid 2 2024-09-16T15:02:06.397 INFO:tasks.rados.rados.0.smithi162.stdout:245: write oid 22 current snap is 22 2024-09-16T15:02:06.397 INFO:tasks.rados.rados.0.smithi162.stdout:245: seq_num 120 ranges {416840=775175,1672545=344295} 2024-09-16T15:02:06.403 INFO:tasks.rados.rados.0.smithi162.stdout:245: writing smithi16234619-22 from 416840 to 1192015 tid 1 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:245: writing smithi16234619-22 from 1672545 to 2016840 tid 2 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:246: read oid 30 snap -1 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:246: expect (ObjNum 77 snap 12 seq_num 77) 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:247: copy_from oid 20 from oid 16 current snap is 22 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:248: read oid 18 snap -1 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:248: expect deleted 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:249: copy_from oid 24 from oid 14 current snap is 22 2024-09-16T15:02:06.408 INFO:tasks.rados.rados.0.smithi162.stdout:250: delete oid 6 current snap is 22 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:248: done (7 left) 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:250: done (6 left) 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:251: read oid 25 snap -1 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:251: expect (ObjNum 117 snap 21 seq_num 117) 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:252: read oid 12 snap -1 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:252: expect (ObjNum 84 snap 12 seq_num 84) 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:253: write (excl) oid 44 current snap is 22 2024-09-16T15:02:06.411 INFO:tasks.rados.rados.0.smithi162.stdout:253: seq_num 122 ranges {450454=642026,1505862=544592} 2024-09-16T15:02:06.417 INFO:tasks.rados.rados.0.smithi162.stdout:253: writing smithi16234619-44 from 450454 to 1092480 tid 1 2024-09-16T15:02:06.423 INFO:tasks.rados.rados.0.smithi162.stdout:253: writing smithi16234619-44 from 1505862 to 2050454 tid 2 2024-09-16T15:02:06.423 INFO:tasks.rados.rados.0.smithi162.stdout:254: snap_create 2024-09-16T15:02:06.423 INFO:tasks.rados.rados.0.smithi162.stdout:246: expect (ObjNum 77 snap 12 seq_num 77) 2024-09-16T15:02:06.585 INFO:tasks.rados.rados.0.smithi162.stdout:251: expect (ObjNum 117 snap 21 seq_num 117) 2024-09-16T15:02:06.801 INFO:tasks.rados.rados.0.smithi162.stdout:243: finishing write tid 1 to smithi16234619-40 2024-09-16T15:02:06.801 INFO:tasks.rados.rados.0.smithi162.stdout:243: finishing write tid 2 to smithi16234619-40 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: finishing write tid 1 to smithi16234619-46 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: finishing write tid 2 to smithi16234619-46 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: finishing write tid 3 to smithi16234619-46 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: finishing write tid 4 to smithi16234619-46 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: oid 46 updating version 0 to 31 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: oid 46 version 31 is already newer than 30 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: oid 46 version 31 is already newer than 29 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 31 (ObjNum 119 snap 22 seq_num 119) dirty exists 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:244: left oid 46 (ObjNum 119 snap 22 seq_num 119) 2024-09-16T15:02:06.802 INFO:tasks.rados.rados.0.smithi162.stdout:252: expect (ObjNum 84 snap 12 seq_num 84) 2024-09-16T15:02:06.942 INFO:tasks.rados.rados.0.smithi162.stdout:243: finishing write tid 3 to smithi16234619-40 2024-09-16T15:02:06.942 INFO:tasks.rados.rados.0.smithi162.stdout:243: finishing write tid 4 to smithi16234619-40 2024-09-16T15:02:06.942 INFO:tasks.rados.rados.0.smithi162.stdout:243: finishing write tid 5 to smithi16234619-40 2024-09-16T15:02:06.942 INFO:tasks.rados.rados.0.smithi162.stdout:243: oid 40 updating version 0 to 53 2024-09-16T15:02:06.942 INFO:tasks.rados.rados.0.smithi162.stdout:243: oid 40 updating version 53 to 54 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:243: oid 40 updating version 54 to 55 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:243: oid 40 updating version 55 to 56 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 56 (ObjNum 118 snap 22 seq_num 118) dirty exists 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:243: left oid 40 (ObjNum 118 snap 22 seq_num 118) 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: finishing write tid 1 to smithi16234619-22 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: finishing write tid 2 to smithi16234619-22 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: finishing write tid 3 to smithi16234619-22 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: finishing write tid 4 to smithi16234619-22 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: oid 22 updating version 0 to 43 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: oid 22 updating version 43 to 44 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: oid 22 version 44 is already newer than 42 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 44 (ObjNum 120 snap 22 seq_num 120) dirty exists 2024-09-16T15:02:06.943 INFO:tasks.rados.rados.0.smithi162.stdout:245: left oid 22 (ObjNum 120 snap 22 seq_num 120) 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:247: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:247: finishing copy_from to smithi16234619-20 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: finishing write tid 1 to smithi16234619-44 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: finishing write tid 2 to smithi16234619-44 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: finishing write tid 3 to smithi16234619-44 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: finishing write tid 4 to smithi16234619-44 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: oid 44 updating version 0 to 61 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: oid 44 version 61 is already newer than 60 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: oid 44 updating version 61 to 62 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 62 (ObjNum 122 snap 22 seq_num 122) dirty exists 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:253: left oid 44 (ObjNum 122 snap 22 seq_num 122) 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:249: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:249: finishing copy_from to smithi16234619-24 2024-09-16T15:02:06.944 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 63 (ObjNum 109 snap 19 seq_num 109) dirty exists 2024-09-16T15:02:07.653 INFO:tasks.rados.rados.0.smithi162.stdout:243: done (9 left) 2024-09-16T15:02:07.653 INFO:tasks.rados.rados.0.smithi162.stdout:244: done (8 left) 2024-09-16T15:02:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:245: done (7 left) 2024-09-16T15:02:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:246: done (6 left) 2024-09-16T15:02:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:247: done (5 left) 2024-09-16T15:02:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:249: done (4 left) 2024-09-16T15:02:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:251: done (3 left) 2024-09-16T15:02:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:252: done (2 left) 2024-09-16T15:02:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:253: done (1 left) 2024-09-16T15:02:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:254: done (0 left) 2024-09-16T15:02:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:255: delete oid 27 current snap is 23 2024-09-16T15:02:07.656 INFO:tasks.rados.rados.0.smithi162.stdout:255: done (0 left) 2024-09-16T15:02:07.656 INFO:tasks.rados.rados.0.smithi162.stdout:256: snap_create 2024-09-16T15:02:08.657 INFO:tasks.rados.rados.0.smithi162.stdout:256: done (0 left) 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:257: read oid 11 snap -1 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:257: expect (ObjNum 107 snap 19 seq_num 107) 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:258: read oid 27 snap -1 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:258: expect deleted 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:259: read oid 40 snap -1 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:259: expect (ObjNum 118 snap 22 seq_num 118) 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:260: copy_from oid 44 from oid 36 current snap is 24 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:261: write oid 48 current snap is 24 2024-09-16T15:02:08.658 INFO:tasks.rados.rados.0.smithi162.stdout:261: seq_num 124 ranges {484068=792524,1906473=177595} 2024-09-16T15:02:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:261: writing smithi16234619-48 from 484068 to 1276592 tid 1 2024-09-16T15:02:08.669 INFO:tasks.rados.rados.0.smithi162.stdout:261: writing smithi16234619-48 from 1906473 to 2084068 tid 2 2024-09-16T15:02:08.670 INFO:tasks.rados.rados.0.smithi162.stdout:262: copy_from oid 14 from oid 24 current snap is 24 2024-09-16T15:02:08.670 INFO:tasks.rados.rados.0.smithi162.stdout:258: done (5 left) 2024-09-16T15:02:08.670 INFO:tasks.rados.rados.0.smithi162.stdout:263: read oid 50 snap -1 2024-09-16T15:02:08.670 INFO:tasks.rados.rados.0.smithi162.stdout:263: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-16T15:02:08.670 INFO:tasks.rados.rados.0.smithi162.stdout:264: snap_remove snap 22 2024-09-16T15:02:09.666 INFO:tasks.rados.rados.0.smithi162.stdout:264: done (6 left) 2024-09-16T15:02:09.666 INFO:tasks.rados.rados.0.smithi162.stdout:265: delete oid 7 current snap is 24 2024-09-16T15:02:09.666 INFO:tasks.rados.rados.0.smithi162.stdout:257: expect (ObjNum 107 snap 19 seq_num 107) 2024-09-16T15:02:09.790 INFO:tasks.rados.rados.0.smithi162.stdout:259: expect (ObjNum 118 snap 22 seq_num 118) 2024-09-16T15:02:10.006 INFO:tasks.rados.rados.0.smithi162.stdout:263: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: finishing write tid 1 to smithi16234619-48 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: finishing write tid 2 to smithi16234619-48 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: finishing write tid 3 to smithi16234619-48 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: finishing write tid 4 to smithi16234619-48 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: oid 48 updating version 0 to 76 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: oid 48 updating version 76 to 77 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:261: oid 48 version 77 is already newer than 75 2024-09-16T15:02:10.170 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 77 (ObjNum 124 snap 24 seq_num 124) dirty exists 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:261: left oid 48 (ObjNum 124 snap 24 seq_num 124) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:262: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:262: finishing copy_from to smithi16234619-14 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 63 (ObjNum 109 snap 19 seq_num 109) dirty exists 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:260: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:257: done (6 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:259: done (5 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:261: done (4 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:262: done (3 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:263: done (2 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:265: done (1 left) 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:266: write oid 28 current snap is 24 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:260: finishing copy_from to smithi16234619-44 2024-09-16T15:02:10.171 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 42 (ObjNum 88 snap 14 seq_num 88) dirty exists 2024-09-16T15:02:10.172 INFO:tasks.rados.rados.0.smithi162.stdout:266: seq_num 126 ranges {517682=543022,1623437=494245} 2024-09-16T15:02:10.175 INFO:tasks.rados.rados.0.smithi162.stdout:266: writing smithi16234619-28 from 517682 to 1060704 tid 1 2024-09-16T15:02:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:266: writing smithi16234619-28 from 1623437 to 2117682 tid 2 2024-09-16T15:02:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:260: done (1 left) 2024-09-16T15:02:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:267: write oid 36 current snap is 24 2024-09-16T15:02:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:267: seq_num 127 ranges {534489=618271,1565566=568923} 2024-09-16T15:02:10.187 INFO:tasks.rados.rados.0.smithi162.stdout:267: writing smithi16234619-36 from 534489 to 1152760 tid 1 2024-09-16T15:02:10.192 INFO:tasks.rados.rados.0.smithi162.stdout:267: writing smithi16234619-36 from 1565566 to 2134489 tid 2 2024-09-16T15:02:10.192 INFO:tasks.rados.rados.0.smithi162.stdout:268: write (excl) oid 18 current snap is 24 2024-09-16T15:02:10.192 INFO:tasks.rados.rados.0.smithi162.stdout:268: seq_num 128 ranges {551296=693520,2024048=127248} 2024-09-16T15:02:10.198 INFO:tasks.rados.rados.0.smithi162.stdout:268: writing smithi16234619-18 from 551296 to 1244816 tid 1 2024-09-16T15:02:10.204 INFO:tasks.rados.rados.0.smithi162.stdout:268: writing smithi16234619-18 from 2024048 to 2151296 tid 2 2024-09-16T15:02:10.204 INFO:tasks.rados.rados.0.smithi162.stdout:269: write (excl) oid 47 current snap is 24 2024-09-16T15:02:10.204 INFO:tasks.rados.rados.0.smithi162.stdout:269: seq_num 129 ranges {568103=768769,2082530=85573} 2024-09-16T15:02:10.210 INFO:tasks.rados.rados.0.smithi162.stdout:269: writing smithi16234619-47 from 568103 to 1336872 tid 1 2024-09-16T15:02:10.215 INFO:tasks.rados.rados.0.smithi162.stdout:269: writing smithi16234619-47 from 2082530 to 2168103 tid 2 2024-09-16T15:02:10.215 INFO:tasks.rados.rados.0.smithi162.stdout:270: write oid 48 current snap is 24 2024-09-16T15:02:10.216 INFO:tasks.rados.rados.0.smithi162.stdout:266: finishing write tid 1 to smithi16234619-28 2024-09-16T15:02:10.216 INFO:tasks.rados.rados.0.smithi162.stdout:266: finishing write tid 2 to smithi16234619-28 2024-09-16T15:02:10.216 INFO:tasks.rados.rados.0.smithi162.stdout:270: seq_num 130 ranges {584910=560371,1857365=327545} 2024-09-16T15:02:10.223 INFO:tasks.rados.rados.0.smithi162.stdout:270: writing smithi16234619-48 from 584910 to 1145281 tid 1 2024-09-16T15:02:10.230 INFO:tasks.rados.rados.0.smithi162.stdout:270: writing smithi16234619-48 from 1857365 to 2184910 tid 2 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:271: rollback oid 42 current snap is 24 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 21 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: finishing write tid 3 to smithi16234619-28 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: finishing write tid 4 to smithi16234619-28 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: oid 28 updating version 0 to 43 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: oid 28 version 43 is already newer than 42 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: oid 28 version 43 is already newer than 41 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 43 (ObjNum 126 snap 24 seq_num 126) dirty exists 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:266: left oid 28 (ObjNum 126 snap 24 seq_num 126) 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: finishing write tid 1 to smithi16234619-18 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: finishing write tid 2 to smithi16234619-18 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: finishing write tid 3 to smithi16234619-18 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: finishing write tid 4 to smithi16234619-18 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: oid 18 updating version 0 to 52 2024-09-16T15:02:10.231 INFO:tasks.rados.rados.0.smithi162.stdout:268: oid 18 updating version 52 to 53 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:268: oid 18 version 53 is already newer than 51 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 53 (ObjNum 128 snap 24 seq_num 128) dirty exists 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:268: left oid 18 (ObjNum 128 snap 24 seq_num 128) 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:267: finishing write tid 1 to smithi16234619-36 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:266: done (5 left) 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:268: done (4 left) 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:272: read oid 50 snap -1 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:272: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:273: rollback oid 14 current snap is 24 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 21 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:274: rollback oid 32 current snap is 24 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 14 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:275: write oid 27 current snap is 24 2024-09-16T15:02:10.232 INFO:tasks.rados.rados.0.smithi162.stdout:275: seq_num 131 ranges {601717=635620,1799494=402223} 2024-09-16T15:02:10.237 INFO:tasks.rados.rados.0.smithi162.stdout:275: writing smithi16234619-27 from 601717 to 1237337 tid 1 2024-09-16T15:02:10.243 INFO:tasks.rados.rados.0.smithi162.stdout:275: writing smithi16234619-27 from 1799494 to 2201717 tid 2 2024-09-16T15:02:10.243 INFO:tasks.rados.rados.0.smithi162.stdout:276: write oid 44 current snap is 24 2024-09-16T15:02:10.243 INFO:tasks.rados.rados.0.smithi162.stdout:276: seq_num 132 ranges {618524=710869,1857976=360548} 2024-09-16T15:02:10.250 INFO:tasks.rados.rados.0.smithi162.stdout:276: writing smithi16234619-44 from 618524 to 1329393 tid 1 2024-09-16T15:02:10.256 INFO:tasks.rados.rados.0.smithi162.stdout:276: writing smithi16234619-44 from 1857976 to 2218524 tid 2 2024-09-16T15:02:10.256 INFO:tasks.rados.rados.0.smithi162.stdout:277: copy_from oid 19 from oid 29 current snap is 24 2024-09-16T15:02:10.256 INFO:tasks.rados.rados.0.smithi162.stdout:271: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:02:10.257 INFO:tasks.rados.rados.0.smithi162.stdout:272: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-16T15:02:10.431 INFO:tasks.rados.rados.0.smithi162.stdout:274: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:02:10.431 INFO:tasks.rados.rados.0.smithi162.stdout:274: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:02:10.431 INFO:tasks.rados.rados.0.smithi162.stdout:269: finishing write tid 1 to smithi16234619-47 2024-09-16T15:02:10.431 INFO:tasks.rados.rados.0.smithi162.stdout:274: finishing rollback tid 2 to smithi16234619-32 2024-09-16T15:02:10.432 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 59 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T15:02:10.432 INFO:tasks.rados.rados.0.smithi162.stdout:272: done (9 left) 2024-09-16T15:02:10.432 INFO:tasks.rados.rados.0.smithi162.stdout:274: done (8 left) 2024-09-16T15:02:10.432 INFO:tasks.rados.rados.0.smithi162.stdout:278: write (excl) oid 26 current snap is 24 2024-09-16T15:02:10.432 INFO:tasks.rados.rados.0.smithi162.stdout:278: seq_num 133 ranges {635331=786118,1916458=318873} 2024-09-16T15:02:10.438 INFO:tasks.rados.rados.0.smithi162.stdout:278: writing smithi16234619-26 from 635331 to 1421449 tid 1 2024-09-16T15:02:10.444 INFO:tasks.rados.rados.0.smithi162.stdout:278: writing smithi16234619-26 from 1916458 to 2235331 tid 2 2024-09-16T15:02:10.444 INFO:tasks.rados.rados.0.smithi162.stdout:279: rollback oid 31 current snap is 24 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 14 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:271: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: finishing write tid 2 to smithi16234619-36 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: finishing write tid 3 to smithi16234619-36 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:271: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 78 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:269: finishing write tid 2 to smithi16234619-47 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:269: finishing write tid 3 to smithi16234619-47 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: finishing write tid 4 to smithi16234619-36 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: oid 36 updating version 0 to 80 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: oid 36 version 80 is already newer than 78 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: oid 36 version 80 is already newer than 79 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:02:10.445 INFO:tasks.rados.rados.0.smithi162.stdout:267: left oid 36 (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:273: finishing rollback tid 0 to smithi16234619-14 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:269: finishing write tid 4 to smithi16234619-47 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:269: oid 47 updating version 0 to 24 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:269: oid 47 updating version 24 to 25 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:269: oid 47 updating version 25 to 26 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 26 (ObjNum 129 snap 24 seq_num 129) dirty exists 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:269: left oid 47 (ObjNum 129 snap 24 seq_num 129) 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:273: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:273: finishing rollback tid 2 to smithi16234619-14 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 83 (ObjNum 109 snap 19 seq_num 109) dirty exists 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:270: finishing write tid 1 to smithi16234619-48 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:270: finishing write tid 2 to smithi16234619-48 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:270: finishing write tid 3 to smithi16234619-48 2024-09-16T15:02:10.446 INFO:tasks.rados.rados.0.smithi162.stdout:267: done (9 left) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:269: done (8 left) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:271: done (7 left) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:273: done (6 left) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:280: read oid 49 snap -1 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:280: expect (ObjNum 76 snap 12 seq_num 76) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:281: read oid 23 snap 23 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:281: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:282: rollback oid 40 current snap is 24 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 40 to 23 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:270: finishing write tid 4 to smithi16234619-48 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:270: oid 48 updating version 0 to 83 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:270: oid 48 version 83 is already newer than 82 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:270: oid 48 updating version 83 to 84 2024-09-16T15:02:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:270: left oid 48 (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: finishing write tid 1 to smithi16234619-27 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: finishing write tid 2 to smithi16234619-27 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: finishing write tid 3 to smithi16234619-27 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: finishing write tid 4 to smithi16234619-27 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: oid 27 updating version 0 to 86 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: oid 27 version 86 is already newer than 85 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: oid 27 updating version 86 to 87 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 87 (ObjNum 131 snap 24 seq_num 131) dirty exists 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:275: left oid 27 (ObjNum 131 snap 24 seq_num 131) 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:276: finishing write tid 1 to smithi16234619-44 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:276: finishing write tid 2 to smithi16234619-44 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:276: finishing write tid 3 to smithi16234619-44 2024-09-16T15:02:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:276: finishing write tid 4 to smithi16234619-44 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:276: oid 44 updating version 0 to 68 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:276: oid 44 updating version 68 to 69 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:276: oid 44 version 69 is already newer than 67 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 69 (ObjNum 132 snap 24 seq_num 132) dirty exists 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:276: left oid 44 (ObjNum 132 snap 24 seq_num 132) 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:277: finishing copy_from racing read to smithi16234619-19 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:277: finishing copy_from to smithi16234619-19 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 27 (ObjNum 67 snap 10 seq_num 67) dirty exists 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:270: done (8 left) 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:275: done (7 left) 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:276: done (6 left) 2024-09-16T15:02:10.449 INFO:tasks.rados.rados.0.smithi162.stdout:277: done (5 left) 2024-09-16T15:02:10.450 INFO:tasks.rados.rados.0.smithi162.stdout:283: delete oid 25 current snap is 24 2024-09-16T15:02:10.455 INFO:tasks.rados.rados.0.smithi162.stdout:281: expect (ObjNum 56 snap 1 seq_num 56) 2024-09-16T15:02:10.626 INFO:tasks.rados.rados.0.smithi162.stdout:282: finishing rollback tid 0 to smithi16234619-40 2024-09-16T15:02:10.626 INFO:tasks.rados.rados.0.smithi162.stdout:282: finishing rollback tid 1 to smithi16234619-40 2024-09-16T15:02:10.626 INFO:tasks.rados.rados.0.smithi162.stdout:281: done (5 left) 2024-09-16T15:02:10.626 INFO:tasks.rados.rados.0.smithi162.stdout:283: done (4 left) 2024-09-16T15:02:10.626 INFO:tasks.rados.rados.0.smithi162.stdout:284: snap_remove snap 21 2024-09-16T15:02:11.370 INFO:tasks.rados.rados.0.smithi162.stdout:284: done (4 left) 2024-09-16T15:02:11.370 INFO:tasks.rados.rados.0.smithi162.stdout:285: write oid 47 current snap is 24 2024-09-16T15:02:11.370 INFO:tasks.rados.rados.0.smithi162.stdout:285: seq_num 135 ranges {668945=536616,1633422=491363} 2024-09-16T15:02:11.376 INFO:tasks.rados.rados.0.smithi162.stdout:285: writing smithi16234619-47 from 668945 to 1205561 tid 1 2024-09-16T15:02:11.382 INFO:tasks.rados.rados.0.smithi162.stdout:285: writing smithi16234619-47 from 1633422 to 2124785 tid 2 2024-09-16T15:02:11.382 INFO:tasks.rados.rados.0.smithi162.stdout:286: snap_remove snap 19 2024-09-16T15:02:12.372 INFO:tasks.rados.rados.0.smithi162.stdout:286: done (5 left) 2024-09-16T15:02:12.372 INFO:tasks.rados.rados.0.smithi162.stdout:287: write oid 23 current snap is 24 2024-09-16T15:02:12.372 INFO:tasks.rados.rados.0.smithi162.stdout:287: seq_num 136 ranges {685752=611865,1975551=310201} 2024-09-16T15:02:12.380 INFO:tasks.rados.rados.0.smithi162.stdout:287: writing smithi16234619-23 from 685752 to 1297617 tid 1 2024-09-16T15:02:12.386 INFO:tasks.rados.rados.0.smithi162.stdout:287: writing smithi16234619-23 from 1975551 to 2285752 tid 2 2024-09-16T15:02:12.386 INFO:tasks.rados.rados.0.smithi162.stdout:288: write oid 9 current snap is 24 2024-09-16T15:02:12.386 INFO:tasks.rados.rados.0.smithi162.stdout:288: seq_num 137 ranges {702559=403467,1750386=495032} 2024-09-16T15:02:12.392 INFO:tasks.rados.rados.0.smithi162.stdout:288: writing smithi16234619-9 from 702559 to 1106026 tid 1 2024-09-16T15:02:12.398 INFO:tasks.rados.rados.0.smithi162.stdout:288: writing smithi16234619-9 from 1750386 to 2245418 tid 2 2024-09-16T15:02:12.399 INFO:tasks.rados.rados.0.smithi162.stdout:289: copy_from oid 35 from oid 36 current snap is 24 2024-09-16T15:02:12.399 INFO:tasks.rados.rados.0.smithi162.stdout:282: finishing rollback tid 2 to smithi16234619-40 2024-09-16T15:02:12.399 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 62 (ObjNum 118 snap 22 seq_num 118) dirty exists 2024-09-16T15:02:12.399 INFO:tasks.rados.rados.0.smithi162.stdout:280: expect (ObjNum 76 snap 12 seq_num 76) 2024-09-16T15:02:12.509 INFO:tasks.rados.rados.0.smithi162.stdout:279: finishing rollback tid 0 to smithi16234619-31 2024-09-16T15:02:12.509 INFO:tasks.rados.rados.0.smithi162.stdout:279: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:02:12.509 INFO:tasks.rados.rados.0.smithi162.stdout:279: finishing rollback tid 2 to smithi16234619-31 2024-09-16T15:02:12.509 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 57 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-16T15:02:12.509 INFO:tasks.rados.rados.0.smithi162.stdout:278: finishing write tid 1 to smithi16234619-26 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: finishing write tid 2 to smithi16234619-26 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:279: done (7 left) 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:280: done (6 left) 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:282: done (5 left) 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:290: write oid 11 current snap is 24 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: finishing write tid 3 to smithi16234619-26 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: finishing write tid 4 to smithi16234619-26 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: oid 26 updating version 0 to 69 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: oid 26 version 69 is already newer than 68 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: oid 26 version 69 is already newer than 67 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 69 (ObjNum 133 snap 24 seq_num 133) dirty exists 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:278: left oid 26 (ObjNum 133 snap 24 seq_num 133) 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:285: finishing write tid 1 to smithi16234619-47 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:285: finishing write tid 2 to smithi16234619-47 2024-09-16T15:02:12.510 INFO:tasks.rados.rados.0.smithi162.stdout:285: finishing write tid 3 to smithi16234619-47 2024-09-16T15:02:12.511 INFO:tasks.rados.rados.0.smithi162.stdout:290: seq_num 138 ranges {719366=478716,1808868=510498} 2024-09-16T15:02:12.515 INFO:tasks.rados.rados.0.smithi162.stdout:290: writing smithi16234619-11 from 719366 to 1198082 tid 1 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:290: writing smithi16234619-11 from 1808868 to 2319366 tid 2 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:278: done (5 left) 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:291: snap_create 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:285: finishing write tid 4 to smithi16234619-47 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:285: oid 47 updating version 0 to 29 2024-09-16T15:02:12.522 INFO:tasks.rados.rados.0.smithi162.stdout:285: oid 47 version 29 is already newer than 28 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:285: oid 47 version 29 is already newer than 27 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 29 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:285: left oid 47 (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:287: finishing write tid 1 to smithi16234619-23 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:287: finishing write tid 2 to smithi16234619-23 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: finishing write tid 1 to smithi16234619-9 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: finishing write tid 2 to smithi16234619-9 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: finishing write tid 3 to smithi16234619-9 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: finishing write tid 4 to smithi16234619-9 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: oid 9 updating version 0 to 99 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: oid 9 updating version 99 to 101 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: oid 9 version 101 is already newer than 100 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 101 (ObjNum 137 snap 24 seq_num 137) dirty exists 2024-09-16T15:02:12.523 INFO:tasks.rados.rados.0.smithi162.stdout:288: left oid 9 (ObjNum 137 snap 24 seq_num 137) 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: finishing write tid 3 to smithi16234619-23 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: finishing write tid 4 to smithi16234619-23 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: oid 23 updating version 0 to 20 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: oid 23 version 20 is already newer than 18 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: oid 23 version 20 is already newer than 19 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 20 (ObjNum 136 snap 24 seq_num 136) dirty exists 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:287: left oid 23 (ObjNum 136 snap 24 seq_num 136) 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:289: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:289: finishing copy_from to smithi16234619-35 2024-09-16T15:02:12.524 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:02:12.540 INFO:tasks.rados.rados.0.smithi162.stdout:290: finishing write tid 1 to smithi16234619-11 2024-09-16T15:02:12.540 INFO:tasks.rados.rados.0.smithi162.stdout:290: finishing write tid 2 to smithi16234619-11 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: finishing write tid 3 to smithi16234619-11 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: finishing write tid 4 to smithi16234619-11 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: oid 11 updating version 0 to 20 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: oid 11 version 20 is already newer than 19 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: oid 11 updating version 20 to 21 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 21 (ObjNum 138 snap 24 seq_num 138) dirty exists 2024-09-16T15:02:12.541 INFO:tasks.rados.rados.0.smithi162.stdout:290: left oid 11 (ObjNum 138 snap 24 seq_num 138) 2024-09-16T15:02:13.373 INFO:tasks.rados.rados.0.smithi162.stdout:285: done (5 left) 2024-09-16T15:02:13.373 INFO:tasks.rados.rados.0.smithi162.stdout:287: done (4 left) 2024-09-16T15:02:13.373 INFO:tasks.rados.rados.0.smithi162.stdout:288: done (3 left) 2024-09-16T15:02:13.373 INFO:tasks.rados.rados.0.smithi162.stdout:289: done (2 left) 2024-09-16T15:02:13.373 INFO:tasks.rados.rados.0.smithi162.stdout:290: done (1 left) 2024-09-16T15:02:13.374 INFO:tasks.rados.rados.0.smithi162.stdout:291: done (0 left) 2024-09-16T15:02:13.374 INFO:tasks.rados.rados.0.smithi162.stdout:292: read oid 5 snap 14 2024-09-16T15:02:13.374 INFO:tasks.rados.rados.0.smithi162.stdout:292: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T15:02:13.374 INFO:tasks.rados.rados.0.smithi162.stdout:293: copy_from oid 28 from oid 41 current snap is 25 2024-09-16T15:02:13.374 INFO:tasks.rados.rados.0.smithi162.stdout:294: snap_create 2024-09-16T15:02:13.378 INFO:tasks.rados.rados.0.smithi162.stdout:292: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T15:02:13.445 INFO:tasks.rados.rados.0.smithi162.stdout:293: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:02:13.445 INFO:tasks.rados.rados.0.smithi162.stdout:293: finishing copy_from to smithi16234619-28 2024-09-16T15:02:13.445 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T15:02:14.735 INFO:tasks.rados.rados.0.smithi162.stdout:292: done (2 left) 2024-09-16T15:02:14.735 INFO:tasks.rados.rados.0.smithi162.stdout:293: done (1 left) 2024-09-16T15:02:14.735 INFO:tasks.rados.rados.0.smithi162.stdout:294: done (0 left) 2024-09-16T15:02:14.735 INFO:tasks.rados.rados.0.smithi162.stdout:295: delete oid 19 current snap is 26 2024-09-16T15:02:14.738 INFO:tasks.rados.rados.0.smithi162.stdout:295: done (0 left) 2024-09-16T15:02:14.738 INFO:tasks.rados.rados.0.smithi162.stdout:296: copy_from oid 5 from oid 16 current snap is 26 2024-09-16T15:02:14.738 INFO:tasks.rados.rados.0.smithi162.stdout:297: snap_remove snap 14 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:297: done (1 left) 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:298: copy_from oid 49 from oid 10 current snap is 26 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:296: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:296: finishing copy_from to smithi16234619-5 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-16T15:02:15.361 INFO:tasks.rados.rados.0.smithi162.stdout:296: done (1 left) 2024-09-16T15:02:15.362 INFO:tasks.rados.rados.0.smithi162.stdout:299: copy_from oid 6 from oid 48 current snap is 26 2024-09-16T15:02:15.362 INFO:tasks.rados.rados.0.smithi162.stdout:300: write (excl) oid 14 current snap is 26 2024-09-16T15:02:15.362 INFO:tasks.rados.rados.0.smithi162.stdout:300: seq_num 140 ranges {752980=629214,1809479=543501} 2024-09-16T15:02:15.369 INFO:tasks.rados.rados.0.smithi162.stdout:300: writing smithi16234619-14 from 752980 to 1382194 tid 1 2024-09-16T15:02:15.375 INFO:tasks.rados.rados.0.smithi162.stdout:300: writing smithi16234619-14 from 1809479 to 2352980 tid 2 2024-09-16T15:02:15.375 INFO:tasks.rados.rados.0.smithi162.stdout:301: snap_remove snap 25 2024-09-16T15:02:16.685 INFO:tasks.rados.rados.0.smithi162.stdout:301: done (3 left) 2024-09-16T15:02:16.685 INFO:tasks.rados.rados.0.smithi162.stdout:302: rollback oid 46 current snap is 26 2024-09-16T15:02:16.685 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 23 2024-09-16T15:02:16.685 INFO:tasks.rados.rados.0.smithi162.stdout:300: finishing write tid 1 to smithi16234619-14 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: finishing write tid 2 to smithi16234619-14 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: finishing write tid 3 to smithi16234619-14 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: finishing write tid 4 to smithi16234619-14 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: oid 14 updating version 0 to 112 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: oid 14 version 112 is already newer than 111 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: oid 14 updating version 112 to 113 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 113 (ObjNum 140 snap 26 seq_num 140) dirty exists 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:300: left oid 14 (ObjNum 140 snap 26 seq_num 140) 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:299: finishing copy_from racing read to smithi16234619-6 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:299: finishing copy_from to smithi16234619-6 2024-09-16T15:02:16.686 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:298: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:298: finishing copy_from to smithi16234619-49 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 63 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:298: done (3 left) 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:299: done (2 left) 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:300: done (1 left) 2024-09-16T15:02:16.687 INFO:tasks.rados.rados.0.smithi162.stdout:303: snap_remove snap 24 2024-09-16T15:02:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:303: done (1 left) 2024-09-16T15:02:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:304: snap_remove snap 20 2024-09-16T15:02:18.710 INFO:tasks.rados.rados.0.smithi162.stdout:304: done (1 left) 2024-09-16T15:02:18.711 INFO:tasks.rados.rados.0.smithi162.stdout:305: snap_create 2024-09-16T15:02:18.711 INFO:tasks.rados.rados.0.smithi162.stdout:302: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:02:18.711 INFO:tasks.rados.rados.0.smithi162.stdout:302: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:02:18.711 INFO:tasks.rados.rados.0.smithi162.stdout:302: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:02:18.711 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 34 (ObjNum 119 snap 22 seq_num 119) dirty exists 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:302: done (1 left) 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:305: done (0 left) 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:306: read oid 28 snap 26 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:306: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:307: write oid 50 current snap is 27 2024-09-16T15:02:19.719 INFO:tasks.rados.rados.0.smithi162.stdout:307: seq_num 141 ranges {769787=704463,2267961=101826} 2024-09-16T15:02:19.726 INFO:tasks.rados.rados.0.smithi162.stdout:307: writing smithi16234619-50 from 769787 to 1474250 tid 1 2024-09-16T15:02:19.732 INFO:tasks.rados.rados.0.smithi162.stdout:307: writing smithi16234619-50 from 2267961 to 2369787 tid 2 2024-09-16T15:02:19.732 INFO:tasks.rados.rados.0.smithi162.stdout:308: copy_from oid 39 from oid 44 current snap is 27 2024-09-16T15:02:19.732 INFO:tasks.rados.rados.0.smithi162.stdout:306: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:19.827 INFO:tasks.rados.rados.0.smithi162.stdout:307: finishing write tid 1 to smithi16234619-50 2024-09-16T15:02:19.827 INFO:tasks.rados.rados.0.smithi162.stdout:307: finishing write tid 2 to smithi16234619-50 2024-09-16T15:02:19.827 INFO:tasks.rados.rados.0.smithi162.stdout:307: finishing write tid 3 to smithi16234619-50 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: finishing write tid 4 to smithi16234619-50 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: oid 50 updating version 0 to 137 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: oid 50 version 137 is already newer than 136 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: oid 50 version 137 is already newer than 135 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 137 (ObjNum 141 snap 27 seq_num 141) dirty exists 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: left oid 50 (ObjNum 141 snap 27 seq_num 141) 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:308: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:308: finishing copy_from to smithi16234619-39 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 69 (ObjNum 132 snap 24 seq_num 132) dirty exists 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:306: done (2 left) 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:307: done (1 left) 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:308: done (0 left) 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:309: copy_from oid 17 from oid 10 current snap is 27 2024-09-16T15:02:19.828 INFO:tasks.rados.rados.0.smithi162.stdout:310: read oid 48 snap -1 2024-09-16T15:02:19.829 INFO:tasks.rados.rados.0.smithi162.stdout:310: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:02:19.829 INFO:tasks.rados.rados.0.smithi162.stdout:311: write (excl) oid 32 current snap is 27 2024-09-16T15:02:19.829 INFO:tasks.rados.rados.0.smithi162.stdout:311: seq_num 142 ranges {786594=779712,2326443=60151} 2024-09-16T15:02:19.835 INFO:tasks.rados.rados.0.smithi162.stdout:311: writing smithi16234619-32 from 786594 to 1566306 tid 1 2024-09-16T15:02:19.840 INFO:tasks.rados.rados.0.smithi162.stdout:311: writing smithi16234619-32 from 2326443 to 2386594 tid 2 2024-09-16T15:02:19.840 INFO:tasks.rados.rados.0.smithi162.stdout:312: copy_from oid 21 from oid 15 current snap is 27 2024-09-16T15:02:19.840 INFO:tasks.rados.rados.0.smithi162.stdout:310: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: finishing write tid 1 to smithi16234619-32 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: finishing write tid 2 to smithi16234619-32 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: finishing write tid 3 to smithi16234619-32 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: finishing write tid 4 to smithi16234619-32 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: oid 32 updating version 0 to 72 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: oid 32 updating version 72 to 74 2024-09-16T15:02:19.978 INFO:tasks.rados.rados.0.smithi162.stdout:311: oid 32 version 74 is already newer than 73 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 74 (ObjNum 142 snap 27 seq_num 142) dirty exists 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:311: left oid 32 (ObjNum 142 snap 27 seq_num 142) 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:309: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:310: done (3 left) 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:311: done (2 left) 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:313: write (excl) oid 30 current snap is 27 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:312: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:309: finishing copy_from to smithi16234619-17 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 63 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:312: finishing copy_from to smithi16234619-21 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 13 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-16T15:02:19.979 INFO:tasks.rados.rados.0.smithi162.stdout:313: seq_num 143 ranges {403401=454961,1584925=506039} 2024-09-16T15:02:19.982 INFO:tasks.rados.rados.0.smithi162.stdout:313: writing smithi16234619-30 from 403401 to 858362 tid 1 2024-09-16T15:02:19.989 INFO:tasks.rados.rados.0.smithi162.stdout:313: writing smithi16234619-30 from 1584925 to 2090964 tid 2 2024-09-16T15:02:19.989 INFO:tasks.rados.rados.0.smithi162.stdout:309: done (2 left) 2024-09-16T15:02:19.989 INFO:tasks.rados.rados.0.smithi162.stdout:312: done (1 left) 2024-09-16T15:02:19.989 INFO:tasks.rados.rados.0.smithi162.stdout:314: write (excl) oid 42 current snap is 27 2024-09-16T15:02:19.989 INFO:tasks.rados.rados.0.smithi162.stdout:314: seq_num 144 ranges {420208=530210,1527054=766050} 2024-09-16T15:02:19.994 INFO:tasks.rados.rados.0.smithi162.stdout:314: writing smithi16234619-42 from 420208 to 950418 tid 1 2024-09-16T15:02:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:314: writing smithi16234619-42 from 1527054 to 2293104 tid 2 2024-09-16T15:02:20.002 INFO:tasks.rados.rados.0.smithi162.stdout:315: delete oid 45 current snap is 27 2024-09-16T15:02:20.005 INFO:tasks.rados.rados.0.smithi162.stdout:315: done (2 left) 2024-09-16T15:02:20.005 INFO:tasks.rados.rados.0.smithi162.stdout:316: snap_create 2024-09-16T15:02:20.006 INFO:tasks.rados.rados.0.smithi162.stdout:313: finishing write tid 1 to smithi16234619-30 2024-09-16T15:02:20.007 INFO:tasks.rados.rados.0.smithi162.stdout:313: finishing write tid 2 to smithi16234619-30 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: finishing write tid 3 to smithi16234619-30 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: finishing write tid 4 to smithi16234619-30 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: oid 30 updating version 0 to 55 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: oid 30 version 55 is already newer than 54 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: oid 30 version 55 is already newer than 53 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 55 (ObjNum 143 snap 27 seq_num 143) dirty exists 2024-09-16T15:02:20.008 INFO:tasks.rados.rados.0.smithi162.stdout:313: left oid 30 (ObjNum 143 snap 27 seq_num 143) 2024-09-16T15:02:20.024 INFO:tasks.rados.rados.0.smithi162.stdout:314: finishing write tid 1 to smithi16234619-42 2024-09-16T15:02:20.026 INFO:tasks.rados.rados.0.smithi162.stdout:314: finishing write tid 2 to smithi16234619-42 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: finishing write tid 3 to smithi16234619-42 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: finishing write tid 4 to smithi16234619-42 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: oid 42 updating version 0 to 103 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: oid 42 version 103 is already newer than 102 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: oid 42 updating version 103 to 104 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 104 (ObjNum 144 snap 27 seq_num 144) dirty exists 2024-09-16T15:02:20.027 INFO:tasks.rados.rados.0.smithi162.stdout:314: left oid 42 (ObjNum 144 snap 27 seq_num 144) 2024-09-16T15:02:20.362 INFO:tasks.rados.rados.0.smithi162.stdout:313: done (2 left) 2024-09-16T15:02:20.362 INFO:tasks.rados.rados.0.smithi162.stdout:314: done (1 left) 2024-09-16T15:02:20.362 INFO:tasks.rados.rados.0.smithi162.stdout:316: done (0 left) 2024-09-16T15:02:20.362 INFO:tasks.rados.rados.0.smithi162.stdout:317: rollback oid 8 current snap is 28 2024-09-16T15:02:20.363 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 23 2024-09-16T15:02:20.363 INFO:tasks.rados.rados.0.smithi162.stdout:318: write oid 17 current snap is 28 2024-09-16T15:02:20.363 INFO:tasks.rados.rados.0.smithi162.stdout:318: seq_num 146 ranges {453822=797061,1760371=653366} 2024-09-16T15:02:20.369 INFO:tasks.rados.rados.0.smithi162.stdout:318: writing smithi16234619-17 from 453822 to 1250883 tid 1 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:318: writing smithi16234619-17 from 1760371 to 2413737 tid 2 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:319: delete oid 11 current snap is 28 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:317: finishing rollback tid 0 to smithi16234619-8 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:317: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:317: finishing rollback tid 2 to smithi16234619-8 2024-09-16T15:02:20.376 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 81 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T15:02:20.378 INFO:tasks.rados.rados.0.smithi162.stdout:317: done (2 left) 2024-09-16T15:02:20.378 INFO:tasks.rados.rados.0.smithi162.stdout:319: done (1 left) 2024-09-16T15:02:20.378 INFO:tasks.rados.rados.0.smithi162.stdout:320: write oid 13 current snap is 28 2024-09-16T15:02:20.378 INFO:tasks.rados.rados.0.smithi162.stdout:320: seq_num 148 ranges {487436=547559,1760982=657035} 2024-09-16T15:02:20.383 INFO:tasks.rados.rados.0.smithi162.stdout:320: writing smithi16234619-13 from 487436 to 1034995 tid 1 2024-09-16T15:02:20.393 INFO:tasks.rados.rados.0.smithi162.stdout:320: writing smithi16234619-13 from 1760982 to 2418017 tid 2 2024-09-16T15:02:20.393 INFO:tasks.rados.rados.0.smithi162.stdout:321: read oid 41 snap -1 2024-09-16T15:02:20.393 INFO:tasks.rados.rados.0.smithi162.stdout:321: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:20.393 INFO:tasks.rados.rados.0.smithi162.stdout:322: write oid 29 current snap is 28 2024-09-16T15:02:20.393 INFO:tasks.rados.rados.0.smithi162.stdout:322: seq_num 149 ranges {504243=622808,1819464=517046} 2024-09-16T15:02:20.401 INFO:tasks.rados.rados.0.smithi162.stdout:322: writing smithi16234619-29 from 504243 to 1127051 tid 1 2024-09-16T15:02:20.408 INFO:tasks.rados.rados.0.smithi162.stdout:322: writing smithi16234619-29 from 1819464 to 2336510 tid 2 2024-09-16T15:02:20.408 INFO:tasks.rados.rados.0.smithi162.stdout:323: snap_remove snap 26 2024-09-16T15:02:21.367 INFO:tasks.rados.rados.0.smithi162.stdout:323: done (4 left) 2024-09-16T15:02:21.367 INFO:tasks.rados.rados.0.smithi162.stdout:324: copy_from oid 1 from oid 36 current snap is 28 2024-09-16T15:02:21.367 INFO:tasks.rados.rados.0.smithi162.stdout:321: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: finishing write tid 1 to smithi16234619-17 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: finishing write tid 2 to smithi16234619-17 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: finishing write tid 3 to smithi16234619-17 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: finishing write tid 4 to smithi16234619-17 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: oid 17 updating version 0 to 96 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: oid 17 updating version 96 to 97 2024-09-16T15:02:21.456 INFO:tasks.rados.rados.0.smithi162.stdout:318: oid 17 updating version 97 to 98 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 98 (ObjNum 146 snap 28 seq_num 146) dirty exists 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:318: left oid 17 (ObjNum 146 snap 28 seq_num 146) 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: finishing write tid 1 to smithi16234619-13 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:318: done (4 left) 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:321: done (3 left) 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:325: read oid 37 snap -1 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:325: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:326: rollback oid 42 current snap is 28 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 23 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: finishing write tid 2 to smithi16234619-13 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: finishing write tid 3 to smithi16234619-13 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: finishing write tid 4 to smithi16234619-13 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: oid 13 updating version 0 to 107 2024-09-16T15:02:21.457 INFO:tasks.rados.rados.0.smithi162.stdout:320: oid 13 version 107 is already newer than 106 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:320: oid 13 updating version 107 to 108 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 108 (ObjNum 148 snap 28 seq_num 148) dirty exists 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:320: left oid 13 (ObjNum 148 snap 28 seq_num 148) 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: finishing write tid 1 to smithi16234619-29 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: finishing write tid 2 to smithi16234619-29 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: finishing write tid 3 to smithi16234619-29 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: finishing write tid 4 to smithi16234619-29 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: oid 29 updating version 0 to 111 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: oid 29 updating version 111 to 112 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: oid 29 version 112 is already newer than 110 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 112 (ObjNum 149 snap 28 seq_num 149) dirty exists 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:322: left oid 29 (ObjNum 149 snap 28 seq_num 149) 2024-09-16T15:02:21.458 INFO:tasks.rados.rados.0.smithi162.stdout:324: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:324: finishing copy_from to smithi16234619-1 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:320: done (4 left) 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:322: done (3 left) 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:324: done (2 left) 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:327: write oid 22 current snap is 28 2024-09-16T15:02:21.459 INFO:tasks.rados.rados.0.smithi162.stdout:327: seq_num 150 ranges {521050=698057,1877946=643104} 2024-09-16T15:02:21.462 INFO:tasks.rados.rados.0.smithi162.stdout:327: writing smithi16234619-22 from 521050 to 1219107 tid 1 2024-09-16T15:02:21.470 INFO:tasks.rados.rados.0.smithi162.stdout:327: writing smithi16234619-22 from 1877946 to 2521050 tid 2 2024-09-16T15:02:21.470 INFO:tasks.rados.rados.0.smithi162.stdout:328: write oid 31 current snap is 28 2024-09-16T15:02:21.470 INFO:tasks.rados.rados.0.smithi162.stdout:328: seq_num 151 ranges {537857=773306,1936428=520715} 2024-09-16T15:02:21.477 INFO:tasks.rados.rados.0.smithi162.stdout:328: writing smithi16234619-31 from 537857 to 1311163 tid 1 2024-09-16T15:02:21.483 INFO:tasks.rados.rados.0.smithi162.stdout:328: writing smithi16234619-31 from 1936428 to 2457143 tid 2 2024-09-16T15:02:21.483 INFO:tasks.rados.rados.0.smithi162.stdout:329: snap_remove snap 27 2024-09-16T15:02:22.373 INFO:tasks.rados.rados.0.smithi162.stdout:329: done (4 left) 2024-09-16T15:02:22.373 INFO:tasks.rados.rados.0.smithi162.stdout:330: snap_create 2024-09-16T15:02:22.373 INFO:tasks.rados.rados.0.smithi162.stdout:325: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:22.559 INFO:tasks.rados.rados.0.smithi162.stdout:326: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:02:22.559 INFO:tasks.rados.rados.0.smithi162.stdout:326: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:02:22.559 INFO:tasks.rados.rados.0.smithi162.stdout:326: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:02:22.559 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 124 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-16T15:02:22.559 INFO:tasks.rados.rados.0.smithi162.stdout:327: finishing write tid 1 to smithi16234619-22 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: finishing write tid 2 to smithi16234619-22 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: finishing write tid 3 to smithi16234619-22 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: finishing write tid 4 to smithi16234619-22 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: oid 22 updating version 0 to 64 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: oid 22 updating version 64 to 65 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: oid 22 updating version 65 to 66 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 66 (ObjNum 150 snap 28 seq_num 150) dirty exists 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:327: left oid 22 (ObjNum 150 snap 28 seq_num 150) 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: finishing write tid 1 to smithi16234619-31 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: finishing write tid 2 to smithi16234619-31 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: finishing write tid 3 to smithi16234619-31 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: finishing write tid 4 to smithi16234619-31 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: oid 31 updating version 0 to 87 2024-09-16T15:02:22.560 INFO:tasks.rados.rados.0.smithi162.stdout:328: oid 31 version 87 is already newer than 85 2024-09-16T15:02:22.561 INFO:tasks.rados.rados.0.smithi162.stdout:328: oid 31 version 87 is already newer than 86 2024-09-16T15:02:22.561 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 87 (ObjNum 151 snap 28 seq_num 151) dirty exists 2024-09-16T15:02:22.561 INFO:tasks.rados.rados.0.smithi162.stdout:328: left oid 31 (ObjNum 151 snap 28 seq_num 151) 2024-09-16T15:02:23.441 INFO:tasks.rados.rados.0.smithi162.stdout:325: done (4 left) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:326: done (3 left) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:327: done (2 left) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:328: done (1 left) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:330: done (0 left) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:331: read oid 1 snap -1 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:331: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:23.442 INFO:tasks.rados.rados.0.smithi162.stdout:332: snap_create 2024-09-16T15:02:23.446 INFO:tasks.rados.rados.0.smithi162.stdout:331: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:24.444 INFO:tasks.rados.rados.0.smithi162.stdout:331: done (1 left) 2024-09-16T15:02:24.445 INFO:tasks.rados.rados.0.smithi162.stdout:332: done (0 left) 2024-09-16T15:02:24.445 INFO:tasks.rados.rados.0.smithi162.stdout:333: delete oid 9 current snap is 30 2024-09-16T15:02:24.447 INFO:tasks.rados.rados.0.smithi162.stdout:333: done (0 left) 2024-09-16T15:02:24.448 INFO:tasks.rados.rados.0.smithi162.stdout:334: delete oid 13 current snap is 30 2024-09-16T15:02:24.452 INFO:tasks.rados.rados.0.smithi162.stdout:334: done (0 left) 2024-09-16T15:02:24.452 INFO:tasks.rados.rados.0.smithi162.stdout:335: snap_remove snap 23 2024-09-16T15:02:25.364 INFO:tasks.rados.rados.0.smithi162.stdout:335: done (0 left) 2024-09-16T15:02:25.364 INFO:tasks.rados.rados.0.smithi162.stdout:336: write oid 39 current snap is 30 2024-09-16T15:02:25.365 INFO:tasks.rados.rados.0.smithi162.stdout:336: seq_num 154 ranges {588278=715406,1711874=668042} 2024-09-16T15:02:25.371 INFO:tasks.rados.rados.0.smithi162.stdout:336: writing smithi16234619-39 from 588278 to 1303684 tid 1 2024-09-16T15:02:25.378 INFO:tasks.rados.rados.0.smithi162.stdout:336: writing smithi16234619-39 from 1711874 to 2379916 tid 2 2024-09-16T15:02:25.378 INFO:tasks.rados.rados.0.smithi162.stdout:337: write oid 23 current snap is 30 2024-09-16T15:02:25.378 INFO:tasks.rados.rados.0.smithi162.stdout:337: seq_num 155 ranges {605085=790655,2170356=434729} 2024-09-16T15:02:25.385 INFO:tasks.rados.rados.0.smithi162.stdout:337: writing smithi16234619-23 from 605085 to 1395740 tid 1 2024-09-16T15:02:25.392 INFO:tasks.rados.rados.0.smithi162.stdout:337: writing smithi16234619-23 from 2170356 to 2605085 tid 2 2024-09-16T15:02:25.392 INFO:tasks.rados.rados.0.smithi162.stdout:338: write oid 33 current snap is 30 2024-09-16T15:02:25.392 INFO:tasks.rados.rados.0.smithi162.stdout:338: seq_num 156 ranges {621892=465904,1712485=671711} 2024-09-16T15:02:25.398 INFO:tasks.rados.rados.0.smithi162.stdout:338: writing smithi16234619-33 from 621892 to 1087796 tid 1 2024-09-16T15:02:25.405 INFO:tasks.rados.rados.0.smithi162.stdout:338: writing smithi16234619-33 from 1712485 to 2384196 tid 2 2024-09-16T15:02:25.405 INFO:tasks.rados.rados.0.smithi162.stdout:339: delete oid 28 current snap is 30 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:339: done (3 left) 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:340: rollback oid 31 current snap is 30 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 28 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:341: rollback oid 10 current snap is 30 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 28 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:342: write oid 36 current snap is 30 2024-09-16T15:02:25.410 INFO:tasks.rados.rados.0.smithi162.stdout:342: seq_num 158 ranges {655506=616402,1829449=675380} 2024-09-16T15:02:25.417 INFO:tasks.rados.rados.0.smithi162.stdout:342: writing smithi16234619-36 from 655506 to 1271908 tid 1 2024-09-16T15:02:25.427 INFO:tasks.rados.rados.0.smithi162.stdout:342: writing smithi16234619-36 from 1829449 to 2504829 tid 2 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:343: copy_from oid 42 from oid 2 current snap is 30 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:340: finishing rollback tid 0 to smithi16234619-31 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:340: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:341: finishing rollback tid 0 to smithi16234619-10 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:341: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:341: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 114 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:336: finishing write tid 1 to smithi16234619-39 2024-09-16T15:02:25.428 INFO:tasks.rados.rados.0.smithi162.stdout:340: finishing rollback tid 2 to smithi16234619-31 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 101 (ObjNum 151 snap 28 seq_num 151) dirty exists 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: finishing write tid 2 to smithi16234619-39 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: finishing write tid 3 to smithi16234619-39 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: finishing write tid 4 to smithi16234619-39 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: oid 39 updating version 0 to 135 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: oid 39 version 135 is already newer than 134 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: oid 39 updating version 135 to 136 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 136 (ObjNum 154 snap 30 seq_num 154) dirty exists 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: left oid 39 (ObjNum 154 snap 30 seq_num 154) 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:337: finishing write tid 1 to smithi16234619-23 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:336: done (6 left) 2024-09-16T15:02:25.429 INFO:tasks.rados.rados.0.smithi162.stdout:340: done (5 left) 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:341: done (4 left) 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:344: delete oid 3 current snap is 30 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: finishing write tid 2 to smithi16234619-23 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: finishing write tid 3 to smithi16234619-23 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: finishing write tid 4 to smithi16234619-23 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: oid 23 updating version 0 to 29 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: oid 23 version 29 is already newer than 28 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: oid 23 updating version 29 to 30 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 30 (ObjNum 155 snap 30 seq_num 155) dirty exists 2024-09-16T15:02:25.430 INFO:tasks.rados.rados.0.smithi162.stdout:337: left oid 23 (ObjNum 155 snap 30 seq_num 155) 2024-09-16T15:02:25.431 INFO:tasks.rados.rados.0.smithi162.stdout:337: done (4 left) 2024-09-16T15:02:25.431 INFO:tasks.rados.rados.0.smithi162.stdout:344: done (3 left) 2024-09-16T15:02:25.431 INFO:tasks.rados.rados.0.smithi162.stdout:345: delete oid 50 current snap is 30 2024-09-16T15:02:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:338: finishing write tid 1 to smithi16234619-33 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: finishing write tid 2 to smithi16234619-33 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: finishing write tid 3 to smithi16234619-33 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: finishing write tid 4 to smithi16234619-33 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: oid 33 updating version 0 to 24 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: oid 33 version 24 is already newer than 23 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:338: oid 33 version 24 is already newer than 22 2024-09-16T15:02:25.438 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 24 (ObjNum 156 snap 30 seq_num 156) dirty exists 2024-09-16T15:02:25.439 INFO:tasks.rados.rados.0.smithi162.stdout:338: left oid 33 (ObjNum 156 snap 30 seq_num 156) 2024-09-16T15:02:25.451 INFO:tasks.rados.rados.0.smithi162.stdout:342: finishing write tid 1 to smithi16234619-36 2024-09-16T15:02:25.451 INFO:tasks.rados.rados.0.smithi162.stdout:343: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: finishing write tid 2 to smithi16234619-36 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: finishing write tid 3 to smithi16234619-36 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: finishing write tid 4 to smithi16234619-36 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: oid 36 updating version 0 to 151 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: oid 36 updating version 151 to 153 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: oid 36 version 153 is already newer than 152 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 153 (ObjNum 158 snap 30 seq_num 158) dirty exists 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: left oid 36 (ObjNum 158 snap 30 seq_num 158) 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:338: done (3 left) 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:342: done (2 left) 2024-09-16T15:02:25.454 INFO:tasks.rados.rados.0.smithi162.stdout:345: done (1 left) 2024-09-16T15:02:25.455 INFO:tasks.rados.rados.0.smithi162.stdout:346: delete oid 40 current snap is 30 2024-09-16T15:02:25.455 INFO:tasks.rados.rados.0.smithi162.stdout:343: finishing copy_from to smithi16234619-42 2024-09-16T15:02:25.455 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T15:02:25.457 INFO:tasks.rados.rados.0.smithi162.stdout:343: done (1 left) 2024-09-16T15:02:25.457 INFO:tasks.rados.rados.0.smithi162.stdout:346: done (0 left) 2024-09-16T15:02:25.457 INFO:tasks.rados.rados.0.smithi162.stdout:347: write (excl) oid 39 current snap is 30 2024-09-16T15:02:25.457 INFO:tasks.rados.rados.0.smithi162.stdout:347: seq_num 162 ranges {722734=633751,2063377=659357} 2024-09-16T15:02:25.464 INFO:tasks.rados.rados.0.smithi162.stdout:347: writing smithi16234619-39 from 722734 to 1356485 tid 1 2024-09-16T15:02:25.471 INFO:tasks.rados.rados.0.smithi162.stdout:347: writing smithi16234619-39 from 2063377 to 2722734 tid 2 2024-09-16T15:02:25.472 INFO:tasks.rados.rados.0.smithi162.stdout:348: write oid 31 current snap is 30 2024-09-16T15:02:25.472 INFO:tasks.rados.rados.0.smithi162.stdout:348: seq_num 163 ranges {739541=709000,2121859=426376} 2024-09-16T15:02:25.479 INFO:tasks.rados.rados.0.smithi162.stdout:348: writing smithi16234619-31 from 739541 to 1448541 tid 1 2024-09-16T15:02:25.485 INFO:tasks.rados.rados.0.smithi162.stdout:348: writing smithi16234619-31 from 2121859 to 2548235 tid 2 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:349: read oid 25 snap 29 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:349: expect deleted 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:350: read oid 47 snap 28 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:350: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:351: read oid 40 snap -1 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:351: expect deleted 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:352: copy_from oid 18 from oid 34 current snap is 30 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:353: copy_from oid 46 from oid 6 current snap is 30 2024-09-16T15:02:25.486 INFO:tasks.rados.rados.0.smithi162.stdout:354: delete oid 16 current snap is 30 2024-09-16T15:02:25.488 INFO:tasks.rados.rados.0.smithi162.stdout:350: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:25.593 INFO:tasks.rados.rados.0.smithi162.stdout:347: finishing write tid 1 to smithi16234619-39 2024-09-16T15:02:25.593 INFO:tasks.rados.rados.0.smithi162.stdout:347: finishing write tid 2 to smithi16234619-39 2024-09-16T15:02:25.593 INFO:tasks.rados.rados.0.smithi162.stdout:347: finishing write tid 3 to smithi16234619-39 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: finishing write tid 4 to smithi16234619-39 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: oid 39 updating version 0 to 150 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: oid 39 version 150 is already newer than 149 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: oid 39 updating version 150 to 151 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 151 (ObjNum 162 snap 30 seq_num 162) dirty exists 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: left oid 39 (ObjNum 162 snap 30 seq_num 162) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:348: finishing write tid 1 to smithi16234619-31 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:348: finishing write tid 2 to smithi16234619-31 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:348: finishing write tid 3 to smithi16234619-31 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:347: done (7 left) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:349: done (6 left) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:350: done (5 left) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:351: done (4 left) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:354: done (3 left) 2024-09-16T15:02:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:355: snap_create 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:348: finishing write tid 4 to smithi16234619-31 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:348: oid 31 updating version 0 to 102 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:348: oid 31 updating version 102 to 103 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:348: oid 31 updating version 103 to 104 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 104 (ObjNum 163 snap 30 seq_num 163) dirty exists 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:348: left oid 31 (ObjNum 163 snap 30 seq_num 163) 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:353: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:353: finishing copy_from to smithi16234619-46 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:02:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:352: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:02:25.751 INFO:tasks.rados.rados.0.smithi162.stdout:352: finishing copy_from to smithi16234619-18 2024-09-16T15:02:25.751 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 67 (ObjNum 111 snap 20 seq_num 111) dirty exists 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:348: done (3 left) 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:352: done (2 left) 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:353: done (1 left) 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:355: done (0 left) 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:356: read oid 8 snap -1 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:356: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:357: read oid 25 snap 29 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:357: expect deleted 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:358: write oid 33 current snap is 31 2024-09-16T15:02:26.454 INFO:tasks.rados.rados.0.smithi162.stdout:358: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2024-09-16T15:02:26.460 INFO:tasks.rados.rados.0.smithi162.stdout:358: writing smithi16234619-33 from 773155 to 1232653 tid 1 2024-09-16T15:02:26.465 INFO:tasks.rados.rados.0.smithi162.stdout:358: writing smithi16234619-33 from 1722470 to 2152515 tid 2 2024-09-16T15:02:26.469 INFO:tasks.rados.rados.0.smithi162.stdout:358: writing smithi16234619-33 from 2648676 to 2773155 tid 3 2024-09-16T15:02:26.469 INFO:tasks.rados.rados.0.smithi162.stdout:359: write (excl) oid 39 current snap is 31 2024-09-16T15:02:26.469 INFO:tasks.rados.rados.0.smithi162.stdout:359: seq_num 166 ranges {789962=534747,1780952=690056} 2024-09-16T15:02:26.476 INFO:tasks.rados.rados.0.smithi162.stdout:359: writing smithi16234619-39 from 789962 to 1324709 tid 1 2024-09-16T15:02:26.483 INFO:tasks.rados.rados.0.smithi162.stdout:359: writing smithi16234619-39 from 1780952 to 2471008 tid 2 2024-09-16T15:02:26.483 INFO:tasks.rados.rados.0.smithi162.stdout:360: snap_create 2024-09-16T15:02:26.483 INFO:tasks.rados.rados.0.smithi162.stdout:356: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-16T15:02:26.553 INFO:tasks.rados.rados.0.smithi162.stdout:358: finishing write tid 1 to smithi16234619-33 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: finishing write tid 2 to smithi16234619-33 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: finishing write tid 3 to smithi16234619-33 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: finishing write tid 4 to smithi16234619-33 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: finishing write tid 5 to smithi16234619-33 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: oid 33 updating version 0 to 29 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: oid 33 version 29 is already newer than 28 2024-09-16T15:02:26.554 INFO:tasks.rados.rados.0.smithi162.stdout:358: oid 33 version 29 is already newer than 27 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:358: oid 33 version 29 is already newer than 26 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 29 (ObjNum 165 snap 31 seq_num 165) dirty exists 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:358: left oid 33 (ObjNum 165 snap 31 seq_num 165) 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:359: finishing write tid 1 to smithi16234619-39 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:359: finishing write tid 2 to smithi16234619-39 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:359: finishing write tid 3 to smithi16234619-39 2024-09-16T15:02:26.555 INFO:tasks.rados.rados.0.smithi162.stdout:359: finishing write tid 4 to smithi16234619-39 2024-09-16T15:02:26.556 INFO:tasks.rados.rados.0.smithi162.stdout:359: oid 39 updating version 0 to 153 2024-09-16T15:02:26.556 INFO:tasks.rados.rados.0.smithi162.stdout:359: oid 39 updating version 153 to 155 2024-09-16T15:02:26.556 INFO:tasks.rados.rados.0.smithi162.stdout:359: oid 39 version 155 is already newer than 154 2024-09-16T15:02:26.556 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 155 (ObjNum 166 snap 31 seq_num 166) dirty exists 2024-09-16T15:02:26.556 INFO:tasks.rados.rados.0.smithi162.stdout:359: left oid 39 (ObjNum 166 snap 31 seq_num 166) 2024-09-16T15:02:27.457 INFO:tasks.rados.rados.0.smithi162.stdout:356: done (4 left) 2024-09-16T15:02:27.457 INFO:tasks.rados.rados.0.smithi162.stdout:357: done (3 left) 2024-09-16T15:02:27.458 INFO:tasks.rados.rados.0.smithi162.stdout:358: done (2 left) 2024-09-16T15:02:27.458 INFO:tasks.rados.rados.0.smithi162.stdout:359: done (1 left) 2024-09-16T15:02:27.458 INFO:tasks.rados.rados.0.smithi162.stdout:360: done (0 left) 2024-09-16T15:02:27.458 INFO:tasks.rados.rados.0.smithi162.stdout:361: snap_remove snap 29 2024-09-16T15:02:28.470 INFO:tasks.rados.rados.0.smithi162.stdout:361: done (0 left) 2024-09-16T15:02:28.471 INFO:tasks.rados.rados.0.smithi162.stdout:362: write oid 14 current snap is 32 2024-09-16T15:02:28.471 INFO:tasks.rados.rados.0.smithi162.stdout:362: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2024-09-16T15:02:28.476 INFO:tasks.rados.rados.0.smithi162.stdout:362: writing smithi16234619-14 from 406769 to 1016765 tid 1 2024-09-16T15:02:28.481 INFO:tasks.rados.rados.0.smithi162.stdout:362: writing smithi16234619-14 from 1439434 to 1873148 tid 2 2024-09-16T15:02:28.486 INFO:tasks.rados.rados.0.smithi162.stdout:362: writing smithi16234619-14 from 2303522 to 2806769 tid 3 2024-09-16T15:02:28.486 INFO:tasks.rados.rados.0.smithi162.stdout:363: read oid 18 snap -1 2024-09-16T15:02:28.486 INFO:tasks.rados.rados.0.smithi162.stdout:363: expect (ObjNum 111 snap 20 seq_num 111) 2024-09-16T15:02:28.486 INFO:tasks.rados.rados.0.smithi162.stdout:364: snap_remove snap 30 2024-09-16T15:02:29.691 INFO:tasks.rados.rados.0.smithi162.stdout:364: done (2 left) 2024-09-16T15:02:29.691 INFO:tasks.rados.rados.0.smithi162.stdout:365: snap_create 2024-09-16T15:02:29.691 INFO:tasks.rados.rados.0.smithi162.stdout:363: expect (ObjNum 111 snap 20 seq_num 111) 2024-09-16T15:02:29.812 INFO:tasks.rados.rados.0.smithi162.stdout:362: finishing write tid 1 to smithi16234619-14 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: finishing write tid 2 to smithi16234619-14 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: finishing write tid 3 to smithi16234619-14 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: finishing write tid 4 to smithi16234619-14 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: finishing write tid 5 to smithi16234619-14 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: oid 14 updating version 0 to 161 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: oid 14 updating version 161 to 162 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: oid 14 updating version 162 to 164 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: oid 14 version 164 is already newer than 163 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 164 (ObjNum 167 snap 32 seq_num 167) dirty exists 2024-09-16T15:02:29.813 INFO:tasks.rados.rados.0.smithi162.stdout:362: left oid 14 (ObjNum 167 snap 32 seq_num 167) 2024-09-16T15:02:30.364 INFO:tasks.rados.rados.0.smithi162.stdout:362: done (2 left) 2024-09-16T15:02:30.364 INFO:tasks.rados.rados.0.smithi162.stdout:363: done (1 left) 2024-09-16T15:02:30.364 INFO:tasks.rados.rados.0.smithi162.stdout:365: done (0 left) 2024-09-16T15:02:30.364 INFO:tasks.rados.rados.0.smithi162.stdout:366: snap_create 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:366: done (0 left) 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:367: rollback oid 23 current snap is 34 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 32 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:368: rollback oid 49 current snap is 34 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 32 2024-09-16T15:02:31.376 INFO:tasks.rados.rados.0.smithi162.stdout:369: read oid 47 snap -1 2024-09-16T15:02:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:369: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:370: write oid 25 current snap is 34 2024-09-16T15:02:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:370: seq_num 168 ranges {423576=401598,1497916=693725} 2024-09-16T15:02:31.380 INFO:tasks.rados.rados.0.smithi162.stdout:370: writing smithi16234619-25 from 423576 to 825174 tid 1 2024-09-16T15:02:31.388 INFO:tasks.rados.rados.0.smithi162.stdout:370: writing smithi16234619-25 from 1497916 to 2191641 tid 2 2024-09-16T15:02:31.388 INFO:tasks.rados.rados.0.smithi162.stdout:371: snap_remove snap 33 2024-09-16T15:02:32.544 INFO:tasks.rados.rados.0.smithi162.stdout:371: done (4 left) 2024-09-16T15:02:32.544 INFO:tasks.rados.rados.0.smithi162.stdout:372: write oid 11 current snap is 34 2024-09-16T15:02:32.544 INFO:tasks.rados.rados.0.smithi162.stdout:372: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2024-09-16T15:02:32.549 INFO:tasks.rados.rados.0.smithi162.stdout:372: writing smithi16234619-11 from 440383 to 917230 tid 1 2024-09-16T15:02:32.555 INFO:tasks.rados.rados.0.smithi162.stdout:372: writing smithi16234619-11 from 1556398 to 1993781 tid 2 2024-09-16T15:02:32.560 INFO:tasks.rados.rados.0.smithi162.stdout:372: writing smithi16234619-11 from 2474721 to 2840383 tid 3 2024-09-16T15:02:32.560 INFO:tasks.rados.rados.0.smithi162.stdout:373: snap_remove snap 28 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:373: done (5 left) 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:374: delete oid 3 current snap is 34 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:367: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:367: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:368: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:367: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 36 (ObjNum 155 snap 30 seq_num 155) dirty exists 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:368: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:368: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 76 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:33.548 INFO:tasks.rados.rados.0.smithi162.stdout:369: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:370: finishing write tid 1 to smithi16234619-25 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:370: finishing write tid 2 to smithi16234619-25 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:367: done (5 left) 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:368: done (4 left) 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:369: done (3 left) 2024-09-16T15:02:33.653 INFO:tasks.rados.rados.0.smithi162.stdout:374: done (2 left) 2024-09-16T15:02:33.654 INFO:tasks.rados.rados.0.smithi162.stdout:375: write oid 34 current snap is 34 2024-09-16T15:02:33.654 INFO:tasks.rados.rados.0.smithi162.stdout:370: finishing write tid 3 to smithi16234619-25 2024-09-16T15:02:33.654 INFO:tasks.rados.rados.0.smithi162.stdout:375: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2024-09-16T15:02:33.659 INFO:tasks.rados.rados.0.smithi162.stdout:375: writing smithi16234619-34 from 473997 to 1101342 tid 1 2024-09-16T15:02:33.664 INFO:tasks.rados.rados.0.smithi162.stdout:375: writing smithi16234619-34 from 1673362 to 2114414 tid 2 2024-09-16T15:02:33.669 INFO:tasks.rados.rados.0.smithi162.stdout:375: writing smithi16234619-34 from 2529567 to 2873997 tid 3 2024-09-16T15:02:33.669 INFO:tasks.rados.rados.0.smithi162.stdout:376: read oid 35 snap -1 2024-09-16T15:02:33.669 INFO:tasks.rados.rados.0.smithi162.stdout:376: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:33.669 INFO:tasks.rados.rados.0.smithi162.stdout:377: read oid 29 snap -1 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:377: expect (ObjNum 149 snap 28 seq_num 149) 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:378: copy_from oid 33 from oid 42 current snap is 34 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:370: finishing write tid 4 to smithi16234619-25 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:370: oid 25 updating version 0 to 167 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:370: oid 25 updating version 167 to 168 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:370: oid 25 updating version 168 to 169 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 169 (ObjNum 168 snap 34 seq_num 168) dirty exists 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:370: left oid 25 (ObjNum 168 snap 34 seq_num 168) 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:372: finishing write tid 1 to smithi16234619-11 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:372: finishing write tid 2 to smithi16234619-11 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:372: finishing write tid 3 to smithi16234619-11 2024-09-16T15:02:33.670 INFO:tasks.rados.rados.0.smithi162.stdout:372: finishing write tid 4 to smithi16234619-11 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: finishing write tid 5 to smithi16234619-11 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: oid 11 updating version 0 to 33 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: oid 11 updating version 33 to 34 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: oid 11 version 34 is already newer than 31 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: oid 11 version 34 is already newer than 32 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 34 (ObjNum 169 snap 34 seq_num 169) dirty exists 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: left oid 11 (ObjNum 169 snap 34 seq_num 169) 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:370: done (5 left) 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:372: done (4 left) 2024-09-16T15:02:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:379: snap_create 2024-09-16T15:02:33.675 INFO:tasks.rados.rados.0.smithi162.stdout:377: expect (ObjNum 149 snap 28 seq_num 149) 2024-09-16T15:02:33.798 INFO:tasks.rados.rados.0.smithi162.stdout:376: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:33.882 INFO:tasks.rados.rados.0.smithi162.stdout:375: finishing write tid 1 to smithi16234619-34 2024-09-16T15:02:33.882 INFO:tasks.rados.rados.0.smithi162.stdout:375: finishing write tid 2 to smithi16234619-34 2024-09-16T15:02:33.882 INFO:tasks.rados.rados.0.smithi162.stdout:375: finishing write tid 3 to smithi16234619-34 2024-09-16T15:02:33.882 INFO:tasks.rados.rados.0.smithi162.stdout:375: finishing write tid 4 to smithi16234619-34 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: finishing write tid 5 to smithi16234619-34 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: oid 34 updating version 0 to 180 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: oid 34 version 180 is already newer than 178 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: oid 34 version 180 is already newer than 179 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: oid 34 version 180 is already newer than 177 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 180 (ObjNum 171 snap 34 seq_num 171) dirty exists 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:375: left oid 34 (ObjNum 171 snap 34 seq_num 171) 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:378: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:378: finishing copy_from to smithi16234619-33 2024-09-16T15:02:33.883 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T15:02:34.558 INFO:tasks.rados.rados.0.smithi162.stdout:375: done (4 left) 2024-09-16T15:02:34.558 INFO:tasks.rados.rados.0.smithi162.stdout:376: done (3 left) 2024-09-16T15:02:34.558 INFO:tasks.rados.rados.0.smithi162.stdout:377: done (2 left) 2024-09-16T15:02:34.558 INFO:tasks.rados.rados.0.smithi162.stdout:378: done (1 left) 2024-09-16T15:02:34.559 INFO:tasks.rados.rados.0.smithi162.stdout:379: done (0 left) 2024-09-16T15:02:34.559 INFO:tasks.rados.rados.0.smithi162.stdout:380: delete oid 16 current snap is 35 2024-09-16T15:02:34.559 INFO:tasks.rados.rados.0.smithi162.stdout:380: done (0 left) 2024-09-16T15:02:34.559 INFO:tasks.rados.rados.0.smithi162.stdout:381: write (excl) oid 23 current snap is 35 2024-09-16T15:02:34.559 INFO:tasks.rados.rados.0.smithi162.stdout:381: seq_num 173 ranges {507611=777843,2073973=444721} 2024-09-16T15:02:34.566 INFO:tasks.rados.rados.0.smithi162.stdout:381: writing smithi16234619-23 from 507611 to 1285454 tid 1 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:381: writing smithi16234619-23 from 2073973 to 2518694 tid 2 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:382: copy_from oid 44 from oid 21 current snap is 35 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:383: read oid 41 snap -1 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:383: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:384: copy_from oid 18 from oid 36 current snap is 35 2024-09-16T15:02:34.573 INFO:tasks.rados.rados.0.smithi162.stdout:385: snap_remove snap 32 2024-09-16T15:02:35.365 INFO:tasks.rados.rados.0.smithi162.stdout:385: done (4 left) 2024-09-16T15:02:35.365 INFO:tasks.rados.rados.0.smithi162.stdout:386: copy_from oid 20 from oid 9 current snap is 35 2024-09-16T15:02:35.365 INFO:tasks.rados.rados.0.smithi162.stdout:387: snap_remove snap 31 2024-09-16T15:02:36.366 INFO:tasks.rados.rados.0.smithi162.stdout:387: done (5 left) 2024-09-16T15:02:36.366 INFO:tasks.rados.rados.0.smithi162.stdout:388: read oid 42 snap -1 2024-09-16T15:02:36.366 INFO:tasks.rados.rados.0.smithi162.stdout:388: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:02:36.366 INFO:tasks.rados.rados.0.smithi162.stdout:389: snap_remove snap 34 2024-09-16T15:02:37.370 INFO:tasks.rados.rados.0.smithi162.stdout:389: done (6 left) 2024-09-16T15:02:37.370 INFO:tasks.rados.rados.0.smithi162.stdout:390: read oid 16 snap -1 2024-09-16T15:02:37.371 INFO:tasks.rados.rados.0.smithi162.stdout:390: expect deleted 2024-09-16T15:02:37.371 INFO:tasks.rados.rados.0.smithi162.stdout:391: rollback oid 34 current snap is 35 2024-09-16T15:02:37.371 INFO:tasks.rados.rados.0.smithi162.stdout:391: done (7 left) 2024-09-16T15:02:37.371 INFO:tasks.rados.rados.0.smithi162.stdout:392: delete oid 35 current snap is 35 2024-09-16T15:02:37.371 INFO:tasks.rados.rados.0.smithi162.stdout:383: expect (ObjNum 40 snap 0 seq_num 40) 2024-09-16T15:02:37.460 INFO:tasks.rados.rados.0.smithi162.stdout:382: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:382: finishing copy_from to smithi16234619-44 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 13 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: finishing write tid 1 to smithi16234619-23 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: finishing write tid 2 to smithi16234619-23 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: finishing write tid 3 to smithi16234619-23 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:382: done (7 left) 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:383: done (6 left) 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:392: done (5 left) 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:393: delete oid 30 current snap is 35 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: finishing write tid 4 to smithi16234619-23 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: oid 23 updating version 0 to 42 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: oid 23 updating version 42 to 44 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: oid 23 version 44 is already newer than 43 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 44 (ObjNum 173 snap 35 seq_num 173) dirty exists 2024-09-16T15:02:37.461 INFO:tasks.rados.rados.0.smithi162.stdout:381: left oid 23 (ObjNum 173 snap 35 seq_num 173) 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:384: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:384: finishing copy_from to smithi16234619-18 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 153 (ObjNum 158 snap 30 seq_num 158) dirty exists 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:386: finishing copy_from to smithi16234619-20 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:386: got expected ENOENT (src dne) 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:386: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:02:37.462 INFO:tasks.rados.rados.0.smithi162.stdout:388: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:02:37.524 INFO:tasks.rados.rados.0.smithi162.stdout:381: done (5 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:384: done (4 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:386: done (3 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:388: done (2 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:390: done (1 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:393: done (0 left) 2024-09-16T15:02:37.525 INFO:tasks.rados.rados.0.smithi162.stdout:394: delete oid 20 current snap is 35 2024-09-16T15:02:37.526 INFO:tasks.rados.rados.0.smithi162.stdout:394: done (0 left) 2024-09-16T15:02:37.526 INFO:tasks.rados.rados.0.smithi162.stdout:395: snap_create 2024-09-16T15:02:38.398 INFO:tasks.rados.rados.0.smithi162.stdout:395: done (0 left) 2024-09-16T15:02:38.398 INFO:tasks.rados.rados.0.smithi162.stdout:396: read oid 25 snap -1 2024-09-16T15:02:38.398 INFO:tasks.rados.rados.0.smithi162.stdout:396: expect (ObjNum 168 snap 34 seq_num 168) 2024-09-16T15:02:38.398 INFO:tasks.rados.rados.0.smithi162.stdout:397: snap_create 2024-09-16T15:02:38.410 INFO:tasks.rados.rados.0.smithi162.stdout:396: expect (ObjNum 168 snap 34 seq_num 168) 2024-09-16T15:02:39.400 INFO:tasks.rados.rados.0.smithi162.stdout:396: done (1 left) 2024-09-16T15:02:39.400 INFO:tasks.rados.rados.0.smithi162.stdout:397: done (0 left) 2024-09-16T15:02:39.401 INFO:tasks.rados.rados.0.smithi162.stdout:398: snap_create 2024-09-16T15:02:40.367 INFO:tasks.rados.rados.0.smithi162.stdout:398: done (0 left) 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:399: read oid 39 snap 36 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:399: expect (ObjNum 166 snap 31 seq_num 166) 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:400: read oid 6 snap -1 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:400: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:401: write (excl) oid 49 current snap is 38 2024-09-16T15:02:40.368 INFO:tasks.rados.rados.0.smithi162.stdout:401: seq_num 177 ranges {574839=795192,1907901=452059} 2024-09-16T15:02:40.375 INFO:tasks.rados.rados.0.smithi162.stdout:401: writing smithi16234619-49 from 574839 to 1370031 tid 1 2024-09-16T15:02:40.381 INFO:tasks.rados.rados.0.smithi162.stdout:401: writing smithi16234619-49 from 1907901 to 2359960 tid 2 2024-09-16T15:02:40.381 INFO:tasks.rados.rados.0.smithi162.stdout:402: copy_from oid 23 from oid 13 current snap is 38 2024-09-16T15:02:40.381 INFO:tasks.rados.rados.0.smithi162.stdout:400: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:02:40.517 INFO:tasks.rados.rados.0.smithi162.stdout:399: expect (ObjNum 166 snap 31 seq_num 166) 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:402: finishing copy_from to smithi16234619-23 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:402: got expected ENOENT (src dne) 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:402: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:401: finishing write tid 1 to smithi16234619-49 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:401: finishing write tid 2 to smithi16234619-49 2024-09-16T15:02:40.691 INFO:tasks.rados.rados.0.smithi162.stdout:401: finishing write tid 3 to smithi16234619-49 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:399: done (3 left) 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:400: done (2 left) 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:402: done (1 left) 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:403: rollback oid 32 current snap is 38 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 36 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: finishing write tid 4 to smithi16234619-49 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: oid 49 updating version 0 to 86 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: oid 49 updating version 86 to 87 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: oid 49 updating version 87 to 88 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 88 (ObjNum 177 snap 38 seq_num 177) dirty exists 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: left oid 49 (ObjNum 177 snap 38 seq_num 177) 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:401: done (1 left) 2024-09-16T15:02:40.692 INFO:tasks.rados.rados.0.smithi162.stdout:404: read oid 1 snap -1 2024-09-16T15:02:40.693 INFO:tasks.rados.rados.0.smithi162.stdout:404: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:40.693 INFO:tasks.rados.rados.0.smithi162.stdout:405: delete oid 34 current snap is 38 2024-09-16T15:02:40.693 INFO:tasks.rados.rados.0.smithi162.stdout:404: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:40.778 INFO:tasks.rados.rados.0.smithi162.stdout:403: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:403: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:404: done (2 left) 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:405: done (1 left) 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:406: write oid 42 current snap is 38 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:403: finishing rollback tid 2 to smithi16234619-32 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 90 (ObjNum 142 snap 27 seq_num 142) dirty exists 2024-09-16T15:02:40.779 INFO:tasks.rados.rados.0.smithi162.stdout:406: seq_num 179 ranges {608453=545690,1624865=455728,2632598=375855} 2024-09-16T15:02:40.784 INFO:tasks.rados.rados.0.smithi162.stdout:406: writing smithi16234619-42 from 608453 to 1154143 tid 1 2024-09-16T15:02:40.790 INFO:tasks.rados.rados.0.smithi162.stdout:406: writing smithi16234619-42 from 1624865 to 2080593 tid 2 2024-09-16T15:02:40.796 INFO:tasks.rados.rados.0.smithi162.stdout:406: writing smithi16234619-42 from 2632598 to 3008453 tid 3 2024-09-16T15:02:40.796 INFO:tasks.rados.rados.0.smithi162.stdout:403: done (1 left) 2024-09-16T15:02:40.796 INFO:tasks.rados.rados.0.smithi162.stdout:407: copy_from oid 28 from oid 21 current snap is 38 2024-09-16T15:02:40.796 INFO:tasks.rados.rados.0.smithi162.stdout:408: read oid 47 snap -1 2024-09-16T15:02:40.796 INFO:tasks.rados.rados.0.smithi162.stdout:408: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:40.797 INFO:tasks.rados.rados.0.smithi162.stdout:409: write oid 43 current snap is 38 2024-09-16T15:02:40.797 INFO:tasks.rados.rados.0.smithi162.stdout:409: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2024-09-16T15:02:40.804 INFO:tasks.rados.rados.0.smithi162.stdout:409: writing smithi16234619-43 from 625260 to 1246199 tid 1 2024-09-16T15:02:40.810 INFO:tasks.rados.rados.0.smithi162.stdout:409: writing smithi16234619-43 from 1683347 to 2282733 tid 2 2024-09-16T15:02:40.815 INFO:tasks.rados.rados.0.smithi162.stdout:409: writing smithi16234619-43 from 2943668 to 3025260 tid 3 2024-09-16T15:02:40.815 INFO:tasks.rados.rados.0.smithi162.stdout:410: copy_from oid 17 from oid 1 current snap is 38 2024-09-16T15:02:40.815 INFO:tasks.rados.rados.0.smithi162.stdout:408: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: finishing write tid 1 to smithi16234619-42 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: finishing write tid 2 to smithi16234619-42 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: finishing write tid 3 to smithi16234619-42 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: finishing write tid 4 to smithi16234619-42 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: finishing write tid 5 to smithi16234619-42 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: oid 42 updating version 0 to 174 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: oid 42 version 174 is already newer than 172 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: oid 42 version 174 is already newer than 171 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:406: oid 42 version 174 is already newer than 173 2024-09-16T15:02:40.934 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 174 (ObjNum 179 snap 38 seq_num 179) dirty exists 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:406: left oid 42 (ObjNum 179 snap 38 seq_num 179) 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:409: finishing write tid 1 to smithi16234619-43 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:406: done (4 left) 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:408: done (3 left) 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:411: read oid 24 snap -1 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:411: expect (ObjNum 109 snap 19 seq_num 109) 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:412: write oid 12 current snap is 38 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:409: finishing write tid 2 to smithi16234619-43 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:409: finishing write tid 3 to smithi16234619-43 2024-09-16T15:02:40.935 INFO:tasks.rados.rados.0.smithi162.stdout:412: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2024-09-16T15:02:40.940 INFO:tasks.rados.rados.0.smithi162.stdout:412: writing smithi16234619-12 from 642067 to 1338255 tid 1 2024-09-16T15:02:40.946 INFO:tasks.rados.rados.0.smithi162.stdout:412: writing smithi16234619-12 from 2025476 to 2484873 tid 2 2024-09-16T15:02:40.950 INFO:tasks.rados.rados.0.smithi162.stdout:412: writing smithi16234619-12 from 2971091 to 3042067 tid 3 2024-09-16T15:02:40.950 INFO:tasks.rados.rados.0.smithi162.stdout:413: rollback oid 30 current snap is 38 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 36 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:414: copy_from oid 2 from oid 29 current snap is 38 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: finishing write tid 4 to smithi16234619-43 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: finishing write tid 5 to smithi16234619-43 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: oid 43 updating version 0 to 40 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: oid 43 updating version 40 to 41 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: oid 43 version 41 is already newer than 38 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: oid 43 version 41 is already newer than 39 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 41 (ObjNum 180 snap 38 seq_num 180) dirty exists 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:409: left oid 43 (ObjNum 180 snap 38 seq_num 180) 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:407: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:407: finishing copy_from to smithi16234619-28 2024-09-16T15:02:40.951 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 13 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:410: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:407: done (6 left) 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:409: done (5 left) 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:415: rollback oid 21 current snap is 38 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 37 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:410: finishing copy_from to smithi16234619-17 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:02:40.952 INFO:tasks.rados.rados.0.smithi162.stdout:411: expect (ObjNum 109 snap 19 seq_num 109) 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:413: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 0 (ObjNum 49 snap 0 seq_num 851383280) dirty dne 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: finishing write tid 1 to smithi16234619-12 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: finishing write tid 2 to smithi16234619-12 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: finishing write tid 3 to smithi16234619-12 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: finishing write tid 4 to smithi16234619-12 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: finishing write tid 5 to smithi16234619-12 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: oid 12 updating version 0 to 92 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: oid 12 updating version 92 to 93 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: oid 12 updating version 93 to 94 2024-09-16T15:02:41.076 INFO:tasks.rados.rados.0.smithi162.stdout:412: oid 12 updating version 94 to 95 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 95 (ObjNum 181 snap 38 seq_num 181) dirty exists 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:412: left oid 12 (ObjNum 181 snap 38 seq_num 181) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:414: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:415: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:415: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:415: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 124 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:414: finishing copy_from to smithi16234619-2 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 112 (ObjNum 149 snap 28 seq_num 149) dirty exists 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:410: done (5 left) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:411: done (4 left) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:412: done (3 left) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:413: done (2 left) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:414: done (1 left) 2024-09-16T15:02:41.077 INFO:tasks.rados.rados.0.smithi162.stdout:415: done (0 left) 2024-09-16T15:02:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:416: read oid 17 snap -1 2024-09-16T15:02:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:416: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:417: write oid 19 current snap is 38 2024-09-16T15:02:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:417: seq_num 182 ranges {658874=771437,2083958=603055} 2024-09-16T15:02:41.083 INFO:tasks.rados.rados.0.smithi162.stdout:417: writing smithi16234619-19 from 658874 to 1430311 tid 1 2024-09-16T15:02:41.089 INFO:tasks.rados.rados.0.smithi162.stdout:417: writing smithi16234619-19 from 2083958 to 2687013 tid 2 2024-09-16T15:02:41.090 INFO:tasks.rados.rados.0.smithi162.stdout:418: snap_remove snap 35 2024-09-16T15:02:41.416 INFO:tasks.rados.rados.0.smithi162.stdout:418: done (2 left) 2024-09-16T15:02:41.416 INFO:tasks.rados.rados.0.smithi162.stdout:419: write oid 18 current snap is 38 2024-09-16T15:02:41.416 INFO:tasks.rados.rados.0.smithi162.stdout:419: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2024-09-16T15:02:41.423 INFO:tasks.rados.rados.0.smithi162.stdout:419: writing smithi16234619-18 from 675681 to 1238720 tid 1 2024-09-16T15:02:41.430 INFO:tasks.rados.rados.0.smithi162.stdout:419: writing smithi16234619-18 from 1858793 to 2605506 tid 2 2024-09-16T15:02:41.433 INFO:tasks.rados.rados.0.smithi162.stdout:419: writing smithi16234619-18 from 3025937 to 3075681 tid 3 2024-09-16T15:02:41.433 INFO:tasks.rados.rados.0.smithi162.stdout:420: snap_remove snap 36 2024-09-16T15:02:42.428 INFO:tasks.rados.rados.0.smithi162.stdout:420: done (3 left) 2024-09-16T15:02:42.429 INFO:tasks.rados.rados.0.smithi162.stdout:421: copy_from oid 5 from oid 33 current snap is 38 2024-09-16T15:02:42.429 INFO:tasks.rados.rados.0.smithi162.stdout:416: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:42.516 INFO:tasks.rados.rados.0.smithi162.stdout:417: finishing write tid 1 to smithi16234619-19 2024-09-16T15:02:42.516 INFO:tasks.rados.rados.0.smithi162.stdout:417: finishing write tid 2 to smithi16234619-19 2024-09-16T15:02:42.516 INFO:tasks.rados.rados.0.smithi162.stdout:417: finishing write tid 3 to smithi16234619-19 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: finishing write tid 4 to smithi16234619-19 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: oid 19 updating version 0 to 191 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: oid 19 version 191 is already newer than 189 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: oid 19 version 191 is already newer than 190 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 191 (ObjNum 182 snap 38 seq_num 182) dirty exists 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: left oid 19 (ObjNum 182 snap 38 seq_num 182) 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:416: done (3 left) 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:417: done (2 left) 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:422: read oid 44 snap 37 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:422: expect (ObjNum 60 snap 3 seq_num 60) 2024-09-16T15:02:42.517 INFO:tasks.rados.rados.0.smithi162.stdout:423: write (excl) oid 35 current snap is 38 2024-09-16T15:02:42.518 INFO:tasks.rados.rados.0.smithi162.stdout:419: finishing write tid 1 to smithi16234619-18 2024-09-16T15:02:42.518 INFO:tasks.rados.rados.0.smithi162.stdout:423: seq_num 184 ranges {692488=638288,1917275=606724} 2024-09-16T15:02:42.523 INFO:tasks.rados.rados.0.smithi162.stdout:423: writing smithi16234619-35 from 692488 to 1330776 tid 1 2024-09-16T15:02:42.530 INFO:tasks.rados.rados.0.smithi162.stdout:423: writing smithi16234619-35 from 1917275 to 2523999 tid 2 2024-09-16T15:02:42.530 INFO:tasks.rados.rados.0.smithi162.stdout:424: read oid 11 snap -1 2024-09-16T15:02:42.530 INFO:tasks.rados.rados.0.smithi162.stdout:424: expect (ObjNum 169 snap 34 seq_num 169) 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:425: write oid 15 current snap is 38 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: finishing write tid 2 to smithi16234619-18 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: finishing write tid 3 to smithi16234619-18 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: finishing write tid 4 to smithi16234619-18 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: finishing write tid 5 to smithi16234619-18 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: oid 18 updating version 0 to 154 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: oid 18 updating version 154 to 155 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: oid 18 updating version 155 to 157 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: oid 18 version 157 is already newer than 156 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 157 (ObjNum 183 snap 38 seq_num 183) dirty exists 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:419: left oid 18 (ObjNum 183 snap 38 seq_num 183) 2024-09-16T15:02:42.531 INFO:tasks.rados.rados.0.smithi162.stdout:421: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:02:42.532 INFO:tasks.rados.rados.0.smithi162.stdout:421: finishing copy_from to smithi16234619-5 2024-09-16T15:02:42.532 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T15:02:42.532 INFO:tasks.rados.rados.0.smithi162.stdout:422: expect (ObjNum 60 snap 3 seq_num 60) 2024-09-16T15:02:42.668 INFO:tasks.rados.rados.0.smithi162.stdout:424: expect (ObjNum 169 snap 34 seq_num 169) 2024-09-16T15:02:42.759 INFO:tasks.rados.rados.0.smithi162.stdout:423: finishing write tid 1 to smithi16234619-35 2024-09-16T15:02:42.759 INFO:tasks.rados.rados.0.smithi162.stdout:423: finishing write tid 2 to smithi16234619-35 2024-09-16T15:02:42.759 INFO:tasks.rados.rados.0.smithi162.stdout:423: finishing write tid 3 to smithi16234619-35 2024-09-16T15:02:42.759 INFO:tasks.rados.rados.0.smithi162.stdout:423: finishing write tid 4 to smithi16234619-35 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:423: oid 35 updating version 0 to 113 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:423: oid 35 updating version 113 to 114 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:423: oid 35 updating version 114 to 115 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 115 (ObjNum 184 snap 38 seq_num 184) dirty exists 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:423: left oid 35 (ObjNum 184 snap 38 seq_num 184) 2024-09-16T15:02:42.760 INFO:tasks.rados.rados.0.smithi162.stdout:425: seq_num 185 ranges {709295=713537,1859404=750382} 2024-09-16T15:02:42.766 INFO:tasks.rados.rados.0.smithi162.stdout:425: writing smithi16234619-15 from 709295 to 1422832 tid 1 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:425: writing smithi16234619-15 from 1859404 to 2609786 tid 2 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:419: done (5 left) 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:421: done (4 left) 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:422: done (3 left) 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:423: done (2 left) 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:424: done (1 left) 2024-09-16T15:02:42.773 INFO:tasks.rados.rados.0.smithi162.stdout:426: write (excl) oid 12 current snap is 38 2024-09-16T15:02:42.774 INFO:tasks.rados.rados.0.smithi162.stdout:426: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2024-09-16T15:02:42.781 INFO:tasks.rados.rados.0.smithi162.stdout:426: writing smithi16234619-12 from 726102 to 1514888 tid 1 2024-09-16T15:02:42.786 INFO:tasks.rados.rados.0.smithi162.stdout:426: writing smithi16234619-12 from 1917886 to 2528279 tid 2 2024-09-16T15:02:42.789 INFO:tasks.rados.rados.0.smithi162.stdout:426: writing smithi16234619-12 from 3108206 to 3126102 tid 3 2024-09-16T15:02:42.789 INFO:tasks.rados.rados.0.smithi162.stdout:427: snap_create 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: finishing write tid 1 to smithi16234619-15 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: finishing write tid 2 to smithi16234619-15 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: finishing write tid 3 to smithi16234619-15 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: finishing write tid 4 to smithi16234619-15 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: oid 15 updating version 0 to 72 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: oid 15 updating version 72 to 73 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: oid 15 updating version 73 to 74 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 74 (ObjNum 185 snap 38 seq_num 185) dirty exists 2024-09-16T15:02:42.807 INFO:tasks.rados.rados.0.smithi162.stdout:425: left oid 15 (ObjNum 185 snap 38 seq_num 185) 2024-09-16T15:02:42.817 INFO:tasks.rados.rados.0.smithi162.stdout:426: finishing write tid 1 to smithi16234619-12 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: finishing write tid 2 to smithi16234619-12 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: finishing write tid 3 to smithi16234619-12 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: finishing write tid 4 to smithi16234619-12 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: finishing write tid 5 to smithi16234619-12 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: oid 12 updating version 0 to 106 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: oid 12 updating version 106 to 107 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: oid 12 version 107 is already newer than 105 2024-09-16T15:02:42.818 INFO:tasks.rados.rados.0.smithi162.stdout:426: oid 12 version 107 is already newer than 104 2024-09-16T15:02:42.819 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 107 (ObjNum 186 snap 38 seq_num 186) dirty exists 2024-09-16T15:02:42.819 INFO:tasks.rados.rados.0.smithi162.stdout:426: left oid 12 (ObjNum 186 snap 38 seq_num 186) 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:425: done (2 left) 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:426: done (1 left) 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:427: done (0 left) 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:428: read oid 3 snap -1 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:428: expect deleted 2024-09-16T15:02:43.433 INFO:tasks.rados.rados.0.smithi162.stdout:429: snap_remove snap 38 2024-09-16T15:02:44.441 INFO:tasks.rados.rados.0.smithi162.stdout:429: done (1 left) 2024-09-16T15:02:44.441 INFO:tasks.rados.rados.0.smithi162.stdout:430: write oid 36 current snap is 39 2024-09-16T15:02:44.441 INFO:tasks.rados.rados.0.smithi162.stdout:430: seq_num 187 ranges {742909=464035,1976368=754051} 2024-09-16T15:02:44.447 INFO:tasks.rados.rados.0.smithi162.stdout:430: writing smithi16234619-36 from 742909 to 1206944 tid 1 2024-09-16T15:02:44.456 INFO:tasks.rados.rados.0.smithi162.stdout:430: writing smithi16234619-36 from 1976368 to 2730419 tid 2 2024-09-16T15:02:44.456 INFO:tasks.rados.rados.0.smithi162.stdout:431: delete oid 5 current snap is 39 2024-09-16T15:02:44.458 INFO:tasks.rados.rados.0.smithi162.stdout:428: done (2 left) 2024-09-16T15:02:44.458 INFO:tasks.rados.rados.0.smithi162.stdout:431: done (1 left) 2024-09-16T15:02:44.458 INFO:tasks.rados.rados.0.smithi162.stdout:432: rollback oid 49 current snap is 39 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 37 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:433: copy_from oid 39 from oid 29 current snap is 39 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:434: read oid 37 snap 37 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:434: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:435: copy_from oid 26 from oid 7 current snap is 39 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:436: write oid 3 current snap is 39 2024-09-16T15:02:44.459 INFO:tasks.rados.rados.0.smithi162.stdout:436: seq_num 189 ranges {776523=614533,1976979=757720} 2024-09-16T15:02:44.466 INFO:tasks.rados.rados.0.smithi162.stdout:436: writing smithi16234619-3 from 776523 to 1391056 tid 1 2024-09-16T15:02:44.474 INFO:tasks.rados.rados.0.smithi162.stdout:436: writing smithi16234619-3 from 1976979 to 2734699 tid 2 2024-09-16T15:02:44.474 INFO:tasks.rados.rados.0.smithi162.stdout:437: delete oid 28 current snap is 39 2024-09-16T15:02:44.474 INFO:tasks.rados.rados.0.smithi162.stdout:434: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:435: finishing copy_from to smithi16234619-26 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:435: got expected ENOENT (src dne) 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:435: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:432: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:432: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:432: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 95 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:02:44.661 INFO:tasks.rados.rados.0.smithi162.stdout:432: done (6 left) 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:434: done (5 left) 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:435: done (4 left) 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:437: done (3 left) 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:438: read oid 37 snap 37 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:438: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:439: write oid 24 current snap is 39 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:430: finishing write tid 1 to smithi16234619-36 2024-09-16T15:02:44.662 INFO:tasks.rados.rados.0.smithi162.stdout:439: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2024-09-16T15:02:44.665 INFO:tasks.rados.rados.0.smithi162.stdout:439: writing smithi16234619-24 from 410137 to 891521 tid 1 2024-09-16T15:02:44.672 INFO:tasks.rados.rados.0.smithi162.stdout:439: writing smithi16234619-24 from 1410296 to 2171685 tid 2 2024-09-16T15:02:44.678 INFO:tasks.rados.rados.0.smithi162.stdout:439: writing smithi16234619-24 from 2728968 to 3210137 tid 3 2024-09-16T15:02:44.678 INFO:tasks.rados.rados.0.smithi162.stdout:440: delete oid 13 current snap is 39 2024-09-16T15:02:44.678 INFO:tasks.rados.rados.0.smithi162.stdout:430: finishing write tid 2 to smithi16234619-36 2024-09-16T15:02:44.678 INFO:tasks.rados.rados.0.smithi162.stdout:430: finishing write tid 3 to smithi16234619-36 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:430: finishing write tid 4 to smithi16234619-36 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:430: oid 36 updating version 0 to 197 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:430: oid 36 updating version 197 to 198 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:430: oid 36 updating version 198 to 199 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 199 (ObjNum 187 snap 39 seq_num 187) dirty exists 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:430: left oid 36 (ObjNum 187 snap 39 seq_num 187) 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:433: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:436: finishing write tid 1 to smithi16234619-3 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:436: finishing write tid 2 to smithi16234619-3 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:436: finishing write tid 3 to smithi16234619-3 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:436: finishing write tid 4 to smithi16234619-3 2024-09-16T15:02:44.679 INFO:tasks.rados.rados.0.smithi162.stdout:436: oid 3 updating version 0 to 23 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:436: oid 3 updating version 23 to 25 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:436: oid 3 version 25 is already newer than 24 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 25 (ObjNum 189 snap 39 seq_num 189) dirty exists 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:436: left oid 3 (ObjNum 189 snap 39 seq_num 189) 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:433: finishing copy_from to smithi16234619-39 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 112 (ObjNum 149 snap 28 seq_num 149) dirty exists 2024-09-16T15:02:44.680 INFO:tasks.rados.rados.0.smithi162.stdout:438: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:02:44.866 INFO:tasks.rados.rados.0.smithi162.stdout:439: finishing write tid 1 to smithi16234619-24 2024-09-16T15:02:44.866 INFO:tasks.rados.rados.0.smithi162.stdout:439: finishing write tid 2 to smithi16234619-24 2024-09-16T15:02:44.866 INFO:tasks.rados.rados.0.smithi162.stdout:430: done (5 left) 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:433: done (4 left) 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:436: done (3 left) 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:438: done (2 left) 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:440: done (1 left) 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:441: delete oid 29 current snap is 39 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: finishing write tid 3 to smithi16234619-24 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: finishing write tid 4 to smithi16234619-24 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: finishing write tid 5 to smithi16234619-24 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: oid 24 updating version 0 to 140 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: oid 24 updating version 140 to 142 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: oid 24 version 142 is already newer than 141 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:439: oid 24 updating version 142 to 143 2024-09-16T15:02:44.867 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 143 (ObjNum 191 snap 39 seq_num 191) dirty exists 2024-09-16T15:02:44.868 INFO:tasks.rados.rados.0.smithi162.stdout:439: left oid 24 (ObjNum 191 snap 39 seq_num 191) 2024-09-16T15:02:44.869 INFO:tasks.rados.rados.0.smithi162.stdout:439: done (1 left) 2024-09-16T15:02:44.869 INFO:tasks.rados.rados.0.smithi162.stdout:441: done (0 left) 2024-09-16T15:02:44.869 INFO:tasks.rados.rados.0.smithi162.stdout:442: delete oid 34 current snap is 39 2024-09-16T15:02:44.870 INFO:tasks.rados.rados.0.smithi162.stdout:442: done (0 left) 2024-09-16T15:02:44.870 INFO:tasks.rados.rados.0.smithi162.stdout:443: write (excl) oid 34 current snap is 39 2024-09-16T15:02:44.870 INFO:tasks.rados.rados.0.smithi162.stdout:443: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2024-09-16T15:02:44.877 INFO:tasks.rados.rados.0.smithi162.stdout:443: writing smithi16234619-34 from 477365 to 1259745 tid 1 2024-09-16T15:02:44.884 INFO:tasks.rados.rados.0.smithi162.stdout:443: writing smithi16234619-34 from 1927871 to 2696598 tid 2 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:443: writing smithi16234619-34 from 3122307 to 3277365 tid 3 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:444: read oid 40 snap -1 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:444: expect deleted 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:445: read oid 17 snap -1 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:445: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:446: copy_from oid 26 from oid 47 current snap is 39 2024-09-16T15:02:44.888 INFO:tasks.rados.rados.0.smithi162.stdout:447: snap_create 2024-09-16T15:02:44.890 INFO:tasks.rados.rados.0.smithi162.stdout:445: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: finishing write tid 1 to smithi16234619-34 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: finishing write tid 2 to smithi16234619-34 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: finishing write tid 3 to smithi16234619-34 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: finishing write tid 4 to smithi16234619-34 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: finishing write tid 5 to smithi16234619-34 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: oid 34 updating version 0 to 202 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: oid 34 updating version 202 to 204 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: oid 34 version 204 is already newer than 201 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: oid 34 version 204 is already newer than 203 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 204 (ObjNum 195 snap 39 seq_num 195) dirty exists 2024-09-16T15:02:44.976 INFO:tasks.rados.rados.0.smithi162.stdout:443: left oid 34 (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:02:44.977 INFO:tasks.rados.rados.0.smithi162.stdout:446: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:02:44.977 INFO:tasks.rados.rados.0.smithi162.stdout:446: finishing copy_from to smithi16234619-26 2024-09-16T15:02:44.977 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 29 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-16T15:02:46.182 INFO:tasks.rados.rados.0.smithi162.stdout:443: done (4 left) 2024-09-16T15:02:46.183 INFO:tasks.rados.rados.0.smithi162.stdout:444: done (3 left) 2024-09-16T15:02:46.183 INFO:tasks.rados.rados.0.smithi162.stdout:445: done (2 left) 2024-09-16T15:02:46.183 INFO:tasks.rados.rados.0.smithi162.stdout:446: done (1 left) 2024-09-16T15:02:46.183 INFO:tasks.rados.rados.0.smithi162.stdout:447: done (0 left) 2024-09-16T15:02:46.183 INFO:tasks.rados.rados.0.smithi162.stdout:448: snap_remove snap 37 2024-09-16T15:02:46.445 INFO:tasks.rados.rados.0.smithi162.stdout:448: done (0 left) 2024-09-16T15:02:46.445 INFO:tasks.rados.rados.0.smithi162.stdout:449: snap_create 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:449: done (0 left) 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:450: read oid 27 snap -1 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:450: expect (ObjNum 131 snap 24 seq_num 131) 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:451: read oid 50 snap -1 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:451: expect deleted 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:452: read oid 47 snap -1 2024-09-16T15:02:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:452: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:47.694 INFO:tasks.rados.rados.0.smithi162.stdout:453: snap_remove snap 39 2024-09-16T15:02:48.705 INFO:tasks.rados.rados.0.smithi162.stdout:453: done (3 left) 2024-09-16T15:02:48.705 INFO:tasks.rados.rados.0.smithi162.stdout:454: snap_remove snap 40 2024-09-16T15:02:49.708 INFO:tasks.rados.rados.0.smithi162.stdout:454: done (3 left) 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:455: read oid 5 snap -1 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:455: expect deleted 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:456: rollback oid 44 current snap is 41 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:456: done (4 left) 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:457: snap_create 2024-09-16T15:02:49.709 INFO:tasks.rados.rados.0.smithi162.stdout:452: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:02:49.812 INFO:tasks.rados.rados.0.smithi162.stdout:450: expect (ObjNum 131 snap 24 seq_num 131) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:450: done (4 left) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:451: done (3 left) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:452: done (2 left) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:455: done (1 left) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:457: done (0 left) 2024-09-16T15:02:50.369 INFO:tasks.rados.rados.0.smithi162.stdout:458: snap_remove snap 41 2024-09-16T15:02:51.379 INFO:tasks.rados.rados.0.smithi162.stdout:458: done (0 left) 2024-09-16T15:02:51.379 INFO:tasks.rados.rados.0.smithi162.stdout:459: read oid 24 snap -1 2024-09-16T15:02:51.379 INFO:tasks.rados.rados.0.smithi162.stdout:459: expect (ObjNum 191 snap 39 seq_num 191) 2024-09-16T15:02:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:460: write oid 9 current snap is 42 2024-09-16T15:02:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:460: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2024-09-16T15:02:51.384 INFO:tasks.rados.rados.0.smithi162.stdout:460: writing smithi16234619-9 from 494172 to 951801 tid 1 2024-09-16T15:02:51.392 INFO:tasks.rados.rados.0.smithi162.stdout:460: writing smithi16234619-9 from 1586353 to 2098738 tid 2 2024-09-16T15:02:51.399 INFO:tasks.rados.rados.0.smithi162.stdout:460: writing smithi16234619-9 from 2749730 to 3294172 tid 3 2024-09-16T15:02:51.399 INFO:tasks.rados.rados.0.smithi162.stdout:461: delete oid 41 current snap is 42 2024-09-16T15:02:51.400 INFO:tasks.rados.rados.0.smithi162.stdout:459: expect (ObjNum 191 snap 39 seq_num 191) 2024-09-16T15:02:51.540 INFO:tasks.rados.rados.0.smithi162.stdout:460: finishing write tid 1 to smithi16234619-9 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: finishing write tid 2 to smithi16234619-9 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: finishing write tid 3 to smithi16234619-9 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: finishing write tid 4 to smithi16234619-9 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: finishing write tid 5 to smithi16234619-9 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: oid 9 updating version 0 to 135 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: oid 9 version 135 is already newer than 134 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: oid 9 updating version 135 to 137 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: oid 9 version 137 is already newer than 136 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 137 (ObjNum 196 snap 42 seq_num 196) dirty exists 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: left oid 9 (ObjNum 196 snap 42 seq_num 196) 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:459: done (2 left) 2024-09-16T15:02:51.541 INFO:tasks.rados.rados.0.smithi162.stdout:460: done (1 left) 2024-09-16T15:02:51.542 INFO:tasks.rados.rados.0.smithi162.stdout:461: done (0 left) 2024-09-16T15:02:51.542 INFO:tasks.rados.rados.0.smithi162.stdout:462: rollback oid 32 current snap is 42 2024-09-16T15:02:51.542 INFO:tasks.rados.rados.0.smithi162.stdout:462: done (0 left) 2024-09-16T15:02:51.542 INFO:tasks.rados.rados.0.smithi162.stdout:463: write (excl) oid 45 current snap is 42 2024-09-16T15:02:51.542 INFO:tasks.rados.rados.0.smithi162.stdout:463: seq_num 198 ranges {527786=724480,1703317=516054,2804576=435035} 2024-09-16T15:02:51.547 INFO:tasks.rados.rados.0.smithi162.stdout:463: writing smithi16234619-45 from 527786 to 1252266 tid 1 2024-09-16T15:02:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:463: writing smithi16234619-45 from 1703317 to 2219371 tid 2 2024-09-16T15:02:51.558 INFO:tasks.rados.rados.0.smithi162.stdout:463: writing smithi16234619-45 from 2804576 to 3239611 tid 3 2024-09-16T15:02:51.558 INFO:tasks.rados.rados.0.smithi162.stdout:464: delete oid 9 current snap is 42 2024-09-16T15:02:51.559 INFO:tasks.rados.rados.0.smithi162.stdout:464: done (1 left) 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:465: rollback oid 49 current snap is 42 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:465: done (1 left) 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:466: read oid 43 snap -1 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:466: expect (ObjNum 180 snap 38 seq_num 180) 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:467: write (excl) oid 7 current snap is 42 2024-09-16T15:02:51.560 INFO:tasks.rados.rados.0.smithi162.stdout:467: seq_num 200 ranges {561400=474978,1820281=519723,2859422=457579} 2024-09-16T15:02:51.565 INFO:tasks.rados.rados.0.smithi162.stdout:467: writing smithi16234619-7 from 561400 to 1036378 tid 1 2024-09-16T15:02:51.572 INFO:tasks.rados.rados.0.smithi162.stdout:467: writing smithi16234619-7 from 1820281 to 2340004 tid 2 2024-09-16T15:02:51.578 INFO:tasks.rados.rados.0.smithi162.stdout:467: writing smithi16234619-7 from 2859422 to 3317001 tid 3 2024-09-16T15:02:51.578 INFO:tasks.rados.rados.0.smithi162.stdout:468: read oid 42 snap -1 2024-09-16T15:02:51.578 INFO:tasks.rados.rados.0.smithi162.stdout:468: expect (ObjNum 179 snap 38 seq_num 179) 2024-09-16T15:02:51.578 INFO:tasks.rados.rados.0.smithi162.stdout:469: rollback oid 36 current snap is 42 2024-09-16T15:02:51.579 INFO:tasks.rados.rados.0.smithi162.stdout:469: done (4 left) 2024-09-16T15:02:51.579 INFO:tasks.rados.rados.0.smithi162.stdout:470: delete oid 32 current snap is 42 2024-09-16T15:02:51.579 INFO:tasks.rados.rados.0.smithi162.stdout:466: expect (ObjNum 180 snap 38 seq_num 180) 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:466: done (4 left) 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:470: done (3 left) 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:471: read oid 36 snap -1 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:471: expect (ObjNum 187 snap 39 seq_num 187) 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:472: write (excl) oid 35 current snap is 42 2024-09-16T15:02:51.677 INFO:tasks.rados.rados.0.smithi162.stdout:468: expect (ObjNum 179 snap 38 seq_num 179) 2024-09-16T15:02:51.804 INFO:tasks.rados.rados.0.smithi162.stdout:463: finishing write tid 1 to smithi16234619-45 2024-09-16T15:02:51.804 INFO:tasks.rados.rados.0.smithi162.stdout:463: finishing write tid 2 to smithi16234619-45 2024-09-16T15:02:51.804 INFO:tasks.rados.rados.0.smithi162.stdout:472: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2024-09-16T15:02:51.810 INFO:tasks.rados.rados.0.smithi162.stdout:472: writing smithi16234619-35 from 595014 to 1220490 tid 1 2024-09-16T15:02:51.816 INFO:tasks.rados.rados.0.smithi162.stdout:472: writing smithi16234619-35 from 1820892 to 2344284 tid 2 2024-09-16T15:02:51.822 INFO:tasks.rados.rados.0.smithi162.stdout:472: writing smithi16234619-35 from 2797915 to 3394391 tid 3 2024-09-16T15:02:51.822 INFO:tasks.rados.rados.0.smithi162.stdout:468: done (4 left) 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:473: write (excl) oid 46 current snap is 42 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: finishing write tid 3 to smithi16234619-45 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: finishing write tid 4 to smithi16234619-45 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: finishing write tid 5 to smithi16234619-45 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: oid 45 updating version 0 to 158 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: oid 45 version 158 is already newer than 157 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: oid 45 updating version 158 to 159 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: oid 45 version 159 is already newer than 156 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 159 (ObjNum 198 snap 42 seq_num 198) dirty exists 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:463: left oid 45 (ObjNum 198 snap 42 seq_num 198) 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:467: finishing write tid 1 to smithi16234619-7 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:467: finishing write tid 2 to smithi16234619-7 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:467: finishing write tid 3 to smithi16234619-7 2024-09-16T15:02:51.823 INFO:tasks.rados.rados.0.smithi162.stdout:467: finishing write tid 4 to smithi16234619-7 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: finishing write tid 5 to smithi16234619-7 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: oid 7 updating version 0 to 158 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: oid 7 updating version 158 to 160 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: oid 7 updating version 160 to 161 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: oid 7 version 161 is already newer than 159 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:467: left oid 7 (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:02:51.824 INFO:tasks.rados.rados.0.smithi162.stdout:473: seq_num 203 ranges {611821=700725,1879374=783403,3341691=70130} 2024-09-16T15:02:51.829 INFO:tasks.rados.rados.0.smithi162.stdout:473: writing smithi16234619-46 from 611821 to 1312546 tid 1 2024-09-16T15:02:51.836 INFO:tasks.rados.rados.0.smithi162.stdout:473: writing smithi16234619-46 from 1879374 to 2662777 tid 2 2024-09-16T15:02:51.841 INFO:tasks.rados.rados.0.smithi162.stdout:473: writing smithi16234619-46 from 3341691 to 3411821 tid 3 2024-09-16T15:02:51.841 INFO:tasks.rados.rados.0.smithi162.stdout:463: done (4 left) 2024-09-16T15:02:51.841 INFO:tasks.rados.rados.0.smithi162.stdout:467: done (3 left) 2024-09-16T15:02:51.841 INFO:tasks.rados.rados.0.smithi162.stdout:474: snap_create 2024-09-16T15:02:51.841 INFO:tasks.rados.rados.0.smithi162.stdout:471: expect (ObjNum 187 snap 39 seq_num 187) 2024-09-16T15:02:52.029 INFO:tasks.rados.rados.0.smithi162.stdout:472: finishing write tid 1 to smithi16234619-35 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: finishing write tid 2 to smithi16234619-35 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: finishing write tid 3 to smithi16234619-35 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: finishing write tid 4 to smithi16234619-35 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: finishing write tid 5 to smithi16234619-35 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: oid 35 updating version 0 to 116 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: oid 35 updating version 116 to 117 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: oid 35 updating version 117 to 118 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: oid 35 updating version 118 to 119 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 119 (ObjNum 202 snap 42 seq_num 202) dirty exists 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:472: left oid 35 (ObjNum 202 snap 42 seq_num 202) 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:473: finishing write tid 1 to smithi16234619-46 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:473: finishing write tid 2 to smithi16234619-46 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:473: finishing write tid 3 to smithi16234619-46 2024-09-16T15:02:52.030 INFO:tasks.rados.rados.0.smithi162.stdout:473: finishing write tid 4 to smithi16234619-46 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: finishing write tid 5 to smithi16234619-46 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: oid 46 updating version 0 to 86 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: oid 46 updating version 86 to 88 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: oid 46 version 88 is already newer than 87 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: oid 46 version 88 is already newer than 85 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 88 (ObjNum 203 snap 42 seq_num 203) dirty exists 2024-09-16T15:02:52.031 INFO:tasks.rados.rados.0.smithi162.stdout:473: left oid 46 (ObjNum 203 snap 42 seq_num 203) 2024-09-16T15:02:52.735 INFO:tasks.rados.rados.0.smithi162.stdout:471: done (3 left) 2024-09-16T15:02:52.735 INFO:tasks.rados.rados.0.smithi162.stdout:472: done (2 left) 2024-09-16T15:02:52.735 INFO:tasks.rados.rados.0.smithi162.stdout:473: done (1 left) 2024-09-16T15:02:52.736 INFO:tasks.rados.rados.0.smithi162.stdout:474: done (0 left) 2024-09-16T15:02:52.736 INFO:tasks.rados.rados.0.smithi162.stdout:475: snap_remove snap 42 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:475: done (0 left) 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:476: read oid 40 snap -1 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:476: expect deleted 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:477: rollback oid 50 current snap is 43 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:477: done (1 left) 2024-09-16T15:02:53.741 INFO:tasks.rados.rados.0.smithi162.stdout:478: snap_create 2024-09-16T15:02:54.119 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:02:54.119 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-16T15:02:54.120 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-16T15:02:54.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-16T15:02:54.123 INFO:tasks.ceph.mon.f:Started 2024-09-16T15:02:54.123 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-16T15:02:54.123 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-16T15:02:54.123 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-16T15:02:54.123 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-16T15:02:54.126 INFO:tasks.ceph.mon.d:Started 2024-09-16T15:02:54.126 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:02:54.126 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-16T15:02:54.126 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-16T15:02:54.126 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-16T15:02:54.128 INFO:tasks.ceph.mon.g:Started 2024-09-16T15:02:54.128 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:02:54.128 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-16T15:02:54.129 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:02:54.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:02:54.131 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:02:54.131 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-16T15:02:54.131 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-16T15:02:54.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-16T15:03:09.132 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-16T15:03:09.133 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-16T15:03:09.133 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:03:09.133 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:09.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.239+0000 7f3113fff640 10 monclient: get_monmap_and_config 2024-09-16T15:03:09.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.239+0000 7f3113fff640 10 monclient: build_initial_monmap 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.239+0000 7f3113fff640 10 monclient: monmap: 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:09.240878+0000 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:09.240878+0000 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:09.242 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.240+0000 7f3113fff640 1 Processor -- start 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.240+0000 7f3113fff640 1 -- start start 2024-09-16T15:03:09.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.240+0000 7f3113fff640 10 monclient: init 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient: _add_conns ranks=[1,0,3,8,5,2,6,9,4,10,7] 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3114136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3114139d90 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:09.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-g con 0x7f311413d8a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3114073250 con 0x7f3114139d90 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140a0f80 con 0x7f3114136e40 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140a1260 con 0x7f311413d8a0 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f3113fff640 10 monclient(hunting): authenticate will time out at 925.830174s 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31127fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3114139d90 0x7f311413c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:48134/0 (socket says 172.21.15.134:48134) 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31127fc640 1 -- 172.21.15.134:0/3769977827 learned_addr learned my addr 172.21.15.134:0/3769977827 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 10 monclient(hunting): get_auth_request con 0x7f311413d8a0 auth_method 0 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:09.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:09.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:09.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:09.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.241+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:09.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.243+0000 7f31127fc640 1 -- 172.21.15.134:0/3769977827 >> v1:172.21.15.134:6789/0 conn(0x7f3114139d90 legacy=0x7f311413c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:09.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.243+0000 7f3112ffd640 1 -- 172.21.15.134:0/3769977827 >> v1:172.21.15.162:6789/0 conn(0x7f3114136e40 legacy=0x7f3114137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 10 monclient(hunting): get_auth_request con 0x7f311413d8a0 auth_method 0 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:09.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.442+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 10 monclient(hunting): get_auth_request con 0x7f311413d8a0 auth_method 0 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:09.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:09.843+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350752519 0 0) 0x7f3114073250 con 0x7f3114139d90 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 10 monclient(hunting): _negotiate 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 10 monclient(hunting): my global_id is 4545 2024-09-16T15:03:10.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.263+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc003520 con 0x7f3114139d90 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702703901 0 0) 0x7f30fc003520 con 0x7f3114139d90 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f311413fc90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f311413fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 >> v1:172.21.15.162:6789/0 conn(0x7f3114136e40 legacy=0x7f3114137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _finish_hunting 0 2024-09-16T15:03:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114137930 con 0x7f3114139d90 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f3104001b90 con 0x7f3114139d90 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: dump: 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 >> v1:172.21.15.134:6789/0 conn(0x7f3114139d90 legacy=0x7f311413c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient: _add_conns ranks=[5,7,3,4,2,8,0,6,1] 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f30fc004270 0x7f30fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): picked mon.h con 0x7f30fc004270 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f30fc004ec0 0x7f30fc009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): picked mon.i con 0x7f30fc004ec0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f30fc00bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): picked mon.g con 0x7f311413d8a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140a1260 con 0x7f311413d8a0 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc00c4e0 con 0x7f30fc004270 2024-09-16T15:03:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc00c650 con 0x7f30fc004ec0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.264+0000 7f3111ffb640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f30fc004ec0 0x7f30fc009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f30fc004270 0x7f30fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f30fc00bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): get_auth_request con 0x7f30fc004ec0 auth_method 0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): get_auth_request con 0x7f311413d8a0 auth_method 0 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): get_auth_request con 0x7f30fc004270 auth_method 0 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient(hunting): handle_auth_done global_id 4545 payload 293 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f30fc004ec0 msgr2=0x7f30fc009970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f30fc004ec0 0x7f30fc009970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f30fc004270 msgr2=0x7f30fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f30fc004270 0x7f30fc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 monclient: found mon.g 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc003520 con 0x7f311413d8a0 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f31127fc640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f30fc004ec0 0x7f30fc009970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3113fff640 5 monclient: authenticate success, global_id 4545 2024-09-16T15:03:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3113fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.265+0000 7f3112ffd640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f30fc004270 0x7f30fc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.268+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f30fc00bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.268+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f30fc00bec0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f310c009fd0 tx=0x7f30fc0057f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ca9bfdeae40a073f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.268+0000 7f31137fe640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f30fc00bec0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f310c009fd0 tx=0x7f30fc0057f0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f310c02bd00 mon_map magic: 0 v1 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f310c02bd00 con 0x7f311413d8a0 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 10 monclient: dump: 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c02f070 con 0x7f311413d8a0 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 4 set_mon_vals no callback set 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 10 monclient: shutdown 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 1 -- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 msgr2=0x7f30fc00bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 1 --2- 172.21.15.134:0/3769977827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f311413d8a0 0x7f30fc00bec0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f310c009fd0 tx=0x7f30fc0057f0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f31137fe640 1 -- 172.21.15.134:0/3769977827 reap_dead start 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 1 -- 172.21.15.134:0/3769977827 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 0 0 0) 0x7f310c02be80 con 0x7f311413d8a0 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3111ffb640 10 monclient: discarding stray monitor message mon_map magic: 0 v1 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 1 -- 172.21.15.134:0/3769977827 shutdown_connections 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.269+0000 7f3113fff640 1 -- 172.21.15.134:0/3769977827 >> 172.21.15.134:0/3769977827 conn(0x7f3114065e30 msgr2=0x7f3114066240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.270+0000 7f3113fff640 1 -- 172.21.15.134:0/3769977827 shutdown_connections 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.270+0000 7f3113fff640 1 -- 172.21.15.134:0/3769977827 wait complete. 2024-09-16T15:03:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.270+0000 7f3113fff640 10 monclient: build_initial_monmap 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.270+0000 7f3113fff640 10 monclient: monmap: 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 Processor -- start 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 -- start start 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient: init 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient: _add_conns ranks=[7,4,5,2,1,6,8,0,3] 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 0x7f311409f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-i con 0x7f311413d8a0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-c con 0x7f311413a170 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 0x7f31141d2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): picked mon.noname-h con 0x7f311413aab0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31141cbdc0 con 0x7f311413a170 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311413ba60 con 0x7f311413aab0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f311413cca0 con 0x7f311413d8a0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.271+0000 7f3113fff640 10 monclient(hunting): authenticate will time out at 926.860954s 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 0x7f31141d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 0x7f311409f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50464/0 (socket says 172.21.15.134:50464) 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): get_auth_request con 0x7f311413d8a0 auth_method 0 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 -- 172.21.15.134:0/749963528 learned_addr learned my addr 172.21.15.134:0/749963528 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): get_auth_request con 0x7f311413a170 auth_method 0 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): get_auth_request con 0x7f311413aab0 auth_method 0 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient(hunting): handle_auth_done global_id 4423 payload 824 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 -- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 msgr2=0x7f311409f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 0x7f311409f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 -- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 msgr2=0x7f31141d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 0x7f31141d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient: _finish_hunting 0 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 monclient: found mon.noname-c 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31127fc640 1 -- 172.21.15.134:0/749963528 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311413cef0 con 0x7f311413a170 2024-09-16T15:03:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f3112ffd640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 0x7f311409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.272+0000 7f31137fe640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 0x7f31141d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f31127fc640 1 -- 172.21.15.134:0/749963528 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 msgr2=0x7f311413a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f31127fc640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3104008600 tx=0x7f310400f530 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3104008e50 con 0x7f311413a170 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: dump: 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f3113fff640 5 monclient: authenticate success, global_id 4423 2024-09-16T15:03:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310400fe50 con 0x7f311413a170 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f3113fff640 10 monclient: _renew_subs 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f3113fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3114137220 con 0x7f311413a170 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.274+0000 7f30fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3104012bc0 con 0x7f311413a170 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f30fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f30fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f30fb7fe640 10 monclient: dump: 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:10.290 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f3113fff640 10 monclient: _renew_subs 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f3113fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140a8d90 con 0x7f311413a170 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f31117fa640 4 set_mon_vals no callback set 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.275+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f31040122e0 con 0x7f311413a170 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3113fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3113fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3113fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f30fb7fe640 1 --2- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 0x7f310003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311413c9c0 con 0x7f311413a170 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f3104049ce0 con 0x7f311413a170 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3112ffd640 1 --2- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 0x7f310003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.276+0000 7f3112ffd640 10 monclient: get_auth_request con 0x7f31000388b0 auth_method 0 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.278+0000 7f3112ffd640 1 --2- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 0x7f310003ad70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3108007fd0 tx=0x7f3108009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.281+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f310401b860 con 0x7f311413a170 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.281+0000 7f30fb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.281+0000 7f30fb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:10.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.505+0000 7f3113fff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:10.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.505+0000 7f3113fff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:10.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.505+0000 7f3113fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:10.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.505+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f31140008d0 con 0x7f311413a170 2024-09-16T15:03:10.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.506+0000 7f30fb7fe640 1 -- 172.21.15.134:0/749963528 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3189 (secure 0 0 0) 0x7f310401a030 con 0x7f311413a170 2024-09-16T15:03:10.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.506+0000 7f30fb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:10.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.506+0000 7f30fb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:10.508 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":26,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:10.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 msgr2=0x7f310003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 0x7f310003ad70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3108007fd0 tx=0x7f3108009040 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 10 monclient: shutdown 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 msgr2=0x7f311413a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3104008600 tx=0x7f310400f530 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 shutdown_connections 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> v2:172.21.15.162:6800/980547263 conn(0x7f31000388b0 0x7f310003ad70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f311413aab0 0x7f31141d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f311413a170 0x7f311413a570 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 --2- 172.21.15.134:0/749963528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f311413d8a0 0x7f311409f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 >> 172.21.15.134:0/749963528 conn(0x7f3114065e30 msgr2=0x7f31140a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 shutdown_connections 2024-09-16T15:03:10.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:10.508+0000 7f3113fff640 1 -- 172.21.15.134:0/749963528 wait complete. 2024-09-16T15:03:10.913 INFO:tasks.rados.rados.0.smithi162.stdout:476: done (1 left) 2024-09-16T15:03:10.913 INFO:tasks.rados.rados.0.smithi162.stdout:478: done (0 left) 2024-09-16T15:03:10.913 INFO:tasks.rados.rados.0.smithi162.stdout:479: delete oid 44 current snap is 44 2024-09-16T15:03:10.923 INFO:tasks.rados.rados.0.smithi162.stdout:479: done (0 left) 2024-09-16T15:03:10.923 INFO:tasks.rados.rados.0.smithi162.stdout:480: read oid 28 snap -1 2024-09-16T15:03:10.924 INFO:tasks.rados.rados.0.smithi162.stdout:480: expect deleted 2024-09-16T15:03:10.924 INFO:tasks.rados.rados.0.smithi162.stdout:481: copy_from oid 14 from oid 9 current snap is 44 2024-09-16T15:03:10.924 INFO:tasks.rados.rados.0.smithi162.stdout:482: copy_from oid 45 from oid 7 current snap is 44 2024-09-16T15:03:10.924 INFO:tasks.rados.rados.0.smithi162.stdout:483: write (excl) oid 39 current snap is 44 2024-09-16T15:03:10.924 INFO:tasks.rados.rados.0.smithi162.stdout:483: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2024-09-16T15:03:10.929 INFO:tasks.rados.rados.0.smithi162.stdout:483: writing smithi16234619-39 from 645435 to 1096658 tid 1 2024-09-16T15:03:10.938 INFO:tasks.rados.rados.0.smithi162.stdout:483: writing smithi16234619-39 from 1879985 to 2667057 tid 2 2024-09-16T15:03:10.942 INFO:tasks.rados.rados.0.smithi162.stdout:483: writing smithi16234619-39 from 3280184 to 3445435 tid 3 2024-09-16T15:03:10.942 INFO:tasks.rados.rados.0.smithi162.stdout:484: write (excl) oid 1 current snap is 44 2024-09-16T15:03:10.942 INFO:tasks.rados.rados.0.smithi162.stdout:484: seq_num 206 ranges {662242=642825,2054820=530730,3307607=154635} 2024-09-16T15:03:10.950 INFO:tasks.rados.rados.0.smithi162.stdout:484: writing smithi16234619-1 from 662242 to 1305067 tid 1 2024-09-16T15:03:10.959 INFO:tasks.rados.rados.0.smithi162.stdout:484: writing smithi16234619-1 from 2054820 to 2585550 tid 2 2024-09-16T15:03:10.965 INFO:tasks.rados.rados.0.smithi162.stdout:484: writing smithi16234619-1 from 3307607 to 3462242 tid 3 2024-09-16T15:03:10.965 INFO:tasks.rados.rados.0.smithi162.stdout:485: read oid 37 snap -1 2024-09-16T15:03:10.965 INFO:tasks.rados.rados.0.smithi162.stdout:485: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:03:10.965 INFO:tasks.rados.rados.0.smithi162.stdout:486: read oid 47 snap -1 2024-09-16T15:03:10.965 INFO:tasks.rados.rados.0.smithi162.stdout:486: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:487: read oid 12 snap -1 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:487: expect (ObjNum 186 snap 38 seq_num 186) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:488: copy_from oid 36 from oid 17 current snap is 44 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:481: finishing copy_from to smithi16234619-14 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:481: got expected ENOENT (src dne) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:481: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:480: done (8 left) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:481: done (7 left) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:489: read oid 24 snap -1 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:489: expect (ObjNum 191 snap 39 seq_num 191) 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:490: rollback oid 14 current snap is 44 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 43 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:491: copy_from oid 11 from oid 5 current snap is 44 2024-09-16T15:03:10.966 INFO:tasks.rados.rados.0.smithi162.stdout:492: read oid 32 snap -1 2024-09-16T15:03:10.967 INFO:tasks.rados.rados.0.smithi162.stdout:492: expect deleted 2024-09-16T15:03:10.967 INFO:tasks.rados.rados.0.smithi162.stdout:493: read oid 38 snap -1 2024-09-16T15:03:10.967 INFO:tasks.rados.rados.0.smithi162.stdout:493: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:10.967 INFO:tasks.rados.rados.0.smithi162.stdout:494: delete oid 10 current snap is 44 2024-09-16T15:03:10.969 INFO:tasks.rados.rados.0.smithi162.stdout:493: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:11.058 INFO:tasks.rados.rados.0.smithi162.stdout:486: expect (ObjNum 135 snap 24 seq_num 135) 2024-09-16T15:03:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:483: finishing write tid 1 to smithi16234619-39 2024-09-16T15:03:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:483: finishing write tid 2 to smithi16234619-39 2024-09-16T15:03:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:487: expect (ObjNum 186 snap 38 seq_num 186) 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:490: finishing rollback tid 0 to smithi16234619-14 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:482: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:491: finishing copy_from to smithi16234619-11 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:491: got expected ENOENT (src dne) 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:491: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:03:11.393 INFO:tasks.rados.rados.0.smithi162.stdout:485: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: finishing write tid 3 to smithi16234619-39 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: finishing write tid 4 to smithi16234619-39 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: finishing write tid 5 to smithi16234619-39 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: oid 39 updating version 0 to 197 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: oid 39 updating version 197 to 198 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: oid 39 updating version 198 to 199 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: oid 39 version 199 is already newer than 196 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 199 (ObjNum 205 snap 44 seq_num 205) dirty exists 2024-09-16T15:03:11.577 INFO:tasks.rados.rados.0.smithi162.stdout:483: left oid 39 (ObjNum 205 snap 44 seq_num 205) 2024-09-16T15:03:11.578 INFO:tasks.rados.rados.0.smithi162.stdout:489: expect (ObjNum 191 snap 39 seq_num 191) 2024-09-16T15:03:11.706 INFO:tasks.rados.rados.0.smithi162.stdout:482: finishing copy_from to smithi16234619-45 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: finishing write tid 1 to smithi16234619-1 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:488: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:490: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:490: finishing rollback tid 2 to smithi16234619-14 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 212 (ObjNum 167 snap 32 seq_num 167) dirty exists 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: finishing write tid 2 to smithi16234619-1 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: finishing write tid 3 to smithi16234619-1 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: finishing write tid 4 to smithi16234619-1 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: finishing write tid 5 to smithi16234619-1 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: oid 1 updating version 0 to 143 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: oid 1 version 143 is already newer than 140 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: oid 1 version 143 is already newer than 142 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:484: oid 1 version 143 is already newer than 141 2024-09-16T15:03:11.707 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 143 (ObjNum 206 snap 44 seq_num 206) dirty exists 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:484: left oid 1 (ObjNum 206 snap 44 seq_num 206) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:488: finishing copy_from to smithi16234619-36 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:482: done (12 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:483: done (11 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:484: done (10 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:485: done (9 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:486: done (8 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:487: done (7 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:488: done (6 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:489: done (5 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:490: done (4 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:491: done (3 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:492: done (2 left) 2024-09-16T15:03:11.708 INFO:tasks.rados.rados.0.smithi162.stdout:493: done (1 left) 2024-09-16T15:03:11.709 INFO:tasks.rados.rados.0.smithi162.stdout:494: done (0 left) 2024-09-16T15:03:11.709 INFO:tasks.rados.rados.0.smithi162.stdout:495: copy_from oid 27 from oid 9 current snap is 44 2024-09-16T15:03:11.709 INFO:tasks.rados.rados.0.smithi162.stdout:496: copy_from oid 16 from oid 12 current snap is 44 2024-09-16T15:03:11.709 INFO:tasks.rados.rados.0.smithi162.stdout:497: snap_remove snap 43 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:497: done (2 left) 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:498: rollback oid 33 current snap is 44 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:498: done (2 left) 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:499: copy_from oid 11 from oid 47 current snap is 44 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:500: read oid 50 snap -1 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:500: expect deleted 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:501: rollback oid 3 current snap is 44 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:501: done (4 left) 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:502: copy_from oid 22 from oid 6 current snap is 44 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:503: delete oid 32 current snap is 44 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:495: finishing copy_from to smithi16234619-27 2024-09-16T15:03:12.270 INFO:tasks.rados.rados.0.smithi162.stdout:495: got expected ENOENT (src dne) 2024-09-16T15:03:12.271 INFO:tasks.rados.rados.0.smithi162.stdout:495: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:03:12.271 INFO:tasks.rados.rados.0.smithi162.stdout:496: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:03:12.271 INFO:tasks.rados.rados.0.smithi162.stdout:496: finishing copy_from to smithi16234619-16 2024-09-16T15:03:12.271 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 107 (ObjNum 186 snap 38 seq_num 186) dirty exists 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:495: done (5 left) 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:496: done (4 left) 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:500: done (3 left) 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:503: done (2 left) 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:504: write (excl) oid 15 current snap is 44 2024-09-16T15:03:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:504: seq_num 209 ranges {712663=468572,1713913=678057,2873523=591733} 2024-09-16T15:03:12.286 INFO:tasks.rados.rados.0.smithi162.stdout:504: writing smithi16234619-15 from 712663 to 1181235 tid 1 2024-09-16T15:03:12.293 INFO:tasks.rados.rados.0.smithi162.stdout:504: writing smithi16234619-15 from 1713913 to 2391970 tid 2 2024-09-16T15:03:12.299 INFO:tasks.rados.rados.0.smithi162.stdout:504: writing smithi16234619-15 from 2873523 to 3465256 tid 3 2024-09-16T15:03:12.299 INFO:tasks.rados.rados.0.smithi162.stdout:505: read oid 37 snap -1 2024-09-16T15:03:12.299 INFO:tasks.rados.rados.0.smithi162.stdout:505: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:03:12.299 INFO:tasks.rados.rados.0.smithi162.stdout:506: write oid 13 current snap is 44 2024-09-16T15:03:12.299 INFO:tasks.rados.rados.0.smithi162.stdout:506: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2024-09-16T15:03:12.306 INFO:tasks.rados.rados.0.smithi162.stdout:506: writing smithi16234619-13 from 729470 to 1273291 tid 1 2024-09-16T15:03:12.311 INFO:tasks.rados.rados.0.smithi162.stdout:506: writing smithi16234619-13 from 1772395 to 2310463 tid 2 2024-09-16T15:03:12.317 INFO:tasks.rados.rados.0.smithi162.stdout:506: writing smithi16234619-13 from 2900946 to 3303951 tid 3 2024-09-16T15:03:12.317 INFO:tasks.rados.rados.0.smithi162.stdout:507: write (excl) oid 18 current snap is 44 2024-09-16T15:03:12.317 INFO:tasks.rados.rados.0.smithi162.stdout:507: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2024-09-16T15:03:12.324 INFO:tasks.rados.rados.0.smithi162.stdout:507: writing smithi16234619-18 from 746277 to 1365347 tid 1 2024-09-16T15:03:12.331 INFO:tasks.rados.rados.0.smithi162.stdout:507: writing smithi16234619-18 from 1830877 to 2512603 tid 2 2024-09-16T15:03:12.337 INFO:tasks.rados.rados.0.smithi162.stdout:507: writing smithi16234619-18 from 2928369 to 3546277 tid 3 2024-09-16T15:03:12.337 INFO:tasks.rados.rados.0.smithi162.stdout:508: read oid 49 snap -1 2024-09-16T15:03:12.337 INFO:tasks.rados.rados.0.smithi162.stdout:508: expect (ObjNum 105 snap 19 seq_num 105) 2024-09-16T15:03:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:509: read oid 38 snap -1 2024-09-16T15:03:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:509: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:510: delete oid 16 current snap is 44 2024-09-16T15:03:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:505: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-16T15:03:12.525 INFO:tasks.rados.rados.0.smithi162.stdout:499: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:03:12.525 INFO:tasks.rados.rados.0.smithi162.stdout:499: finishing copy_from to smithi16234619-11 2024-09-16T15:03:12.525 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 29 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:504: finishing write tid 1 to smithi16234619-15 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:502: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:502: finishing copy_from to smithi16234619-22 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:499: done (8 left) 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:502: done (7 left) 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:505: done (6 left) 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:510: done (5 left) 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:511: rollback oid 43 current snap is 44 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:511: done (5 left) 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:512: write (excl) oid 27 current snap is 44 2024-09-16T15:03:12.526 INFO:tasks.rados.rados.0.smithi162.stdout:509: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:12.608 INFO:tasks.rados.rados.0.smithi162.stdout:504: finishing write tid 2 to smithi16234619-15 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:506: finishing write tid 1 to smithi16234619-13 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: finishing write tid 3 to smithi16234619-15 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: finishing write tid 4 to smithi16234619-15 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: finishing write tid 5 to smithi16234619-15 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: oid 15 updating version 0 to 81 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: oid 15 version 81 is already newer than 79 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: oid 15 version 81 is already newer than 80 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: oid 15 version 81 is already newer than 78 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 81 (ObjNum 209 snap 44 seq_num 209) dirty exists 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:504: left oid 15 (ObjNum 209 snap 44 seq_num 209) 2024-09-16T15:03:12.609 INFO:tasks.rados.rados.0.smithi162.stdout:508: expect (ObjNum 105 snap 19 seq_num 105) 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: finishing write tid 2 to smithi16234619-13 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: finishing write tid 3 to smithi16234619-13 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: finishing write tid 4 to smithi16234619-13 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: finishing write tid 5 to smithi16234619-13 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: oid 13 updating version 0 to 204 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: oid 13 version 204 is already newer than 202 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: oid 13 updating version 204 to 205 2024-09-16T15:03:12.809 INFO:tasks.rados.rados.0.smithi162.stdout:506: oid 13 version 205 is already newer than 203 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 205 (ObjNum 210 snap 44 seq_num 210) dirty exists 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:506: left oid 13 (ObjNum 210 snap 44 seq_num 210) 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:507: finishing write tid 1 to smithi16234619-18 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:507: finishing write tid 2 to smithi16234619-18 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:507: finishing write tid 3 to smithi16234619-18 2024-09-16T15:03:12.810 INFO:tasks.rados.rados.0.smithi162.stdout:512: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2024-09-16T15:03:12.815 INFO:tasks.rados.rados.0.smithi162.stdout:512: writing smithi16234619-27 from 779891 to 1265812 tid 1 2024-09-16T15:03:12.822 INFO:tasks.rados.rados.0.smithi162.stdout:512: writing smithi16234619-27 from 1947841 to 2633236 tid 2 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:512: writing smithi16234619-27 from 3383215 to 3579891 tid 3 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:504: done (5 left) 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:506: done (4 left) 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:508: done (3 left) 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:509: done (2 left) 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:513: copy_from oid 21 from oid 38 current snap is 44 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:507: finishing write tid 4 to smithi16234619-18 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:507: finishing write tid 5 to smithi16234619-18 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:507: oid 18 updating version 0 to 165 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:507: oid 18 version 165 is already newer than 163 2024-09-16T15:03:12.829 INFO:tasks.rados.rados.0.smithi162.stdout:507: oid 18 version 165 is already newer than 162 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:507: oid 18 version 165 is already newer than 164 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 165 (ObjNum 211 snap 44 seq_num 211) dirty exists 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:507: left oid 18 (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:507: done (2 left) 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:514: write (excl) oid 13 current snap is 44 2024-09-16T15:03:12.830 INFO:tasks.rados.rados.0.smithi162.stdout:514: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2024-09-16T15:03:12.835 INFO:tasks.rados.rados.0.smithi162.stdout:514: writing smithi16234619-13 from 796698 to 1357868 tid 1 2024-09-16T15:03:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:514: writing smithi16234619-13 from 2006323 to 2551729 tid 2 2024-09-16T15:03:12.849 INFO:tasks.rados.rados.0.smithi162.stdout:514: writing smithi16234619-13 from 3126991 to 3596698 tid 3 2024-09-16T15:03:12.849 INFO:tasks.rados.rados.0.smithi162.stdout:515: write oid 14 current snap is 44 2024-09-16T15:03:12.849 INFO:tasks.rados.rados.0.smithi162.stdout:512: finishing write tid 1 to smithi16234619-27 2024-09-16T15:03:12.849 INFO:tasks.rados.rados.0.smithi162.stdout:515: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2024-09-16T15:03:12.854 INFO:tasks.rados.rados.0.smithi162.stdout:515: writing smithi16234619-14 from 413505 to 1049924 tid 1 2024-09-16T15:03:12.861 INFO:tasks.rados.rados.0.smithi162.stdout:515: writing smithi16234619-14 from 1664805 to 2353869 tid 2 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:515: writing smithi16234619-14 from 3038061 to 3530132 tid 3 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:516: read oid 23 snap -1 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:516: expect (ObjNum 173 snap 35 seq_num 173) 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:517: write (excl) oid 24 current snap is 44 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:512: finishing write tid 2 to smithi16234619-27 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:512: finishing write tid 3 to smithi16234619-27 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:512: finishing write tid 4 to smithi16234619-27 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:512: finishing write tid 5 to smithi16234619-27 2024-09-16T15:03:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:512: oid 27 updating version 0 to 149 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:512: oid 27 version 149 is already newer than 147 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:512: oid 27 version 149 is already newer than 148 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:512: oid 27 version 149 is already newer than 146 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 149 (ObjNum 213 snap 44 seq_num 213) dirty exists 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:512: left oid 27 (ObjNum 213 snap 44 seq_num 213) 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:513: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:513: finishing copy_from to smithi16234619-21 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T15:03:12.869 INFO:tasks.rados.rados.0.smithi162.stdout:517: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2024-09-16T15:03:12.874 INFO:tasks.rados.rados.0.smithi162.stdout:517: writing smithi16234619-24 from 430312 to 1141980 tid 1 2024-09-16T15:03:12.880 INFO:tasks.rados.rados.0.smithi162.stdout:517: writing smithi16234619-24 from 1723287 to 2272362 tid 2 2024-09-16T15:03:12.887 INFO:tasks.rados.rados.0.smithi162.stdout:517: writing smithi16234619-24 from 2781837 to 3485180 tid 3 2024-09-16T15:03:12.887 INFO:tasks.rados.rados.0.smithi162.stdout:512: done (5 left) 2024-09-16T15:03:12.887 INFO:tasks.rados.rados.0.smithi162.stdout:513: done (4 left) 2024-09-16T15:03:12.887 INFO:tasks.rados.rados.0.smithi162.stdout:518: write oid 37 current snap is 44 2024-09-16T15:03:12.887 INFO:tasks.rados.rados.0.smithi162.stdout:518: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2024-09-16T15:03:12.893 INFO:tasks.rados.rados.0.smithi162.stdout:518: writing smithi16234619-37 from 447119 to 1234036 tid 1 2024-09-16T15:03:12.900 INFO:tasks.rados.rados.0.smithi162.stdout:518: writing smithi16234619-37 from 1665416 to 2358149 tid 2 2024-09-16T15:03:12.906 INFO:tasks.rados.rados.0.smithi162.stdout:518: writing smithi16234619-37 from 2976554 to 3491169 tid 3 2024-09-16T15:03:12.906 INFO:tasks.rados.rados.0.smithi162.stdout:519: write oid 8 current snap is 44 2024-09-16T15:03:12.906 INFO:tasks.rados.rados.0.smithi162.stdout:519: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2024-09-16T15:03:12.911 INFO:tasks.rados.rados.0.smithi162.stdout:519: writing smithi16234619-8 from 463926 to 926092 tid 1 2024-09-16T15:03:12.918 INFO:tasks.rados.rados.0.smithi162.stdout:519: writing smithi16234619-8 from 1723898 to 2160289 tid 2 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:519: writing smithi16234619-8 from 2603977 to 3329864 tid 3 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:520: read oid 3 snap -1 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:520: expect (ObjNum 189 snap 39 seq_num 189) 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:521: rollback oid 12 current snap is 44 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:521: done (7 left) 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:522: write oid 20 current snap is 44 2024-09-16T15:03:12.925 INFO:tasks.rados.rados.0.smithi162.stdout:516: expect (ObjNum 173 snap 35 seq_num 173) 2024-09-16T15:03:13.145 INFO:tasks.rados.rados.0.smithi162.stdout:514: finishing write tid 1 to smithi16234619-13 2024-09-16T15:03:13.145 INFO:tasks.rados.rados.0.smithi162.stdout:514: finishing write tid 2 to smithi16234619-13 2024-09-16T15:03:13.145 INFO:tasks.rados.rados.0.smithi162.stdout:522: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2024-09-16T15:03:13.150 INFO:tasks.rados.rados.0.smithi162.stdout:522: writing smithi16234619-20 from 480733 to 1018148 tid 1 2024-09-16T15:03:13.158 INFO:tasks.rados.rados.0.smithi162.stdout:522: writing smithi16234619-20 from 1782380 to 2478782 tid 2 2024-09-16T15:03:13.165 INFO:tasks.rados.rados.0.smithi162.stdout:522: writing smithi16234619-20 from 3031400 to 3568559 tid 3 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:516: done (7 left) 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:523: snap_create 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: finishing write tid 3 to smithi16234619-13 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: finishing write tid 4 to smithi16234619-13 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: finishing write tid 5 to smithi16234619-13 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: oid 13 updating version 0 to 207 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: oid 13 version 207 is already newer than 206 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: oid 13 updating version 207 to 209 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: oid 13 version 209 is already newer than 208 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 209 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:514: left oid 13 (ObjNum 214 snap 44 seq_num 214) 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:515: finishing write tid 1 to smithi16234619-14 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:515: finishing write tid 2 to smithi16234619-14 2024-09-16T15:03:13.166 INFO:tasks.rados.rados.0.smithi162.stdout:515: finishing write tid 3 to smithi16234619-14 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: finishing write tid 4 to smithi16234619-14 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: finishing write tid 5 to smithi16234619-14 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: oid 14 updating version 0 to 219 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: oid 14 updating version 219 to 222 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: oid 14 version 222 is already newer than 220 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: oid 14 version 222 is already newer than 221 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 222 (ObjNum 215 snap 44 seq_num 215) dirty exists 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:515: left oid 14 (ObjNum 215 snap 44 seq_num 215) 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:517: finishing write tid 1 to smithi16234619-24 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:517: finishing write tid 2 to smithi16234619-24 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:517: finishing write tid 3 to smithi16234619-24 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:517: finishing write tid 4 to smithi16234619-24 2024-09-16T15:03:13.167 INFO:tasks.rados.rados.0.smithi162.stdout:520: expect (ObjNum 189 snap 39 seq_num 189) 2024-09-16T15:03:13.272 INFO:tasks.rados.rados.0.smithi162.stdout:517: finishing write tid 5 to smithi16234619-24 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: oid 24 updating version 0 to 170 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: oid 24 version 170 is already newer than 169 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: oid 24 updating version 170 to 171 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: oid 24 updating version 171 to 172 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 172 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: left oid 24 (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:518: finishing write tid 1 to smithi16234619-37 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:514: done (7 left) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:515: done (6 left) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:517: done (5 left) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:520: done (4 left) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:523: done (3 left) 2024-09-16T15:03:13.273 INFO:tasks.rados.rados.0.smithi162.stdout:524: delete oid 31 current snap is 45 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:519: finishing write tid 1 to smithi16234619-8 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: finishing write tid 2 to smithi16234619-37 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: finishing write tid 3 to smithi16234619-37 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: finishing write tid 4 to smithi16234619-37 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: finishing write tid 5 to smithi16234619-37 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: oid 37 updating version 0 to 212 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: oid 37 version 212 is already newer than 211 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: oid 37 version 212 is already newer than 210 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: oid 37 updating version 212 to 213 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 213 (ObjNum 217 snap 44 seq_num 217) dirty exists 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:518: left oid 37 (ObjNum 217 snap 44 seq_num 217) 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:519: finishing write tid 2 to smithi16234619-8 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:519: finishing write tid 3 to smithi16234619-8 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:519: finishing write tid 4 to smithi16234619-8 2024-09-16T15:03:13.274 INFO:tasks.rados.rados.0.smithi162.stdout:519: finishing write tid 5 to smithi16234619-8 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:519: oid 8 updating version 0 to 166 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:519: oid 8 updating version 166 to 168 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:519: oid 8 version 168 is already newer than 167 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:519: oid 8 updating version 168 to 169 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 169 (ObjNum 218 snap 44 seq_num 218) dirty exists 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:519: left oid 8 (ObjNum 218 snap 44 seq_num 218) 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: finishing write tid 1 to smithi16234619-20 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: finishing write tid 2 to smithi16234619-20 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: finishing write tid 3 to smithi16234619-20 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: finishing write tid 4 to smithi16234619-20 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: finishing write tid 5 to smithi16234619-20 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: oid 20 updating version 0 to 44 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: oid 20 updating version 44 to 45 2024-09-16T15:03:13.275 INFO:tasks.rados.rados.0.smithi162.stdout:522: oid 20 version 45 is already newer than 43 2024-09-16T15:03:13.276 INFO:tasks.rados.rados.0.smithi162.stdout:522: oid 20 updating version 45 to 46 2024-09-16T15:03:13.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 46 (ObjNum 219 snap 44 seq_num 219) dirty exists 2024-09-16T15:03:13.276 INFO:tasks.rados.rados.0.smithi162.stdout:522: left oid 20 (ObjNum 219 snap 44 seq_num 219) 2024-09-16T15:03:13.278 INFO:tasks.rados.rados.0.smithi162.stdout:518: done (3 left) 2024-09-16T15:03:13.278 INFO:tasks.rados.rados.0.smithi162.stdout:519: done (2 left) 2024-09-16T15:03:13.278 INFO:tasks.rados.rados.0.smithi162.stdout:522: done (1 left) 2024-09-16T15:03:13.278 INFO:tasks.rados.rados.0.smithi162.stdout:524: done (0 left) 2024-09-16T15:03:13.278 INFO:tasks.rados.rados.0.smithi162.stdout:525: delete oid 16 current snap is 45 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:525: done (0 left) 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:526: rollback oid 47 current snap is 45 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 44 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:527: rollback oid 15 current snap is 45 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 44 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:528: write oid 48 current snap is 45 2024-09-16T15:03:13.281 INFO:tasks.rados.rados.0.smithi162.stdout:528: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2024-09-16T15:03:13.286 INFO:tasks.rados.rados.0.smithi162.stdout:528: writing smithi16234619-48 from 531154 to 1010669 tid 1 2024-09-16T15:03:13.292 INFO:tasks.rados.rados.0.smithi162.stdout:528: writing smithi16234619-48 from 1557826 to 2001555 tid 2 2024-09-16T15:03:13.299 INFO:tasks.rados.rados.0.smithi162.stdout:528: writing smithi16234619-48 from 2713669 to 3200997 tid 3 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:528: writing smithi16234619-48 from 3679295 to 3731154 tid 4 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:529: write oid 11 current snap is 45 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:526: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:526: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:527: finishing rollback tid 0 to smithi16234619-15 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:527: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:03:13.302 INFO:tasks.rados.rados.0.smithi162.stdout:529: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2024-09-16T15:03:13.308 INFO:tasks.rados.rados.0.smithi162.stdout:529: writing smithi16234619-11 from 547961 to 1102725 tid 1 2024-09-16T15:03:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:529: writing smithi16234619-11 from 1616308 to 2320048 tid 2 2024-09-16T15:03:13.323 INFO:tasks.rados.rados.0.smithi162.stdout:529: writing smithi16234619-11 from 2741092 to 3439692 tid 3 2024-09-16T15:03:13.323 INFO:tasks.rados.rados.0.smithi162.stdout:530: write (excl) oid 44 current snap is 45 2024-09-16T15:03:13.323 INFO:tasks.rados.rados.0.smithi162.stdout:526: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:03:13.323 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 45 (ObjNum 135 snap 24 seq_num 135) dirty exists 2024-09-16T15:03:13.324 INFO:tasks.rados.rados.0.smithi162.stdout:527: finishing rollback tid 2 to smithi16234619-15 2024-09-16T15:03:13.324 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 85 (ObjNum 209 snap 44 seq_num 209) dirty exists 2024-09-16T15:03:13.324 INFO:tasks.rados.rados.0.smithi162.stdout:530: seq_num 224 ranges {564768=630013,1674790=447398,2768515=626225} 2024-09-16T15:03:13.330 INFO:tasks.rados.rados.0.smithi162.stdout:530: writing smithi16234619-44 from 564768 to 1194781 tid 1 2024-09-16T15:03:13.335 INFO:tasks.rados.rados.0.smithi162.stdout:530: writing smithi16234619-44 from 1674790 to 2122188 tid 2 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:530: writing smithi16234619-44 from 2768515 to 3394740 tid 3 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:526: done (4 left) 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:527: done (3 left) 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:531: copy_from oid 14 from oid 49 current snap is 45 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 1 to smithi16234619-48 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 2 to smithi16234619-48 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 3 to smithi16234619-48 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 4 to smithi16234619-48 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 5 to smithi16234619-48 2024-09-16T15:03:13.344 INFO:tasks.rados.rados.0.smithi162.stdout:528: finishing write tid 6 to smithi16234619-48 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: oid 48 updating version 0 to 152 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: oid 48 version 152 is already newer than 151 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: oid 48 updating version 152 to 154 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: oid 48 version 154 is already newer than 153 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: oid 48 updating version 154 to 155 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 155 (ObjNum 222 snap 45 seq_num 222) dirty exists 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: left oid 48 (ObjNum 222 snap 45 seq_num 222) 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:528: done (3 left) 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:532: read oid 40 snap 44 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:532: expect deleted 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:533: rollback oid 23 current snap is 45 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 44 2024-09-16T15:03:13.345 INFO:tasks.rados.rados.0.smithi162.stdout:534: copy_from oid 16 from oid 27 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:535: copy_from oid 48 from oid 18 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:536: copy_from oid 1 from oid 37 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:537: copy_from oid 4 from oid 17 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:538: read oid 33 snap -1 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:538: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:539: copy_from oid 28 from oid 13 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:540: rollback oid 5 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 44 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:541: write (excl) oid 50 current snap is 45 2024-09-16T15:03:13.346 INFO:tasks.rados.rados.0.smithi162.stdout:541: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2024-09-16T15:03:13.351 INFO:tasks.rados.rados.0.smithi162.stdout:541: writing smithi16234619-50 from 581575 to 1286837 tid 1 2024-09-16T15:03:13.359 INFO:tasks.rados.rados.0.smithi162.stdout:541: writing smithi16234619-50 from 1733272 to 2440681 tid 2 2024-09-16T15:03:13.367 INFO:tasks.rados.rados.0.smithi162.stdout:541: writing smithi16234619-50 from 3195938 to 3633435 tid 3 2024-09-16T15:03:13.367 INFO:tasks.rados.rados.0.smithi162.stdout:542: rollback oid 39 current snap is 45 2024-09-16T15:03:13.367 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 44 2024-09-16T15:03:13.367 INFO:tasks.rados.rados.0.smithi162.stdout:538: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:533: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:533: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:533: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 54 (ObjNum 173 snap 35 seq_num 173) dirty exists 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:540: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:529: finishing write tid 1 to smithi16234619-11 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:529: finishing write tid 2 to smithi16234619-11 2024-09-16T15:03:13.440 INFO:tasks.rados.rados.0.smithi162.stdout:529: finishing write tid 3 to smithi16234619-11 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: finishing write tid 4 to smithi16234619-11 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: finishing write tid 5 to smithi16234619-11 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: oid 11 updating version 0 to 38 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: oid 11 version 38 is already newer than 37 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: oid 11 updating version 38 to 39 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: oid 11 updating version 39 to 40 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 40 (ObjNum 223 snap 45 seq_num 223) dirty exists 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:529: left oid 11 (ObjNum 223 snap 45 seq_num 223) 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:542: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:542: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:542: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 217 (ObjNum 205 snap 44 seq_num 205) dirty exists 2024-09-16T15:03:13.441 INFO:tasks.rados.rados.0.smithi162.stdout:530: finishing write tid 1 to smithi16234619-44 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: finishing write tid 2 to smithi16234619-44 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: finishing write tid 3 to smithi16234619-44 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: finishing write tid 4 to smithi16234619-44 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: finishing write tid 5 to smithi16234619-44 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: oid 44 updating version 0 to 129 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: oid 44 version 129 is already newer than 128 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: oid 44 version 129 is already newer than 127 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: oid 44 updating version 129 to 130 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 130 (ObjNum 224 snap 45 seq_num 224) dirty exists 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: left oid 44 (ObjNum 224 snap 45 seq_num 224) 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:531: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:531: finishing copy_from to smithi16234619-14 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 95 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:529: done (13 left) 2024-09-16T15:03:13.442 INFO:tasks.rados.rados.0.smithi162.stdout:530: done (12 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:531: done (11 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:532: done (10 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:533: done (9 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:538: done (8 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:540: done (7 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:542: done (6 left) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:543: read oid 46 snap -1 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:543: expect (ObjNum 203 snap 42 seq_num 203) 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:544: rollback oid 49 current snap is 45 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 44 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:545: write oid 44 current snap is 45 2024-09-16T15:03:13.443 INFO:tasks.rados.rados.0.smithi162.stdout:545: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2024-09-16T15:03:13.446 INFO:tasks.rados.rados.0.smithi162.stdout:545: writing smithi16234619-44 from 598382 to 1378893 tid 1 2024-09-16T15:03:13.453 INFO:tasks.rados.rados.0.smithi162.stdout:545: writing smithi16234619-44 from 2075401 to 2526468 tid 2 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:545: writing smithi16234619-44 from 3107008 to 3755777 tid 3 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:546: copy_from oid 2 from oid 38 current snap is 45 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:544: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:544: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:544: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:03:13.460 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 103 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:03:13.461 INFO:tasks.rados.rados.0.smithi162.stdout:541: finishing write tid 1 to smithi16234619-50 2024-09-16T15:03:13.461 INFO:tasks.rados.rados.0.smithi162.stdout:539: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:03:13.461 INFO:tasks.rados.rados.0.smithi162.stdout:539: finishing copy_from to smithi16234619-28 2024-09-16T15:03:13.461 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 209 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:13.461 INFO:tasks.rados.rados.0.smithi162.stdout:543: expect (ObjNum 203 snap 42 seq_num 203) 2024-09-16T15:03:13.520 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:13.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.614+0000 7f4aafbf4640 10 monclient: get_monmap_and_config 2024-09-16T15:03:13.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.614+0000 7f4aafbf4640 10 monclient: build_initial_monmap 2024-09-16T15:03:13.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.614+0000 7f4aafbf4640 10 monclient: monmap: 2024-09-16T15:03:13.616 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:13.616 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:13.615504+0000 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:13.615504+0000 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:13.617 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:13.618 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 Processor -- start 2024-09-16T15:03:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 -- start start 2024-09-16T15:03:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient: init 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient: _add_conns ranks=[1,2,4,10,3,7,6,8,0,5,9] 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4aa8136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-b con 0x7f4aa809bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-c con 0x7f4aa813bf80 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa8073250 con 0x7f4aa8136030 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa809b600 con 0x7f4aa809bfe0 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80a5740 con 0x7f4aa813bf80 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aafbf4640 10 monclient(hunting): authenticate will time out at 930.204813s 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45172/0 (socket says 172.21.15.134:45172) 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50476/0 (socket says 172.21.15.134:50476) 2024-09-16T15:03:13.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 1 -- 172.21.15.134:0/977161970 learned_addr learned my addr 172.21.15.134:0/977161970 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 10 monclient(hunting): get_auth_request con 0x7f4aa809bfe0 auth_method 0 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aad18f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 10 monclient(hunting): get_auth_request con 0x7f4aa813bf80 auth_method 0 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.615+0000 7f4aae191640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:13.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient(hunting): handle_auth_done global_id 4432 payload 274 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 1 -- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 msgr2=0x7f4aa809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 1 -- 172.21.15.134:0/977161970 >> v1:172.21.15.162:6789/0 conn(0x7f4aa8136030 legacy=0x7f4aa8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient: _finish_hunting 0 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 1 monclient: found mon.noname-c 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aae191640 1 -- 172.21.15.134:0/977161970 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa80a52c0 con 0x7f4aa813bf80 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.616+0000 7f4aad18f640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.618+0000 7f4aae191640 1 -- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 msgr2=0x7f4aa813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.618+0000 7f4aae191640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009dc0 tx=0x7f4aa002a4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bb50a655ea70e430 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:13.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 1 -- 172.21.15.134:0/977161970 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4aa0007bc0 con 0x7f4aa813bf80 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: dump: 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:13.628 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: _finish_auth 0 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: _check_auth_tickets 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 1 -- 172.21.15.134:0/977161970 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa002d050 con 0x7f4aa813bf80 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 5 monclient: authenticate success, global_id 4432 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:13.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 1 -- 172.21.15.134:0/977161970 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4aa0007d40 con 0x7f4aa813bf80 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 4 set_mon_vals no callback set 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aac98e640 10 monclient: dump: 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:13.630 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 10 monclient: shutdown 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 -- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 msgr2=0x7f4aa813e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009dc0 tx=0x7f4aa002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 -- 172.21.15.134:0/977161970 shutdown_connections 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aa813bf80 0x7f4aa813e370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/977161970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa809bfe0 0x7f4aa809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 -- 172.21.15.134:0/977161970 >> 172.21.15.134:0/977161970 conn(0x7f4aa80a9ea0 msgr2=0x7f4aa80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 -- 172.21.15.134:0/977161970 shutdown_connections 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.619+0000 7f4aafbf4640 1 -- 172.21.15.134:0/977161970 wait complete. 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.620+0000 7f4aafbf4640 10 monclient: build_initial_monmap 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.620+0000 7f4aafbf4640 10 monclient: monmap: 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:13.632 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:13.633 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.620+0000 7f4aafbf4640 1 Processor -- start 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.620+0000 7f4aafbf4640 1 -- start start 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.620+0000 7f4aafbf4640 10 monclient: init 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient: _add_conns ranks=[6,3,8,7,1,4,2,0,5] 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-d con 0x7f4aa809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa8136030 0x7f4aa8134060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-g con 0x7f4aa8136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 0x7f4aa81345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): picked mon.noname-e con 0x7f4aa813bf80 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:13.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa813a980 con 0x7f4aa8136030 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81cb9d0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80a82d0 con 0x7f4aa813bf80 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aafbf4640 10 monclient(hunting): authenticate will time out at 930.210207s 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 0x7f4aa81345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:41306/0 (socket says 172.21.15.134:41306) 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 0x7f4aa81345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:39682/0 (socket says 172.21.15.134:39682) 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 -- 172.21.15.134:0/4152075043 learned_addr learned my addr 172.21.15.134:0/4152075043 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): get_auth_request con 0x7f4aa809bfe0 auth_method 0 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:13.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): get_auth_request con 0x7f4aa813bf80 auth_method 0 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa8136030 0x7f4aa8134060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): get_auth_request con 0x7f4aa8136030 auth_method 0 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:13.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aae191640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad18f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient(hunting): handle_auth_done global_id 34107 payload 824 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 -- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 msgr2=0x7f4aa81345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 0x7f4aa81345a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 -- 172.21.15.134:0/4152075043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa8136030 msgr2=0x7f4aa8134060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa8136030 0x7f4aa8134060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient: _finish_hunting 0 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 monclient: found mon.noname-d 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:13.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:13.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.621+0000 7f4aad990640 1 -- 172.21.15.134:0/4152075043 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980094d0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.623+0000 7f4aad990640 1 -- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 msgr2=0x7f4aa81cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.623+0000 7f4aad990640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c430 tx=0x7f4aa400c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.623+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4aa4003860 con 0x7f4aa809bfe0 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.623+0000 7f4a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: dump: 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.640 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:13.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 5 monclient: authenticate success, global_id 34107 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4011c10 con 0x7f4aa809bfe0 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4aa4011dc0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _renew_subs 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa0009a70 con 0x7f4aa809bfe0 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9ffff640 4 set_mon_vals no callback set 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 10 monclient: dump: 2024-09-16T15:03:13.642 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:13.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _renew_subs 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa80a48c0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aad990640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c430 tx=0x7f4aa400c8f0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f4aa4049fa0 osd_map(112..112 src has 1..112) v4 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4aa40117a0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.624+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa809c9c0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.625+0000 7f4a9e7fc640 1 --2- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 0x7f4a8c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.625+0000 7f4aad18f640 1 --2- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 0x7f4a8c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.625+0000 7f4aad18f640 10 monclient: get_auth_request con 0x7f4a8c038880 auth_method 0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.627+0000 7f4aad18f640 1 --2- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 0x7f4a8c03ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a98006cc0 tx=0x7f4a98009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:13.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:13.629+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f4aa4049fa0 con 0x7f4aa809bfe0 2024-09-16T15:03:13.679 INFO:tasks.rados.rados.0.smithi162.stdout:545: finishing write tid 1 to smithi16234619-44 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: finishing write tid 2 to smithi16234619-44 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: finishing write tid 3 to smithi16234619-44 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: finishing write tid 4 to smithi16234619-44 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: finishing write tid 5 to smithi16234619-44 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: oid 44 updating version 0 to 132 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: oid 44 updating version 132 to 133 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: oid 44 version 133 is already newer than 131 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: oid 44 updating version 133 to 134 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 134 (ObjNum 226 snap 45 seq_num 226) dirty exists 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:545: left oid 44 (ObjNum 226 snap 45 seq_num 226) 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:536: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:537: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:03:13.680 INFO:tasks.rados.rados.0.smithi162.stdout:539: done (9 left) 2024-09-16T15:03:13.681 INFO:tasks.rados.rados.0.smithi162.stdout:543: done (8 left) 2024-09-16T15:03:13.681 INFO:tasks.rados.rados.0.smithi162.stdout:544: done (7 left) 2024-09-16T15:03:13.681 INFO:tasks.rados.rados.0.smithi162.stdout:545: done (6 left) 2024-09-16T15:03:13.681 INFO:tasks.rados.rados.0.smithi162.stdout:547: snap_remove snap 44 2024-09-16T15:03:14.288 INFO:tasks.rados.rados.0.smithi162.stdout:547: done (6 left) 2024-09-16T15:03:14.288 INFO:tasks.rados.rados.0.smithi162.stdout:548: copy_from oid 29 from oid 14 current snap is 45 2024-09-16T15:03:14.289 INFO:tasks.rados.rados.0.smithi162.stdout:549: rollback oid 12 current snap is 45 2024-09-16T15:03:14.289 INFO:tasks.rados.rados.0.smithi162.stdout:549: done (7 left) 2024-09-16T15:03:14.289 INFO:tasks.rados.rados.0.smithi162.stdout:550: write oid 47 current snap is 45 2024-09-16T15:03:14.289 INFO:tasks.rados.rados.0.smithi162.stdout:550: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2024-09-16T15:03:14.294 INFO:tasks.rados.rados.0.smithi162.stdout:550: writing smithi16234619-47 from 615189 to 1070949 tid 1 2024-09-16T15:03:14.302 INFO:tasks.rados.rados.0.smithi162.stdout:550: writing smithi16234619-47 from 1733883 to 2444961 tid 2 2024-09-16T15:03:14.309 INFO:tasks.rados.rados.0.smithi162.stdout:550: writing smithi16234619-47 from 3134431 to 3594472 tid 3 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:551: write oid 8 current snap is 45 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: finishing write tid 2 to smithi16234619-50 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:535: finishing copy_from to smithi16234619-48 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 165 (ObjNum 211 snap 44 seq_num 211) dirty exists 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:537: finishing copy_from to smithi16234619-4 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: finishing write tid 3 to smithi16234619-50 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: finishing write tid 4 to smithi16234619-50 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: finishing write tid 5 to smithi16234619-50 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: oid 50 updating version 0 to 226 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: oid 50 version 226 is already newer than 225 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: oid 50 updating version 226 to 228 2024-09-16T15:03:14.310 INFO:tasks.rados.rados.0.smithi162.stdout:541: oid 50 version 228 is already newer than 227 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 228 (ObjNum 225 snap 45 seq_num 225) dirty exists 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:541: left oid 50 (ObjNum 225 snap 45 seq_num 225) 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:535: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:536: finishing copy_from to smithi16234619-1 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 213 (ObjNum 217 snap 44 seq_num 217) dirty exists 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:546: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:546: finishing copy_from to smithi16234619-2 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:534: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:03:14.311 INFO:tasks.rados.rados.0.smithi162.stdout:551: seq_num 228 ranges {631996=531009,1792365=454736,2761854=671313} 2024-09-16T15:03:14.315 INFO:tasks.rados.rados.0.smithi162.stdout:551: writing smithi16234619-8 from 631996 to 1163005 tid 1 2024-09-16T15:03:14.321 INFO:tasks.rados.rados.0.smithi162.stdout:551: writing smithi16234619-8 from 1792365 to 2247101 tid 2 2024-09-16T15:03:14.328 INFO:tasks.rados.rados.0.smithi162.stdout:551: writing smithi16234619-8 from 2761854 to 3433167 tid 3 2024-09-16T15:03:14.328 INFO:tasks.rados.rados.0.smithi162.stdout:535: done (8 left) 2024-09-16T15:03:14.328 INFO:tasks.rados.rados.0.smithi162.stdout:536: done (7 left) 2024-09-16T15:03:14.328 INFO:tasks.rados.rados.0.smithi162.stdout:537: done (6 left) 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:541: done (5 left) 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:546: done (4 left) 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:552: copy_from oid 30 from oid 28 current snap is 45 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:534: finishing copy_from to smithi16234619-16 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 149 (ObjNum 213 snap 44 seq_num 213) dirty exists 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:534: done (4 left) 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:553: write oid 5 current snap is 45 2024-09-16T15:03:14.329 INFO:tasks.rados.rados.0.smithi162.stdout:553: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2024-09-16T15:03:14.335 INFO:tasks.rados.rados.0.smithi162.stdout:553: writing smithi16234619-5 from 648803 to 1371414 tid 1 2024-09-16T15:03:14.342 INFO:tasks.rados.rados.0.smithi162.stdout:553: writing smithi16234619-5 from 1967200 to 2681947 tid 2 2024-09-16T15:03:14.350 INFO:tasks.rados.rados.0.smithi162.stdout:553: writing smithi16234619-5 from 3421983 to 3848803 tid 3 2024-09-16T15:03:14.350 INFO:tasks.rados.rados.0.smithi162.stdout:550: finishing write tid 1 to smithi16234619-47 2024-09-16T15:03:14.350 INFO:tasks.rados.rados.0.smithi162.stdout:550: finishing write tid 2 to smithi16234619-47 2024-09-16T15:03:14.350 INFO:tasks.rados.rados.0.smithi162.stdout:550: finishing write tid 3 to smithi16234619-47 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:554: read oid 18 snap -1 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:554: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:555: rollback oid 31 current snap is 45 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:555: done (6 left) 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:556: read oid 6 snap -1 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:556: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:557: copy_from oid 27 from oid 24 current snap is 45 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: finishing write tid 4 to smithi16234619-47 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: finishing write tid 5 to smithi16234619-47 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: oid 47 updating version 0 to 51 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: oid 47 version 51 is already newer than 49 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: oid 47 version 51 is already newer than 48 2024-09-16T15:03:14.351 INFO:tasks.rados.rados.0.smithi162.stdout:550: oid 47 version 51 is already newer than 50 2024-09-16T15:03:14.352 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 51 (ObjNum 227 snap 45 seq_num 227) dirty exists 2024-09-16T15:03:14.352 INFO:tasks.rados.rados.0.smithi162.stdout:550: left oid 47 (ObjNum 227 snap 45 seq_num 227) 2024-09-16T15:03:14.352 INFO:tasks.rados.rados.0.smithi162.stdout:550: done (7 left) 2024-09-16T15:03:14.352 INFO:tasks.rados.rados.0.smithi162.stdout:558: write oid 17 current snap is 45 2024-09-16T15:03:14.352 INFO:tasks.rados.rados.0.smithi162.stdout:558: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2024-09-16T15:03:14.357 INFO:tasks.rados.rados.0.smithi162.stdout:558: writing smithi16234619-17 from 665610 to 1463470 tid 1 2024-09-16T15:03:14.362 INFO:tasks.rados.rados.0.smithi162.stdout:558: writing smithi16234619-17 from 1909329 to 2367734 tid 2 2024-09-16T15:03:14.367 INFO:tasks.rados.rados.0.smithi162.stdout:558: writing smithi16234619-17 from 2816700 to 3226910 tid 3 2024-09-16T15:03:14.367 INFO:tasks.rados.rados.0.smithi162.stdout:559: copy_from oid 7 from oid 22 current snap is 45 2024-09-16T15:03:14.367 INFO:tasks.rados.rados.0.smithi162.stdout:556: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:14.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.452+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4aa401c740 con 0x7f4aa809bfe0 2024-09-16T15:03:14.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.452+0000 7f4a9e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:14.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.452+0000 7f4a9e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:14.506 INFO:tasks.rados.rados.0.smithi162.stdout:548: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:551: finishing write tid 1 to smithi16234619-8 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:556: done (8 left) 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:560: snap_create 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:551: finishing write tid 2 to smithi16234619-8 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:551: finishing write tid 3 to smithi16234619-8 2024-09-16T15:03:14.507 INFO:tasks.rados.rados.0.smithi162.stdout:554: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:14.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.675+0000 7f4aafbf4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:14.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.675+0000 7f4aafbf4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:14.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.675+0000 7f4aafbf4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:14.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.675+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4aa809c400 con 0x7f4aa809bfe0 2024-09-16T15:03:14.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.676+0000 7f4a9e7fc640 1 -- 172.21.15.134:0/4152075043 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f4aa40102e0 con 0x7f4aa809bfe0 2024-09-16T15:03:14.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.676+0000 7f4a9e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:14.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.676+0000 7f4a9e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:14.678 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:14.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 msgr2=0x7f4a8c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 0x7f4a8c03ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a98006cc0 tx=0x7f4a98009240 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 10 monclient: shutdown 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 msgr2=0x7f4aa81cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400c430 tx=0x7f4aa400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 shutdown_connections 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4aa813bf80 0x7f4aa81345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> v2:172.21.15.162:6800/980547263 conn(0x7f4a8c038880 0x7f4a8c03ad40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4aa8136030 0x7f4aa8134060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 --2- 172.21.15.134:0/4152075043 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4aa809bfe0 0x7f4aa81cb490 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 >> 172.21.15.134:0/4152075043 conn(0x7f4aa80a9ea0 msgr2=0x7f4aa80a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 shutdown_connections 2024-09-16T15:03:14.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.678+0000 7f4aafbf4640 1 -- 172.21.15.134:0/4152075043 wait complete. 2024-09-16T15:03:14.689 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:03:14.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:03:14.690 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: finishing write tid 4 to smithi16234619-8 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: finishing write tid 5 to smithi16234619-8 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: oid 8 updating version 0 to 171 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: oid 8 updating version 171 to 174 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: oid 8 version 174 is already newer than 172 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: oid 8 version 174 is already newer than 173 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 174 (ObjNum 228 snap 45 seq_num 228) dirty exists 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:551: left oid 8 (ObjNum 228 snap 45 seq_num 228) 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:548: finishing copy_from to smithi16234619-29 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 95 (ObjNum 105 snap 19 seq_num 105) dirty exists 2024-09-16T15:03:14.752 INFO:tasks.rados.rados.0.smithi162.stdout:553: finishing write tid 1 to smithi16234619-5 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: finishing write tid 2 to smithi16234619-5 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: finishing write tid 3 to smithi16234619-5 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: finishing write tid 4 to smithi16234619-5 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: finishing write tid 5 to smithi16234619-5 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: oid 5 updating version 0 to 224 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: oid 5 version 224 is already newer than 221 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: oid 5 version 224 is already newer than 222 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: oid 5 version 224 is already newer than 223 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 224 (ObjNum 229 snap 45 seq_num 229) dirty exists 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:553: left oid 5 (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:552: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:557: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:558: finishing write tid 1 to smithi16234619-17 2024-09-16T15:03:14.753 INFO:tasks.rados.rados.0.smithi162.stdout:558: finishing write tid 2 to smithi16234619-17 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: finishing write tid 3 to smithi16234619-17 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: finishing write tid 4 to smithi16234619-17 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: finishing write tid 5 to smithi16234619-17 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: oid 17 updating version 0 to 180 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: oid 17 version 180 is already newer than 177 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: oid 17 version 180 is already newer than 179 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: oid 17 version 180 is already newer than 178 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 180 (ObjNum 230 snap 45 seq_num 230) dirty exists 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:558: left oid 17 (ObjNum 230 snap 45 seq_num 230) 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:552: finishing copy_from to smithi16234619-30 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 209 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:559: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:557: finishing copy_from to smithi16234619-27 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 172 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:559: finishing copy_from to smithi16234619-7 2024-09-16T15:03:14.754 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:03:14.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 10 monclient: get_monmap_and_config 2024-09-16T15:03:14.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 10 monclient: build_initial_monmap 2024-09-16T15:03:14.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 10 monclient: monmap: 2024-09-16T15:03:14.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:14.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:14.787273+0000 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:14.787273+0000 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:14.788 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:14.789 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:14.789 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 1 Processor -- start 2024-09-16T15:03:14.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 1 -- start start 2024-09-16T15:03:14.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.786+0000 7f81b499c640 10 monclient: init 2024-09-16T15:03:14.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient: _add_conns ranks=[8,7,4,2,3,6,0,10,9,5,1] 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-e con 0x7f81b0136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f81b0137220 0x7f81b0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-i con 0x7f81b0137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-c con 0x7f81b0135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b009abc0 con 0x7f81b0135dc0 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f81b009ba50 con 0x7f81b0137220 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00a53c0 con 0x7f81b0136e40 2024-09-16T15:03:14.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81b499c640 10 monclient(hunting): authenticate will time out at 931.376617s 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:39686/0 (socket says 172.21.15.134:39686) 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 1 -- 172.21.15.134:0/4164931111 learned_addr learned my addr 172.21.15.134:0/4164931111 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): get_auth_request con 0x7f81b0136e40 auth_method 0 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): get_auth_request con 0x7f81b0135dc0 auth_method 0 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81aed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81add9b640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f81b0137220 0x7f81b0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81add9b640 10 monclient(hunting): get_auth_request con 0x7f81b0137220 auth_method 0 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81add9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.787+0000 7f81add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 10 monclient(hunting): handle_auth_done global_id 4616 payload 274 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 -- 172.21.15.134:0/4164931111 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f81b0137220 msgr2=0x7f81b0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:14.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f81b0137220 0x7f81b0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 -- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 msgr2=0x7f81b0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 10 monclient: _finish_hunting 0 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 monclient: found mon.noname-e 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81ae59c640 1 -- 172.21.15.134:0/4164931111 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0073250 con 0x7f81b0136e40 2024-09-16T15:03:14.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.788+0000 7f81aed9d640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:14.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.790+0000 7f81ae59c640 1 -- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 msgr2=0x7f81b009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.790+0000 7f81ae59c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098a0 tx=0x7f81a002a080 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3e159f1182a6b28a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.790+0000 7f81ad59a640 1 -- 172.21.15.134:0/4164931111 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a002ade0 con 0x7f81b0136e40 2024-09-16T15:03:14.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: dump: 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.802 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: _finish_auth 0 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: _check_auth_tickets 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 1 -- 172.21.15.134:0/4164931111 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a00307b0 con 0x7f81b0136e40 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 5 monclient: authenticate success, global_id 4616 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 1 -- 172.21.15.134:0/4164931111 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a0030d90 con 0x7f81b0136e40 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 4 set_mon_vals no callback set 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:14.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81ad59a640 10 monclient: dump: 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.804 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 10 monclient: shutdown 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 -- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 msgr2=0x7f81b009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098a0 tx=0x7f81a002a080 comp rx=0 tx=0).stop 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 -- 172.21.15.134:0/4164931111 shutdown_connections 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f81b0137220 0x7f81b0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 --2- 172.21.15.134:0/4164931111 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f81b0136e40 0x7f81b009c150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 -- 172.21.15.134:0/4164931111 >> 172.21.15.134:0/4164931111 conn(0x7f81b00a6110 msgr2=0x7f81b00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 -- 172.21.15.134:0/4164931111 shutdown_connections 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.791+0000 7f81b499c640 1 -- 172.21.15.134:0/4164931111 wait complete. 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.792+0000 7f81b499c640 10 monclient: build_initial_monmap 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.792+0000 7f81b499c640 10 monclient: monmap: 2024-09-16T15:03:14.805 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.792+0000 7f81b499c640 1 Processor -- start 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.792+0000 7f81b499c640 1 -- start start 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.792+0000 7f81b499c640 10 monclient: init 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient: _add_conns ranks=[4,3,5,6,1,7,2,8,0] 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b01c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-c con 0x7f81b0135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81b0136e40 0x7f81b0135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-g con 0x7f81b0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f81b0137220 0x7f81b01cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): picked mon.noname-h con 0x7f81b0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01363b0 con 0x7f81b0136e40 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b01233c0 con 0x7f81b0135dc0 2024-09-16T15:03:14.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0124660 con 0x7f81b0137220 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81b499c640 10 monclient(hunting): authenticate will time out at 931.382342s 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b01c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b01c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50498/0 (socket says 172.21.15.134:50498) 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 -- 172.21.15.134:0/3823501730 learned_addr learned my addr 172.21.15.134:0/3823501730 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): get_auth_request con 0x7f81b0135dc0 auth_method 0 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81aed9d640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f81b0137220 0x7f81b01cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81b0136e40 0x7f81b0135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81aed9d640 10 monclient(hunting): get_auth_request con 0x7f81b0137220 auth_method 0 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81aed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81aed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81aed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f81b0137220 msgr2=0x7f81b01cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f81b0137220 0x7f81b01cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81b0136e40 msgr2=0x7f81b0135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81b0136e40 0x7f81b0135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient: _finish_hunting 0 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 monclient: found mon.noname-c 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81ae59c640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009a70 con 0x7f81b0135dc0 2024-09-16T15:03:14.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81add9b640 10 monclient: get_auth_request con 0x7f81b0136e40 auth_method 0 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.793+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81b0136e40 0x7f81b0135470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.795+0000 7f81ae59c640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 msgr2=0x7f81b01c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.795+0000 7f81ae59c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b01c8180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f81a00090b0 tx=0x7f81a00046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a002ade0 con 0x7f81b0135dc0 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: dump: 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.810 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: _finish_auth 0 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0003610 con 0x7f81b0135dc0 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 5 monclient: authenticate success, global_id 4450 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 monclient: _renew_subs 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a00094d0 con 0x7f81b0135dc0 2024-09-16T15:03:14.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a0003bf0 con 0x7f81b0135dc0 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 10 monclient: dump: 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.812 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81acd99640 4 set_mon_vals no callback set 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 monclient: _renew_subs 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b0125e60 con 0x7f81b0135dc0 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 0x7f81b002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0067a80 con 0x7f81b01359b0 2024-09-16T15:03:14.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.796+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f81a0039040 con 0x7f81b0135dc0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 0x7f81b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient: get_auth_request con 0x7f81b01359b0 auth_method 0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f819effd640 1 --2- 172.21.15.134:0/3823501730 >> v2:172.21.15.162:6800/980547263 conn(0x7f8190038880 0x7f819003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f81a006a630 con 0x7f81b0135dc0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 10 monclient(hunting): handle_auth_done global_id 4554 payload 824 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81add9b640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81b0061ae0 con 0x7f81b01359b0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81aed9d640 1 --2- 172.21.15.134:0/3823501730 >> v2:172.21.15.162:6800/980547263 conn(0x7f8190038880 0x7f819003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.797+0000 7f81aed9d640 10 monclient: get_auth_request con 0x7f8190038880 auth_method 0 2024-09-16T15:03:14.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f81add9b640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 msgr2=0x7f81b002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 0x7f81b002b260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a400ec00 con 0x7f81b01359b0 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f819effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f819effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f819effd640 10 monclient: dump: 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.815 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.799+0000 7f81aed9d640 1 --2- 172.21.15.134:0/3823501730 >> v2:172.21.15.162:6800/980547263 conn(0x7f8190038880 0x7f819003ad40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81b006b550 tx=0x7f81a800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f81a400ed60 con 0x7f81b01359b0 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 msgr2=0x7f81b002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 0x7f81b002b260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a4010c20 comp rx=0 tx=0).stop 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 msgr2=0x7f81b002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:14.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.800+0000 7f819effd640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b01359b0 0x7f81b002b260 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a4010c20 comp rx=0 tx=0).stop 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81b499c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81b499c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 0x7f81b002bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81b499c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0071b20 con 0x7f81b002b7a0 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 0x7f81b002bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient: get_auth_request con 0x7f81b002b7a0 auth_method 0 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 10 monclient(hunting): handle_auth_done global_id 4563 payload 824 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.806+0000 7f81add9b640 1 -- 172.21.15.134:0/3823501730 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f81b007a680 con 0x7f81b002b7a0 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f81add9b640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 msgr2=0x7f81b002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f81add9b640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 0x7f81b002bbb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81a4024070 con 0x7f81b002b7a0 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f819effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f819effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.808+0000 7f819effd640 10 monclient: dump: 2024-09-16T15:03:14.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f81a4004530 con 0x7f81b002b7a0 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 msgr2=0x7f81b002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 0x7f81b002bbb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 msgr2=0x7f81b002bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f819effd640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b002b7a0 0x7f81b002bbb0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81b0121eb0 tx=0x7f81a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:03:14.821 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:14.822 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:14.824 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:14.825 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:14.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:14.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.832 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.834 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.836 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.837 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.841 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:14.842 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:14.843 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:14.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 >> v2:172.21.15.162:6800/980547263 conn(0x7f8190038880 msgr2=0x7f819003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 --2- 172.21.15.134:0/3823501730 >> v2:172.21.15.162:6800/980547263 conn(0x7f8190038880 0x7f819003ad40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81b006b550 tx=0x7f81a800b040 comp rx=0 tx=0).stop 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 10 monclient: shutdown 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 msgr2=0x7f81b01c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 --2- 172.21.15.134:0/3823501730 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f81b0135dc0 0x7f81b01c8180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f81a00090b0 tx=0x7f81a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81aed9d640 1 -- 172.21.15.134:0/3823501730 reap_dead start 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81aed9d640 1 -- 172.21.15.134:0/3823501730 reap_dead start 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 shutdown_connections 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 >> 172.21.15.134:0/3823501730 conn(0x7f81b00a6110 msgr2=0x7f81b009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 shutdown_connections 2024-09-16T15:03:14.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.809+0000 7f81b499c640 1 -- 172.21.15.134:0/3823501730 wait complete. 2024-09-16T15:03:14.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:14.846 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:03:14.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.942+0000 7f7e6cfa0640 10 monclient: get_monmap_and_config 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.942+0000 7f7e6cfa0640 10 monclient: build_initial_monmap 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.942+0000 7f7e6cfa0640 10 monclient: monmap: 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:14.943613+0000 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:14.943613+0000 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.944 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:14.945 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 1 Processor -- start 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 1 -- start start 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 10 monclient: init 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 10 monclient: _add_conns ranks=[7,4,8,9,2,1,6,0,3,5,10] 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7e68135de0 0x7f7e6809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.943+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-i con 0x7f7e68135de0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-c con 0x7f7e681361c0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 0x7f7e680a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-e con 0x7f7e680a3ce0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6809b620 con 0x7f7e681361c0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680a5540 con 0x7f7e68135de0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680a4e80 con 0x7f7e680a3ce0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e6cfa0640 10 monclient(hunting): authenticate will time out at 931.533072s 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 0x7f7e680a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50500/0 (socket says 172.21.15.134:50500) 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 -- 172.21.15.134:0/2107020266 learned_addr learned my addr 172.21.15.134:0/2107020266 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request con 0x7f7e681361c0 auth_method 0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): get_auth_request con 0x7f7e680a3ce0 auth_method 0 2024-09-16T15:03:14.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7e68135de0 0x7f7e6809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): get_auth_request con 0x7f7e68135de0 auth_method 0 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_done global_id 4459 payload 274 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 -- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 msgr2=0x7f7e680a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 0x7f7e680a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 -- 172.21.15.134:0/2107020266 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7e68135de0 msgr2=0x7f7e6809bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7e68135de0 0x7f7e6809bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 monclient: found mon.noname-c 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e66ffd640 1 -- 172.21.15.134:0/2107020266 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68073250 con 0x7f7e681361c0 2024-09-16T15:03:14.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.944+0000 7f7e67fff640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 0x7f7e680a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e66ffd640 1 -- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 msgr2=0x7f7e681365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0098a0 tx=0x7f7e5c02a080 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d6dfe3c92e4b61a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 1 -- 172.21.15.134:0/2107020266 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e5c02ade0 con 0x7f7e681361c0 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: dump: 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.958 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 5 monclient: authenticate success, global_id 4459 2024-09-16T15:03:14.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 1 -- 172.21.15.134:0/2107020266 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0307b0 con 0x7f7e681361c0 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 4 set_mon_vals no callback set 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 1 -- 172.21.15.134:0/2107020266 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e5c030d90 con 0x7f7e681361c0 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e667fc640 10 monclient: dump: 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.960 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 10 monclient: shutdown 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 msgr2=0x7f7e681365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0098a0 tx=0x7f7e5c02a080 comp rx=0 tx=0).stop 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/2107020266 shutdown_connections 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7e680a3ce0 0x7f7e680a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e681361c0 0x7f7e681365c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/2107020266 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7e68135de0 0x7f7e6809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.947+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/2107020266 >> 172.21.15.134:0/2107020266 conn(0x7f7e68066070 msgr2=0x7f7e68066480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.948+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/2107020266 shutdown_connections 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.948+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/2107020266 wait complete. 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.948+0000 7f7e6cfa0640 10 monclient: build_initial_monmap 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.948+0000 7f7e6cfa0640 10 monclient: monmap: 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:14.962 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.948+0000 7f7e6cfa0640 1 Processor -- start 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 -- start start 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient: init 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:14.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient: _add_conns ranks=[4,0,3,2,8,5,7,1,6] 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-c con 0x7f7e680a3ce0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e68135de0 0x7f7e68134c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-a con 0x7f7e68135de0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e681361c0 0x7f7e68135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): picked mon.noname-g con 0x7f7e681361c0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681210f0 con 0x7f7e68135de0 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680a49b0 con 0x7f7e681361c0 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68121d40 con 0x7f7e680a3ce0 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e6cfa0640 10 monclient(hunting): authenticate will time out at 931.538584s 2024-09-16T15:03:14.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50514/0 (socket says 172.21.15.134:50514) 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e68135de0 0x7f7e68134c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 -- 172.21.15.134:0/3544928726 learned_addr learned my addr 172.21.15.134:0/3544928726 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): get_auth_request con 0x7f7e680a3ce0 auth_method 0 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request con 0x7f7e68135de0 auth_method 0 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient(hunting): handle_auth_done global_id 4468 payload 824 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e681361c0 msgr2=0x7f7e68135160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e681361c0 0x7f7e68135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e68135de0 msgr2=0x7f7e68134c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e68135de0 0x7f7e68134c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 monclient: found mon.noname-c 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e677fe640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60009400 con 0x7f7e680a3ce0 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.949+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e68135de0 0x7f7e68134c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e677fe640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 msgr2=0x7f7e6809f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e677fe640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800efc0 tx=0x7f7e5800c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e580148f0 con 0x7f7e680a3ce0 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: dump: 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.967 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: _finish_auth 0 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800f040 con 0x7f7e680a3ce0 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 5 monclient: authenticate success, global_id 4468 2024-09-16T15:03:14.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 10 monclient: _renew_subs 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c0094d0 con 0x7f7e680a3ce0 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e580133e0 con 0x7f7e680a3ce0 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e4ffff640 10 monclient: dump: 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.969 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e65ffb640 4 set_mon_vals no callback set 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 10 monclient: _renew_subs 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.952+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e681225e0 con 0x7f7e680a3ce0 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e6cfa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 0x7f7e6811fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68069fe0 con 0x7f7e6811f930 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7e58014480 con 0x7f7e680a3ce0 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 0x7f7e6811fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient: get_auth_request con 0x7f7e6811f930 auth_method 0 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e4ffff640 1 --2- 172.21.15.134:0/3544928726 >> v2:172.21.15.162:6800/980547263 conn(0x7f7e54038880 0x7f7e5403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f7e5804a970 con 0x7f7e680a3ce0 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_done global_id 34112 payload 824 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e66ffd640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e68073810 con 0x7f7e6811f930 2024-09-16T15:03:14.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e67fff640 1 --2- 172.21.15.134:0/3544928726 >> v2:172.21.15.162:6800/980547263 conn(0x7f7e54038880 0x7f7e5403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.953+0000 7f7e67fff640 10 monclient: get_auth_request con 0x7f7e54038880 auth_method 0 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.955+0000 7f7e66ffd640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 msgr2=0x7f7e6811fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.955+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 0x7f7e6811fd10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009870 tx=0x7f7e5c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e5c003c90 con 0x7f7e6811f930 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 10 monclient: dump: 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e67fff640 1 --2- 172.21.15.134:0/3544928726 >> v2:172.21.15.162:6800/980547263 conn(0x7f7e54038880 0x7f7e5403ad40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006e80 tx=0x7f7e60006dd0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7e5c003df0 con 0x7f7e6811f930 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:14.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 msgr2=0x7f7e6811fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 0x7f7e6811fd10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009870 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 msgr2=0x7f7e6811fd10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:14.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.956+0000 7f7e4ffff640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e6811f930 0x7f7e6811fd10 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009870 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e6cfa0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 0x7f7e680a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e6cfa0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68061ae0 con 0x7f7e68120250 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 0x7f7e680a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient: get_auth_request con 0x7f7e68120250 auth_method 0 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:14.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 10 monclient(hunting): handle_auth_done global_id 34121 payload 824 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.962+0000 7f7e66ffd640 1 -- 172.21.15.134:0/3544928726 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e6807a680 con 0x7f7e68120250 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e66ffd640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 msgr2=0x7f7e680a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e66ffd640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 0x7f7e680a0bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a0c0 tx=0x7f7e5c008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7e5c048be0 con 0x7f7e68120250 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 10 monclient: dump: 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:14.977 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f7e5c04c070 con 0x7f7e68120250 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 msgr2=0x7f7e680a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:14.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 0x7f7e680a0bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a0c0 tx=0x7f7e5c008030 comp rx=0 tx=0).stop 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 msgr2=0x7f7e680a0bf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e4ffff640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e68120250 0x7f7e680a0bf0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a0c0 tx=0x7f7e5c008030 comp rx=0 tx=0).stop 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:14.979 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:14.980 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.981 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:14.982 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:14.983 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:14.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.985 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:14.987 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.990 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.991 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.992 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:14.994 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:14.996 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:14.998 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:14.999 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.001 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.001 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 >> v2:172.21.15.162:6800/980547263 conn(0x7f7e54038880 msgr2=0x7f7e5403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/3544928726 >> v2:172.21.15.162:6800/980547263 conn(0x7f7e54038880 0x7f7e5403ad40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006e80 tx=0x7f7e60006dd0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 10 monclient: shutdown 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 msgr2=0x7f7e6809f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800efc0 tx=0x7f7e5800c490 comp rx=0 tx=0).stop 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e67fff640 1 -- 172.21.15.134:0/3544928726 reap_dead start 2024-09-16T15:03:15.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 shutdown_connections 2024-09-16T15:03:15.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.965+0000 7f7e6cfa0640 1 --2- 172.21.15.134:0/3544928726 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7e680a3ce0 0x7f7e6809f1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.966+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 >> 172.21.15.134:0/3544928726 conn(0x7f7e68066070 msgr2=0x7f7e681365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.966+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 shutdown_connections 2024-09-16T15:03:15.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:14.966+0000 7f7e6cfa0640 1 -- 172.21.15.134:0/3544928726 wait complete. 2024-09-16T15:03:15.003 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:03:15.003 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.099+0000 7f14f1c3a640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.099+0000 7f14f1c3a640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.099+0000 7f14f1c3a640 10 monclient: monmap: 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.101 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.100712+0000 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.100712+0000 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.102 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.100+0000 7f14f1c3a640 1 Processor -- start 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.100+0000 7f14f1c3a640 1 -- start start 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.100+0000 7f14f1c3a640 10 monclient: init 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient: _add_conns ranks=[6,8,9,10,1,4,2,3,0,5,7] 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-d con 0x7f14ec136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-e con 0x7f14ec137290 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f14ec133d10 0x7f14ec1341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-a con 0x7f14ec133d10 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0a5560 con 0x7f14ec133d10 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec09b240 con 0x7f14ec136eb0 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec09b520 con 0x7f14ec137290 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14f1c3a640 10 monclient(hunting): authenticate will time out at 931.690159s 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f14ec133d10 0x7f14ec1341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:39706/0 (socket says 172.21.15.134:39706) 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f14ec133d10 0x7f14ec1341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43580/0 (socket says 172.21.15.134:43580) 2024-09-16T15:03:15.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:41316/0 (socket says 172.21.15.134:41316) 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 -- 172.21.15.134:0/3835579535 learned_addr learned my addr 172.21.15.134:0/3835579535 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 10 monclient(hunting): get_auth_request con 0x7f14ec137290 auth_method 0 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): get_auth_request con 0x7f14ec133d10 auth_method 0 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request con 0x7f14ec136eb0 auth_method 0 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 msgr2=0x7f14ec137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:15.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 msgr2=0x7f14ec137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eaffd640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient(hunting): handle_auth_done global_id 4581 payload 274 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 msgr2=0x7f14ec137690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 msgr2=0x7f14ec09bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 monclient: found mon.noname-a 2024-09-16T15:03:15.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:15.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14ebfff640 1 -- 172.21.15.134:0/3835579535 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec073250 con 0x7f14ec133d10 2024-09-16T15:03:15.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.101+0000 7f14eb7fe640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ebfff640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec133d10 msgr2=0x7f14ec1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ebfff640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec133d10 0x7f14ec1341c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14e40097a0 tx=0x7f14e4029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1430c6984bb9c7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 1 -- 172.21.15.134:0/3835579535 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14e402ad30 con 0x7f14ec133d10 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:15.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: dump: 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.116 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 1 -- 172.21.15.134:0/3835579535 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e40307b0 con 0x7f14ec133d10 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14f1c3a640 5 monclient: authenticate success, global_id 4581 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14f1c3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14f1c3a640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 1 -- 172.21.15.134:0/3835579535 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14e4030d90 con 0x7f14ec133d10 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14f1c3a640 4 set_mon_vals no callback set 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:15.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.104+0000 7f14ea7fc640 10 monclient: dump: 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.118 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 10 monclient: shutdown 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec133d10 msgr2=0x7f14ec1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec133d10 0x7f14ec1341c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14e40097a0 tx=0x7f14e4029fd0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3835579535 shutdown_connections 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f14ec137290 0x7f14ec137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec133d10 0x7f14ec1341c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3835579535 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f14ec136eb0 0x7f14ec09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3835579535 >> 172.21.15.134:0/3835579535 conn(0x7f14ec0a99a0 msgr2=0x7f14ec0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3835579535 shutdown_connections 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.105+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3835579535 wait complete. 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient: monmap: 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.120 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 Processor -- start 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 -- start start 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient: init 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient: _add_conns ranks=[7,3,1,2,5,6,4,8,0] 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f14ec133d10 0x7f14ec132d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-i con 0x7f14ec133d10 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 0x7f14ec1332a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-g con 0x7f14ec136eb0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec137290 0x7f14ec11fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): picked mon.noname-f con 0x7f14ec137290 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:15.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1bd650 con 0x7f14ec137290 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec122030 con 0x7f14ec136eb0 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec136310 con 0x7f14ec133d10 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.106+0000 7f14f1c3a640 10 monclient(hunting): authenticate will time out at 931.695969s 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 0x7f14ec1332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec137290 0x7f14ec11fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f14ec133d10 0x7f14ec132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 0x7f14ec1332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:52380/0 (socket says 172.21.15.134:52380) 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 -- 172.21.15.134:0/3848929593 learned_addr learned my addr 172.21.15.134:0/3848929593 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): get_auth_request con 0x7f14ec136eb0 auth_method 0 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request con 0x7f14ec133d10 auth_method 0 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): get_auth_request con 0x7f14ec137290 auth_method 0 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient(hunting): handle_auth_done global_id 34113 payload 824 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f14ec133d10 msgr2=0x7f14ec132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f14ec133d10 0x7f14ec132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec137290 msgr2=0x7f14ec11fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14ec137290 0x7f14ec11fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 monclient: found mon.noname-g 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:03:15.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.107+0000 7f14eaffd640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009a70 con 0x7f14ec136eb0 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.109+0000 7f14eaffd640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 msgr2=0x7f14ec1332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.109+0000 7f14eaffd640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 0x7f14ec1332a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f14e000e810 tx=0x7f14e000ecd0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14e000ee00 con 0x7f14ec136eb0 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: dump: 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.125 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 5 monclient: authenticate success, global_id 34113 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e000cd50 con 0x7f14ec136eb0 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 monclient: _renew_subs 2024-09-16T15:03:15.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e4009400 con 0x7f14ec136eb0 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14e0013880 con 0x7f14ec136eb0 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 10 monclient: dump: 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.127 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14e9ffb640 4 set_mon_vals no callback set 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 monclient: _renew_subs 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec121900 con 0x7f14ec136eb0 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f14e0013410 con 0x7f14ec136eb0 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 0x7f14ec1204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.110+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec071b20 con 0x7f14ec120110 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 0x7f14ec1204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient: get_auth_request con 0x7f14ec120110 auth_method 0 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14d3fff640 1 --2- 172.21.15.134:0/3848929593 >> v2:172.21.15.162:6800/980547263 conn(0x7f14d8038880 0x7f14d803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f14e0049710 con 0x7f14ec136eb0 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_done global_id 4477 payload 824 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14eb7fe640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14ec061ae0 con 0x7f14ec120110 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14ebfff640 1 --2- 172.21.15.134:0/3848929593 >> v2:172.21.15.162:6800/980547263 conn(0x7f14d8038880 0x7f14d803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.111+0000 7f14ebfff640 10 monclient: get_auth_request con 0x7f14d8038880 auth_method 0 2024-09-16T15:03:15.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14eb7fe640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 msgr2=0x7f14ec1204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14eb7fe640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 0x7f14ec1204f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f410 tx=0x7f14dc00f8d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14dc00e040 con 0x7f14ec120110 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 10 monclient: dump: 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.130 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f14dc00f040 con 0x7f14ec120110 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 msgr2=0x7f14ec1204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 0x7f14ec1204f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f410 tx=0x7f14dc00f8d0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 msgr2=0x7f14ec1204f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14d3fff640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120110 0x7f14ec1204f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.113+0000 7f14ebfff640 1 --2- 172.21.15.134:0/3848929593 >> v2:172.21.15.162:6800/980547263 conn(0x7f14d8038880 0x7f14d803ad40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0a5f80 tx=0x7f14e402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.119+0000 7f14f1c3a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 0x7f14ec1340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14f1c3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec07a680 con 0x7f14ec120a30 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 0x7f14ec1340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient: get_auth_request con 0x7f14ec120a30 auth_method 0 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 10 monclient(hunting): handle_auth_done global_id 4486 payload 824 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.120+0000 7f14eb7fe640 1 -- 172.21.15.134:0/3848929593 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f14ec0a6860 con 0x7f14ec120a30 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14eb7fe640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 msgr2=0x7f14ec1340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14eb7fe640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 0x7f14ec1340f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f410 tx=0x7f14dc004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14dc02c070 con 0x7f14ec120a30 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 10 monclient: dump: 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.134 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f14dc024040 con 0x7f14ec120a30 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 msgr2=0x7f14ec1340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 0x7f14ec1340f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f410 tx=0x7f14dc004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 msgr2=0x7f14ec1340f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.122+0000 7f14d3fff640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14ec120a30 0x7f14ec1340f0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00f410 tx=0x7f14dc004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.136 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.137 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.139 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.140 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.142 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.144 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.147 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.149 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.153 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.155 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.156 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.157 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 >> v2:172.21.15.162:6800/980547263 conn(0x7f14d8038880 msgr2=0x7f14d803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3848929593 >> v2:172.21.15.162:6800/980547263 conn(0x7f14d8038880 0x7f14d803ad40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0a5f80 tx=0x7f14e402e000 comp rx=0 tx=0).stop 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 10 monclient: shutdown 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 msgr2=0x7f14ec1332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 --2- 172.21.15.134:0/3848929593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14ec136eb0 0x7f14ec1332a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f14e000e810 tx=0x7f14e000ecd0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14ebfff640 1 -- 172.21.15.134:0/3848929593 reap_dead start 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14ebfff640 1 -- 172.21.15.134:0/3848929593 reap_dead start 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 shutdown_connections 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 >> 172.21.15.134:0/3848929593 conn(0x7f14ec0a99a0 msgr2=0x7f14ec06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 shutdown_connections 2024-09-16T15:03:15.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.123+0000 7f14f1c3a640 1 -- 172.21.15.134:0/3848929593 wait complete. 2024-09-16T15:03:15.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:03:15.160 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:03:15.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.256+0000 7ff623ce9640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.256+0000 7ff623ce9640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.256+0000 7ff623ce9640 10 monclient: monmap: 2024-09-16T15:03:15.257 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.257172+0000 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.257172+0000 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.258 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.259 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.259 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.256+0000 7ff623ce9640 1 Processor -- start 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 -- start start 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient: init 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient: _add_conns ranks=[5,9,2,4,10,3,8,1,7,0,6] 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff61c1309e0 0x7ff61c130dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-h con 0x7ff61c1309e0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff61c131300 0x7ff61c133b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-a con 0x7ff61c131300 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-b con 0x7ff61c1340a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c09e050 con 0x7ff61c131300 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0a0930 con 0x7ff61c1340a0 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c09e8f0 con 0x7ff61c1309e0 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff623ce9640 10 monclient(hunting): authenticate will time out at 931.846581s 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff61c131300 0x7ff61c133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff61c131300 0x7ff61c133b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43594/0 (socket says 172.21.15.134:43594) 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 -- 172.21.15.134:0/111211811 learned_addr learned my addr 172.21.15.134:0/111211811 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): get_auth_request con 0x7ff61c131300 auth_method 0 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45202/0 (socket says 172.21.15.134:45202) 2024-09-16T15:03:15.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): get_auth_request con 0x7ff61c1340a0 auth_method 0 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621a85640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff61c1309e0 0x7ff61c130dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621a85640 10 monclient(hunting): get_auth_request con 0x7ff61c1309e0 auth_method 0 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621a85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621a85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621a85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient(hunting): handle_auth_done global_id 4590 payload 274 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 -- 172.21.15.134:0/111211811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff61c1309e0 msgr2=0x7ff61c130dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff61c1309e0 0x7ff61c130dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 -- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 msgr2=0x7ff61c134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 monclient: found mon.noname-a 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.257+0000 7ff621284640 1 -- 172.21.15.134:0/111211811 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c073250 con 0x7ff61c131300 2024-09-16T15:03:15.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.258+0000 7ff622286640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff621284640 1 -- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff61c131300 msgr2=0x7ff61c133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff621284640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff61c131300 0x7ff61c133b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff61000d150 tx=0x7ff61002c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983d90e182fa657 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 1 -- 172.21.15.134:0/111211811 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff610009a70 con 0x7ff61c131300 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: dump: 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.272 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff623ce9640 5 monclient: authenticate success, global_id 4590 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 1 -- 172.21.15.134:0/111211811 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61002f050 con 0x7ff61c131300 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff623ce9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff623ce9640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff623ce9640 4 set_mon_vals no callback set 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 1 -- 172.21.15.134:0/111211811 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff610009bf0 con 0x7ff61c131300 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.260+0000 7ff620a83640 10 monclient: dump: 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.274 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 10 monclient: shutdown 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 -- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff61c131300 msgr2=0x7ff61c133b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff61c131300 0x7ff61c133b60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff61000d150 tx=0x7ff61002c760 comp rx=0 tx=0).stop 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 -- 172.21.15.134:0/111211811 shutdown_connections 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff61c1340a0 0x7ff61c134520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff61c131300 0x7ff61c133b60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 --2- 172.21.15.134:0/111211811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff61c1309e0 0x7ff61c130dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 -- 172.21.15.134:0/111211811 >> 172.21.15.134:0/111211811 conn(0x7ff61c0a6470 msgr2=0x7ff61c0a6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 -- 172.21.15.134:0/111211811 shutdown_connections 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.261+0000 7ff623ce9640 1 -- 172.21.15.134:0/111211811 wait complete. 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient: monmap: 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.276 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 Processor -- start 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 -- start start 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient: init 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient: _add_conns ranks=[4,1,8,3,0,7,5,6,2] 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 0x7ff61c12fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-c con 0x7ff61c1309e0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:15.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c131300 0x7ff61c130150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-f con 0x7ff61c131300 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff61c1340a0 0x7ff61c1c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient(hunting): picked mon.noname-e con 0x7ff61c1340a0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c070cb0 con 0x7ff61c131300 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.262+0000 7ff623ce9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c133aa0 con 0x7ff61c1309e0 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff623ce9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c12be20 con 0x7ff61c1340a0 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff623ce9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff623ce9640 10 monclient(hunting): authenticate will time out at 931.852019s 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 0x7ff61c12fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 0x7ff61c12fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50524/0 (socket says 172.21.15.134:50524) 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff61c1340a0 0x7ff61c1c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 -- 172.21.15.134:0/1470479270 learned_addr learned my addr 172.21.15.134:0/1470479270 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): get_auth_request con 0x7ff61c1309e0 auth_method 0 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): get_auth_request con 0x7ff61c1340a0 auth_method 0 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c131300 0x7ff61c130150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): get_auth_request con 0x7ff61c131300 auth_method 0 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient(hunting): handle_auth_done global_id 4495 payload 824 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff61c1340a0 msgr2=0x7ff61c1c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff61c1340a0 0x7ff61c1c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c131300 msgr2=0x7ff61c130150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff61c131300 0x7ff61c130150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 monclient: found mon.noname-c 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff621a85640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c009400 con 0x7ff61c1309e0 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.263+0000 7ff622286640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff61c1340a0 0x7ff61c1c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff621a85640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 msgr2=0x7ff61c12fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff621a85640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 0x7ff61c12fc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d800 tx=0x7ff61400dcc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff61400bd40 con 0x7ff61c1309e0 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: dump: 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614014430 con 0x7ff61c1309e0 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 5 monclient: authenticate success, global_id 4495 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff614014a10 con 0x7ff61c1309e0 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 10 monclient: _renew_subs 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61000bd40 con 0x7ff61c1309e0 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61a7fc640 10 monclient: dump: 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.283 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff61bfff640 4 set_mon_vals no callback set 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 10 monclient: _renew_subs 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.265+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c12c3e0 con 0x7ff61c1309e0 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff623ce9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff623ce9640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 0x7ff61c1c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c06b5d0 con 0x7ff61c1c76b0 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff61400b8d0 con 0x7ff61c1309e0 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff621284640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 0x7ff61c1c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff621284640 10 monclient: get_auth_request con 0x7ff61c1c76b0 auth_method 0 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff621284640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff621284640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.266+0000 7ff621284640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff61a7fc640 1 --2- 172.21.15.134:0/1470479270 >> v2:172.21.15.162:6800/980547263 conn(0x7ff608038880 0x7ff60803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 3943+0+0 (secure 0 0 0) 0x7ff61404a200 con 0x7ff61c1309e0 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff621284640 10 monclient(hunting): handle_auth_done global_id 34125 payload 824 2024-09-16T15:03:15.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff621284640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff61c061ae0 con 0x7ff61c1c76b0 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff622286640 1 --2- 172.21.15.134:0/1470479270 >> v2:172.21.15.162:6800/980547263 conn(0x7ff608038880 0x7ff60803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.267+0000 7ff622286640 10 monclient: get_auth_request con 0x7ff608038880 auth_method 0 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff621284640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 msgr2=0x7ff61c1c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff621284640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 0x7ff61c1c41c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff610035000 tx=0x7ff610004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff61002cee0 con 0x7ff61c1c76b0 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 10 monclient: dump: 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.286 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff622286640 1 --2- 172.21.15.134:0/1470479270 >> v2:172.21.15.162:6800/980547263 conn(0x7ff608038880 0x7ff60803ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0a38a0 tx=0x7ff60c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff610004760 con 0x7ff61c1c76b0 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 msgr2=0x7ff61c1c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 0x7ff61c1c41c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff610035000 tx=0x7ff610004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:15.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 msgr2=0x7ff61c1c41c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.269+0000 7ff61a7fc640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c1c76b0 0x7ff61c1c41c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.275+0000 7ff623ce9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.275+0000 7ff623ce9640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 0x7ff61c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff623ce9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c07a680 con 0x7ff61c02b260 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 0x7ff61c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient: get_auth_request con 0x7ff61c02b260 auth_method 0 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 10 monclient(hunting): handle_auth_done global_id 34134 payload 824 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.276+0000 7ff621284640 1 -- 172.21.15.134:0/1470479270 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff61c09f0f0 con 0x7ff61c02b260 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff621284640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 msgr2=0x7ff61c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff621284640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 0x7ff61c02b670 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff610043000 tx=0x7ff610031000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff610009d30 con 0x7ff61c02b260 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff61a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff61a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff61a7fc640 10 monclient: dump: 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.291 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.278+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7ff610009e90 con 0x7ff61c02b260 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 msgr2=0x7ff61c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 0x7ff61c02b670 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff610043000 tx=0x7ff610031000 comp rx=0 tx=0).stop 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 msgr2=0x7ff61c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff61a7fc640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff61c02b260 0x7ff61c02b670 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff610043000 tx=0x7ff610031000 comp rx=0 tx=0).stop 2024-09-16T15:03:15.292 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.293 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.294 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.295 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.296 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.298 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.300 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.303 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.305 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.306 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.308 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.309 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.311 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.312 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 >> v2:172.21.15.162:6800/980547263 conn(0x7ff608038880 msgr2=0x7ff60803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 --2- 172.21.15.134:0/1470479270 >> v2:172.21.15.162:6800/980547263 conn(0x7ff608038880 0x7ff60803ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0a38a0 tx=0x7ff60c009190 comp rx=0 tx=0).stop 2024-09-16T15:03:15.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 10 monclient: shutdown 2024-09-16T15:03:15.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 msgr2=0x7ff61c12fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 --2- 172.21.15.134:0/1470479270 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff61c1309e0 0x7ff61c12fc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff61400d800 tx=0x7ff61400dcc0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff622286640 1 -- 172.21.15.134:0/1470479270 reap_dead start 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff622286640 1 -- 172.21.15.134:0/1470479270 reap_dead start 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 shutdown_connections 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 >> 172.21.15.134:0/1470479270 conn(0x7ff61c0a6470 msgr2=0x7ff61c0996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 shutdown_connections 2024-09-16T15:03:15.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.279+0000 7ff623ce9640 1 -- 172.21.15.134:0/1470479270 wait complete. 2024-09-16T15:03:15.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:03:15.316 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:03:15.316 INFO:tasks.rados.rados.0.smithi162.stdout:548: done (8 left) 2024-09-16T15:03:15.316 INFO:tasks.rados.rados.0.smithi162.stdout:551: done (7 left) 2024-09-16T15:03:15.316 INFO:tasks.rados.rados.0.smithi162.stdout:552: done (6 left) 2024-09-16T15:03:15.316 INFO:tasks.rados.rados.0.smithi162.stdout:553: done (5 left) 2024-09-16T15:03:15.316 INFO:tasks.rados.rados.0.smithi162.stdout:554: done (4 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:557: done (3 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:558: done (2 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:559: done (1 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:560: done (0 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:561: read oid 18 snap -1 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:561: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:562: delete oid 29 current snap is 46 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:562: done (1 left) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:563: rollback oid 30 current snap is 46 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 45 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:564: rollback oid 24 current snap is 46 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 45 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:565: read oid 48 snap 45 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:565: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:15.317 INFO:tasks.rados.rados.0.smithi162.stdout:566: read oid 10 snap -1 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:566: expect deleted 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:567: copy_from oid 9 from oid 27 current snap is 46 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:568: write oid 7 current snap is 46 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:568: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:568: writing smithi16234619-7 from 699224 to 1247582 tid 1 2024-09-16T15:03:15.318 INFO:tasks.rados.rados.0.smithi162.stdout:568: writing smithi16234619-7 from 2026293 to 2488367 tid 2 2024-09-16T15:03:15.319 INFO:tasks.rados.rados.0.smithi162.stdout:568: writing smithi16234619-7 from 3271546 to 3704300 tid 3 2024-09-16T15:03:15.319 INFO:tasks.rados.rados.0.smithi162.stdout:569: delete oid 23 current snap is 46 2024-09-16T15:03:15.319 INFO:tasks.rados.rados.0.smithi162.stdout:561: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.416+0000 7fc214ee9640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.416+0000 7fc214ee9640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.416+0000 7fc214ee9640 10 monclient: monmap: 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.417793+0000 2024-09-16T15:03:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.417793+0000 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.419 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.417+0000 7fc214ee9640 1 Processor -- start 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.417+0000 7fc214ee9640 1 -- start start 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.417+0000 7fc214ee9640 10 monclient: init 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient: _add_conns ranks=[0,10,4,8,6,1,3,7,9,5,2] 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc210136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc210137220 0x7fc210137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-f con 0x7fc210137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-c con 0x7fc21013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc210073250 con 0x7fc210136e40 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2100a56e0 con 0x7fc210137220 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21009abc0 con 0x7fc21013c370 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc214ee9640 10 monclient(hunting): authenticate will time out at 932.007304s 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20e59c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc210136e40 0x7fc21009c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:48140/0 (socket says 172.21.15.134:48140) 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20e59c640 1 -- 172.21.15.134:0/293654952 learned_addr learned my addr 172.21.15.134:0/293654952 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50532/0 (socket says 172.21.15.134:50532) 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): get_auth_request con 0x7fc21013c370 auth_method 0 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 1 --2- 172.21.15.134:0/293654952 >> v2:172.21.15.162:3300/0 conn(0x7fc210137220 0x7fc210137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): get_auth_request con 0x7fc210137220 auth_method 0 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_done global_id 4504 payload 274 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 -- 172.21.15.134:0/293654952 >> v2:172.21.15.162:3300/0 conn(0x7fc210137220 msgr2=0x7fc210137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/293654952 >> v2:172.21.15.162:3300/0 conn(0x7fc210137220 0x7fc210137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 -- 172.21.15.134:0/293654952 >> v1:172.21.15.134:6789/0 conn(0x7fc210136e40 legacy=0x7fc21009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 monclient: found mon.noname-c 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.418+0000 7fc20ed9d640 1 -- 172.21.15.134:0/293654952 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21009ba50 con 0x7fc21013c370 2024-09-16T15:03:15.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20ed9d640 1 -- 172.21.15.134:0/293654952 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 msgr2=0x7fc21013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/293654952 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc208009d90 tx=0x7fc20802a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=69f0b8d5ce2c6320 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 1 -- 172.21.15.134:0/293654952 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc208007bc0 con 0x7fc21013c370 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: dump: 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.431 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 1 -- 172.21.15.134:0/293654952 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20802d050 con 0x7fc21013c370 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc214ee9640 5 monclient: authenticate success, global_id 4504 2024-09-16T15:03:15.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc214ee9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc214ee9640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 1 -- 172.21.15.134:0/293654952 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc208007d40 con 0x7fc21013c370 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc214ee9640 4 set_mon_vals no callback set 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.421+0000 7fc20d59a640 10 monclient: dump: 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.433 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 10 monclient: shutdown 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 -- 172.21.15.134:0/293654952 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 msgr2=0x7fc21013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 --2- 172.21.15.134:0/293654952 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc208009d90 tx=0x7fc20802a4e0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 -- 172.21.15.134:0/293654952 shutdown_connections 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 --2- 172.21.15.134:0/293654952 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc21013c370 0x7fc21013e760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 --2- 172.21.15.134:0/293654952 >> v2:172.21.15.162:3300/0 conn(0x7fc210137220 0x7fc210137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 -- 172.21.15.134:0/293654952 >> 172.21.15.134:0/293654952 conn(0x7fc2100a6110 msgr2=0x7fc2100a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 -- 172.21.15.134:0/293654952 shutdown_connections 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.422+0000 7fc214ee9640 1 -- 172.21.15.134:0/293654952 wait complete. 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient: monmap: 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.435 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 Processor -- start 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 -- start start 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient: init 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient: _add_conns ranks=[3,8,0,5,7,2,6,4,1] 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc210136e40 0x7fc2100a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-g con 0x7fc210136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc210137220 0x7fc2101cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-e con 0x7fc210137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): picked mon.noname-a con 0x7fc21013c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100a78f0 con 0x7fc21013c370 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100a7410 con 0x7fc210136e40 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101216e0 con 0x7fc210137220 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.423+0000 7fc214ee9640 10 monclient(hunting): authenticate will time out at 932.012933s 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43602/0 (socket says 172.21.15.134:43602) 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc210137220 0x7fc2101cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 learned_addr learned my addr 172.21.15.134:0/1858876827 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): get_auth_request con 0x7fc21013c370 auth_method 0 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): get_auth_request con 0x7fc210137220 auth_method 0 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc210136e40 0x7fc2100a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): get_auth_request con 0x7fc210136e40 auth_method 0 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient(hunting): handle_auth_done global_id 4599 payload 824 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc210137220 msgr2=0x7fc2101cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc210137220 0x7fc2101cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc210136e40 msgr2=0x7fc2100a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc210136e40 0x7fc2100a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 monclient: found mon.noname-a 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208009a70 con 0x7fc21013c370 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.424+0000 7fc20dd9b640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc210137220 0x7fc2101cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 msgr2=0x7fc21011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc20802aa00 tx=0x7fc208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc208007ea0 con 0x7fc21013c370 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: dump: 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.440 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 5 monclient: authenticate success, global_id 4599 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080044b0 con 0x7fc21013c370 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc208004620 con 0x7fc21013c370 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 10 monclient: _renew_subs 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc210121180 con 0x7fc21013c370 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc20cd99640 4 set_mon_vals no callback set 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc1feffd640 10 monclient: dump: 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.442 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 10 monclient: _renew_subs 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.426+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc210122550 con 0x7fc21013c370 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20ed9d640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc20802aa00 tx=0x7fc208004a40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fc20806afc0 osd_map(114..114 src has 1..114) v4 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc208004040 con 0x7fc21013c370 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc214ee9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc214ee9640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 0x7fc2101201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc210071b20 con 0x7fc2101cc100 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 0x7fc2101201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient: get_auth_request con 0x7fc2101cc100 auth_method 0 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.427+0000 7fc1feffd640 1 --2- 172.21.15.134:0/1858876827 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1f0038880 0x7fc1f003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.428+0000 7fc20e59c640 10 monclient(hunting): handle_auth_done global_id 4652 payload 824 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.428+0000 7fc20e59c640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc210061ae0 con 0x7fc2101cc100 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.428+0000 7fc20dd9b640 1 --2- 172.21.15.134:0/1858876827 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1f0038880 0x7fc1f003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.428+0000 7fc20dd9b640 10 monclient: get_auth_request con 0x7fc1f0038880 auth_method 0 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc20e59c640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 msgr2=0x7fc2101201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc20e59c640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 0x7fc2101201a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ea70 tx=0x7fc20000ef30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc20000cc50 con 0x7fc2101cc100 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 10 monclient: dump: 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.448 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc20dd9b640 1 --2- 172.21.15.134:0/1858876827 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1f0038880 0x7fc1f003ad40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc2100a2d70 tx=0x7fc204009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc20000cdb0 con 0x7fc2101cc100 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 msgr2=0x7fc2101201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 0x7fc2101201a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ea70 tx=0x7fc20000ef30 comp rx=0 tx=0).stop 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 msgr2=0x7fc2101201a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.430+0000 7fc1feffd640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101cc100 0x7fc2101201a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.436+0000 7fc214ee9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.436+0000 7fc214ee9640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 0x7fc21009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc214ee9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc210137f30 con 0x7fc2101206e0 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 0x7fc21009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient: get_auth_request con 0x7fc2101206e0 auth_method 0 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 10 monclient(hunting): handle_auth_done global_id 4661 payload 824 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.437+0000 7fc20e59c640 1 -- 172.21.15.134:0/1858876827 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc21007a680 con 0x7fc2101206e0 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc20e59c640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 msgr2=0x7fc21009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc20e59c640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 0x7fc21009d4f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ea70 tx=0x7fc200004940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc20001d300 con 0x7fc2101206e0 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc1feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc1feffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.439+0000 7fc1feffd640 10 monclient: dump: 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.451 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fc200021070 con 0x7fc2101206e0 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 msgr2=0x7fc21009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 0x7fc21009d4f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ea70 tx=0x7fc200004940 comp rx=0 tx=0).stop 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 msgr2=0x7fc21009d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc2101206e0 0x7fc21009d4f0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ea70 tx=0x7fc200004940 comp rx=0 tx=0).stop 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.453 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.454 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.455 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.456 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.457 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.459 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.461 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.464 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.465 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.466 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.467 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.468 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.469 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.470 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.473 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.474 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1f0038880 msgr2=0x7fc1f003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 --2- 172.21.15.134:0/1858876827 >> v2:172.21.15.162:6800/980547263 conn(0x7fc1f0038880 0x7fc1f003ad40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc2100a2d70 tx=0x7fc204009190 comp rx=0 tx=0).stop 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 10 monclient: shutdown 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 msgr2=0x7fc21011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 --2- 172.21.15.134:0/1858876827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21013c370 0x7fc21011fc60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc20802aa00 tx=0x7fc208004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 reap_dead start 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc20ed9d640 1 -- 172.21.15.134:0/1858876827 reap_dead start 2024-09-16T15:03:15.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc1feffd640 1 -- 172.21.15.134:0/1858876827 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 3915+0+0 (unknown 0 0 0) 0x7fc20806afc0 con 0x7fc21013c370 2024-09-16T15:03:15.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 shutdown_connections 2024-09-16T15:03:15.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 >> 172.21.15.134:0/1858876827 conn(0x7fc2100a6110 msgr2=0x7fc21013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 shutdown_connections 2024-09-16T15:03:15.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.440+0000 7fc214ee9640 1 -- 172.21.15.134:0/1858876827 wait complete. 2024-09-16T15:03:15.477 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.477 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:03:15.477 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:564: finishing rollback tid 0 to smithi16234619-24 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:564: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:564: finishing rollback tid 2 to smithi16234619-24 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 184 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:563: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:561: done (7 left) 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:564: done (6 left) 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:566: done (5 left) 2024-09-16T15:03:15.569 INFO:tasks.rados.rados.0.smithi162.stdout:569: done (4 left) 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:570: read oid 22 snap -1 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:570: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:571: rollback oid 5 current snap is 46 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 45 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:563: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:563: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 212 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:15.570 INFO:tasks.rados.rados.0.smithi162.stdout:565: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:15.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 10 monclient: monmap: 2024-09-16T15:03:15.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.576010+0000 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.576010+0000 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.577 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.578 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 1 Processor -- start 2024-09-16T15:03:15.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 1 -- start start 2024-09-16T15:03:15.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.575+0000 7f7fe6b76640 10 monclient: init 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient: _add_conns ranks=[6,8,0,9,10,7,1,3,4,2,5] 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 0x7f7fe009b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-d con 0x7f7fe0069e50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 0x7f7fe006a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-e con 0x7f7fe006a230 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7fe01374a0 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe0073250 con 0x7f7fe01374a0 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe006d500 con 0x7f7fe0069e50 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe009bec0 con 0x7f7fe006a230 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe6b76640 10 monclient(hunting): authenticate will time out at 932.165369s 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 0x7f7fe009b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 0x7f7fe006a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe5113640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7fe01374a0 0x7f7fe0137900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:48144/0 (socket says 172.21.15.134:48144) 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe5113640 1 -- 172.21.15.134:0/1699038773 learned_addr learned my addr 172.21.15.134:0/1699038773 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): get_auth_request con 0x7f7fe0069e50 auth_method 0 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): get_auth_request con 0x7f7fe006a230 auth_method 0 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient(hunting): handle_auth_done global_id 4670 payload 274 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 -- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 msgr2=0x7f7fe009b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 0x7f7fe009b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 -- 172.21.15.134:0/1699038773 >> v1:172.21.15.134:6789/0 conn(0x7f7fe01374a0 legacy=0x7f7fe0137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 monclient: found mon.noname-e 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fdffff640 1 -- 172.21.15.134:0/1699038773 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe009b560 con 0x7f7fe006a230 2024-09-16T15:03:15.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.576+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 0x7f7fe009b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdffff640 1 -- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 msgr2=0x7f7fe006a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 0x7f7fe006a610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009dc0 tx=0x7f7fd002a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a8ff5f9307bc9408 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 1 -- 172.21.15.134:0/1699038773 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fd0007bc0 con 0x7f7fe006a230 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: dump: 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.591 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 1 -- 172.21.15.134:0/1699038773 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd002d050 con 0x7f7fe006a230 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fe6b76640 5 monclient: authenticate success, global_id 4670 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fe6b76640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fe6b76640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 1 -- 172.21.15.134:0/1699038773 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fd0007d40 con 0x7f7fe006a230 2024-09-16T15:03:15.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fe6b76640 4 set_mon_vals no callback set 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.579+0000 7f7fdf7fe640 10 monclient: dump: 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.593 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 10 monclient: shutdown 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 msgr2=0x7f7fe006a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 0x7f7fe006a610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009dc0 tx=0x7f7fd002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1699038773 shutdown_connections 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fe006a230 0x7f7fe006a610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1699038773 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fe0069e50 0x7f7fe009b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1699038773 >> 172.21.15.134:0/1699038773 conn(0x7f7fe00a2e20 msgr2=0x7f7fe00a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1699038773 shutdown_connections 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1699038773 wait complete. 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.580+0000 7f7fe6b76640 10 monclient: monmap: 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.595 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 Processor -- start 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 -- start start 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient: init 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient: _add_conns ranks=[7,3,2,0,6,1,4,5,8] 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7fe0069e50 0x7f7fe01cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-i con 0x7f7fe0069e50 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fe006a230 0x7f7fe01cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-g con 0x7f7fe006a230 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 0x7f7fe0091400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): picked mon.noname-b con 0x7f7fe01374a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00a2090 con 0x7f7fe01374a0 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0095980 con 0x7f7fe006a230 2024-09-16T15:03:15.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0094fa0 con 0x7f7fe0069e50 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe6b76640 10 monclient(hunting): authenticate will time out at 932.170806s 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 0x7f7fe0091400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 0x7f7fe0091400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45208/0 (socket says 172.21.15.134:45208) 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 learned_addr learned my addr 172.21.15.134:0/1967190401 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 10 monclient(hunting): get_auth_request con 0x7f7fe01374a0 auth_method 0 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.581+0000 7f7fe5113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7fe0069e50 0x7f7fe01cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fe006a230 0x7f7fe01cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): get_auth_request con 0x7f7fe0069e50 auth_method 0 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 10 monclient(hunting): get_auth_request con 0x7f7fe006a230 auth_method 0 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 10 monclient(hunting): handle_auth_done global_id 34139 payload 824 2024-09-16T15:03:15.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7fe0069e50 msgr2=0x7f7fe01cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7fe0069e50 0x7f7fe01cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fe006a230 msgr2=0x7f7fe01cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fe006a230 0x7f7fe01cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 monclient: found mon.noname-b 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8009400 con 0x7f7fe01374a0 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.582+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fe006a230 0x7f7fe01cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 msgr2=0x7f7fe0091400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe5113640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 0x7f7fe0091400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800b410 tx=0x7f7fc8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fc8011cb0 con 0x7f7fe01374a0 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:03:15.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: dump: 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.600 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc8011e70 con 0x7f7fe01374a0 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 5 monclient: authenticate success, global_id 34139 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fc80107f0 con 0x7f7fe01374a0 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 10 monclient: _renew_subs 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd0009a70 con 0x7f7fe01374a0 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdeffd640 4 set_mon_vals no callback set 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:15.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fdd7fa640 10 monclient: dump: 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.602 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 10 monclient: _renew_subs 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.584+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe0137cf0 con 0x7f7fe01374a0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7fc8010380 con 0x7f7fe01374a0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe6b76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 0x7f7fe0091d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00947c0 con 0x7f7fe0091940 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fdd7fa640 1 --2- 172.21.15.134:0/1967190401 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fc00388b0 0x7f7fc003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f7fc8049d90 con 0x7f7fe01374a0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 0x7f7fe0091d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient: get_auth_request con 0x7f7fe0091940 auth_method 0 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1967190401 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fc00388b0 0x7f7fc003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.585+0000 7f7fdffff640 10 monclient: get_auth_request con 0x7f7fc00388b0 auth_method 0 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.586+0000 7f7fe4912640 10 monclient(hunting): handle_auth_done global_id 34129 payload 824 2024-09-16T15:03:15.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.586+0000 7f7fe4912640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fe0061ae0 con 0x7f7fe0091940 2024-09-16T15:03:15.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fe4912640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 msgr2=0x7f7fe0091d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 0x7f7fe0091d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd800f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdffff640 1 --2- 172.21.15.134:0/1967190401 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fc00388b0 0x7f7fc003ad70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009dc0 tx=0x7f7fd002f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fd8014ac0 con 0x7f7fe0091940 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 10 monclient: dump: 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.607 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7fd8014c20 con 0x7f7fe0091940 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 msgr2=0x7f7fe0091d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 0x7f7fe0091d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd800f610 comp rx=0 tx=0).stop 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 msgr2=0x7f7fe0091d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.588+0000 7f7fdd7fa640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0091940 0x7f7fe0091d20 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd800f610 comp rx=0 tx=0).stop 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.594+0000 7f7fe6b76640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.594+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 0x7f7fe006f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.594+0000 7f7fe6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.594+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0073880 con 0x7f7fe0092260 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 0x7f7fe006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient: get_auth_request con 0x7f7fe0092260 auth_method 0 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 10 monclient(hunting): handle_auth_done global_id 34138 payload 824 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.595+0000 7f7fe4912640 1 -- 172.21.15.134:0/1967190401 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7fe007a680 con 0x7f7fe0092260 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fe4912640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 msgr2=0x7f7fe006f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fe4912640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 0x7f7fe006f290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fd8022e10 con 0x7f7fe0092260 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fdd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fdd7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.597+0000 7f7fdd7fa640 10 monclient: dump: 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.610 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f7fd8026070 con 0x7f7fe0092260 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 msgr2=0x7f7fe006f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 0x7f7fe006f290 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd80046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 msgr2=0x7f7fe006f290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fdd7fa640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fe0092260 0x7f7fe006f290 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011c00 tx=0x7f7fd80046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.611 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.612 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.613 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.614 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.615 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.617 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.618 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.619 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.620 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.621 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.622 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.624 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.625 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.627 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.629 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.630 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.631 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.632 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fc00388b0 msgr2=0x7f7fc003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1967190401 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fc00388b0 0x7f7fc003ad70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009dc0 tx=0x7f7fd002f000 comp rx=0 tx=0).stop 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 10 monclient: shutdown 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 msgr2=0x7f7fe0091400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 --2- 172.21.15.134:0/1967190401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fe01374a0 0x7f7fe0091400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800b410 tx=0x7f7fc8009f90 comp rx=0 tx=0).stop 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 reap_dead start 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe5113640 1 -- 172.21.15.134:0/1967190401 reap_dead start 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 shutdown_connections 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 >> 172.21.15.134:0/1967190401 conn(0x7f7fe00a2e20 msgr2=0x7f7fe0065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 shutdown_connections 2024-09-16T15:03:15.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.598+0000 7f7fe6b76640 1 -- 172.21.15.134:0/1967190401 wait complete. 2024-09-16T15:03:15.635 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.635 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:03:15.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:03:15.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.736+0000 7f38f1848640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.736+0000 7f38f1848640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.736+0000 7f38f1848640 10 monclient: monmap: 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.737375+0000 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.737375+0000 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.739 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.736+0000 7f38f1848640 1 Processor -- start 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 -- start start 2024-09-16T15:03:15.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient: init 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient: _add_conns ranks=[4,0,3,9,7,8,2,1,10,6,5] 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-c con 0x7f38ec136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f38ec137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:15.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec13c370 0x7f38ec13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-g con 0x7f38ec13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec073250 con 0x7f38ec137220 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0a56e0 con 0x7f38ec13c370 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec09abc0 con 0x7f38ec136e40 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38f1848640 10 monclient(hunting): authenticate will time out at 932.326604s 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50534/0 (socket says 172.21.15.134:50534) 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 -- 172.21.15.134:0/3878683460 learned_addr learned my addr 172.21.15.134:0/3878683460 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): get_auth_request con 0x7f38ec136e40 auth_method 0 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38ea7fc640 1 --1- 172.21.15.134:0/3878683460 >> v1:172.21.15.134:6789/0 conn(0x7f38ec137220 0x7f38ec137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:48146/0 (socket says 172.21.15.134:48146) 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec13c370 0x7f38ec13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): get_auth_request con 0x7f38ec13c370 auth_method 0 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient(hunting): handle_auth_done global_id 4513 payload 274 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 -- 172.21.15.134:0/3878683460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec13c370 msgr2=0x7f38ec13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec13c370 0x7f38ec13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 -- 172.21.15.134:0/3878683460 >> v1:172.21.15.134:6789/0 conn(0x7f38ec137220 legacy=0x7f38ec137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 monclient: found mon.noname-c 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.737+0000 7f38eaffd640 1 -- 172.21.15.134:0/3878683460 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec09ba50 con 0x7f38ec136e40 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.740+0000 7f38eaffd640 1 -- 172.21.15.134:0/3878683460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 msgr2=0x7f38ec09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.740+0000 7f38eaffd640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098a0 tx=0x7f38dc02a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7276f67c7686b325 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 1 -- 172.21.15.134:0/3878683460 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc02adf0 con 0x7f38ec136e40 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: dump: 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.750 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 1 -- 172.21.15.134:0/3878683460 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc0307b0 con 0x7f38ec136e40 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 1 -- 172.21.15.134:0/3878683460 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc030d90 con 0x7f38ec136e40 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 5 monclient: authenticate success, global_id 4513 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 4 set_mon_vals no callback set 2024-09-16T15:03:15.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38e9ffb640 10 monclient: dump: 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.752 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 10 monclient: shutdown 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 -- 172.21.15.134:0/3878683460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 msgr2=0x7f38ec09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098a0 tx=0x7f38dc02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 -- 172.21.15.134:0/3878683460 shutdown_connections 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec13c370 0x7f38ec13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 --2- 172.21.15.134:0/3878683460 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec136e40 0x7f38ec09c150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 -- 172.21.15.134:0/3878683460 >> 172.21.15.134:0/3878683460 conn(0x7f38ec0a6110 msgr2=0x7f38ec0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 -- 172.21.15.134:0/3878683460 shutdown_connections 2024-09-16T15:03:15.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.741+0000 7f38f1848640 1 -- 172.21.15.134:0/3878683460 wait complete. 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.742+0000 7f38f1848640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.742+0000 7f38f1848640 10 monclient: monmap: 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.754 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.742+0000 7f38f1848640 1 Processor -- start 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.742+0000 7f38f1848640 1 -- start start 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.742+0000 7f38f1848640 10 monclient: init 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient: _add_conns ranks=[8,2,4,1,3,6,0,5,7] 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f38ec136e40 0x7f38ec135e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-e con 0x7f38ec136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38ec137220 0x7f38ec136360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-b con 0x7f38ec137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): picked mon.noname-c con 0x7f38ec13c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1cc420 con 0x7f38ec137220 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0a7a60 con 0x7f38ec13c370 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1224f0 con 0x7f38ec136e40 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38f1848640 10 monclient(hunting): authenticate will time out at 932.332284s 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38ec137220 0x7f38ec136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f38ec136e40 0x7f38ec135e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50548/0 (socket says 172.21.15.134:50548) 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 learned_addr learned my addr 172.21.15.134:0/850749522 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): get_auth_request con 0x7f38ec13c370 auth_method 0 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): get_auth_request con 0x7f38ec137220 auth_method 0 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): get_auth_request con 0x7f38ec136e40 auth_method 0 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient(hunting): handle_auth_done global_id 4522 payload 824 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f38ec136e40 msgr2=0x7f38ec135e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f38ec136e40 0x7f38ec135e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38ec137220 msgr2=0x7f38ec136360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38ec137220 0x7f38ec136360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 monclient: found mon.noname-c 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0094d0 con 0x7f38ec13c370 2024-09-16T15:03:15.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38eaffd640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f38ec136e40 0x7f38ec135e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.743+0000 7f38ea7fc640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38ec137220 0x7f38ec136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.745+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 msgr2=0x7f38ec11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.745+0000 7f38eb7fe640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009cb0 tx=0x7f38e400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e4009b20 con 0x7f38ec13c370 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: dump: 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.764 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 5 monclient: authenticate success, global_id 4522 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e400f040 con 0x7f38ec13c370 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 monclient: _renew_subs 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec136c30 con 0x7f38ec13c370 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38e4014d60 con 0x7f38ec13c370 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 10 monclient: dump: 2024-09-16T15:03:15.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 monclient: _renew_subs 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec13ac80 con 0x7f38ec13c370 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e97fa640 4 set_mon_vals no callback set 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 0x7f38ec120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec071b20 con 0x7f38ec1201a0 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.746+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f38e4014480 con 0x7f38ec13c370 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38e3fff640 1 --2- 172.21.15.134:0/850749522 >> v2:172.21.15.162:6800/980547263 conn(0x7f38cc038880 0x7f38cc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 0x7f38ec120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f38e404ad50 con 0x7f38ec13c370 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient: get_auth_request con 0x7f38ec1201a0 auth_method 0 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38ea7fc640 1 --2- 172.21.15.134:0/850749522 >> v2:172.21.15.162:6800/980547263 conn(0x7f38cc038880 0x7f38cc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38ea7fc640 10 monclient: get_auth_request con 0x7f38cc038880 auth_method 0 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 10 monclient(hunting): handle_auth_done global_id 34149 payload 824 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.747+0000 7f38eaffd640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38ec061ae0 con 0x7f38ec1201a0 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38eaffd640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 msgr2=0x7f38ec120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38eaffd640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 0x7f38ec120580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06cb00 tx=0x7f38dc0046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38ea7fc640 1 --2- 172.21.15.134:0/850749522 >> v2:172.21.15.162:6800/980547263 conn(0x7f38cc038880 0x7f38cc03ad40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38ec13a060 tx=0x7f38d4007e60 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc030e80 con 0x7f38ec1201a0 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38e3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.749+0000 7f38e3fff640 10 monclient: dump: 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.770 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f38dc003610 con 0x7f38ec1201a0 2024-09-16T15:03:15.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 msgr2=0x7f38ec120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 0x7f38ec120580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06cb00 tx=0x7f38dc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 msgr2=0x7f38ec120580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.750+0000 7f38e3fff640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec1201a0 0x7f38ec120580 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38ec06cb00 tx=0x7f38dc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38f1848640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38f1848640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 0x7f38ec09e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38f1848640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec07a680 con 0x7f38ec09e520 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 0x7f38ec09e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient: get_auth_request con 0x7f38ec09e520 auth_method 0 2024-09-16T15:03:15.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 10 monclient(hunting): handle_auth_done global_id 34158 payload 824 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.756+0000 7f38eaffd640 1 -- 172.21.15.134:0/850749522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f38ec0a53c0 con 0x7f38ec09e520 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38eaffd640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 msgr2=0x7f38ec09e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38eaffd640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 0x7f38ec09e930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0097c0 tx=0x7f38dc008030 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38dc049300 con 0x7f38ec09e520 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:15.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 10 monclient: dump: 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.774 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f38dc003ce0 con 0x7f38ec09e520 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 msgr2=0x7f38ec09e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 0x7f38ec09e930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0097c0 tx=0x7f38dc008030 comp rx=0 tx=0).stop 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 msgr2=0x7f38ec09e930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38e3fff640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38ec09e520 0x7f38ec09e930 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0097c0 tx=0x7f38dc008030 comp rx=0 tx=0).stop 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.775 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:15.776 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.777 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.778 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.779 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.783 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.784 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.786 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.792 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.796 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.796 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.796 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 >> v2:172.21.15.162:6800/980547263 conn(0x7f38cc038880 msgr2=0x7f38cc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 --2- 172.21.15.134:0/850749522 >> v2:172.21.15.162:6800/980547263 conn(0x7f38cc038880 0x7f38cc03ad40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f38ec13a060 tx=0x7f38d4007e60 comp rx=0 tx=0).stop 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 10 monclient: shutdown 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 msgr2=0x7f38ec11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009cb0 tx=0x7f38e400c490 comp rx=0 tx=0).stop 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38eb7fe640 1 -- 172.21.15.134:0/850749522 reap_dead start 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 shutdown_connections 2024-09-16T15:03:15.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 --2- 172.21.15.134:0/850749522 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f38ec13c370 0x7f38ec11fc60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.759+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 >> 172.21.15.134:0/850749522 conn(0x7f38ec0a6110 msgr2=0x7f38ec06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.760+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 shutdown_connections 2024-09-16T15:03:15.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.760+0000 7f38f1848640 1 -- 172.21.15.134:0/850749522 wait complete. 2024-09-16T15:03:15.798 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.798 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:03:15.798 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:03:15.821 INFO:tasks.rados.rados.0.smithi162.stdout:568: finishing write tid 1 to smithi16234619-7 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: finishing write tid 2 to smithi16234619-7 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: finishing write tid 3 to smithi16234619-7 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: finishing write tid 4 to smithi16234619-7 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: finishing write tid 5 to smithi16234619-7 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: oid 7 updating version 0 to 178 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: oid 7 version 178 is already newer than 177 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: oid 7 version 178 is already newer than 176 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: oid 7 version 178 is already newer than 175 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 178 (ObjNum 232 snap 46 seq_num 232) dirty exists 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:568: left oid 7 (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:567: finishing copy_from racing read to smithi16234619-9 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:567: finishing copy_from to smithi16234619-9 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 172 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:571: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:03:15.822 INFO:tasks.rados.rados.0.smithi162.stdout:571: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:563: done (5 left) 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:565: done (4 left) 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:567: done (3 left) 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:568: done (2 left) 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:572: read oid 30 snap -1 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:572: expect (ObjNum 214 snap 44 seq_num 214) 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:573: write oid 2 current snap is 46 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:571: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 230 (ObjNum 229 snap 45 seq_num 229) dirty exists 2024-09-16T15:03:15.823 INFO:tasks.rados.rados.0.smithi162.stdout:570: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.895+0000 7f221430f640 10 monclient: get_monmap_and_config 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.895+0000 7f221430f640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.895+0000 7f221430f640 10 monclient: monmap: 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:15.896444+0000 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:15.896444+0000 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.897 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 Processor -- start 2024-09-16T15:03:15.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 -- start start 2024-09-16T15:03:15.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient: init 2024-09-16T15:03:15.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient: _add_conns ranks=[7,3,2,0,1,9,6,10,5,8,4] 2024-09-16T15:03:15.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f220c136e40 0x7f220c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): picked mon.noname-i con 0x7f220c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c137220 0x7f220c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): picked mon.noname-g con 0x7f220c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): picked mon.noname-b con 0x7f220c135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c09abc0 con 0x7f220c135dc0 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c09ba50 con 0x7f220c137220 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0a53c0 con 0x7f220c136e40 2024-09-16T15:03:15.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f221430f640 10 monclient(hunting): authenticate will time out at 932.485788s 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45234/0 (socket says 172.21.15.134:45234) 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 1 -- 172.21.15.134:0/486007709 learned_addr learned my addr 172.21.15.134:0/486007709 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 10 monclient(hunting): get_auth_request con 0x7f220c135dc0 auth_method 0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22128ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22118aa640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c137220 0x7f220c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.896+0000 7f22120ab640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f220c136e40 0x7f220c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): get_auth_request con 0x7f220c137220 auth_method 0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): get_auth_request con 0x7f220c136e40 auth_method 0 2024-09-16T15:03:15.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient(hunting): handle_auth_done global_id 34157 payload 274 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 -- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f220c136e40 msgr2=0x7f220c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f220c136e40 0x7f220c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 -- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c137220 msgr2=0x7f220c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c137220 0x7f220c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 monclient: found mon.noname-b 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.897+0000 7f22128ac640 1 -- 172.21.15.134:0/486007709 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c073250 con 0x7f220c135dc0 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.899+0000 7f22128ac640 1 -- 172.21.15.134:0/486007709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 msgr2=0x7f220c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.899+0000 7f22128ac640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e80 tx=0x7f220402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47182bd16b6a2e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 1 -- 172.21.15.134:0/486007709 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2204007a70 con 0x7f220c135dc0 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: dump: 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.912 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 1 -- 172.21.15.134:0/486007709 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220402d050 con 0x7f220c135dc0 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 5 monclient: authenticate success, global_id 34157 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 1 -- 172.21.15.134:0/486007709 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2204007bf0 con 0x7f220c135dc0 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 4 set_mon_vals no callback set 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f22110a9640 10 monclient: dump: 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.914 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 10 monclient: shutdown 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 -- 172.21.15.134:0/486007709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 msgr2=0x7f220c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e80 tx=0x7f220402a530 comp rx=0 tx=0).stop 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 -- 172.21.15.134:0/486007709 shutdown_connections 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c135dc0 0x7f220c136220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c137220 0x7f220c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 --2- 172.21.15.134:0/486007709 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f220c136e40 0x7f220c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 -- 172.21.15.134:0/486007709 >> 172.21.15.134:0/486007709 conn(0x7f220c0a6110 msgr2=0x7f220c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 -- 172.21.15.134:0/486007709 shutdown_connections 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.900+0000 7f221430f640 1 -- 172.21.15.134:0/486007709 wait complete. 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.901+0000 7f221430f640 10 monclient: build_initial_monmap 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.901+0000 7f221430f640 10 monclient: monmap: 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:15.916 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.901+0000 7f221430f640 1 Processor -- start 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.901+0000 7f221430f640 1 -- start start 2024-09-16T15:03:15.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.901+0000 7f221430f640 10 monclient: init 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient: _add_conns ranks=[3,2,6,4,1,7,8,5,0] 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c135dc0 0x7f220c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): picked mon.noname-g con 0x7f220c135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 0x7f220c1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): picked mon.noname-b con 0x7f220c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 0x7f220c1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): picked mon.noname-d con 0x7f220c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c134cb0 con 0x7f220c136e40 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c136350 con 0x7f220c135dc0 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f220c122f00 con 0x7f220c137220 2024-09-16T15:03:15.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f221430f640 10 monclient(hunting): authenticate will time out at 932.491124s 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 0x7f220c1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 0x7f220c1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 0x7f220c1201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45242/0 (socket says 172.21.15.134:45242) 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 1 -- 172.21.15.134:0/3801450521 learned_addr learned my addr 172.21.15.134:0/3801450521 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): get_auth_request con 0x7f220c136e40 auth_method 0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): get_auth_request con 0x7f220c137220 auth_method 0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c135dc0 0x7f220c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): get_auth_request con 0x7f220c135dc0 auth_method 0 2024-09-16T15:03:15.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient(hunting): handle_auth_done global_id 34152 payload 824 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c135dc0 msgr2=0x7f220c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f220c135dc0 0x7f220c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 msgr2=0x7f220c1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 0x7f220c1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient: _finish_hunting 0 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 monclient: found mon.noname-d 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204009b30 con 0x7f220c137220 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.902+0000 7f22118aa640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f220c136e40 0x7f220c1201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 msgr2=0x7f220c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f22128ac640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 0x7f220c1206e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2204030000 tx=0x7f2204004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2204003e20 con 0x7f220c137220 2024-09-16T15:03:15.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: dump: 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.922 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f221430f640 5 monclient: authenticate success, global_id 34152 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204004790 con 0x7f220c137220 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f221430f640 10 monclient: _renew_subs 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f221430f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c0a0800 con 0x7f220c137220 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f220403b400 con 0x7f220c137220 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f2202ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f2202ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:15.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.904+0000 7f22108a8640 4 set_mon_vals no callback set 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f2202ffd640 10 monclient: dump: 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.924 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 10 monclient: _renew_subs 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c122c20 con 0x7f220c137220 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 0x7f220c1bd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0735d0 con 0x7f220c136880 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.905+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f220402ad50 con 0x7f220c137220 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 0x7f220c1bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient: get_auth_request con 0x7f220c136880 auth_method 0 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f2202ffd640 1 --2- 172.21.15.134:0/3801450521 >> v2:172.21.15.162:6800/980547263 conn(0x7f21f0038880 0x7f21f003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f2204069f50 con 0x7f220c137220 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 10 monclient(hunting): handle_auth_done global_id 4451 payload 824 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22120ab640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f220c061ae0 con 0x7f220c136880 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22118aa640 1 --2- 172.21.15.134:0/3801450521 >> v2:172.21.15.162:6800/980547263 conn(0x7f21f0038880 0x7f21f003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.906+0000 7f22118aa640 10 monclient: get_auth_request con 0x7f21f0038880 auth_method 0 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f22120ab640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 msgr2=0x7f220c1bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f22120ab640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 0x7f220c1bd060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22080118e0 tx=0x7f2208011da0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f22118aa640 1 --2- 172.21.15.134:0/3801450521 >> v2:172.21.15.162:6800/980547263 conn(0x7f21f0038880 0x7f21f003ad40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f220c0a30e0 tx=0x7f21fc009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f220800fdc0 con 0x7f220c136880 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f2202ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.908+0000 7f2202ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 10 monclient: dump: 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.929 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.930 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f220800f950 con 0x7f220c136880 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 msgr2=0x7f220c1bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 0x7f220c1bd060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f22080118e0 tx=0x7f2208011da0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 msgr2=0x7f220c1bd060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.909+0000 7f2202ffd640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c136880 0x7f220c1bd060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f221430f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f221430f640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 0x7f220c1bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f221430f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f220c122db0 con 0x7f220c1bd5a0 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 0x7f220c1bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient: get_auth_request con 0x7f220c1bd5a0 auth_method 0 2024-09-16T15:03:15.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 10 monclient(hunting): handle_auth_done global_id 4460 payload 824 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.915+0000 7f22120ab640 1 -- 172.21.15.134:0/3801450521 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f220c07a680 con 0x7f220c1bd5a0 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.917+0000 7f22120ab640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 msgr2=0x7f220c1bd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.917+0000 7f22120ab640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 0x7f220c1bd9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f220801b000 tx=0x7f22080046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f220801ece0 con 0x7f220c1bd5a0 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 10 monclient: dump: 2024-09-16T15:03:15.932 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:15.933 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f2208024040 con 0x7f220c1bd5a0 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 msgr2=0x7f220c1bd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 0x7f220c1bd9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f220801b000 tx=0x7f22080046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 msgr2=0x7f220c1bd9b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f2202ffd640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f220c1bd5a0 0x7f220c1bd9b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.934 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.935 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:15.936 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:15.937 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:15.938 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.939 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:15.943 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:15.945 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:15.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:15.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.953 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:15.954 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:15.956 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:15.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 >> v2:172.21.15.162:6800/980547263 conn(0x7f21f0038880 msgr2=0x7f21f003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 --2- 172.21.15.134:0/3801450521 >> v2:172.21.15.162:6800/980547263 conn(0x7f21f0038880 0x7f21f003ad40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f220c0a30e0 tx=0x7f21fc009190 comp rx=0 tx=0).stop 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 10 monclient: shutdown 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 msgr2=0x7f220c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 --2- 172.21.15.134:0/3801450521 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f220c137220 0x7f220c1206e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2204030000 tx=0x7f2204004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 reap_dead start 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f22128ac640 1 -- 172.21.15.134:0/3801450521 reap_dead start 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 shutdown_connections 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 >> 172.21.15.134:0/3801450521 conn(0x7f220c0a6110 msgr2=0x7f220c0a8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 shutdown_connections 2024-09-16T15:03:15.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:15.918+0000 7f221430f640 1 -- 172.21.15.134:0/3801450521 wait complete. 2024-09-16T15:03:15.958 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:15.959 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:03:15.959 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:03:15.959 INFO:tasks.rados.rados.0.smithi162.stdout:572: expect (ObjNum 214 snap 44 seq_num 214) 2024-09-16T15:03:16.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 10 monclient: get_monmap_and_config 2024-09-16T15:03:16.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 10 monclient: build_initial_monmap 2024-09-16T15:03:16.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 10 monclient: monmap: 2024-09-16T15:03:16.056 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:16.056 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:16.056060+0000 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:16.056060+0000 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:16.057 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:16.058 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:16.058 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 1 Processor -- start 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 1 -- start start 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.055+0000 7f90ac0e9640 10 monclient: init 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient: _add_conns ranks=[4,9,0,2,1,8,7,6,3,10,5] 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-c con 0x7f90a409fb10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 0x7f90a4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-a con 0x7f90a409cd20 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:16.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f90a406a390 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a4073250 con 0x7f90a406a390 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a409fef0 con 0x7f90a409cd20 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40665e0 con 0x7f90a409fb10 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90ac0e9640 10 monclient(hunting): authenticate will time out at 932.645345s 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 0x7f90a4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90aa686640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f90a406a390 0x7f90a413bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:48158/0 (socket says 172.21.15.134:48158) 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): get_auth_request con 0x7f90a409cd20 auth_method 0 2024-09-16T15:03:16.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90aa686640 1 -- 172.21.15.134:0/1131795828 learned_addr learned my addr 172.21.15.134:0/1131795828 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:50552/0 (socket says 172.21.15.134:50552) 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): get_auth_request con 0x7f90a409fb10 auth_method 0 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient(hunting): handle_auth_done global_id 4531 payload 274 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 -- 172.21.15.134:0/1131795828 >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 msgr2=0x7f90a4069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 --2- 172.21.15.134:0/1131795828 >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 0x7f90a4069e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 -- 172.21.15.134:0/1131795828 >> v1:172.21.15.134:6789/0 conn(0x7f90a406a390 legacy=0x7f90a413bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient: _finish_hunting 0 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 monclient: found mon.noname-c 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9e85640 1 -- 172.21.15.134:0/1131795828 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40a61e0 con 0x7f90a409fb10 2024-09-16T15:03:16.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.056+0000 7f90a9684640 1 --2- 172.21.15.134:0/1131795828 >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 0x7f90a4069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:16.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a9e85640 1 -- 172.21.15.134:0/1131795828 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 msgr2=0x7f90a409c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a9e85640 1 --2- 172.21.15.134:0/1131795828 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097a0 tx=0x7f909c02a0a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1ca76796f26695b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 1 -- 172.21.15.134:0/1131795828 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f909c02adc0 con 0x7f90a409fb10 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: dump: 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.071 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: _finish_auth 0 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: _check_auth_tickets 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 1 -- 172.21.15.134:0/1131795828 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c0307b0 con 0x7f90a409fb10 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90ac0e9640 5 monclient: authenticate success, global_id 4531 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90ac0e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90ac0e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 1 -- 172.21.15.134:0/1131795828 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f909c030d90 con 0x7f90a409fb10 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90ac0e9640 4 set_mon_vals no callback set 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.059+0000 7f90a8e83640 10 monclient: dump: 2024-09-16T15:03:16.073 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 10 monclient: shutdown 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 -- 172.21.15.134:0/1131795828 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 msgr2=0x7f90a409c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/1131795828 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097a0 tx=0x7f909c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 -- 172.21.15.134:0/1131795828 shutdown_connections 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/1131795828 >> v2:172.21.15.134:3300/0 conn(0x7f90a409cd20 0x7f90a4069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/1131795828 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90a409fb10 0x7f90a409c7e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 -- 172.21.15.134:0/1131795828 >> 172.21.15.134:0/1131795828 conn(0x7f90a40a6400 msgr2=0x7f90a40a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 -- 172.21.15.134:0/1131795828 shutdown_connections 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.060+0000 7f90ac0e9640 1 -- 172.21.15.134:0/1131795828 wait complete. 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient: build_initial_monmap 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient: monmap: 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:16.075 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 Processor -- start 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 -- start start 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient: init 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:16.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient: _add_conns ranks=[2,0,3,1,6,5,8,7,4] 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90a406a390 0x7f90a41cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-b con 0x7f90a406a390 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-a con 0x7f90a409cd20 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90a409fb10 0x7f90a41c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): picked mon.noname-g con 0x7f90a409fb10 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a406a9e0 con 0x7f90a409cd20 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a409c540 con 0x7f90a406a390 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40a6950 con 0x7f90a409fb10 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:16.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90ac0e9640 10 monclient(hunting): authenticate will time out at 932.650897s 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90a9684640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.061+0000 7f90a9684640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:43622/0 (socket says 172.21.15.134:43622) 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 -- 172.21.15.134:0/17434228 learned_addr learned my addr 172.21.15.134:0/17434228 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): get_auth_request con 0x7f90a409cd20 auth_method 0 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90a406a390 0x7f90a41cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): get_auth_request con 0x7f90a406a390 auth_method 0 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90a409fb10 0x7f90a41c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): get_auth_request con 0x7f90a409fb10 auth_method 0 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90aa686640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient(hunting): handle_auth_done global_id 4617 payload 824 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90a409fb10 msgr2=0x7f90a41c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90a409fb10 0x7f90a41c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90a406a390 msgr2=0x7f90a41cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90a406a390 0x7f90a41cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient: _finish_hunting 0 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 monclient: found mon.noname-a 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9684640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098009a70 con 0x7f90a409cd20 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.062+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90a406a390 0x7f90a41cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a9684640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 msgr2=0x7f90a4131cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a9684640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f909800efc0 tx=0x7f9098009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f909800ce60 con 0x7f90a409cd20 2024-09-16T15:03:16.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: dump: 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.081 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800f040 con 0x7f90a409cd20 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 5 monclient: authenticate success, global_id 4617 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 10 monclient: _renew_subs 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c009400 con 0x7f90a409cd20 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90980133e0 con 0x7f90a409cd20 2024-09-16T15:03:16.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a27fc640 10 monclient: dump: 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.083 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 10 monclient: _renew_subs 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90a3fff640 4 set_mon_vals no callback set 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.064+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a409ddc0 con 0x7f90a409cd20 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9684640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f909800efc0 tx=0x7f9098009f90 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f9098014690 mgrmap(e 5) v1 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90ac0e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9098014690 con 0x7f90a409cd20 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 0x7f90a41c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90a406b5d0 con 0x7f90a4132210 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a27fc640 1 --2- 172.21.15.134:0/17434228 >> v2:172.21.15.162:6800/980547263 conn(0x7f9090038880 0x7f909003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 0x7f90a41c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient: get_auth_request con 0x7f90a4132210 auth_method 0 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90aa686640 1 --2- 172.21.15.134:0/17434228 >> v2:172.21.15.162:6800/980547263 conn(0x7f9090038880 0x7f909003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.065+0000 7f90aa686640 10 monclient: get_auth_request con 0x7f9090038880 auth_method 0 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.066+0000 7f90a9e85640 10 monclient(hunting): handle_auth_done global_id 4471 payload 824 2024-09-16T15:03:16.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.066+0000 7f90a9e85640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90a4061ae0 con 0x7f90a4132210 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90aa686640 1 --2- 172.21.15.134:0/17434228 >> v2:172.21.15.162:6800/980547263 conn(0x7f9090038880 0x7f909003ad40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9094009250 tx=0x7f90940091e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a9e85640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 msgr2=0x7f90a41c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 0x7f90a41c8580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f909c034000 tx=0x7f909c0046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f909c003c90 con 0x7f90a4132210 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 10 monclient: dump: 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.086 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f909c003df0 con 0x7f90a4132210 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 msgr2=0x7f90a41c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 0x7f90a41c8580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f909c034000 tx=0x7f909c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 msgr2=0x7f90a41c8580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:16.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.068+0000 7f90a27fc640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a4132210 0x7f90a41c8580 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f909c034000 tx=0x7f909c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.074+0000 7f90ac0e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:16.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.074+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 0x7f90a40a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.074+0000 7f90ac0e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.074+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4073880 con 0x7f90a41c8ac0 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 0x7f90a40a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient: get_auth_request con 0x7f90a41c8ac0 auth_method 0 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 10 monclient(hunting): handle_auth_done global_id 4480 payload 824 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.075+0000 7f90a9e85640 1 -- 172.21.15.134:0/17434228 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90a407a680 con 0x7f90a41c8ac0 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a9e85640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 msgr2=0x7f90a40a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a9e85640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 0x7f90a40a1d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097a0 tx=0x7f909c008030 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f909c049c80 con 0x7f90a41c8ac0 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a27fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.077+0000 7f90a27fc640 10 monclient: dump: 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.091 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f909c049de0 con 0x7f90a41c8ac0 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 msgr2=0x7f90a40a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 0x7f90a40a1d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097a0 tx=0x7f909c008030 comp rx=0 tx=0).stop 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 msgr2=0x7f90a40a1d10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90a41c8ac0 0x7f90a40a1d10 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097a0 tx=0x7f909c008030 comp rx=0 tx=0).stop 2024-09-16T15:03:16.092 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:16.093 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:16.094 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:16.095 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:16.096 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:16.097 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:16.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:16.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.102 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:16.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:16.106 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:16.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.108 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:16.110 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:16.112 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:16.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:16.114 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:16.115 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:16.116 INFO:tasks.rados.rados.0.smithi162.stdout:573: seq_num 234 ranges {732838=698856,2026904=465743,3210039=571651} 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 >> v2:172.21.15.162:6800/980547263 conn(0x7f9090038880 msgr2=0x7f909003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/17434228 >> v2:172.21.15.162:6800/980547263 conn(0x7f9090038880 0x7f909003ad40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9094009250 tx=0x7f90940091e0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 10 monclient: shutdown 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 msgr2=0x7f90a4131cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 --2- 172.21.15.134:0/17434228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90a409cd20 0x7f90a4131cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f909800efc0 tx=0x7f9098009f90 comp rx=0 tx=0).stop 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90aa686640 1 -- 172.21.15.134:0/17434228 reap_dead start 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90a27fc640 1 -- 172.21.15.134:0/17434228 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 3915+0+0 (unknown 0 0 0) 0x7f909804b030 con 0x7f90a409cd20 2024-09-16T15:03:16.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90aa686640 1 -- 172.21.15.134:0/17434228 reap_dead start 2024-09-16T15:03:16.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 shutdown_connections 2024-09-16T15:03:16.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 >> 172.21.15.134:0/17434228 conn(0x7f90a40a6400 msgr2=0x7f90a41bcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:16.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 shutdown_connections 2024-09-16T15:03:16.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.078+0000 7f90ac0e9640 1 -- 172.21.15.134:0/17434228 wait complete. 2024-09-16T15:03:16.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:16.118 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:03:16.118 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:03:16.122 INFO:tasks.rados.rados.0.smithi162.stdout:573: writing smithi16234619-2 from 732838 to 1431694 tid 1 2024-09-16T15:03:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:573: writing smithi16234619-2 from 2026904 to 2492647 tid 2 2024-09-16T15:03:16.135 INFO:tasks.rados.rados.0.smithi162.stdout:573: writing smithi16234619-2 from 3210039 to 3781690 tid 3 2024-09-16T15:03:16.135 INFO:tasks.rados.rados.0.smithi162.stdout:570: done (3 left) 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:571: done (2 left) 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:572: done (1 left) 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:574: rollback oid 3 current snap is 46 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 45 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:575: write (excl) oid 19 current snap is 46 2024-09-16T15:03:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:575: seq_num 235 ranges {749645=774105,2085386=609401,3237462=712183} 2024-09-16T15:03:16.144 INFO:tasks.rados.rados.0.smithi162.stdout:575: writing smithi16234619-19 from 749645 to 1523750 tid 1 2024-09-16T15:03:16.150 INFO:tasks.rados.rados.0.smithi162.stdout:575: writing smithi16234619-19 from 2085386 to 2694787 tid 2 2024-09-16T15:03:16.157 INFO:tasks.rados.rados.0.smithi162.stdout:575: writing smithi16234619-19 from 3237462 to 3949645 tid 3 2024-09-16T15:03:16.157 INFO:tasks.rados.rados.0.smithi162.stdout:576: copy_from oid 41 from oid 47 current snap is 46 2024-09-16T15:03:16.157 INFO:tasks.rados.rados.0.smithi162.stdout:574: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:03:16.157 INFO:tasks.rados.rados.0.smithi162.stdout:574: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:03:16.158 INFO:tasks.rados.rados.0.smithi162.stdout:574: finishing rollback tid 2 to smithi16234619-3 2024-09-16T15:03:16.158 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 29 (ObjNum 189 snap 39 seq_num 189) dirty exists 2024-09-16T15:03:16.158 INFO:tasks.rados.rados.0.smithi162.stdout:574: done (3 left) 2024-09-16T15:03:16.158 INFO:tasks.rados.rados.0.smithi162.stdout:577: write oid 12 current snap is 46 2024-09-16T15:03:16.158 INFO:tasks.rados.rados.0.smithi162.stdout:577: seq_num 236 ranges {766452=565707,1860221=469412,2981238=594195} 2024-09-16T15:03:16.164 INFO:tasks.rados.rados.0.smithi162.stdout:577: writing smithi16234619-12 from 766452 to 1332159 tid 1 2024-09-16T15:03:16.169 INFO:tasks.rados.rados.0.smithi162.stdout:577: writing smithi16234619-12 from 1860221 to 2329633 tid 2 2024-09-16T15:03:16.176 INFO:tasks.rados.rados.0.smithi162.stdout:577: writing smithi16234619-12 from 2981238 to 3575433 tid 3 2024-09-16T15:03:16.177 INFO:tasks.rados.rados.0.smithi162.stdout:578: write (excl) oid 24 current snap is 46 2024-09-16T15:03:16.177 INFO:tasks.rados.rados.0.smithi162.stdout:573: finishing write tid 1 to smithi16234619-2 2024-09-16T15:03:16.177 INFO:tasks.rados.rados.0.smithi162.stdout:573: finishing write tid 2 to smithi16234619-2 2024-09-16T15:03:16.177 INFO:tasks.rados.rados.0.smithi162.stdout:578: seq_num 237 ranges {783259=640956,1918703=613070,3008661=405467,3821939=161320} 2024-09-16T15:03:16.184 INFO:tasks.rados.rados.0.smithi162.stdout:578: writing smithi16234619-24 from 783259 to 1424215 tid 1 2024-09-16T15:03:16.190 INFO:tasks.rados.rados.0.smithi162.stdout:578: writing smithi16234619-24 from 1918703 to 2531773 tid 2 2024-09-16T15:03:16.195 INFO:tasks.rados.rados.0.smithi162.stdout:578: writing smithi16234619-24 from 3008661 to 3414128 tid 3 2024-09-16T15:03:16.198 INFO:tasks.rados.rados.0.smithi162.stdout:578: writing smithi16234619-24 from 3821939 to 3983259 tid 4 2024-09-16T15:03:16.198 INFO:tasks.rados.rados.0.smithi162.stdout:579: snap_remove snap 45 2024-09-16T15:03:16.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.240+0000 7f1e0c96c640 10 monclient: get_monmap_and_config 2024-09-16T15:03:16.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.240+0000 7f1e0c96c640 10 monclient: build_initial_monmap 2024-09-16T15:03:16.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.240+0000 7f1e0c96c640 10 monclient: monmap: 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:16.241390+0000 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:16.241390+0000 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:16.242 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:16.243 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:16.243 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:16.243 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:16.243 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 Processor -- start 2024-09-16T15:03:16.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 -- start start 2024-09-16T15:03:16.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient: init 2024-09-16T15:03:16.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:16.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient: _add_conns ranks=[7,5,0,1,10,9,4,2,3,6,8] 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 0x7f1e08136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-i con 0x7f1e08136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e0809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-h con 0x7f1e0809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1e08136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e08073250 con 0x7f1e08136c60 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0809b600 con 0x7f1e0809bfe0 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080a5740 con 0x7f1e08136030 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0c96c640 10 monclient(hunting): authenticate will time out at 932.830736s 2024-09-16T15:03:16.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e06d9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1e08136c60 0x7f1e081370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:51984/0 (socket says 172.21.15.134:51984) 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e06d9d640 1 -- 172.21.15.134:0/1653582048 learned_addr learned my addr 172.21.15.134:0/1653582048 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e05d9b640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e0809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.241+0000 7f1e0659c640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 0x7f1e08136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): get_auth_request con 0x7f1e08136030 auth_method 0 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): get_auth_request con 0x7f1e0809bfe0 auth_method 0 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_done global_id 4469 payload 274 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 1 -- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 msgr2=0x7f1e08136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 0x7f1e08136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 1 -- 172.21.15.134:0/1653582048 >> v1:172.21.15.134:6789/0 conn(0x7f1e08136c60 legacy=0x7f1e081370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 1 monclient: found mon.noname-h 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e05d9b640 1 -- 172.21.15.134:0/1653582048 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080a52c0 con 0x7f1e0809bfe0 2024-09-16T15:03:16.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.242+0000 7f1e0659c640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 0x7f1e08136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e05d9b640 1 -- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 msgr2=0x7f1e0809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e05d9b640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e0809c400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097d0 tx=0x7f1dfc02a000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e5f4f4add20740aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 1 -- 172.21.15.134:0/1653582048 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dfc02ad20 con 0x7f1e0809bfe0 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: dump: 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.254 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: _finish_auth 0 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: _check_auth_tickets 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 1 -- 172.21.15.134:0/1653582048 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc0307b0 con 0x7f1e0809bfe0 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0c96c640 5 monclient: authenticate success, global_id 4469 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0c96c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 1 -- 172.21.15.134:0/1653582048 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dfc02aea0 con 0x7f1e0809bfe0 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0559a640 10 monclient: dump: 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0c96c640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.245+0000 7f1e0c96c640 4 set_mon_vals no callback set 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 10 monclient: shutdown 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 -- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 msgr2=0x7f1e0809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e0809c400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097d0 tx=0x7f1dfc02a000 comp rx=0 tx=0).stop 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 -- 172.21.15.134:0/1653582048 shutdown_connections 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e0809c400 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/1653582048 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1e08136030 0x7f1e08136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 -- 172.21.15.134:0/1653582048 >> 172.21.15.134:0/1653582048 conn(0x7f1e080a9ea0 msgr2=0x7f1e080a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 -- 172.21.15.134:0/1653582048 shutdown_connections 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.246+0000 7f1e0c96c640 1 -- 172.21.15.134:0/1653582048 wait complete. 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.247+0000 7f1e0c96c640 10 monclient: build_initial_monmap 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.247+0000 7f1e0c96c640 10 monclient: monmap: 2024-09-16T15:03:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:16.259 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.247+0000 7f1e0c96c640 1 Processor -- start 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.247+0000 7f1e0c96c640 1 -- start start 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.247+0000 7f1e0c96c640 10 monclient: init 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient: _add_conns ranks=[5,8,3,7,1,2,4,0,6] 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e08134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-h con 0x7f1e0809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 0x7f1e081cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-e con 0x7f1e08136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 0x7f1e081cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): picked mon.noname-g con 0x7f1e08136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08120a80 con 0x7f1e08136c60 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080a8290 con 0x7f1e0809bfe0 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080a7df0 con 0x7f1e08136030 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0c96c640 10 monclient(hunting): authenticate will time out at 932.837405s 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 0x7f1e081cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 0x7f1e081cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46138/0 (socket says 172.21.15.134:46138) 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 1 -- 172.21.15.134:0/3958465970 learned_addr learned my addr 172.21.15.134:0/3958465970 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): get_auth_request con 0x7f1e08136030 auth_method 0 2024-09-16T15:03:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e08134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 0x7f1e081cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): get_auth_request con 0x7f1e0809bfe0 auth_method 0 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): get_auth_request con 0x7f1e08136c60 auth_method 0 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e05d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient(hunting): handle_auth_done global_id 34194 payload 824 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 -- 172.21.15.134:0/3958465970 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 msgr2=0x7f1e081cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 0x7f1e081cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 -- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 msgr2=0x7f1e08134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e08134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 monclient: found mon.noname-g 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:16.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e06d9d640 1 -- 172.21.15.134:0/3958465970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009400 con 0x7f1e08136c60 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.248+0000 7f1e0659c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e08134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e06d9d640 1 -- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 msgr2=0x7f1e081cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e06d9d640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 0x7f1e081cb870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bde0 tx=0x7f1e00009ed0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e00011b90 con 0x7f1e08136c60 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: dump: 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.264 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:16.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0000f050 con 0x7f1e08136c60 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1e00011d70 con 0x7f1e08136c60 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1df6ffd640 10 monclient: dump: 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:16.266 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e04d99640 4 set_mon_vals no callback set 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 5 monclient: authenticate success, global_id 34194 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 10 monclient: _renew_subs 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc009400 con 0x7f1e08136c60 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 10 monclient: _renew_subs 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.251+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e08135810 con 0x7f1e08136c60 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.253+0000 7f1e0c96c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.253+0000 7f1e0c96c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.253+0000 7f1e0c96c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.253+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e080679b0 con 0x7f1e08136c60 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.257+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1e00011720 con 0x7f1e08136c60 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.258+0000 7f1df6ffd640 1 --2- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 0x7f1de803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.258+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f1e0004b5c0 con 0x7f1e08136c60 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.258+0000 7f1e0659c640 1 --2- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 0x7f1de803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.258+0000 7f1e0659c640 10 monclient: get_auth_request con 0x7f1de80388b0 auth_method 0 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.260+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1e000119d0 con 0x7f1e08136c60 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.260+0000 7f1df6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.260+0000 7f1df6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.261+0000 7f1e0659c640 1 --2- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 0x7f1de803ad70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1df8011380 tx=0x7f1df8009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:16.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.487+0000 7f1e0c96c640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:03:16.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.487+0000 7f1e0c96c640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:03:16.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.487+0000 7f1e0c96c640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:16.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.487+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1e0809c400 con 0x7f1e08136c60 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=525853532,config=2586702516,health=2157093694,kv=1090497770,logm=2728030756}) 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=10,kv=11,logm=67} crc {auth=4085259141,config=2586702516,health=652400382,kv=1046579507,logm=3666946449}) 2024-09-16T15:03:16.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f1df6ffd640 1 -- 172.21.15.134:0/3958465970 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1e0001b360 con 0x7f1e08136c60 2024-09-16T15:03:16.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f1df6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:03:16.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f1df6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:16.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=9,config=1,health=12,kv=12,logm=66} crc {auth=2021465497,config=1747058506,health=2157093694,kv=1090497770,logm=3669308162}) 2024-09-16T15:03:16.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=1826596890,config=2586702516,health=2157093694,kv=1259556861,logm=3663995581}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=3509175595,config=2586702516,health=2157093694,kv=2983865385,logm=3805768555}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=9,config=2,health=12,kv=12,logm=65} crc {auth=1979115458,config=2586702516,health=2157093694,kv=1090497770,logm=2094828608}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=9,config=1,health=12,kv=11,logm=67} crc {auth=1979115458,config=1228700967,health=2157093694,kv=2676201184,logm=163568835}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=11,kv=12,logm=66} crc {auth=3495956353,config=2586702516,health=1886805112,kv=1090497770,logm=377273464}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.489+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=10,config=2,health=12,kv=11,logm=65} crc {auth=1313617517,config=2586702516,health=2157093694,kv=318701324,logm=522122840}) 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2569082334}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=961352995}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=718393888}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2316536319}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2997150831}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2347770207}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4037672166}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142587199}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.490+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1982616141}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3427643966}) 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2317000568}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.491+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1913725429}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2563745007}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1989440210}) 2024-09-16T15:03:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 msgr2=0x7f1de803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 0x7f1de803ad70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1df8011380 tx=0x7f1df8009190 comp rx=0 tx=0).stop 2024-09-16T15:03:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 10 monclient: shutdown 2024-09-16T15:03:16.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 msgr2=0x7f1e081cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 0x7f1e081cb870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000bde0 tx=0x7f1e00009ed0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=369608613}) 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1720436332}) 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1638418071}) 2024-09-16T15:03:16.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=353938809}) 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 shutdown_connections 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> v2:172.21.15.162:6800/980547263 conn(0x7f1de80388b0 0x7f1de803ad70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e08136c60 0x7f1e081cb870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1e08136030 0x7f1e081cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 --2- 172.21.15.134:0/3958465970 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1e0809bfe0 0x7f1e08134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 >> 172.21.15.134:0/3958465970 conn(0x7f1e080a9ea0 msgr2=0x7f1e0806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 shutdown_connections 2024-09-16T15:03:16.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:16.492+0000 7f1e0c96c640 1 -- 172.21.15.134:0/3958465970 wait complete. 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=170594065}) 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1060096345}) 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.506 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3081697557}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=163999078}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063791810}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=807508516}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.507 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2497599896}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1083402945}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.493+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2629810219}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1018615159}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3878127107}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=198116317}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.508 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1524556813}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=214306042}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2821457251}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2825886770}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.494+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=225353019}) 2024-09-16T15:03:16.509 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.495+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=588779420}) 2024-09-16T15:03:16.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:03:16.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=518722074}) 2024-09-16T15:03:16.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3294874119}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=676536204}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1367636330}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=508095940}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=475646486}) 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4169061699}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2198666565}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.656+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3361019566}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3491556878}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2519581040}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2333335774}) 2024-09-16T15:03:16.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2518475259}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1596112122}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3216339004}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=569171757}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2135590778}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.657+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3587788136}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=17} crc {logm=1905297805,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=431526784,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=84809210,osdmap=1078784257}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=15,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=22,osdmap=17} crc {logm=2229283093,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=4136387080,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=23034103,osdmap=1078784257}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=14,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=21,osdmap=20} crc {logm=438214483,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=2445670369,monmap=2838206317,osd_metadata=2856640748,osd_pg_creating=1527314401,osd_snap=1480789720,osdmap=1423710363}) 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20,osdmap=18} crc {logm=1548998764,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=4136387080,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=3311035557,osdmap=3898022189}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=22,osdmap=16} crc {logm=3523758500,mdsmap=1531151339,mgr=2295072758,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=1492070130,monmap=2838206317,osd_metadata=134110247,osd_pg_creating=1527314401,osd_snap=23034103,osdmap=4163092649}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=16,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=22,osdmap=16} crc {logm=1756848471,mdsmap=2147943554,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=362820133,monmap=2838206317,osd_metadata=1958636322,osd_pg_creating=1527314401,osd_snap=3786836902,osdmap=156090071}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.658+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.659+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.659+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=16,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=16} crc {logm=438214484,mdsmap=1531151339,mgr=3058068630,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=391092224,osdmap=4163092649}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.659+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.659+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=16,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=21,osdmap=18} crc {logm=438214483,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2670705015,mon_config_key=644647321,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=1527314401,osd_snap=1309096152,osdmap=908780651}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.659+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=15,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=6,osd_snap=19,osdmap=19} crc {logm=1172664666,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1893328424,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_snap=600984435,osdmap=1050085076}) 2024-09-16T15:03:16.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1817334735}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=43281274}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2723275088}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2517199499}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1164098187}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3970706901}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2093097029}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1031242188}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2130001798}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=64} crc {osdmap=2041287513}) 2024-09-16T15:03:16.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=63} crc {osdmap=2305528160}) 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=62} crc {osdmap=672523567}) 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=65} crc {osdmap=368268364}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=64} crc {osdmap=2992339736}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=63} crc {osdmap=59202207}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=62} crc {osdmap=872538659}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=298277855}) 2024-09-16T15:03:16.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:03:16.661+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=63} crc {osdmap=1579501668}) 2024-09-16T15:03:17.510 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:03:17.511 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:17.606 INFO:tasks.rados.rados.0.smithi162.stdout:579: done (5 left) 2024-09-16T15:03:17.606 INFO:tasks.rados.rados.0.smithi162.stdout:580: copy_from oid 49 from oid 39 current snap is 46 2024-09-16T15:03:17.606 INFO:tasks.rados.rados.0.smithi162.stdout:573: finishing write tid 3 to smithi16234619-2 2024-09-16T15:03:17.606 INFO:tasks.rados.rados.0.smithi162.stdout:573: finishing write tid 4 to smithi16234619-2 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: finishing write tid 5 to smithi16234619-2 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: oid 2 updating version 0 to 188 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: oid 2 version 188 is already newer than 187 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: oid 2 version 188 is already newer than 186 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: oid 2 updating version 188 to 189 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 189 (ObjNum 234 snap 46 seq_num 234) dirty exists 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:573: left oid 2 (ObjNum 234 snap 46 seq_num 234) 2024-09-16T15:03:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:575: finishing write tid 1 to smithi16234619-19 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:576: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: finishing write tid 2 to smithi16234619-19 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:577: finishing write tid 1 to smithi16234619-12 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:577: finishing write tid 2 to smithi16234619-12 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: finishing write tid 3 to smithi16234619-19 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: finishing write tid 4 to smithi16234619-19 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: finishing write tid 5 to smithi16234619-19 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: oid 19 updating version 0 to 234 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: oid 19 version 234 is already newer than 232 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: oid 19 version 234 is already newer than 233 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:575: oid 19 updating version 234 to 235 2024-09-16T15:03:17.608 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 235 (ObjNum 235 snap 46 seq_num 235) dirty exists 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:575: left oid 19 (ObjNum 235 snap 46 seq_num 235) 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:576: finishing copy_from to smithi16234619-41 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 51 (ObjNum 227 snap 45 seq_num 227) dirty exists 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: finishing write tid 3 to smithi16234619-12 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: finishing write tid 4 to smithi16234619-12 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: finishing write tid 5 to smithi16234619-12 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: oid 12 updating version 0 to 116 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: oid 12 updating version 116 to 117 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: oid 12 version 117 is already newer than 115 2024-09-16T15:03:17.609 INFO:tasks.rados.rados.0.smithi162.stdout:577: oid 12 updating version 117 to 118 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 118 (ObjNum 236 snap 46 seq_num 236) dirty exists 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:577: left oid 12 (ObjNum 236 snap 46 seq_num 236) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:573: done (5 left) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:575: done (4 left) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:576: done (3 left) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:577: done (2 left) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:581: rollback oid 31 current snap is 46 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:581: done (2 left) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:582: read oid 7 snap -1 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:582: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:583: read oid 20 snap -1 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:583: expect (ObjNum 219 snap 44 seq_num 219) 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:584: read oid 48 snap -1 2024-09-16T15:03:17.610 INFO:tasks.rados.rados.0.smithi162.stdout:584: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:17.611 INFO:tasks.rados.rados.0.smithi162.stdout:585: read oid 19 snap -1 2024-09-16T15:03:17.611 INFO:tasks.rados.rados.0.smithi162.stdout:585: expect (ObjNum 235 snap 46 seq_num 235) 2024-09-16T15:03:17.611 INFO:tasks.rados.rados.0.smithi162.stdout:586: write oid 28 current snap is 46 2024-09-16T15:03:17.611 INFO:tasks.rados.rados.0.smithi162.stdout:586: seq_num 238 ranges {400066=716205,1860832=139234} 2024-09-16T15:03:17.612 INFO:tasks.rados.rados.0.smithi162.stdout:586: writing smithi16234619-28 from 400066 to 1116271 tid 1 2024-09-16T15:03:17.617 INFO:tasks.rados.rados.0.smithi162.stdout:586: writing smithi16234619-28 from 1860832 to 2000066 tid 2 2024-09-16T15:03:17.617 INFO:tasks.rados.rados.0.smithi162.stdout:587: snap_create 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 1 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 2 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 3 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 4 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 5 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: finishing write tid 6 to smithi16234619-24 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: oid 24 updating version 0 to 191 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: oid 24 updating version 191 to 193 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: oid 24 version 193 is already newer than 192 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: oid 24 version 193 is already newer than 190 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: oid 24 updating version 193 to 194 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 194 (ObjNum 237 snap 46 seq_num 237) dirty exists 2024-09-16T15:03:17.618 INFO:tasks.rados.rados.0.smithi162.stdout:578: left oid 24 (ObjNum 237 snap 46 seq_num 237) 2024-09-16T15:03:17.619 INFO:tasks.rados.rados.0.smithi162.stdout:584: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:03:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.635+0000 7ff4d0e3a640 10 monclient: get_monmap_and_config 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.635+0000 7ff4d0e3a640 10 monclient: build_initial_monmap 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.635+0000 7ff4d0e3a640 10 monclient: monmap: 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:17.636512+0000 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:17.636512+0000 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:17.638 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.636+0000 7ff4d0e3a640 1 Processor -- start 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.636+0000 7ff4d0e3a640 1 -- start start 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.636+0000 7ff4d0e3a640 10 monclient: init 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient: _add_conns ranks=[6,2,4,0,9,8,3,1,10,7,5] 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff4cc0a1d30 0x7ff4cc06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-d con 0x7ff4cc0a1d30 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 0x7ff4cc0a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-b con 0x7ff4cc070e90 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-c con 0x7ff4cc0a2650 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0a3880 con 0x7ff4cc070e90 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070540 con 0x7ff4cc0a2650 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0a3a80 con 0x7ff4cc0a1d30 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4d0e3a640 10 monclient(hunting): authenticate will time out at 934.226517s 2024-09-16T15:03:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff4cc0a1d30 0x7ff4cc06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54170/0 (socket says 172.21.15.134:54170) 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): get_auth_request con 0x7ff4cc0a1d30 auth_method 0 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 1 -- 172.21.15.134:0/4043393064 learned_addr learned my addr 172.21.15.134:0/4043393064 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): get_auth_request con 0x7ff4cc0a2650 auth_method 0 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 0x7ff4cc0a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): get_auth_request con 0x7ff4cc070e90 auth_method 0 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.637+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_done global_id 4540 payload 274 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 -- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff4cc0a1d30 msgr2=0x7ff4cc06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff4cc0a1d30 0x7ff4cc06f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 -- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 msgr2=0x7ff4cc0a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 0x7ff4cc0a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 monclient: found mon.noname-c 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:17.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4cad9d640 1 -- 172.21.15.134:0/4043393064 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc073250 con 0x7ff4cc0a2650 2024-09-16T15:03:17.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.638+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 0x7ff4cc0a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:17.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4cad9d640 1 -- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 msgr2=0x7ff4cc134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4cad9d640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009e80 tx=0x7ff4c402a5d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c7155479f66835f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 1 -- 172.21.15.134:0/4043393064 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c4007860 con 0x7ff4cc0a2650 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: dump: 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.653 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: _finish_auth 0 2024-09-16T15:03:17.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: _check_auth_tickets 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 1 -- 172.21.15.134:0/4043393064 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c402d050 con 0x7ff4cc0a2650 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4d0e3a640 5 monclient: authenticate success, global_id 4540 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4d0e3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 1 -- 172.21.15.134:0/4043393064 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c4007e50 con 0x7ff4cc0a2650 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:17.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4c959a640 10 monclient: dump: 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:17.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4d0e3a640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.641+0000 7ff4d0e3a640 4 set_mon_vals no callback set 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 10 monclient: shutdown 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 msgr2=0x7ff4cc134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009e80 tx=0x7ff4c402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/4043393064 shutdown_connections 2024-09-16T15:03:17.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc0a2650 0x7ff4cc134b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4cc070e90 0x7ff4cc0a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/4043393064 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff4cc0a1d30 0x7ff4cc06f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/4043393064 >> 172.21.15.134:0/4043393064 conn(0x7ff4cc09abf0 msgr2=0x7ff4cc0a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/4043393064 shutdown_connections 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.642+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/4043393064 wait complete. 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.643+0000 7ff4d0e3a640 10 monclient: build_initial_monmap 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.643+0000 7ff4d0e3a640 10 monclient: monmap: 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.643+0000 7ff4d0e3a640 1 Processor -- start 2024-09-16T15:03:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.643+0000 7ff4d0e3a640 1 -- start start 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.643+0000 7ff4d0e3a640 10 monclient: init 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient: _add_conns ranks=[4,1,8,6,7,0,5,2,3] 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-c con 0x7ff4cc070e90 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 0x7ff4cc11f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-f con 0x7ff4cc0a1d30 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54178/0 (socket says 172.21.15.134:54178) 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff4cc0a2650 0x7ff4cc11fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): picked mon.noname-e con 0x7ff4cc0a2650 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:17.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 -- 172.21.15.134:0/1225012082 learned_addr learned my addr 172.21.15.134:0/1225012082 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc123070 con 0x7ff4cc0a1d30 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1c7d40 con 0x7ff4cc070e90 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc135c00 con 0x7ff4cc0a2650 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4d0e3a640 10 monclient(hunting): authenticate will time out at 934.233600s 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): get_auth_request con 0x7ff4cc070e90 auth_method 0 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff4cc0a2650 0x7ff4cc11fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): get_auth_request con 0x7ff4cc0a2650 auth_method 0 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 0x7ff4cc11f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): get_auth_request con 0x7ff4cc0a1d30 auth_method 0 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:17.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4cad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4c9d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient(hunting): handle_auth_done global_id 4549 payload 824 2024-09-16T15:03:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 -- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff4cc0a2650 msgr2=0x7ff4cc11fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff4cc0a2650 0x7ff4cc11fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 -- 172.21.15.134:0/1225012082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 msgr2=0x7ff4cc11f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 0x7ff4cc11f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient: _finish_hunting 0 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 monclient: found mon.noname-c 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.644+0000 7ff4ca59c640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009400 con 0x7ff4cc070e90 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.645+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 0x7ff4cc11f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4ca59c640 1 -- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 msgr2=0x7ff4cc134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4ca59c640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000e9b0 tx=0x7ff4c000ee70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:17.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c0009670 con 0x7ff4cc070e90 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: dump: 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c000cbb0 con 0x7ff4cc070e90 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 5 monclient: authenticate success, global_id 4549 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 10 monclient: _renew_subs 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c4009b30 con 0x7ff4cc070e90 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4c0013d50 con 0x7ff4cc070e90 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4baffd640 10 monclient: dump: 2024-09-16T15:03:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4c8d99640 4 set_mon_vals no callback set 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 10 monclient: _renew_subs 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.647+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc121180 con 0x7ff4cc070e90 2024-09-16T15:03:17.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff4c000cd20 con 0x7ff4cc070e90 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4d0e3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4d0e3a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4d0e3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4baffd640 1 --2- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 0x7ff4ac03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc06cab0 con 0x7ff4cc070e90 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (secure 0 0 0) 0x7ff4c004a3f0 con 0x7ff4cc070e90 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 0x7ff4ac03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.648+0000 7ff4c9d9b640 10 monclient: get_auth_request con 0x7ff4ac038880 auth_method 0 2024-09-16T15:03:17.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.651+0000 7ff4c9d9b640 1 --2- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 0x7ff4ac03ad40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0093d0 tx=0x7ff4bc008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:17.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.653+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff4c0017020 con 0x7ff4cc070e90 2024-09-16T15:03:17.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.653+0000 7ff4baffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:17.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.653+0000 7ff4baffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:17.868 INFO:tasks.rados.rados.0.smithi162.stdout:583: expect (ObjNum 219 snap 44 seq_num 219) 2024-09-16T15:03:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.882+0000 7ff4d0e3a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.882+0000 7ff4d0e3a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.882+0000 7ff4d0e3a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.882+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff4cc0a6e10 con 0x7ff4cc070e90 2024-09-16T15:03:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.883+0000 7ff4baffd640 1 -- 172.21.15.134:0/1225012082 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7ff4c003c080 con 0x7ff4cc070e90 2024-09-16T15:03:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.883+0000 7ff4baffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.883+0000 7ff4baffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:17.885 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 msgr2=0x7ff4ac03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 0x7ff4ac03ad40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0093d0 tx=0x7ff4bc008040 comp rx=0 tx=0).stop 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 10 monclient: shutdown 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 msgr2=0x7ff4cc134df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000e9b0 tx=0x7ff4c000ee70 comp rx=0 tx=0).stop 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 shutdown_connections 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4ac038880 0x7ff4ac03ad40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff4cc0a2650 0x7ff4cc11fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff4cc0a1d30 0x7ff4cc11f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 --2- 172.21.15.134:0/1225012082 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff4cc070e90 0x7ff4cc134df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:17.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 >> 172.21.15.134:0/1225012082 conn(0x7ff4cc09abf0 msgr2=0x7ff4cc0a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:17.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 shutdown_connections 2024-09-16T15:03:17.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.885+0000 7ff4d0e3a640 1 -- 172.21.15.134:0/1225012082 wait complete. 2024-09-16T15:03:17.897 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:03:17.897 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:03:17.897 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:03:17.898 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:03:17.984 INFO:tasks.rados.rados.0.smithi162.stdout:582: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:03:17.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 10 monclient: get_monmap_and_config 2024-09-16T15:03:17.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 10 monclient: build_initial_monmap 2024-09-16T15:03:17.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 10 monclient: monmap: 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:17.995242+0000 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:17.995242+0000 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:17.996 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:17.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:17.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:17.997 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:17.997 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:17.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:17.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 1 Processor -- start 2024-09-16T15:03:17.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 1 -- start start 2024-09-16T15:03:17.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.994+0000 7f3ac53db640 10 monclient: init 2024-09-16T15:03:17.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:17.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient: _add_conns ranks=[9,0,3,4,5,2,10,6,8,1,7] 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 0x7f3ac0065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-a con 0x7f3ac0069e50 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3ac0066530 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 0x7f3ac013ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-g con 0x7f3ac0066910 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac0073250 con 0x7f3ac0066530 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac009e4a0 con 0x7f3ac0069e50 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac006a230 con 0x7f3ac0066910 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3ac53db640 10 monclient(hunting): authenticate will time out at 934.584443s 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 0x7f3ac0065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 0x7f3ac0065ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:49322/0 (socket says 172.21.15.134:49322) 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 1 -- 172.21.15.134:0/72850744 learned_addr learned my addr 172.21.15.134:0/72850744 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): get_auth_request con 0x7f3ac0069e50 auth_method 0 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 0x7f3ac013ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): get_auth_request con 0x7f3ac0066910 auth_method 0 2024-09-16T15:03:18.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_done global_id 34203 payload 274 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 1 -- 172.21.15.134:0/72850744 >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 msgr2=0x7f3ac0065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.995+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/72850744 >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 0x7f3ac0065ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 1 -- 172.21.15.134:0/72850744 >> v1:172.21.15.134:6789/0 conn(0x7f3ac0066530 legacy=0x7f3ac013b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 1 monclient: found mon.noname-g 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:03:18.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.996+0000 7f3abf7fe640 1 -- 172.21.15.134:0/72850744 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac009e1c0 con 0x7f3ac0066910 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.998+0000 7f3abf7fe640 1 -- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 msgr2=0x7f3ac013ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.998+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 0x7f3ac013ecd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00097a0 tx=0x7f3ab002a130 comp rx=0 tx=0).ready entity=mon.3 client_cookie=2176097712f74705 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 1 -- 172.21.15.134:0/72850744 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ab002ae50 con 0x7f3ac0066910 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: dump: 2024-09-16T15:03:18.010 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.011 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 1 -- 172.21.15.134:0/72850744 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab00307b0 con 0x7f3ac0066910 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 1 -- 172.21.15.134:0/72850744 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ab0030d90 con 0x7f3ac0066910 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3abdffb640 10 monclient: dump: 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.012 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.013 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 5 monclient: authenticate success, global_id 34203 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 4 set_mon_vals no callback set 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 10 monclient: shutdown 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 -- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 msgr2=0x7f3ac013ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 --2- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 0x7f3ac013ecd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00097a0 tx=0x7f3ab002a130 comp rx=0 tx=0).stop 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 -- 172.21.15.134:0/72850744 shutdown_connections 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 --2- 172.21.15.134:0/72850744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ac0066910 0x7f3ac013ecd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 --2- 172.21.15.134:0/72850744 >> v2:172.21.15.134:3300/0 conn(0x7f3ac0069e50 0x7f3ac0065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 -- 172.21.15.134:0/72850744 >> 172.21.15.134:0/72850744 conn(0x7f3ac00a6400 msgr2=0x7f3ac00a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 -- 172.21.15.134:0/72850744 shutdown_connections 2024-09-16T15:03:18.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:17.999+0000 7f3ac53db640 1 -- 172.21.15.134:0/72850744 wait complete. 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.000+0000 7f3ac53db640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.000+0000 7f3ac53db640 10 monclient: monmap: 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.015 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.000+0000 7f3ac53db640 1 Processor -- start 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 -- start start 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient: init 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient: _add_conns ranks=[8,0,7,5,6,1,4,2,3] 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 0x7f3ac01cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-e con 0x7f3ac0066530 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 0x7f3ac01cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-a con 0x7f3ac0066910 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ac0069e50 0x7f3ac01cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): picked mon.noname-i con 0x7f3ac0069e50 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0092670 con 0x7f3ac0066910 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00a1070 con 0x7f3ac0069e50 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00920b0 con 0x7f3ac0066530 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3ac53db640 10 monclient(hunting): authenticate will time out at 934.590492s 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 0x7f3ac01cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 0x7f3ac01cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 0x7f3ac01cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46168/0 (socket says 172.21.15.134:46168) 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 1 -- 172.21.15.134:0/2821349796 learned_addr learned my addr 172.21.15.134:0/2821349796 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): get_auth_request con 0x7f3ac0066910 auth_method 0 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): get_auth_request con 0x7f3ac0066530 auth_method 0 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ac0069e50 0x7f3ac01cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): get_auth_request con 0x7f3ac0069e50 auth_method 0 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient(hunting): handle_auth_done global_id 4635 payload 824 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 msgr2=0x7f3ac01cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 0x7f3ac01cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ac0069e50 msgr2=0x7f3ac01cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ac0069e50 0x7f3ac01cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 monclient: found mon.noname-a 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abe7fc640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0009400 con 0x7f3ac0066910 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.001+0000 7f3abeffd640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ac0066530 0x7f3ac01cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3abe7fc640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 msgr2=0x7f3ac01cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3abe7fc640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 0x7f3ac01cbb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b3e0 tx=0x7f3aa800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3aa8011cb0 con 0x7f3ac0066910 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: dump: 2024-09-16T15:03:18.020 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.021 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8011e50 con 0x7f3ac0066910 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 5 monclient: authenticate success, global_id 4635 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3aa80107f0 con 0x7f3ac0066910 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ab77fe640 10 monclient: dump: 2024-09-16T15:03:18.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3abd7fa640 4 set_mon_vals no callback set 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 10 monclient: _renew_subs 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac0092390 con 0x7f3ac0066910 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 10 monclient: _renew_subs 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.004+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac00718f0 con 0x7f3ac0066910 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3aa8010380 con 0x7f3ac0066910 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ac53db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ac53db640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 0x7f3ac01bce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0073880 con 0x7f3ac0092870 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 0x7f3ac01bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient: get_auth_request con 0x7f3ac0092870 auth_method 0 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ab77fe640 1 --2- 172.21.15.134:0/2821349796 >> v2:172.21.15.162:6800/980547263 conn(0x7f3aa00388b0 0x7f3aa003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f3aa80497e0 con 0x7f3ac0066910 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 10 monclient(hunting): handle_auth_done global_id 4644 payload 824 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abeffd640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ac0061ae0 con 0x7f3ac0092870 2024-09-16T15:03:18.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/2821349796 >> v2:172.21.15.162:6800/980547263 conn(0x7f3aa00388b0 0x7f3aa003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.005+0000 7f3abf7fe640 10 monclient: get_auth_request con 0x7f3aa00388b0 auth_method 0 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3abeffd640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 msgr2=0x7f3ac01bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3abeffd640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 0x7f3ac01bce10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab80077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ab801c070 con 0x7f3ac0092870 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3ab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3ab77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3ab77fe640 10 monclient: dump: 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.026 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.007+0000 7f3abf7fe640 1 --2- 172.21.15.134:0/2821349796 >> v2:172.21.15.162:6800/980547263 conn(0x7f3aa00388b0 0x7f3aa003ad70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ac006ce50 tx=0x7f3ab002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3ab8016850 con 0x7f3ac0092870 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 msgr2=0x7f3ac01bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 0x7f3ac01bce10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab80077c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 msgr2=0x7f3ac01bce10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.008+0000 7f3ab77fe640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0092870 0x7f3ac01bce10 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab80077c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3ac53db640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3ac53db640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 0x7f3ac01bd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3ac53db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac006b5d0 con 0x7f3ac01bd350 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 0x7f3ac01bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient: get_auth_request con 0x7f3ac01bd350 auth_method 0 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 10 monclient(hunting): handle_auth_done global_id 4653 payload 824 2024-09-16T15:03:18.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.014+0000 7f3abeffd640 1 -- 172.21.15.134:0/2821349796 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3ac007a680 con 0x7f3ac01bd350 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3abeffd640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 msgr2=0x7f3ac01bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3abeffd640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 0x7f3ac01bd760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ab8010040 con 0x7f3ac01bd350 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 10 monclient: dump: 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.031 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f3ab8021780 con 0x7f3ac01bd350 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 msgr2=0x7f3ac01bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 0x7f3ac01bd760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab8004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 msgr2=0x7f3ac01bd760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.016+0000 7f3ab77fe640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac01bd350 0x7f3ac01bd760 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800ed60 tx=0x7f3ab8004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.033 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.034 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.035 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.036 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.037 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.038 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.040 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.042 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.043 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.045 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.046 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.047 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.049 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.052 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:18.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.056 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 >> v2:172.21.15.162:6800/980547263 conn(0x7f3aa00388b0 msgr2=0x7f3aa003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 --2- 172.21.15.134:0/2821349796 >> v2:172.21.15.162:6800/980547263 conn(0x7f3aa00388b0 0x7f3aa003ad70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3ac006ce50 tx=0x7f3ab002e000 comp rx=0 tx=0).stop 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 10 monclient: shutdown 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 msgr2=0x7f3ac01cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 --2- 172.21.15.134:0/2821349796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ac0066910 0x7f3ac01cbb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b3e0 tx=0x7f3aa800b8a0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3abf7fe640 1 -- 172.21.15.134:0/2821349796 reap_dead start 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3abf7fe640 1 -- 172.21.15.134:0/2821349796 reap_dead start 2024-09-16T15:03:18.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 shutdown_connections 2024-09-16T15:03:18.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 >> 172.21.15.134:0/2821349796 conn(0x7f3ac00a6400 msgr2=0x7f3ac009c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 shutdown_connections 2024-09-16T15:03:18.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.017+0000 7f3ac53db640 1 -- 172.21.15.134:0/2821349796 wait complete. 2024-09-16T15:03:18.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:03:18.059 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.156+0000 7fb04dfd4640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.156+0000 7fb04dfd4640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.156+0000 7fb04dfd4640 10 monclient: monmap: 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.157557+0000 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.157557+0000 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.158 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.159 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 Processor -- start 2024-09-16T15:03:18.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 -- start start 2024-09-16T15:03:18.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient: init 2024-09-16T15:03:18.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient: _add_conns ranks=[6,0,8,10,5,9,2,7,4,3,1] 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 0x7fb04809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-d con 0x7fb048136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb048137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 0x7fb04813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-e con 0x7fb04813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb048073250 con 0x7fb048137220 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480a56e0 con 0x7fb048136e40 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb04809abc0 con 0x7fb04813c370 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb04dfd4640 10 monclient(hunting): authenticate will time out at 934.746904s 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.157+0000 7fb0477fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 0x7fb04809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb046ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb048137220 0x7fb048137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:51996/0 (socket says 172.21.15.134:51996) 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb046ffd640 1 -- 172.21.15.134:0/4092499227 learned_addr learned my addr 172.21.15.134:0/4092499227 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 0x7fb04813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): get_auth_request con 0x7fb048136e40 auth_method 0 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): get_auth_request con 0x7fb04813c370 auth_method 0 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient(hunting): handle_auth_done global_id 4715 payload 274 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 -- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 msgr2=0x7fb04809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 0x7fb04809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 -- 172.21.15.134:0/4092499227 >> v1:172.21.15.134:6789/0 conn(0x7fb048137220 legacy=0x7fb048137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 monclient: found mon.noname-e 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb047fff640 1 -- 172.21.15.134:0/4092499227 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04809ba50 con 0x7fb04813c370 2024-09-16T15:03:18.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.158+0000 7fb0477fe640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 0x7fb04809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb047fff640 1 -- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 msgr2=0x7fb04813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb047fff640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 0x7fb04813e760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb040009dc0 tx=0x7fb04002a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=f9454104fe4e2c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 1 -- 172.21.15.134:0/4092499227 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb040007bc0 con 0x7fb04813c370 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: dump: 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.175 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 5 monclient: authenticate success, global_id 4715 2024-09-16T15:03:18.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 1 -- 172.21.15.134:0/4092499227 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04002d050 con 0x7fb04813c370 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 4 set_mon_vals no callback set 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 1 -- 172.21.15.134:0/4092499227 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb040007d40 con 0x7fb04813c370 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb0467fc640 10 monclient: dump: 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.177 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 10 monclient: shutdown 2024-09-16T15:03:18.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 -- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 msgr2=0x7fb04813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 0x7fb04813e760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb040009dc0 tx=0x7fb04002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 -- 172.21.15.134:0/4092499227 shutdown_connections 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb04813c370 0x7fb04813e760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/4092499227 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb048136e40 0x7fb04809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 -- 172.21.15.134:0/4092499227 >> 172.21.15.134:0/4092499227 conn(0x7fb0480a6110 msgr2=0x7fb0480a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 -- 172.21.15.134:0/4092499227 shutdown_connections 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.161+0000 7fb04dfd4640 1 -- 172.21.15.134:0/4092499227 wait complete. 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.162+0000 7fb04dfd4640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.162+0000 7fb04dfd4640 10 monclient: monmap: 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.179 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.180 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.162+0000 7fb04dfd4640 1 Processor -- start 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.162+0000 7fb04dfd4640 1 -- start start 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.162+0000 7fb04dfd4640 10 monclient: init 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient: _add_conns ranks=[1,5,7,3,8,6,0,4,2] 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 0x7fb04811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-f con 0x7fb048136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 0x7fb0481201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-h con 0x7fb048137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 0x7fb0481206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): picked mon.noname-i con 0x7fb04813c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:18.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048136540 con 0x7fb048136e40 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480a74a0 con 0x7fb048137220 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb04813b4a0 con 0x7fb04813c370 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb04dfd4640 10 monclient(hunting): authenticate will time out at 934.752283s 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 0x7fb04811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 0x7fb0481201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 0x7fb0481206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 0x7fb04811fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:59706/0 (socket says 172.21.15.134:59706) 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 0x7fb0481201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:32926/0 (socket says 172.21.15.134:32926) 2024-09-16T15:03:18.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 1 -- 172.21.15.134:0/1478759143 learned_addr learned my addr 172.21.15.134:0/1478759143 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): get_auth_request con 0x7fb04813c370 auth_method 0 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): get_auth_request con 0x7fb048136e40 auth_method 0 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): get_auth_request con 0x7fb048137220 auth_method 0 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient(hunting): handle_auth_done global_id 4507 payload 824 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 msgr2=0x7fb0481201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 0x7fb0481201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 msgr2=0x7fb04811fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 0x7fb04811fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 monclient: found mon.noname-i 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb047fff640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038009400 con 0x7fb04813c370 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb0477fe640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb048136e40 0x7fb04811fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.163+0000 7fb046ffd640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb048137220 0x7fb0481201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb047fff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 msgr2=0x7fb0481206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb047fff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 0x7fb0481206e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb040031000 tx=0x7fb040004a40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb040007ea0 con 0x7fb04813c370 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: dump: 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.185 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400044b0 con 0x7fb04813c370 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 5 monclient: authenticate success, global_id 4507 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb040004620 con 0x7fb04813c370 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 10 monclient: _renew_subs 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb040009a70 con 0x7fb04813c370 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb02ffff640 10 monclient: dump: 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.187 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb045ffb640 4 set_mon_vals no callback set 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 10 monclient: _renew_subs 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.166+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04813aa20 con 0x7fb04813c370 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb04dfd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 0x7fb048135880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb048071b20 con 0x7fb0481354a0 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb040004780 con 0x7fb04813c370 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 0x7fb048135880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient: get_auth_request con 0x7fb0481354a0 auth_method 0 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb02ffff640 1 --2- 172.21.15.134:0/1478759143 >> v2:172.21.15.162:6800/980547263 conn(0x7fb034038880 0x7fb03403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fb04006aff0 con 0x7fb04813c370 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 10 monclient(hunting): handle_auth_done global_id 34193 payload 824 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb046ffd640 1 --2- 172.21.15.134:0/1478759143 >> v2:172.21.15.162:6800/980547263 conn(0x7fb034038880 0x7fb03403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb0477fe640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb048061ae0 con 0x7fb0481354a0 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.167+0000 7fb046ffd640 10 monclient: get_auth_request con 0x7fb034038880 auth_method 0 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb0477fe640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 msgr2=0x7fb048135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb0477fe640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 0x7fb048135880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb03800f4a0 tx=0x7fb03800f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0380044a0 con 0x7fb0481354a0 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb02ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb02ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb02ffff640 10 monclient: dump: 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.191 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.169+0000 7fb046ffd640 1 --2- 172.21.15.134:0/1478759143 >> v2:172.21.15.162:6800/980547263 conn(0x7fb034038880 0x7fb03403ad40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007c90 tx=0x7fb03c007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb038004030 con 0x7fb0481354a0 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 msgr2=0x7fb048135880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 0x7fb048135880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb03800f4a0 tx=0x7fb03800f960 comp rx=0 tx=0).stop 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 msgr2=0x7fb048135880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.170+0000 7fb02ffff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0481354a0 0x7fb048135880 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb03800f4a0 tx=0x7fb03800f960 comp rx=0 tx=0).stop 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb04dfd4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 0x7fb04809d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb04dfd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04807a680 con 0x7fb048135dc0 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 0x7fb04809d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient: get_auth_request con 0x7fb048135dc0 auth_method 0 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 10 monclient(hunting): handle_auth_done global_id 34202 payload 824 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.176+0000 7fb0477fe640 1 -- 172.21.15.134:0/1478759143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb0480a53c0 con 0x7fb048135dc0 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb0477fe640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 msgr2=0x7fb04809d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb0477fe640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 0x7fb04809d4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb03800f4a0 tx=0x7fb0380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0380273f0 con 0x7fb048135dc0 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb02ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb02ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.178+0000 7fb02ffff640 10 monclient: dump: 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.195 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fb038023040 con 0x7fb048135dc0 2024-09-16T15:03:18.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 msgr2=0x7fb04809d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 0x7fb04809d4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb03800f4a0 tx=0x7fb0380046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 msgr2=0x7fb04809d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb02ffff640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb048135dc0 0x7fb04809d4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:18.197 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.198 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.199 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.200 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.201 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.202 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.203 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.207 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.214 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.216 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.217 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.218 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.219 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 >> v2:172.21.15.162:6800/980547263 conn(0x7fb034038880 msgr2=0x7fb03403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/1478759143 >> v2:172.21.15.162:6800/980547263 conn(0x7fb034038880 0x7fb03403ad40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007c90 tx=0x7fb03c007be0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 10 monclient: shutdown 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 msgr2=0x7fb0481206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb047fff640 1 -- 172.21.15.134:0/1478759143 reap_dead start 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 0x7fb0481206e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb040031000 tx=0x7fb040004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 shutdown_connections 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 --2- 172.21.15.134:0/1478759143 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb04813c370 0x7fb0481206e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 >> 172.21.15.134:0/1478759143 conn(0x7fb0480a6110 msgr2=0x7fb0480a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 shutdown_connections 2024-09-16T15:03:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.179+0000 7fb04dfd4640 1 -- 172.21.15.134:0/1478759143 wait complete. 2024-09-16T15:03:18.222 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.222 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:03:18.222 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:03:18.231 INFO:tasks.rados.rados.0.smithi162.stdout:585: expect (ObjNum 235 snap 46 seq_num 235) 2024-09-16T15:03:18.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 10 monclient: monmap: 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.320106+0000 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.320106+0000 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.353 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 1 Processor -- start 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 1 -- start start 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.319+0000 7f6ebe480640 10 monclient: init 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient: _add_conns ranks=[8,4,10,0,2,3,6,9,1,7,5] 2024-09-16T15:03:18.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-e con 0x7f6eb80a29a0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-c con 0x7f6eb809f220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 0x7f6eb8089040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-f con 0x7f6eb809d300 addr v2:172.21.15.162:3300/0 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80a5240 con 0x7f6eb809d300 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb809e940 con 0x7f6eb809f220 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a4180 con 0x7f6eb80a29a0 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebe480640 10 monclient(hunting): authenticate will time out at 934.909347s 2024-09-16T15:03:18.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6eb77fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6eb7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6eb77fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54184/0 (socket says 172.21.15.134:54184) 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6eb7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46190/0 (socket says 172.21.15.134:46190) 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 -- 172.21.15.134:0/1067173717 learned_addr learned my addr 172.21.15.134:0/1067173717 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): get_auth_request con 0x7f6eb809f220 auth_method 0 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.320+0000 7f6ebca1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 0x7f6eb8089040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request con 0x7f6eb80a29a0 auth_method 0 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): get_auth_request con 0x7f6eb809d300 auth_method 0 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_done global_id 4558 payload 274 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 -- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 msgr2=0x7f6eb809c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 -- 172.21.15.134:0/1067173717 >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 msgr2=0x7f6eb8089040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/1067173717 >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 0x7f6eb8089040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 monclient: found mon.noname-c 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb77fe640 1 -- 172.21.15.134:0/1067173717 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8073250 con 0x7f6eb809f220 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.351+0000 7f6ebca1d640 1 --2- 172.21.15.134:0/1067173717 >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 0x7f6eb8089040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb77fe640 1 -- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 msgr2=0x7f6eb809cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b9a0 tx=0x7f6eac02a750 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7851a2a48dd5781 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 1 -- 172.21.15.134:0/1067173717 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac030b00 con 0x7f6eb809f220 2024-09-16T15:03:18.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: dump: 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.366 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6ebe480640 5 monclient: authenticate success, global_id 4558 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6ebe480640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 1 -- 172.21.15.134:0/1067173717 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac02d030 con 0x7f6eb809f220 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6ebe480640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6ebe480640 4 set_mon_vals no callback set 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 1 -- 172.21.15.134:0/1067173717 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac030c80 con 0x7f6eb809f220 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.353+0000 7f6eb6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6eb6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6eb6ffd640 10 monclient: dump: 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.368 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 10 monclient: shutdown 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 -- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 msgr2=0x7f6eb809cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b9a0 tx=0x7f6eac02a750 comp rx=0 tx=0).stop 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 -- 172.21.15.134:0/1067173717 shutdown_connections 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 --2- 172.21.15.134:0/1067173717 >> v2:172.21.15.162:3300/0 conn(0x7f6eb809d300 0x7f6eb8089040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb809f220 0x7f6eb809cdc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 --2- 172.21.15.134:0/1067173717 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb80a29a0 0x7f6eb809c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 -- 172.21.15.134:0/1067173717 >> 172.21.15.134:0/1067173717 conn(0x7f6eb80a1e80 msgr2=0x7f6eb80a2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 -- 172.21.15.134:0/1067173717 shutdown_connections 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.354+0000 7f6ebe480640 1 -- 172.21.15.134:0/1067173717 wait complete. 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.355+0000 7f6ebe480640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.355+0000 7f6ebe480640 10 monclient: monmap: 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.370 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.355+0000 7f6ebe480640 1 Processor -- start 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.355+0000 7f6ebe480640 1 -- start start 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.355+0000 7f6ebe480640 10 monclient: init 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient: _add_conns ranks=[7,8,3,4,2,1,6,0,5] 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6eb809d300 0x7f6eb81d8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-i con 0x7f6eb809d300 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-e con 0x7f6eb809f220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6eb80a29a0 0x7f6eb806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): picked mon.noname-g con 0x7f6eb80a29a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8090ad0 con 0x7f6eb80a29a0 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a1280 con 0x7f6eb809d300 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a17d0 con 0x7f6eb809f220 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebe480640 10 monclient(hunting): authenticate will time out at 934.945153s 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46206/0 (socket says 172.21.15.134:46206) 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 -- 172.21.15.134:0/3732808216 learned_addr learned my addr 172.21.15.134:0/3732808216 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): get_auth_request con 0x7f6eb809f220 auth_method 0 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6eb809d300 0x7f6eb81d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request con 0x7f6eb809d300 auth_method 0 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6eb80a29a0 0x7f6eb806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): get_auth_request con 0x7f6eb80a29a0 auth_method 0 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6ebca1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient(hunting): handle_auth_done global_id 4733 payload 824 2024-09-16T15:03:18.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6eb809d300 msgr2=0x7f6eb81d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6eb809d300 0x7f6eb81d8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6eb80a29a0 msgr2=0x7f6eb806ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6eb80a29a0 0x7f6eb806ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 monclient: found mon.noname-e 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.356+0000 7f6eb77fe640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb0009a70 con 0x7f6eb809f220 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.358+0000 7f6eb77fe640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 msgr2=0x7f6eb81d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.358+0000 7f6eb77fe640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00bac0 tx=0x7f6eac0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac004500 con 0x7f6eb809f220 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: dump: 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac003a80 con 0x7f6eb809f220 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 5 monclient: authenticate success, global_id 4733 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 monclient: _renew_subs 2024-09-16T15:03:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea800d8a0 con 0x7f6eb809f220 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eac003bf0 con 0x7f6eb809f220 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb4ff9640 10 monclient: dump: 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 monclient: _renew_subs 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6eb67fc640 4 set_mon_vals no callback set 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac00b5d0 con 0x7f6eb809f220 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 0x7f6eb806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.359+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8072410 con 0x7f6eb8090c10 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6eac03a040 con 0x7f6eb809f220 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 0x7f6eb806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient: get_auth_request con 0x7f6eb8090c10 auth_method 0 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb4ff9640 1 --2- 172.21.15.134:0/3732808216 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ea4038880 0x7f6ea403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_done global_id 4567 payload 824 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f6eac06ad50 con 0x7f6eb809f220 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6eb7fff640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6eb8061ae0 con 0x7f6eb8090c10 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6ebca1d640 1 --2- 172.21.15.134:0/3732808216 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ea4038880 0x7f6ea403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.360+0000 7f6ebca1d640 10 monclient: get_auth_request con 0x7f6ea4038880 auth_method 0 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb7fff640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 msgr2=0x7f6eb806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 0x7f6eb806f9b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800d800 tx=0x7f6ea8013580 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea8012040 con 0x7f6eb8090c10 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.362+0000 7f6eb4ff9640 10 monclient: dump: 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.382 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6ebca1d640 1 --2- 172.21.15.134:0/3732808216 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ea4038880 0x7f6ea403ad40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80914a0 tx=0x7f6eb000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6ea8013040 con 0x7f6eb8090c10 2024-09-16T15:03:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 msgr2=0x7f6eb806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 0x7f6eb806f9b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800d800 tx=0x7f6ea8013580 comp rx=0 tx=0).stop 2024-09-16T15:03:18.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 msgr2=0x7f6eb806f9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.363+0000 7f6eb4ff9640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb8090c10 0x7f6eb806f9b0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800d800 tx=0x7f6ea8013580 comp rx=0 tx=0).stop 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6ebe480640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6ebe480640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 0x7f6eb802b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6ebe480640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb806a4c0 con 0x7f6eb802b260 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 0x7f6eb802b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient: get_auth_request con 0x7f6eb802b260 auth_method 0 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 10 monclient(hunting): handle_auth_done global_id 4576 payload 824 2024-09-16T15:03:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.369+0000 7f6eb7fff640 1 -- 172.21.15.134:0/3732808216 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6eb807a680 con 0x7f6eb802b260 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb7fff640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 msgr2=0x7f6eb802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb7fff640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 0x7f6eb802b670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800f010 tx=0x7f6ea80046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ea8027070 con 0x7f6eb802b260 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.371+0000 7f6eb4ff9640 10 monclient: dump: 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f6ea8013e40 con 0x7f6eb802b260 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 msgr2=0x7f6eb802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 0x7f6eb802b670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800f010 tx=0x7f6ea80046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 msgr2=0x7f6eb802b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6eb4ff9640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6eb802b260 0x7f6eb802b670 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800f010 tx=0x7f6ea80046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.389 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.391 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.392 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.393 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.394 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.395 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.396 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.397 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.400 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.402 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.403 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.405 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.410 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: finishing write tid 1 to smithi16234619-28 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: finishing write tid 2 to smithi16234619-28 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: finishing write tid 3 to smithi16234619-28 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: finishing write tid 4 to smithi16234619-28 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: oid 28 updating version 0 to 213 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: oid 28 updating version 213 to 214 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: oid 28 updating version 214 to 215 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 215 (ObjNum 238 snap 46 seq_num 238) dirty exists 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:586: left oid 28 (ObjNum 238 snap 46 seq_num 238) 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:580: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:580: finishing copy_from to smithi16234619-49 2024-09-16T15:03:18.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 217 (ObjNum 205 snap 44 seq_num 205) dirty exists 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ea4038880 msgr2=0x7f6ea403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 --2- 172.21.15.134:0/3732808216 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ea4038880 0x7f6ea403ad40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80914a0 tx=0x7f6eb000b040 comp rx=0 tx=0).stop 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 10 monclient: shutdown 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 msgr2=0x7f6eb81d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00bac0 tx=0x7f6eac0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebca1d640 1 -- 172.21.15.134:0/3732808216 reap_dead start 2024-09-16T15:03:18.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 shutdown_connections 2024-09-16T15:03:18.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 --2- 172.21.15.134:0/3732808216 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6eb809f220 0x7f6eb81d89b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 >> 172.21.15.134:0/3732808216 conn(0x7f6eb80a1e80 msgr2=0x7f6eb808e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 shutdown_connections 2024-09-16T15:03:18.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.372+0000 7f6ebe480640 1 -- 172.21.15.134:0/3732808216 wait complete. 2024-09-16T15:03:18.414 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.414 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:03:18.414 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:03:18.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.511+0000 7fd837de2640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.511+0000 7fd837de2640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.511+0000 7fd837de2640 10 monclient: monmap: 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.512239+0000 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.512239+0000 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.513 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.514 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.511+0000 7fd837de2640 1 Processor -- start 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 -- start start 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient: init 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient: _add_conns ranks=[6,5,4,1,7,2,10,9,0,8,3] 2024-09-16T15:03:18.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 0x7fd83009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-d con 0x7fd830136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd830137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-h con 0x7fd830137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-c con 0x7fd830135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83009abc0 con 0x7fd830135dc0 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83009ba50 con 0x7fd830137220 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300a53c0 con 0x7fd830136e40 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd837de2640 10 monclient(hunting): authenticate will time out at 935.101596s 2024-09-16T15:03:18.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 0x7fd83009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54220/0 (socket says 172.21.15.134:54220) 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 1 -- 172.21.15.134:0/2739922378 learned_addr learned my addr 172.21.15.134:0/2739922378 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): get_auth_request con 0x7fd830135dc0 auth_method 0 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): get_auth_request con 0x7fd830136e40 auth_method 0 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd830137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): get_auth_request con 0x7fd830137220 auth_method 0 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.512+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 10 monclient(hunting): handle_auth_done global_id 34179 payload 274 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 -- 172.21.15.134:0/2739922378 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 msgr2=0x7fd830137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd830137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 -- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 msgr2=0x7fd830136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 monclient: found mon.noname-d 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd835b7e640 1 -- 172.21.15.134:0/2739922378 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830073250 con 0x7fd830136e40 2024-09-16T15:03:18.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.513+0000 7fd83637f640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd835b7e640 1 -- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 msgr2=0x7fd83009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd835b7e640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 0x7fd83009c150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097d0 tx=0x7fd82c029fb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3865ed5475e99513 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 1 -- 172.21.15.134:0/2739922378 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82c02ad10 con 0x7fd830136e40 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: dump: 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 1 -- 172.21.15.134:0/2739922378 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c02ae90 con 0x7fd830136e40 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd837de2640 5 monclient: authenticate success, global_id 34179 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd837de2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd837de2640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 1 -- 172.21.15.134:0/2739922378 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82c030c20 con 0x7fd830136e40 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd834b7c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.515+0000 7fd837de2640 4 set_mon_vals no callback set 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd834b7c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd834b7c640 10 monclient: dump: 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.530 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 10 monclient: shutdown 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 -- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 msgr2=0x7fd83009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 0x7fd83009c150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097d0 tx=0x7fd82c029fb0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 -- 172.21.15.134:0/2739922378 shutdown_connections 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd830135dc0 0x7fd830136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd830137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 --2- 172.21.15.134:0/2739922378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd830136e40 0x7fd83009c150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 -- 172.21.15.134:0/2739922378 >> 172.21.15.134:0/2739922378 conn(0x7fd8300a6110 msgr2=0x7fd8300a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 -- 172.21.15.134:0/2739922378 shutdown_connections 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.516+0000 7fd837de2640 1 -- 172.21.15.134:0/2739922378 wait complete. 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient: monmap: 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 Processor -- start 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 -- start start 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient: init 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient: _add_conns ranks=[8,1,5,6,7,3,2,4,0] 2024-09-16T15:03:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 0x7fd8301c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-e con 0x7fd830135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd830136e40 0x7fd830135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-f con 0x7fd830136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd8301cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient(hunting): picked mon.noname-h con 0x7fd830137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301363b0 con 0x7fd830136e40 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.517+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd837de2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301233c0 con 0x7fd830137220 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd837de2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd830124660 con 0x7fd830135dc0 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd837de2640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd837de2640 10 monclient(hunting): authenticate will time out at 935.107018s 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 0x7fd8301c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 0x7fd8301c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46208/0 (socket says 172.21.15.134:46208) 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 -- 172.21.15.134:0/408884421 learned_addr learned my addr 172.21.15.134:0/408884421 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): get_auth_request con 0x7fd830135dc0 auth_method 0 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd830136e40 0x7fd830135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd8301cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): get_auth_request con 0x7fd830136e40 auth_method 0 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): get_auth_request con 0x7fd830137220 auth_method 0 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd83637f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient(hunting): handle_auth_done global_id 4742 payload 824 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 msgr2=0x7fd8301cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd830137220 0x7fd8301cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd830136e40 msgr2=0x7fd830135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd830136e40 0x7fd830135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 monclient: found mon.noname-e 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.518+0000 7fd835b7e640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009b30 con 0x7fd830135dc0 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd835b7e640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 msgr2=0x7fd8301c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd835b7e640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 0x7fd8301c8180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd82c035040 tx=0x7fd82c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82c02ad10 con 0x7fd830135dc0 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: dump: 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.537 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.538 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c004500 con 0x7fd830135dc0 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 5 monclient: authenticate success, global_id 4742 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 10 monclient: _renew_subs 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c009400 con 0x7fd830135dc0 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82c003de0 con 0x7fd830135dc0 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd8227fc640 10 monclient: dump: 2024-09-16T15:03:18.539 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.540 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd823fff640 4 set_mon_vals no callback set 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 10 monclient: _renew_subs 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.520+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830125e60 con 0x7fd830135dc0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd837de2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd837de2640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 0x7fd83002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd830067a80 con 0x7fd8301359b0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd82c030340 con 0x7fd830135dc0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 0x7fd83002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient: get_auth_request con 0x7fd8301359b0 auth_method 0 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd8227fc640 1 --2- 172.21.15.134:0/408884421 >> v2:172.21.15.162:6800/980547263 conn(0x7fd814038880 0x7fd81403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 10 monclient(hunting): handle_auth_done global_id 34188 payload 824 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd83537d640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd830061ae0 con 0x7fd8301359b0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.521+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fd82c06ab10 con 0x7fd830135dc0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.522+0000 7fd83637f640 1 --2- 172.21.15.134:0/408884421 >> v2:172.21.15.162:6800/980547263 conn(0x7fd814038880 0x7fd81403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.522+0000 7fd83637f640 10 monclient: get_auth_request con 0x7fd814038880 auth_method 0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd83537d640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 msgr2=0x7fd83002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd83537d640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 0x7fd83002b260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd830121eb0 tx=0x7fd824011cf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82400fcd0 con 0x7fd8301359b0 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 10 monclient: dump: 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd83637f640 1 --2- 172.21.15.134:0/408884421 >> v2:172.21.15.162:6800/980547263 conn(0x7fd814038880 0x7fd81403ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8280072e0 tx=0x7fd82800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd82400fe30 con 0x7fd8301359b0 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 msgr2=0x7fd83002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 0x7fd83002b260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd830121eb0 tx=0x7fd824011cf0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 msgr2=0x7fd83002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.524+0000 7fd8227fc640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8301359b0 0x7fd83002b260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd837de2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd837de2640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 0x7fd83002bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd837de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd830071b20 con 0x7fd83002b7a0 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 0x7fd83002bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient: get_auth_request con 0x7fd83002b7a0 auth_method 0 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 10 monclient(hunting): handle_auth_done global_id 34197 payload 824 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.530+0000 7fd83537d640 1 -- 172.21.15.134:0/408884421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd83007a680 con 0x7fd83002b7a0 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd83537d640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 msgr2=0x7fd83002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd83537d640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 0x7fd83002bbb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd82401b000 tx=0x7fd8240046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd82401ead0 con 0x7fd83002b7a0 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 10 monclient: dump: 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fd82401e660 con 0x7fd83002b7a0 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 msgr2=0x7fd83002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 0x7fd83002bbb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd82401b000 tx=0x7fd8240046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 msgr2=0x7fd83002bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd8227fc640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd83002b7a0 0x7fd83002bbb0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd82401b000 tx=0x7fd8240046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.549 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.550 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.551 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.552 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.553 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.554 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.557 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.559 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.560 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.571 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 >> v2:172.21.15.162:6800/980547263 conn(0x7fd814038880 msgr2=0x7fd81403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 --2- 172.21.15.134:0/408884421 >> v2:172.21.15.162:6800/980547263 conn(0x7fd814038880 0x7fd81403ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8280072e0 tx=0x7fd82800b040 comp rx=0 tx=0).stop 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 10 monclient: shutdown 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 msgr2=0x7fd8301c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 --2- 172.21.15.134:0/408884421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd830135dc0 0x7fd8301c8180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd82c035040 tx=0x7fd82c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd83637f640 1 -- 172.21.15.134:0/408884421 reap_dead start 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd83637f640 1 -- 172.21.15.134:0/408884421 reap_dead start 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 shutdown_connections 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.533+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 >> 172.21.15.134:0/408884421 conn(0x7fd8300a6110 msgr2=0x7fd83009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.534+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 shutdown_connections 2024-09-16T15:03:18.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.534+0000 7fd837de2640 1 -- 172.21.15.134:0/408884421 wait complete. 2024-09-16T15:03:18.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.574 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:03:18.574 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:03:18.611 INFO:tasks.rados.rados.0.smithi162.stdout:578: done (7 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:580: done (6 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:582: done (5 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:583: done (4 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:584: done (3 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:585: done (2 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:586: done (1 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:587: done (0 left) 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:588: rollback oid 3 current snap is 47 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 46 2024-09-16T15:03:18.612 INFO:tasks.rados.rados.0.smithi162.stdout:589: snap_create 2024-09-16T15:03:18.615 INFO:tasks.rados.rados.0.smithi162.stdout:588: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:03:18.617 INFO:tasks.rados.rados.0.smithi162.stdout:588: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:03:18.619 INFO:tasks.rados.rados.0.smithi162.stdout:588: finishing rollback tid 2 to smithi16234619-3 2024-09-16T15:03:18.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 35 (ObjNum 189 snap 39 seq_num 189) dirty exists 2024-09-16T15:03:18.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.674+0000 7f7d9a125640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.674+0000 7f7d9a125640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.674+0000 7f7d9a125640 10 monclient: monmap: 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.675352+0000 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.675352+0000 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.676 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.677 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 Processor -- start 2024-09-16T15:03:18.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 -- start start 2024-09-16T15:03:18.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient: init 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient: _add_conns ranks=[4,9,3,0,6,10,8,7,1,2,5] 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-c con 0x7f7d94136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d94137220 0x7f7d94137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-a con 0x7f7d94137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 0x7f7d94136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-g con 0x7f7d94135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9409abc0 con 0x7f7d94137220 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9409ba50 con 0x7f7d94135dc0 2024-09-16T15:03:18.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940a53c0 con 0x7f7d94136e40 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d9a125640 10 monclient(hunting): authenticate will time out at 935.264694s 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d94137220 0x7f7d94137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7d94137220 0x7f7d94137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:49378/0 (socket says 172.21.15.134:49378) 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2258671235 learned_addr learned my addr 172.21.15.134:0/2258671235 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54228/0 (socket says 172.21.15.134:54228) 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): get_auth_request con 0x7f7d94137220 auth_method 0 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): get_auth_request con 0x7f7d94136e40 auth_method 0 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.675+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 0x7f7d94136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 10 monclient(hunting): get_auth_request con 0x7f7d94135dc0 auth_method 0 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_done global_id 4662 payload 274 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 msgr2=0x7f7d9409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2258671235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 msgr2=0x7f7d94136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 0x7f7d94136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 monclient: found mon.noname-a 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2258671235 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94073250 con 0x7f7d94137220 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d93fff640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 0x7f7d94136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:03:18.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.676+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.678+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d94137220 msgr2=0x7f7d94137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.678+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d94137220 0x7f7d94137600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009870 tx=0x7f7d8402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01ed31deef4c6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 1 -- 172.21.15.134:0/2258671235 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d8402ae00 con 0x7f7d94137220 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: dump: 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.690 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 5 monclient: authenticate success, global_id 4662 2024-09-16T15:03:18.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 1 -- 172.21.15.134:0/2258671235 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d840307b0 con 0x7f7d94137220 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 4 set_mon_vals no callback set 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 1 -- 172.21.15.134:0/2258671235 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d84030d90 con 0x7f7d94137220 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d927fc640 10 monclient: dump: 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.692 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 10 monclient: shutdown 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 -- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d94137220 msgr2=0x7f7d94137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d94137220 0x7f7d94137600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009870 tx=0x7f7d8402a0a0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 -- 172.21.15.134:0/2258671235 shutdown_connections 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d94135dc0 0x7f7d94136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d94137220 0x7f7d94137600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2258671235 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d9409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 -- 172.21.15.134:0/2258671235 >> 172.21.15.134:0/2258671235 conn(0x7f7d940a6110 msgr2=0x7f7d940a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 -- 172.21.15.134:0/2258671235 shutdown_connections 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.679+0000 7f7d9a125640 1 -- 172.21.15.134:0/2258671235 wait complete. 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.680+0000 7f7d9a125640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.680+0000 7f7d9a125640 10 monclient: monmap: 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.694 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.680+0000 7f7d9a125640 1 Processor -- start 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.680+0000 7f7d9a125640 1 -- start start 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.680+0000 7f7d9a125640 10 monclient: init 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient: _add_conns ranks=[7,4,2,3,8,1,0,6,5] 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7d94135dc0 0x7f7d941cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-i con 0x7f7d94135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d94135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-c con 0x7f7d94136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d94137220 0x7f7d94135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): picked mon.noname-b con 0x7f7d94137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94125180 con 0x7f7d94137220 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940a86f0 con 0x7f7d94136e40 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7d941363b0 con 0x7f7d94135dc0 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d9a125640 10 monclient(hunting): authenticate will time out at 935.270233s 2024-09-16T15:03:18.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d94135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d94137220 0x7f7d94135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d94135010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54240/0 (socket says 172.21.15.134:54240) 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2025442303 learned_addr learned my addr 172.21.15.134:0/2025442303 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): get_auth_request con 0x7f7d94137220 auth_method 0 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): get_auth_request con 0x7f7d94136e40 auth_method 0 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7d94135dc0 0x7f7d941cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): get_auth_request con 0x7f7d94135dc0 auth_method 0 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient(hunting): handle_auth_done global_id 4603 payload 824 2024-09-16T15:03:18.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7d94135dc0 msgr2=0x7f7d941cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7d94135dc0 0x7f7d941cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d94137220 msgr2=0x7f7d94135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d94137220 0x7f7d94135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 monclient: found mon.noname-c 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c009400 con 0x7f7d94136e40 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.681+0000 7f7d93fff640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d94137220 0x7f7d94135550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.683+0000 7f7d92ffd640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 msgr2=0x7f7d94135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.683+0000 7f7d92ffd640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d94135010 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d840046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d84030d40 con 0x7f7d94136e40 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:18.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: dump: 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.700 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84003a80 con 0x7f7d94136e40 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 5 monclient: authenticate success, global_id 4603 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 monclient: _renew_subs 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d840094d0 con 0x7f7d94136e40 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d84003bf0 con 0x7f7d94136e40 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 10 monclient: dump: 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 monclient: _renew_subs 2024-09-16T15:03:18.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d91ffb640 4 set_mon_vals no callback set 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d94123da0 con 0x7f7d94136e40 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 0x7f7d9402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7d84039070 con 0x7f7d94136e40 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94071b20 con 0x7f7d94122370 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.684+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 0x7f7d9402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient: get_auth_request con 0x7f7d94122370 auth_method 0 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d8bfff640 1 --2- 172.21.15.134:0/2025442303 >> v2:172.21.15.162:6800/980547263 conn(0x7f7d740388b0 0x7f7d7403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f7d8406a9e0 con 0x7f7d94136e40 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 10 monclient(hunting): handle_auth_done global_id 4751 payload 824 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d937fe640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d94061ae0 con 0x7f7d94122370 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d93fff640 1 --2- 172.21.15.134:0/2025442303 >> v2:172.21.15.162:6800/980547263 conn(0x7f7d740388b0 0x7f7d7403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.685+0000 7f7d93fff640 10 monclient: get_auth_request con 0x7f7d740388b0 auth_method 0 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d937fe640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 msgr2=0x7f7d9402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 0x7f7d9402b260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c009eb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d7c0148b0 con 0x7f7d94122370 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 10 monclient: dump: 2024-09-16T15:03:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d93fff640 1 --2- 172.21.15.134:0/2025442303 >> v2:172.21.15.162:6800/980547263 conn(0x7f7d740388b0 0x7f7d7403ad70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c003e80 tx=0x7f7d8c009850 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7d7c014440 con 0x7f7d94122370 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 msgr2=0x7f7d9402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 0x7f7d9402b260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c009eb0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 msgr2=0x7f7d9402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.687+0000 7f7d8bfff640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d94122370 0x7f7d9402b260 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c009eb0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d9a125640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 0x7f7d9402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d9a125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9407a680 con 0x7f7d941253c0 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 0x7f7d9402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient: get_auth_request con 0x7f7d941253c0 auth_method 0 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 10 monclient(hunting): handle_auth_done global_id 4760 payload 824 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.694+0000 7f7d937fe640 1 -- 172.21.15.134:0/2025442303 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7d9409b490 con 0x7f7d941253c0 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d937fe640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 msgr2=0x7f7d9402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d937fe640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 0x7f7d9402b7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7d7c0222e0 con 0x7f7d941253c0 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d8bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d8bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.696+0000 7f7d8bfff640 10 monclient: dump: 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.710 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f7d7c026070 con 0x7f7d941253c0 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 msgr2=0x7f7d9402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 0x7f7d9402b7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 msgr2=0x7f7d9402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d8bfff640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7d941253c0 0x7f7d9402b7a0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c011c00 tx=0x7f7d7c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.711 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.712 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.713 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.714 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.715 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.716 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.723 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.725 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.729 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:18.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.733 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 >> v2:172.21.15.162:6800/980547263 conn(0x7f7d740388b0 msgr2=0x7f7d7403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2025442303 >> v2:172.21.15.162:6800/980547263 conn(0x7f7d740388b0 0x7f7d7403ad70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c003e80 tx=0x7f7d8c009850 comp rx=0 tx=0).stop 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 10 monclient: shutdown 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 msgr2=0x7f7d94135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 --2- 172.21.15.134:0/2025442303 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7d94136e40 0x7f7d94135010 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d840046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d93fff640 1 -- 172.21.15.134:0/2025442303 reap_dead start 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d93fff640 1 -- 172.21.15.134:0/2025442303 reap_dead start 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 shutdown_connections 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 >> 172.21.15.134:0/2025442303 conn(0x7f7d940a6110 msgr2=0x7f7d9409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 shutdown_connections 2024-09-16T15:03:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.697+0000 7f7d9a125640 1 -- 172.21.15.134:0/2025442303 wait complete. 2024-09-16T15:03:18.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:03:18.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:03:18.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.831+0000 7fe42730f640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.831+0000 7fe42730f640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.831+0000 7fe42730f640 10 monclient: monmap: 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.832373+0000 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.832373+0000 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.834 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.831+0000 7fe42730f640 1 Processor -- start 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 -- start start 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient: init 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient: _add_conns ranks=[3,7,2,5,9,0,6,4,10,1,8] 2024-09-16T15:03:18.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 0x7fe42009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-g con 0x7fe420136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe420137220 0x7fe420137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-i con 0x7fe420137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-b con 0x7fe420135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42009abc0 con 0x7fe420135dc0 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42009ba50 con 0x7fe420136e40 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200a53c0 con 0x7fe420137220 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe42730f640 10 monclient(hunting): authenticate will time out at 935.421678s 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59638/0 (socket says 172.21.15.134:59638) 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 1 -- 172.21.15.134:0/3850779666 learned_addr learned my addr 172.21.15.134:0/3850779666 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): get_auth_request con 0x7fe420135dc0 auth_method 0 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.832+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4250ab640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 0x7fe42009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4248aa640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe420137220 0x7fe420137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 10 monclient(hunting): handle_auth_done global_id 34220 payload 274 2024-09-16T15:03:18.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 -- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe420137220 msgr2=0x7fe420137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe420137220 0x7fe420137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 -- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 msgr2=0x7fe42009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 0x7fe42009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 monclient: found mon.noname-b 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4258ac640 1 -- 172.21.15.134:0/3850779666 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe420073250 con 0x7fe420135dc0 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4250ab640 10 monclient: get_auth_request con 0x7fe420136e40 auth_method 0 2024-09-16T15:03:18.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.833+0000 7fe4250ab640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 0x7fe42009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:03:18.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.835+0000 7fe4258ac640 1 -- 172.21.15.134:0/3850779666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 msgr2=0x7fe420136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.835+0000 7fe4258ac640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe41802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110d090e19671b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 1 -- 172.21.15.134:0/3850779666 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4180307b0 con 0x7fe420135dc0 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: dump: 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.847 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 1 -- 172.21.15.134:0/3850779666 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418030930 con 0x7fe420135dc0 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 5 monclient: authenticate success, global_id 34220 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 1 -- 172.21.15.134:0/3850779666 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4180313a0 con 0x7fe420135dc0 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 4 set_mon_vals no callback set 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe41ffff640 10 monclient: dump: 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.849 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 10 monclient: shutdown 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 -- 172.21.15.134:0/3850779666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 msgr2=0x7fe420136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe41802a080 comp rx=0 tx=0).stop 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 -- 172.21.15.134:0/3850779666 shutdown_connections 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420135dc0 0x7fe420136220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe420137220 0x7fe420137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 --2- 172.21.15.134:0/3850779666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe420136e40 0x7fe42009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 -- 172.21.15.134:0/3850779666 >> 172.21.15.134:0/3850779666 conn(0x7fe4200a6110 msgr2=0x7fe4200a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 -- 172.21.15.134:0/3850779666 shutdown_connections 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.836+0000 7fe42730f640 1 -- 172.21.15.134:0/3850779666 wait complete. 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.837+0000 7fe42730f640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.837+0000 7fe42730f640 10 monclient: monmap: 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.837+0000 7fe42730f640 1 Processor -- start 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.837+0000 7fe42730f640 1 -- start start 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.837+0000 7fe42730f640 10 monclient: init 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient: _add_conns ranks=[8,2,6,0,3,1,4,7,5] 2024-09-16T15:03:18.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 0x7fe420134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-e con 0x7fe420135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 0x7fe420135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-b con 0x7fe420136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe420137220 0x7fe420121b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): picked mon.noname-d con 0x7fe420137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420136430 con 0x7fe420136e40 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201cc1d0 con 0x7fe420137220 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200a8730 con 0x7fe420135dc0 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe42730f640 10 monclient(hunting): authenticate will time out at 935.427214s 2024-09-16T15:03:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 0x7fe420135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 0x7fe420135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59648/0 (socket says 172.21.15.134:59648) 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 1 -- 172.21.15.134:0/1924000628 learned_addr learned my addr 172.21.15.134:0/1924000628 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 0x7fe420134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): get_auth_request con 0x7fe420136e40 auth_method 0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe420137220 0x7fe420121b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): get_auth_request con 0x7fe420135dc0 auth_method 0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): get_auth_request con 0x7fe420137220 auth_method 0 2024-09-16T15:03:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient(hunting): handle_auth_done global_id 4769 payload 824 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe420137220 msgr2=0x7fe420121b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe420137220 0x7fe420121b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 msgr2=0x7fe420135510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 0x7fe420135510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient: _finish_hunting 0 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 monclient: found mon.noname-e 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4250ab640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180094d0 con 0x7fe420135dc0 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4248aa640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe420136e40 0x7fe420135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.838+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe420137220 0x7fe420121b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.840+0000 7fe4250ab640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 msgr2=0x7fe420134fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.840+0000 7fe4250ab640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 0x7fe420134fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c820 tx=0x7fe41000cce0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.840+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe41000ce10 con 0x7fe420135dc0 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: dump: 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.857 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100113f0 con 0x7fe420135dc0 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 5 monclient: authenticate success, global_id 4769 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 10 monclient: _renew_subs 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4201235f0 con 0x7fe420135dc0 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe4100119d0 con 0x7fe420135dc0 2024-09-16T15:03:18.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 10 monclient: dump: 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.859 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 10 monclient: _renew_subs 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41f7fe640 4 set_mon_vals no callback set 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200a8b10 con 0x7fe420135dc0 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.841+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe410011560 con 0x7fe420135dc0 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe42730f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe41dffb640 1 --2- 172.21.15.134:0/1924000628 >> v2:172.21.15.162:6800/980547263 conn(0x7fe40c038880 0x7fe40c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe42730f640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 0x7fe42002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420071b20 con 0x7fe420125640 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fe410049a70 con 0x7fe420135dc0 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4248aa640 1 --2- 172.21.15.134:0/1924000628 >> v2:172.21.15.162:6800/980547263 conn(0x7fe40c038880 0x7fe40c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 0x7fe42002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4248aa640 10 monclient: get_auth_request con 0x7fe40c038880 auth_method 0 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient: get_auth_request con 0x7fe420125640 auth_method 0 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 10 monclient(hunting): handle_auth_done global_id 34183 payload 824 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.842+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe420125120 con 0x7fe420125640 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.844+0000 7fe4248aa640 1 --2- 172.21.15.134:0/1924000628 >> v2:172.21.15.162:6800/980547263 conn(0x7fe40c038880 0x7fe40c03ad40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4200735f0 tx=0x7fe4140077d0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 msgr2=0x7fe42002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 0x7fe42002b260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe418009870 tx=0x7fe4180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe418003c90 con 0x7fe420125640 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 10 monclient: dump: 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.864 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe418003df0 con 0x7fe420125640 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 msgr2=0x7fe42002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 0x7fe42002b260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe418009870 tx=0x7fe4180046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 msgr2=0x7fe42002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.845+0000 7fe41dffb640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe420125640 0x7fe42002b260 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe418009870 tx=0x7fe4180046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.851+0000 7fe42730f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:18.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.851+0000 7fe42730f640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 0x7fe42002bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.851+0000 7fe42730f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.851+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420061ae0 con 0x7fe42002b7a0 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 0x7fe42002bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient: get_auth_request con 0x7fe42002b7a0 auth_method 0 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 10 monclient(hunting): handle_auth_done global_id 34192 payload 824 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.852+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe42007a680 con 0x7fe42002b7a0 2024-09-16T15:03:18.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 msgr2=0x7fe42002bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe4258ac640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 0x7fe42002bbd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe418008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe41803cd60 con 0x7fe42002b7a0 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe41dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe41dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.854+0000 7fe41dffb640 10 monclient: dump: 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:18.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fe41804b070 con 0x7fe42002b7a0 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 msgr2=0x7fe42002bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 0x7fe42002bbd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe418008030 comp rx=0 tx=0).stop 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 msgr2=0x7fe42002bbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:18.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe41dffb640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe42002b7a0 0x7fe42002bbd0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe418008030 comp rx=0 tx=0).stop 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:18.871 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.872 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.873 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:18.874 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:18.875 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:18.876 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.877 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:18.879 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.881 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.882 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:18.884 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.886 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.887 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.888 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:18.889 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.891 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:18.892 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:18.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:18.893 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:18.893 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 >> v2:172.21.15.162:6800/980547263 conn(0x7fe40c038880 msgr2=0x7fe40c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 --2- 172.21.15.134:0/1924000628 >> v2:172.21.15.162:6800/980547263 conn(0x7fe40c038880 0x7fe40c03ad40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4200735f0 tx=0x7fe4140077d0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 10 monclient: shutdown 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 msgr2=0x7fe420134fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 --2- 172.21.15.134:0/1924000628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe420135dc0 0x7fe420134fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe41000c820 tx=0x7fe41000cce0 comp rx=0 tx=0).stop 2024-09-16T15:03:18.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 reap_dead start 2024-09-16T15:03:18.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe4258ac640 1 -- 172.21.15.134:0/1924000628 reap_dead start 2024-09-16T15:03:18.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 shutdown_connections 2024-09-16T15:03:18.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 >> 172.21.15.134:0/1924000628 conn(0x7fe4200a6110 msgr2=0x7fe42009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:18.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 shutdown_connections 2024-09-16T15:03:18.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.855+0000 7fe42730f640 1 -- 172.21.15.134:0/1924000628 wait complete. 2024-09-16T15:03:18.895 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:18.895 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:03:18.895 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.991+0000 7fc8568be640 10 monclient: get_monmap_and_config 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.991+0000 7fc8568be640 10 monclient: build_initial_monmap 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.991+0000 7fc8568be640 10 monclient: monmap: 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:18.992525+0000 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:18.992525+0000 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:18.993 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:18.994 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 Processor -- start 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 -- start start 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient: init 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient: _add_conns ranks=[8,2,6,5,0,4,10,3,9,1,7] 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-e con 0x7fc850136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-b con 0x7fc850137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:18.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-d con 0x7fc850135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85009abc0 con 0x7fc850137220 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc85009ba50 con 0x7fc850135dc0 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500a53c0 con 0x7fc850136e40 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.992+0000 7fc8568be640 10 monclient(hunting): authenticate will time out at 935.581993s 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59658/0 (socket says 172.21.15.134:59658) 2024-09-16T15:03:18.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 1 -- 172.21.15.134:0/2068759540 learned_addr learned my addr 172.21.15.134:0/2068759540 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): get_auth_request con 0x7fc850137220 auth_method 0 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:54556/0 (socket says 172.21.15.134:54556) 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:46228/0 (socket says 172.21.15.134:46228) 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): get_auth_request con 0x7fc850135dc0 auth_method 0 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): get_auth_request con 0x7fc850136e40 auth_method 0 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:18.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient(hunting): handle_auth_done global_id 4778 payload 274 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 -- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 msgr2=0x7fc850136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 -- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 msgr2=0x7fc850137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 monclient: found mon.noname-e 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc84ffff640 1 -- 172.21.15.134:0/2068759540 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850073250 con 0x7fc850136e40 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc854e5b640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:19.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.993+0000 7fc847fff640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84ffff640 1 -- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 msgr2=0x7fc85009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84ffff640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc8400097d0 tx=0x7fc840029fb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=ddac1580a2c2ae29 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 1 -- 172.21.15.134:0/2068759540 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84002ad10 con 0x7fc850136e40 2024-09-16T15:03:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: dump: 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.008 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 1 -- 172.21.15.134:0/2068759540 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84002ae90 con 0x7fc850136e40 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 5 monclient: authenticate success, global_id 4778 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 1 -- 172.21.15.134:0/2068759540 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc840030c20 con 0x7fc850136e40 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 4 set_mon_vals no callback set 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc84f7fe640 10 monclient: dump: 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 10 monclient: shutdown 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 1 -- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 msgr2=0x7fc85009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.996+0000 7fc8568be640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc8400097d0 tx=0x7fc840029fb0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 -- 172.21.15.134:0/2068759540 shutdown_connections 2024-09-16T15:03:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc850136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc850137220 0x7fc850137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 --2- 172.21.15.134:0/2068759540 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc850136e40 0x7fc85009c150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 -- 172.21.15.134:0/2068759540 >> 172.21.15.134:0/2068759540 conn(0x7fc8500a6110 msgr2=0x7fc8500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 -- 172.21.15.134:0/2068759540 shutdown_connections 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 -- 172.21.15.134:0/2068759540 wait complete. 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 10 monclient: monmap: 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.997+0000 7fc8568be640 1 Processor -- start 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 -- start start 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient: init 2024-09-16T15:03:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient: _add_conns ranks=[6,7,5,8,3,4,1,0,2] 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-d con 0x7fc850135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc850136e40 0x7fc850135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-i con 0x7fc850136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc850137220 0x7fc850135680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): picked mon.noname-h con 0x7fc850137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8501cc3d0 con 0x7fc850137220 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8501258e0 con 0x7fc850135dc0 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc850125b30 con 0x7fc850136e40 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc8568be640 10 monclient(hunting): authenticate will time out at 935.587446s 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:54572/0 (socket says 172.21.15.134:54572) 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 -- 172.21.15.134:0/3122634910 learned_addr learned my addr 172.21.15.134:0/3122634910 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): get_auth_request con 0x7fc850135dc0 auth_method 0 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient(hunting): handle_auth_done global_id 34224 payload 824 2024-09-16T15:03:19.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc850136e40 msgr2=0x7fc850135140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc850136e40 0x7fc850135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc854e5b640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc850137220 0x7fc850135680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc850137220 msgr2=0x7fc850135680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc850137220 0x7fc850135680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 monclient: found mon.noname-d 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc84ffff640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848009a70 con 0x7fc850135dc0 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc854e5b640 10 monclient: get_auth_request con 0x7fc850137220 auth_method 0 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:18.998+0000 7fc854e5b640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc850137220 0x7fc850135680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84ffff640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 msgr2=0x7fc8501cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84ffff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc840035040 tx=0x7fc8400046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84002ad10 con 0x7fc850135dc0 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: dump: 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.017 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840004500 con 0x7fc850135dc0 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 5 monclient: authenticate success, global_id 34224 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 10 monclient: _renew_subs 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8380094d0 con 0x7fc850135dc0 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc840003de0 con 0x7fc850135dc0 2024-09-16T15:03:19.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84d7fa640 10 monclient: dump: 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.019 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc84effd640 4 set_mon_vals no callback set 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 10 monclient: _renew_subs 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.001+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc840009400 con 0x7fc850135dc0 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc8568be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc8568be640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 0x7fc85002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850071b20 con 0x7fc850122360 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc840030340 con 0x7fc850135dc0 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 0x7fc85002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient: get_auth_request con 0x7fc850122360 auth_method 0 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc84d7fa640 1 --2- 172.21.15.134:0/3122634910 >> v2:172.21.15.162:6800/980547263 conn(0x7fc830038880 0x7fc83003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc84006ab40 con 0x7fc850135dc0 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 10 monclient(hunting): handle_auth_done global_id 34230 payload 824 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc847fff640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc850061ae0 con 0x7fc850122360 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc854e5b640 1 --2- 172.21.15.134:0/3122634910 >> v2:172.21.15.162:6800/980547263 conn(0x7fc830038880 0x7fc83003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.002+0000 7fc854e5b640 10 monclient: get_auth_request con 0x7fc830038880 auth_method 0 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc847fff640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 msgr2=0x7fc85002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc847fff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 0x7fc85002b260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc850123300 tx=0x7fc83800ec00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc838004030 con 0x7fc850122360 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc84d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc84d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.004+0000 7fc84d7fa640 10 monclient: dump: 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.022 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc854e5b640 1 --2- 172.21.15.134:0/3122634910 >> v2:172.21.15.162:6800/980547263 conn(0x7fc830038880 0x7fc83003ad40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc848009fd0 tx=0x7fc84800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc83800cb30 con 0x7fc850122360 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 msgr2=0x7fc85002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 0x7fc85002b260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc850123300 tx=0x7fc83800ec00 comp rx=0 tx=0).stop 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 msgr2=0x7fc85002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.005+0000 7fc84d7fa640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122360 0x7fc85002b260 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc850123300 tx=0x7fc83800ec00 comp rx=0 tx=0).stop 2024-09-16T15:03:19.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc8568be640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:19.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc8568be640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 0x7fc85002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc8568be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85007a680 con 0x7fc850122740 2024-09-16T15:03:19.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 0x7fc85002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient: get_auth_request con 0x7fc850122740 auth_method 0 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 10 monclient(hunting): handle_auth_done global_id 34239 payload 824 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.011+0000 7fc847fff640 1 -- 172.21.15.134:0/3122634910 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc85009b490 con 0x7fc850122740 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc847fff640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 msgr2=0x7fc85002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc847fff640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 0x7fc85002b7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc838018000 tx=0x7fc8380046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc838005e00 con 0x7fc850122740 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 10 monclient: dump: 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.027 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fc838003ca0 con 0x7fc850122740 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 msgr2=0x7fc85002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 0x7fc85002b7a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc838018000 tx=0x7fc8380046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 msgr2=0x7fc85002b7a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc84d7fa640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc850122740 0x7fc85002b7a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.028 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:19.029 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.030 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.031 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:19.032 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.033 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:19.034 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:19.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:19.037 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.039 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.040 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 >> v2:172.21.15.162:6800/980547263 conn(0x7fc830038880 msgr2=0x7fc83003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.014+0000 7fc8568be640 1 --2- 172.21.15.134:0/3122634910 >> v2:172.21.15.162:6800/980547263 conn(0x7fc830038880 0x7fc83003ad40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc848009fd0 tx=0x7fc84800b040 comp rx=0 tx=0).stop 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 10 monclient: shutdown 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 msgr2=0x7fc8501cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc854e5b640 1 -- 172.21.15.134:0/3122634910 reap_dead start 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc840035040 tx=0x7fc8400046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 shutdown_connections 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 --2- 172.21.15.134:0/3122634910 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc850135dc0 0x7fc8501cb700 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 >> 172.21.15.134:0/3122634910 conn(0x7fc8500a6110 msgr2=0x7fc85009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 shutdown_connections 2024-09-16T15:03:19.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.015+0000 7fc8568be640 1 -- 172.21.15.134:0/3122634910 wait complete. 2024-09-16T15:03:19.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:19.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:03:19.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:03:19.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.151+0000 7f90ba765640 10 monclient: get_monmap_and_config 2024-09-16T15:03:19.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.151+0000 7f90ba765640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.152+0000 7f90ba765640 10 monclient: monmap: 2024-09-16T15:03:19.153 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:19.152932+0000 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:19.152932+0000 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.154 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.155 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:19.155 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:19.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.152+0000 7f90ba765640 1 Processor -- start 2024-09-16T15:03:19.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.152+0000 7f90ba765640 1 -- start start 2024-09-16T15:03:19.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.152+0000 7f90ba765640 10 monclient: init 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient: _add_conns ranks=[4,10,5,1,7,9,8,2,3,0,6] 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-c con 0x7f90b4136eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f90b4137290 0x7f90b4137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-f con 0x7f90b4137290 addr v2:172.21.15.162:3300/0 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4133d10 0x7f90b4134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-h con 0x7f90b4133d10 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:19.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b40a5560 con 0x7f90b4137290 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90b409b240 con 0x7f90b4136eb0 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90b409b520 con 0x7f90b4133d10 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90ba765640 10 monclient(hunting): authenticate will time out at 935.742431s 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54246/0 (socket says 172.21.15.134:54246) 2024-09-16T15:03:19.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 -- 172.21.15.134:0/3938505145 learned_addr learned my addr 172.21.15.134:0/3938505145 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): get_auth_request con 0x7f90b4136eb0 auth_method 0 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4133d10 0x7f90b4134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 1 --2- 172.21.15.134:0/3938505145 >> v2:172.21.15.162:3300/0 conn(0x7f90b4137290 0x7f90b4137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): get_auth_request con 0x7f90b4133d10 auth_method 0 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): get_auth_request con 0x7f90b4137290 auth_method 0 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient(hunting): handle_auth_done global_id 4612 payload 274 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 -- 172.21.15.134:0/3938505145 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4133d10 msgr2=0x7f90b4134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4133d10 0x7f90b4134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 -- 172.21.15.134:0/3938505145 >> v2:172.21.15.162:3300/0 conn(0x7f90b4137290 msgr2=0x7f90b4137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3938505145 >> v2:172.21.15.162:3300/0 conn(0x7f90b4137290 0x7f90b4137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 monclient: found mon.noname-c 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.153+0000 7f90b3fff640 1 -- 172.21.15.134:0/3938505145 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4073250 con 0x7f90b4136eb0 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b3fff640 1 -- 172.21.15.134:0/3938505145 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 msgr2=0x7f90b409bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009870 tx=0x7f90a402a0a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a9ecd00d4ced030 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 1 -- 172.21.15.134:0/3938505145 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90a402ae00 con 0x7f90b4136eb0 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: dump: 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.168 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 1 -- 172.21.15.134:0/3938505145 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a40307b0 con 0x7f90b4136eb0 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90ba765640 5 monclient: authenticate success, global_id 4612 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90ba765640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90ba765640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 1 -- 172.21.15.134:0/3938505145 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90a4030d90 con 0x7f90b4136eb0 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90ba765640 4 set_mon_vals no callback set 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:19.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.156+0000 7f90b37fe640 10 monclient: dump: 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.171 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 10 monclient: shutdown 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 -- 172.21.15.134:0/3938505145 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 msgr2=0x7f90b409bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009870 tx=0x7f90a402a0a0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 -- 172.21.15.134:0/3938505145 shutdown_connections 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4133d10 0x7f90b4134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 --2- 172.21.15.134:0/3938505145 >> v2:172.21.15.162:3300/0 conn(0x7f90b4137290 0x7f90b4137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 --2- 172.21.15.134:0/3938505145 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f90b4136eb0 0x7f90b409bf00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 -- 172.21.15.134:0/3938505145 >> 172.21.15.134:0/3938505145 conn(0x7f90b40a99a0 msgr2=0x7f90b40a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 -- 172.21.15.134:0/3938505145 shutdown_connections 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.157+0000 7f90ba765640 1 -- 172.21.15.134:0/3938505145 wait complete. 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient: monmap: 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.173 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 Processor -- start 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 -- start start 2024-09-16T15:03:19.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient: init 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient: _add_conns ranks=[1,7,5,0,6,3,4,2,8] 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b4133d10 0x7f90b41cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-f con 0x7f90b4133d10 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 0x7f90b4132ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-i con 0x7f90b4136eb0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): picked mon.noname-h con 0x7f90b4137290 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b41200e0 con 0x7f90b4133d10 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90b41cc530 con 0x7f90b4137290 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40a2e20 con 0x7f90b4136eb0 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.158+0000 7f90ba765640 10 monclient(hunting): authenticate will time out at 935.747904s 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 0x7f90b4132ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 0x7f90b4132ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:51002/0 (socket says 172.21.15.134:51002) 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:32968/0 (socket says 172.21.15.134:32968) 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 1 -- 172.21.15.134:0/3094027091 learned_addr learned my addr 172.21.15.134:0/3094027091 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): get_auth_request con 0x7f90b4136eb0 auth_method 0 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): get_auth_request con 0x7f90b4137290 auth_method 0 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b4133d10 0x7f90b41cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): get_auth_request con 0x7f90b4133d10 auth_method 0 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient(hunting): handle_auth_done global_id 4523 payload 824 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 msgr2=0x7f90b4132ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 0x7f90b4132ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b4133d10 msgr2=0x7f90b41cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b4133d10 0x7f90b41cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 monclient: found mon.noname-h 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40094d0 con 0x7f90b4137290 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90abfff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f90b4136eb0 0x7f90b4132ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:19.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.159+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90b4133d10 0x7f90b41cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.161+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 msgr2=0x7f90b40a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.161+0000 7f90b8d02640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00efc0 tx=0x7f90ac00c490 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90ac009c00 con 0x7f90b4137290 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:03:19.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: dump: 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.186 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b8d02640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00efc0 tx=0x7f90ac00c490 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f90ac014d60 mon_map magic: 0 v1 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac00f040 con 0x7f90b4137290 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 5 monclient: authenticate success, global_id 4523 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 monclient: _renew_subs 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:03:19.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b40a2400 con 0x7f90b4137290 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b2ffd640 4 set_mon_vals no callback set 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 monclient: _renew_subs 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b40a7960 con 0x7f90b4137290 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 0x7f90b4120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90b4073880 con 0x7f90b41333e0 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90ac014d60 con 0x7f90b4137290 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:19.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.162+0000 7f90b17fa640 10 monclient: dump: 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.189 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f90ac014480 con 0x7f90b4137290 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 0x7f90b4120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient: get_auth_request con 0x7f90b41333e0 auth_method 0 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b17fa640 1 --2- 172.21.15.134:0/3094027091 >> v2:172.21.15.162:6800/980547263 conn(0x7f9094038880 0x7f909403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 10 monclient(hunting): handle_auth_done global_id 4532 payload 824 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90b3fff640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90b4061ae0 con 0x7f90b41333e0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90abfff640 1 --2- 172.21.15.134:0/3094027091 >> v2:172.21.15.162:6800/980547263 conn(0x7f9094038880 0x7f909403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.163+0000 7f90abfff640 10 monclient: get_auth_request con 0x7f9094038880 auth_method 0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b3fff640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 msgr2=0x7f90b4120500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 0x7f90b4120500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a4034000 tx=0x7f90a40046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90a4003c90 con 0x7f90b41333e0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90abfff640 1 --2- 172.21.15.134:0/3094027091 >> v2:172.21.15.162:6800/980547263 conn(0x7f9094038880 0x7f909403ad40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f90b4066fb0 tx=0x7f909c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.165+0000 7f90b17fa640 10 monclient: dump: 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f90a4003df0 con 0x7f90b41333e0 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 msgr2=0x7f90b4120500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 0x7f90b4120500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90a4034000 tx=0x7f90a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 msgr2=0x7f90b4120500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.166+0000 7f90b17fa640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b41333e0 0x7f90b4120500 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90ba765640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90ba765640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 0x7f90b40a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90ba765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90b404e260 con 0x7f90b40a0ec0 2024-09-16T15:03:19.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 0x7f90b40a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient: get_auth_request con 0x7f90b40a0ec0 auth_method 0 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 10 monclient(hunting): handle_auth_done global_id 4541 payload 824 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.172+0000 7f90b3fff640 1 -- 172.21.15.134:0/3094027091 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90b407a680 con 0x7f90b40a0ec0 2024-09-16T15:03:19.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b3fff640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 msgr2=0x7f90b40a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b3fff640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 0x7f90b40a12d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90a40097c0 tx=0x7f90a4008030 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90a403ce40 con 0x7f90b40a0ec0 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 10 monclient: dump: 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.195 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f90a404f430 con 0x7f90b40a0ec0 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:19.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:19.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 msgr2=0x7f90b40a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 0x7f90b40a12d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90a40097c0 tx=0x7f90a4008030 comp rx=0 tx=0).stop 2024-09-16T15:03:19.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 msgr2=0x7f90b40a12d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b40a0ec0 0x7f90b40a12d0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90a40097c0 tx=0x7f90a4008030 comp rx=0 tx=0).stop 2024-09-16T15:03:19.197 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:19.200 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:19.201 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:19.202 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:19.203 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:19.204 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.205 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.209 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:19.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:19.213 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:19.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.216 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.220 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.222 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.224 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.225 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.225 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.225 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:19.225 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 >> v2:172.21.15.162:6800/980547263 conn(0x7f9094038880 msgr2=0x7f909403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 1 --2- 172.21.15.134:0/3094027091 >> v2:172.21.15.162:6800/980547263 conn(0x7f9094038880 0x7f909403ad40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f90b4066fb0 tx=0x7f909c009190 comp rx=0 tx=0).stop 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 10 monclient: shutdown 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 msgr2=0x7f90b40a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90ba765640 1 --2- 172.21.15.134:0/3094027091 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f90b4137290 0x7f90b40a0980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00efc0 tx=0x7f90ac00c490 comp rx=0 tx=0).stop 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 reap_dead start 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b8d02640 1 -- 172.21.15.134:0/3094027091 reap_dead start 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.175+0000 7f90b17fa640 1 -- 172.21.15.134:0/3094027091 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7f90ac04b390 con 0x7f90b4137290 2024-09-16T15:03:19.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.176+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 shutdown_connections 2024-09-16T15:03:19.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.176+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 >> 172.21.15.134:0/3094027091 conn(0x7f90b40a99a0 msgr2=0x7f90b406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.176+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 shutdown_connections 2024-09-16T15:03:19.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.176+0000 7f90ba765640 1 -- 172.21.15.134:0/3094027091 wait complete. 2024-09-16T15:03:19.228 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:19.228 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:03:19.228 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.324+0000 7fe5e1f8a640 10 monclient: get_monmap_and_config 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.324+0000 7fe5e1f8a640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.324+0000 7fe5e1f8a640 10 monclient: monmap: 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:19.325441+0000 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:19.325441+0000 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.326 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:19.327 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 Processor -- start 2024-09-16T15:03:19.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 -- start start 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient: init 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient: _add_conns ranks=[0,4,3,10,5,1,8,9,7,6,2] 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe5dc136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 0x7fe5dc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-c con 0x7fe5dc137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-g con 0x7fe5dc13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc073250 con 0x7fe5dc136e40 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0a56e0 con 0x7fe5dc13c370 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc09abc0 con 0x7fe5dc137220 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5e1f8a640 10 monclient(hunting): authenticate will time out at 935.914766s 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5db7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe5dc136e40 0x7fe5dc09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:51998/0 (socket says 172.21.15.134:51998) 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3044668052 learned_addr learned my addr 172.21.15.134:0/3044668052 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 0x7fe5dc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): get_auth_request con 0x7fe5dc137220 auth_method 0 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.325+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5dbfff640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5dbfff640 10 monclient(hunting): get_auth_request con 0x7fe5dc13c370 auth_method 0 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 10 monclient(hunting): handle_auth_done global_id 4621 payload 274 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 1 -- 172.21.15.134:0/3044668052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 msgr2=0x7fe5dc13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 1 -- 172.21.15.134:0/3044668052 >> v1:172.21.15.134:6789/0 conn(0x7fe5dc136e40 legacy=0x7fe5dc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 1 monclient: found mon.noname-c 2024-09-16T15:03:19.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.326+0000 7fe5daffd640 1 -- 172.21.15.134:0/3044668052 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc09ba50 con 0x7fe5dc137220 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.328+0000 7fe5daffd640 1 -- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 msgr2=0x7fe5dc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.328+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 0x7fe5dc137600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097d0 tx=0x7fe5d002a000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7fdd08ed1177e88e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 1 -- 172.21.15.134:0/3044668052 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5d002ad20 con 0x7fe5dc137220 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: dump: 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 1 -- 172.21.15.134:0/3044668052 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00307b0 con 0x7fe5dc137220 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 1 -- 172.21.15.134:0/3044668052 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5d002aea0 con 0x7fe5dc137220 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 5 monclient: authenticate success, global_id 4621 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 4 set_mon_vals no callback set 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5da7fc640 10 monclient: dump: 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.337 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 10 monclient: shutdown 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 msgr2=0x7fe5dc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 0x7fe5dc137600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097d0 tx=0x7fe5d002a000 comp rx=0 tx=0).stop 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3044668052 shutdown_connections 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3044668052 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc137220 0x7fe5dc137600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3044668052 >> 172.21.15.134:0/3044668052 conn(0x7fe5dc0a6110 msgr2=0x7fe5dc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3044668052 shutdown_connections 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.329+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3044668052 wait complete. 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.330+0000 7fe5e1f8a640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.330+0000 7fe5e1f8a640 10 monclient: monmap: 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:19.339 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.330+0000 7fe5e1f8a640 1 Processor -- start 2024-09-16T15:03:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.330+0000 7fe5e1f8a640 1 -- start start 2024-09-16T15:03:19.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.330+0000 7fe5e1f8a640 10 monclient: init 2024-09-16T15:03:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient: _add_conns ranks=[4,1,3,7,2,0,8,6,5] 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 0x7fe5dc135b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-c con 0x7fe5dc136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc137220 0x7fe5dc1360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-f con 0x7fe5dc137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): picked mon.noname-g con 0x7fe5dc13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc13b530 con 0x7fe5dc137220 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0a7af0 con 0x7fe5dc13c370 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc122110 con 0x7fe5dc136e40 2024-09-16T15:03:19.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5e1f8a640 10 monclient(hunting): authenticate will time out at 935.920382s 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 0x7fe5dc135b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 0x7fe5dc135b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54270/0 (socket says 172.21.15.134:54270) 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3198368325 learned_addr learned my addr 172.21.15.134:0/3198368325 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): get_auth_request con 0x7fe5dc136e40 auth_method 0 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc137220 0x7fe5dc1360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): get_auth_request con 0x7fe5dc13c370 auth_method 0 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): get_auth_request con 0x7fe5dc137220 auth_method 0 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient(hunting): handle_auth_done global_id 4630 payload 824 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 msgr2=0x7fe5dc11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5dc13c370 0x7fe5dc11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc137220 msgr2=0x7fe5dc1360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5dc137220 0x7fe5dc1360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 monclient: found mon.noname-c 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.331+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009400 con 0x7fe5dc136e40 2024-09-16T15:03:19.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.333+0000 7fe5db7fe640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 msgr2=0x7fe5dc135b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.333+0000 7fe5db7fe640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 0x7fe5dc135b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c460 tx=0x7fe5cc00c920 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5cc005cf0 con 0x7fe5dc136e40 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: dump: 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc005e90 con 0x7fe5dc136e40 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 5 monclient: authenticate success, global_id 4630 2024-09-16T15:03:19.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 10 monclient: _renew_subs 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc122450 con 0x7fe5dc136e40 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5cc0074e0 con 0x7fe5dc136e40 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 10 monclient: dump: 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.353 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 10 monclient: _renew_subs 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5d9ffb640 4 set_mon_vals no callback set 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc1365f0 con 0x7fe5dc136e40 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe5cc007070 con 0x7fe5dc136e40 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.334+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 0x7fe5dc02b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc071b20 con 0x7fe5dc02b260 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5c3fff640 1 --2- 172.21.15.134:0/3198368325 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5c8038880 0x7fe5c803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 0x7fe5dc02b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient: get_auth_request con 0x7fe5dc02b260 auth_method 0 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fe5cc049be0 con 0x7fe5dc136e40 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5dbfff640 1 --2- 172.21.15.134:0/3198368325 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5c8038880 0x7fe5c803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5dbfff640 10 monclient: get_auth_request con 0x7fe5c8038880 auth_method 0 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 10 monclient(hunting): handle_auth_done global_id 4543 payload 824 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.335+0000 7fe5daffd640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5dc061ae0 con 0x7fe5dc02b260 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5daffd640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 msgr2=0x7fe5dc02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 0x7fe5dc02b640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc137f00 tx=0x7fe5d00046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5dbfff640 1 --2- 172.21.15.134:0/3198368325 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5c8038880 0x7fe5c803ad40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0a2f10 tx=0x7fe5d4009780 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5d0003a80 con 0x7fe5dc02b260 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5c3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.337+0000 7fe5c3fff640 10 monclient: dump: 2024-09-16T15:03:19.356 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.357 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe5d0003be0 con 0x7fe5dc02b260 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 msgr2=0x7fe5dc02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 0x7fe5dc02b640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc137f00 tx=0x7fe5d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 msgr2=0x7fe5dc02b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.338+0000 7fe5c3fff640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02b260 0x7fe5dc02b640 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc137f00 tx=0x7fe5d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5e1f8a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 0x7fe5dc02bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5e1f8a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0677b0 con 0x7fe5dc02bb80 2024-09-16T15:03:19.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 0x7fe5dc02bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient: get_auth_request con 0x7fe5dc02bb80 auth_method 0 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 10 monclient(hunting): handle_auth_done global_id 4552 payload 824 2024-09-16T15:03:19.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.344+0000 7fe5daffd640 1 -- 172.21.15.134:0/3198368325 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe5dc07a680 con 0x7fe5dc02bb80 2024-09-16T15:03:19.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5daffd640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 msgr2=0x7fe5dc02bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5daffd640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 0x7fe5dc02bf60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00096f0 tx=0x7fe5d0009190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5d004c040 con 0x7fe5dc02bb80 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 10 monclient: dump: 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.361 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fe5d0051440 con 0x7fe5dc02bb80 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 msgr2=0x7fe5dc02bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 0x7fe5dc02bf60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00096f0 tx=0x7fe5d0009190 comp rx=0 tx=0).stop 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 msgr2=0x7fe5dc02bf60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:19.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.347+0000 7fe5c3fff640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe5dc02bb80 0x7fe5dc02bf60 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00096f0 tx=0x7fe5d0009190 comp rx=0 tx=0).stop 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 28, 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:03:19.363 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.364 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.365 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:19.366 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:19.367 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.368 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:19.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:19.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.375 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:19.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:19.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:19.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.384 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:19.385 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:19.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:19.386 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:19.386 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:19.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5c8038880 msgr2=0x7fe5c803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3198368325 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5c8038880 0x7fe5c803ad40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc0a2f10 tx=0x7fe5d4009780 comp rx=0 tx=0).stop 2024-09-16T15:03:19.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 10 monclient: shutdown 2024-09-16T15:03:19.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 msgr2=0x7fe5dc135b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 --2- 172.21.15.134:0/3198368325 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe5dc136e40 0x7fe5dc135b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c460 tx=0x7fe5cc00c920 comp rx=0 tx=0).stop 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5dbfff640 1 -- 172.21.15.134:0/3198368325 reap_dead start 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5dbfff640 1 -- 172.21.15.134:0/3198368325 reap_dead start 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 shutdown_connections 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 >> 172.21.15.134:0/3198368325 conn(0x7fe5dc0a6110 msgr2=0x7fe5dc13a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 shutdown_connections 2024-09-16T15:03:19.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.348+0000 7fe5e1f8a640 1 -- 172.21.15.134:0/3198368325 wait complete. 2024-09-16T15:03:19.388 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:03:19.389 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'e', 'h', 'b'] 2024-09-16T15:03:19.389 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:03:19.389 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-16T15:03:19.389 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:03:19.389 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-16T15:03:19.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:03:19.400 INFO:tasks.ceph.mon.f:Stopped 2024-09-16T15:03:19.400 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-16T15:03:19.400 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:03:19.400 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-16T15:03:19.400 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:03:19.411 INFO:tasks.ceph.mon.e:Stopped 2024-09-16T15:03:19.411 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-16T15:03:19.411 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-16T15:03:19.412 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-16T15:03:19.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:03:19.452 INFO:tasks.ceph.mon.h:Stopped 2024-09-16T15:03:19.452 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-16T15:03:19.452 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:03:19.453 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:03:19.453 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:03:19.462 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:03:19.463 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-16T15:03:19.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.595+0000 7f5b307ec640 10 monclient: get_monmap_and_config 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.595+0000 7f5b307ec640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.595+0000 7f5b307ec640 10 monclient: monmap: 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:19.596751+0000 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:19.596751+0000 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.596+0000 7f5b307ec640 1 Processor -- start 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.596+0000 7f5b307ec640 1 -- start start 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.596+0000 7f5b307ec640 10 monclient: init 2024-09-16T15:03:19.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient: _add_conns ranks=[3,6,4,1,7,8,2,5,0,10,9] 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b28136030 0x7f5b28136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-g con 0x7f5b28136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-d con 0x7f5b2809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-c con 0x7f5b28136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2809b600 con 0x7f5b28136030 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280a5740 con 0x7f5b28136c60 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280a52c0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b307ec640 10 monclient(hunting): authenticate will time out at 936.186171s 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:54578/0 (socket says 172.21.15.134:54578) 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 -- 172.21.15.134:0/4184023467 learned_addr learned my addr 172.21.15.134:0/4184023467 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54278/0 (socket says 172.21.15.134:54278) 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): get_auth_request con 0x7f5b2809bfe0 auth_method 0 2024-09-16T15:03:19.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): get_auth_request con 0x7f5b28136c60 auth_method 0 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b28136030 0x7f5b28136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): get_auth_request con 0x7f5b28136030 auth_method 0 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient(hunting): handle_auth_done global_id 34233 payload 274 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 -- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 msgr2=0x7f5b281370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 -- 172.21.15.134:0/4184023467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b28136030 msgr2=0x7f5b28136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b28136030 0x7f5b28136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 monclient: found mon.noname-d 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2dd87640 1 -- 172.21.15.134:0/4184023467 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28073250 con 0x7f5b2809bfe0 2024-09-16T15:03:19.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.597+0000 7f5b2ed89640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2dd87640 1 -- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 msgr2=0x7f5b2809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098a0 tx=0x7f5b1802a080 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bb11155750debede server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 1 -- 172.21.15.134:0/4184023467 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b1802ade0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: dump: 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.621 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.622 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 1 -- 172.21.15.134:0/4184023467 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b180307b0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b307ec640 5 monclient: authenticate success, global_id 34233 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b307ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b307ec640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 1 -- 172.21.15.134:0/4184023467 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b18030dd0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b307ec640 4 set_mon_vals no callback set 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.600+0000 7f5b2d586640 10 monclient: dump: 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.625 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.626 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 10 monclient: shutdown 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 -- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 msgr2=0x7f5b2809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098a0 tx=0x7f5b1802a080 comp rx=0 tx=0).stop 2024-09-16T15:03:19.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 -- 172.21.15.134:0/4184023467 shutdown_connections 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b28136c60 0x7f5b281370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5b2809bfe0 0x7f5b2809c400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 --2- 172.21.15.134:0/4184023467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b28136030 0x7f5b28136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 -- 172.21.15.134:0/4184023467 >> 172.21.15.134:0/4184023467 conn(0x7f5b280a9ea0 msgr2=0x7f5b280a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 -- 172.21.15.134:0/4184023467 shutdown_connections 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 1 -- 172.21.15.134:0/4184023467 wait complete. 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 10 monclient: build_initial_monmap 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.601+0000 7f5b307ec640 10 monclient: monmap: 2024-09-16T15:03:19.628 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:19.629 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 Processor -- start 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 -- start start 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient: init 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient: _add_conns ranks=[4,7,5,8,2,3,0,1,6] 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-c con 0x7f5b2809bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b28136030 0x7f5b28134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-i con 0x7f5b28136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 0x7f5b28135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): picked mon.noname-h con 0x7f5b28136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28122200 con 0x7f5b2809bfe0 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2809fe50 con 0x7f5b28136c60 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:19.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5b28121f20 con 0x7f5b28136030 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b307ec640 10 monclient(hunting): authenticate will time out at 936.191681s 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54286/0 (socket says 172.21.15.134:54286) 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 -- 172.21.15.134:0/3885382891 learned_addr learned my addr 172.21.15.134:0/3885382891 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): get_auth_request con 0x7f5b2809bfe0 auth_method 0 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2ed89640 1 -- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 msgr2=0x7f5b28135340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2ed89640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 0x7f5b28135340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b28136030 0x7f5b28134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2dd87640 10 monclient(hunting): get_auth_request con 0x7f5b28136030 auth_method 0 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2dd87640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2dd87640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2dd87640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 10 monclient(hunting): handle_auth_done global_id 4648 payload 824 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 -- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b28136030 msgr2=0x7f5b28134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b28136030 0x7f5b28134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 -- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 msgr2=0x7f5b28135340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:19.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.602+0000 7f5b2e588640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 0x7f5b28135340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.603+0000 7f5b2e588640 10 monclient: _finish_hunting 0 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.603+0000 7f5b2e588640 1 monclient: found mon.noname-c 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.603+0000 7f5b2e588640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.603+0000 7f5b2e588640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.603+0000 7f5b2e588640 1 -- 172.21.15.134:0/3885382891 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b20009a70 con 0x7f5b2809bfe0 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b2e588640 1 -- 172.21.15.134:0/3885382891 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 msgr2=0x7f5b281bebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:19.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b2e588640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400e9b0 tx=0x7f5b2400ee70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b24009670 con 0x7f5b2809bfe0 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: dump: 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.637 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.638 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: _finish_auth 0 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2400cbb0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 5 monclient: authenticate success, global_id 4648 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 10 monclient: _renew_subs 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b180094d0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b24013d50 con 0x7f5b2809bfe0 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b1effd640 10 monclient: dump: 2024-09-16T15:03:19.640 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:19.641 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:19.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b2cd85640 4 set_mon_vals no callback set 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 10 monclient: _renew_subs 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.605+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2809f990 con 0x7f5b2809bfe0 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5b2400cd20 con 0x7f5b2809bfe0 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b307ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b307ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b307ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b281cb6e0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b1effd640 1 --2- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 0x7f5b0c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f5b2404a4f0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 0x7f5b0c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.606+0000 7f5b2dd87640 10 monclient: get_auth_request con 0x7f5b0c038880 auth_method 0 2024-09-16T15:03:19.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.609+0000 7f5b2dd87640 1 --2- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 0x7f5b0c03ad40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:19.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.611+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5b281cb6e0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.611+0000 7f5b1effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:19.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.611+0000 7f5b1effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:19.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.836+0000 7f5b307ec640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.836+0000 7f5b307ec640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.836+0000 7f5b307ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.836+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5b281cb6e0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.837+0000 7f5b1effd640 1 -- 172.21.15.134:0/3885382891 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f5b281cb6e0 con 0x7f5b2809bfe0 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.837+0000 7f5b1effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:19.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.837+0000 7f5b1effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:19.839 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 msgr2=0x7f5b0c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 0x7f5b0c03ad40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).stop 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 10 monclient: shutdown 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 msgr2=0x7f5b281bebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:19.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400e9b0 tx=0x7f5b2400ee70 comp rx=0 tx=0).stop 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 shutdown_connections 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b0c038880 0x7f5b0c03ad40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b28136c60 0x7f5b28135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b28136030 0x7f5b28134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 --2- 172.21.15.134:0/3885382891 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b2809bfe0 0x7f5b281bebd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 >> 172.21.15.134:0/3885382891 conn(0x7f5b280a9ea0 msgr2=0x7f5b2806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 shutdown_connections 2024-09-16T15:03:19.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:19.839+0000 7f5b307ec640 1 -- 172.21.15.134:0/3885382891 wait complete. 2024-09-16T15:03:22.852 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.959+0000 7fc1311ce640 10 monclient: get_monmap_and_config 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.959+0000 7fc1311ce640 10 monclient: build_initial_monmap 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.959+0000 7fc1311ce640 10 monclient: monmap: 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:22.960390+0000 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:22.960390+0000 2024-09-16T15:03:22.961 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:22.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 Processor -- start 2024-09-16T15:03:22.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 -- start start 2024-09-16T15:03:22.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient: init 2024-09-16T15:03:22.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:22.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient: _add_conns ranks=[4,7,0,10,2,3,8,1,5,6,9] 2024-09-16T15:03:22.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-c con 0x7fc12c136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c09bfe0 0x7fc12c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-i con 0x7fc12c09bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc12c136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c073250 con 0x7fc12c136c60 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c09b600 con 0x7fc12c136030 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0a5740 con 0x7fc12c09bfe0 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc1311ce640 10 monclient(hunting): authenticate will time out at 939.549597s 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12b59e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc12c136c60 0x7fc12c1370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:52010/0 (socket says 172.21.15.134:52010) 2024-09-16T15:03:22.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12b59e640 1 -- 172.21.15.134:0/1478340299 learned_addr learned my addr 172.21.15.134:0/1478340299 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:54290/0 (socket says 172.21.15.134:54290) 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): get_auth_request con 0x7fc12c136030 auth_method 0 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12a59c640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c09bfe0 0x7fc12c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12a59c640 10 monclient(hunting): get_auth_request con 0x7fc12c09bfe0 auth_method 0 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.960+0000 7fc12a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:22.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 10 monclient(hunting): handle_auth_done global_id 4657 payload 274 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 1 -- 172.21.15.134:0/1478340299 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c09bfe0 msgr2=0x7fc12c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c09bfe0 0x7fc12c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 1 -- 172.21.15.134:0/1478340299 >> v1:172.21.15.134:6789/0 conn(0x7fc12c136c60 legacy=0x7fc12c1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 1 monclient: found mon.noname-c 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.961+0000 7fc12ad9d640 1 -- 172.21.15.134:0/1478340299 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0a52c0 con 0x7fc12c136030 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.963+0000 7fc12ad9d640 1 -- 172.21.15.134:0/1478340299 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 msgr2=0x7fc12c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.963+0000 7fc12ad9d640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0097d0 tx=0x7fc11c02a000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=198706e0798d3a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 1 -- 172.21.15.134:0/1478340299 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc11c02ad20 con 0x7fc12c136030 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:22.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: dump: 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:22.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: _finish_auth 0 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 1 -- 172.21.15.134:0/1478340299 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c0307b0 con 0x7fc12c136030 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 5 monclient: authenticate success, global_id 4657 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 1 -- 172.21.15.134:0/1478340299 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc11c02aea0 con 0x7fc12c136030 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 4 set_mon_vals no callback set 2024-09-16T15:03:22.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc129d9b640 10 monclient: dump: 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:22.972 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 10 monclient: shutdown 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 -- 172.21.15.134:0/1478340299 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 msgr2=0x7fc12c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0097d0 tx=0x7fc11c02a000 comp rx=0 tx=0).stop 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 -- 172.21.15.134:0/1478340299 shutdown_connections 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c09bfe0 0x7fc12c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 --2- 172.21.15.134:0/1478340299 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc12c136030 0x7fc12c136410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 -- 172.21.15.134:0/1478340299 >> 172.21.15.134:0/1478340299 conn(0x7fc12c0a9ea0 msgr2=0x7fc12c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 -- 172.21.15.134:0/1478340299 shutdown_connections 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.964+0000 7fc1311ce640 1 -- 172.21.15.134:0/1478340299 wait complete. 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.965+0000 7fc1311ce640 10 monclient: build_initial_monmap 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.965+0000 7fc1311ce640 10 monclient: monmap: 2024-09-16T15:03:22.973 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:22.974 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:22.975 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 Processor -- start 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 -- start start 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient: init 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient: _add_conns ranks=[2,7,6,0,8,5,3,1,4] 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 0x7fc12c134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-b con 0x7fc12c09bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c136030 0x7fc12c135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-i con 0x7fc12c136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): picked mon.noname-d con 0x7fc12c136c60 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12ad9d640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 msgr2=0x7fc12c134dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:22.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c120bf0 con 0x7fc12c09bfe0 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c122250 con 0x7fc12c136c60 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0a0350 con 0x7fc12c136030 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12ad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 0x7fc12c134dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc1311ce640 10 monclient(hunting): authenticate will time out at 939.555608s 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:54590/0 (socket says 172.21.15.134:54590) 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 1 -- 172.21.15.134:0/4268715940 learned_addr learned my addr 172.21.15.134:0/4268715940 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): get_auth_request con 0x7fc12c136c60 auth_method 0 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:22.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12a59c640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c136030 0x7fc12c135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12a59c640 10 monclient(hunting): get_auth_request con 0x7fc12c136030 auth_method 0 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.966+0000 7fc12a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 10 monclient(hunting): handle_auth_done global_id 34242 payload 824 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 -- 172.21.15.134:0/4268715940 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c136030 msgr2=0x7fc12c135300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:22.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c136030 0x7fc12c135300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 -- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 msgr2=0x7fc12c134dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 0x7fc12c134dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 10 monclient: _finish_hunting 0 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 monclient: found mon.noname-d 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.967+0000 7fc12b59e640 1 -- 172.21.15.134:0/4268715940 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c009400 con 0x7fc12c136c60 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc12b59e640 1 -- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 msgr2=0x7fc12c1cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc12b59e640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc12400b970 tx=0x7fc12400be30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc124012340 con 0x7fc12c136c60 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: dump: 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:22.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:22.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1240124c0 con 0x7fc12c136c60 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 5 monclient: authenticate success, global_id 34242 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 10 monclient: _renew_subs 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c09feb0 con 0x7fc12c136c60 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc124012aa0 con 0x7fc12c136c60 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1237fe640 10 monclient: dump: 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:22.984 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:22.985 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc12959a640 4 set_mon_vals no callback set 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 10 monclient: _renew_subs 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.969+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c121ee0 con 0x7fc12c136c60 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc124012c80 con 0x7fc12c136c60 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1311ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1311ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1311ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c09c400 con 0x7fc12c136c60 2024-09-16T15:03:22.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1237fe640 1 --2- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 0x7fc10c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc124049ac0 con 0x7fc12c136c60 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc12ad9d640 1 --2- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 0x7fc10c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.970+0000 7fc12ad9d640 10 monclient: get_auth_request con 0x7fc10c0388b0 auth_method 0 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.973+0000 7fc12ad9d640 1 --2- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 0x7fc10c03ad70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc12c06ce90 tx=0x7fc11c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.975+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc12401b8f0 con 0x7fc12c136c60 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.975+0000 7fc1237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:22.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:22.975+0000 7fc1237fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1311ce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1311ce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1311ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc12c072070 con 0x7fc12c136c60 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1237fe640 1 -- 172.21.15.134:0/4268715940 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fc124019070 con 0x7fc12c136c60 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.199+0000 7fc1237fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:23.201 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:23.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.201+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 msgr2=0x7fc10c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:23.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.201+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 0x7fc10c03ad70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc12c06ce90 tx=0x7fc11c02e000 comp rx=0 tx=0).stop 2024-09-16T15:03:23.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.201+0000 7fc1311ce640 10 monclient: shutdown 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.201+0000 7fc1311ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.201+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 msgr2=0x7fc12c1cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc12400b970 tx=0x7fc12400be30 comp rx=0 tx=0).stop 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 shutdown_connections 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> v2:172.21.15.162:6800/980547263 conn(0x7fc10c0388b0 0x7fc10c03ad70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc12c136c60 0x7fc12c1cb270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc12c136030 0x7fc12c135300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 --2- 172.21.15.134:0/4268715940 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc12c09bfe0 0x7fc12c134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 >> 172.21.15.134:0/4268715940 conn(0x7fc12c0a9ea0 msgr2=0x7fc12c09d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 shutdown_connections 2024-09-16T15:03:23.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:23.202+0000 7fc1311ce640 1 -- 172.21.15.134:0/4268715940 wait complete. 2024-09-16T15:03:26.214 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:26.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.322+0000 7f6a6b261640 10 monclient: get_monmap_and_config 2024-09-16T15:03:26.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.322+0000 7f6a6b261640 10 monclient: build_initial_monmap 2024-09-16T15:03:26.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.322+0000 7f6a6b261640 10 monclient: monmap: 2024-09-16T15:03:26.323 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:26.323094+0000 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:26.323094+0000 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:26.324 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:26.325 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:26.325 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.322+0000 7f6a6b261640 1 Processor -- start 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 -- start start 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient: init 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient: _add_conns ranks=[3,8,6,5,4,7,1,10,0,2,9] 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a640a1c30 0x7f6a640a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-g con 0x7f6a640a1c30 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 0x7f6a640a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-e con 0x7f6a6406aae0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-d con 0x7f6a641339e0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406a8d0 con 0x7f6a640a1c30 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6409ba00 con 0x7f6a641339e0 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a2a90 con 0x7f6a6406aae0 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a6b261640 10 monclient(hunting): authenticate will time out at 942.912567s 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a5bfff640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 msgr2=0x7f6a640a2550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 0x7f6a640a2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:26.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:53762/0 (socket says 172.21.15.134:53762) 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 -- 172.21.15.134:0/3870123648 learned_addr learned my addr 172.21.15.134:0/3870123648 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): get_auth_request con 0x7f6a641339e0 auth_method 0 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a68ffd640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a640a1c30 0x7f6a640a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a68ffd640 10 monclient(hunting): get_auth_request con 0x7f6a640a1c30 auth_method 0 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a68ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a68ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a68ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:26.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient(hunting): handle_auth_done global_id 34251 payload 274 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 -- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 msgr2=0x7f6a640a2550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 0x7f6a640a2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 -- 172.21.15.134:0/3870123648 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a640a1c30 msgr2=0x7f6a640a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a640a1c30 0x7f6a640a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 monclient: found mon.noname-d 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:26.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.323+0000 7f6a697fe640 1 -- 172.21.15.134:0/3870123648 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64073250 con 0x7f6a641339e0 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.326+0000 7f6a697fe640 1 -- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 msgr2=0x7f6a64133e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.326+0000 7f6a697fe640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009eb0 tx=0x7f6a5c02a580 comp rx=0 tx=0).ready entity=mon.6 client_cookie=7190917dce3f4528 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 1 -- 172.21.15.134:0/3870123648 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a5c007a70 con 0x7f6a641339e0 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: dump: 2024-09-16T15:03:26.336 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:26.337 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 5 monclient: authenticate success, global_id 34251 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 1 -- 172.21.15.134:0/3870123648 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c02d050 con 0x7f6a641339e0 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 4 set_mon_vals no callback set 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 1 -- 172.21.15.134:0/3870123648 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a5c007bf0 con 0x7f6a641339e0 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a5b7fe640 10 monclient: dump: 2024-09-16T15:03:26.338 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:26.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 10 monclient: shutdown 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 -- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 msgr2=0x7f6a64133e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009eb0 tx=0x7f6a5c02a580 comp rx=0 tx=0).stop 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 -- 172.21.15.134:0/3870123648 shutdown_connections 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6a641339e0 0x7f6a64133e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6a6406aae0 0x7f6a640a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 --2- 172.21.15.134:0/3870123648 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a640a1c30 0x7f6a640a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 -- 172.21.15.134:0/3870123648 >> 172.21.15.134:0/3870123648 conn(0x7f6a64065f50 msgr2=0x7f6a64066360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 -- 172.21.15.134:0/3870123648 shutdown_connections 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.327+0000 7f6a6b261640 1 -- 172.21.15.134:0/3870123648 wait complete. 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.328+0000 7f6a6b261640 10 monclient: build_initial_monmap 2024-09-16T15:03:26.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.328+0000 7f6a6b261640 10 monclient: monmap: 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:26.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.328+0000 7f6a6b261640 1 Processor -- start 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.328+0000 7f6a6b261640 1 -- start start 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.328+0000 7f6a6b261640 10 monclient: init 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient: _add_conns ranks=[3,7,5,1,2,4,6,8,0] 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-g con 0x7f6a6406aae0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 0x7f6a64132d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-i con 0x7f6a640a1c30 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 0x7f6a6411c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): picked mon.noname-h con 0x7f6a641339e0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:26.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6411e120 con 0x7f6a6406aae0 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640a59f0 con 0x7f6a641339e0 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6409f9e0 con 0x7f6a640a1c30 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a6b261640 10 monclient(hunting): authenticate will time out at 942.918525s 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a697fe640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 msgr2=0x7f6a6411c870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a697fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 0x7f6a6411c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 0x7f6a64132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:52916/0 (socket says 172.21.15.134:52916) 2024-09-16T15:03:26.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 1 -- 172.21.15.134:0/1486960867 learned_addr learned my addr 172.21.15.134:0/1486960867 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): get_auth_request con 0x7f6a6406aae0 auth_method 0 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): get_auth_request con 0x7f6a640a1c30 auth_method 0 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a68ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:26.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.329+0000 7f6a5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 10 monclient(hunting): handle_auth_done global_id 4588 payload 824 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 -- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 msgr2=0x7f6a6411c870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 0x7f6a6411c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 -- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 msgr2=0x7f6a641cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 monclient: found mon.noname-i 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a5bfff640 1 -- 172.21.15.134:0/1486960867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009b30 con 0x7f6a640a1c30 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.330+0000 7f6a68ffd640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:26.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a5bfff640 1 -- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 msgr2=0x7f6a64132d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a5bfff640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 0x7f6a64132d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b3e0 tx=0x7f6a5400b8a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a54003da0 con 0x7f6a640a1c30 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: dump: 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:26.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: _finish_auth 0 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: _check_auth_tickets 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540054e0 con 0x7f6a640a1c30 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 5 monclient: authenticate success, global_id 4588 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 10 monclient: _renew_subs 2024-09-16T15:03:26.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a641204f0 con 0x7f6a640a1c30 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6a54005650 con 0x7f6a640a1c30 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a597fa640 10 monclient: dump: 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:26.348 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 10 monclient: _renew_subs 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6411ff30 con 0x7f6a640a1c30 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.332+0000 7f6a5affd640 4 set_mon_vals no callback set 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.333+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6a54005070 con 0x7f6a640a1c30 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.333+0000 7f6a6b261640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.333+0000 7f6a6b261640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:26.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.333+0000 7f6a6b261640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.333+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64120150 con 0x7f6a640a1c30 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.334+0000 7f6a597fa640 1 --2- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.334+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f6a54049150 con 0x7f6a640a1c30 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.334+0000 7f6a68ffd640 1 --2- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.334+0000 7f6a68ffd640 10 monclient: get_auth_request con 0x7f6a50038880 auth_method 0 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.336+0000 7f6a68ffd640 1 --2- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.339+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6a540103a0 con 0x7f6a640a1c30 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.340+0000 7f6a597fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:26.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.340+0000 7f6a597fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.564+0000 7f6a6b261640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.564+0000 7f6a6b261640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.564+0000 7f6a6b261640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.564+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6a64067000 con 0x7f6a640a1c30 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.565+0000 7f6a597fa640 1 -- 172.21.15.134:0/1486960867 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f6a54016070 con 0x7f6a640a1c30 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.565+0000 7f6a597fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:26.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.565+0000 7f6a597fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:26.570 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:26.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 msgr2=0x7f6a5003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 10 monclient: shutdown 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 msgr2=0x7f6a64132d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 0x7f6a64132d60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b3e0 tx=0x7f6a5400b8a0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 shutdown_connections 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6a641339e0 0x7f6a6411c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6a640a1c30 0x7f6a64132d60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> v2:172.21.15.162:6800/980547263 conn(0x7f6a50038880 0x7f6a5003ad40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 --2- 172.21.15.134:0/1486960867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a6406aae0 0x7f6a641cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 >> 172.21.15.134:0/1486960867 conn(0x7f6a64065f50 msgr2=0x7f6a641beea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 shutdown_connections 2024-09-16T15:03:26.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:26.567+0000 7f6a6b261640 1 -- 172.21.15.134:0/1486960867 wait complete. 2024-09-16T15:03:29.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.686+0000 7f6205765640 10 monclient: get_monmap_and_config 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.686+0000 7f6205765640 10 monclient: build_initial_monmap 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.686+0000 7f6205765640 10 monclient: monmap: 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:29.688860+0000 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:29.688860+0000 2024-09-16T15:03:29.689 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:29.690 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:29.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.687+0000 7f6205765640 1 Processor -- start 2024-09-16T15:03:29.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.687+0000 7f6205765640 1 -- start start 2024-09-16T15:03:29.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.687+0000 7f6205765640 10 monclient: init 2024-09-16T15:03:29.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:29.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient: _add_conns ranks=[7,8,2,0,9,10,4,1,3,5,6] 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): picked mon.noname-i con 0x7f6200136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): picked mon.noname-e con 0x7f620009bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): picked mon.noname-b con 0x7f6200136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620009b600 con 0x7f6200136c60 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f62000a5740 con 0x7f6200136030 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f62000a52c0 con 0x7f620009bfe0 2024-09-16T15:03:29.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f6205765640 10 monclient(hunting): authenticate will time out at 946.278257s 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61ff7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 msgr2=0x7f62001370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61f67fc640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 msgr2=0x7f620009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61f67fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61ff7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:48286/0 (socket says 172.21.15.134:48286) 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 learned_addr learned my addr 172.21.15.134:0/2650649754 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f6200136030 auth_method 0 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:29.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:29.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:29.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.688+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:29.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 msgr2=0x7f62001370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:29.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 msgr2=0x7f620009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:29.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:29.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f6200136030 auth_method 0 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:29.889+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 msgr2=0x7f62001370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 msgr2=0x7f620009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f6200136030 auth_method 0 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:30.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:30.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:30.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:30.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:30.290+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:31.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.091+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 msgr2=0x7f62001370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:31.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.091+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 msgr2=0x7f620009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:31.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.091+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.091+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.091+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f6200136030 auth_method 0 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:31.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:31.092+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:32.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): tick 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 monclient(hunting): continuing hunt 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 msgr2=0x7f620009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f620009bfe0 0x7f620009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 msgr2=0x7f6200136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6200136030 0x7f6200136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 msgr2=0x7f62001370c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6200136c60 0x7f62001370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:32.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient: _add_conns ranks=[3,1,6,4,10,5,9,0] 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-g con 0x7f61e0003820 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f61e0006250 addr v1:172.21.15.162:6789/0 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-d con 0x7f61e0009f20 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6200073250 con 0x7f61e0006250 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620009b600 con 0x7f61e0003820 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f62000a5740 con 0x7f61e0009f20 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61fd7fa640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:32.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request con 0x7f61e0009f20 auth_method 0 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.162:6789/0 conn(0x7f61e0006250 legacy=0x7f61e00086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e0003820 auth_method 0 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:32.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:32.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:32.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:32.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.689+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:32.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:32.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request con 0x7f61e0009f20 auth_method 0 2024-09-16T15:03:32.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.162:6789/0 conn(0x7f61e0006250 legacy=0x7f61e00086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:03:32.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:32.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e0003820 auth_method 0 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:32.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:32.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:32.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:32.890+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.162:6789/0 conn(0x7f61e0006250 legacy=0x7f61e00086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request con 0x7f61e0009f20 auth_method 0 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e0003820 auth_method 0 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:33.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:33.291+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.162:6789/0 conn(0x7f61e0006250 legacy=0x7f61e00086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request con 0x7f61e0009f20 auth_method 0 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:34.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e0003820 auth_method 0 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:34.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:34.092+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:35.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): tick 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 monclient(hunting): continuing hunt 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61e0009f20 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61e0003820 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.162:6789/0 conn(0x7f61e0006250 legacy=0x7f61e00086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 reap_dead start 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient: _add_conns ranks=[9,10,0,4,5] 2024-09-16T15:03:35.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 reap_dead start 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-a con 0x7f61e000cbf0 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-f con 0x7f61e000d070 addr v2:172.21.15.162:3300/0 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f61e000d600 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e000dc10 con 0x7f61e000d600 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620009b600 con 0x7f61e000cbf0 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 1 -- 172.21.15.134:0/2650649754 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62000a5740 con 0x7f61e000d070 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.689+0000 7f61fd7fa640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:35.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e000cbf0 auth_method 0 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.690+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:35.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.692+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.134:6789/0 conn(0x7f61e000d600 legacy=0x7f61e0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:35.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e000cbf0 auth_method 0 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:35.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:35.891+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e000cbf0 auth_method 0 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:36.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:36.292+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61e000cbf0 auth_method 0 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:03:37.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:37.093+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139702618 0 0) 0x7f61e000dc10 con 0x7f61e000d600 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 10 monclient(hunting): _negotiate 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 10 monclient(hunting): my global_id is 4671 2024-09-16T15:03:38.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.628+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ec003520 con 0x7f61e000d600 2024-09-16T15:03:38.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36581924 0 0) 0x7f61ec003520 con 0x7f61e000d600 2024-09-16T15:03:38.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 msgr2=0x7f61e000c3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.162:3300/0 conn(0x7f61e000d070 0x7f61e000c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 msgr2=0x7f61e0005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 --2- 172.21.15.134:0/2650649754 >> v2:172.21.15.134:3300/0 conn(0x7f61e000cbf0 0x7f61e0005c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200073250 con 0x7f61e000d600 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f61f800d380 con 0x7f61e000d600 2024-09-16T15:03:38.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f6205765640 5 monclient: authenticate success, global_id 4671 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f6205765640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: dump: 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:38.635 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 >> v1:172.21.15.134:6789/0 conn(0x7f61e000d600 legacy=0x7f61e0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient: _add_conns ranks=[7,3,5,1,0,6,2,4,8] 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61ec0041c0 0x7f61ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): picked mon.i con 0x7f61ec0041c0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 0x7f61ec005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): picked mon.g con 0x7f61ec004dd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec009c90 0x7f61ec00c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): picked mon.h con 0x7f61ec009c90 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620009b600 con 0x7f61ec004dd0 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62000a5740 con 0x7f61ec009c90 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec00ca30 con 0x7f61ec0041c0 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61fe7fc640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f6205765640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:38.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec009c90 msgr2=0x7f61ec00c120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61ff7fe640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec009c90 0x7f61ec00c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 0x7f61ec005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): get_auth_request con 0x7f61ec004dd0 auth_method 0 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61ec0041c0 0x7f61ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61feffd640 10 monclient(hunting): get_auth_request con 0x7f61ec0041c0 auth_method 0 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61feffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61feffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61feffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.629+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61feffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61feffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61feffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient(hunting): handle_auth_done global_id 4671 payload 293 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61ec0041c0 msgr2=0x7f61ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61ec0041c0 0x7f61ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec009c90 msgr2=0x7f61ec00c120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec009c90 0x7f61ec00c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient: _finish_hunting 0 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 monclient: found mon.g 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:38.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61ff7fe640 1 -- 172.21.15.134:0/2650649754 reap_dead start 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec003520 con 0x7f61ec004dd0 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient: _finish_auth 0 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:38.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.630+0000 7f61feffd640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61ec0041c0 0x7f61ec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:38.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61f67fc640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 msgr2=0x7f61ec005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61f67fc640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 0x7f61ec005250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009ec0 tx=0x7f61ec0057f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a812ba27aaaebf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61e8005750 con 0x7f61ec004dd0 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: dump: 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:38.646 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f6205765640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80058d0 con 0x7f61ec004dd0 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f6205765640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f6205765640 4 set_mon_vals no callback set 2024-09-16T15:03:38.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 1 -- 172.21.15.134:0/2650649754 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61e8005a40 con 0x7f61ec004dd0 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.633+0000 7f61fe7fc640 10 monclient: dump: 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:38.648 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 10 monclient: shutdown 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 -- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 msgr2=0x7f61ec005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 0x7f61ec005250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009ec0 tx=0x7f61ec0057f0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 -- 172.21.15.134:0/2650649754 shutdown_connections 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 --2- 172.21.15.134:0/2650649754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61ec004dd0 0x7f61ec005250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 -- 172.21.15.134:0/2650649754 >> 172.21.15.134:0/2650649754 conn(0x7f62000a9ea0 msgr2=0x7f62000a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:38.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 -- 172.21.15.134:0/2650649754 shutdown_connections 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.634+0000 7f6205765640 1 -- 172.21.15.134:0/2650649754 wait complete. 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.635+0000 7f6205765640 10 monclient: build_initial_monmap 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.635+0000 7f6205765640 10 monclient: monmap: 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:38.650 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.635+0000 7f6205765640 1 Processor -- start 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 -- start start 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient: init 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient: _add_conns ranks=[5,3,0,8,6,7,2,1,4] 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 0x7f62001caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): picked mon.noname-h con 0x7f61ec004dd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): picked mon.noname-g con 0x7f6200134d90 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6200122830 0x7f6200122c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): picked mon.noname-a con 0x7f6200122830 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200125ef0 con 0x7f6200122830 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62001243c0 con 0x7f6200134d90 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6200124b10 con 0x7f61ec004dd0 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f6205765640 10 monclient(hunting): authenticate will time out at 955.225936s 2024-09-16T15:03:38.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f61feffd640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 msgr2=0x7f62001caf80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.636+0000 7f61feffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 0x7f62001caf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6200122830 0x7f6200122c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:44736/0 (socket says 172.21.15.134:44736) 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 -- 172.21.15.134:0/1854832386 learned_addr learned my addr 172.21.15.134:0/1854832386 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): get_auth_request con 0x7f6200134d90 auth_method 0 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request con 0x7f6200122830 auth_method 0 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:38.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61ff7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient(hunting): handle_auth_done global_id 34293 payload 824 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 -- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 msgr2=0x7f62001caf80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 0x7f62001caf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 -- 172.21.15.134:0/1854832386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6200122830 msgr2=0x7f6200122c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6200122830 0x7f6200122c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient: _finish_hunting 0 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 monclient: found mon.noname-g 2024-09-16T15:03:38.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.637+0000 7f61f67fc640 1 -- 172.21.15.134:0/1854832386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000a7aa0 con 0x7f6200134d90 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.639+0000 7f61f67fc640 1 -- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 msgr2=0x7f6200135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.639+0000 7f61f67fc640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61e0003770 tx=0x7f61e8005c20 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61e80092b0 con 0x7f6200134d90 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: dump: 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:38.655 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:38.656 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: _finish_auth 0 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8005a40 con 0x7f6200134d90 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 5 monclient: authenticate success, global_id 34293 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61e803ecc0 con 0x7f6200134d90 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61f7fff640 10 monclient: dump: 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:38.657 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:38.658 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f61fdffb640 4 set_mon_vals no callback set 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 10 monclient: _renew_subs 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62001267f0 con 0x7f6200134d90 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 10 monclient: _renew_subs 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.640+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62000a7f40 con 0x7f6200134d90 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f61e8033bc0 con 0x7f6200134d90 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f6205765640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f6205765640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f6205765640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f61f7fff640 1 --2- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 0x7f61e003c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6200126030 con 0x7f6200134d90 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.641+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f61e806abb0 con 0x7f6200134d90 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.642+0000 7f61feffd640 1 --2- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 0x7f61e003c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.642+0000 7f61feffd640 10 monclient: get_auth_request con 0x7f61e003a400 auth_method 0 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.644+0000 7f61feffd640 1 --2- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 0x7f61e003c8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61e000cb00 tx=0x7f61f0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.647+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6200126030 con 0x7f6200134d90 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.647+0000 7f61f7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:38.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.647+0000 7f61f7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:38.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.874+0000 7f6205765640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:03:38.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.874+0000 7f6205765640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:38.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.874+0000 7f6205765640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:03:38.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.874+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6200126030 con 0x7f6200134d90 2024-09-16T15:03:38.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.875+0000 7f61f7fff640 1 -- 172.21.15.134:0/1854832386 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3171 (secure 0 0 0) 0x7f61e8008c80 con 0x7f6200134d90 2024-09-16T15:03:38.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.875+0000 7f61f7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:03:38.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.875+0000 7f61f7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:38.877 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":34,"quorum":[0,3,4,6,7],"quorum_names":["a","g","c","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 msgr2=0x7f61e003c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 0x7f61e003c8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61e000cb00 tx=0x7f61f0008040 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 10 monclient: shutdown 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 msgr2=0x7f6200135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61e0003770 tx=0x7f61e8005c20 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 shutdown_connections 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6200122830 0x7f6200122c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> v2:172.21.15.162:6800/980547263 conn(0x7f61e003a400 0x7f61e003c8c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6200134d90 0x7f6200135170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 --2- 172.21.15.134:0/1854832386 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61ec004dd0 0x7f62001caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 >> 172.21.15.134:0/1854832386 conn(0x7f62000a9ea0 msgr2=0x7f62000a8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:38.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 shutdown_connections 2024-09-16T15:03:38.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.877+0000 7f6205765640 1 -- 172.21.15.134:0/1854832386 wait complete. 2024-09-16T15:03:38.889 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-16T15:03:38.890 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-16T15:03:38.890 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.985+0000 7fcae7fff640 10 monclient: get_monmap_and_config 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.985+0000 7fcae7fff640 10 monclient: build_initial_monmap 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.985+0000 7fcae7fff640 10 monclient: monmap: 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:38.986683+0000 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:38.986683+0000 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:38.987 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:38.988 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:38.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 Processor -- start 2024-09-16T15:03:38.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 -- start start 2024-09-16T15:03:38.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient: init 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient: _add_conns ranks=[9,0,4,8,3,7,1,10,5,2,6] 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 0x7fcae809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-a con 0x7fcae8135de0 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcae81361c0 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-c con 0x7fcae81365c0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae8073250 con 0x7fcae81361c0 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae809b620 con 0x7fcae8135de0 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80a5540 con 0x7fcae81365c0 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.986+0000 7fcae7fff640 10 monclient(hunting): authenticate will time out at 955.575893s 2024-09-16T15:03:38.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 0x7fcae809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41610/0 (socket says 172.21.15.134:41610) 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 -- 172.21.15.134:0/2307368922 learned_addr learned my addr 172.21.15.134:0/2307368922 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): get_auth_request con 0x7fcae81365c0 auth_method 0 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae67fc640 1 --1- 172.21.15.134:0/2307368922 >> v1:172.21.15.134:6789/0 conn(0x7fcae81361c0 0x7fcae80a3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45942/0 (socket says 172.21.15.134:45942) 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): get_auth_request con 0x7fcae8135de0 auth_method 0 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient(hunting): handle_auth_done global_id 4666 payload 274 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 -- 172.21.15.134:0/2307368922 >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 msgr2=0x7fcae809bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 --2- 172.21.15.134:0/2307368922 >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 0x7fcae809bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 -- 172.21.15.134:0/2307368922 >> v1:172.21.15.134:6789/0 conn(0x7fcae81361c0 legacy=0x7fcae80a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 monclient: found mon.noname-c 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae77fe640 1 -- 172.21.15.134:0/2307368922 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80a4e80 con 0x7fcae81365c0 2024-09-16T15:03:38.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.987+0000 7fcae6ffd640 1 --2- 172.21.15.134:0/2307368922 >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 0x7fcae809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:39.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae77fe640 1 -- 172.21.15.134:0/2307368922 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 msgr2=0x7fcae80a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:39.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae77fe640 1 --2- 172.21.15.134:0/2307368922 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009e50 tx=0x7fcae002a5b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=92cae4f0c1cfe2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 1 -- 172.21.15.134:0/2307368922 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcae0007a70 con 0x7fcae81365c0 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: dump: 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.001 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 1 -- 172.21.15.134:0/2307368922 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae002d050 con 0x7fcae81365c0 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 5 monclient: authenticate success, global_id 4666 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 1 -- 172.21.15.134:0/2307368922 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcae0007bf0 con 0x7fcae81365c0 2024-09-16T15:03:39.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 4 set_mon_vals no callback set 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae5ffb640 10 monclient: dump: 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.003 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 10 monclient: shutdown 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 -- 172.21.15.134:0/2307368922 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 msgr2=0x7fcae80a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 --2- 172.21.15.134:0/2307368922 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009e50 tx=0x7fcae002a5b0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 -- 172.21.15.134:0/2307368922 shutdown_connections 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 --2- 172.21.15.134:0/2307368922 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcae81365c0 0x7fcae80a43f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 --2- 172.21.15.134:0/2307368922 >> v2:172.21.15.134:3300/0 conn(0x7fcae8135de0 0x7fcae809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 -- 172.21.15.134:0/2307368922 >> 172.21.15.134:0/2307368922 conn(0x7fcae8066070 msgr2=0x7fcae8066480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 -- 172.21.15.134:0/2307368922 shutdown_connections 2024-09-16T15:03:39.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.990+0000 7fcae7fff640 1 -- 172.21.15.134:0/2307368922 wait complete. 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.991+0000 7fcae7fff640 10 monclient: build_initial_monmap 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.991+0000 7fcae7fff640 10 monclient: monmap: 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:39.005 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.991+0000 7fcae7fff640 1 Processor -- start 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.991+0000 7fcae7fff640 1 -- start start 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.991+0000 7fcae7fff640 10 monclient: init 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient: _add_conns ranks=[5,8,6,7,0,1,2,3,4] 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcae8135de0 0x7fcae81cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-h con 0x7fcae8135de0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcae81361c0 0x7fcae81cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-e con 0x7fcae81361c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): picked mon.noname-d con 0x7fcae81365c0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:39.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8121eb0 con 0x7fcae8135de0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80a73e0 con 0x7fcae81365c0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8122470 con 0x7fcae81361c0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae7fff640 10 monclient(hunting): authenticate will time out at 955.581354s 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae67fc640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcae81361c0 msgr2=0x7fcae81cb890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae67fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcae81361c0 0x7fcae81cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae6ffd640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcae8135de0 msgr2=0x7fcae81cb350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae6ffd640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcae8135de0 0x7fcae81cb350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:32814/0 (socket says 172.21.15.134:32814) 2024-09-16T15:03:39.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 learned_addr learned my addr 172.21.15.134:0/3143487699 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): get_auth_request con 0x7fcae81365c0 auth_method 0 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient(hunting): handle_auth_done global_id 34260 payload 824 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcae81361c0 msgr2=0x7fcae81cb890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcae81361c0 0x7fcae81cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcae8135de0 msgr2=0x7fcae81cb350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcae8135de0 0x7fcae81cb350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:39.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 monclient: found mon.noname-d 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.992+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8009400 con 0x7fcae81365c0 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 msgr2=0x7fcae81cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae77fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcae8122630 tx=0x7fcae0004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcae0004710 con 0x7fcae81365c0 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: dump: 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 5 monclient: authenticate success, global_id 34260 2024-09-16T15:03:39.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0007a70 con 0x7fcae81365c0 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 10 monclient: _renew_subs 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae0009b30 con 0x7fcae81365c0 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcae0007be0 con 0x7fcae81365c0 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcacf7fe640 10 monclient: dump: 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae57fa640 4 set_mon_vals no callback set 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 10 monclient: _renew_subs 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:39.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.995+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae8120f50 con 0x7fcae81365c0 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcae0007d40 con 0x7fcae81365c0 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae7fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae7fff640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 0x7fcae8134790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae8069fe0 con 0x7fcae81343b0 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 0x7fcae8134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient: get_auth_request con 0x7fcae81343b0 auth_method 0 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcacf7fe640 1 --2- 172.21.15.134:0/3143487699 >> v2:172.21.15.162:6800/980547263 conn(0x7fcad4038880 0x7fcad403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:39.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fcae006abd0 con 0x7fcae81365c0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_done global_id 4698 payload 824 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae6ffd640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcae8061ae0 con 0x7fcae81343b0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae67fc640 1 --2- 172.21.15.134:0/3143487699 >> v2:172.21.15.162:6800/980547263 conn(0x7fcad4038880 0x7fcad403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.996+0000 7fcae67fc640 10 monclient: get_auth_request con 0x7fcad4038880 auth_method 0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.998+0000 7fcae6ffd640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 msgr2=0x7fcae8134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.998+0000 7fcae6ffd640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 0x7fcae8134790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcad800e8b0 tx=0x7fcad800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcad8013470 con 0x7fcae81343b0 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 10 monclient: dump: 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.014 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.015 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcad800cb70 con 0x7fcae81343b0 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 msgr2=0x7fcae8134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 0x7fcae8134790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcad800e8b0 tx=0x7fcad800ed70 comp rx=0 tx=0).stop 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 msgr2=0x7fcae8134790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcacf7fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae81343b0 0x7fcae8134790 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcad800e8b0 tx=0x7fcad800ed70 comp rx=0 tx=0).stop 2024-09-16T15:03:39.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:38.999+0000 7fcae67fc640 1 --2- 172.21.15.134:0/3143487699 >> v2:172.21.15.162:6800/980547263 conn(0x7fcad4038880 0x7fcad403ad40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007fd0 tx=0x7fcadc007aa0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae7fff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae7fff640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 0x7fcae809d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae7fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae807a680 con 0x7fcae8134cd0 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 0x7fcae809d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient: get_auth_request con 0x7fcae8134cd0 auth_method 0 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 10 monclient(hunting): handle_auth_done global_id 4707 payload 824 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.005+0000 7fcae6ffd640 1 -- 172.21.15.134:0/3143487699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcae809b240 con 0x7fcae8134cd0 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae6ffd640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 msgr2=0x7fcae809d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae6ffd640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 0x7fcae809d600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcad8019000 tx=0x7fcad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcad8023070 con 0x7fcae8134cd0 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 10 monclient: dump: 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9319 (secure 0 0 0) 0x7fcad80067c0 con 0x7fcae8134cd0 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:39.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 msgr2=0x7fcae809d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 0x7fcae809d600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcad8019000 tx=0x7fcad8004730 comp rx=0 tx=0).stop 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 msgr2=0x7fcae809d600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcacf7fe640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae8134cd0 0x7fcae809d600 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcad8019000 tx=0x7fcad8004730 comp rx=0 tx=0).stop 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 34, 2024-09-16T15:03:39.022 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:39.023 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:39.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:39.025 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:39.026 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:39.028 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:39.030 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:39.032 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.034 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.035 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.036 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:39.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:39.041 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.042 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:39.043 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 >> v2:172.21.15.162:6800/980547263 conn(0x7fcad4038880 msgr2=0x7fcad403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 --2- 172.21.15.134:0/3143487699 >> v2:172.21.15.162:6800/980547263 conn(0x7fcad4038880 0x7fcad403ad40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007fd0 tx=0x7fcadc007aa0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 10 monclient: shutdown 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 msgr2=0x7fcae81cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcae8122630 tx=0x7fcae0004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae77fe640 1 -- 172.21.15.134:0/3143487699 reap_dead start 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 shutdown_connections 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 --2- 172.21.15.134:0/3143487699 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcae81365c0 0x7fcae81cbdd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 >> 172.21.15.134:0/3143487699 conn(0x7fcae8066070 msgr2=0x7fcae81bec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 shutdown_connections 2024-09-16T15:03:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.008+0000 7fcae7fff640 1 -- 172.21.15.134:0/3143487699 wait complete. 2024-09-16T15:03:39.046 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:03:39.046 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-09-16T15:03:39.046 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.142+0000 7f0c427aa640 10 monclient: get_monmap_and_config 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.142+0000 7f0c427aa640 10 monclient: build_initial_monmap 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.143+0000 7f0c427aa640 10 monclient: monmap: 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:39.143916+0000 2024-09-16T15:03:39.144 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:39.143916+0000 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:39.145 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.143+0000 7f0c427aa640 1 Processor -- start 2024-09-16T15:03:39.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.143+0000 7f0c427aa640 1 -- start start 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.143+0000 7f0c427aa640 10 monclient: init 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient: _add_conns ranks=[0,9,3,5,1,6,2,8,7,4,10] 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c3c136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c3c137220 0x7f0c3c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-a con 0x7f0c3c137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c3c13c370 0x7f0c3c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-g con 0x7f0c3c13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c073250 con 0x7f0c3c136e40 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c0a56e0 con 0x7f0c3c137220 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c09abc0 con 0x7f0c3c13c370 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c427aa640 10 monclient(hunting): authenticate will time out at 955.733386s 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c3c137220 0x7f0c3c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0c3c136e40 0x7f0c3c09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45944/0 (socket says 172.21.15.134:45944) 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3bfff640 1 -- 172.21.15.134:0/4187082317 learned_addr learned my addr 172.21.15.134:0/4187082317 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): get_auth_request con 0x7f0c3c137220 auth_method 0 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:39.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c40d47640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c3c13c370 0x7f0c3c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c40d47640 10 monclient(hunting): get_auth_request con 0x7f0c3c13c370 auth_method 0 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c40d47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c40d47640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c40d47640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_done global_id 4716 payload 274 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/4187082317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c3c13c370 0x7f0c3c13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/4187082317 >> v1:172.21.15.134:6789/0 conn(0x7f0c3c136e40 legacy=0x7f0c3c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 monclient: found mon.noname-a 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:39.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.144+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/4187082317 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c09ba50 con 0x7f0c3c137220 2024-09-16T15:03:39.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.147+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/4187082317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:39.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.147+0000 7f0c3b7fe640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3c137220 0x7f0c3c137600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0c300097a0 tx=0x7f0c3002a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c139810ac7ab0339 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.148+0000 7f0c3affd640 1 -- 172.21.15.134:0/4187082317 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c3002ad30 con 0x7f0c3c137220 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: dump: 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: _finish_auth 0 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 1 -- 172.21.15.134:0/4187082317 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c300307b0 con 0x7f0c3c137220 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 5 monclient: authenticate success, global_id 4716 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:39.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 1 -- 172.21.15.134:0/4187082317 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c30030d90 con 0x7f0c3c137220 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 4 set_mon_vals no callback set 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c3affd640 10 monclient: dump: 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:39.156 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 10 monclient: shutdown 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 -- 172.21.15.134:0/4187082317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3c137220 0x7f0c3c137600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0c300097a0 tx=0x7f0c3002a010 comp rx=0 tx=0).stop 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 -- 172.21.15.134:0/4187082317 shutdown_connections 2024-09-16T15:03:39.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c3c13c370 0x7f0c3c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 --2- 172.21.15.134:0/4187082317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c3c137220 0x7f0c3c137600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 -- 172.21.15.134:0/4187082317 >> 172.21.15.134:0/4187082317 conn(0x7f0c3c0a6110 msgr2=0x7f0c3c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 -- 172.21.15.134:0/4187082317 shutdown_connections 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.149+0000 7f0c427aa640 1 -- 172.21.15.134:0/4187082317 wait complete. 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.150+0000 7f0c427aa640 10 monclient: build_initial_monmap 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.150+0000 7f0c427aa640 10 monclient: monmap: 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:39.158 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.150+0000 7f0c427aa640 1 Processor -- start 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.150+0000 7f0c427aa640 1 -- start start 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.150+0000 7f0c427aa640 10 monclient: init 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:39.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient: _add_conns ranks=[5,2,1,6,7,0,3,8,4] 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-h con 0x7f0c3c136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-b con 0x7f0c3c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): picked mon.noname-f con 0x7f0c3c13c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1228e0 con 0x7f0c3c13c370 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0a75a0 con 0x7f0c3c137220 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0a0170 con 0x7f0c3c136e40 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c427aa640 10 monclient(hunting): authenticate will time out at 955.740329s 2024-09-16T15:03:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c3b7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c1366c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c3bfff640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 msgr2=0x7f0c3c136180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c40d47640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c1cb780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c40d47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.151+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:39.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c3b7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c1366c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:39.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:39.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c3bfff640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 msgr2=0x7f0c3c136180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:39.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:39.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c40d47640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c1cb780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:39.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.352+0000 7f0c40d47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:03:39.638 INFO:tasks.rados.rados.0.smithi162.stdout:588: done (1 left) 2024-09-16T15:03:39.638 INFO:tasks.rados.rados.0.smithi162.stdout:589: done (0 left) 2024-09-16T15:03:39.638 INFO:tasks.rados.rados.0.smithi162.stdout:590: rollback oid 9 current snap is 48 2024-09-16T15:03:39.638 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 9 to 47 2024-09-16T15:03:39.638 INFO:tasks.rados.rados.0.smithi162.stdout:591: snap_create 2024-09-16T15:03:39.642 INFO:tasks.rados.rados.0.smithi162.stdout:590: finishing rollback tid 0 to smithi16234619-9 2024-09-16T15:03:39.643 INFO:tasks.rados.rados.0.smithi162.stdout:590: finishing rollback tid 1 to smithi16234619-9 2024-09-16T15:03:39.645 INFO:tasks.rados.rados.0.smithi162.stdout:590: finishing rollback tid 2 to smithi16234619-9 2024-09-16T15:03:39.645 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 216 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:39.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c3b7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c1366c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:39.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:39.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c40d47640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c1cb780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c3bfff640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 msgr2=0x7f0c3c136180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c40d47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:39.753+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:03:40.373 INFO:tasks.rados.rados.0.smithi162.stdout:590: done (1 left) 2024-09-16T15:03:40.373 INFO:tasks.rados.rados.0.smithi162.stdout:591: done (0 left) 2024-09-16T15:03:40.373 INFO:tasks.rados.rados.0.smithi162.stdout:592: delete oid 46 current snap is 49 2024-09-16T15:03:40.378 INFO:tasks.rados.rados.0.smithi162.stdout:592: done (0 left) 2024-09-16T15:03:40.378 INFO:tasks.rados.rados.0.smithi162.stdout:593: read oid 27 snap -1 2024-09-16T15:03:40.378 INFO:tasks.rados.rados.0.smithi162.stdout:593: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:03:40.378 INFO:tasks.rados.rados.0.smithi162.stdout:594: snap_remove snap 47 2024-09-16T15:03:40.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c3b7fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c1366c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:40.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:40.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c40d47640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c1cb780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:40.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c3bfff640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 msgr2=0x7f0c3c136180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:40.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c40d47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:40.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:40.554+0000 7f0c3bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:594: done (1 left) 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:595: read oid 10 snap -1 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:595: expect deleted 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:596: rollback oid 23 current snap is 49 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 48 2024-09-16T15:03:41.377 INFO:tasks.rados.rados.0.smithi162.stdout:593: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:03:41.574 INFO:tasks.rados.rados.0.smithi162.stdout:596: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:03:41.574 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:03:41.574 INFO:tasks.rados.rados.0.smithi162.stdout:593: done (2 left) 2024-09-16T15:03:41.574 INFO:tasks.rados.rados.0.smithi162.stdout:595: done (1 left) 2024-09-16T15:03:41.574 INFO:tasks.rados.rados.0.smithi162.stdout:596: done (0 left) 2024-09-16T15:03:41.575 INFO:tasks.rados.rados.0.smithi162.stdout:597: copy_from oid 6 from oid 9 current snap is 49 2024-09-16T15:03:41.575 INFO:tasks.rados.rados.0.smithi162.stdout:598: copy_from oid 37 from oid 45 current snap is 49 2024-09-16T15:03:41.575 INFO:tasks.rados.rados.0.smithi162.stdout:599: snap_remove snap 48 2024-09-16T15:03:42.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): tick 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 monclient(hunting): continuing hunt 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 msgr2=0x7f0c3c136180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c3c136e40 0x7f0c3c136180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 msgr2=0x7f0c3c1366c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c3c137220 0x7f0c3c1366c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 msgr2=0x7f0c3c1cb780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c3c13c370 0x7f0c3c1cb780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient: _add_conns ranks=[7,6,8,3,4,0] 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0c1c0037c0 0x7f0c1c005c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): picked mon.noname-i con 0x7f0c1c0037c0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:42.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): picked mon.noname-d con 0x7f0c1c006210 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0c1c008d20 0x7f0c1c00b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): picked mon.noname-e con 0x7f0c1c008d20 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1228e0 con 0x7f0c1c006210 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0a75a0 con 0x7f0c1c0037c0 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0a0170 con 0x7f0c1c008d20 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c237fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c40d47640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0c1c008d20 msgr2=0x7f0c1c00b1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:03:42.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c40d47640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0c1c008d20 0x7f0c1c00b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:32816/0 (socket says 172.21.15.134:32816) 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/1846473920 learned_addr learned my addr 172.21.15.134:0/1846473920 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 10 monclient(hunting): get_auth_request con 0x7f0c1c006210 auth_method 0 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.151+0000 7f0c3b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0c1c0037c0 0x7f0c1c005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): get_auth_request con 0x7f0c1c0037c0 auth_method 0 2024-09-16T15:03:42.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient(hunting): handle_auth_done global_id 34269 payload 824 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0c1c008d20 msgr2=0x7f0c1c00b1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0c1c008d20 0x7f0c1c00b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0c1c0037c0 msgr2=0x7f0c1c005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0c1c0037c0 0x7f0c1c005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 monclient: found mon.noname-d 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c40d47640 1 -- 172.21.15.134:0/1846473920 reap_dead start 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:03:42.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.152+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30009400 con 0x7f0c1c006210 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c3b7fe640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 msgr2=0x7f0c1c008680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c3b7fe640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c3002afd0 tx=0x7f0c300046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c30030d30 con 0x7f0c1c006210 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 10 monclient: dump: 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.154+0000 7f0c38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 5 monclient: authenticate success, global_id 34269 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30004460 con 0x7f0c1c006210 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 10 monclient: _renew_subs 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c122290 con 0x7f0c1c006210 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c30032440 con 0x7f0c1c006210 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c38ff9640 10 monclient: dump: 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.164 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c3a7fc640 4 set_mon_vals no callback set 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 10 monclient: _renew_subs 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:03:42.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.155+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c121330 con 0x7f0c1c006210 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0c30003610 con 0x7f0c1c006210 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c427aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1214c0 con 0x7f0c3c120170 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient: get_auth_request con 0x7f0c3c120170 auth_method 0 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c38ff9640 1 --2- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 0x7f0c2803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 3927+0+0 (secure 0 0 0) 0x7f0c3006a870 con 0x7f0c1c006210 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_done global_id 4675 payload 824 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c3bfff640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c3c061ae0 con 0x7f0c3c120170 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c40d47640 1 --2- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 0x7f0c2803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.156+0000 7f0c40d47640 10 monclient: get_auth_request con 0x7f0c280387f0 auth_method 0 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c3bfff640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 msgr2=0x7f0c3c136180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c3bfff640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c003660 tx=0x7f0c2c00ee40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c2c018070 con 0x7f0c3c120170 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 10 monclient: dump: 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.167 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.168 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c40d47640 1 --2- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 0x7f0c2803acb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c09fe60 tx=0x7f0c340049a0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0c2c00cb70 con 0x7f0c3c120170 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 msgr2=0x7f0c3c136180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c003660 tx=0x7f0c2c00ee40 comp rx=0 tx=0).stop 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 msgr2=0x7f0c3c136180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.159+0000 7f0c38ff9640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c003660 tx=0x7f0c2c00ee40 comp rx=0 tx=0).stop 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c427aa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c427aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c071b20 con 0x7f0c3c09f850 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient: get_auth_request con 0x7f0c3c09f850 auth_method 0 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 10 monclient(hunting): handle_auth_done global_id 4684 payload 824 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.166+0000 7f0c3bfff640 1 -- 172.21.15.134:0/1846473920 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c3c07a680 con 0x7f0c3c09f850 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.168+0000 7f0c3bfff640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 msgr2=0x7f0c3c1366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.168+0000 7f0c3bfff640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c007f90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c2c014d10 con 0x7f0c3c09f850 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 10 monclient: dump: 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.172 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7f0c2c0148a0 con 0x7f0c3c09f850 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:42.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 msgr2=0x7f0c3c1366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c007f90 comp rx=0 tx=0).stop 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 msgr2=0x7f0c3c1366c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c38ff9640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c000c00 tx=0x7f0c2c007f90 comp rx=0 tx=0).stop 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 34, 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.175 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.176 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:42.177 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:42.178 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.179 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.182 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:42.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.188 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.193 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:42.195 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 msgr2=0x7f0c2803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 0x7f0c2803acb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c09fe60 tx=0x7f0c340049a0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 10 monclient: shutdown 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 msgr2=0x7f0c1c008680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c3002afd0 tx=0x7f0c300046c0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 shutdown_connections 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c280387f0 0x7f0c2803acb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0c1c006210 0x7f0c1c008680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c09f850 0x7f0c3c1366c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.169+0000 7f0c427aa640 1 --2- 172.21.15.134:0/1846473920 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c3c120170 0x7f0c3c136180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.170+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 >> 172.21.15.134:0/1846473920 conn(0x7f0c3c0a6110 msgr2=0x7f0c3c13b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.170+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 shutdown_connections 2024-09-16T15:03:42.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.170+0000 7f0c427aa640 1 -- 172.21.15.134:0/1846473920 wait complete. 2024-09-16T15:03:42.198 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:03:42.198 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-16T15:03:42.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.303+0000 7f0ad8557640 10 monclient: get_monmap_and_config 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.303+0000 7f0ad8557640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.303+0000 7f0ad8557640 10 monclient: monmap: 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:42.304624+0000 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:42.304624+0000 2024-09-16T15:03:42.305 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:42.306 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 Processor -- start 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 -- start start 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient: init 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient: _add_conns ranks=[7,2,5,8,0,6,4,10,1,9,3] 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-i con 0x7f0ad0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 0x7f0ad0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-b con 0x7f0ad0137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 0x7f0ad0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-h con 0x7f0ad0135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad009abc0 con 0x7f0ad0137220 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad009ba50 con 0x7f0ad0135dc0 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00a53c0 con 0x7f0ad0136e40 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.304+0000 7f0ad8557640 10 monclient(hunting): authenticate will time out at 958.893966s 2024-09-16T15:03:42.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad5af2640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 msgr2=0x7f0ad0137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad6af4640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 msgr2=0x7f0ad0136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad5af2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 0x7f0ad0137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad6af4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 0x7f0ad0136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:58002/0 (socket says 172.21.15.134:58002) 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 -- 172.21.15.134:0/776501247 learned_addr learned my addr 172.21.15.134:0/776501247 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): get_auth_request con 0x7f0ad0136e40 auth_method 0 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_done global_id 4606 payload 274 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 -- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 msgr2=0x7f0ad0136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 0x7f0ad0136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 -- 172.21.15.134:0/776501247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 msgr2=0x7f0ad0137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 0x7f0ad0137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 monclient: found mon.noname-i 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.305+0000 7f0ad62f3640 1 -- 172.21.15.134:0/776501247 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0073250 con 0x7f0ad0136e40 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad62f3640 1 -- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 msgr2=0x7f0ad009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0098a0 tx=0x7f0acc02a0b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=f1236344d00aca6e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 1 -- 172.21.15.134:0/776501247 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0acc0307b0 con 0x7f0ad0136e40 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: dump: 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.321 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 1 -- 172.21.15.134:0/776501247 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc030930 con 0x7f0ad0136e40 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad8557640 5 monclient: authenticate success, global_id 4606 2024-09-16T15:03:42.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad8557640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 1 -- 172.21.15.134:0/776501247 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0acc0313a0 con 0x7f0ad0136e40 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad52f1640 10 monclient: dump: 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.323 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad8557640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.308+0000 7f0ad8557640 4 set_mon_vals no callback set 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 10 monclient: shutdown 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 -- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 msgr2=0x7f0ad009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0098a0 tx=0x7f0acc02a0b0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 -- 172.21.15.134:0/776501247 shutdown_connections 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0ad0135dc0 0x7f0ad0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ad0137220 0x7f0ad0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 --2- 172.21.15.134:0/776501247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0ad0136e40 0x7f0ad009c150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 -- 172.21.15.134:0/776501247 >> 172.21.15.134:0/776501247 conn(0x7f0ad00a6110 msgr2=0x7f0ad00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 -- 172.21.15.134:0/776501247 shutdown_connections 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.309+0000 7f0ad8557640 1 -- 172.21.15.134:0/776501247 wait complete. 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient: monmap: 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:42.325 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 Processor -- start 2024-09-16T15:03:42.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 -- start start 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient: init 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient: _add_conns ranks=[4,3,6,1,5,8,2,7,0] 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 0x7f0ad011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-c con 0x7f0ad0135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ad0136e40 0x7f0ad01201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-g con 0x7f0ad0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad0137220 0x7f0ad01206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): picked mon.noname-d con 0x7f0ad0137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00a0470 con 0x7f0ad0136e40 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad01220d0 con 0x7f0ad0135dc0 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0122320 con 0x7f0ad0137220 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.310+0000 7f0ad8557640 10 monclient(hunting): authenticate will time out at 958.899935s 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 0x7f0ad011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 0x7f0ad011fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41628/0 (socket says 172.21.15.134:41628) 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 -- 172.21.15.134:0/3763579192 learned_addr learned my addr 172.21.15.134:0/3763579192 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): get_auth_request con 0x7f0ad0135dc0 auth_method 0 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad0137220 0x7f0ad01206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): get_auth_request con 0x7f0ad0137220 auth_method 0 2024-09-16T15:03:42.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ad0136e40 0x7f0ad01201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): get_auth_request con 0x7f0ad0136e40 auth_method 0 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient(hunting): handle_auth_done global_id 4693 payload 824 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad0137220 msgr2=0x7f0ad01206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad0137220 0x7f0ad01206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ad0136e40 msgr2=0x7f0ad01201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ad0136e40 0x7f0ad01201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 monclient: found mon.noname-c 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad62f3640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc0094d0 con 0x7f0ad0135dc0 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.311+0000 7f0ad6af4640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad0137220 0x7f0ad01206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:03:42.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.313+0000 7f0ad62f3640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 msgr2=0x7f0ad011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.313+0000 7f0ad62f3640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 0x7f0ad011fc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0acc000c00 tx=0x7f0acc004730 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.313+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0acc030770 con 0x7f0ad0135dc0 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.313+0000 7f0ac6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: dump: 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc004460 con 0x7f0ad0135dc0 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 5 monclient: authenticate success, global_id 4693 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 monclient: _renew_subs 2024-09-16T15:03:42.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00a8b50 con 0x7f0ad0135dc0 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0acc03d470 con 0x7f0ad0135dc0 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 10 monclient: dump: 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 monclient: _renew_subs 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad4af0640 4 set_mon_vals no callback set 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad01369f0 con 0x7f0ad0135dc0 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 0x7f0ad0134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0071b20 con 0x7f0ad01224b0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0acc039070 con 0x7f0ad0135dc0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad5af2640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 0x7f0ad0134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad5af2640 10 monclient: get_auth_request con 0x7f0ad01224b0 auth_method 0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad5af2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad5af2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.314+0000 7f0ad5af2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ac6ffd640 1 --2- 172.21.15.134:0/3763579192 >> v2:172.21.15.162:6800/980547263 conn(0x7f0ab40388b0 0x7f0ab403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 3927+0+0 (secure 0 0 0) 0x7f0acc06a9c0 con 0x7f0ad0135dc0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_done global_id 34287 payload 824 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad5af2640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ad0061ae0 con 0x7f0ad01224b0 2024-09-16T15:03:42.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad6af4640 1 --2- 172.21.15.134:0/3763579192 >> v2:172.21.15.162:6800/980547263 conn(0x7f0ab40388b0 0x7f0ab403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.315+0000 7f0ad6af4640 10 monclient: get_auth_request con 0x7f0ab40388b0 auth_method 0 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ad5af2640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 msgr2=0x7f0ad0134bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ad5af2640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 0x7f0ad0134bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0003980 tx=0x7f0ac0009aa0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ac000edc0 con 0x7f0ad01224b0 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 10 monclient: dump: 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.338 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ad6af4640 1 --2- 172.21.15.134:0/3763579192 >> v2:172.21.15.162:6800/980547263 conn(0x7f0ab40388b0 0x7f0ab403ad70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ad009f810 tx=0x7f0ac80095f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0ac000e950 con 0x7f0ad01224b0 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 msgr2=0x7f0ad0134bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 0x7f0ad0134bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0003980 tx=0x7f0ac0009aa0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 msgr2=0x7f0ad0134bd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.317+0000 7f0ac6ffd640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad01224b0 0x7f0ad0134bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.323+0000 7f0ad8557640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad8557640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 0x7f0ad002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad8557640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad007a680 con 0x7f0ad002b260 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 0x7f0ad002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient: get_auth_request con 0x7f0ad002b260 auth_method 0 2024-09-16T15:03:42.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 10 monclient(hunting): handle_auth_done global_id 34296 payload 824 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.324+0000 7f0ad5af2640 1 -- 172.21.15.134:0/3763579192 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ad009b490 con 0x7f0ad002b260 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ad5af2640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 msgr2=0x7f0ad002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ad5af2640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 0x7f0ad002b670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009fd0 tx=0x7f0ac00059d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ac0012a70 con 0x7f0ad002b260 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ac6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ac6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ac6ffd640 10 monclient: dump: 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.341 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.326+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7f0ac0012bd0 con 0x7f0ad002b260 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 msgr2=0x7f0ad002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 0x7f0ad002b670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009fd0 tx=0x7f0ac00059d0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 msgr2=0x7f0ad002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ac6ffd640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0ad002b260 0x7f0ad002b670 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009fd0 tx=0x7f0ac00059d0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 34, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:42.366 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 >> v2:172.21.15.162:6800/980547263 conn(0x7f0ab40388b0 msgr2=0x7f0ab403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 --2- 172.21.15.134:0/3763579192 >> v2:172.21.15.162:6800/980547263 conn(0x7f0ab40388b0 0x7f0ab403ad70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ad009f810 tx=0x7f0ac80095f0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 10 monclient: shutdown 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 msgr2=0x7f0ad011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 --2- 172.21.15.134:0/3763579192 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0ad0135dc0 0x7f0ad011fc60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0acc000c00 tx=0x7f0acc004730 comp rx=0 tx=0).stop 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad6af4640 1 -- 172.21.15.134:0/3763579192 reap_dead start 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad6af4640 1 -- 172.21.15.134:0/3763579192 reap_dead start 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 shutdown_connections 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 >> 172.21.15.134:0/3763579192 conn(0x7f0ad00a6110 msgr2=0x7f0ad00a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 shutdown_connections 2024-09-16T15:03:42.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.327+0000 7f0ad8557640 1 -- 172.21.15.134:0/3763579192 wait complete. 2024-09-16T15:03:42.368 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:03:42.368 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-09-16T15:03:42.368 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:03:42.378 INFO:tasks.rados.rados.0.smithi162.stdout:599: done (2 left) 2024-09-16T15:03:42.378 INFO:tasks.rados.rados.0.smithi162.stdout:600: write (excl) oid 50 current snap is 49 2024-09-16T15:03:42.378 INFO:tasks.rados.rados.0.smithi162.stdout:600: seq_num 240 ranges {433680=466703,1577796=455884} 2024-09-16T15:03:42.382 INFO:tasks.rados.rados.0.smithi162.stdout:600: writing smithi16234619-50 from 433680 to 900383 tid 1 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:600: writing smithi16234619-50 from 1577796 to 2033680 tid 2 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:601: rollback oid 47 current snap is 49 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 46 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:598: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:598: finishing copy_from to smithi16234619-37 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:03:42.389 INFO:tasks.rados.rados.0.smithi162.stdout:597: finishing copy_from racing read to smithi16234619-6 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:597: finishing copy_from to smithi16234619-6 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 216 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:597: done (3 left) 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:598: done (2 left) 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:602: read oid 38 snap 46 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:602: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:603: read oid 42 snap -1 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:603: expect (ObjNum 179 snap 38 seq_num 179) 2024-09-16T15:03:42.390 INFO:tasks.rados.rados.0.smithi162.stdout:604: delete oid 49 current snap is 49 2024-09-16T15:03:42.393 INFO:tasks.rados.rados.0.smithi162.stdout:604: done (4 left) 2024-09-16T15:03:42.393 INFO:tasks.rados.rados.0.smithi162.stdout:605: write oid 15 current snap is 49 2024-09-16T15:03:42.393 INFO:tasks.rados.rados.0.smithi162.stdout:605: seq_num 242 ranges {467294=617201,1578407=488887} 2024-09-16T15:03:42.399 INFO:tasks.rados.rados.0.smithi162.stdout:605: writing smithi16234619-15 from 467294 to 1084495 tid 1 2024-09-16T15:03:42.405 INFO:tasks.rados.rados.0.smithi162.stdout:605: writing smithi16234619-15 from 1578407 to 2067294 tid 2 2024-09-16T15:03:42.405 INFO:tasks.rados.rados.0.smithi162.stdout:606: copy_from oid 29 from oid 2 current snap is 49 2024-09-16T15:03:42.405 INFO:tasks.rados.rados.0.smithi162.stdout:603: expect (ObjNum 179 snap 38 seq_num 179) 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.479+0000 7f400295a640 10 monclient: get_monmap_and_config 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.479+0000 7f400295a640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.480+0000 7f400295a640 10 monclient: monmap: 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:42.480932+0000 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:42.480932+0000 2024-09-16T15:03:42.481 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.482 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.483 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:42.483 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:42.483 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.480+0000 7f400295a640 1 Processor -- start 2024-09-16T15:03:42.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.480+0000 7f400295a640 1 -- start start 2024-09-16T15:03:42.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.480+0000 7f400295a640 10 monclient: init 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient: _add_conns ranks=[2,10,7,1,3,8,6,9,0,4,5] 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 0x7f3ffc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): picked mon.noname-b con 0x7f3ffc136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 0x7f3ffc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): picked mon.noname-f con 0x7f3ffc137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): picked mon.noname-i con 0x7f3ffc135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc09abc0 con 0x7f3ffc137220 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc09ba50 con 0x7f3ffc136e40 2024-09-16T15:03:42.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0a53c0 con 0x7f3ffc135dc0 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f400295a640 10 monclient(hunting): authenticate will time out at 959.070372s 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f3ffbfff640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 msgr2=0x7f3ffc09c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 0x7f3ffc09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f3ffb7fe640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 msgr2=0x7f3ffc137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f3ffb7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 0x7f3ffc137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:58016/0 (socket says 172.21.15.134:58016) 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 -- 172.21.15.134:0/196589949 learned_addr learned my addr 172.21.15.134:0/196589949 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): get_auth_request con 0x7f3ffc135dc0 auth_method 0 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient(hunting): handle_auth_done global_id 4615 payload 274 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 -- 172.21.15.134:0/196589949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 msgr2=0x7f3ffc09c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 --2- 172.21.15.134:0/196589949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 0x7f3ffc09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 -- 172.21.15.134:0/196589949 >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 msgr2=0x7f3ffc137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 --2- 172.21.15.134:0/196589949 >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 0x7f3ffc137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 monclient: found mon.noname-i 2024-09-16T15:03:42.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:03:42.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.481+0000 7f4000ef7640 1 -- 172.21.15.134:0/196589949 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc073250 con 0x7f3ffc135dc0 2024-09-16T15:03:42.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.484+0000 7f4000ef7640 1 -- 172.21.15.134:0/196589949 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 msgr2=0x7f3ffc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.484+0000 7f4000ef7640 1 --2- 172.21.15.134:0/196589949 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009e80 tx=0x7f3ff402a5d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ea0b3e2b6813fd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 1 -- 172.21.15.134:0/196589949 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff4007a70 con 0x7f3ffc135dc0 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: dump: 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.497 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 5 monclient: authenticate success, global_id 4615 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 1 -- 172.21.15.134:0/196589949 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff402d050 con 0x7f3ffc135dc0 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 4 set_mon_vals no callback set 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 1 -- 172.21.15.134:0/196589949 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff4007c30 con 0x7f3ffc135dc0 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f3ffaffd640 10 monclient: dump: 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.499 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 10 monclient: shutdown 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 -- 172.21.15.134:0/196589949 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 msgr2=0x7f3ffc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 --2- 172.21.15.134:0/196589949 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009e80 tx=0x7f3ff402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 -- 172.21.15.134:0/196589949 shutdown_connections 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 --2- 172.21.15.134:0/196589949 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ffc135dc0 0x7f3ffc136220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 --2- 172.21.15.134:0/196589949 >> v2:172.21.15.162:3300/0 conn(0x7f3ffc137220 0x7f3ffc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 --2- 172.21.15.134:0/196589949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ffc136e40 0x7f3ffc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 -- 172.21.15.134:0/196589949 >> 172.21.15.134:0/196589949 conn(0x7f3ffc0a6110 msgr2=0x7f3ffc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 -- 172.21.15.134:0/196589949 shutdown_connections 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.485+0000 7f400295a640 1 -- 172.21.15.134:0/196589949 wait complete. 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.486+0000 7f400295a640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.486+0000 7f400295a640 10 monclient: monmap: 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:42.501 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.486+0000 7f400295a640 1 Processor -- start 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.486+0000 7f400295a640 1 -- start start 2024-09-16T15:03:42.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.486+0000 7f400295a640 10 monclient: init 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient: _add_conns ranks=[5,3,0,8,6,2,1,4,7] 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ffc135dc0 0x7f3ffc135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): picked mon.noname-h con 0x7f3ffc135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc136e40 0x7f3ffc1355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): picked mon.noname-g con 0x7f3ffc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 0x7f3ffc121b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): picked mon.noname-a con 0x7f3ffc137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc136430 con 0x7f3ffc137220 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0a8ac0 con 0x7f3ffc136e40 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc136980 con 0x7f3ffc135dc0 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f400295a640 10 monclient(hunting): authenticate will time out at 959.076248s 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffbfff640 1 -- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ffc135dc0 msgr2=0x7f3ffc135080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ffc135dc0 0x7f3ffc135080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 0x7f3ffc121b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 0x7f3ffc121b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38946/0 (socket says 172.21.15.134:38946) 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 learned_addr learned my addr 172.21.15.134:0/3940902693 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): get_auth_request con 0x7f3ffc137220 auth_method 0 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc136e40 0x7f3ffc1355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): get_auth_request con 0x7f3ffc136e40 auth_method 0 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f3ffb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient(hunting): handle_auth_done global_id 4725 payload 824 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ffc135dc0 msgr2=0x7f3ffc135080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ffc135dc0 0x7f3ffc135080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc136e40 msgr2=0x7f3ffc1355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc136e40 0x7f3ffc1355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 monclient: found mon.noname-a 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.487+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff4009b30 con 0x7f3ffc137220 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.489+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 msgr2=0x7f3ffc121b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.489+0000 7f4000ef7640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 0x7f3ffc121b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009fa0 tx=0x7f3ff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff4004790 con 0x7f3ffc137220 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:03:42.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: dump: 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.507 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4007a70 con 0x7f3ffc137220 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 5 monclient: authenticate success, global_id 4725 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ff4007be0 con 0x7f3ffc137220 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 monclient: _renew_subs 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc125790 con 0x7f3ffc137220 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ffa7fc640 4 set_mon_vals no callback set 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f3ff8ff9640 10 monclient: dump: 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.509 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.511 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.511 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.511 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 monclient: _renew_subs 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc1236a0 con 0x7f3ffc137220 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 0x7f3ffc122050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.490+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc071b20 con 0x7f3ffc1cb5c0 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3ff4007d40 con 0x7f3ffc137220 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 0x7f3ffc122050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient: get_auth_request con 0x7f3ffc1cb5c0 auth_method 0 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ff8ff9640 1 --2- 172.21.15.134:0/3940902693 >> v2:172.21.15.162:6800/980547263 conn(0x7f3fe8038880 0x7f3fe803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 3927+0+0 (secure 0 0 0) 0x7f3ff406ad70 con 0x7f3ffc137220 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_done global_id 34329 payload 824 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffbfff640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ffc1251d0 con 0x7f3ffc1cb5c0 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffb7fe640 1 --2- 172.21.15.134:0/3940902693 >> v2:172.21.15.162:6800/980547263 conn(0x7f3fe8038880 0x7f3fe803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.491+0000 7f3ffb7fe640 10 monclient: get_auth_request con 0x7f3fe8038880 auth_method 0 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ffbfff640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 msgr2=0x7f3ffc122050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ffbfff640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 0x7f3ffc122050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b450 tx=0x7f3fec00b910 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ffb7fe640 1 --2- 172.21.15.134:0/3940902693 >> v2:172.21.15.162:6800/980547263 conn(0x7f3fe8038880 0x7f3fe803ad40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00093d0 tx=0x7f3ff0009360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3fec0054e0 con 0x7f3ffc1cb5c0 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ff8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ff8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.493+0000 7f3ff8ff9640 10 monclient: dump: 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.514 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3fec005640 con 0x7f3ffc1cb5c0 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 msgr2=0x7f3ffc122050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 0x7f3ffc122050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b450 tx=0x7f3fec00b910 comp rx=0 tx=0).stop 2024-09-16T15:03:42.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 msgr2=0x7f3ffc122050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.494+0000 7f3ff8ff9640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc1cb5c0 0x7f3ffc122050 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b450 tx=0x7f3fec00b910 comp rx=0 tx=0).stop 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f400295a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f400295a640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 0x7f3ffc0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f400295a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc125920 con 0x7f3ffc122590 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 0x7f3ffc0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient: get_auth_request con 0x7f3ffc122590 auth_method 0 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 10 monclient(hunting): handle_auth_done global_id 34338 payload 824 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.500+0000 7f3ffbfff640 1 -- 172.21.15.134:0/3940902693 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3ffc061ae0 con 0x7f3ffc122590 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.502+0000 7f3ffbfff640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 msgr2=0x7f3ffc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.502+0000 7f3ffbfff640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 0x7f3ffc0a1a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fec012040 tx=0x7f3fec0059d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3fec01f070 con 0x7f3ffc122590 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 10 monclient: dump: 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.519 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7f3fec010400 con 0x7f3ffc122590 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 msgr2=0x7f3ffc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 0x7f3ffc0a1a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fec012040 tx=0x7f3fec0059d0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 msgr2=0x7f3ffc0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f3ff8ff9640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ffc122590 0x7f3ffc0a1a20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 34, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:42.522 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.523 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:42.525 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:42.526 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.527 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:42.528 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.529 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:42.530 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.532 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.534 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.537 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.541 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.542 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.543 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:42.544 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:42.546 INFO:tasks.rados.rados.0.smithi162.stdout:602: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T15:03:42.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 >> v2:172.21.15.162:6800/980547263 conn(0x7f3fe8038880 msgr2=0x7f3fe803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 --2- 172.21.15.134:0/3940902693 >> v2:172.21.15.162:6800/980547263 conn(0x7f3fe8038880 0x7f3fe803ad40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00093d0 tx=0x7f3ff0009360 comp rx=0 tx=0).stop 2024-09-16T15:03:42.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 10 monclient: shutdown 2024-09-16T15:03:42.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 msgr2=0x7f3ffc121b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 --2- 172.21.15.134:0/3940902693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ffc137220 0x7f3ffc121b10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009fa0 tx=0x7f3ff4004a40 comp rx=0 tx=0).stop 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 reap_dead start 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f4000ef7640 1 -- 172.21.15.134:0/3940902693 reap_dead start 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 shutdown_connections 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 >> 172.21.15.134:0/3940902693 conn(0x7f3ffc0a6110 msgr2=0x7f3ffc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 shutdown_connections 2024-09-16T15:03:42.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.503+0000 7f400295a640 1 -- 172.21.15.134:0/3940902693 wait complete. 2024-09-16T15:03:42.547 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:03:42.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon i 2024-09-16T15:03:42.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:03:42.618 INFO:tasks.rados.rados.0.smithi162.stdout:601: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:03:42.618 INFO:tasks.rados.rados.0.smithi162.stdout:601: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:601: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 55 (ObjNum 227 snap 45 seq_num 227) dirty exists 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: finishing write tid 1 to smithi16234619-50 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: finishing write tid 2 to smithi16234619-50 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: finishing write tid 3 to smithi16234619-50 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: finishing write tid 4 to smithi16234619-50 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: oid 50 updating version 0 to 240 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: oid 50 updating version 240 to 241 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: oid 50 version 241 is already newer than 239 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 241 (ObjNum 240 snap 49 seq_num 240) dirty exists 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:600: left oid 50 (ObjNum 240 snap 49 seq_num 240) 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:605: finishing write tid 1 to smithi16234619-15 2024-09-16T15:03:42.619 INFO:tasks.rados.rados.0.smithi162.stdout:605: finishing write tid 2 to smithi16234619-15 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: finishing write tid 3 to smithi16234619-15 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: finishing write tid 4 to smithi16234619-15 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: oid 15 updating version 0 to 217 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: oid 15 updating version 217 to 218 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: oid 15 version 218 is already newer than 216 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 218 (ObjNum 242 snap 49 seq_num 242) dirty exists 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:605: left oid 15 (ObjNum 242 snap 49 seq_num 242) 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:606: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:606: finishing copy_from to smithi16234619-29 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 189 (ObjNum 234 snap 46 seq_num 234) dirty exists 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:600: done (5 left) 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:601: done (4 left) 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:602: done (3 left) 2024-09-16T15:03:42.620 INFO:tasks.rados.rados.0.smithi162.stdout:603: done (2 left) 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:605: done (1 left) 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:606: done (0 left) 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:607: rollback oid 37 current snap is 49 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 46 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:608: write (excl) oid 13 current snap is 49 2024-09-16T15:03:42.621 INFO:tasks.rados.rados.0.smithi162.stdout:608: seq_num 243 ranges {484101=692450,1636889=447212} 2024-09-16T15:03:42.624 INFO:tasks.rados.rados.0.smithi162.stdout:608: writing smithi16234619-13 from 484101 to 1176551 tid 1 2024-09-16T15:03:42.629 INFO:tasks.rados.rados.0.smithi162.stdout:608: writing smithi16234619-13 from 1636889 to 2084101 tid 2 2024-09-16T15:03:42.630 INFO:tasks.rados.rados.0.smithi162.stdout:609: delete oid 2 current snap is 49 2024-09-16T15:03:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:607: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:03:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:607: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:03:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:607: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:03:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 242 (ObjNum 217 snap 44 seq_num 217) dirty exists 2024-09-16T15:03:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:607: done (2 left) 2024-09-16T15:03:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:609: done (1 left) 2024-09-16T15:03:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:610: read oid 36 snap -1 2024-09-16T15:03:42.635 INFO:tasks.rados.rados.0.smithi162.stdout:610: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:03:42.635 INFO:tasks.rados.rados.0.smithi162.stdout:611: snap_remove snap 46 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.645+0000 7f141d391640 10 monclient: get_monmap_and_config 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.645+0000 7f141d391640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.646+0000 7f141d391640 10 monclient: monmap: 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:03:42.646954+0000 2024-09-16T15:03:42.647 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:03:42.646954+0000 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:03:42.648 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.646+0000 7f141d391640 1 Processor -- start 2024-09-16T15:03:42.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.646+0000 7f141d391640 1 -- start start 2024-09-16T15:03:42.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.646+0000 7f141d391640 10 monclient: init 2024-09-16T15:03:42.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient: _add_conns ranks=[0,1,2,7,10,6,5,3,8,9,4] 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1418069e50 addr v1:172.21.15.134:6789/0 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1418066700 addr v1:172.21.15.162:6789/0 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f141813fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): picked mon.noname-b con 0x7f1418066ae0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1418073250 con 0x7f1418069e50 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141809e4a0 con 0x7f1418066700 2024-09-16T15:03:42.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141806a230 con 0x7f1418066ae0 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f141d391640 10 monclient(hunting): authenticate will time out at 959.236125s 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f14177fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 msgr2=0x7f141813fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f14177fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f141813fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f14167fc640 1 -- >> v1:172.21.15.162:6789/0 conn(0x7f1418066700 legacy=0x7f141813c5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.162:6789/0 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f1416ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1418069e50 0x7f1418065ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45952/0 (socket says 172.21.15.134:45952) 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.647+0000 7f1416ffd640 1 -- 172.21.15.134:0/2602226943 learned_addr learned my addr 172.21.15.134:0/2602226943 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1416ffd640 1 -- 172.21.15.134:0/2602226943 >> v1:172.21.15.134:6789/0 conn(0x7f1418069e50 legacy=0x7f1418065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668729763 0 0) 0x7f1418073250 con 0x7f1418069e50 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): _negotiate 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): my global_id is 4734 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400003520 con 0x7f1418069e50 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142407019 0 0) 0x7f1400003520 con 0x7f1418069e50 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 msgr2=0x7f141813fce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f141813fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 >> v1:172.21.15.162:6789/0 conn(0x7f1418066700 legacy=0x7f141813c5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141809e1c0 con 0x7f1418069e50 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f141d391640 5 monclient: authenticate success, global_id 4734 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f140c001b90 con 0x7f1418069e50 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f141d391640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:42.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: dump: 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.658 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 >> v1:172.21.15.134:6789/0 conn(0x7f1418069e50 legacy=0x7f1418065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient: _add_conns ranks=[2,5,0,1,7,3,6,8,4] 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f14000041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): picked mon.b con 0x7f1418066ae0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:42.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f14000049a0 0x7f1400004e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 10 monclient(hunting): picked mon.h con 0x7f14000049a0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.649+0000 7f1415ffb640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 0x7f140000bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): picked mon.a con 0x7f1400009970 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141806a230 con 0x7f1400009970 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140000c530 con 0x7f1418066ae0 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f140000c6a0 con 0x7f14000049a0 2024-09-16T15:03:42.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1415ffb640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f141d391640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1416ffd640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 msgr2=0x7f14000041f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f1416ffd640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f14000041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14167fc640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f14000049a0 msgr2=0x7f1400004e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3302/0 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14167fc640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f14000049a0 0x7f1400004e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 0x7f140000bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): get_auth_request con 0x7f1400009970 auth_method 0 2024-09-16T15:03:42.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient(hunting): handle_auth_done global_id 4734 payload 293 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f14000049a0 msgr2=0x7f1400004e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f14000049a0 0x7f1400004e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 msgr2=0x7f14000041f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:03:42.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1418066ae0 0x7f14000041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 monclient: found mon.a 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 1 -- 172.21.15.134:0/2602226943 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400003520 con 0x7f1400009970 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.650+0000 7f14177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f14177fe640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 msgr2=0x7f140000bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f14177fe640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 0x7f140000bd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1410009e30 tx=0x7f14000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adedc869744caa98 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1410005750 con 0x7f1400009970 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: dump: 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.670 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.671 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f141d391640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100058d0 con 0x7f1400009970 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f141d391640 10 monclient: get_monmap_and_config success 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 1 -- 172.21.15.134:0/2602226943 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1410005a40 con 0x7f1400009970 2024-09-16T15:03:42.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f141d391640 4 set_mon_vals no callback set 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.653+0000 7f1415ffb640 10 monclient: dump: 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.673 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 10 monclient: shutdown 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 -- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 msgr2=0x7f140000bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 --2- 172.21.15.134:0/2602226943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1400009970 0x7f140000bd60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1410009e30 tx=0x7f14000057f0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f14177fe640 1 -- 172.21.15.134:0/2602226943 reap_dead start 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 -- 172.21.15.134:0/2602226943 shutdown_connections 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 -- 172.21.15.134:0/2602226943 >> 172.21.15.134:0/2602226943 conn(0x7f14180a6400 msgr2=0x7f14180a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 -- 172.21.15.134:0/2602226943 shutdown_connections 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 1 -- 172.21.15.134:0/2602226943 wait complete. 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.654+0000 7f141d391640 10 monclient: build_initial_monmap 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.655+0000 7f141d391640 10 monclient: monmap: 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:03:42.675 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.655+0000 7f141d391640 1 Processor -- start 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.655+0000 7f141d391640 1 -- start start 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.655+0000 7f141d391640 10 monclient: init 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient: _reopen_session rank -1 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient: _add_conns ranks=[4,2,1,0,6,5,3,8,7] 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 0x7f14181cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): picked mon.noname-c con 0x7f14181cb780 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14181cc0a0 0x7f1418091eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): picked mon.noname-b con 0x7f14181cc0a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14180923f0 0x7f1418092870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f14167fc640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14181cc0a0 msgr2=0x7f1418091eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f14167fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14181cc0a0 0x7f1418091eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): picked mon.noname-f con 0x7f14180923f0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180a1120 con 0x7f14180923f0 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1418095b60 con 0x7f14181cc0a0 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1418095e40 con 0x7f14181cb780 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): _renew_subs 2024-09-16T15:03:42.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f141d391640 10 monclient(hunting): authenticate will time out at 959.245254s 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f14177fe640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14180923f0 msgr2=0x7f1418092870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 0x7f14181cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f14177fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14180923f0 0x7f1418092870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 0x7f14181cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41632/0 (socket says 172.21.15.134:41632) 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 -- 172.21.15.134:0/459458354 learned_addr learned my addr 172.21.15.134:0/459458354 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): get_auth_request con 0x7f14181cb780 auth_method 0 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient(hunting): handle_auth_done global_id 4702 payload 824 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14181cc0a0 msgr2=0x7f1418091eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14181cc0a0 0x7f1418091eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14180923f0 msgr2=0x7f1418092870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14180923f0 0x7f1418092870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 monclient: found mon.noname-c 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.656+0000 7f1416ffd640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418093ae0 con 0x7f14181cb780 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.658+0000 7f1416ffd640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 msgr2=0x7f14181cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.658+0000 7f1416ffd640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 0x7f14181cbb60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c700 tx=0x7f140c00cbc0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.659+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f140c004d60 con 0x7f14181cb780 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.659+0000 7f13ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: dump: 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.680 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: _finish_auth 0 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c007540 con 0x7f14181cb780 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 5 monclient: authenticate success, global_id 4702 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 monclient: _renew_subs 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1418093ff0 con 0x7f14181cb780 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:03:42.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f140c0076b0 con 0x7f14181cb780 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 10 monclient: dump: 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.682 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f14157fa640 4 set_mon_vals no callback set 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 monclient: _renew_subs 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180955a0 con 0x7f14181cb780 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f140c0070d0 con 0x7f14181cb780 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 0x7f1418070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.660+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f14180a6db0 con 0x7f1418092db0 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f13ff7fe640 1 --2- 172.21.15.134:0/459458354 >> v2:172.21.15.162:6800/980547263 conn(0x7f1404038880 0x7f140403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 3927+0+0 (secure 0 0 0) 0x7f140c049540 con 0x7f14181cb780 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 0x7f1418070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient: get_auth_request con 0x7f1418092db0 auth_method 0 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14177fe640 1 --2- 172.21.15.134:0/459458354 >> v2:172.21.15.162:6800/980547263 conn(0x7f1404038880 0x7f140403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14177fe640 10 monclient: get_auth_request con 0x7f1404038880 auth_method 0 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 10 monclient(hunting): handle_auth_done global_id 4624 payload 824 2024-09-16T15:03:42.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.661+0000 7f14167fc640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1418061ae0 con 0x7f1418092db0 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.663+0000 7f14177fe640 1 --2- 172.21.15.134:0/459458354 >> v2:172.21.15.162:6800/980547263 conn(0x7f1404038880 0x7f140403ad40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1410003d00 tx=0x7f1410005000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.663+0000 7f14167fc640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 msgr2=0x7f1418070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.663+0000 7f14167fc640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 0x7f1418070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1418093610 tx=0x7f140800b8e0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f14080054e0 con 0x7f1418092db0 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 10 monclient: dump: 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.685 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1408005640 con 0x7f1418092db0 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 msgr2=0x7f1418070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 0x7f1418070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1418093610 tx=0x7f140800b8e0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 msgr2=0x7f1418070300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.664+0000 7f13ff7fe640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418092db0 0x7f1418070300 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1418093610 tx=0x7f140800b8e0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f141d391640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:03:42.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f141d391640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 0x7f1418070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f141d391640 10 monclient(hunting): start opening mon connection 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f141806b5d0 con 0x7f1418070840 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 0x7f1418070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient: get_auth_request con 0x7f1418070840 auth_method 0 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.670+0000 7f14167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.671+0000 7f14167fc640 10 monclient(hunting): handle_auth_done global_id 4633 payload 824 2024-09-16T15:03:42.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.671+0000 7f14167fc640 1 -- 172.21.15.134:0/459458354 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f141807a680 con 0x7f1418070840 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f14167fc640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 msgr2=0x7f1418070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f14167fc640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 0x7f1418070c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1408014000 tx=0x7f1408005a10 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1408020070 con 0x7f1418070840 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f13ff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f13ff7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.673+0000 7f13ff7fe640 10 monclient: dump: 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:03:42.690 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9317 (secure 0 0 0) 0x7f1408025430 con 0x7f1418070840 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 msgr2=0x7f1418070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 0x7f1418070c50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1408014000 tx=0x7f1408005a10 comp rx=0 tx=0).stop 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 msgr2=0x7f1418070c50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f13ff7fe640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1418070840 0x7f1418070c50 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1408014000 tx=0x7f1408005a10 comp rx=0 tx=0).stop 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 34, 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:03:42.692 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:03:42.695 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:03:42.696 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:03:42.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:03:42.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:03:42.709 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:03:42.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:03:42.712 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.713 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:03:42.714 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:03:42.715 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:03:42.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:03:42.716 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:03:42.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:03:42.716 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:03:42.716 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:03:42.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 >> v2:172.21.15.162:6800/980547263 conn(0x7f1404038880 msgr2=0x7f140403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 --2- 172.21.15.134:0/459458354 >> v2:172.21.15.162:6800/980547263 conn(0x7f1404038880 0x7f140403ad40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1410003d00 tx=0x7f1410005000 comp rx=0 tx=0).stop 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 10 monclient: shutdown 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 msgr2=0x7f14181cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 --2- 172.21.15.134:0/459458354 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f14181cb780 0x7f14181cbb60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c700 tx=0x7f140c00cbc0 comp rx=0 tx=0).stop 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f14177fe640 1 -- 172.21.15.134:0/459458354 reap_dead start 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f14177fe640 1 -- 172.21.15.134:0/459458354 reap_dead start 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 shutdown_connections 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 >> 172.21.15.134:0/459458354 conn(0x7f14180a6400 msgr2=0x7f141809a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 shutdown_connections 2024-09-16T15:03:42.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:03:42.674+0000 7f141d391640 1 -- 172.21.15.134:0/459458354 wait complete. 2024-09-16T15:03:42.719 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-16T15:03:42.719 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:03:43.379 INFO:tasks.rados.rados.0.smithi162.stdout:611: done (2 left) 2024-09-16T15:03:43.379 INFO:tasks.rados.rados.0.smithi162.stdout:612: snap_create 2024-09-16T15:03:43.379 INFO:tasks.rados.rados.0.smithi162.stdout:610: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: finishing write tid 1 to smithi16234619-13 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: finishing write tid 2 to smithi16234619-13 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: finishing write tid 3 to smithi16234619-13 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: finishing write tid 4 to smithi16234619-13 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: oid 13 updating version 0 to 244 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: oid 13 updating version 244 to 246 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: oid 13 version 246 is already newer than 245 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 246 (ObjNum 243 snap 49 seq_num 243) dirty exists 2024-09-16T15:03:43.465 INFO:tasks.rados.rados.0.smithi162.stdout:608: left oid 13 (ObjNum 243 snap 49 seq_num 243) 2024-09-16T15:03:44.379 INFO:tasks.rados.rados.0.smithi162.stdout:608: done (2 left) 2024-09-16T15:03:44.379 INFO:tasks.rados.rados.0.smithi162.stdout:610: done (1 left) 2024-09-16T15:03:44.379 INFO:tasks.rados.rados.0.smithi162.stdout:612: done (0 left) 2024-09-16T15:03:44.379 INFO:tasks.rados.rados.0.smithi162.stdout:613: write oid 26 current snap is 50 2024-09-16T15:03:44.379 INFO:tasks.rados.rados.0.smithi162.stdout:613: seq_num 245 ranges {517715=559301,1870206=247509} 2024-09-16T15:03:44.385 INFO:tasks.rados.rados.0.smithi162.stdout:613: writing smithi16234619-26 from 517715 to 1077016 tid 1 2024-09-16T15:03:44.390 INFO:tasks.rados.rados.0.smithi162.stdout:613: writing smithi16234619-26 from 1870206 to 2117715 tid 2 2024-09-16T15:03:44.391 INFO:tasks.rados.rados.0.smithi162.stdout:614: write oid 4 current snap is 50 2024-09-16T15:03:44.391 INFO:tasks.rados.rados.0.smithi162.stdout:614: seq_num 246 ranges {534522=634550,1812335=322187} 2024-09-16T15:03:44.397 INFO:tasks.rados.rados.0.smithi162.stdout:614: writing smithi16234619-4 from 534522 to 1169072 tid 1 2024-09-16T15:03:44.402 INFO:tasks.rados.rados.0.smithi162.stdout:614: writing smithi16234619-4 from 1812335 to 2134522 tid 2 2024-09-16T15:03:44.402 INFO:tasks.rados.rados.0.smithi162.stdout:615: read oid 15 snap -1 2024-09-16T15:03:44.402 INFO:tasks.rados.rados.0.smithi162.stdout:615: expect (ObjNum 242 snap 49 seq_num 242) 2024-09-16T15:03:44.403 INFO:tasks.rados.rados.0.smithi162.stdout:616: snap_remove snap 49 2024-09-16T15:03:45.375 INFO:tasks.rados.rados.0.smithi162.stdout:616: done (3 left) 2024-09-16T15:03:45.376 INFO:tasks.rados.rados.0.smithi162.stdout:617: write oid 24 current snap is 50 2024-09-16T15:03:45.376 INFO:tasks.rados.rados.0.smithi162.stdout:617: seq_num 247 ranges {551329=709799,1870817=280512} 2024-09-16T15:03:45.383 INFO:tasks.rados.rados.0.smithi162.stdout:617: writing smithi16234619-24 from 551329 to 1261128 tid 1 2024-09-16T15:03:45.388 INFO:tasks.rados.rados.0.smithi162.stdout:617: writing smithi16234619-24 from 1870817 to 2151329 tid 2 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:618: rollback oid 46 current snap is 50 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:618: done (4 left) 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:619: read oid 12 snap -1 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:619: expect (ObjNum 236 snap 46 seq_num 236) 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:620: read oid 17 snap -1 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:620: expect (ObjNum 230 snap 45 seq_num 230) 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:621: write oid 49 current snap is 50 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:613: finishing write tid 1 to smithi16234619-26 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:613: finishing write tid 2 to smithi16234619-26 2024-09-16T15:03:45.389 INFO:tasks.rados.rados.0.smithi162.stdout:621: seq_num 248 ranges {568136=785048,1929299=238837} 2024-09-16T15:03:45.396 INFO:tasks.rados.rados.0.smithi162.stdout:621: writing smithi16234619-49 from 568136 to 1353184 tid 1 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:621: writing smithi16234619-49 from 1929299 to 2168136 tid 2 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:622: read oid 7 snap -1 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:622: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:623: rollback oid 29 current snap is 50 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:623: done (8 left) 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:624: read oid 16 snap -1 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:624: expect (ObjNum 213 snap 44 seq_num 213) 2024-09-16T15:03:45.401 INFO:tasks.rados.rados.0.smithi162.stdout:625: read oid 43 snap -1 2024-09-16T15:03:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:625: expect (ObjNum 180 snap 38 seq_num 180) 2024-09-16T15:03:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:626: copy_from oid 11 from oid 40 current snap is 50 2024-09-16T15:03:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:615: expect (ObjNum 242 snap 49 seq_num 242) 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: finishing write tid 3 to smithi16234619-26 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: finishing write tid 4 to smithi16234619-26 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: oid 26 updating version 0 to 204 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: oid 26 updating version 204 to 205 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: oid 26 updating version 205 to 206 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 206 (ObjNum 245 snap 50 seq_num 245) dirty exists 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:613: left oid 26 (ObjNum 245 snap 50 seq_num 245) 2024-09-16T15:03:45.505 INFO:tasks.rados.rados.0.smithi162.stdout:614: finishing write tid 1 to smithi16234619-4 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: finishing write tid 2 to smithi16234619-4 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: finishing write tid 3 to smithi16234619-4 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:613: done (10 left) 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:615: done (9 left) 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:627: rollback oid 14 current snap is 50 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:627: done (9 left) 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:628: rollback oid 8 current snap is 50 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:628: done (9 left) 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:629: write oid 38 current snap is 50 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: finishing write tid 4 to smithi16234619-4 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: oid 4 updating version 0 to 219 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: oid 4 version 219 is already newer than 218 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: oid 4 version 219 is already newer than 217 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 219 (ObjNum 246 snap 50 seq_num 246) dirty exists 2024-09-16T15:03:45.506 INFO:tasks.rados.rados.0.smithi162.stdout:614: left oid 4 (ObjNum 246 snap 50 seq_num 246) 2024-09-16T15:03:45.507 INFO:tasks.rados.rados.0.smithi162.stdout:629: seq_num 249 ranges {584943=460297,1587781=597162} 2024-09-16T15:03:45.510 INFO:tasks.rados.rados.0.smithi162.stdout:629: writing smithi16234619-38 from 584943 to 1045240 tid 1 2024-09-16T15:03:45.518 INFO:tasks.rados.rados.0.smithi162.stdout:629: writing smithi16234619-38 from 1587781 to 2184943 tid 2 2024-09-16T15:03:45.518 INFO:tasks.rados.rados.0.smithi162.stdout:614: done (9 left) 2024-09-16T15:03:45.518 INFO:tasks.rados.rados.0.smithi162.stdout:630: rollback oid 25 current snap is 50 2024-09-16T15:03:45.518 INFO:tasks.rados.rados.0.smithi162.stdout:630: done (9 left) 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:631: read oid 22 snap -1 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:631: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:632: read oid 36 snap -1 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:632: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:633: write (excl) oid 33 current snap is 50 2024-09-16T15:03:45.519 INFO:tasks.rados.rados.0.smithi162.stdout:633: seq_num 250 ranges {601750=535546,1929910=271840} 2024-09-16T15:03:45.525 INFO:tasks.rados.rados.0.smithi162.stdout:633: writing smithi16234619-33 from 601750 to 1137296 tid 1 2024-09-16T15:03:45.531 INFO:tasks.rados.rados.0.smithi162.stdout:633: writing smithi16234619-33 from 1929910 to 2201750 tid 2 2024-09-16T15:03:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:634: delete oid 3 current snap is 50 2024-09-16T15:03:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:626: finishing copy_from to smithi16234619-11 2024-09-16T15:03:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:626: got expected ENOENT (src dne) 2024-09-16T15:03:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:626: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:03:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:619: expect (ObjNum 236 snap 46 seq_num 236) 2024-09-16T15:03:45.845 INFO:tasks.rados.rados.0.smithi162.stdout:625: expect (ObjNum 180 snap 38 seq_num 180) 2024-09-16T15:03:45.943 INFO:tasks.rados.rados.0.smithi162.stdout:617: finishing write tid 1 to smithi16234619-24 2024-09-16T15:03:45.943 INFO:tasks.rados.rados.0.smithi162.stdout:617: finishing write tid 2 to smithi16234619-24 2024-09-16T15:03:45.943 INFO:tasks.rados.rados.0.smithi162.stdout:622: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:03:46.189 INFO:tasks.rados.rados.0.smithi162.stdout:624: expect (ObjNum 213 snap 44 seq_num 213) 2024-09-16T15:03:46.343 INFO:tasks.rados.rados.0.smithi162.stdout:620: expect (ObjNum 230 snap 45 seq_num 230) 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: finishing write tid 3 to smithi16234619-24 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: finishing write tid 4 to smithi16234619-24 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: oid 24 updating version 0 to 210 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: oid 24 version 210 is already newer than 209 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: oid 24 updating version 210 to 211 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 211 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:617: left oid 24 (ObjNum 247 snap 50 seq_num 247) 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:621: finishing write tid 1 to smithi16234619-49 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:621: finishing write tid 2 to smithi16234619-49 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:621: finishing write tid 3 to smithi16234619-49 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:621: finishing write tid 4 to smithi16234619-49 2024-09-16T15:03:46.513 INFO:tasks.rados.rados.0.smithi162.stdout:621: oid 49 updating version 0 to 220 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:621: oid 49 version 220 is already newer than 219 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:621: oid 49 updating version 220 to 221 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 221 (ObjNum 248 snap 50 seq_num 248) dirty exists 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:621: left oid 49 (ObjNum 248 snap 50 seq_num 248) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:617: done (12 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:619: done (11 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:620: done (10 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:621: done (9 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:622: done (8 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:624: done (7 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:625: done (6 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:626: done (5 left) 2024-09-16T15:03:46.514 INFO:tasks.rados.rados.0.smithi162.stdout:634: done (4 left) 2024-09-16T15:03:46.515 INFO:tasks.rados.rados.0.smithi162.stdout:635: snap_create 2024-09-16T15:03:46.515 INFO:tasks.rados.rados.0.smithi162.stdout:631: expect (ObjNum 130 snap 24 seq_num 130) 2024-09-16T15:03:46.645 INFO:tasks.rados.rados.0.smithi162.stdout:632: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: finishing write tid 1 to smithi16234619-38 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: finishing write tid 2 to smithi16234619-38 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: finishing write tid 3 to smithi16234619-38 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: finishing write tid 4 to smithi16234619-38 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: oid 38 updating version 0 to 219 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: oid 38 version 219 is already newer than 218 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: oid 38 version 219 is already newer than 217 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 219 (ObjNum 249 snap 50 seq_num 249) dirty exists 2024-09-16T15:03:46.729 INFO:tasks.rados.rados.0.smithi162.stdout:629: left oid 38 (ObjNum 249 snap 50 seq_num 249) 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: finishing write tid 1 to smithi16234619-33 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: finishing write tid 2 to smithi16234619-33 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: finishing write tid 3 to smithi16234619-33 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: finishing write tid 4 to smithi16234619-33 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: oid 33 updating version 0 to 47 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: oid 33 updating version 47 to 48 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: oid 33 updating version 48 to 49 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 49 (ObjNum 250 snap 50 seq_num 250) dirty exists 2024-09-16T15:03:46.730 INFO:tasks.rados.rados.0.smithi162.stdout:633: left oid 33 (ObjNum 250 snap 50 seq_num 250) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:629: done (4 left) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:631: done (3 left) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:632: done (2 left) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:633: done (1 left) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:635: done (0 left) 2024-09-16T15:03:47.398 INFO:tasks.rados.rados.0.smithi162.stdout:636: delete oid 26 current snap is 51 2024-09-16T15:03:47.400 INFO:tasks.rados.rados.0.smithi162.stdout:636: done (0 left) 2024-09-16T15:03:47.400 INFO:tasks.rados.rados.0.smithi162.stdout:637: snap_remove snap 50 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:637: done (0 left) 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:638: rollback oid 29 current snap is 51 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:638: done (0 left) 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:639: copy_from oid 12 from oid 22 current snap is 51 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:640: rollback oid 9 current snap is 51 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:640: done (1 left) 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:641: rollback oid 38 current snap is 51 2024-09-16T15:03:48.409 INFO:tasks.rados.rados.0.smithi162.stdout:641: done (1 left) 2024-09-16T15:03:48.410 INFO:tasks.rados.rados.0.smithi162.stdout:642: rollback oid 44 current snap is 51 2024-09-16T15:03:48.410 INFO:tasks.rados.rados.0.smithi162.stdout:642: done (1 left) 2024-09-16T15:03:48.410 INFO:tasks.rados.rados.0.smithi162.stdout:643: write oid 9 current snap is 51 2024-09-16T15:03:48.410 INFO:tasks.rados.rados.0.smithi162.stdout:643: seq_num 253 ranges {652171=477646,1821709=430462} 2024-09-16T15:03:48.415 INFO:tasks.rados.rados.0.smithi162.stdout:643: writing smithi16234619-9 from 652171 to 1129817 tid 1 2024-09-16T15:03:48.421 INFO:tasks.rados.rados.0.smithi162.stdout:643: writing smithi16234619-9 from 1821709 to 2252171 tid 2 2024-09-16T15:03:48.421 INFO:tasks.rados.rados.0.smithi162.stdout:644: write (excl) oid 20 current snap is 51 2024-09-16T15:03:48.421 INFO:tasks.rados.rados.0.smithi162.stdout:644: seq_num 254 ranges {668978=552895,1763838=505140} 2024-09-16T15:03:48.428 INFO:tasks.rados.rados.0.smithi162.stdout:644: writing smithi16234619-20 from 668978 to 1221873 tid 1 2024-09-16T15:03:48.435 INFO:tasks.rados.rados.0.smithi162.stdout:644: writing smithi16234619-20 from 1763838 to 2268978 tid 2 2024-09-16T15:03:48.435 INFO:tasks.rados.rados.0.smithi162.stdout:645: snap_create 2024-09-16T15:03:48.444 INFO:tasks.rados.rados.0.smithi162.stdout:643: finishing write tid 1 to smithi16234619-9 2024-09-16T15:03:48.444 INFO:tasks.rados.rados.0.smithi162.stdout:643: finishing write tid 2 to smithi16234619-9 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: finishing write tid 3 to smithi16234619-9 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: finishing write tid 4 to smithi16234619-9 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: oid 9 updating version 0 to 222 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: oid 9 version 222 is already newer than 220 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: oid 9 version 222 is already newer than 221 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 222 (ObjNum 253 snap 51 seq_num 253) dirty exists 2024-09-16T15:03:48.447 INFO:tasks.rados.rados.0.smithi162.stdout:643: left oid 9 (ObjNum 253 snap 51 seq_num 253) 2024-09-16T15:03:48.455 INFO:tasks.rados.rados.0.smithi162.stdout:644: finishing write tid 1 to smithi16234619-20 2024-09-16T15:03:48.456 INFO:tasks.rados.rados.0.smithi162.stdout:644: finishing write tid 2 to smithi16234619-20 2024-09-16T15:03:48.456 INFO:tasks.rados.rados.0.smithi162.stdout:644: finishing write tid 3 to smithi16234619-20 2024-09-16T15:03:48.456 INFO:tasks.rados.rados.0.smithi162.stdout:644: finishing write tid 4 to smithi16234619-20 2024-09-16T15:03:48.456 INFO:tasks.rados.rados.0.smithi162.stdout:644: oid 20 updating version 0 to 52 2024-09-16T15:03:48.457 INFO:tasks.rados.rados.0.smithi162.stdout:644: oid 20 version 52 is already newer than 50 2024-09-16T15:03:48.457 INFO:tasks.rados.rados.0.smithi162.stdout:644: oid 20 version 52 is already newer than 51 2024-09-16T15:03:48.457 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 52 (ObjNum 254 snap 51 seq_num 254) dirty exists 2024-09-16T15:03:48.457 INFO:tasks.rados.rados.0.smithi162.stdout:644: left oid 20 (ObjNum 254 snap 51 seq_num 254) 2024-09-16T15:03:48.460 INFO:tasks.rados.rados.0.smithi162.stdout:639: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:03:48.465 INFO:tasks.rados.rados.0.smithi162.stdout:639: finishing copy_from to smithi16234619-12 2024-09-16T15:03:48.465 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:639: done (3 left) 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:643: done (2 left) 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:644: done (1 left) 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:645: done (0 left) 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:646: write (excl) oid 16 current snap is 52 2024-09-16T15:03:49.417 INFO:tasks.rados.rados.0.smithi162.stdout:646: seq_num 255 ranges {685785=628144,1822320=463465} 2024-09-16T15:03:49.424 INFO:tasks.rados.rados.0.smithi162.stdout:646: writing smithi16234619-16 from 685785 to 1313929 tid 1 2024-09-16T15:03:49.429 INFO:tasks.rados.rados.0.smithi162.stdout:646: writing smithi16234619-16 from 1822320 to 2285785 tid 2 2024-09-16T15:03:49.429 INFO:tasks.rados.rados.0.smithi162.stdout:647: read oid 20 snap -1 2024-09-16T15:03:49.430 INFO:tasks.rados.rados.0.smithi162.stdout:647: expect (ObjNum 254 snap 51 seq_num 254) 2024-09-16T15:03:49.430 INFO:tasks.rados.rados.0.smithi162.stdout:648: rollback oid 30 current snap is 52 2024-09-16T15:03:49.430 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 51 2024-09-16T15:03:49.430 INFO:tasks.rados.rados.0.smithi162.stdout:649: delete oid 37 current snap is 52 2024-09-16T15:03:49.433 INFO:tasks.rados.rados.0.smithi162.stdout:649: done (3 left) 2024-09-16T15:03:49.433 INFO:tasks.rados.rados.0.smithi162.stdout:650: snap_create 2024-09-16T15:03:49.434 INFO:tasks.rados.rados.0.smithi162.stdout:647: expect (ObjNum 254 snap 51 seq_num 254) 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:648: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:648: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:648: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 221 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:646: finishing write tid 1 to smithi16234619-16 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:646: finishing write tid 2 to smithi16234619-16 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:646: finishing write tid 3 to smithi16234619-16 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:646: finishing write tid 4 to smithi16234619-16 2024-09-16T15:03:49.516 INFO:tasks.rados.rados.0.smithi162.stdout:646: oid 16 updating version 0 to 222 2024-09-16T15:03:49.517 INFO:tasks.rados.rados.0.smithi162.stdout:646: oid 16 version 222 is already newer than 221 2024-09-16T15:03:49.517 INFO:tasks.rados.rados.0.smithi162.stdout:646: oid 16 version 222 is already newer than 220 2024-09-16T15:03:49.517 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 222 (ObjNum 255 snap 52 seq_num 255) dirty exists 2024-09-16T15:03:49.517 INFO:tasks.rados.rados.0.smithi162.stdout:646: left oid 16 (ObjNum 255 snap 52 seq_num 255) 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:646: done (3 left) 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:647: done (2 left) 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:648: done (1 left) 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:650: done (0 left) 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:651: read oid 46 snap 52 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:651: expect deleted 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:652: rollback oid 30 current snap is 53 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 52 2024-09-16T15:03:50.378 INFO:tasks.rados.rados.0.smithi162.stdout:653: snap_remove snap 51 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:653: done (2 left) 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:654: read oid 5 snap -1 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:654: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:655: read oid 29 snap -1 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:655: expect (ObjNum 234 snap 46 seq_num 234) 2024-09-16T15:03:51.380 INFO:tasks.rados.rados.0.smithi162.stdout:656: delete oid 47 current snap is 53 2024-09-16T15:03:51.381 INFO:tasks.rados.rados.0.smithi162.stdout:652: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:03:51.381 INFO:tasks.rados.rados.0.smithi162.stdout:652: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:03:51.381 INFO:tasks.rados.rados.0.smithi162.stdout:652: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:03:51.381 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 224 (ObjNum 214 snap 44 seq_num 214) dirty exists 2024-09-16T15:03:51.385 INFO:tasks.rados.rados.0.smithi162.stdout:651: done (4 left) 2024-09-16T15:03:51.385 INFO:tasks.rados.rados.0.smithi162.stdout:652: done (3 left) 2024-09-16T15:03:51.385 INFO:tasks.rados.rados.0.smithi162.stdout:656: done (2 left) 2024-09-16T15:03:51.385 INFO:tasks.rados.rados.0.smithi162.stdout:657: write (excl) oid 12 current snap is 53 2024-09-16T15:03:51.385 INFO:tasks.rados.rados.0.smithi162.stdout:657: seq_num 258 ranges {736206=453891,1881413=454793} 2024-09-16T15:03:51.392 INFO:tasks.rados.rados.0.smithi162.stdout:657: writing smithi16234619-12 from 736206 to 1190097 tid 1 2024-09-16T15:03:51.399 INFO:tasks.rados.rados.0.smithi162.stdout:657: writing smithi16234619-12 from 1881413 to 2336206 tid 2 2024-09-16T15:03:51.399 INFO:tasks.rados.rados.0.smithi162.stdout:658: write oid 3 current snap is 53 2024-09-16T15:03:51.399 INFO:tasks.rados.rados.0.smithi162.stdout:658: seq_num 259 ranges {753013=645493,2056248=296765} 2024-09-16T15:03:51.406 INFO:tasks.rados.rados.0.smithi162.stdout:658: writing smithi16234619-3 from 753013 to 1398506 tid 1 2024-09-16T15:03:51.412 INFO:tasks.rados.rados.0.smithi162.stdout:658: writing smithi16234619-3 from 2056248 to 2353013 tid 2 2024-09-16T15:03:51.412 INFO:tasks.rados.rados.0.smithi162.stdout:659: write oid 33 current snap is 53 2024-09-16T15:03:51.412 INFO:tasks.rados.rados.0.smithi162.stdout:659: seq_num 260 ranges {769820=720742,2114730=255090} 2024-09-16T15:03:51.420 INFO:tasks.rados.rados.0.smithi162.stdout:659: writing smithi16234619-33 from 769820 to 1490562 tid 1 2024-09-16T15:03:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:659: writing smithi16234619-33 from 2114730 to 2369820 tid 2 2024-09-16T15:03:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:660: rollback oid 23 current snap is 53 2024-09-16T15:03:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 52 2024-09-16T15:03:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:654: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:03:51.553 INFO:tasks.rados.rados.0.smithi162.stdout:655: expect (ObjNum 234 snap 46 seq_num 234) 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: finishing write tid 1 to smithi16234619-12 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: finishing write tid 2 to smithi16234619-12 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: finishing write tid 3 to smithi16234619-12 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: finishing write tid 4 to smithi16234619-12 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: oid 12 updating version 0 to 123 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: oid 12 updating version 123 to 124 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: oid 12 updating version 124 to 125 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 125 (ObjNum 258 snap 53 seq_num 258) dirty exists 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:657: left oid 12 (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:660: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:03:51.734 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 0 (ObjNum 129 snap 0 seq_num 861975352) dirty dne 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: finishing write tid 1 to smithi16234619-33 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: finishing write tid 2 to smithi16234619-33 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: finishing write tid 3 to smithi16234619-33 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: finishing write tid 4 to smithi16234619-33 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: oid 33 updating version 0 to 55 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: oid 33 updating version 55 to 56 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: oid 33 version 56 is already newer than 54 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 56 (ObjNum 260 snap 53 seq_num 260) dirty exists 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:659: left oid 33 (ObjNum 260 snap 53 seq_num 260) 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: finishing write tid 1 to smithi16234619-3 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: finishing write tid 2 to smithi16234619-3 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: finishing write tid 3 to smithi16234619-3 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: finishing write tid 4 to smithi16234619-3 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: oid 3 updating version 0 to 40 2024-09-16T15:03:51.735 INFO:tasks.rados.rados.0.smithi162.stdout:658: oid 3 version 40 is already newer than 39 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:658: oid 3 updating version 40 to 41 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 41 (ObjNum 259 snap 53 seq_num 259) dirty exists 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:658: left oid 3 (ObjNum 259 snap 53 seq_num 259) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:654: done (5 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:655: done (4 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:657: done (3 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:658: done (2 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:659: done (1 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:660: done (0 left) 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:661: copy_from oid 33 from oid 22 current snap is 53 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:662: read oid 32 snap -1 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:662: expect deleted 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:663: write oid 27 current snap is 53 2024-09-16T15:03:51.736 INFO:tasks.rados.rados.0.smithi162.stdout:663: seq_num 261 ranges {786627=795991,2173212=213415} 2024-09-16T15:03:51.743 INFO:tasks.rados.rados.0.smithi162.stdout:663: writing smithi16234619-27 from 786627 to 1582618 tid 1 2024-09-16T15:03:51.748 INFO:tasks.rados.rados.0.smithi162.stdout:663: writing smithi16234619-27 from 2173212 to 2386627 tid 2 2024-09-16T15:03:51.748 INFO:tasks.rados.rados.0.smithi162.stdout:664: write oid 35 current snap is 53 2024-09-16T15:03:51.748 INFO:tasks.rados.rados.0.smithi162.stdout:664: seq_num 262 ranges {403434=471240,1315341=400756} 2024-09-16T15:03:51.754 INFO:tasks.rados.rados.0.smithi162.stdout:664: writing smithi16234619-35 from 403434 to 874674 tid 1 2024-09-16T15:03:51.759 INFO:tasks.rados.rados.0.smithi162.stdout:664: writing smithi16234619-35 from 1315341 to 1716097 tid 2 2024-09-16T15:03:51.759 INFO:tasks.rados.rados.0.smithi162.stdout:665: snap_remove snap 52 2024-09-16T15:03:52.451 INFO:tasks.rados.rados.0.smithi162.stdout:665: done (4 left) 2024-09-16T15:03:52.451 INFO:tasks.rados.rados.0.smithi162.stdout:666: write oid 11 current snap is 53 2024-09-16T15:03:52.451 INFO:tasks.rados.rados.0.smithi162.stdout:666: seq_num 263 ranges {420241=546489,1373823=544414} 2024-09-16T15:03:52.456 INFO:tasks.rados.rados.0.smithi162.stdout:666: writing smithi16234619-11 from 420241 to 966730 tid 1 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:666: writing smithi16234619-11 from 1373823 to 1918237 tid 2 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:667: read oid 34 snap -1 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:667: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:668: copy_from oid 19 from oid 24 current snap is 53 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: finishing write tid 1 to smithi16234619-27 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: finishing write tid 2 to smithi16234619-27 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: finishing write tid 3 to smithi16234619-27 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: finishing write tid 4 to smithi16234619-27 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: oid 27 updating version 0 to 225 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: oid 27 version 225 is already newer than 223 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:663: oid 27 version 225 is already newer than 224 2024-09-16T15:03:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 225 (ObjNum 261 snap 53 seq_num 261) dirty exists 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:663: left oid 27 (ObjNum 261 snap 53 seq_num 261) 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:661: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:662: done (6 left) 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:663: done (5 left) 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:669: write (excl) oid 44 current snap is 53 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:661: finishing copy_from to smithi16234619-33 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 84 (ObjNum 130 snap 24 seq_num 130) dirty exists 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:664: finishing write tid 1 to smithi16234619-35 2024-09-16T15:03:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:669: seq_num 264 ranges {437048=621738,1832305=404425} 2024-09-16T15:03:52.468 INFO:tasks.rados.rados.0.smithi162.stdout:669: writing smithi16234619-44 from 437048 to 1058786 tid 1 2024-09-16T15:03:52.475 INFO:tasks.rados.rados.0.smithi162.stdout:669: writing smithi16234619-44 from 1832305 to 2236730 tid 2 2024-09-16T15:03:52.475 INFO:tasks.rados.rados.0.smithi162.stdout:661: done (5 left) 2024-09-16T15:03:52.475 INFO:tasks.rados.rados.0.smithi162.stdout:670: copy_from oid 16 from oid 50 current snap is 53 2024-09-16T15:03:52.475 INFO:tasks.rados.rados.0.smithi162.stdout:664: finishing write tid 2 to smithi16234619-35 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: finishing write tid 3 to smithi16234619-35 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: finishing write tid 4 to smithi16234619-35 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: oid 35 updating version 0 to 225 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: oid 35 version 225 is already newer than 224 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: oid 35 version 225 is already newer than 223 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 225 (ObjNum 262 snap 53 seq_num 262) dirty exists 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:664: left oid 35 (ObjNum 262 snap 53 seq_num 262) 2024-09-16T15:03:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:667: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:03:52.586 INFO:tasks.rados.rados.0.smithi162.stdout:666: finishing write tid 1 to smithi16234619-11 2024-09-16T15:03:52.586 INFO:tasks.rados.rados.0.smithi162.stdout:666: finishing write tid 2 to smithi16234619-11 2024-09-16T15:03:52.586 INFO:tasks.rados.rados.0.smithi162.stdout:666: finishing write tid 3 to smithi16234619-11 2024-09-16T15:03:52.586 INFO:tasks.rados.rados.0.smithi162.stdout:666: finishing write tid 4 to smithi16234619-11 2024-09-16T15:03:52.586 INFO:tasks.rados.rados.0.smithi162.stdout:666: oid 11 updating version 0 to 43 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:666: oid 11 updating version 43 to 45 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:666: oid 11 version 45 is already newer than 44 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 45 (ObjNum 263 snap 53 seq_num 263) dirty exists 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:666: left oid 11 (ObjNum 263 snap 53 seq_num 263) 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: finishing write tid 1 to smithi16234619-44 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:664: done (5 left) 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:666: done (4 left) 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:667: done (3 left) 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:671: snap_create 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: finishing write tid 2 to smithi16234619-44 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: finishing write tid 3 to smithi16234619-44 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: finishing write tid 4 to smithi16234619-44 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: oid 44 updating version 0 to 226 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: oid 44 updating version 226 to 228 2024-09-16T15:03:52.587 INFO:tasks.rados.rados.0.smithi162.stdout:669: oid 44 version 228 is already newer than 227 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 228 (ObjNum 264 snap 53 seq_num 264) dirty exists 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:669: left oid 44 (ObjNum 264 snap 53 seq_num 264) 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:668: finishing copy_from racing read to smithi16234619-19 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:668: finishing copy_from to smithi16234619-19 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 211 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:670: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:670: finishing copy_from to smithi16234619-16 2024-09-16T15:03:52.588 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 241 (ObjNum 240 snap 49 seq_num 240) dirty exists 2024-09-16T15:03:53.456 INFO:tasks.rados.rados.0.smithi162.stdout:668: done (3 left) 2024-09-16T15:03:53.456 INFO:tasks.rados.rados.0.smithi162.stdout:669: done (2 left) 2024-09-16T15:03:53.456 INFO:tasks.rados.rados.0.smithi162.stdout:670: done (1 left) 2024-09-16T15:03:53.456 INFO:tasks.rados.rados.0.smithi162.stdout:671: done (0 left) 2024-09-16T15:03:53.456 INFO:tasks.rados.rados.0.smithi162.stdout:672: write oid 28 current snap is 54 2024-09-16T15:03:53.457 INFO:tasks.rados.rados.0.smithi162.stdout:672: seq_num 265 ranges {453855=696987,1890787=548083} 2024-09-16T15:03:53.462 INFO:tasks.rados.rados.0.smithi162.stdout:672: writing smithi16234619-28 from 453855 to 1150842 tid 1 2024-09-16T15:03:53.469 INFO:tasks.rados.rados.0.smithi162.stdout:672: writing smithi16234619-28 from 1890787 to 2438870 tid 2 2024-09-16T15:03:53.469 INFO:tasks.rados.rados.0.smithi162.stdout:673: snap_create 2024-09-16T15:03:53.502 INFO:tasks.rados.rados.0.smithi162.stdout:672: finishing write tid 1 to smithi16234619-28 2024-09-16T15:03:53.503 INFO:tasks.rados.rados.0.smithi162.stdout:672: finishing write tid 2 to smithi16234619-28 2024-09-16T15:03:53.503 INFO:tasks.rados.rados.0.smithi162.stdout:672: finishing write tid 3 to smithi16234619-28 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:672: finishing write tid 4 to smithi16234619-28 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:672: oid 28 updating version 0 to 225 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:672: oid 28 updating version 225 to 227 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:672: oid 28 version 227 is already newer than 226 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 227 (ObjNum 265 snap 54 seq_num 265) dirty exists 2024-09-16T15:03:53.504 INFO:tasks.rados.rados.0.smithi162.stdout:672: left oid 28 (ObjNum 265 snap 54 seq_num 265) 2024-09-16T15:03:54.470 INFO:tasks.rados.rados.0.smithi162.stdout:672: done (1 left) 2024-09-16T15:03:54.470 INFO:tasks.rados.rados.0.smithi162.stdout:673: done (0 left) 2024-09-16T15:03:54.470 INFO:tasks.rados.rados.0.smithi162.stdout:674: snap_create 2024-09-16T15:03:55.379 INFO:tasks.rados.rados.0.smithi162.stdout:674: done (0 left) 2024-09-16T15:03:55.379 INFO:tasks.rados.rados.0.smithi162.stdout:675: delete oid 42 current snap is 56 2024-09-16T15:03:55.385 INFO:tasks.rados.rados.0.smithi162.stdout:675: done (0 left) 2024-09-16T15:03:55.385 INFO:tasks.rados.rados.0.smithi162.stdout:676: read oid 11 snap -1 2024-09-16T15:03:55.385 INFO:tasks.rados.rados.0.smithi162.stdout:676: expect (ObjNum 263 snap 53 seq_num 263) 2024-09-16T15:03:55.385 INFO:tasks.rados.rados.0.smithi162.stdout:677: read oid 46 snap 55 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:677: expect deleted 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:678: copy_from oid 21 from oid 19 current snap is 56 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:679: read oid 34 snap -1 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:679: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:680: write oid 18 current snap is 56 2024-09-16T15:03:55.386 INFO:tasks.rados.rados.0.smithi162.stdout:680: seq_num 267 ranges {487469=563838,1607751=551752} 2024-09-16T15:03:55.391 INFO:tasks.rados.rados.0.smithi162.stdout:680: writing smithi16234619-18 from 487469 to 1051307 tid 1 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:680: writing smithi16234619-18 from 1607751 to 2159503 tid 2 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:681: read oid 16 snap -1 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:681: expect (ObjNum 240 snap 49 seq_num 240) 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:682: rollback oid 27 current snap is 56 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 27 to 55 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:677: done (6 left) 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:683: write (excl) oid 37 current snap is 56 2024-09-16T15:03:55.399 INFO:tasks.rados.rados.0.smithi162.stdout:683: seq_num 268 ranges {504276=639087,1666233=695410} 2024-09-16T15:03:55.405 INFO:tasks.rados.rados.0.smithi162.stdout:683: writing smithi16234619-37 from 504276 to 1143363 tid 1 2024-09-16T15:03:55.412 INFO:tasks.rados.rados.0.smithi162.stdout:683: writing smithi16234619-37 from 1666233 to 2361643 tid 2 2024-09-16T15:03:55.412 INFO:tasks.rados.rados.0.smithi162.stdout:684: snap_create 2024-09-16T15:03:55.412 INFO:tasks.rados.rados.0.smithi162.stdout:676: expect (ObjNum 263 snap 53 seq_num 263) 2024-09-16T15:03:55.581 INFO:tasks.rados.rados.0.smithi162.stdout:682: finishing rollback tid 0 to smithi16234619-27 2024-09-16T15:03:55.581 INFO:tasks.rados.rados.0.smithi162.stdout:682: finishing rollback tid 1 to smithi16234619-27 2024-09-16T15:03:55.581 INFO:tasks.rados.rados.0.smithi162.stdout:682: finishing rollback tid 2 to smithi16234619-27 2024-09-16T15:03:55.581 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 228 (ObjNum 261 snap 53 seq_num 261) dirty exists 2024-09-16T15:03:55.581 INFO:tasks.rados.rados.0.smithi162.stdout:681: expect (ObjNum 240 snap 49 seq_num 240) 2024-09-16T15:03:55.661 INFO:tasks.rados.rados.0.smithi162.stdout:679: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:03:55.761 INFO:tasks.rados.rados.0.smithi162.stdout:680: finishing write tid 1 to smithi16234619-18 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: finishing write tid 2 to smithi16234619-18 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: finishing write tid 3 to smithi16234619-18 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: finishing write tid 4 to smithi16234619-18 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: oid 18 updating version 0 to 179 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: oid 18 updating version 179 to 181 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: oid 18 version 181 is already newer than 180 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 181 (ObjNum 267 snap 56 seq_num 267) dirty exists 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:680: left oid 18 (ObjNum 267 snap 56 seq_num 267) 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:683: finishing write tid 1 to smithi16234619-37 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:683: finishing write tid 2 to smithi16234619-37 2024-09-16T15:03:55.762 INFO:tasks.rados.rados.0.smithi162.stdout:683: finishing write tid 3 to smithi16234619-37 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:683: finishing write tid 4 to smithi16234619-37 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:683: oid 37 updating version 0 to 259 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:683: oid 37 version 259 is already newer than 258 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:683: oid 37 version 259 is already newer than 257 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 259 (ObjNum 268 snap 56 seq_num 268) dirty exists 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:683: left oid 37 (ObjNum 268 snap 56 seq_num 268) 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:678: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:678: finishing copy_from to smithi16234619-21 2024-09-16T15:03:55.763 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 211 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:03:56.517 INFO:tasks.rados.rados.0.smithi162.stdout:676: done (7 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:678: done (6 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:679: done (5 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:680: done (4 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:681: done (3 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:682: done (2 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:683: done (1 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:684: done (0 left) 2024-09-16T15:03:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:685: snap_remove snap 53 2024-09-16T15:03:57.864 INFO:tasks.rados.rados.0.smithi162.stdout:685: done (0 left) 2024-09-16T15:03:57.864 INFO:tasks.rados.rados.0.smithi162.stdout:686: copy_from oid 29 from oid 41 current snap is 57 2024-09-16T15:03:57.864 INFO:tasks.rados.rados.0.smithi162.stdout:687: copy_from oid 25 from oid 46 current snap is 57 2024-09-16T15:03:57.865 INFO:tasks.rados.rados.0.smithi162.stdout:688: write oid 4 current snap is 57 2024-09-16T15:03:57.865 INFO:tasks.rados.rados.0.smithi162.stdout:688: seq_num 269 ranges {521083=714336,1724715=555421} 2024-09-16T15:03:57.871 INFO:tasks.rados.rados.0.smithi162.stdout:688: writing smithi16234619-4 from 521083 to 1235419 tid 1 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:688: writing smithi16234619-4 from 1724715 to 2280136 tid 2 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:689: copy_from oid 19 from oid 18 current snap is 57 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:687: finishing copy_from to smithi16234619-25 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:687: got expected ENOENT (src dne) 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:687: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:687: done (3 left) 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:690: copy_from oid 47 from oid 2 current snap is 57 2024-09-16T15:03:57.879 INFO:tasks.rados.rados.0.smithi162.stdout:691: snap_remove snap 54 2024-09-16T15:03:58.875 INFO:tasks.rados.rados.0.smithi162.stdout:691: done (4 left) 2024-09-16T15:03:58.875 INFO:tasks.rados.rados.0.smithi162.stdout:692: write (excl) oid 32 current snap is 57 2024-09-16T15:03:58.876 INFO:tasks.rados.rados.0.smithi162.stdout:692: seq_num 270 ranges {537890=789585,1783197=699079} 2024-09-16T15:03:58.883 INFO:tasks.rados.rados.0.smithi162.stdout:692: writing smithi16234619-32 from 537890 to 1327475 tid 1 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:692: writing smithi16234619-32 from 1783197 to 2482276 tid 2 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:693: rollback oid 10 current snap is 57 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 55 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:688: finishing write tid 1 to smithi16234619-4 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:688: finishing write tid 2 to smithi16234619-4 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:688: finishing write tid 3 to smithi16234619-4 2024-09-16T15:03:58.890 INFO:tasks.rados.rados.0.smithi162.stdout:688: finishing write tid 4 to smithi16234619-4 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:688: oid 4 updating version 0 to 230 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:688: oid 4 updating version 230 to 231 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:688: oid 4 version 231 is already newer than 229 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 231 (ObjNum 269 snap 57 seq_num 269) dirty exists 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:688: left oid 4 (ObjNum 269 snap 57 seq_num 269) 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:686: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:690: finishing copy_from to smithi16234619-47 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:690: got expected ENOENT (src dne) 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:690: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:689: finishing copy_from racing read to smithi16234619-19 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:689: finishing copy_from to smithi16234619-19 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 181 (ObjNum 267 snap 56 seq_num 267) dirty exists 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:688: done (5 left) 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:689: done (4 left) 2024-09-16T15:03:58.891 INFO:tasks.rados.rados.0.smithi162.stdout:690: done (3 left) 2024-09-16T15:03:58.892 INFO:tasks.rados.rados.0.smithi162.stdout:694: delete oid 14 current snap is 57 2024-09-16T15:03:58.892 INFO:tasks.rados.rados.0.smithi162.stdout:686: finishing copy_from to smithi16234619-29 2024-09-16T15:03:58.892 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 51 (ObjNum 227 snap 45 seq_num 227) dirty exists 2024-09-16T15:03:58.892 INFO:tasks.rados.rados.0.smithi162.stdout:693: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:03:58.892 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 0 (ObjNum 337 snap 0 seq_num 865363136) dirty dne 2024-09-16T15:03:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:686: done (3 left) 2024-09-16T15:03:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:693: done (2 left) 2024-09-16T15:03:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:694: done (1 left) 2024-09-16T15:03:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:695: rollback oid 31 current snap is 57 2024-09-16T15:03:58.896 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 56 2024-09-16T15:03:58.896 INFO:tasks.rados.rados.0.smithi162.stdout:696: read oid 12 snap -1 2024-09-16T15:03:58.896 INFO:tasks.rados.rados.0.smithi162.stdout:696: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:03:58.896 INFO:tasks.rados.rados.0.smithi162.stdout:697: write oid 23 current snap is 57 2024-09-16T15:03:58.896 INFO:tasks.rados.rados.0.smithi162.stdout:697: seq_num 272 ranges {571504=540083,1783808=702748} 2024-09-16T15:03:58.901 INFO:tasks.rados.rados.0.smithi162.stdout:697: writing smithi16234619-23 from 571504 to 1111587 tid 1 2024-09-16T15:03:58.909 INFO:tasks.rados.rados.0.smithi162.stdout:697: writing smithi16234619-23 from 1783808 to 2486556 tid 2 2024-09-16T15:03:58.909 INFO:tasks.rados.rados.0.smithi162.stdout:698: write (excl) oid 37 current snap is 57 2024-09-16T15:03:58.909 INFO:tasks.rados.rados.0.smithi162.stdout:695: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:03:58.910 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 0 (ObjNum 929 snap 0 seq_num 2027921520) dirty dne 2024-09-16T15:03:58.910 INFO:tasks.rados.rados.0.smithi162.stdout:698: seq_num 273 ranges {588311=615332,1842290=562759} 2024-09-16T15:03:58.915 INFO:tasks.rados.rados.0.smithi162.stdout:698: writing smithi16234619-37 from 588311 to 1203643 tid 1 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:698: writing smithi16234619-37 from 1842290 to 2405049 tid 2 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:695: done (4 left) 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:699: copy_from oid 22 from oid 48 current snap is 57 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:692: finishing write tid 1 to smithi16234619-32 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:700: write (excl) oid 33 current snap is 57 2024-09-16T15:03:58.924 INFO:tasks.rados.rados.0.smithi162.stdout:700: seq_num 274 ranges {605118=406934,1617125=706417} 2024-09-16T15:03:58.932 INFO:tasks.rados.rados.0.smithi162.stdout:700: writing smithi16234619-33 from 605118 to 1012052 tid 1 2024-09-16T15:03:58.940 INFO:tasks.rados.rados.0.smithi162.stdout:700: writing smithi16234619-33 from 1617125 to 2323542 tid 2 2024-09-16T15:03:58.940 INFO:tasks.rados.rados.0.smithi162.stdout:701: read oid 14 snap -1 2024-09-16T15:03:58.940 INFO:tasks.rados.rados.0.smithi162.stdout:701: expect deleted 2024-09-16T15:03:58.940 INFO:tasks.rados.rados.0.smithi162.stdout:702: write (excl) oid 30 current snap is 57 2024-09-16T15:03:58.940 INFO:tasks.rados.rados.0.smithi162.stdout:702: seq_num 275 ranges {621925=482183,1559254=566428} 2024-09-16T15:03:58.946 INFO:tasks.rados.rados.0.smithi162.stdout:702: writing smithi16234619-30 from 621925 to 1104108 tid 1 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:702: writing smithi16234619-30 from 1559254 to 2125682 tid 2 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:703: read oid 2 snap -1 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:703: expect deleted 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:704: rollback oid 6 current snap is 57 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 56 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:692: finishing write tid 2 to smithi16234619-32 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:692: finishing write tid 3 to smithi16234619-32 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:705: snap_create 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:692: finishing write tid 4 to smithi16234619-32 2024-09-16T15:03:58.952 INFO:tasks.rados.rados.0.smithi162.stdout:692: oid 32 updating version 0 to 129 2024-09-16T15:03:58.953 INFO:tasks.rados.rados.0.smithi162.stdout:692: oid 32 updating version 129 to 130 2024-09-16T15:03:58.953 INFO:tasks.rados.rados.0.smithi162.stdout:692: oid 32 version 130 is already newer than 128 2024-09-16T15:03:58.953 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 130 (ObjNum 270 snap 57 seq_num 270) dirty exists 2024-09-16T15:03:58.953 INFO:tasks.rados.rados.0.smithi162.stdout:692: left oid 32 (ObjNum 270 snap 57 seq_num 270) 2024-09-16T15:03:58.953 INFO:tasks.rados.rados.0.smithi162.stdout:696: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: finishing write tid 1 to smithi16234619-23 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: finishing write tid 2 to smithi16234619-23 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: finishing write tid 3 to smithi16234619-23 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: finishing write tid 4 to smithi16234619-23 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: oid 23 updating version 0 to 62 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: oid 23 updating version 62 to 63 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: oid 23 updating version 63 to 64 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 64 (ObjNum 272 snap 57 seq_num 272) dirty exists 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:697: left oid 23 (ObjNum 272 snap 57 seq_num 272) 2024-09-16T15:03:59.121 INFO:tasks.rados.rados.0.smithi162.stdout:698: finishing write tid 1 to smithi16234619-37 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:704: finishing rollback tid 0 to smithi16234619-6 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: finishing write tid 2 to smithi16234619-37 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:704: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: finishing write tid 3 to smithi16234619-37 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: finishing write tid 4 to smithi16234619-37 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: oid 37 updating version 0 to 270 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: oid 37 version 270 is already newer than 269 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: oid 37 updating version 270 to 271 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 271 (ObjNum 273 snap 57 seq_num 273) dirty exists 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:698: left oid 37 (ObjNum 273 snap 57 seq_num 273) 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:704: finishing rollback tid 2 to smithi16234619-6 2024-09-16T15:03:59.122 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 244 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: finishing write tid 1 to smithi16234619-33 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: finishing write tid 2 to smithi16234619-33 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: finishing write tid 3 to smithi16234619-33 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: finishing write tid 4 to smithi16234619-33 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: oid 33 updating version 0 to 85 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: oid 33 updating version 85 to 87 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: oid 33 version 87 is already newer than 86 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 87 (ObjNum 274 snap 57 seq_num 274) dirty exists 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:700: left oid 33 (ObjNum 274 snap 57 seq_num 274) 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: finishing write tid 1 to smithi16234619-30 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: finishing write tid 2 to smithi16234619-30 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: finishing write tid 3 to smithi16234619-30 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: finishing write tid 4 to smithi16234619-30 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: oid 30 updating version 0 to 228 2024-09-16T15:03:59.123 INFO:tasks.rados.rados.0.smithi162.stdout:702: oid 30 updating version 228 to 229 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:702: oid 30 updating version 229 to 230 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 230 (ObjNum 275 snap 57 seq_num 275) dirty exists 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:702: left oid 30 (ObjNum 275 snap 57 seq_num 275) 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:699: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:699: finishing copy_from to smithi16234619-22 2024-09-16T15:03:59.124 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 165 (ObjNum 211 snap 44 seq_num 211) dirty exists 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:692: done (10 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:696: done (9 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:697: done (8 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:698: done (7 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:699: done (6 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:700: done (5 left) 2024-09-16T15:03:59.873 INFO:tasks.rados.rados.0.smithi162.stdout:701: done (4 left) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:702: done (3 left) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:703: done (2 left) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:704: done (1 left) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:705: done (0 left) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:706: read oid 37 snap -1 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:706: expect (ObjNum 273 snap 57 seq_num 273) 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:707: write (excl) oid 26 current snap is 58 2024-09-16T15:03:59.874 INFO:tasks.rados.rados.0.smithi162.stdout:707: seq_num 276 ranges {638732=557432,1617736=710086} 2024-09-16T15:03:59.879 INFO:tasks.rados.rados.0.smithi162.stdout:707: writing smithi16234619-26 from 638732 to 1196164 tid 1 2024-09-16T15:03:59.886 INFO:tasks.rados.rados.0.smithi162.stdout:707: writing smithi16234619-26 from 1617736 to 2327822 tid 2 2024-09-16T15:03:59.886 INFO:tasks.rados.rados.0.smithi162.stdout:708: delete oid 38 current snap is 58 2024-09-16T15:03:59.886 INFO:tasks.rados.rados.0.smithi162.stdout:706: expect (ObjNum 273 snap 57 seq_num 273) 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: finishing write tid 1 to smithi16234619-26 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: finishing write tid 2 to smithi16234619-26 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: finishing write tid 3 to smithi16234619-26 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: finishing write tid 4 to smithi16234619-26 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: oid 26 updating version 0 to 222 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: oid 26 updating version 222 to 224 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: oid 26 version 224 is already newer than 223 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 224 (ObjNum 276 snap 58 seq_num 276) dirty exists 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: left oid 26 (ObjNum 276 snap 58 seq_num 276) 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:706: done (2 left) 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:707: done (1 left) 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:708: done (0 left) 2024-09-16T15:04:00.007 INFO:tasks.rados.rados.0.smithi162.stdout:709: rollback oid 18 current snap is 58 2024-09-16T15:04:00.008 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 56 2024-09-16T15:04:00.008 INFO:tasks.rados.rados.0.smithi162.stdout:710: read oid 15 snap -1 2024-09-16T15:04:00.008 INFO:tasks.rados.rados.0.smithi162.stdout:710: expect (ObjNum 242 snap 49 seq_num 242) 2024-09-16T15:04:00.008 INFO:tasks.rados.rados.0.smithi162.stdout:711: snap_remove snap 57 2024-09-16T15:04:00.383 INFO:tasks.rados.rados.0.smithi162.stdout:711: done (2 left) 2024-09-16T15:04:00.383 INFO:tasks.rados.rados.0.smithi162.stdout:712: read oid 5 snap 56 2024-09-16T15:04:00.383 INFO:tasks.rados.rados.0.smithi162.stdout:712: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:04:00.383 INFO:tasks.rados.rados.0.smithi162.stdout:713: write oid 24 current snap is 58 2024-09-16T15:04:00.384 INFO:tasks.rados.rados.0.smithi162.stdout:713: seq_num 278 ranges {672346=707930,2134700=537646} 2024-09-16T15:04:00.391 INFO:tasks.rados.rados.0.smithi162.stdout:713: writing smithi16234619-24 from 672346 to 1380276 tid 1 2024-09-16T15:04:00.398 INFO:tasks.rados.rados.0.smithi162.stdout:713: writing smithi16234619-24 from 2134700 to 2672346 tid 2 2024-09-16T15:04:00.398 INFO:tasks.rados.rados.0.smithi162.stdout:714: read oid 25 snap 55 2024-09-16T15:04:00.398 INFO:tasks.rados.rados.0.smithi162.stdout:714: expect (ObjNum 168 snap 34 seq_num 168) 2024-09-16T15:04:00.398 INFO:tasks.rados.rados.0.smithi162.stdout:715: read oid 8 snap -1 2024-09-16T15:04:00.398 INFO:tasks.rados.rados.0.smithi162.stdout:715: expect (ObjNum 228 snap 45 seq_num 228) 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:716: write oid 38 current snap is 58 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:709: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:709: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:709: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 184 (ObjNum 267 snap 56 seq_num 267) dirty exists 2024-09-16T15:04:00.399 INFO:tasks.rados.rados.0.smithi162.stdout:710: expect (ObjNum 242 snap 49 seq_num 242) 2024-09-16T15:04:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:712: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:04:00.621 INFO:tasks.rados.rados.0.smithi162.stdout:715: expect (ObjNum 228 snap 45 seq_num 228) 2024-09-16T15:04:00.850 INFO:tasks.rados.rados.0.smithi162.stdout:714: expect (ObjNum 168 snap 34 seq_num 168) 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: finishing write tid 1 to smithi16234619-24 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: finishing write tid 2 to smithi16234619-24 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: finishing write tid 3 to smithi16234619-24 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: finishing write tid 4 to smithi16234619-24 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: oid 24 updating version 0 to 228 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: oid 24 version 228 is already newer than 226 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: oid 24 version 228 is already newer than 227 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 228 (ObjNum 278 snap 58 seq_num 278) dirty exists 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:713: left oid 24 (ObjNum 278 snap 58 seq_num 278) 2024-09-16T15:04:00.944 INFO:tasks.rados.rados.0.smithi162.stdout:716: seq_num 279 ranges {689153=783179,2076829=457413} 2024-09-16T15:04:00.951 INFO:tasks.rados.rados.0.smithi162.stdout:716: writing smithi16234619-38 from 689153 to 1472332 tid 1 2024-09-16T15:04:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:716: writing smithi16234619-38 from 2076829 to 2534242 tid 2 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:709: done (6 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:710: done (5 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:712: done (4 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:713: done (3 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:714: done (2 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:715: done (1 left) 2024-09-16T15:04:00.959 INFO:tasks.rados.rados.0.smithi162.stdout:717: snap_create 2024-09-16T15:04:00.980 INFO:tasks.rados.rados.0.smithi162.stdout:716: finishing write tid 1 to smithi16234619-38 2024-09-16T15:04:00.980 INFO:tasks.rados.rados.0.smithi162.stdout:716: finishing write tid 2 to smithi16234619-38 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: finishing write tid 3 to smithi16234619-38 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: finishing write tid 4 to smithi16234619-38 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: oid 38 updating version 0 to 248 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: oid 38 version 248 is already newer than 247 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: oid 38 version 248 is already newer than 246 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 248 (ObjNum 279 snap 58 seq_num 279) dirty exists 2024-09-16T15:04:00.983 INFO:tasks.rados.rados.0.smithi162.stdout:716: left oid 38 (ObjNum 279 snap 58 seq_num 279) 2024-09-16T15:04:01.894 INFO:tasks.rados.rados.0.smithi162.stdout:716: done (1 left) 2024-09-16T15:04:01.894 INFO:tasks.rados.rados.0.smithi162.stdout:717: done (0 left) 2024-09-16T15:04:01.894 INFO:tasks.rados.rados.0.smithi162.stdout:718: write (excl) oid 47 current snap is 59 2024-09-16T15:04:01.894 INFO:tasks.rados.rados.0.smithi162.stdout:718: seq_num 280 ranges {705960=458428,1735311=717424} 2024-09-16T15:04:01.900 INFO:tasks.rados.rados.0.smithi162.stdout:718: writing smithi16234619-47 from 705960 to 1164388 tid 1 2024-09-16T15:04:01.908 INFO:tasks.rados.rados.0.smithi162.stdout:718: writing smithi16234619-47 from 1735311 to 2452735 tid 2 2024-09-16T15:04:01.908 INFO:tasks.rados.rados.0.smithi162.stdout:719: write oid 46 current snap is 59 2024-09-16T15:04:01.908 INFO:tasks.rados.rados.0.smithi162.stdout:719: seq_num 281 ranges {722767=533677,1793793=461082} 2024-09-16T15:04:01.916 INFO:tasks.rados.rados.0.smithi162.stdout:719: writing smithi16234619-46 from 722767 to 1256444 tid 1 2024-09-16T15:04:01.921 INFO:tasks.rados.rados.0.smithi162.stdout:719: writing smithi16234619-46 from 1793793 to 2254875 tid 2 2024-09-16T15:04:01.921 INFO:tasks.rados.rados.0.smithi162.stdout:720: write (excl) oid 22 current snap is 59 2024-09-16T15:04:01.922 INFO:tasks.rados.rados.0.smithi162.stdout:720: seq_num 282 ranges {739574=725279,1968628=721093} 2024-09-16T15:04:01.929 INFO:tasks.rados.rados.0.smithi162.stdout:720: writing smithi16234619-22 from 739574 to 1464853 tid 1 2024-09-16T15:04:01.936 INFO:tasks.rados.rados.0.smithi162.stdout:720: writing smithi16234619-22 from 1968628 to 2689721 tid 2 2024-09-16T15:04:01.936 INFO:tasks.rados.rados.0.smithi162.stdout:721: snap_remove snap 55 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:721: done (3 left) 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:722: read oid 7 snap -1 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:722: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:723: write oid 28 current snap is 59 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:718: finishing write tid 1 to smithi16234619-47 2024-09-16T15:04:02.906 INFO:tasks.rados.rados.0.smithi162.stdout:723: seq_num 283 ranges {756381=400528,1910757=464751} 2024-09-16T15:04:02.912 INFO:tasks.rados.rados.0.smithi162.stdout:723: writing smithi16234619-28 from 756381 to 1156909 tid 1 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:723: writing smithi16234619-28 from 1910757 to 2375508 tid 2 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:724: read oid 42 snap -1 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:724: expect deleted 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:725: write oid 18 current snap is 59 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:718: finishing write tid 2 to smithi16234619-47 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:718: finishing write tid 3 to smithi16234619-47 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:718: finishing write tid 4 to smithi16234619-47 2024-09-16T15:04:02.919 INFO:tasks.rados.rados.0.smithi162.stdout:718: oid 47 updating version 0 to 63 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:718: oid 47 version 63 is already newer than 62 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:718: oid 47 updating version 63 to 64 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 64 (ObjNum 280 snap 59 seq_num 280) dirty exists 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:718: left oid 47 (ObjNum 280 snap 59 seq_num 280) 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: finishing write tid 1 to smithi16234619-46 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: finishing write tid 2 to smithi16234619-46 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: finishing write tid 3 to smithi16234619-46 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: finishing write tid 4 to smithi16234619-46 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: oid 46 updating version 0 to 92 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: oid 46 version 92 is already newer than 90 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: oid 46 version 92 is already newer than 91 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:04:02.920 INFO:tasks.rados.rados.0.smithi162.stdout:719: left oid 46 (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:04:02.921 INFO:tasks.rados.rados.0.smithi162.stdout:720: finishing write tid 1 to smithi16234619-22 2024-09-16T15:04:02.921 INFO:tasks.rados.rados.0.smithi162.stdout:720: finishing write tid 2 to smithi16234619-22 2024-09-16T15:04:02.921 INFO:tasks.rados.rados.0.smithi162.stdout:725: seq_num 284 ranges {773188=475777,1969239=724762} 2024-09-16T15:04:02.926 INFO:tasks.rados.rados.0.smithi162.stdout:725: writing smithi16234619-18 from 773188 to 1248965 tid 1 2024-09-16T15:04:02.934 INFO:tasks.rados.rados.0.smithi162.stdout:725: writing smithi16234619-18 from 1969239 to 2694001 tid 2 2024-09-16T15:04:02.934 INFO:tasks.rados.rados.0.smithi162.stdout:718: done (6 left) 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:719: done (5 left) 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:726: rollback oid 15 current snap is 59 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 56 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: finishing write tid 3 to smithi16234619-22 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: finishing write tid 4 to smithi16234619-22 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: oid 22 updating version 0 to 223 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: oid 22 updating version 223 to 224 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: oid 22 version 224 is already newer than 222 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 224 (ObjNum 282 snap 59 seq_num 282) dirty exists 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:720: left oid 22 (ObjNum 282 snap 59 seq_num 282) 2024-09-16T15:04:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:722: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:04:03.181 INFO:tasks.rados.rados.0.smithi162.stdout:723: finishing write tid 1 to smithi16234619-28 2024-09-16T15:04:03.181 INFO:tasks.rados.rados.0.smithi162.stdout:723: finishing write tid 2 to smithi16234619-28 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: finishing write tid 3 to smithi16234619-28 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: finishing write tid 4 to smithi16234619-28 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: oid 28 updating version 0 to 231 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: oid 28 updating version 231 to 233 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: oid 28 version 233 is already newer than 232 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 233 (ObjNum 283 snap 59 seq_num 283) dirty exists 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: left oid 28 (ObjNum 283 snap 59 seq_num 283) 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:726: finishing rollback tid 0 to smithi16234619-15 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:726: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:720: done (5 left) 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:722: done (4 left) 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:723: done (3 left) 2024-09-16T15:04:03.182 INFO:tasks.rados.rados.0.smithi162.stdout:724: done (2 left) 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:727: rollback oid 21 current snap is 59 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 56 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:726: finishing rollback tid 2 to smithi16234619-15 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 236 (ObjNum 242 snap 49 seq_num 242) dirty exists 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: finishing write tid 1 to smithi16234619-18 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: finishing write tid 2 to smithi16234619-18 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: finishing write tid 3 to smithi16234619-18 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: finishing write tid 4 to smithi16234619-18 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: oid 18 updating version 0 to 187 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: oid 18 version 187 is already newer than 185 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: oid 18 version 187 is already newer than 186 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 187 (ObjNum 284 snap 59 seq_num 284) dirty exists 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: left oid 18 (ObjNum 284 snap 59 seq_num 284) 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:725: done (2 left) 2024-09-16T15:04:03.183 INFO:tasks.rados.rados.0.smithi162.stdout:726: done (1 left) 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:728: copy_from oid 1 from oid 8 current snap is 59 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:729: rollback oid 28 current snap is 59 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 58 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:730: copy_from oid 22 from oid 6 current snap is 59 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:731: rollback oid 30 current snap is 59 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 58 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:732: write oid 38 current snap is 59 2024-09-16T15:04:03.184 INFO:tasks.rados.rados.0.smithi162.stdout:732: seq_num 285 ranges {789995=551026,2027721=468420} 2024-09-16T15:04:03.188 INFO:tasks.rados.rados.0.smithi162.stdout:732: writing smithi16234619-38 from 789995 to 1341021 tid 1 2024-09-16T15:04:03.195 INFO:tasks.rados.rados.0.smithi162.stdout:732: writing smithi16234619-38 from 2027721 to 2496141 tid 2 2024-09-16T15:04:03.195 INFO:tasks.rados.rados.0.smithi162.stdout:733: write oid 43 current snap is 59 2024-09-16T15:04:03.195 INFO:tasks.rados.rados.0.smithi162.stdout:727: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:727: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:729: finishing rollback tid 0 to smithi16234619-28 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:729: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:727: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 236 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:729: finishing rollback tid 2 to smithi16234619-28 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 239 (ObjNum 265 snap 54 seq_num 265) dirty exists 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:731: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:731: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:731: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 242 (ObjNum 275 snap 57 seq_num 275) dirty exists 2024-09-16T15:04:03.196 INFO:tasks.rados.rados.0.smithi162.stdout:733: seq_num 286 ranges {406802=626275,1686203=728431} 2024-09-16T15:04:03.200 INFO:tasks.rados.rados.0.smithi162.stdout:733: writing smithi16234619-43 from 406802 to 1033077 tid 1 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:733: writing smithi16234619-43 from 1686203 to 2414634 tid 2 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:727: done (6 left) 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:729: done (5 left) 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:731: done (4 left) 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:734: read oid 5 snap -1 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:734: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:04:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:735: snap_create 2024-09-16T15:04:03.217 INFO:tasks.rados.rados.0.smithi162.stdout:734: expect (ObjNum 229 snap 45 seq_num 229) 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: finishing write tid 1 to smithi16234619-38 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: finishing write tid 2 to smithi16234619-38 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: finishing write tid 3 to smithi16234619-38 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: finishing write tid 4 to smithi16234619-38 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: oid 38 updating version 0 to 251 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: oid 38 version 251 is already newer than 249 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: oid 38 version 251 is already newer than 250 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 251 (ObjNum 285 snap 59 seq_num 285) dirty exists 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:732: left oid 38 (ObjNum 285 snap 59 seq_num 285) 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:733: finishing write tid 1 to smithi16234619-43 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:733: finishing write tid 2 to smithi16234619-43 2024-09-16T15:04:03.341 INFO:tasks.rados.rados.0.smithi162.stdout:733: finishing write tid 3 to smithi16234619-43 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:733: finishing write tid 4 to smithi16234619-43 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:733: oid 43 updating version 0 to 68 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:733: oid 43 version 68 is already newer than 66 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:733: oid 43 version 68 is already newer than 67 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 68 (ObjNum 286 snap 59 seq_num 286) dirty exists 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:733: left oid 43 (ObjNum 286 snap 59 seq_num 286) 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:728: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:728: finishing copy_from to smithi16234619-1 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 174 (ObjNum 228 snap 45 seq_num 228) dirty exists 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:730: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:730: finishing copy_from to smithi16234619-22 2024-09-16T15:04:03.342 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 244 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:04:03.908 INFO:tasks.rados.rados.0.smithi162.stdout:728: done (5 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:730: done (4 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:732: done (3 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:733: done (2 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:734: done (1 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:735: done (0 left) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:736: read oid 33 snap -1 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:736: expect (ObjNum 274 snap 57 seq_num 274) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:737: read oid 9 snap 59 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:737: expect (ObjNum 253 snap 51 seq_num 253) 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:738: write oid 29 current snap is 60 2024-09-16T15:04:03.909 INFO:tasks.rados.rados.0.smithi162.stdout:738: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2024-09-16T15:04:03.914 INFO:tasks.rados.rados.0.smithi162.stdout:738: writing smithi16234619-29 from 423609 to 1125133 tid 1 2024-09-16T15:04:03.920 INFO:tasks.rados.rados.0.smithi162.stdout:738: writing smithi16234619-29 from 1628332 to 2100421 tid 2 2024-09-16T15:04:03.924 INFO:tasks.rados.rados.0.smithi162.stdout:738: writing smithi16234619-29 from 2648987 to 2823609 tid 3 2024-09-16T15:04:03.924 INFO:tasks.rados.rados.0.smithi162.stdout:739: read oid 3 snap -1 2024-09-16T15:04:03.924 INFO:tasks.rados.rados.0.smithi162.stdout:739: expect (ObjNum 259 snap 53 seq_num 259) 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:740: read oid 22 snap -1 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:740: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:741: read oid 35 snap -1 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:741: expect (ObjNum 262 snap 53 seq_num 262) 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:742: copy_from oid 31 from oid 48 current snap is 60 2024-09-16T15:04:03.925 INFO:tasks.rados.rados.0.smithi162.stdout:737: expect (ObjNum 253 snap 51 seq_num 253) 2024-09-16T15:04:04.068 INFO:tasks.rados.rados.0.smithi162.stdout:736: expect (ObjNum 274 snap 57 seq_num 274) 2024-09-16T15:04:04.248 INFO:tasks.rados.rados.0.smithi162.stdout:739: expect (ObjNum 259 snap 53 seq_num 259) 2024-09-16T15:04:04.324 INFO:tasks.rados.rados.0.smithi162.stdout:741: expect (ObjNum 262 snap 53 seq_num 262) 2024-09-16T15:04:04.464 INFO:tasks.rados.rados.0.smithi162.stdout:740: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: finishing write tid 1 to smithi16234619-29 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: finishing write tid 2 to smithi16234619-29 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: finishing write tid 3 to smithi16234619-29 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: finishing write tid 4 to smithi16234619-29 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: finishing write tid 5 to smithi16234619-29 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: oid 29 updating version 0 to 278 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: oid 29 updating version 278 to 279 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: oid 29 updating version 279 to 280 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: oid 29 version 280 is already newer than 277 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 280 (ObjNum 287 snap 60 seq_num 287) dirty exists 2024-09-16T15:04:04.660 INFO:tasks.rados.rados.0.smithi162.stdout:738: left oid 29 (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:742: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:742: finishing copy_from to smithi16234619-31 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 165 (ObjNum 211 snap 44 seq_num 211) dirty exists 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:736: done (6 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:737: done (5 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:738: done (4 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:739: done (3 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:740: done (2 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:741: done (1 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:742: done (0 left) 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:743: write oid 38 current snap is 60 2024-09-16T15:04:04.661 INFO:tasks.rados.rados.0.smithi162.stdout:743: seq_num 288 ranges {440416=776773,1686814=732100} 2024-09-16T15:04:04.665 INFO:tasks.rados.rados.0.smithi162.stdout:743: writing smithi16234619-38 from 440416 to 1217189 tid 1 2024-09-16T15:04:04.674 INFO:tasks.rados.rados.0.smithi162.stdout:743: writing smithi16234619-38 from 1686814 to 2418914 tid 2 2024-09-16T15:04:04.674 INFO:tasks.rados.rados.0.smithi162.stdout:744: read oid 26 snap -1 2024-09-16T15:04:04.674 INFO:tasks.rados.rados.0.smithi162.stdout:744: expect (ObjNum 276 snap 58 seq_num 276) 2024-09-16T15:04:04.674 INFO:tasks.rados.rados.0.smithi162.stdout:745: snap_create 2024-09-16T15:04:04.679 INFO:tasks.rados.rados.0.smithi162.stdout:744: expect (ObjNum 276 snap 58 seq_num 276) 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: finishing write tid 1 to smithi16234619-38 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: finishing write tid 2 to smithi16234619-38 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: finishing write tid 3 to smithi16234619-38 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: finishing write tid 4 to smithi16234619-38 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: oid 38 updating version 0 to 253 2024-09-16T15:04:04.772 INFO:tasks.rados.rados.0.smithi162.stdout:743: oid 38 version 253 is already newer than 252 2024-09-16T15:04:04.773 INFO:tasks.rados.rados.0.smithi162.stdout:743: oid 38 updating version 253 to 254 2024-09-16T15:04:04.773 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 254 (ObjNum 288 snap 60 seq_num 288) dirty exists 2024-09-16T15:04:04.773 INFO:tasks.rados.rados.0.smithi162.stdout:743: left oid 38 (ObjNum 288 snap 60 seq_num 288) 2024-09-16T15:04:05.593 INFO:tasks.rados.rados.0.smithi162.stdout:743: done (2 left) 2024-09-16T15:04:05.593 INFO:tasks.rados.rados.0.smithi162.stdout:744: done (1 left) 2024-09-16T15:04:05.593 INFO:tasks.rados.rados.0.smithi162.stdout:745: done (0 left) 2024-09-16T15:04:05.593 INFO:tasks.rados.rados.0.smithi162.stdout:746: copy_from oid 33 from oid 45 current snap is 61 2024-09-16T15:04:05.594 INFO:tasks.rados.rados.0.smithi162.stdout:747: copy_from oid 5 from oid 3 current snap is 61 2024-09-16T15:04:05.594 INFO:tasks.rados.rados.0.smithi162.stdout:748: write (excl) oid 28 current snap is 61 2024-09-16T15:04:05.594 INFO:tasks.rados.rados.0.smithi162.stdout:748: seq_num 289 ranges {457223=568375,1461649=475758,2420186=437037} 2024-09-16T15:04:05.599 INFO:tasks.rados.rados.0.smithi162.stdout:748: writing smithi16234619-28 from 457223 to 1025598 tid 1 2024-09-16T15:04:05.605 INFO:tasks.rados.rados.0.smithi162.stdout:748: writing smithi16234619-28 from 1461649 to 1937407 tid 2 2024-09-16T15:04:05.611 INFO:tasks.rados.rados.0.smithi162.stdout:748: writing smithi16234619-28 from 2420186 to 2857223 tid 3 2024-09-16T15:04:05.611 INFO:tasks.rados.rados.0.smithi162.stdout:749: delete oid 14 current snap is 61 2024-09-16T15:04:05.613 INFO:tasks.rados.rados.0.smithi162.stdout:749: done (3 left) 2024-09-16T15:04:05.613 INFO:tasks.rados.rados.0.smithi162.stdout:750: write (excl) oid 13 current snap is 61 2024-09-16T15:04:05.613 INFO:tasks.rados.rados.0.smithi162.stdout:750: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2024-09-16T15:04:05.621 INFO:tasks.rados.rados.0.smithi162.stdout:750: writing smithi16234619-13 from 490837 to 1209710 tid 1 2024-09-16T15:04:05.628 INFO:tasks.rados.rados.0.smithi162.stdout:750: writing smithi16234619-13 from 1862260 to 2341687 tid 2 2024-09-16T15:04:05.632 INFO:tasks.rados.rados.0.smithi162.stdout:750: writing smithi16234619-13 from 2875032 to 2890837 tid 3 2024-09-16T15:04:05.632 INFO:tasks.rados.rados.0.smithi162.stdout:751: snap_create 2024-09-16T15:04:05.632 INFO:tasks.rados.rados.0.smithi162.stdout:747: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:04:05.637 INFO:tasks.rados.rados.0.smithi162.stdout:748: finishing write tid 1 to smithi16234619-28 2024-09-16T15:04:05.642 INFO:tasks.rados.rados.0.smithi162.stdout:748: finishing write tid 2 to smithi16234619-28 2024-09-16T15:04:05.642 INFO:tasks.rados.rados.0.smithi162.stdout:748: finishing write tid 3 to smithi16234619-28 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: finishing write tid 4 to smithi16234619-28 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: finishing write tid 5 to smithi16234619-28 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: oid 28 updating version 0 to 246 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: oid 28 version 246 is already newer than 243 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: oid 28 version 246 is already newer than 244 2024-09-16T15:04:05.643 INFO:tasks.rados.rados.0.smithi162.stdout:748: oid 28 version 246 is already newer than 245 2024-09-16T15:04:05.644 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 246 (ObjNum 289 snap 61 seq_num 289) dirty exists 2024-09-16T15:04:05.644 INFO:tasks.rados.rados.0.smithi162.stdout:748: left oid 28 (ObjNum 289 snap 61 seq_num 289) 2024-09-16T15:04:05.654 INFO:tasks.rados.rados.0.smithi162.stdout:746: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:04:05.657 INFO:tasks.rados.rados.0.smithi162.stdout:746: finishing copy_from to smithi16234619-33 2024-09-16T15:04:05.657 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:04:05.659 INFO:tasks.rados.rados.0.smithi162.stdout:747: finishing copy_from to smithi16234619-5 2024-09-16T15:04:05.659 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 41 (ObjNum 259 snap 53 seq_num 259) dirty exists 2024-09-16T15:04:05.661 INFO:tasks.rados.rados.0.smithi162.stdout:750: finishing write tid 1 to smithi16234619-13 2024-09-16T15:04:05.661 INFO:tasks.rados.rados.0.smithi162.stdout:750: finishing write tid 2 to smithi16234619-13 2024-09-16T15:04:05.661 INFO:tasks.rados.rados.0.smithi162.stdout:750: finishing write tid 3 to smithi16234619-13 2024-09-16T15:04:05.661 INFO:tasks.rados.rados.0.smithi162.stdout:750: finishing write tid 4 to smithi16234619-13 2024-09-16T15:04:05.661 INFO:tasks.rados.rados.0.smithi162.stdout:750: finishing write tid 5 to smithi16234619-13 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:750: oid 13 updating version 0 to 285 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:750: oid 13 updating version 285 to 286 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:750: oid 13 version 286 is already newer than 284 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:750: oid 13 updating version 286 to 287 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 287 (ObjNum 291 snap 61 seq_num 291) dirty exists 2024-09-16T15:04:05.662 INFO:tasks.rados.rados.0.smithi162.stdout:750: left oid 13 (ObjNum 291 snap 61 seq_num 291) 2024-09-16T15:04:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:746: done (4 left) 2024-09-16T15:04:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:747: done (3 left) 2024-09-16T15:04:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:748: done (2 left) 2024-09-16T15:04:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:750: done (1 left) 2024-09-16T15:04:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:751: done (0 left) 2024-09-16T15:04:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:752: copy_from oid 40 from oid 1 current snap is 62 2024-09-16T15:04:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:753: copy_from oid 18 from oid 24 current snap is 62 2024-09-16T15:04:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:754: snap_create 2024-09-16T15:04:06.924 INFO:tasks.rados.rados.0.smithi162.stdout:752: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:04:06.925 INFO:tasks.rados.rados.0.smithi162.stdout:753: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:04:06.939 INFO:tasks.rados.rados.0.smithi162.stdout:752: finishing copy_from to smithi16234619-40 2024-09-16T15:04:06.939 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 174 (ObjNum 228 snap 45 seq_num 228) dirty exists 2024-09-16T15:04:06.941 INFO:tasks.rados.rados.0.smithi162.stdout:753: finishing copy_from to smithi16234619-18 2024-09-16T15:04:06.941 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 228 (ObjNum 278 snap 58 seq_num 278) dirty exists 2024-09-16T15:04:07.867 INFO:tasks.rados.rados.0.smithi162.stdout:752: done (2 left) 2024-09-16T15:04:07.867 INFO:tasks.rados.rados.0.smithi162.stdout:753: done (1 left) 2024-09-16T15:04:07.867 INFO:tasks.rados.rados.0.smithi162.stdout:754: done (0 left) 2024-09-16T15:04:07.867 INFO:tasks.rados.rados.0.smithi162.stdout:755: write oid 49 current snap is 63 2024-09-16T15:04:07.867 INFO:tasks.rados.rados.0.smithi162.stdout:755: seq_num 292 ranges {507644=794122,1920742=623085} 2024-09-16T15:04:07.874 INFO:tasks.rados.rados.0.smithi162.stdout:755: writing smithi16234619-49 from 507644 to 1301766 tid 1 2024-09-16T15:04:07.881 INFO:tasks.rados.rados.0.smithi162.stdout:755: writing smithi16234619-49 from 1920742 to 2543827 tid 2 2024-09-16T15:04:07.881 INFO:tasks.rados.rados.0.smithi162.stdout:756: write (excl) oid 9 current snap is 63 2024-09-16T15:04:07.881 INFO:tasks.rados.rados.0.smithi162.stdout:756: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2024-09-16T15:04:07.886 INFO:tasks.rados.rados.0.smithi162.stdout:756: writing smithi16234619-9 from 524451 to 993822 tid 1 2024-09-16T15:04:07.892 INFO:tasks.rados.rados.0.smithi162.stdout:756: writing smithi16234619-9 from 1579224 to 2062320 tid 2 2024-09-16T15:04:07.897 INFO:tasks.rados.rados.0.smithi162.stdout:756: writing smithi16234619-9 from 2529878 to 2924451 tid 3 2024-09-16T15:04:07.897 INFO:tasks.rados.rados.0.smithi162.stdout:757: write (excl) oid 43 current snap is 63 2024-09-16T15:04:07.897 INFO:tasks.rados.rados.0.smithi162.stdout:757: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2024-09-16T15:04:07.903 INFO:tasks.rados.rados.0.smithi162.stdout:757: writing smithi16234619-43 from 541258 to 1085878 tid 1 2024-09-16T15:04:07.911 INFO:tasks.rados.rados.0.smithi162.stdout:757: writing smithi16234619-43 from 1637706 to 2264460 tid 2 2024-09-16T15:04:07.915 INFO:tasks.rados.rados.0.smithi162.stdout:757: writing smithi16234619-43 from 2840948 to 2941258 tid 3 2024-09-16T15:04:07.915 INFO:tasks.rados.rados.0.smithi162.stdout:758: delete oid 2 current snap is 63 2024-09-16T15:04:07.916 INFO:tasks.rados.rados.0.smithi162.stdout:755: finishing write tid 1 to smithi16234619-49 2024-09-16T15:04:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:758: done (3 left) 2024-09-16T15:04:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:759: write (excl) oid 23 current snap is 63 2024-09-16T15:04:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:759: seq_num 296 ranges {574872=695118,2038317=630423} 2024-09-16T15:04:07.925 INFO:tasks.rados.rados.0.smithi162.stdout:759: writing smithi16234619-23 from 574872 to 1269990 tid 1 2024-09-16T15:04:07.934 INFO:tasks.rados.rados.0.smithi162.stdout:759: writing smithi16234619-23 from 2038317 to 2668740 tid 2 2024-09-16T15:04:07.934 INFO:tasks.rados.rados.0.smithi162.stdout:760: write oid 50 current snap is 63 2024-09-16T15:04:07.934 INFO:tasks.rados.rados.0.smithi162.stdout:755: finishing write tid 2 to smithi16234619-49 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: finishing write tid 3 to smithi16234619-49 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: finishing write tid 4 to smithi16234619-49 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: oid 49 updating version 0 to 246 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: oid 49 updating version 246 to 247 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: oid 49 version 247 is already newer than 245 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 247 (ObjNum 292 snap 63 seq_num 292) dirty exists 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:755: left oid 49 (ObjNum 292 snap 63 seq_num 292) 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: finishing write tid 1 to smithi16234619-9 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: finishing write tid 2 to smithi16234619-9 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: finishing write tid 3 to smithi16234619-9 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: finishing write tid 4 to smithi16234619-9 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: finishing write tid 5 to smithi16234619-9 2024-09-16T15:04:07.935 INFO:tasks.rados.rados.0.smithi162.stdout:756: oid 9 updating version 0 to 235 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:756: oid 9 version 235 is already newer than 232 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:756: oid 9 version 235 is already newer than 233 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:756: oid 9 version 235 is already newer than 234 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 235 (ObjNum 293 snap 63 seq_num 293) dirty exists 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:756: left oid 9 (ObjNum 293 snap 63 seq_num 293) 2024-09-16T15:04:07.936 INFO:tasks.rados.rados.0.smithi162.stdout:760: seq_num 297 ranges {591679=486720,1813152=490434} 2024-09-16T15:04:07.941 INFO:tasks.rados.rados.0.smithi162.stdout:760: writing smithi16234619-50 from 591679 to 1078399 tid 1 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:760: writing smithi16234619-50 from 1813152 to 2303586 tid 2 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:755: done (4 left) 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:756: done (3 left) 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:761: write oid 31 current snap is 63 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:757: finishing write tid 1 to smithi16234619-43 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:757: finishing write tid 2 to smithi16234619-43 2024-09-16T15:04:07.948 INFO:tasks.rados.rados.0.smithi162.stdout:761: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2024-09-16T15:04:07.954 INFO:tasks.rados.rados.0.smithi162.stdout:761: writing smithi16234619-31 from 608486 to 1170455 tid 1 2024-09-16T15:04:07.961 INFO:tasks.rados.rados.0.smithi162.stdout:761: writing smithi16234619-31 from 1871634 to 2505726 tid 2 2024-09-16T15:04:07.964 INFO:tasks.rados.rados.0.smithi162.stdout:761: writing smithi16234619-31 from 2950640 to 3008486 tid 3 2024-09-16T15:04:07.964 INFO:tasks.rados.rados.0.smithi162.stdout:762: write (excl) oid 49 current snap is 63 2024-09-16T15:04:07.964 INFO:tasks.rados.rados.0.smithi162.stdout:762: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2024-09-16T15:04:07.970 INFO:tasks.rados.rados.0.smithi162.stdout:762: writing smithi16234619-49 from 625293 to 1262511 tid 1 2024-09-16T15:04:07.976 INFO:tasks.rados.rados.0.smithi162.stdout:762: writing smithi16234619-49 from 1813763 to 2307866 tid 2 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:762: writing smithi16234619-49 from 2978063 to 3025293 tid 3 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:763: delete oid 32 current snap is 63 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: finishing write tid 3 to smithi16234619-43 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: finishing write tid 4 to smithi16234619-43 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: finishing write tid 5 to smithi16234619-43 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: oid 43 updating version 0 to 73 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: oid 43 version 73 is already newer than 71 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: oid 43 version 73 is already newer than 70 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:757: oid 43 version 73 is already newer than 72 2024-09-16T15:04:07.981 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 73 (ObjNum 294 snap 63 seq_num 294) dirty exists 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:757: left oid 43 (ObjNum 294 snap 63 seq_num 294) 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: finishing write tid 1 to smithi16234619-23 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: finishing write tid 2 to smithi16234619-23 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: finishing write tid 3 to smithi16234619-23 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: finishing write tid 4 to smithi16234619-23 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: oid 23 updating version 0 to 66 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: oid 23 updating version 66 to 67 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: oid 23 updating version 67 to 68 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 68 (ObjNum 296 snap 63 seq_num 296) dirty exists 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:759: left oid 23 (ObjNum 296 snap 63 seq_num 296) 2024-09-16T15:04:07.982 INFO:tasks.rados.rados.0.smithi162.stdout:760: finishing write tid 1 to smithi16234619-50 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: finishing write tid 2 to smithi16234619-50 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: finishing write tid 3 to smithi16234619-50 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: finishing write tid 4 to smithi16234619-50 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: oid 50 updating version 0 to 258 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: oid 50 updating version 258 to 259 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: oid 50 version 259 is already newer than 257 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 259 (ObjNum 297 snap 63 seq_num 297) dirty exists 2024-09-16T15:04:07.983 INFO:tasks.rados.rados.0.smithi162.stdout:760: left oid 50 (ObjNum 297 snap 63 seq_num 297) 2024-09-16T15:04:07.990 INFO:tasks.rados.rados.0.smithi162.stdout:761: finishing write tid 1 to smithi16234619-31 2024-09-16T15:04:07.997 INFO:tasks.rados.rados.0.smithi162.stdout:757: done (5 left) 2024-09-16T15:04:07.997 INFO:tasks.rados.rados.0.smithi162.stdout:759: done (4 left) 2024-09-16T15:04:07.997 INFO:tasks.rados.rados.0.smithi162.stdout:760: done (3 left) 2024-09-16T15:04:07.997 INFO:tasks.rados.rados.0.smithi162.stdout:763: done (2 left) 2024-09-16T15:04:07.998 INFO:tasks.rados.rados.0.smithi162.stdout:764: snap_remove snap 61 2024-09-16T15:04:08.880 INFO:tasks.rados.rados.0.smithi162.stdout:764: done (2 left) 2024-09-16T15:04:08.880 INFO:tasks.rados.rados.0.smithi162.stdout:765: read oid 17 snap 58 2024-09-16T15:04:08.880 INFO:tasks.rados.rados.0.smithi162.stdout:765: expect (ObjNum 230 snap 45 seq_num 230) 2024-09-16T15:04:08.880 INFO:tasks.rados.rados.0.smithi162.stdout:766: snap_remove snap 56 2024-09-16T15:04:09.883 INFO:tasks.rados.rados.0.smithi162.stdout:766: done (3 left) 2024-09-16T15:04:09.883 INFO:tasks.rados.rados.0.smithi162.stdout:767: read oid 39 snap -1 2024-09-16T15:04:09.883 INFO:tasks.rados.rados.0.smithi162.stdout:767: expect (ObjNum 205 snap 44 seq_num 205) 2024-09-16T15:04:09.883 INFO:tasks.rados.rados.0.smithi162.stdout:768: write oid 41 current snap is 63 2024-09-16T15:04:09.883 INFO:tasks.rados.rados.0.smithi162.stdout:768: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2024-09-16T15:04:09.891 INFO:tasks.rados.rados.0.smithi162.stdout:768: writing smithi16234619-41 from 658907 to 1446623 tid 1 2024-09-16T15:04:09.897 INFO:tasks.rados.rados.0.smithi162.stdout:768: writing smithi16234619-41 from 1930727 to 2428499 tid 2 2024-09-16T15:04:09.901 INFO:tasks.rados.rados.0.smithi162.stdout:768: writing smithi16234619-41 from 3032909 to 3058907 tid 3 2024-09-16T15:04:09.901 INFO:tasks.rados.rados.0.smithi162.stdout:769: snap_create 2024-09-16T15:04:09.901 INFO:tasks.rados.rados.0.smithi162.stdout:761: finishing write tid 2 to smithi16234619-31 2024-09-16T15:04:09.901 INFO:tasks.rados.rados.0.smithi162.stdout:761: finishing write tid 3 to smithi16234619-31 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: finishing write tid 4 to smithi16234619-31 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: finishing write tid 5 to smithi16234619-31 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: oid 31 updating version 0 to 230 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: oid 31 updating version 230 to 232 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: oid 31 version 232 is already newer than 231 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: oid 31 version 232 is already newer than 229 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 232 (ObjNum 298 snap 63 seq_num 298) dirty exists 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:761: left oid 31 (ObjNum 298 snap 63 seq_num 298) 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:762: finishing write tid 1 to smithi16234619-49 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:762: finishing write tid 2 to smithi16234619-49 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:762: finishing write tid 3 to smithi16234619-49 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:762: finishing write tid 4 to smithi16234619-49 2024-09-16T15:04:09.902 INFO:tasks.rados.rados.0.smithi162.stdout:762: finishing write tid 5 to smithi16234619-49 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:762: oid 49 updating version 0 to 249 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:762: oid 49 version 249 is already newer than 248 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:762: oid 49 updating version 249 to 250 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:762: oid 49 updating version 250 to 251 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 251 (ObjNum 299 snap 63 seq_num 299) dirty exists 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:762: left oid 49 (ObjNum 299 snap 63 seq_num 299) 2024-09-16T15:04:09.903 INFO:tasks.rados.rados.0.smithi162.stdout:765: expect (ObjNum 230 snap 45 seq_num 230) 2024-09-16T15:04:10.072 INFO:tasks.rados.rados.0.smithi162.stdout:767: expect (ObjNum 205 snap 44 seq_num 205) 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: finishing write tid 1 to smithi16234619-41 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: finishing write tid 2 to smithi16234619-41 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: finishing write tid 3 to smithi16234619-41 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: finishing write tid 4 to smithi16234619-41 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: finishing write tid 5 to smithi16234619-41 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: oid 41 updating version 0 to 253 2024-09-16T15:04:10.240 INFO:tasks.rados.rados.0.smithi162.stdout:768: oid 41 updating version 253 to 255 2024-09-16T15:04:10.241 INFO:tasks.rados.rados.0.smithi162.stdout:768: oid 41 version 255 is already newer than 252 2024-09-16T15:04:10.241 INFO:tasks.rados.rados.0.smithi162.stdout:768: oid 41 version 255 is already newer than 254 2024-09-16T15:04:10.241 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 255 (ObjNum 301 snap 63 seq_num 301) dirty exists 2024-09-16T15:04:10.241 INFO:tasks.rados.rados.0.smithi162.stdout:768: left oid 41 (ObjNum 301 snap 63 seq_num 301) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:761: done (5 left) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:762: done (4 left) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:765: done (3 left) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:767: done (2 left) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:768: done (1 left) 2024-09-16T15:04:10.812 INFO:tasks.rados.rados.0.smithi162.stdout:769: done (0 left) 2024-09-16T15:04:10.813 INFO:tasks.rados.rados.0.smithi162.stdout:770: delete oid 15 current snap is 64 2024-09-16T15:04:10.822 INFO:tasks.rados.rados.0.smithi162.stdout:770: done (0 left) 2024-09-16T15:04:10.822 INFO:tasks.rados.rados.0.smithi162.stdout:771: write oid 27 current snap is 64 2024-09-16T15:04:10.823 INFO:tasks.rados.rados.0.smithi162.stdout:771: seq_num 303 ranges {692521=538214,1931338=785088} 2024-09-16T15:04:10.829 INFO:tasks.rados.rados.0.smithi162.stdout:771: writing smithi16234619-27 from 692521 to 1230735 tid 1 2024-09-16T15:04:10.837 INFO:tasks.rados.rados.0.smithi162.stdout:771: writing smithi16234619-27 from 1931338 to 2716426 tid 2 2024-09-16T15:04:10.837 INFO:tasks.rados.rados.0.smithi162.stdout:772: read oid 12 snap -1 2024-09-16T15:04:10.837 INFO:tasks.rados.rados.0.smithi162.stdout:772: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:04:10.837 INFO:tasks.rados.rados.0.smithi162.stdout:773: snap_remove snap 63 2024-09-16T15:04:11.819 INFO:tasks.rados.rados.0.smithi162.stdout:773: done (2 left) 2024-09-16T15:04:11.819 INFO:tasks.rados.rados.0.smithi162.stdout:774: write (excl) oid 3 current snap is 64 2024-09-16T15:04:11.819 INFO:tasks.rados.rados.0.smithi162.stdout:774: seq_num 304 ranges {709328=613463,1989820=645099} 2024-09-16T15:04:11.826 INFO:tasks.rados.rados.0.smithi162.stdout:774: writing smithi16234619-3 from 709328 to 1322791 tid 1 2024-09-16T15:04:11.834 INFO:tasks.rados.rados.0.smithi162.stdout:774: writing smithi16234619-3 from 1989820 to 2634919 tid 2 2024-09-16T15:04:11.834 INFO:tasks.rados.rados.0.smithi162.stdout:775: rollback oid 39 current snap is 64 2024-09-16T15:04:11.834 INFO:tasks.rados.rados.0.smithi162.stdout:772: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: finishing write tid 1 to smithi16234619-27 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: finishing write tid 2 to smithi16234619-27 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: finishing write tid 3 to smithi16234619-27 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: finishing write tid 4 to smithi16234619-27 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: oid 27 updating version 0 to 237 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: oid 27 version 237 is already newer than 236 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: oid 27 updating version 237 to 238 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 238 (ObjNum 303 snap 64 seq_num 303) dirty exists 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:771: left oid 27 (ObjNum 303 snap 64 seq_num 303) 2024-09-16T15:04:12.004 INFO:tasks.rados.rados.0.smithi162.stdout:774: finishing write tid 1 to smithi16234619-3 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: finishing write tid 2 to smithi16234619-3 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: finishing write tid 3 to smithi16234619-3 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 58 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: finishing write tid 4 to smithi16234619-3 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: oid 3 updating version 0 to 43 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: oid 3 updating version 43 to 44 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: oid 3 updating version 44 to 45 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 45 (ObjNum 304 snap 64 seq_num 304) dirty exists 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: left oid 3 (ObjNum 304 snap 64 seq_num 304) 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:771: done (3 left) 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:772: done (2 left) 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:774: done (1 left) 2024-09-16T15:04:12.005 INFO:tasks.rados.rados.0.smithi162.stdout:776: delete oid 3 current snap is 64 2024-09-16T15:04:12.006 INFO:tasks.rados.rados.0.smithi162.stdout:776: done (1 left) 2024-09-16T15:04:12.006 INFO:tasks.rados.rados.0.smithi162.stdout:777: snap_remove snap 60 2024-09-16T15:04:13.148 INFO:tasks.rados.rados.0.smithi162.stdout:777: done (1 left) 2024-09-16T15:04:13.148 INFO:tasks.rados.rados.0.smithi162.stdout:778: write (excl) oid 43 current snap is 64 2024-09-16T15:04:13.148 INFO:tasks.rados.rados.0.smithi162.stdout:775: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:04:13.148 INFO:tasks.rados.rados.0.smithi162.stdout:778: seq_num 306 ranges {742942=480314,1823137=648768} 2024-09-16T15:04:13.154 INFO:tasks.rados.rados.0.smithi162.stdout:778: writing smithi16234619-43 from 742942 to 1223256 tid 1 2024-09-16T15:04:13.162 INFO:tasks.rados.rados.0.smithi162.stdout:778: writing smithi16234619-43 from 1823137 to 2471905 tid 2 2024-09-16T15:04:13.162 INFO:tasks.rados.rados.0.smithi162.stdout:779: rollback oid 21 current snap is 64 2024-09-16T15:04:13.162 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 59 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:775: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:775: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 299 (ObjNum 205 snap 44 seq_num 205) dirty exists 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:775: done (2 left) 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:780: copy_from oid 6 from oid 31 current snap is 64 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:781: read oid 29 snap -1 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:781: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:782: read oid 18 snap -1 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:782: expect (ObjNum 278 snap 58 seq_num 278) 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:783: read oid 2 snap -1 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:783: expect deleted 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:784: read oid 23 snap -1 2024-09-16T15:04:13.163 INFO:tasks.rados.rados.0.smithi162.stdout:784: expect (ObjNum 296 snap 63 seq_num 296) 2024-09-16T15:04:13.164 INFO:tasks.rados.rados.0.smithi162.stdout:785: write oid 44 current snap is 64 2024-09-16T15:04:13.164 INFO:tasks.rados.rados.0.smithi162.stdout:785: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2024-09-16T15:04:13.170 INFO:tasks.rados.rados.0.smithi162.stdout:785: writing smithi16234619-44 from 759749 to 1315312 tid 1 2024-09-16T15:04:13.179 INFO:tasks.rados.rados.0.smithi162.stdout:785: writing smithi16234619-44 from 1765266 to 2557692 tid 2 2024-09-16T15:04:13.184 INFO:tasks.rados.rados.0.smithi162.stdout:785: writing smithi16234619-44 from 2964741 to 3159749 tid 3 2024-09-16T15:04:13.184 INFO:tasks.rados.rados.0.smithi162.stdout:786: write oid 38 current snap is 64 2024-09-16T15:04:13.184 INFO:tasks.rados.rados.0.smithi162.stdout:779: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:04:13.184 INFO:tasks.rados.rados.0.smithi162.stdout:779: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:04:13.184 INFO:tasks.rados.rados.0.smithi162.stdout:781: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:04:13.372 INFO:tasks.rados.rados.0.smithi162.stdout:779: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:04:13.372 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 245 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:04:13.372 INFO:tasks.rados.rados.0.smithi162.stdout:782: expect (ObjNum 278 snap 58 seq_num 278) 2024-09-16T15:04:13.555 INFO:tasks.rados.rados.0.smithi162.stdout:784: expect (ObjNum 296 snap 63 seq_num 296) 2024-09-16T15:04:13.687 INFO:tasks.rados.rados.0.smithi162.stdout:778: finishing write tid 1 to smithi16234619-43 2024-09-16T15:04:13.687 INFO:tasks.rados.rados.0.smithi162.stdout:778: finishing write tid 2 to smithi16234619-43 2024-09-16T15:04:13.687 INFO:tasks.rados.rados.0.smithi162.stdout:786: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2024-09-16T15:04:13.694 INFO:tasks.rados.rados.0.smithi162.stdout:786: writing smithi16234619-38 from 776556 to 1407368 tid 1 2024-09-16T15:04:13.700 INFO:tasks.rados.rados.0.smithi162.stdout:786: writing smithi16234619-38 from 1823748 to 2476185 tid 2 2024-09-16T15:04:13.705 INFO:tasks.rados.rados.0.smithi162.stdout:786: writing smithi16234619-38 from 3108517 to 3176556 tid 3 2024-09-16T15:04:13.705 INFO:tasks.rados.rados.0.smithi162.stdout:779: done (8 left) 2024-09-16T15:04:13.705 INFO:tasks.rados.rados.0.smithi162.stdout:781: done (7 left) 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:782: done (6 left) 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:783: done (5 left) 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:784: done (4 left) 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:787: write (excl) oid 15 current snap is 64 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: finishing write tid 3 to smithi16234619-43 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: finishing write tid 4 to smithi16234619-43 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: oid 43 updating version 0 to 81 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: oid 43 version 81 is already newer than 80 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: oid 43 updating version 81 to 82 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 82 (ObjNum 306 snap 64 seq_num 306) dirty exists 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:778: left oid 43 (ObjNum 306 snap 64 seq_num 306) 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:780: finishing copy_from racing read to smithi16234619-6 2024-09-16T15:04:13.706 INFO:tasks.rados.rados.0.smithi162.stdout:785: finishing write tid 1 to smithi16234619-44 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: finishing write tid 2 to smithi16234619-44 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:780: finishing copy_from to smithi16234619-6 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 232 (ObjNum 298 snap 63 seq_num 298) dirty exists 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: finishing write tid 3 to smithi16234619-44 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: finishing write tid 4 to smithi16234619-44 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: finishing write tid 5 to smithi16234619-44 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: oid 44 updating version 0 to 257 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: oid 44 version 257 is already newer than 256 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: oid 44 version 257 is already newer than 255 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: oid 44 updating version 257 to 258 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 258 (ObjNum 307 snap 64 seq_num 307) dirty exists 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:785: left oid 44 (ObjNum 307 snap 64 seq_num 307) 2024-09-16T15:04:13.707 INFO:tasks.rados.rados.0.smithi162.stdout:787: seq_num 309 ranges {793363=706061,2282230=796095} 2024-09-16T15:04:13.713 INFO:tasks.rados.rados.0.smithi162.stdout:787: writing smithi16234619-15 from 793363 to 1499424 tid 1 2024-09-16T15:04:13.722 INFO:tasks.rados.rados.0.smithi162.stdout:787: writing smithi16234619-15 from 2282230 to 3078325 tid 2 2024-09-16T15:04:13.722 INFO:tasks.rados.rados.0.smithi162.stdout:778: done (4 left) 2024-09-16T15:04:13.722 INFO:tasks.rados.rados.0.smithi162.stdout:780: done (3 left) 2024-09-16T15:04:13.722 INFO:tasks.rados.rados.0.smithi162.stdout:785: done (2 left) 2024-09-16T15:04:13.723 INFO:tasks.rados.rados.0.smithi162.stdout:788: delete oid 5 current snap is 64 2024-09-16T15:04:13.725 INFO:tasks.rados.rados.0.smithi162.stdout:788: done (2 left) 2024-09-16T15:04:13.725 INFO:tasks.rados.rados.0.smithi162.stdout:789: read oid 50 snap -1 2024-09-16T15:04:13.725 INFO:tasks.rados.rados.0.smithi162.stdout:789: expect (ObjNum 297 snap 63 seq_num 297) 2024-09-16T15:04:13.725 INFO:tasks.rados.rados.0.smithi162.stdout:790: copy_from oid 27 from oid 30 current snap is 64 2024-09-16T15:04:13.725 INFO:tasks.rados.rados.0.smithi162.stdout:791: snap_remove snap 59 2024-09-16T15:04:13.829 INFO:tasks.rados.rados.0.smithi162.stdout:791: done (4 left) 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:792: snap_create 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: finishing write tid 1 to smithi16234619-38 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: finishing write tid 2 to smithi16234619-38 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: finishing write tid 3 to smithi16234619-38 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: finishing write tid 4 to smithi16234619-38 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: finishing write tid 5 to smithi16234619-38 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: oid 38 updating version 0 to 262 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: oid 38 version 262 is already newer than 259 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: oid 38 version 262 is already newer than 261 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: oid 38 version 262 is already newer than 260 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 262 (ObjNum 308 snap 64 seq_num 308) dirty exists 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:786: left oid 38 (ObjNum 308 snap 64 seq_num 308) 2024-09-16T15:04:13.830 INFO:tasks.rados.rados.0.smithi162.stdout:789: expect (ObjNum 297 snap 63 seq_num 297) 2024-09-16T15:04:13.981 INFO:tasks.rados.rados.0.smithi162.stdout:787: finishing write tid 1 to smithi16234619-15 2024-09-16T15:04:13.981 INFO:tasks.rados.rados.0.smithi162.stdout:787: finishing write tid 2 to smithi16234619-15 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: finishing write tid 3 to smithi16234619-15 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: finishing write tid 4 to smithi16234619-15 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: oid 15 updating version 0 to 250 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: oid 15 version 250 is already newer than 248 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: oid 15 version 250 is already newer than 249 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 250 (ObjNum 309 snap 64 seq_num 309) dirty exists 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:787: left oid 15 (ObjNum 309 snap 64 seq_num 309) 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:790: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:790: finishing copy_from to smithi16234619-27 2024-09-16T15:04:13.982 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 242 (ObjNum 275 snap 57 seq_num 275) dirty exists 2024-09-16T15:04:14.837 INFO:tasks.rados.rados.0.smithi162.stdout:786: done (4 left) 2024-09-16T15:04:14.837 INFO:tasks.rados.rados.0.smithi162.stdout:787: done (3 left) 2024-09-16T15:04:14.837 INFO:tasks.rados.rados.0.smithi162.stdout:789: done (2 left) 2024-09-16T15:04:14.838 INFO:tasks.rados.rados.0.smithi162.stdout:790: done (1 left) 2024-09-16T15:04:14.838 INFO:tasks.rados.rados.0.smithi162.stdout:792: done (0 left) 2024-09-16T15:04:14.838 INFO:tasks.rados.rados.0.smithi162.stdout:793: snap_create 2024-09-16T15:04:15.519 INFO:tasks.rados.rados.0.smithi162.stdout:793: done (0 left) 2024-09-16T15:04:15.519 INFO:tasks.rados.rados.0.smithi162.stdout:794: rollback oid 20 current snap is 66 2024-09-16T15:04:15.519 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 64 2024-09-16T15:04:15.519 INFO:tasks.rados.rados.0.smithi162.stdout:795: write oid 27 current snap is 66 2024-09-16T15:04:15.519 INFO:tasks.rados.rados.0.smithi162.stdout:795: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2024-09-16T15:04:15.524 INFO:tasks.rados.rados.0.smithi162.stdout:795: writing smithi16234619-27 from 426977 to 883536 tid 1 2024-09-16T15:04:15.531 INFO:tasks.rados.rados.0.smithi162.stdout:795: writing smithi16234619-27 from 1482841 to 2282605 tid 2 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:795: writing smithi16234619-27 from 2958080 to 3226977 tid 3 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:796: rollback oid 41 current snap is 66 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 41 to 64 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:794: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:794: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:794: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 164 (ObjNum 254 snap 51 seq_num 254) dirty exists 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:794: done (2 left) 2024-09-16T15:04:15.537 INFO:tasks.rados.rados.0.smithi162.stdout:797: delete oid 28 current snap is 66 2024-09-16T15:04:15.541 INFO:tasks.rados.rados.0.smithi162.stdout:797: done (2 left) 2024-09-16T15:04:15.541 INFO:tasks.rados.rados.0.smithi162.stdout:798: snap_create 2024-09-16T15:04:15.544 INFO:tasks.rados.rados.0.smithi162.stdout:796: finishing rollback tid 0 to smithi16234619-41 2024-09-16T15:04:15.544 INFO:tasks.rados.rados.0.smithi162.stdout:796: finishing rollback tid 1 to smithi16234619-41 2024-09-16T15:04:15.545 INFO:tasks.rados.rados.0.smithi162.stdout:796: finishing rollback tid 2 to smithi16234619-41 2024-09-16T15:04:15.545 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 258 (ObjNum 301 snap 63 seq_num 301) dirty exists 2024-09-16T15:04:15.558 INFO:tasks.rados.rados.0.smithi162.stdout:795: finishing write tid 1 to smithi16234619-27 2024-09-16T15:04:15.562 INFO:tasks.rados.rados.0.smithi162.stdout:795: finishing write tid 2 to smithi16234619-27 2024-09-16T15:04:15.562 INFO:tasks.rados.rados.0.smithi162.stdout:795: finishing write tid 3 to smithi16234619-27 2024-09-16T15:04:15.562 INFO:tasks.rados.rados.0.smithi162.stdout:795: finishing write tid 4 to smithi16234619-27 2024-09-16T15:04:15.562 INFO:tasks.rados.rados.0.smithi162.stdout:795: finishing write tid 5 to smithi16234619-27 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:795: oid 27 updating version 0 to 243 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:795: oid 27 updating version 243 to 246 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:795: oid 27 version 246 is already newer than 245 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:795: oid 27 version 246 is already newer than 244 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 246 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:04:15.563 INFO:tasks.rados.rados.0.smithi162.stdout:795: left oid 27 (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:04:16.383 INFO:tasks.rados.rados.0.smithi162.stdout:795: done (2 left) 2024-09-16T15:04:16.383 INFO:tasks.rados.rados.0.smithi162.stdout:796: done (1 left) 2024-09-16T15:04:16.383 INFO:tasks.rados.rados.0.smithi162.stdout:798: done (0 left) 2024-09-16T15:04:16.383 INFO:tasks.rados.rados.0.smithi162.stdout:799: write (excl) oid 19 current snap is 67 2024-09-16T15:04:16.383 INFO:tasks.rados.rados.0.smithi162.stdout:799: seq_num 313 ranges {460591=723410,1716158=403433,2729279=440140} 2024-09-16T15:04:16.389 INFO:tasks.rados.rados.0.smithi162.stdout:799: writing smithi16234619-19 from 460591 to 1184001 tid 1 2024-09-16T15:04:16.394 INFO:tasks.rados.rados.0.smithi162.stdout:799: writing smithi16234619-19 from 1716158 to 2119591 tid 2 2024-09-16T15:04:16.400 INFO:tasks.rados.rados.0.smithi162.stdout:799: writing smithi16234619-19 from 2729279 to 3169419 tid 3 2024-09-16T15:04:16.400 INFO:tasks.rados.rados.0.smithi162.stdout:800: write (excl) oid 13 current snap is 67 2024-09-16T15:04:16.400 INFO:tasks.rados.rados.0.smithi162.stdout:800: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2024-09-16T15:04:16.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: writing smithi16234619-13 from 477398 to 1276057 tid 1 2024-09-16T15:04:16.413 INFO:tasks.rados.rados.0.smithi162.stdout:800: writing smithi16234619-13 from 1774640 to 2438084 tid 2 2024-09-16T15:04:16.417 INFO:tasks.rados.rados.0.smithi162.stdout:800: writing smithi16234619-13 from 2873055 to 3277398 tid 3 2024-09-16T15:04:16.417 INFO:tasks.rados.rados.0.smithi162.stdout:801: snap_remove snap 58 2024-09-16T15:04:17.389 INFO:tasks.rados.rados.0.smithi162.stdout:801: done (2 left) 2024-09-16T15:04:17.389 INFO:tasks.rados.rados.0.smithi162.stdout:802: write (excl) oid 42 current snap is 67 2024-09-16T15:04:17.389 INFO:tasks.rados.rados.0.smithi162.stdout:802: seq_num 315 ranges {494205=473908,1433122=407102,2384125=462684} 2024-09-16T15:04:17.394 INFO:tasks.rados.rados.0.smithi162.stdout:802: writing smithi16234619-42 from 494205 to 968113 tid 1 2024-09-16T15:04:17.399 INFO:tasks.rados.rados.0.smithi162.stdout:802: writing smithi16234619-42 from 1433122 to 1840224 tid 2 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:802: writing smithi16234619-42 from 2384125 to 2846809 tid 3 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:803: snap_create 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: finishing write tid 1 to smithi16234619-19 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: finishing write tid 2 to smithi16234619-19 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: finishing write tid 3 to smithi16234619-19 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: finishing write tid 4 to smithi16234619-19 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: finishing write tid 5 to smithi16234619-19 2024-09-16T15:04:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:799: oid 19 updating version 0 to 273 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:799: oid 19 updating version 273 to 275 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:799: oid 19 version 275 is already newer than 274 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:799: oid 19 updating version 275 to 276 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 276 (ObjNum 313 snap 67 seq_num 313) dirty exists 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:799: left oid 19 (ObjNum 313 snap 67 seq_num 313) 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: finishing write tid 1 to smithi16234619-13 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: finishing write tid 2 to smithi16234619-13 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: finishing write tid 3 to smithi16234619-13 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: finishing write tid 4 to smithi16234619-13 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: finishing write tid 5 to smithi16234619-13 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: oid 13 updating version 0 to 318 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: oid 13 version 318 is already newer than 317 2024-09-16T15:04:17.406 INFO:tasks.rados.rados.0.smithi162.stdout:800: oid 13 updating version 318 to 320 2024-09-16T15:04:17.407 INFO:tasks.rados.rados.0.smithi162.stdout:800: oid 13 version 320 is already newer than 319 2024-09-16T15:04:17.407 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 320 (ObjNum 314 snap 67 seq_num 314) dirty exists 2024-09-16T15:04:17.407 INFO:tasks.rados.rados.0.smithi162.stdout:800: left oid 13 (ObjNum 314 snap 67 seq_num 314) 2024-09-16T15:04:17.423 INFO:tasks.rados.rados.0.smithi162.stdout:802: finishing write tid 1 to smithi16234619-42 2024-09-16T15:04:17.433 INFO:tasks.rados.rados.0.smithi162.stdout:802: finishing write tid 2 to smithi16234619-42 2024-09-16T15:04:17.436 INFO:tasks.rados.rados.0.smithi162.stdout:802: finishing write tid 3 to smithi16234619-42 2024-09-16T15:04:17.436 INFO:tasks.rados.rados.0.smithi162.stdout:802: finishing write tid 4 to smithi16234619-42 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: finishing write tid 5 to smithi16234619-42 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: oid 42 updating version 0 to 328 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: oid 42 version 328 is already newer than 326 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: oid 42 version 328 is already newer than 327 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: oid 42 version 328 is already newer than 321 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 328 (ObjNum 315 snap 67 seq_num 315) dirty exists 2024-09-16T15:04:17.437 INFO:tasks.rados.rados.0.smithi162.stdout:802: left oid 42 (ObjNum 315 snap 67 seq_num 315) 2024-09-16T15:04:18.806 INFO:tasks.rados.rados.0.smithi162.stdout:799: done (3 left) 2024-09-16T15:04:18.806 INFO:tasks.rados.rados.0.smithi162.stdout:800: done (2 left) 2024-09-16T15:04:18.806 INFO:tasks.rados.rados.0.smithi162.stdout:802: done (1 left) 2024-09-16T15:04:18.806 INFO:tasks.rados.rados.0.smithi162.stdout:803: done (0 left) 2024-09-16T15:04:18.806 INFO:tasks.rados.rados.0.smithi162.stdout:804: snap_create 2024-09-16T15:04:19.399 INFO:tasks.rados.rados.0.smithi162.stdout:804: done (0 left) 2024-09-16T15:04:19.399 INFO:tasks.rados.rados.0.smithi162.stdout:805: snap_create 2024-09-16T15:04:20.382 INFO:tasks.rados.rados.0.smithi162.stdout:805: done (0 left) 2024-09-16T15:04:20.382 INFO:tasks.rados.rados.0.smithi162.stdout:806: read oid 48 snap -1 2024-09-16T15:04:20.382 INFO:tasks.rados.rados.0.smithi162.stdout:806: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:04:20.382 INFO:tasks.rados.rados.0.smithi162.stdout:807: snap_remove snap 68 2024-09-16T15:04:21.390 INFO:tasks.rados.rados.0.smithi162.stdout:807: done (1 left) 2024-09-16T15:04:21.391 INFO:tasks.rados.rados.0.smithi162.stdout:808: write (excl) oid 35 current snap is 70 2024-09-16T15:04:21.391 INFO:tasks.rados.rados.0.smithi162.stdout:808: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2024-09-16T15:04:21.396 INFO:tasks.rados.rados.0.smithi162.stdout:808: writing smithi16234619-35 from 511012 to 1060169 tid 1 2024-09-16T15:04:21.403 INFO:tasks.rados.rados.0.smithi162.stdout:808: writing smithi16234619-35 from 1775251 to 2326011 tid 2 2024-09-16T15:04:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:808: writing smithi16234619-35 from 3095195 to 3311012 tid 3 2024-09-16T15:04:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:809: write (excl) oid 17 current snap is 70 2024-09-16T15:04:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:809: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2024-09-16T15:04:21.416 INFO:tasks.rados.rados.0.smithi162.stdout:809: writing smithi16234619-17 from 527819 to 1152225 tid 1 2024-09-16T15:04:21.423 INFO:tasks.rados.rados.0.smithi162.stdout:809: writing smithi16234619-17 from 1833733 to 2244504 tid 2 2024-09-16T15:04:21.430 INFO:tasks.rados.rados.0.smithi162.stdout:809: writing smithi16234619-17 from 2722618 to 3324199 tid 3 2024-09-16T15:04:21.430 INFO:tasks.rados.rados.0.smithi162.stdout:810: snap_create 2024-09-16T15:04:21.430 INFO:tasks.rados.rados.0.smithi162.stdout:806: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:04:21.678 INFO:tasks.rados.rados.0.smithi162.stdout:808: finishing write tid 1 to smithi16234619-35 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: finishing write tid 2 to smithi16234619-35 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: finishing write tid 3 to smithi16234619-35 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: finishing write tid 4 to smithi16234619-35 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: finishing write tid 5 to smithi16234619-35 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: oid 35 updating version 0 to 263 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: oid 35 updating version 263 to 264 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: oid 35 updating version 264 to 265 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: oid 35 updating version 265 to 266 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 266 (ObjNum 316 snap 70 seq_num 316) dirty exists 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:808: left oid 35 (ObjNum 316 snap 70 seq_num 316) 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:809: finishing write tid 1 to smithi16234619-17 2024-09-16T15:04:21.679 INFO:tasks.rados.rados.0.smithi162.stdout:809: finishing write tid 2 to smithi16234619-17 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: finishing write tid 3 to smithi16234619-17 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: finishing write tid 4 to smithi16234619-17 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: finishing write tid 5 to smithi16234619-17 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: oid 17 updating version 0 to 253 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: oid 17 version 253 is already newer than 252 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: oid 17 updating version 253 to 254 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: oid 17 version 254 is already newer than 251 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 254 (ObjNum 317 snap 70 seq_num 317) dirty exists 2024-09-16T15:04:21.680 INFO:tasks.rados.rados.0.smithi162.stdout:809: left oid 17 (ObjNum 317 snap 70 seq_num 317) 2024-09-16T15:04:22.424 INFO:tasks.rados.rados.0.smithi162.stdout:806: done (3 left) 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:808: done (2 left) 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:809: done (1 left) 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:810: done (0 left) 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:811: copy_from oid 37 from oid 9 current snap is 71 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:812: read oid 12 snap -1 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:812: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:813: write (excl) oid 49 current snap is 71 2024-09-16T15:04:22.425 INFO:tasks.rados.rados.0.smithi162.stdout:813: seq_num 318 ranges {544626=699655,1892215=554429,3150041=194585} 2024-09-16T15:04:22.431 INFO:tasks.rados.rados.0.smithi162.stdout:813: writing smithi16234619-49 from 544626 to 1244281 tid 1 2024-09-16T15:04:22.438 INFO:tasks.rados.rados.0.smithi162.stdout:813: writing smithi16234619-49 from 1892215 to 2446644 tid 2 2024-09-16T15:04:22.443 INFO:tasks.rados.rados.0.smithi162.stdout:813: writing smithi16234619-49 from 3150041 to 3344626 tid 3 2024-09-16T15:04:22.443 INFO:tasks.rados.rados.0.smithi162.stdout:814: rollback oid 18 current snap is 71 2024-09-16T15:04:22.443 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 70 2024-09-16T15:04:22.443 INFO:tasks.rados.rados.0.smithi162.stdout:812: expect (ObjNum 258 snap 53 seq_num 258) 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:814: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:814: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:814: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 235 (ObjNum 278 snap 58 seq_num 278) dirty exists 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:813: finishing write tid 1 to smithi16234619-49 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:813: finishing write tid 2 to smithi16234619-49 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:813: finishing write tid 3 to smithi16234619-49 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:813: finishing write tid 4 to smithi16234619-49 2024-09-16T15:04:22.612 INFO:tasks.rados.rados.0.smithi162.stdout:813: finishing write tid 5 to smithi16234619-49 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: oid 49 updating version 0 to 260 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: oid 49 updating version 260 to 262 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: oid 49 version 262 is already newer than 261 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: oid 49 version 262 is already newer than 259 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 262 (ObjNum 318 snap 71 seq_num 318) dirty exists 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: left oid 49 (ObjNum 318 snap 71 seq_num 318) 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:811: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:812: done (3 left) 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:813: done (2 left) 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:814: done (1 left) 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:815: copy_from oid 28 from oid 33 current snap is 71 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:811: finishing copy_from to smithi16234619-37 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 235 (ObjNum 293 snap 63 seq_num 293) dirty exists 2024-09-16T15:04:22.613 INFO:tasks.rados.rados.0.smithi162.stdout:811: done (1 left) 2024-09-16T15:04:22.614 INFO:tasks.rados.rados.0.smithi162.stdout:816: snap_create 2024-09-16T15:04:22.658 INFO:tasks.rados.rados.0.smithi162.stdout:815: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:04:22.659 INFO:tasks.rados.rados.0.smithi162.stdout:815: finishing copy_from to smithi16234619-28 2024-09-16T15:04:22.659 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:04:23.429 INFO:tasks.rados.rados.0.smithi162.stdout:815: done (1 left) 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:816: done (0 left) 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:817: read oid 36 snap -1 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:817: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:818: read oid 30 snap -1 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:818: expect (ObjNum 275 snap 57 seq_num 275) 2024-09-16T15:04:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:819: snap_remove snap 65 2024-09-16T15:04:24.438 INFO:tasks.rados.rados.0.smithi162.stdout:819: done (2 left) 2024-09-16T15:04:24.438 INFO:tasks.rados.rados.0.smithi162.stdout:820: rollback oid 21 current snap is 72 2024-09-16T15:04:24.438 INFO:tasks.rados.rados.0.smithi162.stdout:817: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:04:24.524 INFO:tasks.rados.rados.0.smithi162.stdout:818: expect (ObjNum 275 snap 57 seq_num 275) 2024-09-16T15:04:24.641 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 70 2024-09-16T15:04:24.641 INFO:tasks.rados.rados.0.smithi162.stdout:817: done (2 left) 2024-09-16T15:04:24.641 INFO:tasks.rados.rados.0.smithi162.stdout:818: done (1 left) 2024-09-16T15:04:24.642 INFO:tasks.rados.rados.0.smithi162.stdout:821: copy_from oid 22 from oid 32 current snap is 72 2024-09-16T15:04:24.642 INFO:tasks.rados.rados.0.smithi162.stdout:822: read oid 27 snap -1 2024-09-16T15:04:24.642 INFO:tasks.rados.rados.0.smithi162.stdout:822: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:04:24.642 INFO:tasks.rados.rados.0.smithi162.stdout:823: snap_remove snap 64 2024-09-16T15:04:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:823: done (3 left) 2024-09-16T15:04:25.594 INFO:tasks.rados.rados.0.smithi162.stdout:824: read oid 48 snap -1 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:824: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:825: delete oid 4 current snap is 72 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:821: finishing copy_from to smithi16234619-22 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:821: got expected ENOENT (src dne) 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:821: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:820: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:820: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:04:25.595 INFO:tasks.rados.rados.0.smithi162.stdout:822: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:04:25.774 INFO:tasks.rados.rados.0.smithi162.stdout:820: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:04:25.774 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 260 (ObjNum 247 snap 50 seq_num 247) dirty exists 2024-09-16T15:04:25.774 INFO:tasks.rados.rados.0.smithi162.stdout:824: expect (ObjNum 211 snap 44 seq_num 211) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:820: done (4 left) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:821: done (3 left) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:822: done (2 left) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:824: done (1 left) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:825: done (0 left) 2024-09-16T15:04:26.020 INFO:tasks.rados.rados.0.smithi162.stdout:826: read oid 41 snap -1 2024-09-16T15:04:26.021 INFO:tasks.rados.rados.0.smithi162.stdout:826: expect (ObjNum 301 snap 63 seq_num 301) 2024-09-16T15:04:26.021 INFO:tasks.rados.rados.0.smithi162.stdout:827: snap_remove snap 66 2024-09-16T15:04:26.612 INFO:tasks.rados.rados.0.smithi162.stdout:827: done (1 left) 2024-09-16T15:04:26.612 INFO:tasks.rados.rados.0.smithi162.stdout:828: write oid 49 current snap is 72 2024-09-16T15:04:26.612 INFO:tasks.rados.rados.0.smithi162.stdout:828: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2024-09-16T15:04:26.618 INFO:tasks.rados.rados.0.smithi162.stdout:828: writing smithi16234619-49 from 578240 to 1144746 tid 1 2024-09-16T15:04:26.624 INFO:tasks.rados.rados.0.smithi162.stdout:828: writing smithi16234619-49 from 1609179 to 2167277 tid 2 2024-09-16T15:04:26.630 INFO:tasks.rados.rados.0.smithi162.stdout:828: writing smithi16234619-49 from 2804887 to 3356637 tid 3 2024-09-16T15:04:26.630 INFO:tasks.rados.rados.0.smithi162.stdout:829: snap_remove snap 70 2024-09-16T15:04:27.614 INFO:tasks.rados.rados.0.smithi162.stdout:829: done (2 left) 2024-09-16T15:04:27.615 INFO:tasks.rados.rados.0.smithi162.stdout:830: read oid 10 snap 67 2024-09-16T15:04:27.615 INFO:tasks.rados.rados.0.smithi162.stdout:830: expect deleted 2024-09-16T15:04:27.615 INFO:tasks.rados.rados.0.smithi162.stdout:831: write (excl) oid 24 current snap is 72 2024-09-16T15:04:27.615 INFO:tasks.rados.rados.0.smithi162.stdout:831: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2024-09-16T15:04:27.621 INFO:tasks.rados.rados.0.smithi162.stdout:831: writing smithi16234619-24 from 595047 to 1236802 tid 1 2024-09-16T15:04:27.626 INFO:tasks.rados.rados.0.smithi162.stdout:831: writing smithi16234619-24 from 1667661 to 2085770 tid 2 2024-09-16T15:04:27.633 INFO:tasks.rados.rados.0.smithi162.stdout:831: writing smithi16234619-24 from 2548663 to 3311685 tid 3 2024-09-16T15:04:27.633 INFO:tasks.rados.rados.0.smithi162.stdout:832: snap_remove snap 71 2024-09-16T15:04:28.626 INFO:tasks.rados.rados.0.smithi162.stdout:832: done (4 left) 2024-09-16T15:04:28.626 INFO:tasks.rados.rados.0.smithi162.stdout:833: snap_remove snap 62 2024-09-16T15:04:29.630 INFO:tasks.rados.rados.0.smithi162.stdout:833: done (4 left) 2024-09-16T15:04:29.630 INFO:tasks.rados.rados.0.smithi162.stdout:834: snap_remove snap 69 2024-09-16T15:04:30.389 INFO:tasks.rados.rados.0.smithi162.stdout:834: done (4 left) 2024-09-16T15:04:30.390 INFO:tasks.rados.rados.0.smithi162.stdout:835: delete oid 15 current snap is 72 2024-09-16T15:04:30.390 INFO:tasks.rados.rados.0.smithi162.stdout:826: expect (ObjNum 301 snap 63 seq_num 301) 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: finishing write tid 1 to smithi16234619-49 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: finishing write tid 2 to smithi16234619-49 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: finishing write tid 3 to smithi16234619-49 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: finishing write tid 4 to smithi16234619-49 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: finishing write tid 5 to smithi16234619-49 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: oid 49 updating version 0 to 266 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: oid 49 version 266 is already newer than 263 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: oid 49 version 266 is already newer than 265 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: oid 49 version 266 is already newer than 264 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 266 (ObjNum 320 snap 72 seq_num 320) dirty exists 2024-09-16T15:04:30.570 INFO:tasks.rados.rados.0.smithi162.stdout:828: left oid 49 (ObjNum 320 snap 72 seq_num 320) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:831: finishing write tid 1 to smithi16234619-24 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:826: done (4 left) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:828: done (3 left) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:830: done (2 left) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:835: done (1 left) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:836: read oid 31 snap -1 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:836: expect (ObjNum 298 snap 63 seq_num 298) 2024-09-16T15:04:30.571 INFO:tasks.rados.rados.0.smithi162.stdout:837: snap_remove snap 67 2024-09-16T15:04:31.553 INFO:tasks.rados.rados.0.smithi162.stdout:837: done (2 left) 2024-09-16T15:04:31.553 INFO:tasks.rados.rados.0.smithi162.stdout:838: write (excl) oid 4 current snap is 72 2024-09-16T15:04:31.554 INFO:tasks.rados.rados.0.smithi162.stdout:838: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2024-09-16T15:04:31.561 INFO:tasks.rados.rados.0.smithi162.stdout:838: writing smithi16234619-4 from 628661 to 1420914 tid 1 2024-09-16T15:04:31.568 INFO:tasks.rados.rados.0.smithi162.stdout:838: writing smithi16234619-4 from 2184625 to 2606403 tid 2 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:838: writing smithi16234619-4 from 3403509 to 3428661 tid 3 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:839: read oid 34 snap -1 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:839: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:840: write (excl) oid 32 current snap is 72 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:831: finishing write tid 2 to smithi16234619-24 2024-09-16T15:04:31.573 INFO:tasks.rados.rados.0.smithi162.stdout:840: seq_num 324 ranges {645468=467502,1726754=565436,2798226=596838} 2024-09-16T15:04:31.578 INFO:tasks.rados.rados.0.smithi162.stdout:840: writing smithi16234619-32 from 645468 to 1112970 tid 1 2024-09-16T15:04:31.585 INFO:tasks.rados.rados.0.smithi162.stdout:840: writing smithi16234619-32 from 1726754 to 2292190 tid 2 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:840: writing smithi16234619-32 from 2798226 to 3395064 tid 3 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:841: delete oid 1 current snap is 72 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:831: finishing write tid 3 to smithi16234619-24 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:831: finishing write tid 4 to smithi16234619-24 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:831: finishing write tid 5 to smithi16234619-24 2024-09-16T15:04:31.592 INFO:tasks.rados.rados.0.smithi162.stdout:831: oid 24 updating version 0 to 272 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:831: oid 24 updating version 272 to 274 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:831: oid 24 version 274 is already newer than 273 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:831: oid 24 updating version 274 to 275 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 275 (ObjNum 321 snap 72 seq_num 321) dirty exists 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:831: left oid 24 (ObjNum 321 snap 72 seq_num 321) 2024-09-16T15:04:31.593 INFO:tasks.rados.rados.0.smithi162.stdout:836: expect (ObjNum 298 snap 63 seq_num 298) 2024-09-16T15:04:31.870 INFO:tasks.rados.rados.0.smithi162.stdout:839: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:838: finishing write tid 1 to smithi16234619-4 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:838: finishing write tid 2 to smithi16234619-4 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:831: done (5 left) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:836: done (4 left) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:839: done (3 left) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:841: done (2 left) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:842: read oid 22 snap -1 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:842: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:843: read oid 13 snap -1 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:843: expect (ObjNum 314 snap 67 seq_num 314) 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:844: read oid 34 snap -1 2024-09-16T15:04:31.974 INFO:tasks.rados.rados.0.smithi162.stdout:844: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:845: rollback oid 40 current snap is 72 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:845: done (5 left) 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:846: write oid 19 current snap is 72 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:838: finishing write tid 3 to smithi16234619-4 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:838: finishing write tid 4 to smithi16234619-4 2024-09-16T15:04:31.975 INFO:tasks.rados.rados.0.smithi162.stdout:846: seq_num 326 ranges {679082=618000,1843718=569105,2853072=626010} 2024-09-16T15:04:31.980 INFO:tasks.rados.rados.0.smithi162.stdout:846: writing smithi16234619-19 from 679082 to 1297082 tid 1 2024-09-16T15:04:31.986 INFO:tasks.rados.rados.0.smithi162.stdout:846: writing smithi16234619-19 from 1843718 to 2412823 tid 2 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:846: writing smithi16234619-19 from 2853072 to 3479082 tid 3 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:847: snap_create 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: finishing write tid 5 to smithi16234619-4 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: oid 4 updating version 0 to 272 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: oid 4 version 272 is already newer than 271 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: oid 4 updating version 272 to 273 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: oid 4 version 273 is already newer than 270 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 273 (ObjNum 323 snap 72 seq_num 323) dirty exists 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:838: left oid 4 (ObjNum 323 snap 72 seq_num 323) 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:840: finishing write tid 1 to smithi16234619-32 2024-09-16T15:04:31.993 INFO:tasks.rados.rados.0.smithi162.stdout:840: finishing write tid 2 to smithi16234619-32 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: finishing write tid 3 to smithi16234619-32 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: finishing write tid 4 to smithi16234619-32 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: finishing write tid 5 to smithi16234619-32 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: oid 32 updating version 0 to 177 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: oid 32 updating version 177 to 178 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: oid 32 updating version 178 to 179 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: oid 32 version 179 is already newer than 176 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 179 (ObjNum 324 snap 72 seq_num 324) dirty exists 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:840: left oid 32 (ObjNum 324 snap 72 seq_num 324) 2024-09-16T15:04:31.994 INFO:tasks.rados.rados.0.smithi162.stdout:843: expect (ObjNum 314 snap 67 seq_num 314) 2024-09-16T15:04:32.147 INFO:tasks.rados.rados.0.smithi162.stdout:844: expect (ObjNum 195 snap 39 seq_num 195) 2024-09-16T15:04:32.248 INFO:tasks.rados.rados.0.smithi162.stdout:842: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:32.443 INFO:tasks.rados.rados.0.smithi162.stdout:846: finishing write tid 1 to smithi16234619-19 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: finishing write tid 2 to smithi16234619-19 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: finishing write tid 3 to smithi16234619-19 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: finishing write tid 4 to smithi16234619-19 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: finishing write tid 5 to smithi16234619-19 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: oid 19 updating version 0 to 292 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: oid 19 version 292 is already newer than 291 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: oid 19 updating version 292 to 293 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: oid 19 updating version 293 to 294 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 294 (ObjNum 326 snap 72 seq_num 326) dirty exists 2024-09-16T15:04:32.444 INFO:tasks.rados.rados.0.smithi162.stdout:846: left oid 19 (ObjNum 326 snap 72 seq_num 326) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:838: done (6 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:840: done (5 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:842: done (4 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:843: done (3 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:844: done (2 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:846: done (1 left) 2024-09-16T15:04:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:847: done (0 left) 2024-09-16T15:04:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:848: snap_create 2024-09-16T15:04:33.731 INFO:tasks.rados.rados.0.smithi162.stdout:848: done (0 left) 2024-09-16T15:04:33.732 INFO:tasks.rados.rados.0.smithi162.stdout:849: snap_remove snap 72 2024-09-16T15:04:35.155 INFO:tasks.rados.rados.0.smithi162.stdout:849: done (0 left) 2024-09-16T15:04:35.156 INFO:tasks.rados.rados.0.smithi162.stdout:850: snap_create 2024-09-16T15:04:35.386 INFO:tasks.rados.rados.0.smithi162.stdout:850: done (0 left) 2024-09-16T15:04:35.387 INFO:tasks.rados.rados.0.smithi162.stdout:851: write (excl) oid 1 current snap is 75 2024-09-16T15:04:35.387 INFO:tasks.rados.rados.0.smithi162.stdout:851: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2024-09-16T15:04:35.392 INFO:tasks.rados.rados.0.smithi162.stdout:851: writing smithi16234619-1 from 695889 to 1105491 tid 1 2024-09-16T15:04:35.399 INFO:tasks.rados.rados.0.smithi162.stdout:851: writing smithi16234619-1 from 1618553 to 2331316 tid 2 2024-09-16T15:04:35.405 INFO:tasks.rados.rados.0.smithi162.stdout:851: writing smithi16234619-1 from 2996848 to 3495889 tid 3 2024-09-16T15:04:35.405 INFO:tasks.rados.rados.0.smithi162.stdout:852: write oid 10 current snap is 75 2024-09-16T15:04:35.405 INFO:tasks.rados.rados.0.smithi162.stdout:852: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2024-09-16T15:04:35.412 INFO:tasks.rados.rados.0.smithi162.stdout:852: writing smithi16234619-10 from 712696 to 1197547 tid 1 2024-09-16T15:04:35.419 INFO:tasks.rados.rados.0.smithi162.stdout:852: writing smithi16234619-10 from 1960682 to 2533456 tid 2 2024-09-16T15:04:35.425 INFO:tasks.rados.rados.0.smithi162.stdout:852: writing smithi16234619-10 from 3307918 to 3512696 tid 3 2024-09-16T15:04:35.425 INFO:tasks.rados.rados.0.smithi162.stdout:853: snap_remove snap 73 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:853: done (2 left) 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:854: snap_create 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: finishing write tid 1 to smithi16234619-1 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: finishing write tid 2 to smithi16234619-1 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: finishing write tid 3 to smithi16234619-1 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: finishing write tid 4 to smithi16234619-1 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: finishing write tid 5 to smithi16234619-1 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: oid 1 updating version 0 to 275 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: oid 1 updating version 275 to 278 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: oid 1 version 278 is already newer than 277 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: oid 1 version 278 is already newer than 276 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 278 (ObjNum 327 snap 75 seq_num 327) dirty exists 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: left oid 1 (ObjNum 327 snap 75 seq_num 327) 2024-09-16T15:04:36.390 INFO:tasks.rados.rados.0.smithi162.stdout:852: finishing write tid 1 to smithi16234619-10 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: finishing write tid 2 to smithi16234619-10 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: finishing write tid 3 to smithi16234619-10 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: finishing write tid 4 to smithi16234619-10 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: finishing write tid 5 to smithi16234619-10 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: oid 10 updating version 0 to 298 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: oid 10 updating version 298 to 299 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: oid 10 updating version 299 to 301 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: oid 10 version 301 is already newer than 300 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 301 (ObjNum 328 snap 75 seq_num 328) dirty exists 2024-09-16T15:04:36.391 INFO:tasks.rados.rados.0.smithi162.stdout:852: left oid 10 (ObjNum 328 snap 75 seq_num 328) 2024-09-16T15:04:37.390 INFO:tasks.rados.rados.0.smithi162.stdout:851: done (2 left) 2024-09-16T15:04:37.390 INFO:tasks.rados.rados.0.smithi162.stdout:852: done (1 left) 2024-09-16T15:04:37.390 INFO:tasks.rados.rados.0.smithi162.stdout:854: done (0 left) 2024-09-16T15:04:37.390 INFO:tasks.rados.rados.0.smithi162.stdout:855: rollback oid 23 current snap is 76 2024-09-16T15:04:37.390 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 74 2024-09-16T15:04:37.391 INFO:tasks.rados.rados.0.smithi162.stdout:856: delete oid 8 current snap is 76 2024-09-16T15:04:37.397 INFO:tasks.rados.rados.0.smithi162.stdout:855: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:04:37.397 INFO:tasks.rados.rados.0.smithi162.stdout:856: done (1 left) 2024-09-16T15:04:37.397 INFO:tasks.rados.rados.0.smithi162.stdout:857: snap_create 2024-09-16T15:04:37.398 INFO:tasks.rados.rados.0.smithi162.stdout:855: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:04:37.399 INFO:tasks.rados.rados.0.smithi162.stdout:855: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:04:37.399 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 82 (ObjNum 296 snap 63 seq_num 296) dirty exists 2024-09-16T15:04:38.406 INFO:tasks.rados.rados.0.smithi162.stdout:855: done (1 left) 2024-09-16T15:04:38.406 INFO:tasks.rados.rados.0.smithi162.stdout:857: done (0 left) 2024-09-16T15:04:38.406 INFO:tasks.rados.rados.0.smithi162.stdout:858: write (excl) oid 20 current snap is 77 2024-09-16T15:04:38.407 INFO:tasks.rados.rados.0.smithi162.stdout:858: seq_num 330 ranges {746310=635349,2077646=576443,3362764=183546} 2024-09-16T15:04:38.414 INFO:tasks.rados.rados.0.smithi162.stdout:858: writing smithi16234619-20 from 746310 to 1381659 tid 1 2024-09-16T15:04:38.422 INFO:tasks.rados.rados.0.smithi162.stdout:858: writing smithi16234619-20 from 2077646 to 2654089 tid 2 2024-09-16T15:04:38.427 INFO:tasks.rados.rados.0.smithi162.stdout:858: writing smithi16234619-20 from 3362764 to 3546310 tid 3 2024-09-16T15:04:38.428 INFO:tasks.rados.rados.0.smithi162.stdout:859: snap_create 2024-09-16T15:04:38.450 INFO:tasks.rados.rados.0.smithi162.stdout:858: finishing write tid 1 to smithi16234619-20 2024-09-16T15:04:38.450 INFO:tasks.rados.rados.0.smithi162.stdout:858: finishing write tid 2 to smithi16234619-20 2024-09-16T15:04:38.450 INFO:tasks.rados.rados.0.smithi162.stdout:858: finishing write tid 3 to smithi16234619-20 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: finishing write tid 4 to smithi16234619-20 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: finishing write tid 5 to smithi16234619-20 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: oid 20 updating version 0 to 166 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: oid 20 updating version 166 to 167 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: oid 20 version 167 is already newer than 165 2024-09-16T15:04:38.451 INFO:tasks.rados.rados.0.smithi162.stdout:858: oid 20 updating version 167 to 168 2024-09-16T15:04:38.452 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 168 (ObjNum 330 snap 77 seq_num 330) dirty exists 2024-09-16T15:04:38.452 INFO:tasks.rados.rados.0.smithi162.stdout:858: left oid 20 (ObjNum 330 snap 77 seq_num 330) 2024-09-16T15:04:39.417 INFO:tasks.rados.rados.0.smithi162.stdout:858: done (1 left) 2024-09-16T15:04:39.417 INFO:tasks.rados.rados.0.smithi162.stdout:859: done (0 left) 2024-09-16T15:04:39.417 INFO:tasks.rados.rados.0.smithi162.stdout:860: write oid 48 current snap is 78 2024-09-16T15:04:39.417 INFO:tasks.rados.rados.0.smithi162.stdout:860: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2024-09-16T15:04:39.425 INFO:tasks.rados.rados.0.smithi162.stdout:860: writing smithi16234619-48 from 763117 to 1473715 tid 1 2024-09-16T15:04:39.433 INFO:tasks.rados.rados.0.smithi162.stdout:860: writing smithi16234619-48 from 2136128 to 2856229 tid 2 2024-09-16T15:04:39.437 INFO:tasks.rados.rados.0.smithi162.stdout:860: writing smithi16234619-48 from 3390187 to 3563117 tid 3 2024-09-16T15:04:39.437 INFO:tasks.rados.rados.0.smithi162.stdout:861: write (excl) oid 34 current snap is 78 2024-09-16T15:04:39.437 INFO:tasks.rados.rados.0.smithi162.stdout:861: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2024-09-16T15:04:39.445 INFO:tasks.rados.rados.0.smithi162.stdout:861: writing smithi16234619-34 from 779924 to 1565771 tid 1 2024-09-16T15:04:39.451 INFO:tasks.rados.rados.0.smithi162.stdout:861: writing smithi16234619-34 from 2078257 to 2658369 tid 2 2024-09-16T15:04:39.456 INFO:tasks.rados.rados.0.smithi162.stdout:861: writing smithi16234619-34 from 3301257 to 3579924 tid 3 2024-09-16T15:04:39.457 INFO:tasks.rados.rados.0.smithi162.stdout:862: read oid 35 snap -1 2024-09-16T15:04:39.457 INFO:tasks.rados.rados.0.smithi162.stdout:862: expect (ObjNum 316 snap 70 seq_num 316) 2024-09-16T15:04:39.457 INFO:tasks.rados.rados.0.smithi162.stdout:863: snap_remove snap 74 2024-09-16T15:04:40.388 INFO:tasks.rados.rados.0.smithi162.stdout:863: done (3 left) 2024-09-16T15:04:40.388 INFO:tasks.rados.rados.0.smithi162.stdout:864: rollback oid 23 current snap is 78 2024-09-16T15:04:40.388 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 75 2024-09-16T15:04:40.388 INFO:tasks.rados.rados.0.smithi162.stdout:862: expect (ObjNum 316 snap 70 seq_num 316) 2024-09-16T15:04:40.600 INFO:tasks.rados.rados.0.smithi162.stdout:860: finishing write tid 1 to smithi16234619-48 2024-09-16T15:04:40.600 INFO:tasks.rados.rados.0.smithi162.stdout:860: finishing write tid 2 to smithi16234619-48 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:862: done (3 left) 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:865: copy_from oid 28 from oid 46 current snap is 78 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: finishing write tid 3 to smithi16234619-48 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: finishing write tid 4 to smithi16234619-48 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: finishing write tid 5 to smithi16234619-48 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: oid 48 updating version 0 to 281 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: oid 48 updating version 281 to 283 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: oid 48 version 283 is already newer than 282 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: oid 48 version 283 is already newer than 280 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 283 (ObjNum 331 snap 78 seq_num 331) dirty exists 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:860: left oid 48 (ObjNum 331 snap 78 seq_num 331) 2024-09-16T15:04:40.601 INFO:tasks.rados.rados.0.smithi162.stdout:861: finishing write tid 1 to smithi16234619-34 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: finishing write tid 2 to smithi16234619-34 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: finishing write tid 3 to smithi16234619-34 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: finishing write tid 4 to smithi16234619-34 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: finishing write tid 5 to smithi16234619-34 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: oid 34 updating version 0 to 298 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: oid 34 version 298 is already newer than 297 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: oid 34 version 298 is already newer than 296 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: oid 34 updating version 298 to 299 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 299 (ObjNum 332 snap 78 seq_num 332) dirty exists 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:861: left oid 34 (ObjNum 332 snap 78 seq_num 332) 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:864: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:04:40.602 INFO:tasks.rados.rados.0.smithi162.stdout:864: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:864: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 88 (ObjNum 296 snap 63 seq_num 296) dirty exists 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:860: done (3 left) 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:861: done (2 left) 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:864: done (1 left) 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:866: write oid 32 current snap is 78 2024-09-16T15:04:40.603 INFO:tasks.rados.rados.0.smithi162.stdout:866: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2024-09-16T15:04:40.606 INFO:tasks.rados.rados.0.smithi162.stdout:866: writing smithi16234619-32 from 796731 to 1257827 tid 1 2024-09-16T15:04:40.612 INFO:tasks.rados.rados.0.smithi162.stdout:866: writing smithi16234619-32 from 1736739 to 2460509 tid 2 2024-09-16T15:04:40.618 INFO:tasks.rados.rados.0.smithi162.stdout:866: writing smithi16234619-32 from 2928680 to 3596731 tid 3 2024-09-16T15:04:40.619 INFO:tasks.rados.rados.0.smithi162.stdout:867: rollback oid 13 current snap is 78 2024-09-16T15:04:40.619 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 76 2024-09-16T15:04:40.619 INFO:tasks.rados.rados.0.smithi162.stdout:868: write (excl) oid 44 current snap is 78 2024-09-16T15:04:40.619 INFO:tasks.rados.rados.0.smithi162.stdout:868: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2024-09-16T15:04:40.624 INFO:tasks.rados.rados.0.smithi162.stdout:868: writing smithi16234619-44 from 413538 to 949883 tid 1 2024-09-16T15:04:40.629 INFO:tasks.rados.rados.0.smithi162.stdout:868: writing smithi16234619-44 from 1395221 to 1979002 tid 2 2024-09-16T15:04:40.636 INFO:tasks.rados.rados.0.smithi162.stdout:868: writing smithi16234619-44 from 2556103 to 3214720 tid 3 2024-09-16T15:04:40.636 INFO:tasks.rados.rados.0.smithi162.stdout:869: read oid 38 snap -1 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:869: expect (ObjNum 308 snap 64 seq_num 308) 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:870: read oid 15 snap -1 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:870: expect deleted 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:871: delete oid 48 current snap is 78 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:867: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:867: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:867: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 364 (ObjNum 314 snap 67 seq_num 314) dirty exists 2024-09-16T15:04:40.637 INFO:tasks.rados.rados.0.smithi162.stdout:865: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:04:40.638 INFO:tasks.rados.rados.0.smithi162.stdout:867: done (6 left) 2024-09-16T15:04:40.639 INFO:tasks.rados.rados.0.smithi162.stdout:870: done (5 left) 2024-09-16T15:04:40.639 INFO:tasks.rados.rados.0.smithi162.stdout:871: done (4 left) 2024-09-16T15:04:40.639 INFO:tasks.rados.rados.0.smithi162.stdout:872: snap_create 2024-09-16T15:04:40.641 INFO:tasks.rados.rados.0.smithi162.stdout:865: finishing copy_from to smithi16234619-28 2024-09-16T15:04:40.641 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:04:40.657 INFO:tasks.rados.rados.0.smithi162.stdout:866: finishing write tid 1 to smithi16234619-32 2024-09-16T15:04:40.658 INFO:tasks.rados.rados.0.smithi162.stdout:866: finishing write tid 2 to smithi16234619-32 2024-09-16T15:04:40.660 INFO:tasks.rados.rados.0.smithi162.stdout:866: finishing write tid 3 to smithi16234619-32 2024-09-16T15:04:40.660 INFO:tasks.rados.rados.0.smithi162.stdout:866: finishing write tid 4 to smithi16234619-32 2024-09-16T15:04:40.660 INFO:tasks.rados.rados.0.smithi162.stdout:866: finishing write tid 5 to smithi16234619-32 2024-09-16T15:04:40.660 INFO:tasks.rados.rados.0.smithi162.stdout:866: oid 32 updating version 0 to 183 2024-09-16T15:04:40.661 INFO:tasks.rados.rados.0.smithi162.stdout:866: oid 32 version 183 is already newer than 180 2024-09-16T15:04:40.661 INFO:tasks.rados.rados.0.smithi162.stdout:866: oid 32 version 183 is already newer than 181 2024-09-16T15:04:40.661 INFO:tasks.rados.rados.0.smithi162.stdout:866: oid 32 version 183 is already newer than 182 2024-09-16T15:04:40.661 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 183 (ObjNum 333 snap 78 seq_num 333) dirty exists 2024-09-16T15:04:40.661 INFO:tasks.rados.rados.0.smithi162.stdout:866: left oid 32 (ObjNum 333 snap 78 seq_num 333) 2024-09-16T15:04:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:868: finishing write tid 1 to smithi16234619-44 2024-09-16T15:04:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:868: finishing write tid 2 to smithi16234619-44 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: finishing write tid 3 to smithi16234619-44 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: finishing write tid 4 to smithi16234619-44 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: finishing write tid 5 to smithi16234619-44 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: oid 44 updating version 0 to 267 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: oid 44 updating version 267 to 269 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: oid 44 updating version 269 to 270 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:868: oid 44 version 270 is already newer than 268 2024-09-16T15:04:40.671 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 270 (ObjNum 334 snap 78 seq_num 334) dirty exists 2024-09-16T15:04:40.672 INFO:tasks.rados.rados.0.smithi162.stdout:868: left oid 44 (ObjNum 334 snap 78 seq_num 334) 2024-09-16T15:04:40.672 INFO:tasks.rados.rados.0.smithi162.stdout:869: expect (ObjNum 308 snap 64 seq_num 308) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:865: done (4 left) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:866: done (3 left) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:868: done (2 left) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:869: done (1 left) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:872: done (0 left) 2024-09-16T15:04:41.426 INFO:tasks.rados.rados.0.smithi162.stdout:873: snap_create 2024-09-16T15:04:42.434 INFO:tasks.rados.rados.0.smithi162.stdout:873: done (0 left) 2024-09-16T15:04:42.434 INFO:tasks.rados.rados.0.smithi162.stdout:874: snap_remove snap 77 2024-09-16T15:04:43.438 INFO:tasks.rados.rados.0.smithi162.stdout:874: done (0 left) 2024-09-16T15:04:43.438 INFO:tasks.rados.rados.0.smithi162.stdout:875: delete oid 49 current snap is 80 2024-09-16T15:04:43.447 INFO:tasks.rados.rados.0.smithi162.stdout:875: done (0 left) 2024-09-16T15:04:43.447 INFO:tasks.rados.rados.0.smithi162.stdout:876: read oid 7 snap -1 2024-09-16T15:04:43.447 INFO:tasks.rados.rados.0.smithi162.stdout:876: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:04:43.447 INFO:tasks.rados.rados.0.smithi162.stdout:877: delete oid 16 current snap is 80 2024-09-16T15:04:43.449 INFO:tasks.rados.rados.0.smithi162.stdout:877: done (1 left) 2024-09-16T15:04:43.449 INFO:tasks.rados.rados.0.smithi162.stdout:878: read oid 18 snap 75 2024-09-16T15:04:43.449 INFO:tasks.rados.rados.0.smithi162.stdout:878: expect (ObjNum 278 snap 58 seq_num 278) 2024-09-16T15:04:43.450 INFO:tasks.rados.rados.0.smithi162.stdout:879: snap_remove snap 78 2024-09-16T15:04:44.440 INFO:tasks.rados.rados.0.smithi162.stdout:879: done (2 left) 2024-09-16T15:04:44.440 INFO:tasks.rados.rados.0.smithi162.stdout:880: write oid 23 current snap is 80 2024-09-16T15:04:44.440 INFO:tasks.rados.rados.0.smithi162.stdout:880: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2024-09-16T15:04:44.445 INFO:tasks.rados.rados.0.smithi162.stdout:880: writing smithi16234619-23 from 480766 to 1034460 tid 1 2024-09-16T15:04:44.452 INFO:tasks.rados.rados.0.smithi162.stdout:880: writing smithi16234619-23 from 1629149 to 2220268 tid 2 2024-09-16T15:04:44.458 INFO:tasks.rados.rados.0.smithi162.stdout:880: writing smithi16234619-23 from 2782148 to 3202206 tid 3 2024-09-16T15:04:44.458 INFO:tasks.rados.rados.0.smithi162.stdout:881: snap_remove snap 76 2024-09-16T15:04:45.391 INFO:tasks.rados.rados.0.smithi162.stdout:881: done (3 left) 2024-09-16T15:04:45.391 INFO:tasks.rados.rados.0.smithi162.stdout:882: snap_remove snap 79 2024-09-16T15:04:46.390 INFO:tasks.rados.rados.0.smithi162.stdout:882: done (3 left) 2024-09-16T15:04:46.390 INFO:tasks.rados.rados.0.smithi162.stdout:883: delete oid 33 current snap is 80 2024-09-16T15:04:46.391 INFO:tasks.rados.rados.0.smithi162.stdout:876: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:04:46.639 INFO:tasks.rados.rados.0.smithi162.stdout:878: expect (ObjNum 278 snap 58 seq_num 278) 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: finishing write tid 1 to smithi16234619-23 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: finishing write tid 2 to smithi16234619-23 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: finishing write tid 3 to smithi16234619-23 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: finishing write tid 4 to smithi16234619-23 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: finishing write tid 5 to smithi16234619-23 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: oid 23 updating version 0 to 95 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: oid 23 version 95 is already newer than 92 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: oid 23 version 95 is already newer than 93 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: oid 23 version 95 is already newer than 94 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 95 (ObjNum 338 snap 80 seq_num 338) dirty exists 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: left oid 23 (ObjNum 338 snap 80 seq_num 338) 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:876: done (3 left) 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:878: done (2 left) 2024-09-16T15:04:46.822 INFO:tasks.rados.rados.0.smithi162.stdout:880: done (1 left) 2024-09-16T15:04:46.823 INFO:tasks.rados.rados.0.smithi162.stdout:883: done (0 left) 2024-09-16T15:04:46.823 INFO:tasks.rados.rados.0.smithi162.stdout:884: delete oid 50 current snap is 80 2024-09-16T15:04:46.825 INFO:tasks.rados.rados.0.smithi162.stdout:884: done (0 left) 2024-09-16T15:04:46.825 INFO:tasks.rados.rados.0.smithi162.stdout:885: read oid 45 snap -1 2024-09-16T15:04:46.825 INFO:tasks.rados.rados.0.smithi162.stdout:885: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:04:46.825 INFO:tasks.rados.rados.0.smithi162.stdout:886: delete oid 31 current snap is 80 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:886: done (1 left) 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:887: read oid 19 snap -1 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:887: expect (ObjNum 326 snap 72 seq_num 326) 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:888: rollback oid 46 current snap is 80 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 75 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:889: write oid 9 current snap is 80 2024-09-16T15:04:46.828 INFO:tasks.rados.rados.0.smithi162.stdout:889: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2024-09-16T15:04:46.833 INFO:tasks.rados.rados.0.smithi162.stdout:889: writing smithi16234619-9 from 547994 to 1002684 tid 1 2024-09-16T15:04:46.840 INFO:tasks.rados.rados.0.smithi162.stdout:889: writing smithi16234619-9 from 1746724 to 2228828 tid 2 2024-09-16T15:04:46.845 INFO:tasks.rados.rados.0.smithi162.stdout:889: writing smithi16234619-9 from 2659134 to 3124280 tid 3 2024-09-16T15:04:46.848 INFO:tasks.rados.rados.0.smithi162.stdout:889: writing smithi16234619-9 from 3605976 to 3747994 tid 4 2024-09-16T15:04:46.849 INFO:tasks.rados.rados.0.smithi162.stdout:890: rollback oid 4 current snap is 80 2024-09-16T15:04:46.849 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 75 2024-09-16T15:04:46.849 INFO:tasks.rados.rados.0.smithi162.stdout:885: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:04:46.958 INFO:tasks.rados.rados.0.smithi162.stdout:888: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:04:46.958 INFO:tasks.rados.rados.0.smithi162.stdout:888: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:04:46.958 INFO:tasks.rados.rados.0.smithi162.stdout:888: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:04:46.959 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 95 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:04:46.959 INFO:tasks.rados.rados.0.smithi162.stdout:887: expect (ObjNum 326 snap 72 seq_num 326) 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 1 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 2 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 3 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 4 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 5 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: finishing write tid 6 to smithi16234619-9 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: oid 9 updating version 0 to 293 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: oid 9 updating version 293 to 294 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: oid 9 updating version 294 to 295 2024-09-16T15:04:47.225 INFO:tasks.rados.rados.0.smithi162.stdout:889: oid 9 updating version 295 to 296 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:889: oid 9 version 296 is already newer than 292 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 296 (ObjNum 342 snap 80 seq_num 342) dirty exists 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:889: left oid 9 (ObjNum 342 snap 80 seq_num 342) 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:890: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:885: done (4 left) 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:887: done (3 left) 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:888: done (2 left) 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:889: done (1 left) 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:891: write (excl) oid 48 current snap is 80 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:890: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:890: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 300 (ObjNum 323 snap 72 seq_num 323) dirty exists 2024-09-16T15:04:47.226 INFO:tasks.rados.rados.0.smithi162.stdout:891: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2024-09-16T15:04:47.231 INFO:tasks.rados.rados.0.smithi162.stdout:891: writing smithi16234619-48 from 564801 to 1211093 tid 1 2024-09-16T15:04:47.240 INFO:tasks.rados.rados.0.smithi162.stdout:891: writing smithi16234619-48 from 1921559 to 2663674 tid 2 2024-09-16T15:04:47.247 INFO:tasks.rados.rados.0.smithi162.stdout:891: writing smithi16234619-48 from 3202910 to 3764801 tid 3 2024-09-16T15:04:47.247 INFO:tasks.rados.rados.0.smithi162.stdout:890: done (1 left) 2024-09-16T15:04:47.247 INFO:tasks.rados.rados.0.smithi162.stdout:892: read oid 4 snap -1 2024-09-16T15:04:47.247 INFO:tasks.rados.rados.0.smithi162.stdout:892: expect (ObjNum 323 snap 72 seq_num 323) 2024-09-16T15:04:47.248 INFO:tasks.rados.rados.0.smithi162.stdout:893: read oid 21 snap -1 2024-09-16T15:04:47.248 INFO:tasks.rados.rados.0.smithi162.stdout:893: expect (ObjNum 247 snap 50 seq_num 247) 2024-09-16T15:04:47.248 INFO:tasks.rados.rados.0.smithi162.stdout:894: snap_remove snap 75 2024-09-16T15:04:47.835 INFO:tasks.rados.rados.0.smithi162.stdout:894: done (3 left) 2024-09-16T15:04:47.835 INFO:tasks.rados.rados.0.smithi162.stdout:895: delete oid 18 current snap is 80 2024-09-16T15:04:47.835 INFO:tasks.rados.rados.0.smithi162.stdout:893: expect (ObjNum 247 snap 50 seq_num 247) 2024-09-16T15:04:47.976 INFO:tasks.rados.rados.0.smithi162.stdout:891: finishing write tid 1 to smithi16234619-48 2024-09-16T15:04:47.976 INFO:tasks.rados.rados.0.smithi162.stdout:891: finishing write tid 2 to smithi16234619-48 2024-09-16T15:04:47.976 INFO:tasks.rados.rados.0.smithi162.stdout:891: finishing write tid 3 to smithi16234619-48 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: finishing write tid 4 to smithi16234619-48 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: finishing write tid 5 to smithi16234619-48 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: oid 48 updating version 0 to 301 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: oid 48 updating version 301 to 302 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: oid 48 updating version 302 to 303 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: oid 48 updating version 303 to 304 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 304 (ObjNum 343 snap 80 seq_num 343) dirty exists 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:891: left oid 48 (ObjNum 343 snap 80 seq_num 343) 2024-09-16T15:04:47.977 INFO:tasks.rados.rados.0.smithi162.stdout:892: expect (ObjNum 323 snap 72 seq_num 323) 2024-09-16T15:04:48.089 INFO:tasks.rados.rados.0.smithi162.stdout:891: done (3 left) 2024-09-16T15:04:48.089 INFO:tasks.rados.rados.0.smithi162.stdout:892: done (2 left) 2024-09-16T15:04:48.090 INFO:tasks.rados.rados.0.smithi162.stdout:893: done (1 left) 2024-09-16T15:04:48.090 INFO:tasks.rados.rados.0.smithi162.stdout:895: done (0 left) 2024-09-16T15:04:48.090 INFO:tasks.rados.rados.0.smithi162.stdout:896: snap_create 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:896: done (0 left) 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:897: read oid 31 snap 80 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:897: expect deleted 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:898: read oid 33 snap -1 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:898: expect deleted 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:899: write (excl) oid 21 current snap is 81 2024-09-16T15:04:48.843 INFO:tasks.rados.rados.0.smithi162.stdout:899: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2024-09-16T15:04:48.850 INFO:tasks.rados.rados.0.smithi162.stdout:899: writing smithi16234619-21 from 598415 to 1395205 tid 1 2024-09-16T15:04:48.857 INFO:tasks.rados.rados.0.smithi162.stdout:899: writing smithi16234619-21 from 1922170 to 2667954 tid 2 2024-09-16T15:04:48.862 INFO:tasks.rados.rados.0.smithi162.stdout:899: writing smithi16234619-21 from 3141403 to 3556718 tid 3 2024-09-16T15:04:48.862 INFO:tasks.rados.rados.0.smithi162.stdout:900: read oid 42 snap -1 2024-09-16T15:04:48.862 INFO:tasks.rados.rados.0.smithi162.stdout:900: expect (ObjNum 315 snap 67 seq_num 315) 2024-09-16T15:04:48.862 INFO:tasks.rados.rados.0.smithi162.stdout:901: write oid 5 current snap is 81 2024-09-16T15:04:48.863 INFO:tasks.rados.rados.0.smithi162.stdout:901: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2024-09-16T15:04:48.868 INFO:tasks.rados.rados.0.smithi162.stdout:901: writing smithi16234619-5 from 615222 to 1087261 tid 1 2024-09-16T15:04:48.874 INFO:tasks.rados.rados.0.smithi162.stdout:901: writing smithi16234619-5 from 1580652 to 2070094 tid 2 2024-09-16T15:04:48.881 INFO:tasks.rados.rados.0.smithi162.stdout:901: writing smithi16234619-5 from 2768826 to 3395413 tid 3 2024-09-16T15:04:48.881 INFO:tasks.rados.rados.0.smithi162.stdout:902: delete oid 32 current snap is 81 2024-09-16T15:04:48.882 INFO:tasks.rados.rados.0.smithi162.stdout:900: expect (ObjNum 315 snap 67 seq_num 315) 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:899: finishing write tid 1 to smithi16234619-21 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:899: finishing write tid 2 to smithi16234619-21 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:899: finishing write tid 3 to smithi16234619-21 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:897: done (5 left) 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:898: done (4 left) 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:900: done (3 left) 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:902: done (2 left) 2024-09-16T15:04:48.991 INFO:tasks.rados.rados.0.smithi162.stdout:903: snap_remove snap 80 2024-09-16T15:04:49.846 INFO:tasks.rados.rados.0.smithi162.stdout:903: done (2 left) 2024-09-16T15:04:49.846 INFO:tasks.rados.rados.0.smithi162.stdout:904: write (excl) oid 35 current snap is 81 2024-09-16T15:04:49.846 INFO:tasks.rados.rados.0.smithi162.stdout:904: seq_num 348 ranges {648836=622537,1981263=493111,3107319=741517} 2024-09-16T15:04:49.853 INFO:tasks.rados.rados.0.smithi162.stdout:904: writing smithi16234619-35 from 648836 to 1271373 tid 1 2024-09-16T15:04:49.859 INFO:tasks.rados.rados.0.smithi162.stdout:904: writing smithi16234619-35 from 1981263 to 2474374 tid 2 2024-09-16T15:04:49.867 INFO:tasks.rados.rados.0.smithi162.stdout:904: writing smithi16234619-35 from 3107319 to 3848836 tid 3 2024-09-16T15:04:49.867 INFO:tasks.rados.rados.0.smithi162.stdout:905: write (excl) oid 15 current snap is 81 2024-09-16T15:04:49.867 INFO:tasks.rados.rados.0.smithi162.stdout:905: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2024-09-16T15:04:49.874 INFO:tasks.rados.rados.0.smithi162.stdout:905: writing smithi16234619-15 from 665643 to 1363429 tid 1 2024-09-16T15:04:49.882 INFO:tasks.rados.rados.0.smithi162.stdout:905: writing smithi16234619-15 from 2039745 to 2792867 tid 2 2024-09-16T15:04:49.889 INFO:tasks.rados.rados.0.smithi162.stdout:905: writing smithi16234619-15 from 3534742 to 3865643 tid 3 2024-09-16T15:04:49.889 INFO:tasks.rados.rados.0.smithi162.stdout:906: delete oid 17 current snap is 81 2024-09-16T15:04:49.889 INFO:tasks.rados.rados.0.smithi162.stdout:899: finishing write tid 4 to smithi16234619-21 2024-09-16T15:04:49.889 INFO:tasks.rados.rados.0.smithi162.stdout:899: finishing write tid 5 to smithi16234619-21 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:899: oid 21 updating version 0 to 303 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:899: oid 21 updating version 303 to 305 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:899: oid 21 version 305 is already newer than 304 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:899: oid 21 updating version 305 to 306 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 306 (ObjNum 345 snap 81 seq_num 345) dirty exists 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:899: left oid 21 (ObjNum 345 snap 81 seq_num 345) 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:901: finishing write tid 1 to smithi16234619-5 2024-09-16T15:04:49.890 INFO:tasks.rados.rados.0.smithi162.stdout:901: finishing write tid 2 to smithi16234619-5 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: finishing write tid 3 to smithi16234619-5 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: finishing write tid 4 to smithi16234619-5 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: finishing write tid 5 to smithi16234619-5 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: oid 5 updating version 0 to 373 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: oid 5 updating version 373 to 374 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: oid 5 version 374 is already newer than 372 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: oid 5 version 374 is already newer than 371 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 374 (ObjNum 346 snap 81 seq_num 346) dirty exists 2024-09-16T15:04:49.891 INFO:tasks.rados.rados.0.smithi162.stdout:901: left oid 5 (ObjNum 346 snap 81 seq_num 346) 2024-09-16T15:04:49.892 INFO:tasks.rados.rados.0.smithi162.stdout:899: done (4 left) 2024-09-16T15:04:49.892 INFO:tasks.rados.rados.0.smithi162.stdout:901: done (3 left) 2024-09-16T15:04:49.892 INFO:tasks.rados.rados.0.smithi162.stdout:906: done (2 left) 2024-09-16T15:04:49.892 INFO:tasks.rados.rados.0.smithi162.stdout:907: delete oid 25 current snap is 81 2024-09-16T15:04:49.892 INFO:tasks.rados.rados.0.smithi162.stdout:907: done (2 left) 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:908: rollback oid 39 current snap is 81 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:908: done (2 left) 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:909: rollback oid 5 current snap is 81 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:909: done (2 left) 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:910: copy_from oid 37 from oid 24 current snap is 81 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:911: write oid 40 current snap is 81 2024-09-16T15:04:49.893 INFO:tasks.rados.rados.0.smithi162.stdout:911: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2024-09-16T15:04:49.900 INFO:tasks.rados.rados.0.smithi162.stdout:911: writing smithi16234619-40 from 716064 to 1355950 tid 1 2024-09-16T15:04:49.905 INFO:tasks.rados.rados.0.smithi162.stdout:911: writing smithi16234619-40 from 1815191 to 2315640 tid 2 2024-09-16T15:04:49.911 INFO:tasks.rados.rados.0.smithi162.stdout:911: writing smithi16234619-40 from 2817011 to 3343936 tid 3 2024-09-16T15:04:49.911 INFO:tasks.rados.rados.0.smithi162.stdout:912: write oid 4 current snap is 81 2024-09-16T15:04:49.911 INFO:tasks.rados.rados.0.smithi162.stdout:912: seq_num 353 ranges {732871=715135,1873673=644107,3244434=688437} 2024-09-16T15:04:49.919 INFO:tasks.rados.rados.0.smithi162.stdout:912: writing smithi16234619-4 from 732871 to 1448006 tid 1 2024-09-16T15:04:49.925 INFO:tasks.rados.rados.0.smithi162.stdout:912: writing smithi16234619-4 from 1873673 to 2517780 tid 2 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:912: writing smithi16234619-4 from 3244434 to 3932871 tid 3 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:913: delete oid 17 current snap is 81 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:904: finishing write tid 1 to smithi16234619-35 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:904: finishing write tid 2 to smithi16234619-35 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:904: finishing write tid 3 to smithi16234619-35 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:905: finishing write tid 1 to smithi16234619-15 2024-09-16T15:04:49.933 INFO:tasks.rados.rados.0.smithi162.stdout:904: finishing write tid 4 to smithi16234619-35 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: finishing write tid 2 to smithi16234619-15 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: finishing write tid 5 to smithi16234619-35 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: oid 35 updating version 0 to 272 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: oid 35 updating version 272 to 275 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: oid 35 version 275 is already newer than 273 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: oid 35 version 275 is already newer than 274 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 275 (ObjNum 348 snap 81 seq_num 348) dirty exists 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:904: left oid 35 (ObjNum 348 snap 81 seq_num 348) 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: finishing write tid 3 to smithi16234619-15 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: finishing write tid 4 to smithi16234619-15 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: finishing write tid 5 to smithi16234619-15 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: oid 15 updating version 0 to 253 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: oid 15 updating version 253 to 255 2024-09-16T15:04:49.934 INFO:tasks.rados.rados.0.smithi162.stdout:905: oid 15 version 255 is already newer than 254 2024-09-16T15:04:49.935 INFO:tasks.rados.rados.0.smithi162.stdout:905: oid 15 updating version 255 to 256 2024-09-16T15:04:49.935 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 256 (ObjNum 349 snap 81 seq_num 349) dirty exists 2024-09-16T15:04:49.935 INFO:tasks.rados.rados.0.smithi162.stdout:905: left oid 15 (ObjNum 349 snap 81 seq_num 349) 2024-09-16T15:04:49.935 INFO:tasks.rados.rados.0.smithi162.stdout:904: done (5 left) 2024-09-16T15:04:49.936 INFO:tasks.rados.rados.0.smithi162.stdout:905: done (4 left) 2024-09-16T15:04:49.936 INFO:tasks.rados.rados.0.smithi162.stdout:913: done (3 left) 2024-09-16T15:04:49.936 INFO:tasks.rados.rados.0.smithi162.stdout:914: copy_from oid 5 from oid 22 current snap is 81 2024-09-16T15:04:49.936 INFO:tasks.rados.rados.0.smithi162.stdout:915: write (excl) oid 6 current snap is 81 2024-09-16T15:04:49.936 INFO:tasks.rados.rados.0.smithi162.stdout:915: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2024-09-16T15:04:49.941 INFO:tasks.rados.rados.0.smithi162.stdout:915: writing smithi16234619-6 from 766485 to 1232118 tid 1 2024-09-16T15:04:49.950 INFO:tasks.rados.rados.0.smithi162.stdout:915: writing smithi16234619-6 from 1990637 to 2638413 tid 2 2024-09-16T15:04:49.958 INFO:tasks.rados.rados.0.smithi162.stdout:915: writing smithi16234619-6 from 3299280 to 3966485 tid 3 2024-09-16T15:04:49.958 INFO:tasks.rados.rados.0.smithi162.stdout:916: write oid 39 current snap is 81 2024-09-16T15:04:49.958 INFO:tasks.rados.rados.0.smithi162.stdout:916: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2024-09-16T15:04:49.966 INFO:tasks.rados.rados.0.smithi162.stdout:916: writing smithi16234619-39 from 783292 to 1324174 tid 1 2024-09-16T15:04:49.973 INFO:tasks.rados.rados.0.smithi162.stdout:916: writing smithi16234619-39 from 1932766 to 2440553 tid 2 2024-09-16T15:04:49.982 INFO:tasks.rados.rados.0.smithi162.stdout:916: writing smithi16234619-39 from 3210350 to 3782363 tid 3 2024-09-16T15:04:49.982 INFO:tasks.rados.rados.0.smithi162.stdout:917: rollback oid 11 current snap is 81 2024-09-16T15:04:49.982 INFO:tasks.rados.rados.0.smithi162.stdout:917: done (6 left) 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:918: delete oid 44 current snap is 81 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: finishing write tid 1 to smithi16234619-40 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: finishing write tid 2 to smithi16234619-40 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: finishing write tid 3 to smithi16234619-40 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: finishing write tid 4 to smithi16234619-40 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: finishing write tid 5 to smithi16234619-40 2024-09-16T15:04:49.983 INFO:tasks.rados.rados.0.smithi162.stdout:911: oid 40 updating version 0 to 188 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:911: oid 40 version 188 is already newer than 185 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:911: oid 40 version 188 is already newer than 186 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:911: oid 40 version 188 is already newer than 187 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 188 (ObjNum 352 snap 81 seq_num 352) dirty exists 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:911: left oid 40 (ObjNum 352 snap 81 seq_num 352) 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:910: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:04:49.984 INFO:tasks.rados.rados.0.smithi162.stdout:912: finishing write tid 1 to smithi16234619-4 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: finishing write tid 2 to smithi16234619-4 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: finishing write tid 3 to smithi16234619-4 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: finishing write tid 4 to smithi16234619-4 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: finishing write tid 5 to smithi16234619-4 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: oid 4 updating version 0 to 311 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: oid 4 updating version 311 to 312 2024-09-16T15:04:49.985 INFO:tasks.rados.rados.0.smithi162.stdout:912: oid 4 updating version 312 to 313 2024-09-16T15:04:49.986 INFO:tasks.rados.rados.0.smithi162.stdout:912: oid 4 updating version 313 to 314 2024-09-16T15:04:49.986 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 314 (ObjNum 353 snap 81 seq_num 353) dirty exists 2024-09-16T15:04:49.986 INFO:tasks.rados.rados.0.smithi162.stdout:912: left oid 4 (ObjNum 353 snap 81 seq_num 353) 2024-09-16T15:04:49.997 INFO:tasks.rados.rados.0.smithi162.stdout:915: finishing write tid 1 to smithi16234619-6 2024-09-16T15:04:50.000 INFO:tasks.rados.rados.0.smithi162.stdout:915: finishing write tid 2 to smithi16234619-6 2024-09-16T15:04:50.001 INFO:tasks.rados.rados.0.smithi162.stdout:915: finishing write tid 3 to smithi16234619-6 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: finishing write tid 4 to smithi16234619-6 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: finishing write tid 5 to smithi16234619-6 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: oid 6 updating version 0 to 278 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: oid 6 version 278 is already newer than 277 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: oid 6 updating version 278 to 279 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: oid 6 version 279 is already newer than 276 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 279 (ObjNum 355 snap 81 seq_num 355) dirty exists 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: left oid 6 (ObjNum 355 snap 81 seq_num 355) 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:911: done (6 left) 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:912: done (5 left) 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:915: done (4 left) 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:918: done (3 left) 2024-09-16T15:04:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:919: read oid 1 snap -1 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:919: expect (ObjNum 327 snap 75 seq_num 327) 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:920: copy_from oid 19 from oid 36 current snap is 81 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:921: rollback oid 8 current snap is 81 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:921: done (5 left) 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:922: write (excl) oid 50 current snap is 81 2024-09-16T15:04:50.003 INFO:tasks.rados.rados.0.smithi162.stdout:922: seq_num 358 ranges {416906=407733,1366083=511456} 2024-09-16T15:04:50.007 INFO:tasks.rados.rados.0.smithi162.stdout:922: writing smithi16234619-50 from 416906 to 824639 tid 1 2024-09-16T15:04:50.012 INFO:tasks.rados.rados.0.smithi162.stdout:922: writing smithi16234619-50 from 1366083 to 1877539 tid 2 2024-09-16T15:04:50.012 INFO:tasks.rados.rados.0.smithi162.stdout:923: rollback oid 18 current snap is 81 2024-09-16T15:04:50.012 INFO:tasks.rados.rados.0.smithi162.stdout:923: done (6 left) 2024-09-16T15:04:50.013 INFO:tasks.rados.rados.0.smithi162.stdout:924: copy_from oid 25 from oid 27 current snap is 81 2024-09-16T15:04:50.013 INFO:tasks.rados.rados.0.smithi162.stdout:919: expect (ObjNum 327 snap 75 seq_num 327) 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:920: finishing copy_from racing read to smithi16234619-19 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:914: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:910: finishing copy_from to smithi16234619-37 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 275 (ObjNum 321 snap 72 seq_num 321) dirty exists 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:916: finishing write tid 1 to smithi16234619-39 2024-09-16T15:04:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:914: finishing copy_from to smithi16234619-5 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 244 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:916: finishing write tid 2 to smithi16234619-39 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:910: done (6 left) 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:914: done (5 left) 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:919: done (4 left) 2024-09-16T15:04:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:925: write oid 22 current snap is 81 2024-09-16T15:04:50.136 INFO:tasks.rados.rados.0.smithi162.stdout:916: finishing write tid 3 to smithi16234619-39 2024-09-16T15:04:50.136 INFO:tasks.rados.rados.0.smithi162.stdout:925: seq_num 359 ranges {433713=482982,1424565=609148} 2024-09-16T15:04:50.138 INFO:tasks.rados.rados.0.smithi162.stdout:925: writing smithi16234619-22 from 433713 to 916695 tid 1 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:925: writing smithi16234619-22 from 1424565 to 2033713 tid 2 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:926: read oid 31 snap -1 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:926: expect deleted 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:927: read oid 3 snap -1 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:927: expect deleted 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:928: write (excl) oid 12 current snap is 81 2024-09-16T15:04:50.146 INFO:tasks.rados.rados.0.smithi162.stdout:916: finishing write tid 4 to smithi16234619-39 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:916: finishing write tid 5 to smithi16234619-39 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:916: oid 39 updating version 0 to 379 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:916: oid 39 version 379 is already newer than 376 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:916: oid 39 updating version 379 to 380 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:916: oid 39 version 380 is already newer than 378 2024-09-16T15:04:50.147 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 380 (ObjNum 356 snap 81 seq_num 356) dirty exists 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:916: left oid 39 (ObjNum 356 snap 81 seq_num 356) 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:924: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:920: finishing copy_from to smithi16234619-19 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 80 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:922: finishing write tid 1 to smithi16234619-50 2024-09-16T15:04:50.148 INFO:tasks.rados.rados.0.smithi162.stdout:928: seq_num 360 ranges {450520=558231,1483047=515125} 2024-09-16T15:04:50.149 INFO:tasks.rados.rados.0.smithi162.stdout:928: writing smithi16234619-12 from 450520 to 1008751 tid 1 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:928: writing smithi16234619-12 from 1483047 to 1998172 tid 2 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:916: done (7 left) 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:920: done (6 left) 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:929: delete oid 19 current snap is 81 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:922: finishing write tid 2 to smithi16234619-50 2024-09-16T15:04:50.156 INFO:tasks.rados.rados.0.smithi162.stdout:922: finishing write tid 3 to smithi16234619-50 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:922: finishing write tid 4 to smithi16234619-50 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:922: oid 50 updating version 0 to 316 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:922: oid 50 version 316 is already newer than 314 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:922: oid 50 version 316 is already newer than 315 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 316 (ObjNum 358 snap 81 seq_num 358) dirty exists 2024-09-16T15:04:50.157 INFO:tasks.rados.rados.0.smithi162.stdout:922: left oid 50 (ObjNum 358 snap 81 seq_num 358) 2024-09-16T15:04:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:924: finishing copy_from to smithi16234619-25 2024-09-16T15:04:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 246 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:04:50.159 INFO:tasks.rados.rados.0.smithi162.stdout:922: done (6 left) 2024-09-16T15:04:50.159 INFO:tasks.rados.rados.0.smithi162.stdout:924: done (5 left) 2024-09-16T15:04:50.159 INFO:tasks.rados.rados.0.smithi162.stdout:926: done (4 left) 2024-09-16T15:04:50.159 INFO:tasks.rados.rados.0.smithi162.stdout:927: done (3 left) 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:929: done (2 left) 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:930: read oid 2 snap -1 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:930: expect deleted 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:931: read oid 45 snap -1 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:931: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:04:50.160 INFO:tasks.rados.rados.0.smithi162.stdout:932: read oid 19 snap -1 2024-09-16T15:04:50.161 INFO:tasks.rados.rados.0.smithi162.stdout:932: expect deleted 2024-09-16T15:04:50.161 INFO:tasks.rados.rados.0.smithi162.stdout:933: snap_create 2024-09-16T15:04:50.161 INFO:tasks.rados.rados.0.smithi162.stdout:931: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:04:50.268 INFO:tasks.rados.rados.0.smithi162.stdout:925: finishing write tid 1 to smithi16234619-22 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: finishing write tid 2 to smithi16234619-22 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: finishing write tid 3 to smithi16234619-22 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: finishing write tid 4 to smithi16234619-22 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: oid 22 updating version 0 to 269 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: oid 22 version 269 is already newer than 268 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: oid 22 updating version 269 to 270 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 270 (ObjNum 359 snap 81 seq_num 359) dirty exists 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:925: left oid 22 (ObjNum 359 snap 81 seq_num 359) 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:928: finishing write tid 1 to smithi16234619-12 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:928: finishing write tid 2 to smithi16234619-12 2024-09-16T15:04:50.269 INFO:tasks.rados.rados.0.smithi162.stdout:928: finishing write tid 3 to smithi16234619-12 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:928: finishing write tid 4 to smithi16234619-12 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:928: oid 12 updating version 0 to 191 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:928: oid 12 version 191 is already newer than 190 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:928: oid 12 version 191 is already newer than 189 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 191 (ObjNum 360 snap 81 seq_num 360) dirty exists 2024-09-16T15:04:50.270 INFO:tasks.rados.rados.0.smithi162.stdout:928: left oid 12 (ObjNum 360 snap 81 seq_num 360) 2024-09-16T15:04:50.391 INFO:tasks.rados.rados.0.smithi162.stdout:925: done (5 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:928: done (4 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:930: done (3 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:931: done (2 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:932: done (1 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:933: done (0 left) 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:934: read oid 3 snap 81 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:934: expect deleted 2024-09-16T15:04:50.392 INFO:tasks.rados.rados.0.smithi162.stdout:935: delete oid 27 current snap is 82 2024-09-16T15:04:50.396 INFO:tasks.rados.rados.0.smithi162.stdout:934: done (1 left) 2024-09-16T15:04:50.396 INFO:tasks.rados.rados.0.smithi162.stdout:935: done (0 left) 2024-09-16T15:04:50.396 INFO:tasks.rados.rados.0.smithi162.stdout:936: delete oid 11 current snap is 82 2024-09-16T15:04:50.401 INFO:tasks.rados.rados.0.smithi162.stdout:936: done (0 left) 2024-09-16T15:04:50.402 INFO:tasks.rados.rados.0.smithi162.stdout:937: read oid 48 snap 81 2024-09-16T15:04:50.402 INFO:tasks.rados.rados.0.smithi162.stdout:937: expect (ObjNum 343 snap 80 seq_num 343) 2024-09-16T15:04:50.402 INFO:tasks.rados.rados.0.smithi162.stdout:938: delete oid 42 current snap is 82 2024-09-16T15:04:50.404 INFO:tasks.rados.rados.0.smithi162.stdout:938: done (1 left) 2024-09-16T15:04:50.404 INFO:tasks.rados.rados.0.smithi162.stdout:939: rollback oid 19 current snap is 82 2024-09-16T15:04:50.404 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 81 2024-09-16T15:04:50.404 INFO:tasks.rados.rados.0.smithi162.stdout:940: write oid 9 current snap is 82 2024-09-16T15:04:50.404 INFO:tasks.rados.rados.0.smithi162.stdout:940: seq_num 365 ranges {534555=650829,1659104=475451} 2024-09-16T15:04:50.410 INFO:tasks.rados.rados.0.smithi162.stdout:940: writing smithi16234619-9 from 534555 to 1185384 tid 1 2024-09-16T15:04:50.416 INFO:tasks.rados.rados.0.smithi162.stdout:940: writing smithi16234619-9 from 1659104 to 2134555 tid 2 2024-09-16T15:04:50.416 INFO:tasks.rados.rados.0.smithi162.stdout:941: rollback oid 20 current snap is 82 2024-09-16T15:04:50.416 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 81 2024-09-16T15:04:50.416 INFO:tasks.rados.rados.0.smithi162.stdout:937: expect (ObjNum 343 snap 80 seq_num 343) 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:939: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 0 (ObjNum 13633 snap 0 seq_num 2027922400) dirty dne 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:941: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:941: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:941: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 172 (ObjNum 330 snap 77 seq_num 330) dirty exists 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:940: finishing write tid 1 to smithi16234619-9 2024-09-16T15:04:50.536 INFO:tasks.rados.rados.0.smithi162.stdout:937: done (3 left) 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:939: done (2 left) 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:941: done (1 left) 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:942: read oid 28 snap -1 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:942: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:943: copy_from oid 10 from oid 5 current snap is 82 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:940: finishing write tid 2 to smithi16234619-9 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:940: finishing write tid 3 to smithi16234619-9 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:940: finishing write tid 4 to smithi16234619-9 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:940: oid 9 updating version 0 to 320 2024-09-16T15:04:50.537 INFO:tasks.rados.rados.0.smithi162.stdout:940: oid 9 version 320 is already newer than 318 2024-09-16T15:04:50.540 INFO:tasks.rados.rados.0.smithi162.stdout:940: oid 9 version 320 is already newer than 319 2024-09-16T15:04:50.540 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 320 (ObjNum 365 snap 82 seq_num 365) dirty exists 2024-09-16T15:04:50.540 INFO:tasks.rados.rados.0.smithi162.stdout:940: left oid 9 (ObjNum 365 snap 82 seq_num 365) 2024-09-16T15:04:50.540 INFO:tasks.rados.rados.0.smithi162.stdout:940: done (2 left) 2024-09-16T15:04:50.540 INFO:tasks.rados.rados.0.smithi162.stdout:944: snap_remove snap 81 2024-09-16T15:04:51.712 INFO:tasks.rados.rados.0.smithi162.stdout:944: done (2 left) 2024-09-16T15:04:51.712 INFO:tasks.rados.rados.0.smithi162.stdout:945: write (excl) oid 50 current snap is 82 2024-09-16T15:04:51.712 INFO:tasks.rados.rados.0.smithi162.stdout:945: seq_num 366 ranges {551362=726078,1717586=409779} 2024-09-16T15:04:51.720 INFO:tasks.rados.rados.0.smithi162.stdout:945: writing smithi16234619-50 from 551362 to 1277440 tid 1 2024-09-16T15:04:51.725 INFO:tasks.rados.rados.0.smithi162.stdout:945: writing smithi16234619-50 from 1717586 to 2127365 tid 2 2024-09-16T15:04:51.725 INFO:tasks.rados.rados.0.smithi162.stdout:946: delete oid 17 current snap is 82 2024-09-16T15:04:51.725 INFO:tasks.rados.rados.0.smithi162.stdout:942: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:04:51.816 INFO:tasks.rados.rados.0.smithi162.stdout:943: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:04:51.816 INFO:tasks.rados.rados.0.smithi162.stdout:943: finishing copy_from to smithi16234619-10 2024-09-16T15:04:51.816 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 244 (ObjNum 216 snap 44 seq_num 216) dirty exists 2024-09-16T15:04:51.816 INFO:tasks.rados.rados.0.smithi162.stdout:945: finishing write tid 1 to smithi16234619-50 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: finishing write tid 2 to smithi16234619-50 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: finishing write tid 3 to smithi16234619-50 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: finishing write tid 4 to smithi16234619-50 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: oid 50 updating version 0 to 320 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: oid 50 updating version 320 to 321 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: oid 50 updating version 321 to 322 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 322 (ObjNum 366 snap 82 seq_num 366) dirty exists 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: left oid 50 (ObjNum 366 snap 82 seq_num 366) 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:942: done (3 left) 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:943: done (2 left) 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:945: done (1 left) 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:946: done (0 left) 2024-09-16T15:04:51.817 INFO:tasks.rados.rados.0.smithi162.stdout:947: read oid 36 snap -1 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:947: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:948: read oid 14 snap -1 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:948: expect deleted 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:949: read oid 19 snap -1 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:949: expect deleted 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:950: copy_from oid 30 from oid 32 current snap is 82 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:951: write (excl) oid 11 current snap is 82 2024-09-16T15:04:51.818 INFO:tasks.rados.rados.0.smithi162.stdout:951: seq_num 368 ranges {584976=476576,1834550=350426} 2024-09-16T15:04:51.821 INFO:tasks.rados.rados.0.smithi162.stdout:951: writing smithi16234619-11 from 584976 to 1061552 tid 1 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:951: writing smithi16234619-11 from 1834550 to 2184976 tid 2 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:952: read oid 26 snap -1 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:952: expect (ObjNum 276 snap 58 seq_num 276) 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:953: read oid 21 snap -1 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:953: expect (ObjNum 345 snap 81 seq_num 345) 2024-09-16T15:04:51.828 INFO:tasks.rados.rados.0.smithi162.stdout:954: delete oid 50 current snap is 82 2024-09-16T15:04:51.829 INFO:tasks.rados.rados.0.smithi162.stdout:950: finishing copy_from to smithi16234619-30 2024-09-16T15:04:51.829 INFO:tasks.rados.rados.0.smithi162.stdout:950: got expected ENOENT (src dne) 2024-09-16T15:04:51.829 INFO:tasks.rados.rados.0.smithi162.stdout:950: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:04:51.829 INFO:tasks.rados.rados.0.smithi162.stdout:947: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:04:51.916 INFO:tasks.rados.rados.0.smithi162.stdout:952: expect (ObjNum 276 snap 58 seq_num 276) 2024-09-16T15:04:51.999 INFO:tasks.rados.rados.0.smithi162.stdout:953: expect (ObjNum 345 snap 81 seq_num 345) 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: finishing write tid 1 to smithi16234619-11 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: finishing write tid 2 to smithi16234619-11 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: finishing write tid 3 to smithi16234619-11 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: finishing write tid 4 to smithi16234619-11 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: oid 11 updating version 0 to 51 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: oid 11 version 51 is already newer than 50 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: oid 11 updating version 51 to 52 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 52 (ObjNum 368 snap 82 seq_num 368) dirty exists 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:951: left oid 11 (ObjNum 368 snap 82 seq_num 368) 2024-09-16T15:04:52.243 INFO:tasks.rados.rados.0.smithi162.stdout:947: done (7 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:948: done (6 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:949: done (5 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:950: done (4 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:951: done (3 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:952: done (2 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:953: done (1 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:954: done (0 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:955: read oid 42 snap -1 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:955: expect deleted 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:956: rollback oid 40 current snap is 82 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:956: done (1 left) 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:957: write oid 9 current snap is 82 2024-09-16T15:04:52.244 INFO:tasks.rados.rados.0.smithi162.stdout:957: seq_num 370 ranges {618590=627074,1835161=383429} 2024-09-16T15:04:52.248 INFO:tasks.rados.rados.0.smithi162.stdout:957: writing smithi16234619-9 from 618590 to 1245664 tid 1 2024-09-16T15:04:52.254 INFO:tasks.rados.rados.0.smithi162.stdout:957: writing smithi16234619-9 from 1835161 to 2218590 tid 2 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:958: read oid 29 snap -1 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:958: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:959: rollback oid 49 current snap is 82 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:959: done (3 left) 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:960: copy_from oid 45 from oid 8 current snap is 82 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:955: done (3 left) 2024-09-16T15:04:52.255 INFO:tasks.rados.rados.0.smithi162.stdout:961: delete oid 6 current snap is 82 2024-09-16T15:04:52.261 INFO:tasks.rados.rados.0.smithi162.stdout:960: finishing copy_from to smithi16234619-45 2024-09-16T15:04:52.261 INFO:tasks.rados.rados.0.smithi162.stdout:960: got expected ENOENT (src dne) 2024-09-16T15:04:52.261 INFO:tasks.rados.rados.0.smithi162.stdout:960: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:04:52.262 INFO:tasks.rados.rados.0.smithi162.stdout:960: done (3 left) 2024-09-16T15:04:52.262 INFO:tasks.rados.rados.0.smithi162.stdout:961: done (2 left) 2024-09-16T15:04:52.262 INFO:tasks.rados.rados.0.smithi162.stdout:962: read oid 28 snap -1 2024-09-16T15:04:52.262 INFO:tasks.rados.rados.0.smithi162.stdout:962: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:04:52.262 INFO:tasks.rados.rados.0.smithi162.stdout:963: snap_create 2024-09-16T15:04:52.263 INFO:tasks.rados.rados.0.smithi162.stdout:958: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:04:52.442 INFO:tasks.rados.rados.0.smithi162.stdout:962: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:04:52.531 INFO:tasks.rados.rados.0.smithi162.stdout:957: finishing write tid 1 to smithi16234619-9 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: finishing write tid 2 to smithi16234619-9 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: finishing write tid 3 to smithi16234619-9 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: finishing write tid 4 to smithi16234619-9 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: oid 9 updating version 0 to 327 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: oid 9 version 327 is already newer than 325 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: oid 9 version 327 is already newer than 326 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 327 (ObjNum 370 snap 82 seq_num 370) dirty exists 2024-09-16T15:04:52.532 INFO:tasks.rados.rados.0.smithi162.stdout:957: left oid 9 (ObjNum 370 snap 82 seq_num 370) 2024-09-16T15:04:52.717 INFO:tasks.rados.rados.0.smithi162.stdout:957: done (3 left) 2024-09-16T15:04:52.718 INFO:tasks.rados.rados.0.smithi162.stdout:958: done (2 left) 2024-09-16T15:04:52.718 INFO:tasks.rados.rados.0.smithi162.stdout:962: done (1 left) 2024-09-16T15:04:52.718 INFO:tasks.rados.rados.0.smithi162.stdout:963: done (0 left) 2024-09-16T15:04:52.718 INFO:tasks.rados.rados.0.smithi162.stdout:964: snap_create 2024-09-16T15:04:53.719 INFO:tasks.rados.rados.0.smithi162.stdout:964: done (0 left) 2024-09-16T15:04:53.719 INFO:tasks.rados.rados.0.smithi162.stdout:965: read oid 19 snap 82 2024-09-16T15:04:53.719 INFO:tasks.rados.rados.0.smithi162.stdout:965: expect deleted 2024-09-16T15:04:53.720 INFO:tasks.rados.rados.0.smithi162.stdout:966: snap_create 2024-09-16T15:04:54.721 INFO:tasks.rados.rados.0.smithi162.stdout:965: done (1 left) 2024-09-16T15:04:54.721 INFO:tasks.rados.rados.0.smithi162.stdout:966: done (0 left) 2024-09-16T15:04:54.721 INFO:tasks.rados.rados.0.smithi162.stdout:967: write oid 8 current snap is 85 2024-09-16T15:04:54.721 INFO:tasks.rados.rados.0.smithi162.stdout:967: seq_num 372 ranges {652204=777572,1952125=300079} 2024-09-16T15:04:54.728 INFO:tasks.rados.rados.0.smithi162.stdout:967: writing smithi16234619-8 from 652204 to 1429776 tid 1 2024-09-16T15:04:54.732 INFO:tasks.rados.rados.0.smithi162.stdout:967: writing smithi16234619-8 from 1952125 to 2252204 tid 2 2024-09-16T15:04:54.732 INFO:tasks.rados.rados.0.smithi162.stdout:968: snap_create 2024-09-16T15:04:54.757 INFO:tasks.rados.rados.0.smithi162.stdout:967: finishing write tid 1 to smithi16234619-8 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: finishing write tid 2 to smithi16234619-8 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: finishing write tid 3 to smithi16234619-8 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: finishing write tid 4 to smithi16234619-8 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: oid 8 updating version 0 to 239 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: oid 8 updating version 239 to 240 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: oid 8 updating version 240 to 241 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 241 (ObjNum 372 snap 85 seq_num 372) dirty exists 2024-09-16T15:04:54.760 INFO:tasks.rados.rados.0.smithi162.stdout:967: left oid 8 (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:04:55.512 INFO:tasks.rados.rados.0.smithi162.stdout:967: done (1 left) 2024-09-16T15:04:55.512 INFO:tasks.rados.rados.0.smithi162.stdout:968: done (0 left) 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:969: rollback oid 21 current snap is 86 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 83 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:970: read oid 14 snap -1 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:970: expect deleted 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:971: read oid 48 snap -1 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:971: expect (ObjNum 343 snap 80 seq_num 343) 2024-09-16T15:04:55.513 INFO:tasks.rados.rados.0.smithi162.stdout:972: snap_create 2024-09-16T15:04:55.519 INFO:tasks.rados.rados.0.smithi162.stdout:971: expect (ObjNum 343 snap 80 seq_num 343) 2024-09-16T15:04:55.636 INFO:tasks.rados.rados.0.smithi162.stdout:969: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:04:55.637 INFO:tasks.rados.rados.0.smithi162.stdout:969: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:04:55.637 INFO:tasks.rados.rados.0.smithi162.stdout:969: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:04:55.637 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 319 (ObjNum 345 snap 81 seq_num 345) dirty exists 2024-09-16T15:04:56.515 INFO:tasks.rados.rados.0.smithi162.stdout:969: done (3 left) 2024-09-16T15:04:56.515 INFO:tasks.rados.rados.0.smithi162.stdout:970: done (2 left) 2024-09-16T15:04:56.515 INFO:tasks.rados.rados.0.smithi162.stdout:971: done (1 left) 2024-09-16T15:04:56.515 INFO:tasks.rados.rados.0.smithi162.stdout:972: done (0 left) 2024-09-16T15:04:56.515 INFO:tasks.rados.rados.0.smithi162.stdout:973: delete oid 42 current snap is 87 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:973: done (0 left) 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:974: copy_from oid 1 from oid 17 current snap is 87 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:975: read oid 5 snap 83 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:975: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:976: write oid 10 current snap is 87 2024-09-16T15:04:56.518 INFO:tasks.rados.rados.0.smithi162.stdout:976: seq_num 374 ranges {685818=644423,2069089=216729} 2024-09-16T15:04:56.525 INFO:tasks.rados.rados.0.smithi162.stdout:976: writing smithi16234619-10 from 685818 to 1330241 tid 1 2024-09-16T15:04:56.531 INFO:tasks.rados.rados.0.smithi162.stdout:976: writing smithi16234619-10 from 2069089 to 2285818 tid 2 2024-09-16T15:04:56.531 INFO:tasks.rados.rados.0.smithi162.stdout:977: copy_from oid 45 from oid 42 current snap is 87 2024-09-16T15:04:56.531 INFO:tasks.rados.rados.0.smithi162.stdout:974: finishing copy_from to smithi16234619-1 2024-09-16T15:04:56.531 INFO:tasks.rados.rados.0.smithi162.stdout:974: got expected ENOENT (src dne) 2024-09-16T15:04:56.531 INFO:tasks.rados.rados.0.smithi162.stdout:974: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:04:56.532 INFO:tasks.rados.rados.0.smithi162.stdout:975: expect (ObjNum 216 snap 44 seq_num 216) 2024-09-16T15:04:56.730 INFO:tasks.rados.rados.0.smithi162.stdout:976: finishing write tid 1 to smithi16234619-10 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: finishing write tid 2 to smithi16234619-10 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:977: finishing copy_from to smithi16234619-45 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:977: got expected ENOENT (src dne) 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:977: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: finishing write tid 3 to smithi16234619-10 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: finishing write tid 4 to smithi16234619-10 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: oid 10 updating version 0 to 321 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: oid 10 updating version 321 to 322 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: oid 10 updating version 322 to 323 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 323 (ObjNum 374 snap 87 seq_num 374) dirty exists 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:976: left oid 10 (ObjNum 374 snap 87 seq_num 374) 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:974: done (3 left) 2024-09-16T15:04:56.731 INFO:tasks.rados.rados.0.smithi162.stdout:975: done (2 left) 2024-09-16T15:04:56.732 INFO:tasks.rados.rados.0.smithi162.stdout:976: done (1 left) 2024-09-16T15:04:56.732 INFO:tasks.rados.rados.0.smithi162.stdout:977: done (0 left) 2024-09-16T15:04:56.732 INFO:tasks.rados.rados.0.smithi162.stdout:978: write oid 26 current snap is 87 2024-09-16T15:04:56.732 INFO:tasks.rados.rados.0.smithi162.stdout:978: seq_num 375 ranges {702625=719672,2127571=175054} 2024-09-16T15:04:56.738 INFO:tasks.rados.rados.0.smithi162.stdout:978: writing smithi16234619-26 from 702625 to 1422297 tid 1 2024-09-16T15:04:56.742 INFO:tasks.rados.rados.0.smithi162.stdout:978: writing smithi16234619-26 from 2127571 to 2302625 tid 2 2024-09-16T15:04:56.743 INFO:tasks.rados.rados.0.smithi162.stdout:979: write oid 41 current snap is 87 2024-09-16T15:04:56.743 INFO:tasks.rados.rados.0.smithi162.stdout:979: seq_num 376 ranges {719432=794921,2186053=133379} 2024-09-16T15:04:56.751 INFO:tasks.rados.rados.0.smithi162.stdout:979: writing smithi16234619-41 from 719432 to 1514353 tid 1 2024-09-16T15:04:56.755 INFO:tasks.rados.rados.0.smithi162.stdout:979: writing smithi16234619-41 from 2186053 to 2319432 tid 2 2024-09-16T15:04:56.756 INFO:tasks.rados.rados.0.smithi162.stdout:980: rollback oid 35 current snap is 87 2024-09-16T15:04:56.756 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 85 2024-09-16T15:04:56.756 INFO:tasks.rados.rados.0.smithi162.stdout:981: snap_create 2024-09-16T15:04:56.759 INFO:tasks.rados.rados.0.smithi162.stdout:980: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:04:56.760 INFO:tasks.rados.rados.0.smithi162.stdout:980: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:04:56.761 INFO:tasks.rados.rados.0.smithi162.stdout:980: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:04:56.761 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 284 (ObjNum 348 snap 81 seq_num 348) dirty exists 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: finishing write tid 1 to smithi16234619-26 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: finishing write tid 2 to smithi16234619-26 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: finishing write tid 3 to smithi16234619-26 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: finishing write tid 4 to smithi16234619-26 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: oid 26 updating version 0 to 325 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: oid 26 updating version 325 to 327 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: oid 26 version 327 is already newer than 326 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 327 (ObjNum 375 snap 87 seq_num 375) dirty exists 2024-09-16T15:04:56.763 INFO:tasks.rados.rados.0.smithi162.stdout:978: left oid 26 (ObjNum 375 snap 87 seq_num 375) 2024-09-16T15:04:56.778 INFO:tasks.rados.rados.0.smithi162.stdout:979: finishing write tid 1 to smithi16234619-41 2024-09-16T15:04:56.778 INFO:tasks.rados.rados.0.smithi162.stdout:979: finishing write tid 2 to smithi16234619-41 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: finishing write tid 3 to smithi16234619-41 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: finishing write tid 4 to smithi16234619-41 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: oid 41 updating version 0 to 271 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: oid 41 updating version 271 to 273 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: oid 41 version 273 is already newer than 272 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 273 (ObjNum 376 snap 87 seq_num 376) dirty exists 2024-09-16T15:04:56.779 INFO:tasks.rados.rados.0.smithi162.stdout:979: left oid 41 (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:04:57.519 INFO:tasks.rados.rados.0.smithi162.stdout:978: done (3 left) 2024-09-16T15:04:57.520 INFO:tasks.rados.rados.0.smithi162.stdout:979: done (2 left) 2024-09-16T15:04:57.520 INFO:tasks.rados.rados.0.smithi162.stdout:980: done (1 left) 2024-09-16T15:04:57.520 INFO:tasks.rados.rados.0.smithi162.stdout:981: done (0 left) 2024-09-16T15:04:57.520 INFO:tasks.rados.rados.0.smithi162.stdout:982: snap_remove snap 85 2024-09-16T15:04:58.523 INFO:tasks.rados.rados.0.smithi162.stdout:982: done (0 left) 2024-09-16T15:04:58.523 INFO:tasks.rados.rados.0.smithi162.stdout:983: snap_create 2024-09-16T15:04:59.525 INFO:tasks.rados.rados.0.smithi162.stdout:983: done (0 left) 2024-09-16T15:04:59.525 INFO:tasks.rados.rados.0.smithi162.stdout:984: write oid 20 current snap is 89 2024-09-16T15:04:59.525 INFO:tasks.rados.rados.0.smithi162.stdout:984: seq_num 377 ranges {736239=470170,1728182=571782} 2024-09-16T15:04:59.531 INFO:tasks.rados.rados.0.smithi162.stdout:984: writing smithi16234619-20 from 736239 to 1206409 tid 1 2024-09-16T15:04:59.537 INFO:tasks.rados.rados.0.smithi162.stdout:984: writing smithi16234619-20 from 1728182 to 2299964 tid 2 2024-09-16T15:04:59.537 INFO:tasks.rados.rados.0.smithi162.stdout:985: rollback oid 49 current snap is 89 2024-09-16T15:04:59.537 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 82 2024-09-16T15:04:59.537 INFO:tasks.rados.rados.0.smithi162.stdout:986: snap_remove snap 83 2024-09-16T15:05:00.391 INFO:tasks.rados.rados.0.smithi162.stdout:986: done (2 left) 2024-09-16T15:05:00.391 INFO:tasks.rados.rados.0.smithi162.stdout:987: snap_remove snap 87 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:987: done (2 left) 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:988: delete oid 30 current snap is 89 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:985: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: finishing write tid 1 to smithi16234619-20 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: finishing write tid 2 to smithi16234619-20 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: finishing write tid 3 to smithi16234619-20 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: finishing write tid 4 to smithi16234619-20 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: oid 20 updating version 0 to 175 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: oid 20 version 175 is already newer than 174 2024-09-16T15:05:01.395 INFO:tasks.rados.rados.0.smithi162.stdout:984: oid 20 version 175 is already newer than 173 2024-09-16T15:05:01.396 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 175 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:05:01.396 INFO:tasks.rados.rados.0.smithi162.stdout:984: left oid 20 (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:05:01.403 INFO:tasks.rados.rados.0.smithi162.stdout:984: done (2 left) 2024-09-16T15:05:01.403 INFO:tasks.rados.rados.0.smithi162.stdout:985: done (1 left) 2024-09-16T15:05:01.403 INFO:tasks.rados.rados.0.smithi162.stdout:988: done (0 left) 2024-09-16T15:05:01.403 INFO:tasks.rados.rados.0.smithi162.stdout:989: copy_from oid 9 from oid 14 current snap is 89 2024-09-16T15:05:01.403 INFO:tasks.rados.rados.0.smithi162.stdout:990: delete oid 3 current snap is 89 2024-09-16T15:05:01.404 INFO:tasks.rados.rados.0.smithi162.stdout:989: finishing copy_from to smithi16234619-9 2024-09-16T15:05:01.404 INFO:tasks.rados.rados.0.smithi162.stdout:989: got expected ENOENT (src dne) 2024-09-16T15:05:01.404 INFO:tasks.rados.rados.0.smithi162.stdout:989: finishing copy_from racing read to smithi16234619-9 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:989: done (1 left) 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:990: done (0 left) 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:991: read oid 30 snap -1 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:991: expect deleted 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:992: rollback oid 13 current snap is 89 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 84 2024-09-16T15:05:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:993: snap_create 2024-09-16T15:05:01.416 INFO:tasks.rados.rados.0.smithi162.stdout:992: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:05:01.416 INFO:tasks.rados.rados.0.smithi162.stdout:992: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:05:01.416 INFO:tasks.rados.rados.0.smithi162.stdout:992: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:05:01.416 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 389 (ObjNum 314 snap 67 seq_num 314) dirty exists 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:991: done (2 left) 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:992: done (1 left) 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:993: done (0 left) 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:994: read oid 12 snap -1 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:994: expect (ObjNum 360 snap 81 seq_num 360) 2024-09-16T15:05:02.403 INFO:tasks.rados.rados.0.smithi162.stdout:995: snap_create 2024-09-16T15:05:02.410 INFO:tasks.rados.rados.0.smithi162.stdout:994: expect (ObjNum 360 snap 81 seq_num 360) 2024-09-16T15:05:03.404 INFO:tasks.rados.rados.0.smithi162.stdout:994: done (1 left) 2024-09-16T15:05:03.404 INFO:tasks.rados.rados.0.smithi162.stdout:995: done (0 left) 2024-09-16T15:05:03.404 INFO:tasks.rados.rados.0.smithi162.stdout:996: delete oid 46 current snap is 91 2024-09-16T15:05:03.407 INFO:tasks.rados.rados.0.smithi162.stdout:996: done (0 left) 2024-09-16T15:05:03.407 INFO:tasks.rados.rados.0.smithi162.stdout:997: write oid 17 current snap is 91 2024-09-16T15:05:03.407 INFO:tasks.rados.rados.0.smithi162.stdout:997: seq_num 381 ranges {403467=487519,1562110=579120} 2024-09-16T15:05:03.412 INFO:tasks.rados.rados.0.smithi162.stdout:997: writing smithi16234619-17 from 403467 to 890986 tid 1 2024-09-16T15:05:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:997: writing smithi16234619-17 from 1562110 to 2141230 tid 2 2024-09-16T15:05:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:998: read oid 6 snap 89 2024-09-16T15:05:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:998: expect deleted 2024-09-16T15:05:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:999: delete oid 34 current snap is 91 2024-09-16T15:05:03.423 INFO:tasks.rados.rados.0.smithi162.stdout:998: done (2 left) 2024-09-16T15:05:03.423 INFO:tasks.rados.rados.0.smithi162.stdout:999: done (1 left) 2024-09-16T15:05:03.423 INFO:tasks.rados.rados.0.smithi162.stdout:1000: read oid 21 snap 86 2024-09-16T15:05:03.423 INFO:tasks.rados.rados.0.smithi162.stdout:1000: expect (ObjNum 345 snap 81 seq_num 345) 2024-09-16T15:05:03.423 INFO:tasks.rados.rados.0.smithi162.stdout:1001: delete oid 30 current snap is 91 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:1001: done (2 left) 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:1002: copy_from oid 12 from oid 11 current snap is 91 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:1003: rollback oid 4 current snap is 91 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 86 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:1004: write oid 10 current snap is 91 2024-09-16T15:05:03.425 INFO:tasks.rados.rados.0.smithi162.stdout:1004: seq_num 384 ranges {453888=713266,1737556=442800} 2024-09-16T15:05:03.431 INFO:tasks.rados.rados.0.smithi162.stdout:1004: writing smithi16234619-10 from 453888 to 1167154 tid 1 2024-09-16T15:05:03.437 INFO:tasks.rados.rados.0.smithi162.stdout:1004: writing smithi16234619-10 from 1737556 to 2180356 tid 2 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1005: copy_from oid 22 from oid 41 current snap is 91 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1003: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1003: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1003: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 331 (ObjNum 353 snap 81 seq_num 353) dirty exists 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1003: done (5 left) 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1006: read oid 8 snap 86 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1006: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1007: read oid 31 snap -1 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1007: expect deleted 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1008: read oid 15 snap -1 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1008: expect (ObjNum 349 snap 81 seq_num 349) 2024-09-16T15:05:03.438 INFO:tasks.rados.rados.0.smithi162.stdout:1009: rollback oid 49 current snap is 91 2024-09-16T15:05:03.439 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 84 2024-09-16T15:05:03.439 INFO:tasks.rados.rados.0.smithi162.stdout:1010: write (excl) oid 39 current snap is 91 2024-09-16T15:05:03.439 INFO:tasks.rados.rados.0.smithi162.stdout:1010: seq_num 385 ranges {470695=788515,1679685=586458} 2024-09-16T15:05:03.444 INFO:tasks.rados.rados.0.smithi162.stdout:1010: writing smithi16234619-39 from 470695 to 1259210 tid 1 2024-09-16T15:05:03.450 INFO:tasks.rados.rados.0.smithi162.stdout:1010: writing smithi16234619-39 from 1679685 to 2266143 tid 2 2024-09-16T15:05:03.450 INFO:tasks.rados.rados.0.smithi162.stdout:1011: write oid 42 current snap is 91 2024-09-16T15:05:03.451 INFO:tasks.rados.rados.0.smithi162.stdout:1011: seq_num 386 ranges {487502=463764,1738167=446469} 2024-09-16T15:05:03.455 INFO:tasks.rados.rados.0.smithi162.stdout:1011: writing smithi16234619-42 from 487502 to 951266 tid 1 2024-09-16T15:05:03.462 INFO:tasks.rados.rados.0.smithi162.stdout:1011: writing smithi16234619-42 from 1738167 to 2184636 tid 2 2024-09-16T15:05:03.462 INFO:tasks.rados.rados.0.smithi162.stdout:1012: write oid 38 current snap is 91 2024-09-16T15:05:03.462 INFO:tasks.rados.rados.0.smithi162.stdout:1012: seq_num 387 ranges {504309=539013,1796649=590127} 2024-09-16T15:05:03.468 INFO:tasks.rados.rados.0.smithi162.stdout:1012: writing smithi16234619-38 from 504309 to 1043322 tid 1 2024-09-16T15:05:03.476 INFO:tasks.rados.rados.0.smithi162.stdout:1012: writing smithi16234619-38 from 1796649 to 2386776 tid 2 2024-09-16T15:05:03.476 INFO:tasks.rados.rados.0.smithi162.stdout:1013: write oid 2 current snap is 91 2024-09-16T15:05:03.476 INFO:tasks.rados.rados.0.smithi162.stdout:997: finishing write tid 1 to smithi16234619-17 2024-09-16T15:05:03.476 INFO:tasks.rados.rados.0.smithi162.stdout:997: finishing write tid 2 to smithi16234619-17 2024-09-16T15:05:03.476 INFO:tasks.rados.rados.0.smithi162.stdout:1000: expect (ObjNum 345 snap 81 seq_num 345) 2024-09-16T15:05:03.722 INFO:tasks.rados.rados.0.smithi162.stdout:1006: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:05:03.792 INFO:tasks.rados.rados.0.smithi162.stdout:1009: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:05:03.792 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 0 (ObjNum 65 snap 0 seq_num 1840189970) dirty dne 2024-09-16T15:05:03.793 INFO:tasks.rados.rados.0.smithi162.stdout:1008: expect (ObjNum 349 snap 81 seq_num 349) 2024-09-16T15:05:03.914 INFO:tasks.rados.rados.0.smithi162.stdout:1002: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:05:03.914 INFO:tasks.rados.rados.0.smithi162.stdout:1002: finishing copy_from to smithi16234619-12 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 52 (ObjNum 368 snap 82 seq_num 368) dirty exists 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: finishing write tid 3 to smithi16234619-17 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: finishing write tid 4 to smithi16234619-17 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: oid 17 updating version 0 to 341 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: oid 17 version 341 is already newer than 340 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: oid 17 updating version 341 to 342 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 342 (ObjNum 381 snap 91 seq_num 381) dirty exists 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:997: left oid 17 (ObjNum 381 snap 91 seq_num 381) 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:1005: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:1004: finishing write tid 1 to smithi16234619-10 2024-09-16T15:05:03.915 INFO:tasks.rados.rados.0.smithi162.stdout:1013: seq_num 388 ranges {521116=730615,1971484=549632} 2024-09-16T15:05:03.920 INFO:tasks.rados.rados.0.smithi162.stdout:1013: writing smithi16234619-2 from 521116 to 1251731 tid 1 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1013: writing smithi16234619-2 from 1971484 to 2521116 tid 2 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:997: done (12 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1000: done (11 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1002: done (10 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1006: done (9 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1007: done (8 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1008: done (7 left) 2024-09-16T15:05:03.928 INFO:tasks.rados.rados.0.smithi162.stdout:1009: done (6 left) 2024-09-16T15:05:03.929 INFO:tasks.rados.rados.0.smithi162.stdout:1014: snap_remove snap 90 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1014: done (6 left) 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1015: read oid 7 snap -1 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1015: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1016: rollback oid 4 current snap is 91 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 89 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1004: finishing write tid 2 to smithi16234619-10 2024-09-16T15:05:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:1004: finishing write tid 3 to smithi16234619-10 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1004: finishing write tid 4 to smithi16234619-10 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1004: oid 10 updating version 0 to 344 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1004: oid 10 updating version 344 to 346 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1004: oid 10 version 346 is already newer than 345 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 346 (ObjNum 384 snap 91 seq_num 384) dirty exists 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1004: left oid 10 (ObjNum 384 snap 91 seq_num 384) 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1005: finishing copy_from to smithi16234619-22 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 273 (ObjNum 376 snap 87 seq_num 376) dirty exists 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: finishing write tid 1 to smithi16234619-39 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: finishing write tid 2 to smithi16234619-39 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: finishing write tid 3 to smithi16234619-39 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1012: finishing write tid 1 to smithi16234619-38 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: finishing write tid 4 to smithi16234619-39 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: oid 39 updating version 0 to 391 2024-09-16T15:05:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:1010: oid 39 updating version 391 to 392 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1010: oid 39 updating version 392 to 393 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 393 (ObjNum 385 snap 91 seq_num 385) dirty exists 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1010: left oid 39 (ObjNum 385 snap 91 seq_num 385) 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1011: finishing write tid 1 to smithi16234619-42 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1012: finishing write tid 2 to smithi16234619-38 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1004: done (7 left) 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1005: done (6 left) 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1010: done (5 left) 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1017: write oid 30 current snap is 91 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1011: finishing write tid 2 to smithi16234619-42 2024-09-16T15:05:04.420 INFO:tasks.rados.rados.0.smithi162.stdout:1017: seq_num 389 ranges {537923=405864,1513613=593796} 2024-09-16T15:05:04.422 INFO:tasks.rados.rados.0.smithi162.stdout:1017: writing smithi16234619-30 from 537923 to 943787 tid 1 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1017: writing smithi16234619-30 from 1513613 to 2107409 tid 2 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1018: snap_create 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1011: finishing write tid 3 to smithi16234619-42 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1011: finishing write tid 4 to smithi16234619-42 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1011: oid 42 updating version 0 to 396 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1011: oid 42 version 396 is already newer than 395 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:1011: oid 42 version 396 is already newer than 394 2024-09-16T15:05:04.430 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 396 (ObjNum 386 snap 91 seq_num 386) dirty exists 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1011: left oid 42 (ObjNum 386 snap 91 seq_num 386) 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: finishing write tid 3 to smithi16234619-38 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: finishing write tid 4 to smithi16234619-38 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: oid 38 updating version 0 to 287 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: oid 38 version 287 is already newer than 285 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: oid 38 version 287 is already newer than 286 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 287 (ObjNum 387 snap 91 seq_num 387) dirty exists 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1012: left oid 38 (ObjNum 387 snap 91 seq_num 387) 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1013: finishing write tid 1 to smithi16234619-2 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1013: finishing write tid 2 to smithi16234619-2 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1013: finishing write tid 3 to smithi16234619-2 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1013: finishing write tid 4 to smithi16234619-2 2024-09-16T15:05:04.431 INFO:tasks.rados.rados.0.smithi162.stdout:1013: oid 2 updating version 0 to 348 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1013: oid 2 updating version 348 to 349 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1013: oid 2 version 349 is already newer than 347 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 349 (ObjNum 388 snap 91 seq_num 388) dirty exists 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1013: left oid 2 (ObjNum 388 snap 91 seq_num 388) 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1016: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1016: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:1016: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:05:04.432 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 336 (ObjNum 353 snap 81 seq_num 353) dirty exists 2024-09-16T15:05:04.434 INFO:tasks.rados.rados.0.smithi162.stdout:1015: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: finishing write tid 1 to smithi16234619-30 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: finishing write tid 2 to smithi16234619-30 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: finishing write tid 3 to smithi16234619-30 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: finishing write tid 4 to smithi16234619-30 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: oid 30 updating version 0 to 259 2024-09-16T15:05:04.678 INFO:tasks.rados.rados.0.smithi162.stdout:1017: oid 30 updating version 259 to 260 2024-09-16T15:05:04.679 INFO:tasks.rados.rados.0.smithi162.stdout:1017: oid 30 version 260 is already newer than 258 2024-09-16T15:05:04.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 260 (ObjNum 389 snap 91 seq_num 389) dirty exists 2024-09-16T15:05:04.679 INFO:tasks.rados.rados.0.smithi162.stdout:1017: left oid 30 (ObjNum 389 snap 91 seq_num 389) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1011: done (6 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1012: done (5 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1013: done (4 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1015: done (3 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1016: done (2 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1017: done (1 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1018: done (0 left) 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1019: read oid 26 snap -1 2024-09-16T15:05:05.396 INFO:tasks.rados.rados.0.smithi162.stdout:1019: expect (ObjNum 375 snap 87 seq_num 375) 2024-09-16T15:05:05.397 INFO:tasks.rados.rados.0.smithi162.stdout:1020: read oid 25 snap -1 2024-09-16T15:05:05.397 INFO:tasks.rados.rados.0.smithi162.stdout:1020: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:05:05.397 INFO:tasks.rados.rados.0.smithi162.stdout:1021: delete oid 3 current snap is 92 2024-09-16T15:05:05.405 INFO:tasks.rados.rados.0.smithi162.stdout:1021: done (2 left) 2024-09-16T15:05:05.405 INFO:tasks.rados.rados.0.smithi162.stdout:1022: delete oid 38 current snap is 92 2024-09-16T15:05:05.409 INFO:tasks.rados.rados.0.smithi162.stdout:1019: expect (ObjNum 375 snap 87 seq_num 375) 2024-09-16T15:05:05.512 INFO:tasks.rados.rados.0.smithi162.stdout:1020: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1019: done (2 left) 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1020: done (1 left) 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1022: done (0 left) 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1023: read oid 32 snap -1 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1023: expect deleted 2024-09-16T15:05:05.685 INFO:tasks.rados.rados.0.smithi162.stdout:1024: snap_create 2024-09-16T15:05:06.406 INFO:tasks.rados.rados.0.smithi162.stdout:1023: done (1 left) 2024-09-16T15:05:06.407 INFO:tasks.rados.rados.0.smithi162.stdout:1024: done (0 left) 2024-09-16T15:05:06.407 INFO:tasks.rados.rados.0.smithi162.stdout:1025: write oid 21 current snap is 93 2024-09-16T15:05:06.407 INFO:tasks.rados.rados.0.smithi162.stdout:1025: seq_num 392 ranges {588344=631611,1689059=741123} 2024-09-16T15:05:06.413 INFO:tasks.rados.rados.0.smithi162.stdout:1025: writing smithi16234619-21 from 588344 to 1219955 tid 1 2024-09-16T15:05:06.420 INFO:tasks.rados.rados.0.smithi162.stdout:1025: writing smithi16234619-21 from 1689059 to 2430182 tid 2 2024-09-16T15:05:06.420 INFO:tasks.rados.rados.0.smithi162.stdout:1026: delete oid 5 current snap is 93 2024-09-16T15:05:06.424 INFO:tasks.rados.rados.0.smithi162.stdout:1026: done (1 left) 2024-09-16T15:05:06.424 INFO:tasks.rados.rados.0.smithi162.stdout:1027: write (excl) oid 50 current snap is 93 2024-09-16T15:05:06.424 INFO:tasks.rados.rados.0.smithi162.stdout:1027: seq_num 394 ranges {621958=782109,2089670=532288} 2024-09-16T15:05:06.431 INFO:tasks.rados.rados.0.smithi162.stdout:1027: writing smithi16234619-50 from 621958 to 1404067 tid 1 2024-09-16T15:05:06.438 INFO:tasks.rados.rados.0.smithi162.stdout:1027: writing smithi16234619-50 from 2089670 to 2621958 tid 2 2024-09-16T15:05:06.438 INFO:tasks.rados.rados.0.smithi162.stdout:1028: write oid 46 current snap is 93 2024-09-16T15:05:06.438 INFO:tasks.rados.rados.0.smithi162.stdout:1028: seq_num 395 ranges {638765=457358,1748152=604803} 2024-09-16T15:05:06.444 INFO:tasks.rados.rados.0.smithi162.stdout:1028: writing smithi16234619-46 from 638765 to 1096123 tid 1 2024-09-16T15:05:06.451 INFO:tasks.rados.rados.0.smithi162.stdout:1028: writing smithi16234619-46 from 1748152 to 2352955 tid 2 2024-09-16T15:05:06.451 INFO:tasks.rados.rados.0.smithi162.stdout:1029: delete oid 40 current snap is 93 2024-09-16T15:05:06.451 INFO:tasks.rados.rados.0.smithi162.stdout:1025: finishing write tid 1 to smithi16234619-21 2024-09-16T15:05:06.452 INFO:tasks.rados.rados.0.smithi162.stdout:1025: finishing write tid 2 to smithi16234619-21 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: finishing write tid 3 to smithi16234619-21 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: finishing write tid 4 to smithi16234619-21 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: oid 21 updating version 0 to 354 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: oid 21 updating version 354 to 355 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: oid 21 version 355 is already newer than 353 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 355 (ObjNum 392 snap 93 seq_num 392) dirty exists 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: left oid 21 (ObjNum 392 snap 93 seq_num 392) 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1025: done (3 left) 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1029: done (2 left) 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1030: copy_from oid 31 from oid 28 current snap is 93 2024-09-16T15:05:06.455 INFO:tasks.rados.rados.0.smithi162.stdout:1031: snap_remove snap 89 2024-09-16T15:05:07.653 INFO:tasks.rados.rados.0.smithi162.stdout:1031: done (3 left) 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1032: delete oid 22 current snap is 93 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: finishing write tid 1 to smithi16234619-50 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: finishing write tid 2 to smithi16234619-50 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: finishing write tid 3 to smithi16234619-50 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: finishing write tid 4 to smithi16234619-50 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: oid 50 updating version 0 to 330 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: oid 50 version 330 is already newer than 329 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: oid 50 version 330 is already newer than 328 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 330 (ObjNum 394 snap 93 seq_num 394) dirty exists 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1027: left oid 50 (ObjNum 394 snap 93 seq_num 394) 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1028: finishing write tid 1 to smithi16234619-46 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1028: finishing write tid 2 to smithi16234619-46 2024-09-16T15:05:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1028: finishing write tid 3 to smithi16234619-46 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1028: finishing write tid 4 to smithi16234619-46 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1028: oid 46 updating version 0 to 97 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1028: oid 46 updating version 97 to 98 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1028: oid 46 updating version 98 to 99 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 99 (ObjNum 395 snap 93 seq_num 395) dirty exists 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1028: left oid 46 (ObjNum 395 snap 93 seq_num 395) 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1030: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:1030: finishing copy_from to smithi16234619-31 2024-09-16T15:05:07.655 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:05:07.664 INFO:tasks.rados.rados.0.smithi162.stdout:1027: done (3 left) 2024-09-16T15:05:07.664 INFO:tasks.rados.rados.0.smithi162.stdout:1028: done (2 left) 2024-09-16T15:05:07.664 INFO:tasks.rados.rados.0.smithi162.stdout:1030: done (1 left) 2024-09-16T15:05:07.664 INFO:tasks.rados.rados.0.smithi162.stdout:1032: done (0 left) 2024-09-16T15:05:07.664 INFO:tasks.rados.rados.0.smithi162.stdout:1033: snap_remove snap 86 2024-09-16T15:05:08.663 INFO:tasks.rados.rados.0.smithi162.stdout:1033: done (0 left) 2024-09-16T15:05:08.663 INFO:tasks.rados.rados.0.smithi162.stdout:1034: read oid 45 snap -1 2024-09-16T15:05:08.663 INFO:tasks.rados.rados.0.smithi162.stdout:1034: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:05:08.663 INFO:tasks.rados.rados.0.smithi162.stdout:1035: read oid 29 snap -1 2024-09-16T15:05:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:1035: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:05:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:1036: copy_from oid 8 from oid 49 current snap is 93 2024-09-16T15:05:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:1037: rollback oid 30 current snap is 93 2024-09-16T15:05:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 84 2024-09-16T15:05:08.664 INFO:tasks.rados.rados.0.smithi162.stdout:1038: rollback oid 43 current snap is 93 2024-09-16T15:05:08.665 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 84 2024-09-16T15:05:08.665 INFO:tasks.rados.rados.0.smithi162.stdout:1039: snap_create 2024-09-16T15:05:08.671 INFO:tasks.rados.rados.0.smithi162.stdout:1037: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:05:08.728 INFO:tasks.rados.rados.0.smithi162.stdout:1037: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:05:08.728 INFO:tasks.rados.rados.0.smithi162.stdout:1037: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:05:08.728 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 263 (ObjNum 275 snap 57 seq_num 275) dirty exists 2024-09-16T15:05:08.729 INFO:tasks.rados.rados.0.smithi162.stdout:1036: finishing copy_from to smithi16234619-8 2024-09-16T15:05:08.729 INFO:tasks.rados.rados.0.smithi162.stdout:1036: got expected ENOENT (src dne) 2024-09-16T15:05:08.729 INFO:tasks.rados.rados.0.smithi162.stdout:1036: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:05:08.729 INFO:tasks.rados.rados.0.smithi162.stdout:1038: finishing rollback tid 0 to smithi16234619-43 2024-09-16T15:05:08.729 INFO:tasks.rados.rados.0.smithi162.stdout:1034: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:05:08.784 INFO:tasks.rados.rados.0.smithi162.stdout:1035: expect (ObjNum 287 snap 60 seq_num 287) 2024-09-16T15:05:08.963 INFO:tasks.rados.rados.0.smithi162.stdout:1038: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:05:08.963 INFO:tasks.rados.rados.0.smithi162.stdout:1038: finishing rollback tid 2 to smithi16234619-43 2024-09-16T15:05:08.963 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 92 (ObjNum 306 snap 64 seq_num 306) dirty exists 2024-09-16T15:05:09.664 INFO:tasks.rados.rados.0.smithi162.stdout:1034: done (5 left) 2024-09-16T15:05:09.664 INFO:tasks.rados.rados.0.smithi162.stdout:1035: done (4 left) 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1036: done (3 left) 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1037: done (2 left) 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1038: done (1 left) 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1039: done (0 left) 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1040: write (excl) oid 19 current snap is 94 2024-09-16T15:05:09.665 INFO:tasks.rados.rados.0.smithi162.stdout:1040: seq_num 398 ranges {689186=799458,1923598=752130} 2024-09-16T15:05:09.672 INFO:tasks.rados.rados.0.smithi162.stdout:1040: writing smithi16234619-19 from 689186 to 1488644 tid 1 2024-09-16T15:05:09.679 INFO:tasks.rados.rados.0.smithi162.stdout:1040: writing smithi16234619-19 from 1923598 to 2675728 tid 2 2024-09-16T15:05:09.679 INFO:tasks.rados.rados.0.smithi162.stdout:1041: snap_remove snap 92 2024-09-16T15:05:10.393 INFO:tasks.rados.rados.0.smithi162.stdout:1041: done (1 left) 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1042: read oid 14 snap 91 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1042: expect deleted 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1043: delete oid 42 current snap is 94 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: finishing write tid 1 to smithi16234619-19 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: finishing write tid 2 to smithi16234619-19 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: finishing write tid 3 to smithi16234619-19 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: finishing write tid 4 to smithi16234619-19 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: oid 19 updating version 0 to 337 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: oid 19 version 337 is already newer than 336 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: oid 19 version 337 is already newer than 335 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 337 (ObjNum 398 snap 94 seq_num 398) dirty exists 2024-09-16T15:05:10.394 INFO:tasks.rados.rados.0.smithi162.stdout:1040: left oid 19 (ObjNum 398 snap 94 seq_num 398) 2024-09-16T15:05:10.399 INFO:tasks.rados.rados.0.smithi162.stdout:1040: done (2 left) 2024-09-16T15:05:10.399 INFO:tasks.rados.rados.0.smithi162.stdout:1042: done (1 left) 2024-09-16T15:05:10.399 INFO:tasks.rados.rados.0.smithi162.stdout:1043: done (0 left) 2024-09-16T15:05:10.400 INFO:tasks.rados.rados.0.smithi162.stdout:1044: snap_remove snap 91 2024-09-16T15:05:11.396 INFO:tasks.rados.rados.0.smithi162.stdout:1044: done (0 left) 2024-09-16T15:05:11.396 INFO:tasks.rados.rados.0.smithi162.stdout:1045: rollback oid 19 current snap is 94 2024-09-16T15:05:11.396 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 82 2024-09-16T15:05:11.396 INFO:tasks.rados.rados.0.smithi162.stdout:1046: delete oid 38 current snap is 94 2024-09-16T15:05:11.399 INFO:tasks.rados.rados.0.smithi162.stdout:1046: done (1 left) 2024-09-16T15:05:11.399 INFO:tasks.rados.rados.0.smithi162.stdout:1047: write (excl) oid 9 current snap is 94 2024-09-16T15:05:11.399 INFO:tasks.rados.rados.0.smithi162.stdout:1047: seq_num 401 ranges {739607=625205,1982691=499457} 2024-09-16T15:05:11.407 INFO:tasks.rados.rados.0.smithi162.stdout:1047: writing smithi16234619-9 from 739607 to 1364812 tid 1 2024-09-16T15:05:11.413 INFO:tasks.rados.rados.0.smithi162.stdout:1047: writing smithi16234619-9 from 1982691 to 2482148 tid 2 2024-09-16T15:05:11.414 INFO:tasks.rados.rados.0.smithi162.stdout:1048: delete oid 14 current snap is 94 2024-09-16T15:05:11.414 INFO:tasks.rados.rados.0.smithi162.stdout:1045: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:05:11.414 INFO:tasks.rados.rados.0.smithi162.stdout:1045: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:05:11.414 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 339 (ObjNum 33 snap 0 seq_num 862661616) dirty dne 2024-09-16T15:05:11.415 INFO:tasks.rados.rados.0.smithi162.stdout:1045: done (2 left) 2024-09-16T15:05:11.415 INFO:tasks.rados.rados.0.smithi162.stdout:1048: done (1 left) 2024-09-16T15:05:11.415 INFO:tasks.rados.rados.0.smithi162.stdout:1049: read oid 33 snap -1 2024-09-16T15:05:11.415 INFO:tasks.rados.rados.0.smithi162.stdout:1049: expect deleted 2024-09-16T15:05:11.415 INFO:tasks.rados.rados.0.smithi162.stdout:1050: snap_remove snap 88 2024-09-16T15:05:12.400 INFO:tasks.rados.rados.0.smithi162.stdout:1050: done (2 left) 2024-09-16T15:05:12.400 INFO:tasks.rados.rados.0.smithi162.stdout:1051: write oid 32 current snap is 94 2024-09-16T15:05:12.400 INFO:tasks.rados.rados.0.smithi162.stdout:1051: seq_num 403 ranges {773221=492056,1816008=503126} 2024-09-16T15:05:12.407 INFO:tasks.rados.rados.0.smithi162.stdout:1051: writing smithi16234619-32 from 773221 to 1265277 tid 1 2024-09-16T15:05:12.413 INFO:tasks.rados.rados.0.smithi162.stdout:1051: writing smithi16234619-32 from 1816008 to 2319134 tid 2 2024-09-16T15:05:12.413 INFO:tasks.rados.rados.0.smithi162.stdout:1052: read oid 15 snap -1 2024-09-16T15:05:12.413 INFO:tasks.rados.rados.0.smithi162.stdout:1052: expect (ObjNum 349 snap 81 seq_num 349) 2024-09-16T15:05:12.414 INFO:tasks.rados.rados.0.smithi162.stdout:1053: snap_remove snap 93 2024-09-16T15:05:12.720 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:05:12.720 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-16T15:05:12.720 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-16T15:05:12.720 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-16T15:05:12.724 INFO:tasks.ceph.mon.f:Started 2024-09-16T15:05:12.724 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:05:12.724 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-16T15:05:12.724 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-16T15:05:12.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-16T15:05:12.727 INFO:tasks.ceph.mon.e:Started 2024-09-16T15:05:12.727 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-16T15:05:12.727 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-16T15:05:12.727 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-16T15:05:12.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-16T15:05:12.730 INFO:tasks.ceph.mon.h:Started 2024-09-16T15:05:12.730 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:05:12.730 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-16T15:05:12.730 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:05:12.730 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:05:12.732 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:05:12.732 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:05:12.732 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.842+0000 7f2193bbc640 10 monclient: get_monmap_and_config 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.842+0000 7f2193bbc640 10 monclient: build_initial_monmap 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.842+0000 7f2193bbc640 10 monclient: monmap: 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:12.843470+0000 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:12.843470+0000 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:12.844 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:12.845 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.843+0000 7f2193bbc640 1 Processor -- start 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.843+0000 7f2193bbc640 1 -- start start 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.843+0000 7f2193bbc640 10 monclient: init 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient: _add_conns ranks=[4,3,1,0,8,10,2,6,9,7,5] 2024-09-16T15:05:12.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-c con 0x7f218c136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-g con 0x7f218c09bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f218c136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c073250 con 0x7f218c136c60 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09b600 con 0x7f218c09bfe0 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0a5740 con 0x7f218c136030 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2193bbc640 10 monclient(hunting): authenticate will time out at 1049.433487s 2024-09-16T15:05:12.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:34392/0 (socket says 172.21.15.134:34392) 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 learned_addr learned my addr 172.21.15.134:0/438382029 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f218c136030 auth_method 0 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f218c09bfe0 auth_method 0 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:12.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.844+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:12.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:12.846+0000 7f2192159640 1 -- 172.21.15.134:0/438382029 >> v1:172.21.15.162:6789/0 conn(0x7f218c136c60 legacy=0x7f218c1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:13.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:13.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f218c136030 auth_method 0 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f218c09bfe0 auth_method 0 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:13.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:13.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:13.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:13.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.045+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:13.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:13.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f218c136030 auth_method 0 2024-09-16T15:05:13.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:13.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:13.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f218c09bfe0 auth_method 0 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:13.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:13.446+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f218c136030 auth_method 0 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f218c09bfe0 auth_method 0 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:14.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:14.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:14.247+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): tick 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 monclient(hunting): continuing hunt 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 msgr2=0x7f218c136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f218c136030 0x7f218c136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 msgr2=0x7f218c09c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f218c09bfe0 0x7f218c09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 >> v1:172.21.15.162:6789/0 conn(0x7f218c136c60 legacy=0x7f218c1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:15.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient: _add_conns ranks=[10,6,0,5,2,8,9,7] 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): picked mon.noname-f con 0x7f21700038c0 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): picked mon.noname-d con 0x7f21700062f0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2170008e00 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218c0a52c0 con 0x7f2170008e00 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218c09b600 con 0x7f21700038c0 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 1 -- 172.21.15.134:0/438382029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0a5740 con 0x7f21700062f0 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f21837fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.844+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:15.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21700062f0 auth_method 0 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f21700038c0 auth_method 0 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:15.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:15.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.845+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:15.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:15.847+0000 7f2192159640 1 -- 172.21.15.134:0/438382029 >> v1:172.21.15.134:6789/0 conn(0x7f2170008e00 legacy=0x7f217000b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21700062f0 auth_method 0 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:16.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f21700038c0 auth_method 0 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:16.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.046+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21700062f0 auth_method 0 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:16.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f21700038c0 auth_method 0 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:16.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:16.447+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21700062f0 auth_method 0 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:17.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f21700038c0 auth_method 0 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.248+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997788203 0 0) 0x7f218c0a52c0 con 0x7f2170008e00 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 10 monclient(hunting): _negotiate 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 10 monclient(hunting): my global_id is 4743 2024-09-16T15:05:17.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.861+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2178003520 con 0x7f2170008e00 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.862+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778913969 0 0) 0x7f2178003520 con 0x7f2170008e00 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 msgr2=0x7f2170008740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21700062f0 0x7f2170008740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 msgr2=0x7f2170005cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 --2- 172.21.15.134:0/438382029 >> v2:172.21.15.162:3300/0 conn(0x7f21700038c0 0x7f2170005cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _finish_hunting 0 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:05:17.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 1 -- 172.21.15.134:0/438382029 reap_dead start 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217000dcb0 con 0x7f2170008e00 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _finish_auth 0 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _check_auth_tickets 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f2184001f60 con 0x7f2170008e00 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: dump: 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:17.867 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:17.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 >> v1:172.21.15.134:6789/0 conn(0x7f2170008e00 legacy=0x7f217000b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient: _add_conns ranks=[6,8,2,4,3,7,5,0,1] 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 0x7f21780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): picked mon.d con 0x7f21780041c0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2178004dd0 0x7f2178005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): picked mon.e con 0x7f2178004dd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 0x7f217800c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): picked mon.b con 0x7f2178009bb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09b600 con 0x7f2178009bb0 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f218c0a5740 con 0x7f21780041c0 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f217800c950 con 0x7f2178004dd0 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:17.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2190956640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 0x7f21780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f21780041c0 auth_method 0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 0x7f217800c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2178004dd0 0x7f2178005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f2178009bb0 auth_method 0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f2178004dd0 auth_method 0 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient(hunting): handle_auth_done global_id 4743 payload 293 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2178004dd0 msgr2=0x7f2178005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:17.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2178004dd0 0x7f2178005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 msgr2=0x7f217800c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 0x7f217800c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient: _finish_hunting 0 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 monclient: found mon.d 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178003520 con 0x7f21780041c0 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient: _finish_auth 0 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 10 monclient: _check_auth_tickets 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2191958640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2193bbc640 5 monclient: authenticate success, global_id 4743 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2193bbc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:17.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.863+0000 7f2192159640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 0x7f217800c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:17.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.866+0000 7f2191958640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 msgr2=0x7f21780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:17.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.866+0000 7f2191958640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 0x7f21780047c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2188019b00 tx=0x7f21780057f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a480bb356d046f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f218802ca90 con 0x7f21780041c0 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: dump: 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:17.877 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218800c040 con 0x7f21780041c0 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2193bbc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2193bbc640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 1 -- 172.21.15.134:0/438382029 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f218802cc10 con 0x7f21780041c0 2024-09-16T15:05:17.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2193bbc640 4 set_mon_vals no callback set 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.868+0000 7f2190956640 10 monclient: dump: 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:17.879 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 10 monclient: shutdown 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 -- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 msgr2=0x7f21780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 0x7f21780047c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2188019b00 tx=0x7f21780057f0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 -- 172.21.15.134:0/438382029 shutdown_connections 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2178009bb0 0x7f217800c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2178004dd0 0x7f2178005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 --2- 172.21.15.134:0/438382029 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f21780041c0 0x7f21780047c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:17.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 -- 172.21.15.134:0/438382029 >> 172.21.15.134:0/438382029 conn(0x7f218c0a9ea0 msgr2=0x7f218c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 -- 172.21.15.134:0/438382029 shutdown_connections 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.869+0000 7f2193bbc640 1 -- 172.21.15.134:0/438382029 wait complete. 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.870+0000 7f2193bbc640 10 monclient: build_initial_monmap 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.870+0000 7f2193bbc640 10 monclient: monmap: 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:17.881 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.870+0000 7f2193bbc640 1 Processor -- start 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.870+0000 7f2193bbc640 1 -- start start 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.870+0000 7f2193bbc640 10 monclient: init 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient: _add_conns ranks=[1,2,5,3,4,7,8,0,6] 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-f con 0x7f2170008e00 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-b con 0x7f21780041c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:17.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): picked mon.noname-h con 0x7f2178004dd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09fa90 con 0x7f2170008e00 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09f6d0 con 0x7f21780041c0 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1245b0 con 0x7f2178004dd0 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2193bbc640 10 monclient(hunting): authenticate will time out at 1054.460182s 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34404/0 (socket says 172.21.15.134:34404) 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 learned_addr learned my addr 172.21.15.134:0/3126600861 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21780041c0 auth_method 0 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f2178004dd0 auth_method 0 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f2170008e00 auth_method 0 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:17.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:17.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:17.871+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21780041c0 auth_method 0 2024-09-16T15:05:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f2178004dd0 auth_method 0 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f2170008e00 auth_method 0 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.072+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:18.175 INFO:tasks.rados.rados.0.smithi162.stdout:1053: done (4 left) 2024-09-16T15:05:18.175 INFO:tasks.rados.rados.0.smithi162.stdout:1054: read oid 19 snap -1 2024-09-16T15:05:18.175 INFO:tasks.rados.rados.0.smithi162.stdout:1054: expect deleted 2024-09-16T15:05:18.175 INFO:tasks.rados.rados.0.smithi162.stdout:1055: snap_remove snap 84 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21780041c0 auth_method 0 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f2178004dd0 auth_method 0 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f2170008e00 auth_method 0 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:18.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:18.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:18.473+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:1055: done (5 left) 2024-09-16T15:05:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:1056: write (excl) oid 34 current snap is 94 2024-09-16T15:05:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:1056: seq_num 404 ranges {790028=567305,1874490=763137} 2024-09-16T15:05:19.187 INFO:tasks.rados.rados.0.smithi162.stdout:1056: writing smithi16234619-34 from 790028 to 1357333 tid 1 2024-09-16T15:05:19.194 INFO:tasks.rados.rados.0.smithi162.stdout:1056: writing smithi16234619-34 from 1874490 to 2637627 tid 2 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: finishing write tid 1 to smithi16234619-9 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: finishing write tid 2 to smithi16234619-9 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: finishing write tid 3 to smithi16234619-9 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: finishing write tid 4 to smithi16234619-9 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: oid 9 updating version 0 to 343 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: oid 9 version 343 is already newer than 342 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: oid 9 updating version 343 to 344 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 344 (ObjNum 401 snap 94 seq_num 401) dirty exists 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1047: left oid 9 (ObjNum 401 snap 94 seq_num 401) 2024-09-16T15:05:19.195 INFO:tasks.rados.rados.0.smithi162.stdout:1052: expect (ObjNum 349 snap 81 seq_num 349) 2024-09-16T15:05:19.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:19.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f2170008e00 auth_method 0 2024-09-16T15:05:19.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:19.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f21780041c0 auth_method 0 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:19.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f2178004dd0 auth_method 0 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:19.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:19.274+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1051: finishing write tid 1 to smithi16234619-32 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1051: finishing write tid 2 to smithi16234619-32 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1057: read oid 42 snap -1 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1057: expect deleted 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1047: done (6 left) 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1049: done (5 left) 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1052: done (4 left) 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1058: write oid 37 current snap is 94 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1051: finishing write tid 3 to smithi16234619-32 2024-09-16T15:05:19.323 INFO:tasks.rados.rados.0.smithi162.stdout:1051: finishing write tid 4 to smithi16234619-32 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:1051: oid 32 updating version 0 to 194 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:1051: oid 32 version 194 is already newer than 193 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:1051: oid 32 updating version 194 to 195 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 195 (ObjNum 403 snap 94 seq_num 403) dirty exists 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:1051: left oid 32 (ObjNum 403 snap 94 seq_num 403) 2024-09-16T15:05:19.324 INFO:tasks.rados.rados.0.smithi162.stdout:1058: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2024-09-16T15:05:19.327 INFO:tasks.rados.rados.0.smithi162.stdout:1058: writing smithi16234619-37 from 406835 to 1049389 tid 1 2024-09-16T15:05:19.333 INFO:tasks.rados.rados.0.smithi162.stdout:1058: writing smithi16234619-37 from 1532972 to 2039767 tid 2 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1058: writing smithi16234619-37 from 2772312 to 2806835 tid 3 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1051: done (4 left) 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1054: done (3 left) 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1059: rollback oid 15 current snap is 94 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 82 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1056: finishing write tid 1 to smithi16234619-34 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1056: finishing write tid 2 to smithi16234619-34 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1056: finishing write tid 3 to smithi16234619-34 2024-09-16T15:05:19.338 INFO:tasks.rados.rados.0.smithi162.stdout:1056: finishing write tid 4 to smithi16234619-34 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1056: oid 34 updating version 0 to 347 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1056: oid 34 version 347 is already newer than 346 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1056: oid 34 version 347 is already newer than 345 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 347 (ObjNum 404 snap 94 seq_num 404) dirty exists 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1056: left oid 34 (ObjNum 404 snap 94 seq_num 404) 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1056: done (3 left) 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1057: done (2 left) 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1060: write oid 9 current snap is 94 2024-09-16T15:05:19.339 INFO:tasks.rados.rados.0.smithi162.stdout:1060: seq_num 406 ranges {423642=717803,1875101=766806} 2024-09-16T15:05:19.343 INFO:tasks.rados.rados.0.smithi162.stdout:1060: writing smithi16234619-9 from 423642 to 1141445 tid 1 2024-09-16T15:05:19.352 INFO:tasks.rados.rados.0.smithi162.stdout:1060: writing smithi16234619-9 from 1875101 to 2641907 tid 2 2024-09-16T15:05:19.352 INFO:tasks.rados.rados.0.smithi162.stdout:1061: write oid 2 current snap is 94 2024-09-16T15:05:19.352 INFO:tasks.rados.rados.0.smithi162.stdout:1059: finishing rollback tid 0 to smithi16234619-15 2024-09-16T15:05:19.352 INFO:tasks.rados.rados.0.smithi162.stdout:1059: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:05:19.352 INFO:tasks.rados.rados.0.smithi162.stdout:1059: finishing rollback tid 2 to smithi16234619-15 2024-09-16T15:05:19.353 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 266 (ObjNum 349 snap 81 seq_num 349) dirty exists 2024-09-16T15:05:19.353 INFO:tasks.rados.rados.0.smithi162.stdout:1061: seq_num 407 ranges {440449=793052,1933583=510464} 2024-09-16T15:05:19.358 INFO:tasks.rados.rados.0.smithi162.stdout:1061: writing smithi16234619-2 from 440449 to 1233501 tid 1 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1061: writing smithi16234619-2 from 1933583 to 2444047 tid 2 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1059: done (3 left) 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1062: read oid 1 snap -1 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1062: expect (ObjNum 327 snap 75 seq_num 327) 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1063: write (excl) oid 16 current snap is 94 2024-09-16T15:05:19.366 INFO:tasks.rados.rados.0.smithi162.stdout:1063: seq_num 408 ranges {457256=468301,1592065=770475,2854581=2675} 2024-09-16T15:05:19.371 INFO:tasks.rados.rados.0.smithi162.stdout:1063: writing smithi16234619-16 from 457256 to 925557 tid 1 2024-09-16T15:05:19.381 INFO:tasks.rados.rados.0.smithi162.stdout:1063: writing smithi16234619-16 from 1592065 to 2362540 tid 2 2024-09-16T15:05:19.384 INFO:tasks.rados.rados.0.smithi162.stdout:1063: writing smithi16234619-16 from 2854581 to 2857256 tid 3 2024-09-16T15:05:19.384 INFO:tasks.rados.rados.0.smithi162.stdout:1064: snap_remove snap 82 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1064: done (5 left) 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1065: delete oid 21 current snap is 94 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: finishing write tid 1 to smithi16234619-37 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: finishing write tid 2 to smithi16234619-37 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: finishing write tid 3 to smithi16234619-37 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: finishing write tid 4 to smithi16234619-37 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: finishing write tid 5 to smithi16234619-37 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: oid 37 updating version 0 to 437 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: oid 37 version 437 is already newer than 435 2024-09-16T15:05:20.183 INFO:tasks.rados.rados.0.smithi162.stdout:1058: oid 37 version 437 is already newer than 436 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1058: oid 37 version 437 is already newer than 434 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 437 (ObjNum 405 snap 94 seq_num 405) dirty exists 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1058: left oid 37 (ObjNum 405 snap 94 seq_num 405) 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1060: finishing write tid 1 to smithi16234619-9 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: finishing write tid 1 to smithi16234619-2 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: finishing write tid 2 to smithi16234619-2 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: finishing write tid 3 to smithi16234619-2 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: finishing write tid 4 to smithi16234619-2 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: oid 2 updating version 0 to 381 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: oid 2 updating version 381 to 382 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:1061: oid 2 version 382 is already newer than 380 2024-09-16T15:05:20.184 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 382 (ObjNum 407 snap 94 seq_num 407) dirty exists 2024-09-16T15:05:20.185 INFO:tasks.rados.rados.0.smithi162.stdout:1061: left oid 2 (ObjNum 407 snap 94 seq_num 407) 2024-09-16T15:05:20.185 INFO:tasks.rados.rados.0.smithi162.stdout:1062: expect (ObjNum 327 snap 75 seq_num 327) 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: finishing write tid 2 to smithi16234619-9 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: finishing write tid 3 to smithi16234619-9 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: finishing write tid 4 to smithi16234619-9 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: oid 9 updating version 0 to 357 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: oid 9 version 357 is already newer than 356 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:1060: oid 9 version 357 is already newer than 355 2024-09-16T15:05:20.298 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 357 (ObjNum 406 snap 94 seq_num 406) dirty exists 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1060: left oid 9 (ObjNum 406 snap 94 seq_num 406) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: finishing write tid 1 to smithi16234619-16 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: finishing write tid 2 to smithi16234619-16 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1058: done (5 left) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1060: done (4 left) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1061: done (3 left) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1062: done (2 left) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1065: done (1 left) 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1066: snap_create 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: finishing write tid 3 to smithi16234619-16 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: finishing write tid 4 to smithi16234619-16 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: finishing write tid 5 to smithi16234619-16 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: oid 16 updating version 0 to 297 2024-09-16T15:05:20.299 INFO:tasks.rados.rados.0.smithi162.stdout:1063: oid 16 updating version 297 to 299 2024-09-16T15:05:20.300 INFO:tasks.rados.rados.0.smithi162.stdout:1063: oid 16 version 299 is already newer than 298 2024-09-16T15:05:20.300 INFO:tasks.rados.rados.0.smithi162.stdout:1063: oid 16 updating version 299 to 300 2024-09-16T15:05:20.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 300 (ObjNum 408 snap 94 seq_num 408) dirty exists 2024-09-16T15:05:20.300 INFO:tasks.rados.rados.0.smithi162.stdout:1063: left oid 16 (ObjNum 408 snap 94 seq_num 408) 2024-09-16T15:05:20.395 INFO:tasks.rados.rados.0.smithi162.stdout:1063: done (1 left) 2024-09-16T15:05:20.395 INFO:tasks.rados.rados.0.smithi162.stdout:1066: done (0 left) 2024-09-16T15:05:20.395 INFO:tasks.rados.rados.0.smithi162.stdout:1067: write (excl) oid 42 current snap is 95 2024-09-16T15:05:20.395 INFO:tasks.rados.rados.0.smithi162.stdout:1067: seq_num 410 ranges {490870=618799,1592676=774144,2793074=97796} 2024-09-16T15:05:20.401 INFO:tasks.rados.rados.0.smithi162.stdout:1067: writing smithi16234619-42 from 490870 to 1109669 tid 1 2024-09-16T15:05:20.408 INFO:tasks.rados.rados.0.smithi162.stdout:1067: writing smithi16234619-42 from 1592676 to 2366820 tid 2 2024-09-16T15:05:20.411 INFO:tasks.rados.rados.0.smithi162.stdout:1067: writing smithi16234619-42 from 2793074 to 2890870 tid 3 2024-09-16T15:05:20.411 INFO:tasks.rados.rados.0.smithi162.stdout:1068: write (excl) oid 23 current snap is 95 2024-09-16T15:05:20.411 INFO:tasks.rados.rados.0.smithi162.stdout:1068: seq_num 411 ranges {507677=410401,1367511=517802,2420497=487180} 2024-09-16T15:05:20.416 INFO:tasks.rados.rados.0.smithi162.stdout:1068: writing smithi16234619-23 from 507677 to 918078 tid 1 2024-09-16T15:05:20.422 INFO:tasks.rados.rados.0.smithi162.stdout:1068: writing smithi16234619-23 from 1367511 to 1885313 tid 2 2024-09-16T15:05:20.429 INFO:tasks.rados.rados.0.smithi162.stdout:1068: writing smithi16234619-23 from 2420497 to 2907677 tid 3 2024-09-16T15:05:20.429 INFO:tasks.rados.rados.0.smithi162.stdout:1069: write (excl) oid 28 current snap is 95 2024-09-16T15:05:20.429 INFO:tasks.rados.rados.0.smithi162.stdout:1069: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2024-09-16T15:05:20.435 INFO:tasks.rados.rados.0.smithi162.stdout:1069: writing smithi16234619-28 from 524484 to 1010134 tid 1 2024-09-16T15:05:20.441 INFO:tasks.rados.rados.0.smithi162.stdout:1069: writing smithi16234619-28 from 1425993 to 2087453 tid 2 2024-09-16T15:05:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:1069: writing smithi16234619-28 from 2847920 to 2924484 tid 3 2024-09-16T15:05:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:1070: write (excl) oid 13 current snap is 95 2024-09-16T15:05:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:1067: finishing write tid 1 to smithi16234619-42 2024-09-16T15:05:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:1070: seq_num 413 ranges {541291=560899,1884475=521471} 2024-09-16T15:05:20.451 INFO:tasks.rados.rados.0.smithi162.stdout:1070: writing smithi16234619-13 from 541291 to 1102190 tid 1 2024-09-16T15:05:20.459 INFO:tasks.rados.rados.0.smithi162.stdout:1070: writing smithi16234619-13 from 1884475 to 2405946 tid 2 2024-09-16T15:05:20.459 INFO:tasks.rados.rados.0.smithi162.stdout:1071: snap_create 2024-09-16T15:05:20.459 INFO:tasks.rados.rados.0.smithi162.stdout:1067: finishing write tid 2 to smithi16234619-42 2024-09-16T15:05:20.459 INFO:tasks.rados.rados.0.smithi162.stdout:1067: finishing write tid 3 to smithi16234619-42 2024-09-16T15:05:20.459 INFO:tasks.rados.rados.0.smithi162.stdout:1067: finishing write tid 4 to smithi16234619-42 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: finishing write tid 5 to smithi16234619-42 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: oid 42 updating version 0 to 448 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: oid 42 version 448 is already newer than 447 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: oid 42 updating version 448 to 449 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: oid 42 version 449 is already newer than 446 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 449 (ObjNum 410 snap 95 seq_num 410) dirty exists 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1067: left oid 42 (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: finishing write tid 1 to smithi16234619-23 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: finishing write tid 2 to smithi16234619-23 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: finishing write tid 3 to smithi16234619-23 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: finishing write tid 4 to smithi16234619-23 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: finishing write tid 5 to smithi16234619-23 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: oid 23 updating version 0 to 104 2024-09-16T15:05:20.460 INFO:tasks.rados.rados.0.smithi162.stdout:1068: oid 23 version 104 is already newer than 103 2024-09-16T15:05:20.461 INFO:tasks.rados.rados.0.smithi162.stdout:1068: oid 23 updating version 104 to 106 2024-09-16T15:05:20.461 INFO:tasks.rados.rados.0.smithi162.stdout:1068: oid 23 version 106 is already newer than 105 2024-09-16T15:05:20.461 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 106 (ObjNum 411 snap 95 seq_num 411) dirty exists 2024-09-16T15:05:20.461 INFO:tasks.rados.rados.0.smithi162.stdout:1068: left oid 23 (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:05:20.465 INFO:tasks.rados.rados.0.smithi162.stdout:1069: finishing write tid 1 to smithi16234619-28 2024-09-16T15:05:20.465 INFO:tasks.rados.rados.0.smithi162.stdout:1069: finishing write tid 2 to smithi16234619-28 2024-09-16T15:05:20.468 INFO:tasks.rados.rados.0.smithi162.stdout:1069: finishing write tid 3 to smithi16234619-28 2024-09-16T15:05:20.468 INFO:tasks.rados.rados.0.smithi162.stdout:1069: finishing write tid 4 to smithi16234619-28 2024-09-16T15:05:20.468 INFO:tasks.rados.rados.0.smithi162.stdout:1069: finishing write tid 5 to smithi16234619-28 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:1069: oid 28 updating version 0 to 268 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:1069: oid 28 updating version 268 to 270 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:1069: oid 28 version 270 is already newer than 269 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:1069: oid 28 version 270 is already newer than 267 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 270 (ObjNum 412 snap 95 seq_num 412) dirty exists 2024-09-16T15:05:20.469 INFO:tasks.rados.rados.0.smithi162.stdout:1069: left oid 28 (ObjNum 412 snap 95 seq_num 412) 2024-09-16T15:05:20.477 INFO:tasks.rados.rados.0.smithi162.stdout:1070: finishing write tid 1 to smithi16234619-13 2024-09-16T15:05:20.477 INFO:tasks.rados.rados.0.smithi162.stdout:1070: finishing write tid 2 to smithi16234619-13 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: finishing write tid 3 to smithi16234619-13 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: finishing write tid 4 to smithi16234619-13 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: oid 13 updating version 0 to 451 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: oid 13 updating version 451 to 453 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: oid 13 version 453 is already newer than 452 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 453 (ObjNum 413 snap 95 seq_num 413) dirty exists 2024-09-16T15:05:20.479 INFO:tasks.rados.rados.0.smithi162.stdout:1070: left oid 13 (ObjNum 413 snap 95 seq_num 413) 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): tick 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 monclient(hunting): continuing hunt 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 msgr2=0x7f218c1354c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2178004dd0 0x7f218c1354c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 msgr2=0x7f218c134f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:20.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21780041c0 0x7f218c134f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 msgr2=0x7f218c1cb380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2170008e00 0x7f218c1cb380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient: _add_conns ranks=[7,0,8,3,4,6] 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f217000cb70 0x7f2170005cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): picked mon.noname-i con 0x7f217000cb70 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 0x7f21700086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): picked mon.noname-a con 0x7f217000d030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 0x7f217000b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): picked mon.noname-e con 0x7f217000d570 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09fa90 con 0x7f217000d030 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:20.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f218c09f6d0 con 0x7f217000cb70 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f218c1245b0 con 0x7f217000d570 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f21817fa640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 0x7f21700086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 0x7f217000b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 10 monclient(hunting): get_auth_request con 0x7f217000d570 auth_method 0 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): get_auth_request con 0x7f217000d030 auth_method 0 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:20.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 msgr2=0x7f217000b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 msgr2=0x7f217000b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 0x7f217000b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 0x7f217000b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f217000cb70 0x7f2170005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 10 monclient(hunting): get_auth_request con 0x7f217000cb70 auth_method 0 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:20.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient(hunting): handle_auth_done global_id 4752 payload 824 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 msgr2=0x7f217000b220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f217000d570 0x7f217000b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f217000cb70 msgr2=0x7f2170005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f217000cb70 0x7f2170005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient: _finish_hunting 0 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 monclient: found mon.noname-a 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2192159640 1 -- 172.21.15.134:0/3126600861 reap_dead start 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c124d00 con 0x7f217000d030 2024-09-16T15:05:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.871+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f217000cb70 0x7f2170005cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2191157640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 msgr2=0x7f21700086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2191157640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 0x7f21700086e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f217c006a90 tx=0x7f217c003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f217c026430 con 0x7f217000d030 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: dump: 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c01f070 con 0x7f217000d030 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 5 monclient: authenticate success, global_id 4752 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f217c026a00 con 0x7f217000d030 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 10 monclient: _renew_subs 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c1266a0 con 0x7f217000d030 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2183fff640 4 set_mon_vals no callback set 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f21827fc640 10 monclient: dump: 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 10 monclient: _renew_subs 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.874+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c1cb560 con 0x7f217000d030 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f217c026590 con 0x7f217000d030 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f21827fc640 1 --2- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 0x7f217403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f217c04c700 con 0x7f217000d030 2024-09-16T15:05:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2193bbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2193bbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2193bbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c0679b0 con 0x7f217000d030 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 0x7f217403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.875+0000 7f2191958640 10 monclient: get_auth_request con 0x7f21740387f0 auth_method 0 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.878+0000 7f2191958640 1 --2- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 0x7f217403acb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2188033000 tx=0x7f2188030000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.880+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f217c034080 con 0x7f217000d030 2024-09-16T15:05:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.880+0000 7f21827fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:20.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:20.880+0000 7f21827fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:21.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.102+0000 7f2193bbc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:21.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.102+0000 7f2193bbc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.102+0000 7f2193bbc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.102+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f218c09c400 con 0x7f217000d030 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.103+0000 7f21827fc640 1 -- 172.21.15.134:0/3126600861 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f217c02db50 con 0x7f217000d030 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.103+0000 7f21827fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.103+0000 7f21827fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:21.105 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":42,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 msgr2=0x7f217403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 --2- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 0x7f217403acb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2188033000 tx=0x7f2188030000 comp rx=0 tx=0).stop 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 10 monclient: shutdown 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 msgr2=0x7f21700086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 0x7f21700086e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f217c006a90 tx=0x7f217c003aa0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 shutdown_connections 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 --2- 172.21.15.134:0/3126600861 >> v2:172.21.15.162:6800/980547263 conn(0x7f21740387f0 0x7f217403acb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 --2- 172.21.15.134:0/3126600861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217000d030 0x7f21700086e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 >> 172.21.15.134:0/3126600861 conn(0x7f218c0a9ea0 msgr2=0x7f218c09fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:21.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 shutdown_connections 2024-09-16T15:05:21.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.105+0000 7f2193bbc640 1 -- 172.21.15.134:0/3126600861 wait complete. 2024-09-16T15:05:21.118 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:05:21.118 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:05:21.118 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.217+0000 7f44d50d0640 10 monclient: get_monmap_and_config 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.217+0000 7f44d50d0640 10 monclient: build_initial_monmap 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.217+0000 7f44d50d0640 10 monclient: monmap: 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.219 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:21.218613+0000 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:21.218613+0000 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:21.220 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:21.221 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 Processor -- start 2024-09-16T15:05:21.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 -- start start 2024-09-16T15:05:21.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient: init 2024-09-16T15:05:21.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient: _add_conns ranks=[5,3,9,0,10,8,1,7,4,2,6] 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f44d0136eb0 0x7f44d009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-h con 0x7f44d0136eb0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44d0137290 0x7f44d0137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-g con 0x7f44d0137290 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f44d0133d10 0x7f44d01341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-a con 0x7f44d0133d10 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00a5560 con 0x7f44d0133d10 2024-09-16T15:05:21.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d009b240 con 0x7f44d0137290 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44d009b520 con 0x7f44d0136eb0 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44d50d0640 10 monclient(hunting): authenticate will time out at 1057.807765s 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f44d0133d10 0x7f44d01341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f44d0133d10 0x7f44d01341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46318/0 (socket says 172.21.15.134:46318) 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 1 -- 172.21.15.134:0/3951612521 learned_addr learned my addr 172.21.15.134:0/3951612521 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 10 monclient(hunting): get_auth_request con 0x7f44d0133d10 auth_method 0 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.218+0000 7f44cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ced9d640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f44d0136eb0 0x7f44d009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ce59c640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44d0137290 0x7f44d0137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ced9d640 10 monclient(hunting): get_auth_request con 0x7f44d0136eb0 auth_method 0 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ce59c640 10 monclient(hunting): get_auth_request con 0x7f44d0137290 auth_method 0 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient(hunting): handle_auth_done global_id 4761 payload 274 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 -- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f44d0136eb0 msgr2=0x7f44d009bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f44d0136eb0 0x7f44d009bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 -- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44d0137290 msgr2=0x7f44d0137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44d0137290 0x7f44d0137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient: _finish_hunting 0 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 monclient: found mon.noname-a 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.219+0000 7f44cf59e640 1 -- 172.21.15.134:0/3951612521 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0073250 con 0x7f44d0133d10 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.221+0000 7f44cf59e640 1 -- 172.21.15.134:0/3951612521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0133d10 msgr2=0x7f44d01341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.221+0000 7f44cf59e640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0133d10 0x7f44d01341c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8ace5c71ec53e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 1 -- 172.21.15.134:0/3951612521 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c8007a70 con 0x7f44d0133d10 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: dump: 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.232 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: _finish_auth 0 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 5 monclient: authenticate success, global_id 4761 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 1 -- 172.21.15.134:0/3951612521 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c802d050 con 0x7f44d0133d10 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 1 -- 172.21.15.134:0/3951612521 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c8007bf0 con 0x7f44d0133d10 2024-09-16T15:05:21.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 4 set_mon_vals no callback set 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44cdd9b640 10 monclient: dump: 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.234 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 10 monclient: shutdown 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 -- 172.21.15.134:0/3951612521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0133d10 msgr2=0x7f44d01341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0133d10 0x7f44d01341c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c802a560 comp rx=0 tx=0).stop 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 -- 172.21.15.134:0/3951612521 shutdown_connections 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0133d10 0x7f44d01341c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f44d0137290 0x7f44d0137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 --2- 172.21.15.134:0/3951612521 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f44d0136eb0 0x7f44d009bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 -- 172.21.15.134:0/3951612521 >> 172.21.15.134:0/3951612521 conn(0x7f44d00a99a0 msgr2=0x7f44d00a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 -- 172.21.15.134:0/3951612521 shutdown_connections 2024-09-16T15:05:21.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.222+0000 7f44d50d0640 1 -- 172.21.15.134:0/3951612521 wait complete. 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.223+0000 7f44d50d0640 10 monclient: build_initial_monmap 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.223+0000 7f44d50d0640 10 monclient: monmap: 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:21.236 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.223+0000 7f44d50d0640 1 Processor -- start 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 -- start start 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient: init 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient: _add_conns ranks=[2,0,8,1,3,5,7,4,6] 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 0x7f44d0132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-b con 0x7f44d0133d10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 0x7f44d0133100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-a con 0x7f44d0136eb0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:21.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 0x7f44d009d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): picked mon.noname-e con 0x7f44d0137290 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d009ec60 con 0x7f44d0136eb0 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d01363a0 con 0x7f44d0133d10 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00a7af0 con 0x7f44d0137290 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44d50d0640 10 monclient(hunting): authenticate will time out at 1057.813471s 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 0x7f44d0133100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 0x7f44d0132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 0x7f44d0133100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46326/0 (socket says 172.21.15.134:46326) 2024-09-16T15:05:21.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 -- 172.21.15.134:0/311382928 learned_addr learned my addr 172.21.15.134:0/311382928 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): get_auth_request con 0x7f44d0136eb0 auth_method 0 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 10 monclient(hunting): get_auth_request con 0x7f44d0133d10 auth_method 0 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 0x7f44d009d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 10 monclient(hunting): get_auth_request con 0x7f44d0137290 auth_method 0 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 msgr2=0x7f44d0132bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 msgr2=0x7f44d0132bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 0x7f44d0132bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 0x7f44d0132bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 msgr2=0x7f44d009d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 msgr2=0x7f44d009d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 0x7f44d009d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44cf59e640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 0x7f44d009d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:21.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient(hunting): handle_auth_done global_id 4770 payload 824 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 msgr2=0x7f44d009d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f44d0137290 0x7f44d009d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 msgr2=0x7f44d0132bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44d0133d10 0x7f44d0132bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 monclient: found mon.noname-a 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.224+0000 7f44ce59c640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8009b30 con 0x7f44d0136eb0 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44ce59c640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 msgr2=0x7f44d0133100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44ce59c640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 0x7f44d0133100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d5d0 tx=0x7f44c400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c4004450 con 0x7f44d0136eb0 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: dump: 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.246 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.247 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c4003a80 con 0x7f44d0136eb0 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 5 monclient: authenticate success, global_id 4770 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 10 monclient: _renew_subs 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c4003bf0 con 0x7f44d0136eb0 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00a75a0 con 0x7f44d0136eb0 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44cd59a640 4 set_mon_vals no callback set 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44b77fe640 10 monclient: dump: 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.249 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 10 monclient: _renew_subs 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.227+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d01202c0 con 0x7f44d0136eb0 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f44c4003d50 con 0x7f44d0136eb0 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44d50d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44d50d0640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 0x7f44d009dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0138070 con 0x7f44d009d9c0 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44b77fe640 1 --2- 172.21.15.134:0/311382928 >> v2:172.21.15.162:6800/980547263 conn(0x7f44bc0388b0 0x7f44bc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 0x7f44d009dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient: get_auth_request con 0x7f44d009d9c0 auth_method 0 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f44c404a430 con 0x7f44d0136eb0 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 10 monclient(hunting): handle_auth_done global_id 4779 payload 824 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44d0061ae0 con 0x7f44d009d9c0 2024-09-16T15:05:21.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44cf59e640 1 --2- 172.21.15.134:0/311382928 >> v2:172.21.15.162:6800/980547263 conn(0x7f44bc0388b0 0x7f44bc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.228+0000 7f44cf59e640 10 monclient: get_auth_request con 0x7f44bc0388b0 auth_method 0 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 msgr2=0x7f44d009dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 0x7f44d009dda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f44c0006fd0 tx=0x7f44c000f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c0014cf0 con 0x7f44d009d9c0 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44b77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.230+0000 7f44b77fe640 10 monclient: dump: 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.252 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f44c0014e50 con 0x7f44d009d9c0 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 msgr2=0x7f44d009dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 0x7f44d009dda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f44c0006fd0 tx=0x7f44c000f910 comp rx=0 tx=0).stop 2024-09-16T15:05:21.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 msgr2=0x7f44d009dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44b77fe640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009d9c0 0x7f44d009dda0 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f44c0006fd0 tx=0x7f44c000f910 comp rx=0 tx=0).stop 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.231+0000 7f44cf59e640 1 --2- 172.21.15.134:0/311382928 >> v2:172.21.15.162:6800/980547263 conn(0x7f44bc0388b0 0x7f44bc03ad70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44d00a5f80 tx=0x7f44c8009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44d50d0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44d50d0640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 0x7f44d01340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44d50d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0071b20 con 0x7f44d009e2e0 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 0x7f44d01340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient: get_auth_request con 0x7f44d009e2e0 auth_method 0 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 10 monclient(hunting): handle_auth_done global_id 4788 payload 824 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.237+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f44d007a680 con 0x7f44d009e2e0 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44ced9d640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 msgr2=0x7f44d01340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44ced9d640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 0x7f44d01340f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f44c001a000 tx=0x7f44c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44c0004490 con 0x7f44d009e2e0 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44b77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.239+0000 7f44b77fe640 10 monclient: dump: 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.255 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f44c0004020 con 0x7f44d009e2e0 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 msgr2=0x7f44d01340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 0x7f44d01340f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f44c001a000 tx=0x7f44c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 msgr2=0x7f44d01340f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44b77fe640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d009e2e0 0x7f44d01340f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:21.257 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:21.258 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:21.259 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:21.260 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:21.261 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:21.262 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:21.264 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.265 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:21.268 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:21.269 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.270 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.271 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.272 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.273 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:21.274 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.276 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:21.277 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:21.278 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:21.279 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:21.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:21.280 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:21.280 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 >> v2:172.21.15.162:6800/980547263 conn(0x7f44bc0388b0 msgr2=0x7f44bc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 --2- 172.21.15.134:0/311382928 >> v2:172.21.15.162:6800/980547263 conn(0x7f44bc0388b0 0x7f44bc03ad70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44d00a5f80 tx=0x7f44c8009000 comp rx=0 tx=0).stop 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 10 monclient: shutdown 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 msgr2=0x7f44d0133100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 --2- 172.21.15.134:0/311382928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44d0136eb0 0x7f44d0133100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f44c400d5d0 tx=0x7f44c400da90 comp rx=0 tx=0).stop 2024-09-16T15:05:21.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44cf59e640 1 -- 172.21.15.134:0/311382928 reap_dead start 2024-09-16T15:05:21.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44cf59e640 1 -- 172.21.15.134:0/311382928 reap_dead start 2024-09-16T15:05:21.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 shutdown_connections 2024-09-16T15:05:21.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 >> 172.21.15.134:0/311382928 conn(0x7f44d00a99a0 msgr2=0x7f44d00a8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:21.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 shutdown_connections 2024-09-16T15:05:21.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.240+0000 7f44d50d0640 1 -- 172.21.15.134:0/311382928 wait complete. 2024-09-16T15:05:21.282 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:21.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:05:21.283 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:05:21.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 10 monclient: get_monmap_and_config 2024-09-16T15:05:21.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 10 monclient: build_initial_monmap 2024-09-16T15:05:21.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 10 monclient: monmap: 2024-09-16T15:05:21.381 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:21.381102+0000 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:21.381102+0000 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:21.382 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:21.383 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:21.383 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:21.383 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:21.383 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 1 Processor -- start 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 1 -- start start 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.380+0000 7f31b58e9640 10 monclient: init 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient: _add_conns ranks=[1,5,9,8,6,0,4,3,10,7,2] 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f31b0136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31b0137220 0x7f31b0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-h con 0x7f31b0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31b013c370 0x7f31b013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-a con 0x7f31b013c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:21.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b0073250 con 0x7f31b0136e40 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b00a56e0 con 0x7f31b013c370 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31b009abc0 con 0x7f31b0137220 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31b58e9640 10 monclient(hunting): authenticate will time out at 1057.970461s 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31b013c370 0x7f31b013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f31b013c370 0x7f31b013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46354/0 (socket says 172.21.15.134:46354) 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 -- 172.21.15.134:0/148253918 learned_addr learned my addr 172.21.15.134:0/148253918 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): get_auth_request con 0x7f31b013c370 auth_method 0 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31a7fff640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31b0137220 0x7f31b0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient(hunting): handle_auth_done global_id 4797 payload 274 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 -- 172.21.15.134:0/148253918 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31b0137220 msgr2=0x7f31b0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31b0137220 0x7f31b0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 -- 172.21.15.134:0/148253918 >> v1:172.21.15.162:6789/0 conn(0x7f31b0136e40 legacy=0x7f31b009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 monclient: found mon.noname-a 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:21.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:21.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.381+0000 7f31af7fe640 1 -- 172.21.15.134:0/148253918 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b009ba50 con 0x7f31b013c370 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31af7fe640 1 -- 172.21.15.134:0/148253918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31b013c370 msgr2=0x7f31b013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31af7fe640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31b013c370 0x7f31b013e760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e80 tx=0x7f31a802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedbd38488cab25a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 1 -- 172.21.15.134:0/148253918 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31a8007a70 con 0x7f31b013c370 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: dump: 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.394 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:21.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 1 -- 172.21.15.134:0/148253918 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a802d050 con 0x7f31b013c370 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31b58e9640 5 monclient: authenticate success, global_id 4797 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31b58e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31b58e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 1 -- 172.21.15.134:0/148253918 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31a8007bf0 con 0x7f31b013c370 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31b58e9640 4 set_mon_vals no callback set 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.384+0000 7f31ae7fc640 10 monclient: dump: 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.396 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.397 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 10 monclient: shutdown 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 -- 172.21.15.134:0/148253918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31b013c370 msgr2=0x7f31b013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31b013c370 0x7f31b013e760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009e80 tx=0x7f31a802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 -- 172.21.15.134:0/148253918 shutdown_connections 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31b013c370 0x7f31b013e760 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 --2- 172.21.15.134:0/148253918 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f31b0137220 0x7f31b0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 -- 172.21.15.134:0/148253918 >> 172.21.15.134:0/148253918 conn(0x7f31b00a6110 msgr2=0x7f31b00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 -- 172.21.15.134:0/148253918 shutdown_connections 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.385+0000 7f31b58e9640 1 -- 172.21.15.134:0/148253918 wait complete. 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 10 monclient: build_initial_monmap 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 10 monclient: monmap: 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:21.398 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:21.399 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 1 Processor -- start 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 1 -- start start 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 10 monclient: init 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.386+0000 7f31b58e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient: _add_conns ranks=[4,3,2,1,8,7,6,5,0] 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 0x7f31b00a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-c con 0x7f31b0136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b0137220 0x7f31b0135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-g con 0x7f31b0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): picked mon.noname-b con 0x7f31b013c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b01223c0 con 0x7f31b013c370 2024-09-16T15:05:21.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00a75c0 con 0x7f31b0137220 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31b00a7b10 con 0x7f31b0136e40 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31b58e9640 10 monclient(hunting): authenticate will time out at 1057.976094s 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 0x7f31b00a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 0x7f31b00a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:58718/0 (socket says 172.21.15.134:58718) 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34454/0 (socket says 172.21.15.134:34454) 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 -- 172.21.15.134:0/864182170 learned_addr learned my addr 172.21.15.134:0/864182170 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): get_auth_request con 0x7f31b0136e40 auth_method 0 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 10 monclient(hunting): get_auth_request con 0x7f31b013c370 auth_method 0 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b0137220 0x7f31b0135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): get_auth_request con 0x7f31b0137220 auth_method 0 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 msgr2=0x7f31b0136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 msgr2=0x7f31b0136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31af7fe640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient(hunting): handle_auth_done global_id 4711 payload 824 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b0137220 msgr2=0x7f31b0135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b0137220 0x7f31b0135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 msgr2=0x7f31b0136150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b013c370 0x7f31b0136150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:21.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 monclient: found mon.noname-c 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.387+0000 7f31aeffd640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8009b30 con 0x7f31b0136e40 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31aeffd640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 msgr2=0x7f31b00a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31aeffd640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 0x7f31b00a2680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c430 tx=0x7f31a000c8f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31a0011c10 con 0x7f31b0136e40 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: dump: 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.404 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: _finish_auth 0 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: _check_auth_tickets 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a0011d90 con 0x7f31b0136e40 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31b58e9640 5 monclient: authenticate success, global_id 4711 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31b58e9640 10 monclient: _renew_subs 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31b58e9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b01225c0 con 0x7f31b0136e40 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31a0012890 con 0x7f31b0136e40 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:21.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31acff9640 10 monclient: dump: 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:21.407 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.389+0000 7f31adffb640 4 set_mon_vals no callback set 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 10 monclient: _renew_subs 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b00a32a0 con 0x7f31b0136e40 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0071b20 con 0x7f31b0120420 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f31a0012420 con 0x7f31b0136e40 2024-09-16T15:05:21.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120420 auth_method 0 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.390+0000 7f31a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31acff9640 1 --2- 172.21.15.134:0/864182170 >> v2:172.21.15.162:6800/980547263 conn(0x7f31900388b0 0x7f319003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f31a0049fa0 con 0x7f31b0136e40 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31af7fe640 1 --2- 172.21.15.134:0/864182170 >> v2:172.21.15.162:6800/980547263 conn(0x7f31900388b0 0x7f319003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.391+0000 7f31af7fe640 10 monclient: get_auth_request con 0x7f31900388b0 auth_method 0 2024-09-16T15:05:21.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.393+0000 7f31af7fe640 1 --2- 172.21.15.134:0/864182170 >> v2:172.21.15.162:6800/980547263 conn(0x7f31900388b0 0x7f319003ad70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31a8033000 tx=0x7f31a8009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:21.410 INFO:tasks.rados.rados.0.smithi162.stdout:1067: done (4 left) 2024-09-16T15:05:21.410 INFO:tasks.rados.rados.0.smithi162.stdout:1068: done (3 left) 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1069: done (2 left) 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1070: done (1 left) 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1071: done (0 left) 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1072: read oid 23 snap -1 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1072: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1073: rollback oid 37 current snap is 96 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 94 2024-09-16T15:05:21.411 INFO:tasks.rados.rados.0.smithi162.stdout:1074: snap_remove snap 95 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120420 auth_method 0 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.592+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120420 auth_method 0 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:21.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:21.993+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:22.412 INFO:tasks.rados.rados.0.smithi162.stdout:1073: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:05:22.412 INFO:tasks.rados.rados.0.smithi162.stdout:1073: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:05:22.412 INFO:tasks.rados.rados.0.smithi162.stdout:1073: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:05:22.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 457 (ObjNum 405 snap 94 seq_num 405) dirty exists 2024-09-16T15:05:22.413 INFO:tasks.rados.rados.0.smithi162.stdout:1072: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:05:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1072: done (2 left) 2024-09-16T15:05:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1073: done (1 left) 2024-09-16T15:05:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1074: done (0 left) 2024-09-16T15:05:22.567 INFO:tasks.rados.rados.0.smithi162.stdout:1075: delete oid 15 current snap is 96 2024-09-16T15:05:22.571 INFO:tasks.rados.rados.0.smithi162.stdout:1075: done (0 left) 2024-09-16T15:05:22.571 INFO:tasks.rados.rados.0.smithi162.stdout:1076: rollback oid 2 current snap is 96 2024-09-16T15:05:22.572 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 94 2024-09-16T15:05:22.572 INFO:tasks.rados.rados.0.smithi162.stdout:1077: snap_create 2024-09-16T15:05:22.581 INFO:tasks.rados.rados.0.smithi162.stdout:1076: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:05:22.581 INFO:tasks.rados.rados.0.smithi162.stdout:1076: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:05:22.582 INFO:tasks.rados.rados.0.smithi162.stdout:1076: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:05:22.582 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 393 (ObjNum 407 snap 94 seq_num 407) dirty exists 2024-09-16T15:05:22.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:22.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120420 auth_method 0 2024-09-16T15:05:22.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:22.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:22.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:22.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:05:22.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:22.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:22.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:22.794+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:24.032 INFO:tasks.rados.rados.0.smithi162.stdout:1076: done (1 left) 2024-09-16T15:05:24.032 INFO:tasks.rados.rados.0.smithi162.stdout:1077: done (0 left) 2024-09-16T15:05:24.032 INFO:tasks.rados.rados.0.smithi162.stdout:1078: read oid 43 snap -1 2024-09-16T15:05:24.032 INFO:tasks.rados.rados.0.smithi162.stdout:1078: expect (ObjNum 306 snap 64 seq_num 306) 2024-09-16T15:05:24.033 INFO:tasks.rados.rados.0.smithi162.stdout:1079: snap_create 2024-09-16T15:05:24.040 INFO:tasks.rados.rados.0.smithi162.stdout:1078: expect (ObjNum 306 snap 64 seq_num 306) 2024-09-16T15:05:24.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.387+0000 7f31a6ffd640 10 monclient: tick 2024-09-16T15:05:24.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.387+0000 7f31a6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.387+0000 7f31a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.387+0000 7f31a6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120420 auth_method 0 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 10 monclient(hunting): handle_auth_done global_id 44102 payload 824 2024-09-16T15:05:24.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.397+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31b0061ae0 con 0x7f31b0120420 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31980189f0 tx=0x7f3198018eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3198022860 con 0x7f31b0120420 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 10 monclient: dump: 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.401 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f319800c040 con 0x7f31b0120420 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31980189f0 tx=0x7f3198018eb0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 msgr2=0x7f31b002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.399+0000 7f31acff9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120420 0x7f31b002b260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.406+0000 7f31b58e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.406+0000 7f31b58e9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 0x7f31b002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.406+0000 7f31b58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.406+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b0122750 con 0x7f31b0120800 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 0x7f31b002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient: get_auth_request con 0x7f31b0120800 auth_method 0 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 10 monclient(hunting): handle_auth_done global_id 44111 payload 824 2024-09-16T15:05:24.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.407+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f31b007a680 con 0x7f31b0120800 2024-09-16T15:05:24.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31a7fff640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 msgr2=0x7f31b002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31a7fff640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 0x7f31b002b7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f31980189f0 tx=0x7f31980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3198005ea0 con 0x7f31b0120800 2024-09-16T15:05:24.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31acff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31acff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.409+0000 7f31acff9640 10 monclient: dump: 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.412 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f3198029310 con 0x7f31b0120800 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 msgr2=0x7f31b002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 0x7f31b002b7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f31980189f0 tx=0x7f31980046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 msgr2=0x7f31b002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31acff9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31b0120800 0x7f31b002b7a0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f31980189f0 tx=0x7f31980046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:24.414 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.416 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:24.417 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.418 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.419 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:24.421 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:24.422 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.426 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.427 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:24.428 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.429 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.432 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.434 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.435 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:24.436 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:24.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 >> v2:172.21.15.162:6800/980547263 conn(0x7f31900388b0 msgr2=0x7f319003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 --2- 172.21.15.134:0/864182170 >> v2:172.21.15.162:6800/980547263 conn(0x7f31900388b0 0x7f319003ad70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f31a8033000 tx=0x7f31a8009000 comp rx=0 tx=0).stop 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 10 monclient: shutdown 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 msgr2=0x7f31b00a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 --2- 172.21.15.134:0/864182170 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f31b0136e40 0x7f31b00a2680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c430 tx=0x7f31a000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31af7fe640 1 -- 172.21.15.134:0/864182170 reap_dead start 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31af7fe640 1 -- 172.21.15.134:0/864182170 reap_dead start 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 shutdown_connections 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 >> 172.21.15.134:0/864182170 conn(0x7f31b00a6110 msgr2=0x7f31b013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 shutdown_connections 2024-09-16T15:05:24.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.410+0000 7f31b58e9640 1 -- 172.21.15.134:0/864182170 wait complete. 2024-09-16T15:05:24.439 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:24.439 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:05:24.439 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.546+0000 7f8230d83640 10 monclient: get_monmap_and_config 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.546+0000 7f8230d83640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.546+0000 7f8230d83640 10 monclient: monmap: 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:24.547761+0000 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:24.547761+0000 2024-09-16T15:05:24.548 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:24.549 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.547+0000 7f8230d83640 1 Processor -- start 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.547+0000 7f8230d83640 1 -- start start 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.547+0000 7f8230d83640 10 monclient: init 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient: _add_conns ranks=[7,9,8,2,1,6,10,5,3,4,0] 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-i con 0x7f822c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f822c137220 0x7f822c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-a con 0x7f822c137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:24.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f822c135dc0 0x7f822c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-e con 0x7f822c135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c09abc0 con 0x7f822c137220 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f822c09ba50 con 0x7f822c136e40 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0a53c0 con 0x7f822c135dc0 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8230d83640 10 monclient(hunting): authenticate will time out at 1061.137237s 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f822c137220 0x7f822c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f822c135dc0 0x7f822c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f822c137220 0x7f822c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46356/0 (socket says 172.21.15.134:46356) 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 -- 172.21.15.134:0/2721054129 learned_addr learned my addr 172.21.15.134:0/2721054129 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): get_auth_request con 0x7f822c137220 auth_method 0 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): get_auth_request con 0x7f822c135dc0 auth_method 0 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822a59c640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822a59c640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 msgr2=0x7f822c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822a59c640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 msgr2=0x7f822c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822a59c640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient(hunting): handle_auth_done global_id 4806 payload 274 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f822a59c640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f822c135dc0 msgr2=0x7f822c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f822c135dc0 0x7f822c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 msgr2=0x7f822c09c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 monclient: found mon.noname-a 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:24.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.548+0000 7f8229d9b640 1 -- 172.21.15.134:0/2721054129 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c073250 con 0x7f822c137220 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8229d9b640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f822c137220 msgr2=0x7f822c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8229d9b640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f822c137220 0x7f822c137600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d415d79efbaa1aff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 1 -- 172.21.15.134:0/2721054129 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f822002ae00 con 0x7f822c137220 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: dump: 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.563 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 1 -- 172.21.15.134:0/2721054129 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200307b0 con 0x7f822c137220 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8230d83640 5 monclient: authenticate success, global_id 4806 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8230d83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8230d83640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 1 -- 172.21.15.134:0/2721054129 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8220030d90 con 0x7f822c137220 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f8230d83640 4 set_mon_vals no callback set 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.551+0000 7f822959a640 10 monclient: dump: 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.565 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.566 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 10 monclient: shutdown 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 -- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f822c137220 msgr2=0x7f822c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f822c137220 0x7f822c137600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 -- 172.21.15.134:0/2721054129 shutdown_connections 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f822c135dc0 0x7f822c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f822c137220 0x7f822c137600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 --2- 172.21.15.134:0/2721054129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f822c136e40 0x7f822c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 -- 172.21.15.134:0/2721054129 >> 172.21.15.134:0/2721054129 conn(0x7f822c0a6110 msgr2=0x7f822c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 -- 172.21.15.134:0/2721054129 shutdown_connections 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.552+0000 7f8230d83640 1 -- 172.21.15.134:0/2721054129 wait complete. 2024-09-16T15:05:24.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient: monmap: 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.568 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 Processor -- start 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 -- start start 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient: init 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient: _add_conns ranks=[2,5,6,0,3,4,7,1,8] 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 0x7f822c1cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-b con 0x7f822c135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f822c136e40 0x7f822c134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-h con 0x7f822c136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 0x7f822c135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): picked mon.noname-d con 0x7f822c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1cb760 con 0x7f822c135dc0 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f822c125030 con 0x7f822c136e40 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f822c136390 con 0x7f822c137220 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.553+0000 7f8230d83640 10 monclient(hunting): authenticate will time out at 1061.142938s 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 0x7f822c1cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 0x7f822c1cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34522/0 (socket says 172.21.15.134:34522) 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 1 -- 172.21.15.134:0/1784020365 learned_addr learned my addr 172.21.15.134:0/1784020365 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 0x7f822c135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): get_auth_request con 0x7f822c135dc0 auth_method 0 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): get_auth_request con 0x7f822c137220 auth_method 0 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f822c136e40 0x7f822c134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): get_auth_request con 0x7f822c136e40 auth_method 0 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f8229d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient(hunting): handle_auth_done global_id 34305 payload 824 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f822c136e40 msgr2=0x7f822c134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f822c136e40 0x7f822c134ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 msgr2=0x7f822c1cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 0x7f822c1cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 monclient: found mon.noname-d 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200094d0 con 0x7f822c137220 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.554+0000 7f822a59c640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f822c135dc0 0x7f822c1cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 msgr2=0x7f822c135420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f822ad9d640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 0x7f822c135420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f822400e7f0 tx=0x7f822400ecb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8224009590 con 0x7f822c137220 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: dump: 2024-09-16T15:05:24.573 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.574 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8224013470 con 0x7f822c137220 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8224013a50 con 0x7f822c137220 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 5 monclient: authenticate success, global_id 34305 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 10 monclient: _renew_subs 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c0a8a30 con 0x7f822c137220 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8228d99640 4 set_mon_vals no callback set 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:24.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f821affd640 10 monclient: dump: 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.576 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 10 monclient: _renew_subs 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.556+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c1258d0 con 0x7f822c137220 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f8230d83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f8230d83640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 0x7f822c121cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f822c071b20 con 0x7f822c1218f0 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8224013bb0 con 0x7f822c137220 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 0x7f822c121cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient: get_auth_request con 0x7f822c1218f0 auth_method 0 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.557+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f821affd640 1 --2- 172.21.15.134:0/1784020365 >> v2:172.21.15.162:6800/980547263 conn(0x7f820c038880 0x7f820c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f822404a0d0 con 0x7f822c137220 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f822a59c640 10 monclient(hunting): handle_auth_done global_id 4720 payload 824 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f822a59c640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f822c061ae0 con 0x7f822c1218f0 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f8229d9b640 1 --2- 172.21.15.134:0/1784020365 >> v2:172.21.15.162:6800/980547263 conn(0x7f820c038880 0x7f820c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.558+0000 7f8229d9b640 10 monclient: get_auth_request con 0x7f820c038880 auth_method 0 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f822a59c640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 msgr2=0x7f822c121cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f822a59c640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 0x7f822c121cd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e720 tx=0x7f821c00ebe0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f821c00de40 con 0x7f822c1218f0 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 10 monclient: dump: 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.579 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f821c010260 con 0x7f822c1218f0 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 msgr2=0x7f822c121cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 0x7f822c121cd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e720 tx=0x7f821c00ebe0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 msgr2=0x7f822c121cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f821affd640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c1218f0 0x7f822c121cd0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e720 tx=0x7f821c00ebe0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.560+0000 7f8229d9b640 1 --2- 172.21.15.134:0/1784020365 >> v2:172.21.15.162:6800/980547263 conn(0x7f820c038880 0x7f820c03ad40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.566+0000 7f8230d83640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:24.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.566+0000 7f8230d83640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 0x7f822c0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.566+0000 7f8230d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.566+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f822c07a680 con 0x7f822c122210 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 0x7f822c0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient: get_auth_request con 0x7f822c122210 auth_method 0 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 10 monclient(hunting): handle_auth_done global_id 4729 payload 824 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.567+0000 7f822a59c640 1 -- 172.21.15.134:0/1784020365 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f822c09b490 con 0x7f822c122210 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f822a59c640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 msgr2=0x7f822c0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f822a59c640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 0x7f822c0a1a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f821c000f80 tx=0x7f821c004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f821c003db0 con 0x7f822c122210 2024-09-16T15:05:24.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 10 monclient: dump: 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.584 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f821c004030 con 0x7f822c122210 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 msgr2=0x7f822c0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 0x7f822c0a1a20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f821c000f80 tx=0x7f821c004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 msgr2=0x7f822c0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.569+0000 7f821affd640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f822c122210 0x7f822c0a1a20 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f821c000f80 tx=0x7f821c004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:24.586 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.587 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.588 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.589 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.590 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:24.591 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.594 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.595 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:24.596 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.598 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.599 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.600 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.601 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.603 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.606 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.607 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:24.608 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 >> v2:172.21.15.162:6800/980547263 conn(0x7f820c038880 msgr2=0x7f820c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 --2- 172.21.15.134:0/1784020365 >> v2:172.21.15.162:6800/980547263 conn(0x7f820c038880 0x7f820c03ad40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8220009870 tx=0x7f822002e000 comp rx=0 tx=0).stop 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 10 monclient: shutdown 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 msgr2=0x7f822c135420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 --2- 172.21.15.134:0/1784020365 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f822c137220 0x7f822c135420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f822400e7f0 tx=0x7f822400ecb0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 reap_dead start 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f822ad9d640 1 -- 172.21.15.134:0/1784020365 reap_dead start 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 shutdown_connections 2024-09-16T15:05:24.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 >> 172.21.15.134:0/1784020365 conn(0x7f822c0a6110 msgr2=0x7f822c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 shutdown_connections 2024-09-16T15:05:24.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.570+0000 7f8230d83640 1 -- 172.21.15.134:0/1784020365 wait complete. 2024-09-16T15:05:24.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:24.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:05:24.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:05:24.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.708+0000 7fe0031ae640 10 monclient: get_monmap_and_config 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.708+0000 7fe0031ae640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.708+0000 7fe0031ae640 10 monclient: monmap: 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:24.709541+0000 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:24.709541+0000 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.710 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:24.711 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 Processor -- start 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 -- start start 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient: init 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient: _add_conns ranks=[5,6,2,3,8,4,10,7,0,1,9] 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdffc136e40 0x7fdffc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-h con 0x7fdffc136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-d con 0x7fdffc137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 0x7fdffc136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-b con 0x7fdffc135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc09abc0 con 0x7fdffc135dc0 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc09ba50 con 0x7fdffc136e40 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0a53c0 con 0x7fdffc137220 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe0031ae640 10 monclient(hunting): authenticate will time out at 1061.298838s 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fdffbfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fe00174b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 0x7fdffc136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.709+0000 7fdffbfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57900/0 (socket says 172.21.15.134:57900) 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): get_auth_request con 0x7fdffc135dc0 auth_method 0 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 1 -- 172.21.15.134:0/369339200 learned_addr learned my addr 172.21.15.134:0/369339200 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): get_auth_request con 0x7fdffc137220 auth_method 0 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdffc136e40 0x7fdffc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): get_auth_request con 0x7fdffc136e40 auth_method 0 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient(hunting): handle_auth_done global_id 44129 payload 274 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 -- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 msgr2=0x7fdffc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 -- 172.21.15.134:0/369339200 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdffc136e40 msgr2=0x7fdffc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdffc136e40 0x7fdffc09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 monclient: found mon.noname-b 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fe00174b640 1 -- 172.21.15.134:0/369339200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc073250 con 0x7fdffc135dc0 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.710+0000 7fdffbfff640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe00174b640 1 -- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 msgr2=0x7fdffc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe00174b640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 0x7fdffc136220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009dc0 tx=0x7fdff402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a996e85dbe89beac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 1 -- 172.21.15.134:0/369339200 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff4007bc0 con 0x7fdffc135dc0 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: dump: 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.720 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.721 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 5 monclient: authenticate success, global_id 44129 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 1 -- 172.21.15.134:0/369339200 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff402d050 con 0x7fdffc135dc0 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 1 -- 172.21.15.134:0/369339200 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff4007d40 con 0x7fdffc135dc0 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 4 set_mon_vals no callback set 2024-09-16T15:05:24.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fdffb7fe640 10 monclient: dump: 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.723 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 10 monclient: shutdown 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 -- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 msgr2=0x7fdffc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 0x7fdffc136220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009dc0 tx=0x7fdff402a480 comp rx=0 tx=0).stop 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 -- 172.21.15.134:0/369339200 shutdown_connections 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdffc135dc0 0x7fdffc136220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc137220 0x7fdffc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 --2- 172.21.15.134:0/369339200 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdffc136e40 0x7fdffc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 -- 172.21.15.134:0/369339200 >> 172.21.15.134:0/369339200 conn(0x7fdffc0a6110 msgr2=0x7fdffc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 -- 172.21.15.134:0/369339200 shutdown_connections 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.713+0000 7fe0031ae640 1 -- 172.21.15.134:0/369339200 wait complete. 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.714+0000 7fe0031ae640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.714+0000 7fe0031ae640 10 monclient: monmap: 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:24.725 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.714+0000 7fe0031ae640 1 Processor -- start 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.714+0000 7fe0031ae640 1 -- start start 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.714+0000 7fe0031ae640 10 monclient: init 2024-09-16T15:05:24.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient: _add_conns ranks=[8,3,0,1,2,6,7,5,4] 2024-09-16T15:05:24.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-e con 0x7fdffc135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:24.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdffc136e40 0x7fdffc134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-g con 0x7fdffc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 0x7fdffc135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): picked mon.noname-a con 0x7fdffc137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc123660 con 0x7fdffc137220 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc1cb760 con 0x7fdffc136e40 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0a89a0 con 0x7fdffc135dc0 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe0031ae640 10 monclient(hunting): authenticate will time out at 1061.304418s 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 0x7fdffc135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:50636/0 (socket says 172.21.15.134:50636) 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 -- 172.21.15.134:0/1367477453 learned_addr learned my addr 172.21.15.134:0/1367477453 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): get_auth_request con 0x7fdffc135dc0 auth_method 0 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 0x7fdffc135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46370/0 (socket says 172.21.15.134:46370) 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): get_auth_request con 0x7fdffc137220 auth_method 0 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fdffbfff640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdffc136e40 0x7fdffc134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fdffbfff640 10 monclient(hunting): get_auth_request con 0x7fdffc136e40 auth_method 0 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fdffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fdffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fdffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient(hunting): handle_auth_done global_id 44117 payload 824 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdffc136e40 msgr2=0x7fdffc134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdffc136e40 0x7fdffc134ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 msgr2=0x7fdffc135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 0x7fdffc135400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 monclient: found mon.noname-e 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe000f4a640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80094d0 con 0x7fdffc135dc0 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.715+0000 7fe00174b640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdffc137220 0x7fdffc135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe000f4a640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 msgr2=0x7fdffc1cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe000f4a640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e9b0 tx=0x7fdff000ee70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff0009670 con 0x7fdffc135dc0 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: dump: 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.737 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 5 monclient: authenticate success, global_id 44117 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000cbb0 con 0x7fdffc135dc0 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdff0013d90 con 0x7fdffc135dc0 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 10 monclient: _renew_subs 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4009a70 con 0x7fdffc135dc0 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdffaffd640 4 set_mon_vals no callback set 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fdff97fa640 10 monclient: dump: 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.740 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 10 monclient: _renew_subs 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.718+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc136310 con 0x7fdffc135dc0 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe0031ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe000f4a640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e9b0 tx=0x7fdff000ee70 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fdff000cd20 mgrmap(e 5) v1 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe0031ae640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 0x7fdffc121cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc071b20 con 0x7fdffc1218d0 2024-09-16T15:05:24.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdff000cd20 con 0x7fdffc135dc0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 0x7fdffc121cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient: get_auth_request con 0x7fdffc1218d0 auth_method 0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdff97fa640 1 --2- 172.21.15.134:0/1367477453 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfe0038880 0x7fdfe003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 10 monclient(hunting): handle_auth_done global_id 34323 payload 824 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fdffbfff640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdffc061ae0 con 0x7fdffc1218d0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe00174b640 1 --2- 172.21.15.134:0/1367477453 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfe0038880 0x7fdfe003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.719+0000 7fe00174b640 10 monclient: get_auth_request con 0x7fdfe0038880 auth_method 0 2024-09-16T15:05:24.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdffbfff640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 msgr2=0x7fdffc121cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdffbfff640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 0x7fdffc121cb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011800 tx=0x7fdfe8011cc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe800feb0 con 0x7fdffc1218d0 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdff97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdff97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.721+0000 7fdff97fa640 10 monclient: dump: 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.743 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fe00174b640 1 --2- 172.21.15.134:0/1367477453 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfe0038880 0x7fdfe003ad40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009d90 tx=0x7fdff402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdfe800fa40 con 0x7fdffc1218d0 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 msgr2=0x7fdffc121cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 0x7fdffc121cb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011800 tx=0x7fdfe8011cc0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 msgr2=0x7fdffc121cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.722+0000 7fdff97fa640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1218d0 0x7fdffc121cb0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8011800 tx=0x7fdfe8011cc0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fe0031ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fe0031ae640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 0x7fdffc0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fe0031ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc07a680 con 0x7fdffc1221f0 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 0x7fdffc0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient: get_auth_request con 0x7fdffc1221f0 auth_method 0 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 10 monclient(hunting): handle_auth_done global_id 34332 payload 824 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.728+0000 7fdffbfff640 1 -- 172.21.15.134:0/1367477453 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdffc09b490 con 0x7fdffc1221f0 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.730+0000 7fdffbfff640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 msgr2=0x7fdffc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.730+0000 7fdffbfff640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 0x7fdffc0a1a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8008010 tx=0x7fdfe80046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfe8022580 con 0x7fdffc1221f0 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 10 monclient: dump: 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.746 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fdfe80064d0 con 0x7fdffc1221f0 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:24.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 msgr2=0x7fdffc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 0x7fdffc0a1a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8008010 tx=0x7fdfe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 msgr2=0x7fdffc0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdffc1221f0 0x7fdffc0a1a20 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8008010 tx=0x7fdfe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:24.748 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.750 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.751 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:24.752 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.753 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:24.754 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:24.756 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.757 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.761 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.763 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.765 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:24.768 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:24.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:24.770 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfe0038880 msgr2=0x7fdfe003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 --2- 172.21.15.134:0/1367477453 >> v2:172.21.15.162:6800/980547263 conn(0x7fdfe0038880 0x7fdfe003ad40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009d90 tx=0x7fdff402f000 comp rx=0 tx=0).stop 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 10 monclient: shutdown 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 msgr2=0x7fdffc1cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e9b0 tx=0x7fdff000ee70 comp rx=0 tx=0).stop 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe00174b640 1 -- 172.21.15.134:0/1367477453 reap_dead start 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fdff97fa640 1 -- 172.21.15.134:0/1367477453 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 3947+0+0 (unknown 0 0 0) 0x7fdff004a460 con 0x7fdffc135dc0 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 shutdown_connections 2024-09-16T15:05:24.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 --2- 172.21.15.134:0/1367477453 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdffc135dc0 0x7fdffc1cb220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 >> 172.21.15.134:0/1367477453 conn(0x7fdffc0a6110 msgr2=0x7fdffc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 shutdown_connections 2024-09-16T15:05:24.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.731+0000 7fe0031ae640 1 -- 172.21.15.134:0/1367477453 wait complete. 2024-09-16T15:05:24.773 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:24.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:05:24.773 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.869+0000 7f5efebfb640 10 monclient: get_monmap_and_config 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.869+0000 7f5efebfb640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.869+0000 7f5efebfb640 10 monclient: monmap: 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:24.870629+0000 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:24.870629+0000 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:24.871 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:24.872 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 Processor -- start 2024-09-16T15:05:24.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 -- start start 2024-09-16T15:05:24.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient: init 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient: _add_conns ranks=[6,0,9,5,8,10,4,7,3,1,2] 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-d con 0x7f5ef8136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ef8137290 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5ef813c450 0x7f5ef813e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-a con 0x7f5ef813c450 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef8073250 con 0x7f5ef8137290 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef80a5560 con 0x7f5ef813c450 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef809b240 con 0x7f5ef8136eb0 2024-09-16T15:05:24.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.870+0000 7f5efebfb640 10 monclient(hunting): authenticate will time out at 1061.459961s 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5ef813c450 0x7f5ef813e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57920/0 (socket says 172.21.15.134:57920) 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 1 -- 172.21.15.134:0/771751341 learned_addr learned my addr 172.21.15.134:0/771751341 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): get_auth_request con 0x7f5ef8136eb0 auth_method 0 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5ef7fff640 1 --1- 172.21.15.134:0/771751341 >> v1:172.21.15.134:6789/0 conn(0x7f5ef8137290 0x7f5ef8137690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41512/0 (socket says 172.21.15.134:41512) 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): get_auth_request con 0x7f5ef813c450 auth_method 0 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient(hunting): handle_auth_done global_id 4824 payload 274 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 -- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 msgr2=0x7f5ef809bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 -- 172.21.15.134:0/771751341 >> v1:172.21.15.134:6789/0 conn(0x7f5ef8137290 legacy=0x7f5ef8137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:24.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 monclient: found mon.noname-a 2024-09-16T15:05:24.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:24.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efd198640 1 -- 172.21.15.134:0/771751341 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef809b520 con 0x7f5ef813c450 2024-09-16T15:05:24.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.871+0000 7f5efc997640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:24.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efd198640 1 -- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 msgr2=0x7f5ef813e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efd198640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef813e840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009d90 tx=0x7f5ef002a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b80bfb75554bc98 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 1 -- 172.21.15.134:0/771751341 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ef0007bc0 con 0x7f5ef813c450 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: dump: 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.885 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 1 -- 172.21.15.134:0/771751341 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef002d050 con 0x7f5ef813c450 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 5 monclient: authenticate success, global_id 4824 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:24.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 1 -- 172.21.15.134:0/771751341 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ef0007d40 con 0x7f5ef813c450 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 4 set_mon_vals no callback set 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5ef77fe640 10 monclient: dump: 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.887 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 10 monclient: shutdown 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 -- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 msgr2=0x7f5ef813e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef813e840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009d90 tx=0x7f5ef002a4e0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 -- 172.21.15.134:0/771751341 shutdown_connections 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef813e840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 --2- 172.21.15.134:0/771751341 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef809bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 -- 172.21.15.134:0/771751341 >> 172.21.15.134:0/771751341 conn(0x7f5ef80a99a0 msgr2=0x7f5ef80a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 -- 172.21.15.134:0/771751341 shutdown_connections 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.874+0000 7f5efebfb640 1 -- 172.21.15.134:0/771751341 wait complete. 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.875+0000 7f5efebfb640 10 monclient: build_initial_monmap 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.875+0000 7f5efebfb640 10 monclient: monmap: 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:24.889 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.875+0000 7f5efebfb640 1 Processor -- start 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.875+0000 7f5efebfb640 1 -- start start 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.875+0000 7f5efebfb640 10 monclient: init 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:24.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient: _add_conns ranks=[6,5,0,8,3,7,4,1,2] 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-d con 0x7f5ef8136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5ef8137290 0x7f5ef8120110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-h con 0x7f5ef8137290 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef8120650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): picked mon.noname-a con 0x7f5ef813c450 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef81224d0 con 0x7f5ef813c450 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef813b440 con 0x7f5ef8137290 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef813b940 con 0x7f5ef8136eb0 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:24.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efebfb640 10 monclient(hunting): authenticate will time out at 1061.465485s 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57922/0 (socket says 172.21.15.134:57922) 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef8120650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 -- 172.21.15.134:0/246317965 learned_addr learned my addr 172.21.15.134:0/246317965 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): get_auth_request con 0x7f5ef8136eb0 auth_method 0 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): get_auth_request con 0x7f5ef813c450 auth_method 0 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient(hunting): handle_auth_done global_id 34350 payload 824 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5ef8137290 msgr2=0x7f5ef8120110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5ef8137290 0x7f5ef8120110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 msgr2=0x7f5ef8120650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef8120650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient: _finish_hunting 0 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 monclient: found mon.noname-d 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:24.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efc997640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0094d0 con 0x7f5ef8136eb0 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.876+0000 7f5efd198640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ef813c450 0x7f5ef8120650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efc997640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 msgr2=0x7f5ef811fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efc997640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00ea00 tx=0x7f5eec00eec0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5eec00cc70 con 0x7f5ef8136eb0 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: dump: 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.894 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: _finish_auth 0 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:05:24.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec00cdf0 con 0x7f5ef8136eb0 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 5 monclient: authenticate success, global_id 34350 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 10 monclient: _renew_subs 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef0009a70 con 0x7f5ef8136eb0 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5eec013d50 con 0x7f5ef8136eb0 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef57fa640 10 monclient: dump: 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.896 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.897 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5ef6ffd640 4 set_mon_vals no callback set 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 10 monclient: _renew_subs 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.879+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef80a3310 con 0x7f5ef8136eb0 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efebfb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efebfb640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 0x7f5ef802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8073880 con 0x7f5ef8135f50 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5eec013470 con 0x7f5ef8136eb0 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 0x7f5ef802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient: get_auth_request con 0x7f5ef8135f50 auth_method 0 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef57fa640 1 --2- 172.21.15.134:0/246317965 >> v2:172.21.15.162:6800/980547263 conn(0x7f5edc038880 0x7f5edc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f5eec04a590 con 0x7f5ef8136eb0 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_done global_id 44126 payload 824 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5ef7fff640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ef8061ae0 con 0x7f5ef8135f50 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efd198640 1 --2- 172.21.15.134:0/246317965 >> v2:172.21.15.162:6800/980547263 conn(0x7f5edc038880 0x7f5edc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.880+0000 7f5efd198640 10 monclient: get_auth_request con 0x7f5edc038880 auth_method 0 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef7fff640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 msgr2=0x7f5ef802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef7fff640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 0x7f5ef802b260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a39f0 tx=0x7f5ee400c7a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ee40075d0 con 0x7f5ef8135f50 2024-09-16T15:05:24.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.882+0000 7f5ef57fa640 10 monclient: dump: 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.900 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5ee40057f0 con 0x7f5ef8135f50 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 msgr2=0x7f5ef802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 0x7f5ef802b260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a39f0 tx=0x7f5ee400c7a0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 msgr2=0x7f5ef802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5ef57fa640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8135f50 0x7f5ef802b260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.883+0000 7f5efd198640 1 --2- 172.21.15.134:0/246317965 >> v2:172.21.15.162:6800/980547263 conn(0x7f5edc038880 0x7f5edc03ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a67d0 tx=0x7f5ef002f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5efebfb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:24.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5efebfb640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 0x7f5ef802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:24.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5efebfb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:24.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8071b20 con 0x7f5ef8136330 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 0x7f5ef802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient: get_auth_request con 0x7f5ef8136330 auth_method 0 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 10 monclient(hunting): handle_auth_done global_id 44135 payload 824 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.889+0000 7f5ef7fff640 1 -- 172.21.15.134:0/246317965 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ef807a680 con 0x7f5ef8136330 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.891+0000 7f5ef7fff640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 msgr2=0x7f5ef802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.891+0000 7f5ef7fff640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 0x7f5ef802b7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4014000 tx=0x7f5ee4004930 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5ee401f070 con 0x7f5ef8136330 2024-09-16T15:05:24.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 10 monclient: dump: 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:24.905 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f5ee4012610 con 0x7f5ef8136330 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 msgr2=0x7f5ef802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 0x7f5ef802b7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4014000 tx=0x7f5ee4004930 comp rx=0 tx=0).stop 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 msgr2=0x7f5ef802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5ef57fa640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5ef8136330 0x7f5ef802b7a0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4014000 tx=0x7f5ee4004930 comp rx=0 tx=0).stop 2024-09-16T15:05:24.906 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:24.907 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.908 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.909 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:24.910 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:24.911 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:24.914 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.916 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.917 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:24.918 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.919 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.920 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.921 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.922 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.923 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.924 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:24.925 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.927 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:24.928 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 >> v2:172.21.15.162:6800/980547263 conn(0x7f5edc038880 msgr2=0x7f5edc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 --2- 172.21.15.134:0/246317965 >> v2:172.21.15.162:6800/980547263 conn(0x7f5edc038880 0x7f5edc03ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80a67d0 tx=0x7f5ef002f000 comp rx=0 tx=0).stop 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 10 monclient: shutdown 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 msgr2=0x7f5ef811fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efd198640 1 -- 172.21.15.134:0/246317965 reap_dead start 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00ea00 tx=0x7f5eec00eec0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 shutdown_connections 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 --2- 172.21.15.134:0/246317965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5ef8136eb0 0x7f5ef811fbd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:24.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 >> 172.21.15.134:0/246317965 conn(0x7f5ef80a99a0 msgr2=0x7f5ef813a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:24.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 shutdown_connections 2024-09-16T15:05:24.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:24.892+0000 7f5efebfb640 1 -- 172.21.15.134:0/246317965 wait complete. 2024-09-16T15:05:24.931 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:24.931 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:05:24.931 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.027+0000 7fc7cbf5a640 10 monclient: get_monmap_and_config 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.027+0000 7fc7cbf5a640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.027+0000 7fc7cbf5a640 10 monclient: monmap: 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:25.028548+0000 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:25.028548+0000 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:25.029 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:25.030 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 Processor -- start 2024-09-16T15:05:25.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 -- start start 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient: init 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient: _add_conns ranks=[2,4,7,3,5,8,9,10,6,1,0] 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-b con 0x7fc7c4133f10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 0x7fc7c4134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-c con 0x7fc7c4134830 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 0x7fc7c41351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-i con 0x7fc7c4071bd0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40a3d50 con 0x7fc7c4133f10 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c406f630 con 0x7fc7c4134830 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40a1e10 con 0x7fc7c4071bd0 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7cbf5a640 10 monclient(hunting): authenticate will time out at 1061.617871s 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7c9cf6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.028+0000 7fc7c94f5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 0x7fc7c4134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34528/0 (socket says 172.21.15.134:34528) 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 1 -- 172.21.15.134:0/2202538784 learned_addr learned my addr 172.21.15.134:0/2202538784 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 0x7fc7c41351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request con 0x7fc7c4133f10 auth_method 0 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): get_auth_request con 0x7fc7c4134830 auth_method 0 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): get_auth_request con 0x7fc7c4071bd0 auth_method 0 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_done global_id 4738 payload 274 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 -- 172.21.15.134:0/2202538784 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 msgr2=0x7fc7c41351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 0x7fc7c41351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 -- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 msgr2=0x7fc7c41342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 monclient: found mon.noname-c 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:25.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c94f5640 1 -- 172.21.15.134:0/2202538784 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4073250 con 0x7fc7c4134830 2024-09-16T15:05:25.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.029+0000 7fc7ca4f7640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 0x7fc7c41351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c94f5640 1 -- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 msgr2=0x7fc7c4134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 0x7fc7c4134cb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40097d0 tx=0x7fc7b4029fb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=bcf84149a92f931 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 1 -- 172.21.15.134:0/2202538784 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b402ad10 con 0x7fc7c4134830 2024-09-16T15:05:25.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: dump: 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.045 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 1 -- 172.21.15.134:0/2202538784 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b402ae90 con 0x7fc7c4134830 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 1 -- 172.21.15.134:0/2202538784 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b4030c20 con 0x7fc7c4134830 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7cbf5a640 5 monclient: authenticate success, global_id 4738 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7cbf5a640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:25.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7cbf5a640 4 set_mon_vals no callback set 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.032+0000 7fc7c8cf4640 10 monclient: dump: 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.047 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 10 monclient: shutdown 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 msgr2=0x7fc7c4134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 0x7fc7c4134cb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40097d0 tx=0x7fc7b4029fb0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/2202538784 shutdown_connections 2024-09-16T15:05:25.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4071bd0 0x7fc7c41351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4134830 0x7fc7c4134cb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/2202538784 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7c4133f10 0x7fc7c41342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/2202538784 >> 172.21.15.134:0/2202538784 conn(0x7fc7c40a7410 msgr2=0x7fc7c40a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/2202538784 shutdown_connections 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.033+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/2202538784 wait complete. 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.034+0000 7fc7cbf5a640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.034+0000 7fc7cbf5a640 10 monclient: monmap: 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.049 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.034+0000 7fc7cbf5a640 1 Processor -- start 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.034+0000 7fc7cbf5a640 1 -- start start 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.034+0000 7fc7cbf5a640 10 monclient: init 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient: _add_conns ranks=[8,4,7,5,0,1,6,2,3] 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc7c4071bd0 0x7fc7c40a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-e con 0x7fc7c4071bd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 0x7fc7c4132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-c con 0x7fc7c4133f10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4134830 0x7fc7c41332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): picked mon.noname-i con 0x7fc7c4134830 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c4120170 con 0x7fc7c4133f10 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c409dd80 con 0x7fc7c4134830 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40a6320 con 0x7fc7c4071bd0 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7cbf5a640 10 monclient(hunting): authenticate will time out at 1061.624170s 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 0x7fc7c4132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc7c4071bd0 0x7fc7c40a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 0x7fc7c4132d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:58738/0 (socket says 172.21.15.134:58738) 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 -- 172.21.15.134:0/471841146 learned_addr learned my addr 172.21.15.134:0/471841146 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): get_auth_request con 0x7fc7c4133f10 auth_method 0 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request con 0x7fc7c4071bd0 auth_method 0 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4134830 0x7fc7c41332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): get_auth_request con 0x7fc7c4134830 auth_method 0 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7ca4f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient(hunting): handle_auth_done global_id 4747 payload 824 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc7c4071bd0 msgr2=0x7fc7c40a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc7c4071bd0 0x7fc7c40a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4134830 msgr2=0x7fc7c41332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc7c4134830 0x7fc7c41332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 monclient: found mon.noname-c 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c94f5640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc009a70 con 0x7fc7c4133f10 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.035+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc7c4071bd0 0x7fc7c40a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.037+0000 7fc7c94f5640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 msgr2=0x7fc7c4132d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.037+0000 7fc7c94f5640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 0x7fc7c4132d70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40096f0 tx=0x7fc7b40046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.037+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b4004500 con 0x7fc7c4133f10 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.037+0000 7fc7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: dump: 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.055 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4003d60 con 0x7fc7c4133f10 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 5 monclient: authenticate success, global_id 4747 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7b4032d20 con 0x7fc7c4133f10 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 monclient: _renew_subs 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c00094d0 con 0x7fc7c4133f10 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7bbfff640 4 set_mon_vals no callback set 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 10 monclient: dump: 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.057 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 monclient: _renew_subs 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:25.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4009400 con 0x7fc7c4133f10 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc7b4039070 con 0x7fc7c4133f10 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 0x7fc7c411da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.038+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c4073880 con 0x7fc7c411efa0 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7ba7fc640 1 --2- 172.21.15.134:0/471841146 >> v2:172.21.15.162:6800/980547263 conn(0x7fc7a8038880 0x7fc7a803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fc7b406a6c0 con 0x7fc7c4133f10 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 0x7fc7c411da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient: get_auth_request con 0x7fc7c411efa0 auth_method 0 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7ca4f7640 1 --2- 172.21.15.134:0/471841146 >> v2:172.21.15.162:6800/980547263 conn(0x7fc7a8038880 0x7fc7a803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7ca4f7640 10 monclient: get_auth_request con 0x7fc7a8038880 auth_method 0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_done global_id 44101 payload 824 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.039+0000 7fc7c9cf6640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7c4061ae0 con 0x7fc7c411efa0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7c9cf6640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 msgr2=0x7fc7c411da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 0x7fc7c411da60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009f80 tx=0x7fc7c000f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7ca4f7640 1 --2- 172.21.15.134:0/471841146 >> v2:172.21.15.162:6800/980547263 conn(0x7fc7a8038880 0x7fc7a803ad40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7c000fee0 con 0x7fc7c411efa0 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7ba7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.041+0000 7fc7ba7fc640 10 monclient: dump: 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.062 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc7c0014860 con 0x7fc7c411efa0 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 msgr2=0x7fc7c411da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 0x7fc7c411da60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009f80 tx=0x7fc7c000f610 comp rx=0 tx=0).stop 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 msgr2=0x7fc7c411da60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.042+0000 7fc7ba7fc640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411efa0 0x7fc7c411da60 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009f80 tx=0x7fc7c000f610 comp rx=0 tx=0).stop 2024-09-16T15:05:25.064 INFO:tasks.rados.rados.0.smithi162.stdout:1078: done (1 left) 2024-09-16T15:05:25.065 INFO:tasks.rados.rados.0.smithi162.stdout:1079: done (0 left) 2024-09-16T15:05:25.065 INFO:tasks.rados.rados.0.smithi162.stdout:1080: copy_from oid 32 from oid 31 current snap is 98 2024-09-16T15:05:25.065 INFO:tasks.rados.rados.0.smithi162.stdout:1081: write oid 12 current snap is 98 2024-09-16T15:05:25.065 INFO:tasks.rados.rados.0.smithi162.stdout:1081: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7cbf5a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 0x7fc7c411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7cbf5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c406a3f0 con 0x7fc7c411dfa0 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 0x7fc7c411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient: get_auth_request con 0x7fc7c411dfa0 auth_method 0 2024-09-16T15:05:25.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.048+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.049+0000 7fc7c9cf6640 10 monclient(hunting): handle_auth_done global_id 44110 payload 824 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.049+0000 7fc7c9cf6640 1 -- 172.21.15.134:0/471841146 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc7c407a680 con 0x7fc7c411dfa0 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7c9cf6640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 msgr2=0x7fc7c411e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7c9cf6640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 0x7fc7c411e3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009f80 tx=0x7fc7c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7c0004490 con 0x7fc7c411dfa0 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7ba7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7ba7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:25.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.051+0000 7fc7ba7fc640 10 monclient: dump: 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.068 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fc7c0026070 con 0x7fc7c411dfa0 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 msgr2=0x7fc7c411e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 0x7fc7c411e3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009f80 tx=0x7fc7c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 msgr2=0x7fc7c411e3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ba7fc640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7c411dfa0 0x7fc7c411e3b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.069 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:25.070 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.071 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.072 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:25.073 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.074 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.076 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.077 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:25.078 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.081 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:25.084 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.086 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.088 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.092 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.092 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.092 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:25.092 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 >> v2:172.21.15.162:6800/980547263 conn(0x7fc7a8038880 msgr2=0x7fc7a803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/471841146 >> v2:172.21.15.162:6800/980547263 conn(0x7fc7a8038880 0x7fc7a803ad40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009fd0 tx=0x7fc7bc00b040 comp rx=0 tx=0).stop 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 10 monclient: shutdown 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 msgr2=0x7fc7c4132d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 --2- 172.21.15.134:0/471841146 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc7c4133f10 0x7fc7c4132d70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40096f0 tx=0x7fc7b40046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ca4f7640 1 -- 172.21.15.134:0/471841146 reap_dead start 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7ca4f7640 1 -- 172.21.15.134:0/471841146 reap_dead start 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 shutdown_connections 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 >> 172.21.15.134:0/471841146 conn(0x7fc7c40a7410 msgr2=0x7fc7c41b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 shutdown_connections 2024-09-16T15:05:25.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.052+0000 7fc7cbf5a640 1 -- 172.21.15.134:0/471841146 wait complete. 2024-09-16T15:05:25.095 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:25.095 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:05:25.095 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:05:25.096 INFO:tasks.rados.rados.0.smithi162.stdout:1081: writing smithi16234619-12 from 574905 to 1286302 tid 1 2024-09-16T15:05:25.096 INFO:tasks.rados.rados.0.smithi162.stdout:1081: writing smithi16234619-12 from 1885086 to 2410226 tid 2 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1081: writing smithi16234619-12 from 2930189 to 2974905 tid 3 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1082: delete oid 3 current snap is 98 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1082: done (2 left) 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1083: read oid 36 snap 94 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1083: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1084: write oid 10 current snap is 98 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1084: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1084: writing smithi16234619-10 from 608519 to 1070414 tid 1 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1084: writing smithi16234619-10 from 1602050 to 2130859 tid 2 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1084: writing smithi16234619-10 from 2585035 to 3008519 tid 3 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1085: copy_from oid 28 from oid 38 current snap is 98 2024-09-16T15:05:25.097 INFO:tasks.rados.rados.0.smithi162.stdout:1083: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:1080: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:1085: finishing copy_from to smithi16234619-28 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:1085: got expected ENOENT (src dne) 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:1085: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:1080: finishing copy_from to smithi16234619-32 2024-09-16T15:05:25.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: finishing write tid 1 to smithi16234619-10 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1080: done (4 left) 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1083: done (3 left) 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1085: done (2 left) 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1086: read oid 6 snap -1 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1086: expect deleted 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1087: delete oid 22 current snap is 98 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: finishing write tid 2 to smithi16234619-10 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: finishing write tid 3 to smithi16234619-10 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: finishing write tid 4 to smithi16234619-10 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: finishing write tid 5 to smithi16234619-10 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: oid 10 updating version 0 to 396 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: oid 10 updating version 396 to 398 2024-09-16T15:05:25.184 INFO:tasks.rados.rados.0.smithi162.stdout:1084: oid 10 version 398 is already newer than 397 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1084: oid 10 version 398 is already newer than 395 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 398 (ObjNum 417 snap 98 seq_num 417) dirty exists 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1084: left oid 10 (ObjNum 417 snap 98 seq_num 417) 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: finishing write tid 1 to smithi16234619-12 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: finishing write tid 2 to smithi16234619-12 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: finishing write tid 3 to smithi16234619-12 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: finishing write tid 4 to smithi16234619-12 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: finishing write tid 5 to smithi16234619-12 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: oid 12 updating version 0 to 208 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: oid 12 version 208 is already newer than 207 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: oid 12 version 208 is already newer than 206 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: oid 12 updating version 208 to 209 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 209 (ObjNum 415 snap 98 seq_num 415) dirty exists 2024-09-16T15:05:25.185 INFO:tasks.rados.rados.0.smithi162.stdout:1081: left oid 12 (ObjNum 415 snap 98 seq_num 415) 2024-09-16T15:05:25.186 INFO:tasks.rados.rados.0.smithi162.stdout:1081: done (3 left) 2024-09-16T15:05:25.186 INFO:tasks.rados.rados.0.smithi162.stdout:1084: done (2 left) 2024-09-16T15:05:25.186 INFO:tasks.rados.rados.0.smithi162.stdout:1086: done (1 left) 2024-09-16T15:05:25.186 INFO:tasks.rados.rados.0.smithi162.stdout:1087: done (0 left) 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:1088: rollback oid 20 current snap is 98 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 96 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:1089: copy_from oid 27 from oid 25 current snap is 98 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:1090: read oid 44 snap -1 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:1090: expect deleted 2024-09-16T15:05:25.187 INFO:tasks.rados.rados.0.smithi162.stdout:1091: snap_remove snap 94 2024-09-16T15:05:25.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.201+0000 7f31276b1640 10 monclient: get_monmap_and_config 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.201+0000 7f31276b1640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.201+0000 7f31276b1640 10 monclient: monmap: 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:25.202678+0000 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:25.202678+0000 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:25.204 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.202+0000 7f31276b1640 1 Processor -- start 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.202+0000 7f31276b1640 1 -- start start 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.202+0000 7f31276b1640 10 monclient: init 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient: _add_conns ranks=[7,8,10,5,9,2,3,0,1,6,4] 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3120136e40 0x7f312009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-i con 0x7f3120136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-e con 0x7f3120137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3120135dc0 0x7f3120136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-f con 0x7f3120135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312009abc0 con 0x7f3120135dc0 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f312009ba50 con 0x7f3120136e40 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31200a53c0 con 0x7f3120137220 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f31276b1640 10 monclient(hunting): authenticate will time out at 1061.792139s 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:50676/0 (socket says 172.21.15.134:50676) 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 learned_addr learned my addr 172.21.15.134:0/2980681550 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): get_auth_request con 0x7f3120137220 auth_method 0 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3120136e40 0x7f312009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): get_auth_request con 0x7f3120136e40 auth_method 0 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2980681550 >> v2:172.21.15.162:3300/0 conn(0x7f3120135dc0 0x7f3120136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): get_auth_request con 0x7f3120135dc0 auth_method 0 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient(hunting): handle_auth_done global_id 44153 payload 274 2024-09-16T15:05:25.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3120136e40 msgr2=0x7f312009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3120136e40 0x7f312009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> v2:172.21.15.162:3300/0 conn(0x7f3120135dc0 msgr2=0x7f3120136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> v2:172.21.15.162:3300/0 conn(0x7f3120135dc0 0x7f3120136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 monclient: found mon.noname-e 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.203+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120073250 con 0x7f3120137220 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.204+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 msgr2=0x7f3120137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send send error: (32) Broken pipe 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.204+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f311c0098a0 tx=0x7f311c02a0d0 comp rx=0 tx=0).write compression request write failed r=-32 ((32) Broken pipe) 2024-09-16T15:05:25.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.204+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient: get_auth_request con 0x7f3120137220 auth_method 0 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 0 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 msgr2=0x7f3120137600 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120137220 0x7f3120137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient: _add_conns ranks=[1,9,3,6,5,2,4,0] 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f310c003630 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 --2- 172.21.15.134:0/2980681550 >> v2:172.21.15.134:3300/0 conn(0x7f310c0074b0 0x7f310c009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): picked mon.noname-a con 0x7f310c0074b0 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310c00a280 0x7f310c00c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): picked mon.noname-g con 0x7f310c00a280 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c00d0c0 con 0x7f310c003630 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c00e2e0 con 0x7f310c0074b0 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c00e470 con 0x7f310c00a280 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3117fff640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> v2:172.21.15.134:3300/0 conn(0x7f310c0074b0 0x7f310c009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): get_auth_request con 0x7f310c0074b0 auth_method 0 2024-09-16T15:05:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient(hunting): handle_auth_done global_id 44153 payload 293 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310c00a280 msgr2=0x7f310c00c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310c00a280 0x7f310c00c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f310c00a280 0x7f310c00c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> v1:172.21.15.162:6789/0 conn(0x7f310c003630 legacy=0x7f310c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 monclient: found mon.noname-a 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3125c4e640 1 -- 172.21.15.134:0/2980681550 reap_dead start 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.405+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c00e6f0 con 0x7f310c0074b0 2024-09-16T15:05:25.412 INFO:tasks.rados.rados.0.smithi162.stdout:1091: done (3 left) 2024-09-16T15:05:25.412 INFO:tasks.rados.rados.0.smithi162.stdout:1092: read oid 26 snap -1 2024-09-16T15:05:25.412 INFO:tasks.rados.rados.0.smithi162.stdout:1092: expect (ObjNum 375 snap 87 seq_num 375) 2024-09-16T15:05:25.412 INFO:tasks.rados.rados.0.smithi162.stdout:1093: write (excl) oid 43 current snap is 98 2024-09-16T15:05:25.412 INFO:tasks.rados.rados.0.smithi162.stdout:1093: seq_num 419 ranges {642133=728746,2119014=532478} 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3124c4c640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 msgr2=0x7f310c009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f310c009970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f311c02afd0 tx=0x7f311c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cbb8db0a7d340b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f311c0044a0 con 0x7f310c0074b0 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: dump: 2024-09-16T15:05:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c032830 con 0x7f310c0074b0 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f31276b1640 5 monclient: authenticate success, global_id 44153 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f31276b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 1 -- 172.21.15.134:0/2980681550 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f311c032e10 con 0x7f310c0074b0 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f31276b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f31276b1640 4 set_mon_vals no callback set 2024-09-16T15:05:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.407+0000 7f3117fff640 10 monclient: dump: 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 10 monclient: shutdown 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 -- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 msgr2=0x7f310c009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f310c009970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f311c02afd0 tx=0x7f311c009240 comp rx=0 tx=0).stop 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 -- 172.21.15.134:0/2980681550 shutdown_connections 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 --2- 172.21.15.134:0/2980681550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f310c009970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 -- 172.21.15.134:0/2980681550 >> 172.21.15.134:0/2980681550 conn(0x7f31200a6110 msgr2=0x7f31200a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 -- 172.21.15.134:0/2980681550 shutdown_connections 2024-09-16T15:05:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.408+0000 7f31276b1640 1 -- 172.21.15.134:0/2980681550 wait complete. 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.409+0000 7f31276b1640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.409+0000 7f31276b1640 10 monclient: monmap: 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.409+0000 7f31276b1640 1 Processor -- start 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.409+0000 7f31276b1640 1 -- start start 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.409+0000 7f31276b1640 10 monclient: init 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient: _add_conns ranks=[0,6,8,7,3,2,1,4,5] 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f31201cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-a con 0x7f310c0074b0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 0x7f31201353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-d con 0x7f3120135010 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120121a50 0x7f3120121ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): picked mon.noname-e con 0x7f3120121a50 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201cb7f0 con 0x7f310c0074b0 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f31200a8b70 con 0x7f3120135010 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f31201364c0 con 0x7f3120121a50 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f31276b1640 10 monclient(hunting): authenticate will time out at 1061.999262s 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f31201cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 0x7f31201353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120121a50 0x7f3120121ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f31201cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46404/0 (socket says 172.21.15.134:46404) 2024-09-16T15:05:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 1 -- 172.21.15.134:0/2726341914 learned_addr learned my addr 172.21.15.134:0/2726341914 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): get_auth_request con 0x7f310c0074b0 auth_method 0 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): get_auth_request con 0x7f3120121a50 auth_method 0 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): get_auth_request con 0x7f3120135010 auth_method 0 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f312544d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient(hunting): handle_auth_done global_id 34359 payload 824 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120121a50 msgr2=0x7f3120121ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120121a50 0x7f3120121ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 msgr2=0x7f31201cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f310c0074b0 0x7f31201cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 monclient: found mon.noname-d 2024-09-16T15:05:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3124c4c640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120123630 con 0x7f3120135010 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.410+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3120121a50 0x7f3120121ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.413+0000 7f3124c4c640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 msgr2=0x7f31201353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.413+0000 7f3124c4c640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 0x7f31201353f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f31200a5500 tx=0x7f311c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f311c02a710 con 0x7f3120135010 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: dump: 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.430 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f31276b1640 5 monclient: authenticate success, global_id 34359 2024-09-16T15:05:25.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f31276b1640 10 monclient: _renew_subs 2024-09-16T15:05:25.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f31276b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:25.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3120136a10 con 0x7f3120135010 2024-09-16T15:05:25.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c0044e0 con 0x7f3120135010 2024-09-16T15:05:25.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f311c003c90 con 0x7f3120135010 2024-09-16T15:05:25.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f3115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.415+0000 7f3115ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:25.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.415+0000 7f3115ffb640 10 monclient: dump: 2024-09-16T15:05:25.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.433 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.434 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.435 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.415+0000 7f31276b1640 10 monclient: _renew_subs 2024-09-16T15:05:25.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.415+0000 7f31276b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:25.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.415+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3120125a00 con 0x7f3120135010 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1093: writing smithi16234619-43 from 642133 to 1370879 tid 1 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1093: writing smithi16234619-43 from 2119014 to 2651492 tid 2 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1094: delete oid 47 current snap is 98 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1088: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1088: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1088: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 178 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1089: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1089: finishing copy_from to smithi16234619-27 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 246 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:05:25.437 INFO:tasks.rados.rados.0.smithi162.stdout:1092: expect (ObjNum 375 snap 87 seq_num 375) 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.414+0000 7f31177fe640 4 set_mon_vals no callback set 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f311c031040 con 0x7f3120135010 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f3115ffb640 1 --2- 172.21.15.134:0/2726341914 >> v2:172.21.15.162:6800/980547263 conn(0x7f3104038880 0x7f310403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 3987+0+0 (secure 0 0 0) 0x7f311c044030 con 0x7f3120135010 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f31276b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f31276b1640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 0x7f3120122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.416+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120122950 con 0x7f312002c3e0 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 0x7f3120122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f312544d640 1 --2- 172.21.15.134:0/2726341914 >> v2:172.21.15.162:6800/980547263 conn(0x7f3104038880 0x7f310403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f312544d640 10 monclient: get_auth_request con 0x7f3104038880 auth_method 0 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f3125c4e640 10 monclient: get_auth_request con 0x7f312002c3e0 auth_method 0 2024-09-16T15:05:25.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f3125c4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f3125c4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.417+0000 7f3125c4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.418+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.418+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.418+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.418+0000 7f3125c4e640 10 monclient(hunting): handle_auth_done global_id 34374 payload 824 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.418+0000 7f3125c4e640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31200a8b70 con 0x7f312002c3e0 2024-09-16T15:05:25.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f312544d640 1 --2- 172.21.15.134:0/2726341914 >> v2:172.21.15.162:6800/980547263 conn(0x7f3104038880 0x7f310403ad40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3110009b60 tx=0x7f3110009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3125c4e640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 msgr2=0x7f3120122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 0x7f3120122410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f312006b430 tx=0x7f31180128f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3118017db0 con 0x7f312002c3e0 2024-09-16T15:05:25.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3115ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:05:25.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.420+0000 7f3115ffb640 10 monclient: dump: 2024-09-16T15:05:25.445 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.445 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.445 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.445 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.446 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.447 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.447 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.447 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.447 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.447 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3118011040 con 0x7f312002c3e0 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 msgr2=0x7f3120122410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 0x7f3120122410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f312006b430 tx=0x7f31180128f0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 msgr2=0x7f3120122410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.421+0000 7f3115ffb640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f312002c3e0 0x7f3120122410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.427+0000 7f31276b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:25.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.427+0000 7f31276b1640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 0x7f3120061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.427+0000 7f31276b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.427+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120071b20 con 0x7f31200610b0 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 0x7f3120061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient: get_auth_request con 0x7f31200610b0 auth_method 0 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 10 monclient(hunting): handle_auth_done global_id 34383 payload 824 2024-09-16T15:05:25.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.428+0000 7f3125c4e640 1 -- 172.21.15.134:0/2726341914 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3120061ae0 con 0x7f31200610b0 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3125c4e640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 msgr2=0x7f3120061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3125c4e640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 0x7f3120061490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3118000f80 tx=0x7f3118004ab0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3118023300 con 0x7f31200610b0 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3115ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3115ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.430+0000 7f3115ffb640 10 monclient: dump: 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f31180040c0 con 0x7f31200610b0 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:25.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 msgr2=0x7f3120061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 0x7f3120061490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3118000f80 tx=0x7f3118004ab0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 msgr2=0x7f3120061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.431+0000 7f3115ffb640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31200610b0 0x7f3120061490 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3118000f80 tx=0x7f3118004ab0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:25.453 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:25.454 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.455 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:25.456 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:25.457 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:25.458 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.459 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.460 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.461 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.463 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.465 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.466 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:25.466 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:25.466 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.469 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.472 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.473 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.474 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:25.475 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.476 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.477 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.478 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.479 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.479 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:25.479 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 >> v2:172.21.15.162:6800/980547263 conn(0x7f3104038880 msgr2=0x7f310403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 --2- 172.21.15.134:0/2726341914 >> v2:172.21.15.162:6800/980547263 conn(0x7f3104038880 0x7f310403ad40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3110009b60 tx=0x7f3110009190 comp rx=0 tx=0).stop 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 10 monclient: shutdown 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 msgr2=0x7f31201353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f3125c4e640 1 -- 172.21.15.134:0/2726341914 reap_dead start 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 0x7f31201353f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f31200a5500 tx=0x7f311c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 shutdown_connections 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 --2- 172.21.15.134:0/2726341914 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3120135010 0x7f31201353f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 >> 172.21.15.134:0/2726341914 conn(0x7f31200a6110 msgr2=0x7f312009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 shutdown_connections 2024-09-16T15:05:25.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.432+0000 7f31276b1640 1 -- 172.21.15.134:0/2726341914 wait complete. 2024-09-16T15:05:25.482 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:25.482 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:05:25.482 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: finishing write tid 1 to smithi16234619-43 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: finishing write tid 2 to smithi16234619-43 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: finishing write tid 3 to smithi16234619-43 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: finishing write tid 4 to smithi16234619-43 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: oid 43 updating version 0 to 105 2024-09-16T15:05:25.536 INFO:tasks.rados.rados.0.smithi162.stdout:1093: oid 43 updating version 105 to 106 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1093: oid 43 version 106 is already newer than 104 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 106 (ObjNum 419 snap 98 seq_num 419) dirty exists 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1093: left oid 43 (ObjNum 419 snap 98 seq_num 419) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1088: done (5 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1089: done (4 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1090: done (3 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1092: done (2 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1093: done (1 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1094: done (0 left) 2024-09-16T15:05:25.537 INFO:tasks.rados.rados.0.smithi162.stdout:1095: delete oid 32 current snap is 98 2024-09-16T15:05:25.538 INFO:tasks.rados.rados.0.smithi162.stdout:1095: done (0 left) 2024-09-16T15:05:25.538 INFO:tasks.rados.rados.0.smithi162.stdout:1096: rollback oid 34 current snap is 98 2024-09-16T15:05:25.538 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 96 2024-09-16T15:05:25.538 INFO:tasks.rados.rados.0.smithi162.stdout:1097: write oid 18 current snap is 98 2024-09-16T15:05:25.538 INFO:tasks.rados.rados.0.smithi162.stdout:1097: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2024-09-16T15:05:25.544 INFO:tasks.rados.rados.0.smithi162.stdout:1097: writing smithi16234619-18 from 692554 to 1247047 tid 1 2024-09-16T15:05:25.551 INFO:tasks.rados.rados.0.smithi162.stdout:1097: writing smithi16234619-18 from 1778107 to 2457912 tid 2 2024-09-16T15:05:25.554 INFO:tasks.rados.rados.0.smithi162.stdout:1097: writing smithi16234619-18 from 2889444 to 3092554 tid 3 2024-09-16T15:05:25.554 INFO:tasks.rados.rados.0.smithi162.stdout:1098: snap_remove snap 97 2024-09-16T15:05:25.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.583+0000 7ffa82272640 10 monclient: get_monmap_and_config 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.583+0000 7ffa82272640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.583+0000 7ffa82272640 10 monclient: monmap: 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:25.584573+0000 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:25.584573+0000 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.585 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:25.586 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.584+0000 7ffa82272640 1 Processor -- start 2024-09-16T15:05:25.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.584+0000 7ffa82272640 1 -- start start 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.584+0000 7ffa82272640 10 monclient: init 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient: _add_conns ranks=[9,5,6,0,2,7,4,10,1,8,3] 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-a con 0x7ffa7c0a3940 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 0x7ffa7c0a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-h con 0x7ffa7c09ffb0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 0x7ffa7c06a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-d con 0x7ffa7c069e50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:25.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c09d820 con 0x7ffa7c0a3940 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a3d20 con 0x7ffa7c09ffb0 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c09dae0 con 0x7ffa7c069e50 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa82272640 10 monclient(hunting): authenticate will time out at 1062.174122s 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 0x7ffa7c06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46416/0 (socket says 172.21.15.134:46416) 2024-09-16T15:05:25.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 1 -- 172.21.15.134:0/1199425296 learned_addr learned my addr 172.21.15.134:0/1199425296 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request con 0x7ffa7c0a3940 auth_method 0 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): get_auth_request con 0x7ffa7c069e50 auth_method 0 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 0x7ffa7c0a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 10 monclient(hunting): get_auth_request con 0x7ffa7c09ffb0 auth_method 0 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient(hunting): handle_auth_done global_id 34368 payload 274 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 -- 172.21.15.134:0/1199425296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 msgr2=0x7ffa7c0a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 0x7ffa7c0a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 -- 172.21.15.134:0/1199425296 >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 msgr2=0x7ffa7c134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 --2- 172.21.15.134:0/1199425296 >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 monclient: found mon.noname-d 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa8080f640 1 -- 172.21.15.134:0/1199425296 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c073250 con 0x7ffa7c069e50 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa73fff640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 0x7ffa7c0a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.585+0000 7ffa7bfff640 1 --2- 172.21.15.134:0/1199425296 >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa8080f640 1 -- 172.21.15.134:0/1199425296 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 msgr2=0x7ffa7c06a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa8080f640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 0x7ffa7c06a300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009d90 tx=0x7ffa7402a490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=35cc305789c37a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 1 -- 172.21.15.134:0/1199425296 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa74007bc0 con 0x7ffa7c069e50 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: dump: 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.599 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 1 -- 172.21.15.134:0/1199425296 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7402d050 con 0x7ffa7c069e50 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa82272640 5 monclient: authenticate success, global_id 34368 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa82272640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa82272640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 1 -- 172.21.15.134:0/1199425296 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa74007d40 con 0x7ffa7c069e50 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa82272640 4 set_mon_vals no callback set 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.588+0000 7ffa7b7fe640 10 monclient: dump: 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 10 monclient: shutdown 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 -- 172.21.15.134:0/1199425296 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 msgr2=0x7ffa7c06a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 0x7ffa7c06a300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009d90 tx=0x7ffa7402a490 comp rx=0 tx=0).stop 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 -- 172.21.15.134:0/1199425296 shutdown_connections 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c09ffb0 0x7ffa7c0a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 --2- 172.21.15.134:0/1199425296 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ffa7c069e50 0x7ffa7c06a300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 --2- 172.21.15.134:0/1199425296 >> v2:172.21.15.134:3300/0 conn(0x7ffa7c0a3940 0x7ffa7c134ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 -- 172.21.15.134:0/1199425296 >> 172.21.15.134:0/1199425296 conn(0x7ffa7c0a2e20 msgr2=0x7ffa7c0a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 -- 172.21.15.134:0/1199425296 shutdown_connections 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 -- 172.21.15.134:0/1199425296 wait complete. 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 10 monclient: monmap: 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.605 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.589+0000 7ffa82272640 1 Processor -- start 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 -- start start 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient: init 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient: _add_conns ranks=[2,1,5,8,6,0,3,4,7] 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 0x7ffa7c099340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-b con 0x7ffa7c069e50 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c09ffb0 0x7ffa7c099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-f con 0x7ffa7c09ffb0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c0a3940 0x7ffa7c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): picked mon.noname-h con 0x7ffa7c0a3940 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c06a8e0 con 0x7ffa7c09ffb0 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c070a10 con 0x7ffa7c069e50 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c0a2680 con 0x7ffa7c0a3940 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa82272640 10 monclient(hunting): authenticate will time out at 1062.179483s 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 0x7ffa7c099340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 0x7ffa7c099340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34542/0 (socket says 172.21.15.134:34542) 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 -- 172.21.15.134:0/3601115065 learned_addr learned my addr 172.21.15.134:0/3601115065 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request con 0x7ffa7c069e50 auth_method 0 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c0a3940 0x7ffa7c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): get_auth_request con 0x7ffa7c0a3940 auth_method 0 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c09ffb0 0x7ffa7c099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): get_auth_request con 0x7ffa7c09ffb0 auth_method 0 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa8080f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient(hunting): handle_auth_done global_id 44147 payload 824 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c0a3940 msgr2=0x7ffa7c06e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c0a3940 0x7ffa7c06e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c09ffb0 msgr2=0x7ffa7c099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa7c09ffb0 0x7ffa7c099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 monclient: found mon.noname-b 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.590+0000 7ffa7bfff640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0094d0 con 0x7ffa7c069e50 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa7bfff640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 msgr2=0x7ffa7c099340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa7bfff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 0x7ffa7c099340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e9b0 tx=0x7ffa6c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa6c00cbb0 con 0x7ffa7c069e50 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:25.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: dump: 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.616 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c00cd30 con 0x7ffa7c069e50 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 5 monclient: authenticate success, global_id 44147 2024-09-16T15:05:25.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa6c00cea0 con 0x7ffa7c069e50 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa797fa640 10 monclient: dump: 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.618 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa7affd640 4 set_mon_vals no callback set 2024-09-16T15:05:25.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 10 monclient: _renew_subs 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa74009a70 con 0x7ffa7c069e50 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 10 monclient: _renew_subs 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.593+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c071d10 con 0x7ffa7c069e50 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ffa6c013470 con 0x7ffa7c069e50 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa82272640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa82272640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 0x7ffa7c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c06b5d0 con 0x7ffa7c06fcc0 2024-09-16T15:05:25.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa797fa640 1 --2- 172.21.15.134:0/3601115065 >> v2:172.21.15.162:6800/980547263 conn(0x7ffa5c038880 0x7ffa5c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 3987+0+0 (secure 0 0 0) 0x7ffa6c04a110 con 0x7ffa7c069e50 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 0x7ffa7c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient: get_auth_request con 0x7ffa7c06fcc0 auth_method 0 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa8080f640 1 --2- 172.21.15.134:0/3601115065 >> v2:172.21.15.162:6800/980547263 conn(0x7ffa5c038880 0x7ffa5c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa8080f640 10 monclient: get_auth_request con 0x7ffa5c038880 auth_method 0 2024-09-16T15:05:25.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 10 monclient(hunting): handle_auth_done global_id 44141 payload 824 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.594+0000 7ffa73fff640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa7c061ae0 con 0x7ffa7c06fcc0 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa73fff640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 msgr2=0x7ffa7c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa73fff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 0x7ffa7c06e7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64011c90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa8080f640 1 --2- 172.21.15.134:0/3601115065 >> v2:172.21.15.162:6800/980547263 conn(0x7ffa5c038880 0x7ffa5c03ad40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06d810 tx=0x7ffa7402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa6400c070 con 0x7ffa7c06fcc0 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 10 monclient: dump: 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.622 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ffa64011ea0 con 0x7ffa7c06fcc0 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 msgr2=0x7ffa7c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 0x7ffa7c06e7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64011c90 comp rx=0 tx=0).stop 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 msgr2=0x7ffa7c06e7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.597+0000 7ffa797fa640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06fcc0 0x7ffa7c06e7c0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64011c90 comp rx=0 tx=0).stop 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.603+0000 7ffa82272640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.603+0000 7ffa82272640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 0x7ffa7c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.603+0000 7ffa82272640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.603+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c071f00 con 0x7ffa7c06ed00 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 0x7ffa7c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient: get_auth_request con 0x7ffa7c06ed00 auth_method 0 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 10 monclient(hunting): handle_auth_done global_id 44150 payload 824 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.604+0000 7ffa73fff640 1 -- 172.21.15.134:0/3601115065 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa7c07a680 con 0x7ffa7c06ed00 2024-09-16T15:05:25.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.606+0000 7ffa73fff640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 msgr2=0x7ffa7c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.606+0000 7ffa73fff640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 0x7ffa7c06f110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64004730 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa64023040 con 0x7ffa7c06ed00 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 10 monclient: dump: 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.626 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7ffa64004050 con 0x7ffa7c06ed00 2024-09-16T15:05:25.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 msgr2=0x7ffa7c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 0x7ffa7c06f110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64004730 comp rx=0 tx=0).stop 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 msgr2=0x7ffa7c06f110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa797fa640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ffa7c06ed00 0x7ffa7c06f110 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64004730 comp rx=0 tx=0).stop 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:25.628 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:25.629 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.630 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.631 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:25.632 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.633 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:25.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.635 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.636 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.637 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.638 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.639 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.640 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.641 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.642 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.645 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.648 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.653 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:25.654 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:25.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 >> v2:172.21.15.162:6800/980547263 conn(0x7ffa5c038880 msgr2=0x7ffa5c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 --2- 172.21.15.134:0/3601115065 >> v2:172.21.15.162:6800/980547263 conn(0x7ffa5c038880 0x7ffa5c03ad40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06d810 tx=0x7ffa7402f000 comp rx=0 tx=0).stop 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 10 monclient: shutdown 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 msgr2=0x7ffa7c099340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 --2- 172.21.15.134:0/3601115065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa7c069e50 0x7ffa7c099340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e9b0 tx=0x7ffa6c00ee70 comp rx=0 tx=0).stop 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa8080f640 1 -- 172.21.15.134:0/3601115065 reap_dead start 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa8080f640 1 -- 172.21.15.134:0/3601115065 reap_dead start 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 shutdown_connections 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 >> 172.21.15.134:0/3601115065 conn(0x7ffa7c0a2e20 msgr2=0x7ffa7c09df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 shutdown_connections 2024-09-16T15:05:25.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.607+0000 7ffa82272640 1 -- 172.21.15.134:0/3601115065 wait complete. 2024-09-16T15:05:25.657 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:25.657 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:05:25.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.753+0000 7f8cb608f640 10 monclient: get_monmap_and_config 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.753+0000 7f8cb608f640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.753+0000 7f8cb608f640 10 monclient: monmap: 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:25.754514+0000 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:25.754514+0000 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.755 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:25.756 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 Processor -- start 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 -- start start 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient: init 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient: _add_conns ranks=[5,0,3,1,2,9,10,6,8,7,4] 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8cb0136e40 0x7f8cb009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-h con 0x7f8cb0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8cb0137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 0x7f8cb013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-g con 0x7f8cb013c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb0073250 con 0x7f8cb0137220 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a56e0 con 0x7f8cb013c370 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009abc0 con 0x7f8cb0136e40 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8cb608f640 10 monclient(hunting): authenticate will time out at 1062.343804s 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8caeffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8cb0137220 0x7f8cb0137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41514/0 (socket says 172.21.15.134:41514) 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.754+0000 7f8caeffd640 1 -- 172.21.15.134:0/1319939107 learned_addr learned my addr 172.21.15.134:0/1319939107 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 0x7f8cb013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8cb0136e40 0x7f8cb009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): get_auth_request con 0x7f8cb013c370 auth_method 0 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): get_auth_request con 0x7f8cb0136e40 auth_method 0 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient(hunting): handle_auth_done global_id 34392 payload 274 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 -- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8cb0136e40 msgr2=0x7f8cb009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8cb0136e40 0x7f8cb009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 -- 172.21.15.134:0/1319939107 >> v1:172.21.15.134:6789/0 conn(0x7f8cb0137220 legacy=0x7f8cb0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 monclient: found mon.noname-g 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.755+0000 7f8caffff640 1 -- 172.21.15.134:0/1319939107 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb009ba50 con 0x7f8cb013c370 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8caffff640 1 -- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 msgr2=0x7f8cb013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8caffff640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 0x7f8cb013e760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009dc0 tx=0x7f8ca802a4d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=ca99f9d65b5e5bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 1 -- 172.21.15.134:0/1319939107 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca8007bc0 con 0x7f8cb013c370 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: dump: 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.764 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 1 -- 172.21.15.134:0/1319939107 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca802d050 con 0x7f8cb013c370 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 5 monclient: authenticate success, global_id 34392 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 1 -- 172.21.15.134:0/1319939107 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca8007d40 con 0x7f8cb013c370 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 4 set_mon_vals no callback set 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cae7fc640 10 monclient: dump: 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.766 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 10 monclient: shutdown 2024-09-16T15:05:25.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 -- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 msgr2=0x7f8cb013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 0x7f8cb013e760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009dc0 tx=0x7f8ca802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 -- 172.21.15.134:0/1319939107 shutdown_connections 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8cb013c370 0x7f8cb013e760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 --2- 172.21.15.134:0/1319939107 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8cb0136e40 0x7f8cb009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 -- 172.21.15.134:0/1319939107 >> 172.21.15.134:0/1319939107 conn(0x7f8cb00a6110 msgr2=0x7f8cb00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 -- 172.21.15.134:0/1319939107 shutdown_connections 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.758+0000 7f8cb608f640 1 -- 172.21.15.134:0/1319939107 wait complete. 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.759+0000 7f8cb608f640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.759+0000 7f8cb608f640 10 monclient: monmap: 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:25.768 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.759+0000 7f8cb608f640 1 Processor -- start 2024-09-16T15:05:25.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.759+0000 7f8cb608f640 1 -- start start 2024-09-16T15:05:25.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.759+0000 7f8cb608f640 10 monclient: init 2024-09-16T15:05:25.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient: _add_conns ranks=[6,1,0,2,4,5,8,7,3] 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8cb0136e40 0x7f8cb0135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-d con 0x7f8cb0136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cb0137220 0x7f8cb0135db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-f con 0x7f8cb0137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): picked mon.noname-a con 0x7f8cb013c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a7610 con 0x7f8cb013c370 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb01221d0 con 0x7f8cb0137220 2024-09-16T15:05:25.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb00a3400 con 0x7f8cb0136e40 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8cb608f640 10 monclient(hunting): authenticate will time out at 1062.349393s 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8cb0136e40 0x7f8cb0135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46426/0 (socket says 172.21.15.134:46426) 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 learned_addr learned my addr 172.21.15.134:0/4199927133 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): get_auth_request con 0x7f8cb013c370 auth_method 0 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): get_auth_request con 0x7f8cb0136e40 auth_method 0 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cb0137220 0x7f8cb0135db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): get_auth_request con 0x7f8cb0137220 auth_method 0 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient(hunting): handle_auth_done global_id 4860 payload 824 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8cb0136e40 msgr2=0x7f8cb0135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8cb0136e40 0x7f8cb0135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cb0137220 msgr2=0x7f8cb0135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8cb0137220 0x7f8cb0135db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 monclient: found mon.noname-a 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009a70 con 0x7f8cb013c370 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.760+0000 7f8caf7fe640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8cb0136e40 0x7f8cb0135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 msgr2=0x7f8cb01362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8caffff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ca802aac0 tx=0x7f8ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca80044b0 con 0x7f8cb013c370 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: dump: 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.780 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8004650 con 0x7f8cb013c370 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 5 monclient: authenticate success, global_id 4860 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 10 monclient: _renew_subs 2024-09-16T15:05:25.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb0121ef0 con 0x7f8cb013c370 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca80047c0 con 0x7f8cb013c370 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8c97fff640 10 monclient: dump: 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.782 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 10 monclient: _renew_subs 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb00a2aa0 con 0x7f8cb013c370 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.763+0000 7f8cadffb640 4 set_mon_vals no callback set 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8ca8004040 con 0x7f8cb013c370 2024-09-16T15:05:25.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8cb608f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8c97fff640 1 --2- 172.21.15.134:0/4199927133 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c9c0388b0 0x7f8c9c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8cb608f640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 0x7f8cb009e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0071b20 con 0x7f8cb009e520 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 3987+0+0 (secure 0 0 0) 0x7f8ca806b630 con 0x7f8cb013c370 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 0x7f8cb009e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caf7fe640 1 --2- 172.21.15.134:0/4199927133 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c9c0388b0 0x7f8c9c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient: get_auth_request con 0x7f8cb009e520 auth_method 0 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caf7fe640 10 monclient: get_auth_request con 0x7f8c9c0388b0 auth_method 0 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 10 monclient(hunting): handle_auth_done global_id 4678 payload 824 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.764+0000 7f8caeffd640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb0122820 con 0x7f8cb009e520 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.766+0000 7f8caf7fe640 1 --2- 172.21.15.134:0/4199927133 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c9c0388b0 0x7f8c9c03ad70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009eb0 tx=0x7f8ca0009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.766+0000 7f8caeffd640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 msgr2=0x7f8cb009e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.766+0000 7f8caeffd640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 0x7f8cb009e900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400ea70 tx=0x7f8ca400ef30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca400cc50 con 0x7f8cb009e520 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 10 monclient: dump: 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8ca400cdb0 con 0x7f8cb009e520 2024-09-16T15:05:25.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 msgr2=0x7f8cb009e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 0x7f8cb009e900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400ea70 tx=0x7f8ca400ef30 comp rx=0 tx=0).stop 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 msgr2=0x7f8cb009e900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.767+0000 7f8c97fff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009e520 0x7f8cb009e900 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400ea70 tx=0x7f8ca400ef30 comp rx=0 tx=0).stop 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8cb608f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8cb608f640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 0x7f8cb011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8cb608f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0122080 con 0x7f8cb009ee40 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 0x7f8cb011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient: get_auth_request con 0x7f8cb009ee40 auth_method 0 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 10 monclient(hunting): handle_auth_done global_id 4687 payload 824 2024-09-16T15:05:25.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.773+0000 7f8caeffd640 1 -- 172.21.15.134:0/4199927133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8cb0061ae0 con 0x7f8cb009ee40 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8caeffd640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 msgr2=0x7f8cb011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8caeffd640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 0x7f8cb011fc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4017000 tx=0x7f8ca4004940 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8ca4015cc0 con 0x7f8cb009ee40 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 10 monclient: dump: 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.789 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f8ca4015850 con 0x7f8cb009ee40 2024-09-16T15:05:25.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 msgr2=0x7f8cb011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 0x7f8cb011fc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4017000 tx=0x7f8ca4004940 comp rx=0 tx=0).stop 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 msgr2=0x7f8cb011fc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8c97fff640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8cb009ee40 0x7f8cb011fc60 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4017000 tx=0x7f8ca4004940 comp rx=0 tx=0).stop 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:25.791 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:05:25.792 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.793 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:25.795 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:25.796 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.798 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.799 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.800 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.801 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.802 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.803 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.805 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:25.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:25.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:25.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:25.812 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:25.813 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:25.814 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:25.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c9c0388b0 msgr2=0x7f8c9c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 --2- 172.21.15.134:0/4199927133 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c9c0388b0 0x7f8c9c03ad70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009eb0 tx=0x7f8ca0009190 comp rx=0 tx=0).stop 2024-09-16T15:05:25.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 10 monclient: shutdown 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 msgr2=0x7f8cb01362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ca802aac0 tx=0x7f8ca8004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8caffff640 1 -- 172.21.15.134:0/4199927133 reap_dead start 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 shutdown_connections 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 --2- 172.21.15.134:0/4199927133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cb013c370 0x7f8cb01362f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.776+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 >> 172.21.15.134:0/4199927133 conn(0x7f8cb00a6110 msgr2=0x7f8cb006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.777+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 shutdown_connections 2024-09-16T15:05:25.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.777+0000 7f8cb608f640 1 -- 172.21.15.134:0/4199927133 wait complete. 2024-09-16T15:05:25.816 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:25.817 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:05:25.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.912+0000 7f50a11bc640 10 monclient: get_monmap_and_config 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.912+0000 7f50a11bc640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.912+0000 7f50a11bc640 10 monclient: monmap: 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:25.913603+0000 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:25.913603+0000 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:25.914 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:25.915 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 Processor -- start 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 -- start start 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient: init 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient: _add_conns ranks=[7,5,0,2,1,9,10,8,3,6,4] 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 0x7f509c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-i con 0x7f509c136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 0x7f509c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-h con 0x7f509c09bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f509c136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:25.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c073250 con 0x7f509c136c60 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f509c09b600 con 0x7f509c09bfe0 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a5740 con 0x7f509c136030 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f50a11bc640 10 monclient(hunting): authenticate will time out at 1062.502781s 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509b59e640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f509c136c60 0x7f509c1370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41524/0 (socket says 172.21.15.134:41524) 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509b59e640 1 -- 172.21.15.134:0/3451316137 learned_addr learned my addr 172.21.15.134:0/3451316137 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509a59c640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 0x7f509c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509ad9d640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 0x7f509c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509a59c640 10 monclient(hunting): get_auth_request con 0x7f509c09bfe0 auth_method 0 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509ad9d640 10 monclient(hunting): get_auth_request con 0x7f509c136030 auth_method 0 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.913+0000 7f509ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient(hunting): handle_auth_done global_id 44168 payload 274 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 1 -- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 msgr2=0x7f509c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 0x7f509c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 1 -- 172.21.15.134:0/3451316137 >> v1:172.21.15.134:6789/0 conn(0x7f509c136c60 legacy=0x7f509c1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 1 monclient: found mon.noname-h 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509a59c640 1 -- 172.21.15.134:0/3451316137 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0a52c0 con 0x7f509c09bfe0 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.914+0000 7f509ad9d640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 0x7f509c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.916+0000 7f509a59c640 1 -- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 msgr2=0x7f509c09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.916+0000 7f509a59c640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 0x7f509c09c400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50900097d0 tx=0x7f509002a000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=d301ce716b2d2a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 1 -- 172.21.15.134:0/3451316137 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f509002ad20 con 0x7f509c09bfe0 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: dump: 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.927 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 1 -- 172.21.15.134:0/3451316137 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50900307b0 con 0x7f509c09bfe0 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 1 -- 172.21.15.134:0/3451316137 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f509002aea0 con 0x7f509c09bfe0 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f5099d9b640 10 monclient: dump: 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.929 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 5 monclient: authenticate success, global_id 44168 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 4 set_mon_vals no callback set 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 10 monclient: shutdown 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:25.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 -- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 msgr2=0x7f509c09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 0x7f509c09c400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50900097d0 tx=0x7f509002a000 comp rx=0 tx=0).stop 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 -- 172.21.15.134:0/3451316137 shutdown_connections 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f509c09bfe0 0x7f509c09c400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3451316137 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136030 0x7f509c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 -- 172.21.15.134:0/3451316137 >> 172.21.15.134:0/3451316137 conn(0x7f509c0a9ea0 msgr2=0x7f509c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 -- 172.21.15.134:0/3451316137 shutdown_connections 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.917+0000 7f50a11bc640 1 -- 172.21.15.134:0/3451316137 wait complete. 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.918+0000 7f50a11bc640 10 monclient: build_initial_monmap 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.918+0000 7f50a11bc640 10 monclient: monmap: 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:25.931 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.918+0000 7f50a11bc640 1 Processor -- start 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 -- start start 2024-09-16T15:05:25.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient: init 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient: _add_conns ranks=[2,1,7,5,8,4,6,3,0] 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-b con 0x7f509c09bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f509c136030 0x7f509c1cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-f con 0x7f509c136030 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136c60 0x7f509c1cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): picked mon.noname-i con 0x7f509c136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c120a80 con 0x7f509c136030 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a8290 con 0x7f509c09bfe0 2024-09-16T15:05:25.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a7df0 con 0x7f509c136c60 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f50a11bc640 10 monclient(hunting): authenticate will time out at 1062.508497s 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34548/0 (socket says 172.21.15.134:34548) 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 -- 172.21.15.134:0/3066577604 learned_addr learned my addr 172.21.15.134:0/3066577604 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): get_auth_request con 0x7f509c09bfe0 auth_method 0 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f509c136030 0x7f509c1cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136c60 0x7f509c1cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): get_auth_request con 0x7f509c136030 auth_method 0 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): get_auth_request con 0x7f509c136c60 auth_method 0 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient(hunting): handle_auth_done global_id 44156 payload 824 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 -- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136c60 msgr2=0x7f509c1cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136c60 0x7f509c1cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 -- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f509c136030 msgr2=0x7f509c1cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f509c136030 0x7f509c1cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 monclient: found mon.noname-b 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:25.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.919+0000 7f509ad9d640 1 -- 172.21.15.134:0/3066577604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009400 con 0x7f509c09bfe0 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f509ad9d640 1 -- 172.21.15.134:0/3066577604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 msgr2=0x7f509c134ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f509ad9d640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f508c00ebe0 tx=0x7f508c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f508c009760 con 0x7f509c09bfe0 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: dump: 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.937 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 5 monclient: authenticate success, global_id 44156 2024-09-16T15:05:25.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00ce40 con 0x7f509c09bfe0 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f508c00eee0 con 0x7f509c09bfe0 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 10 monclient: _renew_subs 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5090009400 con 0x7f509c09bfe0 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f509959a640 4 set_mon_vals no callback set 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50837fe640 10 monclient: dump: 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:25.939 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 10 monclient: _renew_subs 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.922+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c135810 con 0x7f509c09bfe0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f508c013470 con 0x7f509c09bfe0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50837fe640 1 --2- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 0x7f508803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 3987+0+0 (secure 0 0 0) 0x7f508c04a500 con 0x7f509c09bfe0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50a11bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50a11bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50a11bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c122430 con 0x7f509c09bfe0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f509a59c640 1 --2- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 0x7f508803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.923+0000 7f509a59c640 10 monclient: get_auth_request con 0x7f50880388b0 auth_method 0 2024-09-16T15:05:25.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.926+0000 7f509a59c640 1 --2- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 0x7f508803ad70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5090031000 tx=0x7f509002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:25.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.928+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f508c017080 con 0x7f509c09bfe0 2024-09-16T15:05:25.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.928+0000 7f50837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:25.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:25.928+0000 7f50837fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:26.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.156+0000 7f50a11bc640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:05:26.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.156+0000 7f50a11bc640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:05:26.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.156+0000 7f50a11bc640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:26.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.156+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f509c09c400 con 0x7f509c09bfe0 2024-09-16T15:05:26.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.158+0000 7f50837fe640 1 -- 172.21.15.134:0/3066577604 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f508c01b770 con 0x7f509c09bfe0 2024-09-16T15:05:26.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.158+0000 7f50837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:05:26.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.158+0000 7f50837fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.158+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=10,config=2,health=11,kv=12,logm=65} crc {auth=3671718148,config=2586702516,health=1647462659,kv=1090497770,logm=1392283165}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=2019528823,config=2586702516,health=1535303502,kv=1090497770,logm=2903879612}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=2019528823,config=2586702516,health=1535303502,kv=1090497770,logm=533456924}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=11,config=2,health=11,kv=11,logm=65} crc {auth=2019528823,config=2586702516,health=3542941888,kv=1302766305,logm=3646241477}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=11,config=2,health=12,kv=11,logm=64} crc {auth=2019528823,config=2586702516,health=1535303502,kv=2676201184,logm=259169483}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=10,health=11,kv=10,logm=69} crc {auth=3491424031,health=1096256309,kv=1031078199,logm=1343820318}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=11,config=2,health=12,kv=12,logm=63} crc {auth=2019528823,config=2586702516,health=1535303502,kv=1090497770,logm=2396897406}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=10,config=2,health=11,kv=10,logm=67} crc {auth=2956965071,config=2586702516,health=1096256309,kv=1283996808,logm=2877499126}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.159+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=10,config=2,health=12,kv=12,logm=64} crc {auth=97393872,config=2586702516,health=1535303502,kv=1090497770,logm=1469270907}) 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=745206601}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2322519864}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1159013401}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1854901635}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3921835548}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2609612937}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2575861200}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3735240848}) 2024-09-16T15:05:26.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.160+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4047061664}) 2024-09-16T15:05:26.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 msgr2=0x7f508803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:26.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 0x7f508803ad70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5090031000 tx=0x7f509002e000 comp rx=0 tx=0).stop 2024-09-16T15:05:26.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 10 monclient: shutdown 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 msgr2=0x7f509c134ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f508c00ebe0 tx=0x7f508c00c520 comp rx=0 tx=0).stop 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 shutdown_connections 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> v2:172.21.15.162:6800/980547263 conn(0x7f50880388b0 0x7f508803ad70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f509c136c60 0x7f509c1cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f509c136030 0x7f509c1cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 --2- 172.21.15.134:0/3066577604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f509c09bfe0 0x7f509c134ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.161+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 >> 172.21.15.134:0/3066577604 conn(0x7f509c0a9ea0 msgr2=0x7f509c06be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 shutdown_connections 2024-09-16T15:05:26.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f50a11bc640 1 -- 172.21.15.134:0/3066577604 wait complete. 2024-09-16T15:05:26.179 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.179 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.179 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4269259707}) 2024-09-16T15:05:26.179 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1518770330}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2357285920}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2662444010}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1654971715}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=221185200}) 2024-09-16T15:05:26.180 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3969737395}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2879596050}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.162+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=289015908}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=528573584}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2848635636}) 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.181 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4119337881}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3553624830}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4009824815}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3092565813}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2699509230}) 2024-09-16T15:05:26.182 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.163+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.164+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=214813690}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.164+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3231197338}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3113175774}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3475544024}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=427173983}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3724236275}) 2024-09-16T15:05:26.183 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=199215655}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=165597482}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3648104624}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=772663963}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.165+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3760233347}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.184 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3179022571}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3930137862}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1213129122}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3522948901}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1534698102}) 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.185 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3655590232}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.166+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3719093753}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.167+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.167+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=406741310}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.167+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165976802}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1665572083}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=700407860}) 2024-09-16T15:05:26.186 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1783996331}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1080645712}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=614633657}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1859121542}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.187 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=939849805}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=545232894}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.168+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2909683687}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2144364102}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2557071381}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.188 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2190783911}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.169+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3172051618}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=252985081}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3160351067}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3119582584}) 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.189 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2149548714}) 2024-09-16T15:05:26.190 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.170+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1891379440}) 2024-09-16T15:05:26.190 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1911470531}) 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=48944665}) 2024-09-16T15:05:26.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3329741874}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=891788924}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811615086}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1717327677}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=837023506}) 2024-09-16T15:05:26.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3109868203}) 2024-09-16T15:05:26.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.248+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=775720708}) 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3869394046}) 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=905870240}) 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3623724558}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1586870612}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=738331686}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2207287551}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3671514437}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1463039909}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.249+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=492713210}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4104425991}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2021991607}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=567472918}) 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1079268182}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.250+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3942345166}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=288315738}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2816651129}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1088424375}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.251+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3576146476}) 2024-09-16T15:05:26.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2276025178}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600625416}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2838871584}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=168610709}) 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3431219339}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2588848227}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2235298010}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006259782}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.252+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3998173388}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=8} crc {logm=1386408922,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4174070643,mon_config_key=351612246}) 2024-09-16T15:05:26.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8} crc {logm=3779597440,mdsmap=1531151339,mgr=1802989191,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2067745025,mon_config_key=351612246}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=68,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=10} crc {logm=4216041786,mdsmap=3585761405,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4174070643,mon_config_key=2117612215}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=9} crc {logm=3334203438,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgrstat=4174070643,mon_config_key=3834705962}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=9} crc {logm=596283916,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2541634014,mon_config_key=2736598628}) 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=69,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=11} crc {logm=3284654935,mdsmap=1531151339,mgr=2460804549,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3492803976,mon_config_key=4155428396}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=67,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=9} crc {logm=4016837513,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4174070644,mon_config_key=3798191598}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=11} crc {logm=1757342135,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2698597462,mon_config_key=4155428396}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.253+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=8} crc {logm=2317960685,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4174070643,mon_config_key=351612246}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=40,osdmap=44} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=3471750090,osdmap=4072570942}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=42} crc {mon_config_key=804276461,monmap=2838206317,osd_metadata=2617104446,osd_pg_creating=2123140811,osd_snap=4211882876,osdmap=1189080024}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=42,osdmap=43} crc {mon_config_key=524852399,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=2905970097,osdmap=1011237936}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=41} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=2503814208,osdmap=4069705025}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=42} crc {mon_config_key=2518386621,monmap=2838206317,osd_metadata=764497481,osd_pg_creating=2123140811,osd_snap=3120394156,osdmap=2373879517}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=41} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=3110136761,osdmap=1991139004}) 2024-09-16T15:05:26.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=43,osdmap=44} crc {mon_config_key=1011971200,monmap=3140514009,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=715583899,osdmap=1881943371}) 2024-09-16T15:05:26.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=5,osd_snap=43,osdmap=43} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=134110247,osd_snap=1877970537,osdmap=3148867741}) 2024-09-16T15:05:26.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.255+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=39,osdmap=46} crc {mon_config_key=804276461,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2123140811,osd_snap=1902123826,osdmap=456030843}) 2024-09-16T15:05:26.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=830198369}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3734013462}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=319705583}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1703327581}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=493302459}) 2024-09-16T15:05:26.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3111894259}) 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2124447265}) 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2324679381}) 2024-09-16T15:05:26.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.409+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2614713430}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2437655038}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3603074873}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4124613103}) 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2519119556}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3603074873}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=902741686}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2407448128}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3007521734}) 2024-09-16T15:05:26.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.413+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=109951226}) 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1098: done (2 left) 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1099: read oid 15 snap 96 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1099: expect deleted 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1100: rollback oid 33 current snap is 98 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 96 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1096: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1096: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1096: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 353 (ObjNum 404 snap 94 seq_num 404) dirty exists 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1097: finishing write tid 1 to smithi16234619-18 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1097: finishing write tid 2 to smithi16234619-18 2024-09-16T15:05:26.419 INFO:tasks.rados.rados.0.smithi162.stdout:1097: finishing write tid 3 to smithi16234619-18 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1096: done (3 left) 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1101: snap_create 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: finishing write tid 4 to smithi16234619-18 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: finishing write tid 5 to smithi16234619-18 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: oid 18 updating version 0 to 243 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: oid 18 updating version 243 to 244 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: oid 18 version 244 is already newer than 242 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: oid 18 updating version 244 to 245 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 245 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1097: left oid 18 (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:1100: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:05:26.420 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:05:26.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=89} crc {osdmap=2316700124}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=90} crc {osdmap=1476383408}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=88} crc {osdmap=4174012294}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=92} crc {osdmap=1966186969}) 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=90} crc {osdmap=1476383408}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=94} crc {osdmap=575220595}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=93} crc {osdmap=3039479418}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=89} crc {osdmap=4006820618}) 2024-09-16T15:05:26.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:05:26.484+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=91} crc {osdmap=745633338}) 2024-09-16T15:05:27.192 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:05:27.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:27.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.290+0000 7f42fddc7640 10 monclient: get_monmap_and_config 2024-09-16T15:05:27.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.290+0000 7f42fddc7640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.291+0000 7f42fddc7640 10 monclient: monmap: 2024-09-16T15:05:27.292 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.292 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:27.291937+0000 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:27.291937+0000 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.293 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.294 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.294 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:27.294 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:27.294 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.291+0000 7f42fddc7640 1 Processor -- start 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.291+0000 7f42fddc7640 1 -- start start 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.291+0000 7f42fddc7640 10 monclient: init 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient: _add_conns ranks=[4,0,8,6,9,10,7,5,2,1,3] 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 0x7f42f8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-c con 0x7f42f8136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f42f809bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:27.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 0x7f42f8137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-e con 0x7f42f809c400 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f8073250 con 0x7f42f809bfe0 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42f809c880 con 0x7f42f8136030 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f42f809b600 con 0x7f42f809c400 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42fddc7640 10 monclient(hunting): authenticate will time out at 1063.881231s 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 0x7f42f8137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 0x7f42f8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f6ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f42f809bfe0 0x7f42f8136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42908/0 (socket says 172.21.15.134:42908) 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f6ffd640 1 -- 172.21.15.134:0/149044491 learned_addr learned my addr 172.21.15.134:0/149044491 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:27.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): get_auth_request con 0x7f42f809c400 auth_method 0 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): get_auth_request con 0x7f42f8136030 auth_method 0 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient(hunting): handle_auth_done global_id 44171 payload 274 2024-09-16T15:05:27.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 -- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 msgr2=0x7f42f8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 0x7f42f8136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 -- 172.21.15.134:0/149044491 >> v1:172.21.15.134:6789/0 conn(0x7f42f809bfe0 legacy=0x7f42f8136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 monclient: found mon.noname-e 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f7fff640 1 -- 172.21.15.134:0/149044491 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80a5740 con 0x7f42f809c400 2024-09-16T15:05:27.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.292+0000 7f42f77fe640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 0x7f42f8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:27.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f7fff640 1 -- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 msgr2=0x7f42f8137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f7fff640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 0x7f42f8137370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009dc0 tx=0x7f42f002a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1d6680d7738fe49a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 1 -- 172.21.15.134:0/149044491 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42f0007bc0 con 0x7f42f809c400 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: dump: 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.307 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 1 -- 172.21.15.134:0/149044491 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f002d050 con 0x7f42f809c400 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42fddc7640 5 monclient: authenticate success, global_id 44171 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42fddc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42fddc7640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 1 -- 172.21.15.134:0/149044491 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42f0007d40 con 0x7f42f809c400 2024-09-16T15:05:27.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42fddc7640 4 set_mon_vals no callback set 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.295+0000 7f42f67fc640 10 monclient: dump: 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.309 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 10 monclient: shutdown 2024-09-16T15:05:27.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 -- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 msgr2=0x7f42f8137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 0x7f42f8137370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009dc0 tx=0x7f42f002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 -- 172.21.15.134:0/149044491 shutdown_connections 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42f809c400 0x7f42f8137370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 --2- 172.21.15.134:0/149044491 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42f8136030 0x7f42f8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 -- 172.21.15.134:0/149044491 >> 172.21.15.134:0/149044491 conn(0x7f42f80a9ea0 msgr2=0x7f42f80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 -- 172.21.15.134:0/149044491 shutdown_connections 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 1 -- 172.21.15.134:0/149044491 wait complete. 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.296+0000 7f42fddc7640 10 monclient: monmap: 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:27.311 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 Processor -- start 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 -- start start 2024-09-16T15:05:27.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient: init 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient: _add_conns ranks=[5,2,0,4,3,1,8,6,7] 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f42f809bfe0 0x7f42f81bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-h con 0x7f42f809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-b con 0x7f42f809c400 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): picked mon.noname-a con 0x7f42f8136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80a8b20 con 0x7f42f8136030 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f8121ef0 con 0x7f42f809c400 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42f81bfd10 con 0x7f42f809bfe0 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42fddc7640 10 monclient(hunting): authenticate will time out at 1063.886690s 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48954/0 (socket says 172.21.15.134:48954) 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41106/0 (socket says 172.21.15.134:41106) 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 1 -- 172.21.15.134:0/2956298506 learned_addr learned my addr 172.21.15.134:0/2956298506 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): get_auth_request con 0x7f42f8136030 auth_method 0 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): get_auth_request con 0x7f42f809c400 auth_method 0 2024-09-16T15:05:27.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.297+0000 7f42f77fe640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f42f809bfe0 0x7f42f81bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f77fe640 10 monclient(hunting): get_auth_request con 0x7f42f809bfe0 auth_method 0 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 10 monclient(hunting): handle_auth_done global_id 4869 payload 824 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 -- 172.21.15.134:0/2956298506 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f42f809bfe0 msgr2=0x7f42f81bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f42f809bfe0 0x7f42f81bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 -- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 msgr2=0x7f42f811fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 monclient: found mon.noname-a 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f7fff640 1 -- 172.21.15.134:0/2956298506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8009400 con 0x7f42f8136030 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.298+0000 7f42f6ffd640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42f7fff640 1 -- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 msgr2=0x7f42f811ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42f7fff640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42f002aac0 tx=0x7f42f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42f0007ea0 con 0x7f42f8136030 2024-09-16T15:05:27.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: dump: 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 5 monclient: authenticate success, global_id 4869 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f00044b0 con 0x7f42f8136030 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42f0004620 con 0x7f42f8136030 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 10 monclient: _renew_subs 2024-09-16T15:05:27.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f0009a70 con 0x7f42f8136030 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42dffff640 10 monclient: dump: 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.319 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42f5ffb640 4 set_mon_vals no callback set 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 10 monclient: _renew_subs 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.300+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f809f660 con 0x7f42f8136030 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f42f0004040 con 0x7f42f8136030 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42dffff640 1 --2- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 0x7f42e403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f42f006b080 con 0x7f42f8136030 2024-09-16T15:05:27.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42fddc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42fddc7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42fddc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f8122030 con 0x7f42f8136030 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42f77fe640 1 --2- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 0x7f42e403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.301+0000 7f42f77fe640 10 monclient: get_auth_request con 0x7f42e4038880 auth_method 0 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.304+0000 7f42f77fe640 1 --2- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 0x7f42e403ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009bc0 tx=0x7f42e8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.306+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f42f8122030 con 0x7f42f8136030 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.306+0000 7f42dffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.306+0000 7f42dffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1097: done (3 left) 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1099: done (2 left) 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1100: done (1 left) 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1101: done (0 left) 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1102: write (excl) oid 2 current snap is 99 2024-09-16T15:05:27.418 INFO:tasks.rados.rados.0.smithi162.stdout:1102: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2024-09-16T15:05:27.426 INFO:tasks.rados.rados.0.smithi162.stdout:1102: writing smithi16234619-2 from 709361 to 1339103 tid 1 2024-09-16T15:05:27.432 INFO:tasks.rados.rados.0.smithi162.stdout:1102: writing smithi16234619-2 from 1836589 to 2376405 tid 2 2024-09-16T15:05:27.436 INFO:tasks.rados.rados.0.smithi162.stdout:1102: writing smithi16234619-2 from 3033220 to 3109361 tid 3 2024-09-16T15:05:27.436 INFO:tasks.rados.rados.0.smithi162.stdout:1103: snap_remove snap 98 2024-09-16T15:05:27.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.539+0000 7f42fddc7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:27.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.539+0000 7f42fddc7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:27.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.539+0000 7f42fddc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:27.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.539+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f42f8122030 con 0x7f42f8136030 2024-09-16T15:05:27.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.540+0000 7f42dffff640 1 -- 172.21.15.134:0/2956298506 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f42f8122030 con 0x7f42f8136030 2024-09-16T15:05:27.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.540+0000 7f42dffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:27.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.540+0000 7f42dffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:27.542 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":42,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 msgr2=0x7f42e403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 0x7f42e403ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009bc0 tx=0x7f42e8008040 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 10 monclient: shutdown 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 msgr2=0x7f42f811ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f42f002aac0 tx=0x7f42f0004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 shutdown_connections 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> v2:172.21.15.162:6800/980547263 conn(0x7f42e4038880 0x7f42e403ad40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42f8136030 0x7f42f811ffc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42f809c400 0x7f42f811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 --2- 172.21.15.134:0/2956298506 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f42f809bfe0 0x7f42f81bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 >> 172.21.15.134:0/2956298506 conn(0x7f42f80a9ea0 msgr2=0x7f42f809e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 shutdown_connections 2024-09-16T15:05:27.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.542+0000 7f42fddc7640 1 -- 172.21.15.134:0/2956298506 wait complete. 2024-09-16T15:05:27.554 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:05:27.554 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:05:27.554 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:05:27.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:05:27.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.651+0000 7f7c01661640 10 monclient: get_monmap_and_config 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.651+0000 7f7c01661640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.651+0000 7f7c01661640 10 monclient: monmap: 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:27.652334+0000 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:27.652334+0000 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:27.653 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:27.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.651+0000 7f7c01661640 1 Processor -- start 2024-09-16T15:05:27.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 -- start start 2024-09-16T15:05:27.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient: init 2024-09-16T15:05:27.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient: _add_conns ranks=[3,0,8,2,1,7,10,9,4,5,6] 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-g con 0x7f7bfc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7bfc137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-e con 0x7f7bfc13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc073250 con 0x7f7bfc137220 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0a56e0 con 0x7f7bfc136e40 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc09abc0 con 0x7f7bfc13c370 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7c01661640 10 monclient(hunting): authenticate will time out at 1064.241558s 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfa7fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7bfc137220 0x7f7bfc137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42920/0 (socket says 172.21.15.134:42920) 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/4186013104 learned_addr learned my addr 172.21.15.134:0/4186013104 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): get_auth_request con 0x7f7bfc13c370 auth_method 0 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_done global_id 44180 payload 274 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/4186013104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 msgr2=0x7f7bfc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/4186013104 >> v1:172.21.15.134:6789/0 conn(0x7f7bfc137220 legacy=0x7f7bfc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 monclient: found mon.noname-e 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.652+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/4186013104 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc09ba50 con 0x7f7bfc13c370 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 msgr2=0x7f7bfc13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc13e760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009dc0 tx=0x7f7bf402a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d1cf2a1b28716481 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 1 -- 172.21.15.134:0/4186013104 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf4007ca0 con 0x7f7bfc13c370 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: dump: 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.666 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 1 -- 172.21.15.134:0/4186013104 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf402d050 con 0x7f7bfc13c370 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7c01661640 5 monclient: authenticate success, global_id 44180 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7c01661640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7c01661640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 1 -- 172.21.15.134:0/4186013104 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf4007e20 con 0x7f7bfc13c370 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7c01661640 4 set_mon_vals no callback set 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.655+0000 7f7bf9ffb640 10 monclient: dump: 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.668 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.669 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 10 monclient: shutdown 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 -- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 msgr2=0x7f7bfc13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc13e760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009dc0 tx=0x7f7bf402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 -- 172.21.15.134:0/4186013104 shutdown_connections 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc13e760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 --2- 172.21.15.134:0/4186013104 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 -- 172.21.15.134:0/4186013104 >> 172.21.15.134:0/4186013104 conn(0x7f7bfc0a6110 msgr2=0x7f7bfc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 -- 172.21.15.134:0/4186013104 shutdown_connections 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.656+0000 7f7c01661640 1 -- 172.21.15.134:0/4186013104 wait complete. 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 10 monclient: monmap: 2024-09-16T15:05:27.670 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.671 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 1 Processor -- start 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 1 -- start start 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 10 monclient: init 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 10 monclient: _add_conns ranks=[3,6,8,5,1,0,2,4,7] 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.657+0000 7f7c01661640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-g con 0x7f7bfc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bfc137220 0x7f7bfc136510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-d con 0x7f7bfc137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): picked mon.noname-e con 0x7f7bfc13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0a7b10 con 0x7f7bfc136e40 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc0a75c0 con 0x7f7bfc137220 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc1cc520 con 0x7f7bfc13c370 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7c01661640 10 monclient(hunting): authenticate will time out at 1064.247058s 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bfc137220 0x7f7bfc136510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc11fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58552/0 (socket says 172.21.15.134:58552) 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 learned_addr learned my addr 172.21.15.134:0/416130665 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): get_auth_request con 0x7f7bfc13c370 auth_method 0 2024-09-16T15:05:27.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): get_auth_request con 0x7f7bfc137220 auth_method 0 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): get_auth_request con 0x7f7bfc136e40 auth_method 0 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient(hunting): handle_auth_done global_id 44189 payload 824 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bfc137220 msgr2=0x7f7bfc136510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bfc137220 0x7f7bfc136510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 msgr2=0x7f7bfc135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bfc136e40 0x7f7bfc135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 monclient: found mon.noname-e 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4009a70 con 0x7f7bfc13c370 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.658+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bfc137220 0x7f7bfc136510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 msgr2=0x7f7bfc11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bfb7fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc11fc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009d90 tx=0x7f7bf4004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf40044b0 con 0x7f7bfc13c370 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: dump: 2024-09-16T15:05:27.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4004630 con 0x7f7bfc13c370 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 5 monclient: authenticate success, global_id 44189 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 10 monclient: _renew_subs 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc122980 con 0x7f7bfc13c370 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf40047a0 con 0x7f7bfc13c370 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf37fe640 10 monclient: dump: 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 10 monclient: _renew_subs 2024-09-16T15:05:27.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7bf97fa640 4 set_mon_vals no callback set 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.660+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc1223c0 con 0x7f7bfc13c370 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7bf4004040 con 0x7f7bfc13c370 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7c01661640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7c01661640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 0x7f7bfc120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc071b20 con 0x7f7bfc1201a0 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 0x7f7bfc120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bf37fe640 1 --2- 172.21.15.134:0/416130665 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bdc038880 0x7f7bdc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient: get_auth_request con 0x7f7bfc1201a0 auth_method 0 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f7bf406b130 con 0x7f7bfc13c370 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.661+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.662+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/416130665 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bdc038880 0x7f7bdc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.662+0000 7f7bfa7fc640 10 monclient: get_auth_request con 0x7f7bdc038880 auth_method 0 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.662+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_done global_id 4878 payload 824 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.662+0000 7f7bfaffd640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bfc061ae0 con 0x7f7bfc1201a0 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bfaffd640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 msgr2=0x7f7bfc120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bfaffd640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 0x7f7bfc120580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be400ee70 con 0x7f7bfc1201a0 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 10 monclient: dump: 2024-09-16T15:05:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.683 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/416130665 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bdc038880 0x7f7bdc03ad40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007c90 tx=0x7f7bec007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7be400cbc0 con 0x7f7bfc1201a0 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 msgr2=0x7f7bfc120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 0x7f7bfc120580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be400ec60 comp rx=0 tx=0).stop 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 msgr2=0x7f7bfc120580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:27.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.664+0000 7f7bf37fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc1201a0 0x7f7bfc120580 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be400ec60 comp rx=0 tx=0).stop 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7c01661640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7c01661640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 0x7f7bfc09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7c01661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc122b10 con 0x7f7bfc09d4f0 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 0x7f7bfc09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient: get_auth_request con 0x7f7bfc09d4f0 auth_method 0 2024-09-16T15:05:27.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.670+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.671+0000 7f7bfaffd640 10 monclient(hunting): handle_auth_done global_id 4887 payload 824 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.671+0000 7f7bfaffd640 1 -- 172.21.15.134:0/416130665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7bfc07a680 con 0x7f7bfc09d4f0 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bfaffd640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 msgr2=0x7f7bfc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bfaffd640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 0x7f7bfc09d900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7be4023070 con 0x7f7bfc09d4f0 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:27.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 10 monclient: dump: 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.688 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f7be4006880 con 0x7f7bfc09d4f0 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 msgr2=0x7f7bfc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 0x7f7bfc09d900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be40047d0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 msgr2=0x7f7bfc09d900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.673+0000 7f7bf37fe640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bfc09d4f0 0x7f7bfc09d900 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7be400e7a0 tx=0x7f7be40047d0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:27.690 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 9, 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.691 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.692 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:27.693 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.694 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.695 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.704 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.705 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.707 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:27.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:27.711 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:27.712 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.713 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:27.714 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bdc038880 msgr2=0x7f7bdc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 --2- 172.21.15.134:0/416130665 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bdc038880 0x7f7bdc03ad40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007c90 tx=0x7f7bec007be0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 10 monclient: shutdown 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 msgr2=0x7f7bfc11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 --2- 172.21.15.134:0/416130665 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bfc13c370 0x7f7bfc11fc60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009d90 tx=0x7f7bf4004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 reap_dead start 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7bfb7fe640 1 -- 172.21.15.134:0/416130665 reap_dead start 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 shutdown_connections 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 >> 172.21.15.134:0/416130665 conn(0x7f7bfc0a6110 msgr2=0x7f7bfc13b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 shutdown_connections 2024-09-16T15:05:27.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.674+0000 7f7c01661640 1 -- 172.21.15.134:0/416130665 wait complete. 2024-09-16T15:05:27.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:27.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:05:27.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:05:27.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.814+0000 7f67ee71f640 10 monclient: get_monmap_and_config 2024-09-16T15:05:27.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.814+0000 7f67ee71f640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.814+0000 7f67ee71f640 10 monclient: monmap: 2024-09-16T15:05:27.815 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:27.815092+0000 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:27.815092+0000 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.816 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.817 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.817 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.817 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:27.817 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:27.817 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 Processor -- start 2024-09-16T15:05:27.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 -- start start 2024-09-16T15:05:27.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient: init 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient: _add_conns ranks=[8,5,6,3,7,9,1,0,4,2,10] 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-e con 0x7f67e80a1e10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 0x7f67e80a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-h con 0x7f67e80a21f0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e81373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-d con 0x7f67e8136fc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806f340 con 0x7f67e80a21f0 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80a37a0 con 0x7f67e8136fc0 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806c7f0 con 0x7f67e80a1e10 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67ee71f640 10 monclient(hunting): authenticate will time out at 1064.404692s 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58566/0 (socket says 172.21.15.134:58566) 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e81373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 1 -- 172.21.15.134:0/2196176554 learned_addr learned my addr 172.21.15.134:0/2196176554 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): get_auth_request con 0x7f67e80a1e10 auth_method 0 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): get_auth_request con 0x7f67e8136fc0 auth_method 0 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67dffff640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 0x7f67e80a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67dffff640 10 monclient(hunting): get_auth_request con 0x7f67e80a21f0 auth_method 0 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.815+0000 7f67dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 10 monclient(hunting): handle_auth_done global_id 44198 payload 274 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 -- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 msgr2=0x7f67e81373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e81373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 -- 172.21.15.134:0/2196176554 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 msgr2=0x7f67e80a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 0x7f67e80a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 monclient: found mon.noname-e 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67e7fff640 1 -- 172.21.15.134:0/2196176554 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e8073250 con 0x7f67e80a1e10 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67eccbc640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e81373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.816+0000 7f67dffff640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 0x7f67e80a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:27.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.818+0000 7f67e7fff640 1 -- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 msgr2=0x7f67e806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.818+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098a0 tx=0x7f67d802a080 comp rx=0 tx=0).ready entity=mon.8 client_cookie=9eda69d72f4e4d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.818+0000 7f67e77fe640 1 -- 172.21.15.134:0/2196176554 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d802ade0 con 0x7f67e80a1e10 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: dump: 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.828 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 5 monclient: authenticate success, global_id 44198 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 1 -- 172.21.15.134:0/2196176554 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d80307b0 con 0x7f67e80a1e10 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 4 set_mon_vals no callback set 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 1 -- 172.21.15.134:0/2196176554 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d8030d90 con 0x7f67e80a1e10 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67e77fe640 10 monclient: dump: 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.830 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 10 monclient: shutdown 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 -- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 msgr2=0x7f67e806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098a0 tx=0x7f67d802a080 comp rx=0 tx=0).stop 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 -- 172.21.15.134:0/2196176554 shutdown_connections 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e81373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f67e80a21f0 0x7f67e80a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2196176554 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a1e10 0x7f67e806cc10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 -- 172.21.15.134:0/2196176554 >> 172.21.15.134:0/2196176554 conn(0x7f67e80a7410 msgr2=0x7f67e80a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 -- 172.21.15.134:0/2196176554 shutdown_connections 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.819+0000 7f67ee71f640 1 -- 172.21.15.134:0/2196176554 wait complete. 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.820+0000 7f67ee71f640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.820+0000 7f67ee71f640 10 monclient: monmap: 2024-09-16T15:05:27.832 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.833 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.820+0000 7f67ee71f640 1 Processor -- start 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.820+0000 7f67ee71f640 1 -- start start 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.820+0000 7f67ee71f640 10 monclient: init 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient: _add_conns ranks=[4,8,6,5,2,1,3,0,7] 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f67e80a1e10 0x7f67e8136350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-c con 0x7f67e80a1e10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a21f0 0x7f67e8122b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-e con 0x7f67e80a21f0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e8123050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): picked mon.noname-d con 0x7f67e8136fc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8125490 con 0x7f67e80a1e10 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81cc170 con 0x7f67e8136fc0 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806fe40 con 0x7f67e80a21f0 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67ee71f640 10 monclient(hunting): authenticate will time out at 1064.410092s 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f67e80a1e10 0x7f67e8136350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a21f0 0x7f67e8122b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e8123050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f67e80a1e10 0x7f67e8136350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:57692/0 (socket says 172.21.15.134:57692) 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 1 -- 172.21.15.134:0/2893421226 learned_addr learned my addr 172.21.15.134:0/2893421226 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): get_auth_request con 0x7f67e8136fc0 auth_method 0 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): get_auth_request con 0x7f67e80a21f0 auth_method 0 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): get_auth_request con 0x7f67e80a1e10 auth_method 0 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient(hunting): handle_auth_done global_id 34404 payload 824 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a21f0 msgr2=0x7f67e8122b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f67e80a21f0 0x7f67e8122b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f67e80a1e10 msgr2=0x7f67e8136350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f67e80a1e10 0x7f67e8136350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 monclient: found mon.noname-d 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:27.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.821+0000 7f67eccbc640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0009a70 con 0x7f67e8136fc0 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67eccbc640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 msgr2=0x7f67e8123050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67eccbc640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e8123050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67e000efc0 tx=0x7f67e000c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67e00148f0 con 0x7f67e8136fc0 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: dump: 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.843 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e000f040 con 0x7f67e8136fc0 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 5 monclient: authenticate success, global_id 34404 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67e00133e0 con 0x7f67e8136fc0 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 10 monclient: _renew_subs 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d80094d0 con 0x7f67e8136fc0 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e6ffd640 4 set_mon_vals no callback set 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67e57fa640 10 monclient: dump: 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.845 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.846 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 10 monclient: _renew_subs 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.823+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81246a0 con 0x7f67e8136fc0 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67ee71f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 0x7f67e8123590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e806a3f0 con 0x7f67e8126410 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f67e0014480 con 0x7f67e8136fc0 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.824+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 0x7f67e8123590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient: get_auth_request con 0x7f67e8126410 auth_method 0 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e57fa640 1 --2- 172.21.15.134:0/2893421226 >> v2:172.21.15.162:6800/980547263 conn(0x7f67c8038880 0x7f67c803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f67e004adf0 con 0x7f67e8136fc0 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 10 monclient(hunting): handle_auth_done global_id 44174 payload 824 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67e7fff640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67e8061ae0 con 0x7f67e8126410 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67dffff640 1 --2- 172.21.15.134:0/2893421226 >> v2:172.21.15.162:6800/980547263 conn(0x7f67c8038880 0x7f67c803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.825+0000 7f67dffff640 10 monclient: get_auth_request con 0x7f67c8038880 auth_method 0 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e7fff640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 msgr2=0x7f67e8123590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 0x7f67e8123590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097c0 tx=0x7f67d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67dffff640 1 --2- 172.21.15.134:0/2893421226 >> v2:172.21.15.162:6800/980547263 conn(0x7f67c8038880 0x7f67c803ad40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67e8124270 tx=0x7f67d0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d8003610 con 0x7f67e8126410 2024-09-16T15:05:27.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.827+0000 7f67e57fa640 10 monclient: dump: 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.849 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f67d8030b30 con 0x7f67e8126410 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 msgr2=0x7f67e8123590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 0x7f67e8123590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097c0 tx=0x7f67d80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 msgr2=0x7f67e8123590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.828+0000 7f67e57fa640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8126410 0x7f67e8123590 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097c0 tx=0x7f67d80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67ee71f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:27.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 0x7f67e809e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67ee71f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e807a680 con 0x7f67e8123ad0 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 0x7f67e809e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient: get_auth_request con 0x7f67e8123ad0 auth_method 0 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 10 monclient(hunting): handle_auth_done global_id 44183 payload 824 2024-09-16T15:05:27.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.834+0000 7f67e7fff640 1 -- 172.21.15.134:0/2893421226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67e806bf50 con 0x7f67e8123ad0 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e7fff640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 msgr2=0x7f67e809e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e7fff640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 0x7f67e809e810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a0c0 tx=0x7f67d8008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d8003e80 con 0x7f67e8123ad0 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 10 monclient: dump: 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.852 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f67d804c2b0 con 0x7f67e8123ad0 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:27.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 msgr2=0x7f67e809e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 0x7f67e809e810 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a0c0 tx=0x7f67d8008030 comp rx=0 tx=0).stop 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 msgr2=0x7f67e809e810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67e57fa640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67e8123ad0 0x7f67e809e810 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a0c0 tx=0x7f67d8008030 comp rx=0 tx=0).stop 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:27.854 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 9, 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:27.855 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.856 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.857 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:27.858 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:27.859 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.863 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.864 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.865 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:27.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.868 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:27.870 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:27.873 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:27.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:27.875 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:27.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:27.877 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:27.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 >> v2:172.21.15.162:6800/980547263 conn(0x7f67c8038880 msgr2=0x7f67c803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2893421226 >> v2:172.21.15.162:6800/980547263 conn(0x7f67c8038880 0x7f67c803ad40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67e8124270 tx=0x7f67d0008040 comp rx=0 tx=0).stop 2024-09-16T15:05:27.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 10 monclient: shutdown 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 msgr2=0x7f67e8123050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e8123050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67e000efc0 tx=0x7f67e000c490 comp rx=0 tx=0).stop 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67eccbc640 1 -- 172.21.15.134:0/2893421226 reap_dead start 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 shutdown_connections 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 --2- 172.21.15.134:0/2893421226 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67e8136fc0 0x7f67e8123050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 >> 172.21.15.134:0/2893421226 conn(0x7f67e80a7410 msgr2=0x7f67e80a5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 shutdown_connections 2024-09-16T15:05:27.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.837+0000 7f67ee71f640 1 -- 172.21.15.134:0/2893421226 wait complete. 2024-09-16T15:05:27.879 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:27.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:05:27.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:05:27.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.975+0000 7f735eea3640 10 monclient: get_monmap_and_config 2024-09-16T15:05:27.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.975+0000 7f735eea3640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.975+0000 7f735eea3640 10 monclient: monmap: 2024-09-16T15:05:27.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:27.976631+0000 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:27.976631+0000 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 Processor -- start 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 -- start start 2024-09-16T15:05:27.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient: init 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient: _add_conns ranks=[4,1,10,6,3,7,2,8,9,0,5] 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-c con 0x7f7358136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7358137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f735813c370 0x7f735813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-f con 0x7f735813c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7358073250 con 0x7f7358137220 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73580a56e0 con 0x7f735813c370 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f735809abc0 con 0x7f7358136e40 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.976+0000 7f735eea3640 10 monclient(hunting): authenticate will time out at 1064.565944s 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:57694/0 (socket says 172.21.15.134:57694) 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 -- 172.21.15.134:0/3401687234 learned_addr learned my addr 172.21.15.134:0/3401687234 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): get_auth_request con 0x7f7358136e40 auth_method 0 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735d440640 1 --2- 172.21.15.134:0/3401687234 >> v2:172.21.15.162:3300/0 conn(0x7f735813c370 0x7f735813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient(hunting): handle_auth_done global_id 4774 payload 274 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 -- 172.21.15.134:0/3401687234 >> v2:172.21.15.162:3300/0 conn(0x7f735813c370 msgr2=0x7f735813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 --2- 172.21.15.134:0/3401687234 >> v2:172.21.15.162:3300/0 conn(0x7f735813c370 0x7f735813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 -- 172.21.15.134:0/3401687234 >> v1:172.21.15.162:6789/0 conn(0x7f7358137220 legacy=0x7f7358137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 monclient: found mon.noname-c 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.977+0000 7f735cc3f640 1 -- 172.21.15.134:0/3401687234 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735809ba50 con 0x7f7358136e40 2024-09-16T15:05:27.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735cc3f640 1 -- 172.21.15.134:0/3401687234 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 msgr2=0x7f735809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735cc3f640 1 --2- 172.21.15.134:0/3401687234 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f734c009870 tx=0x7f734c02a100 comp rx=0 tx=0).ready entity=mon.4 client_cookie=59e2bee5833c01a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 1 -- 172.21.15.134:0/3401687234 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c0307b0 con 0x7f7358136e40 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: dump: 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 1 -- 172.21.15.134:0/3401687234 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c030930 con 0x7f7358136e40 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 5 monclient: authenticate success, global_id 4774 2024-09-16T15:05:27.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 4 set_mon_vals no callback set 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 1 -- 172.21.15.134:0/3401687234 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c0313a0 con 0x7f7358136e40 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f73577fe640 10 monclient: dump: 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:27.990 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 10 monclient: shutdown 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:27.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 1 -- 172.21.15.134:0/3401687234 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 msgr2=0x7f735809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.980+0000 7f735eea3640 1 --2- 172.21.15.134:0/3401687234 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f734c009870 tx=0x7f734c02a100 comp rx=0 tx=0).stop 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 -- 172.21.15.134:0/3401687234 shutdown_connections 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 --2- 172.21.15.134:0/3401687234 >> v2:172.21.15.162:3300/0 conn(0x7f735813c370 0x7f735813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 --2- 172.21.15.134:0/3401687234 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f735809c150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 -- 172.21.15.134:0/3401687234 >> 172.21.15.134:0/3401687234 conn(0x7f73580a6110 msgr2=0x7f73580a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 -- 172.21.15.134:0/3401687234 shutdown_connections 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 -- 172.21.15.134:0/3401687234 wait complete. 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 10 monclient: build_initial_monmap 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 10 monclient: monmap: 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:27.992 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:27.993 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 Processor -- start 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 1 -- start start 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.981+0000 7f735eea3640 10 monclient: init 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient: _add_conns ranks=[4,3,7,8,0,2,6,5,1] 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-c con 0x7f7358136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7358137220 0x7f7358135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-g con 0x7f7358137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f735813c370 0x7f7358136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): picked mon.noname-i con 0x7f735813c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:27.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73580a75a0 con 0x7f7358137220 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7358121e70 con 0x7f7358136e40 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f73581223a0 con 0x7f735813c370 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735eea3640 10 monclient(hunting): authenticate will time out at 1064.571546s 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:57700/0 (socket says 172.21.15.134:57700) 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 -- 172.21.15.134:0/304175695 learned_addr learned my addr 172.21.15.134:0/304175695 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): get_auth_request con 0x7f7358136e40 auth_method 0 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:27.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient(hunting): handle_auth_done global_id 4783 payload 824 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735d440640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f735813c370 0x7f7358136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7358137220 0x7f7358135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f735813c370 msgr2=0x7f7358136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f735813c370 0x7f7358136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7358137220 msgr2=0x7f7358135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7358137220 0x7f7358135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:27.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient: _finish_hunting 0 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 monclient: found mon.noname-c 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735cc3f640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0094d0 con 0x7f7358136e40 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735d440640 10 monclient: get_auth_request con 0x7f735813c370 auth_method 0 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f7357fff640 10 monclient: get_auth_request con 0x7f7358137220 auth_method 0 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f735d440640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f735813c370 0x7f7358136150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:27.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.982+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7358137220 0x7f7358135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735cc3f640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 msgr2=0x7f73581356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735cc3f640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f734c0090b0 tx=0x7f734c0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c004460 con 0x7f7358136e40 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:28.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: dump: 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.002 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c003a80 con 0x7f7358136e40 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 5 monclient: authenticate success, global_id 4783 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f734c003bf0 con 0x7f7358136e40 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f73557fa640 10 monclient: dump: 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.003 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.004 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f7356ffd640 4 set_mon_vals no callback set 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 10 monclient: _renew_subs 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7358122680 con 0x7f7358136e40 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 10 monclient: _renew_subs 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.985+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735813b510 con 0x7f7358136e40 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f734c039040 con 0x7f7358136e40 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735eea3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735eea3640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 0x7f735802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7358071b20 con 0x7f7358120420 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f73557fa640 1 --2- 172.21.15.134:0/304175695 >> v2:172.21.15.162:6800/980547263 conn(0x7f733c038880 0x7f733c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f734c06aa70 con 0x7f7358136e40 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 0x7f735802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient: get_auth_request con 0x7f7358120420 auth_method 0 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735d440640 1 --2- 172.21.15.134:0/304175695 >> v2:172.21.15.162:6800/980547263 conn(0x7f733c038880 0x7f733c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f735d440640 10 monclient: get_auth_request con 0x7f733c038880 auth_method 0 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 10 monclient(hunting): handle_auth_done global_id 4792 payload 824 2024-09-16T15:05:28.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.986+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7358061ae0 con 0x7f7358120420 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 msgr2=0x7f735802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 0x7f735802b260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f734400d950 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7344016070 con 0x7f7358120420 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f73557fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f73557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.988+0000 7f73557fa640 10 monclient: dump: 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.007 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f735d440640 1 --2- 172.21.15.134:0/304175695 >> v2:172.21.15.162:6800/980547263 conn(0x7f733c038880 0x7f733c03ad40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f735806b7d0 tx=0x7f7350007360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7344005490 con 0x7f7358120420 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 msgr2=0x7f735802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 0x7f735802b260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f734400d950 comp rx=0 tx=0).stop 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 msgr2=0x7f735802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.989+0000 7f73557fa640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120420 0x7f735802b260 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f734400d950 comp rx=0 tx=0).stop 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f735eea3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f735eea3640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f735eea3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f735807a680 con 0x7f7358120800 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient: get_auth_request con 0x7f7358120800 auth_method 0 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 10 monclient(hunting): handle_auth_done global_id 4801 payload 824 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f73580a53c0 con 0x7f7358120800 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 msgr2=0x7f735802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 msgr2=0x7f735802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f7344001020 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:28.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:27.995+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:28.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: get_auth_request con 0x7f7358120800 auth_method 0 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: _init_auth method 2 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: _init_auth already have auth, reseting 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: handle_auth_reply_more payload 9 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:28.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.196+0000 7f7357fff640 10 monclient: handle_auth_done global_id 4801 payload 293 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f7357fff640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 msgr2=0x7f735802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f7357fff640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f7344011000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7344020070 con 0x7f7358120800 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f73557fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f73557fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:28.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.198+0000 7f73557fa640 10 monclient: dump: 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.201 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f73440069a0 con 0x7f7358120800 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 msgr2=0x7f735802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f7344011000 comp rx=0 tx=0).stop 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 msgr2=0x7f735802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f73557fa640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358120800 0x7f735802b7a0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73580a26e0 tx=0x7f7344011000 comp rx=0 tx=0).stop 2024-09-16T15:05:28.202 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:28.203 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.204 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.205 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:28.206 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.207 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.209 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.211 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:28.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.214 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.216 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.217 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.219 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:28.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:28.225 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:28.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 >> v2:172.21.15.162:6800/980547263 conn(0x7f733c038880 msgr2=0x7f733c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 --2- 172.21.15.134:0/304175695 >> v2:172.21.15.162:6800/980547263 conn(0x7f733c038880 0x7f733c03ad40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f735806b7d0 tx=0x7f7350007360 comp rx=0 tx=0).stop 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 10 monclient: shutdown 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 msgr2=0x7f73581356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f734c0090b0 tx=0x7f734c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735d440640 1 -- 172.21.15.134:0/304175695 reap_dead start 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 shutdown_connections 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 --2- 172.21.15.134:0/304175695 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7358136e40 0x7f73581356d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 >> 172.21.15.134:0/304175695 conn(0x7f73580a6110 msgr2=0x7f735813a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 shutdown_connections 2024-09-16T15:05:28.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.199+0000 7f735eea3640 1 -- 172.21.15.134:0/304175695 wait complete. 2024-09-16T15:05:28.228 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:28.228 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:05:28.228 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:05:28.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 10 monclient: get_monmap_and_config 2024-09-16T15:05:28.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 10 monclient: monmap: 2024-09-16T15:05:28.325 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.325 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:28.325020+0000 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:28.325020+0000 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.326 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:28.327 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:28.327 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 1 Processor -- start 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 1 -- start start 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.324+0000 7fbf520f0640 10 monclient: init 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient: _add_conns ranks=[8,2,3,5,6,7,10,4,1,0,9] 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-e con 0x7fbf4c136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-b con 0x7fbf4c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf4c135dc0 0x7fbf4c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-g con 0x7fbf4c135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c09abc0 con 0x7fbf4c137220 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c09ba50 con 0x7fbf4c135dc0 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0a53c0 con 0x7fbf4c136e40 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf520f0640 10 monclient(hunting): authenticate will time out at 1064.914360s 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41132/0 (socket says 172.21.15.134:41132) 2024-09-16T15:05:28.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58576/0 (socket says 172.21.15.134:58576) 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 -- 172.21.15.134:0/1276989090 learned_addr learned my addr 172.21.15.134:0/1276989090 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): get_auth_request con 0x7fbf4c137220 auth_method 0 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request con 0x7fbf4c136e40 auth_method 0 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4bfff640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf4c135dc0 0x7fbf4c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4bfff640 10 monclient(hunting): get_auth_request con 0x7fbf4c135dc0 auth_method 0 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient(hunting): handle_auth_done global_id 44192 payload 274 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 -- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 msgr2=0x7fbf4c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 -- 172.21.15.134:0/1276989090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf4c135dc0 msgr2=0x7fbf4c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf4c135dc0 0x7fbf4c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 monclient: found mon.noname-b 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4affd640 1 -- 172.21.15.134:0/1276989090 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c073250 con 0x7fbf4c137220 2024-09-16T15:05:28.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.325+0000 7fbf4b7fe640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:28.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4affd640 1 -- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 msgr2=0x7fbf4c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4affd640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf340098a0 tx=0x7fbf3402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ff4cf49630b6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 1 -- 172.21.15.134:0/1276989090 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf3402ade0 con 0x7fbf4c137220 2024-09-16T15:05:28.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: dump: 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 1 -- 172.21.15.134:0/1276989090 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf340307b0 con 0x7fbf4c137220 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf520f0640 5 monclient: authenticate success, global_id 44192 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf520f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf520f0640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf520f0640 4 set_mon_vals no callback set 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 1 -- 172.21.15.134:0/1276989090 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf34030d90 con 0x7fbf4c137220 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.328+0000 7fbf4a7fc640 10 monclient: dump: 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.343 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 10 monclient: shutdown 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 -- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 msgr2=0x7fbf4c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf340098a0 tx=0x7fbf3402a080 comp rx=0 tx=0).stop 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 -- 172.21.15.134:0/1276989090 shutdown_connections 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbf4c135dc0 0x7fbf4c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c137220 0x7fbf4c137600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 --2- 172.21.15.134:0/1276989090 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbf4c136e40 0x7fbf4c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 -- 172.21.15.134:0/1276989090 >> 172.21.15.134:0/1276989090 conn(0x7fbf4c0a6110 msgr2=0x7fbf4c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 -- 172.21.15.134:0/1276989090 shutdown_connections 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 1 -- 172.21.15.134:0/1276989090 wait complete. 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.329+0000 7fbf520f0640 10 monclient: monmap: 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:28.345 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 Processor -- start 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 -- start start 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient: init 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient: _add_conns ranks=[4,2,1,5,0,7,8,3,6] 2024-09-16T15:05:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbf4c135dc0 0x7fbf4c134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-c con 0x7fbf4c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-b con 0x7fbf4c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf4c137220 0x7fbf4c121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): picked mon.noname-f con 0x7fbf4c137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0a8a50 con 0x7fbf4c137220 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c136890 con 0x7fbf4c136e40 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c136340 con 0x7fbf4c135dc0 2024-09-16T15:05:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf520f0640 10 monclient(hunting): authenticate will time out at 1064.919888s 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbf4c135dc0 0x7fbf4c134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41148/0 (socket says 172.21.15.134:41148) 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 1 -- 172.21.15.134:0/4222217210 learned_addr learned my addr 172.21.15.134:0/4222217210 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 10 monclient(hunting): get_auth_request con 0x7fbf4c136e40 auth_method 0 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.330+0000 7fbf4affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request con 0x7fbf4c135dc0 auth_method 0 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf4c137220 0x7fbf4c121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): get_auth_request con 0x7fbf4c137220 auth_method 0 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient(hunting): handle_auth_done global_id 44201 payload 824 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbf4c135dc0 msgr2=0x7fbf4c134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbf4c135dc0 0x7fbf4c134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf4c137220 msgr2=0x7fbf4c121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbf4c137220 0x7fbf4c121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 monclient: found mon.noname-b 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.331+0000 7fbf4affd640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c009a70 con 0x7fbf4c136e40 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf4affd640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 msgr2=0x7fbf4c134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf4affd640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf340090b0 tx=0x7fbf340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf34030d40 con 0x7fbf4c136e40 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: dump: 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.351 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 5 monclient: authenticate success, global_id 44201 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34003a80 con 0x7fbf4c136e40 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf34003bf0 con 0x7fbf4c136e40 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 10 monclient: _renew_subs 2024-09-16T15:05:28.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf340094d0 con 0x7fbf4c136e40 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf49ffb640 4 set_mon_vals no callback set 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf43fff640 10 monclient: dump: 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.353 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 10 monclient: _renew_subs 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.333+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c125210 con 0x7fbf4c136e40 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4affd640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf340090b0 tx=0x7fbf340046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fbf34039040 mgrmap(e 5) v1 2024-09-16T15:05:28.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbf34039040 con 0x7fbf4c136e40 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf520f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf520f0640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 0x7fbf4c0a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c073620 con 0x7fbf4c0a1a20 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf43fff640 1 --2- 172.21.15.134:0/4222217210 >> v2:172.21.15.162:6800/980547263 conn(0x7fbf2c038880 0x7fbf2c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 0x7fbf4c0a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient: get_auth_request con 0x7fbf4c0a1a20 auth_method 0 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.334+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.335+0000 7fbf4bfff640 1 --2- 172.21.15.134:0/4222217210 >> v2:172.21.15.162:6800/980547263 conn(0x7fbf2c038880 0x7fbf2c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.335+0000 7fbf4bfff640 10 monclient: get_auth_request con 0x7fbf2c038880 auth_method 0 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.335+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_done global_id 34413 payload 824 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.335+0000 7fbf4b7fe640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf4c061ae0 con 0x7fbf4c0a1a20 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf4b7fe640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 msgr2=0x7fbf4c0a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf4b7fe640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 0x7fbf4c0a1e00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c00f8d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf3c00e040 con 0x7fbf4c0a1a20 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 10 monclient: dump: 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.357 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf4bfff640 1 --2- 172.21.15.134:0/4222217210 >> v2:172.21.15.162:6800/980547263 conn(0x7fbf2c038880 0x7fbf2c03ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c123000 tx=0x7fbf44008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbf3c00f040 con 0x7fbf4c0a1a20 2024-09-16T15:05:28.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 msgr2=0x7fbf4c0a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 0x7fbf4c0a1e00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c00f8d0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 msgr2=0x7fbf4c0a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.337+0000 7fbf43fff640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a1a20 0x7fbf4c0a1e00 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c00f8d0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf520f0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf520f0640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 0x7fbf4c121d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf520f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1253a0 con 0x7fbf4c0a2340 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 0x7fbf4c121d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient: get_auth_request con 0x7fbf4c0a2340 auth_method 0 2024-09-16T15:05:28.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.343+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.344+0000 7fbf4b7fe640 10 monclient(hunting): handle_auth_done global_id 34422 payload 824 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.344+0000 7fbf4b7fe640 1 -- 172.21.15.134:0/4222217210 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbf4c07a680 con 0x7fbf4c0a2340 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf4b7fe640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 msgr2=0x7fbf4c121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf4b7fe640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 0x7fbf4c121d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf3c024070 con 0x7fbf4c0a2340 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:28.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 10 monclient: dump: 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.363 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fbf3c003e90 con 0x7fbf4c0a2340 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 msgr2=0x7fbf4c121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 0x7fbf4c121d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 msgr2=0x7fbf4c121d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.346+0000 7fbf43fff640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbf4c0a2340 0x7fbf4c121d60 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00f410 tx=0x7fbf3c004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:28.364 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:28.365 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.366 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:28.368 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.369 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:28.373 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:28.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.376 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.377 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.378 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:28.381 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.384 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.385 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:28.386 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:28.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 >> v2:172.21.15.162:6800/980547263 conn(0x7fbf2c038880 msgr2=0x7fbf2c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 --2- 172.21.15.134:0/4222217210 >> v2:172.21.15.162:6800/980547263 conn(0x7fbf2c038880 0x7fbf2c03ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c123000 tx=0x7fbf44008040 comp rx=0 tx=0).stop 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 10 monclient: shutdown 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 msgr2=0x7fbf4c134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 --2- 172.21.15.134:0/4222217210 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf4c136e40 0x7fbf4c134fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf340090b0 tx=0x7fbf340046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf4bfff640 1 -- 172.21.15.134:0/4222217210 reap_dead start 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf43fff640 1 -- 172.21.15.134:0/4222217210 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7fbf3406a670 con 0x7fbf4c136e40 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf4bfff640 1 -- 172.21.15.134:0/4222217210 reap_dead start 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 shutdown_connections 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 >> 172.21.15.134:0/4222217210 conn(0x7fbf4c0a6110 msgr2=0x7fbf4c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 shutdown_connections 2024-09-16T15:05:28.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.347+0000 7fbf520f0640 1 -- 172.21.15.134:0/4222217210 wait complete. 2024-09-16T15:05:28.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:28.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:05:28.389 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.488+0000 7f456048b640 10 monclient: get_monmap_and_config 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.488+0000 7f456048b640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.488+0000 7f456048b640 10 monclient: monmap: 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:28.489548+0000 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:28.489548+0000 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:28.491 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 Processor -- start 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 -- start start 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient: init 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient: _add_conns ranks=[9,0,4,7,8,3,6,1,5,2,10] 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 0x7f455809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): picked mon.noname-a con 0x7f4558136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4558137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 0x7f455813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): picked mon.noname-c con 0x7f455813c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4558073250 con 0x7f4558137220 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580a56e0 con 0x7f4558136e40 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f455809abc0 con 0x7f455813c370 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f456048b640 10 monclient(hunting): authenticate will time out at 1065.078870s 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f455ea28640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 0x7f455813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.489+0000 7f455e227640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 0x7f455809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455da26640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4558137220 0x7f4558137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42932/0 (socket says 172.21.15.134:42932) 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455da26640 1 -- 172.21.15.134:0/3118040144 learned_addr learned my addr 172.21.15.134:0/3118040144 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): get_auth_request con 0x7f455813c370 auth_method 0 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): get_auth_request con 0x7f4558136e40 auth_method 0 2024-09-16T15:05:28.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient(hunting): handle_auth_done global_id 4819 payload 274 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 1 -- 172.21.15.134:0/3118040144 >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 msgr2=0x7f455809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 1 --2- 172.21.15.134:0/3118040144 >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 0x7f455809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 1 -- 172.21.15.134:0/3118040144 >> v1:172.21.15.134:6789/0 conn(0x7f4558137220 legacy=0x7f4558137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 1 monclient: found mon.noname-c 2024-09-16T15:05:28.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455ea28640 1 -- 172.21.15.134:0/3118040144 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455809ba50 con 0x7f455813c370 2024-09-16T15:05:28.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.490+0000 7f455e227640 1 --2- 172.21.15.134:0/3118040144 >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 0x7f455809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:28.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455ea28640 1 -- 172.21.15.134:0/3118040144 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 msgr2=0x7f455813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455ea28640 1 --2- 172.21.15.134:0/3118040144 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 0x7f455813e760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4550009d90 tx=0x7f455002a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=367854b1d2204ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 1 -- 172.21.15.134:0/3118040144 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4550007bc0 con 0x7f455813c370 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: dump: 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.504 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 1 -- 172.21.15.134:0/3118040144 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455002d050 con 0x7f455813c370 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 5 monclient: authenticate success, global_id 4819 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:28.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 1 -- 172.21.15.134:0/3118040144 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4550007d80 con 0x7f455813c370 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 4 set_mon_vals no callback set 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f455d225640 10 monclient: dump: 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.506 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 10 monclient: shutdown 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 -- 172.21.15.134:0/3118040144 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 msgr2=0x7f455813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 --2- 172.21.15.134:0/3118040144 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 0x7f455813e760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4550009d90 tx=0x7f455002a4e0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 -- 172.21.15.134:0/3118040144 shutdown_connections 2024-09-16T15:05:28.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 --2- 172.21.15.134:0/3118040144 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f455813c370 0x7f455813e760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 --2- 172.21.15.134:0/3118040144 >> v2:172.21.15.134:3300/0 conn(0x7f4558136e40 0x7f455809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 -- 172.21.15.134:0/3118040144 >> 172.21.15.134:0/3118040144 conn(0x7f45580a6110 msgr2=0x7f45580a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 -- 172.21.15.134:0/3118040144 shutdown_connections 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.493+0000 7f456048b640 1 -- 172.21.15.134:0/3118040144 wait complete. 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.494+0000 7f456048b640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.494+0000 7f456048b640 10 monclient: monmap: 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:28.508 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.494+0000 7f456048b640 1 Processor -- start 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.494+0000 7f456048b640 1 -- start start 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.494+0000 7f456048b640 10 monclient: init 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient: _add_conns ranks=[6,7,1,8,2,3,5,4,0] 2024-09-16T15:05:28.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): picked mon.noname-d con 0x7f4558136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4558137220 0x7f45581201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): picked mon.noname-i con 0x7f4558137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455813c370 0x7f45581206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): picked mon.noname-f con 0x7f455813c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558121e70 con 0x7f455813c370 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4558136b00 con 0x7f4558136e40 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f45581223a0 con 0x7f4558137220 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f456048b640 10 monclient(hunting): authenticate will time out at 1065.084257s 2024-09-16T15:05:28.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:47056/0 (socket says 172.21.15.134:47056) 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 -- 172.21.15.134:0/1195942099 learned_addr learned my addr 172.21.15.134:0/1195942099 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): get_auth_request con 0x7f4558136e40 auth_method 0 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient(hunting): handle_auth_done global_id 34431 payload 824 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4558137220 msgr2=0x7f45581201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455ea28640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455813c370 0x7f45581206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4558137220 0x7f45581201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4558137220 0x7f45581201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455813c370 msgr2=0x7f45581206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455813c370 0x7f45581206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 monclient: found mon.noname-d 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455e227640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009400 con 0x7f4558136e40 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455ea28640 10 monclient: get_auth_request con 0x7f455813c370 auth_method 0 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455da26640 10 monclient: get_auth_request con 0x7f4558137220 auth_method 0 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455ea28640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455813c370 0x7f45581206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.495+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4558137220 0x7f45581201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.497+0000 7f455e227640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 msgr2=0x7f455811fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.497+0000 7f455e227640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f455400ebe0 tx=0x7f455400c520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.497+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f455400eee0 con 0x7f4558136e40 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.497+0000 7f454effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: dump: 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.513 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400ce40 con 0x7f4558136e40 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 5 monclient: authenticate success, global_id 34431 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 monclient: _renew_subs 2024-09-16T15:05:28.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4554013d50 con 0x7f4558136e40 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4550009a70 con 0x7f4558136e40 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 10 monclient: dump: 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454ffff640 4 set_mon_vals no callback set 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 monclient: _renew_subs 2024-09-16T15:05:28.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45580a3a30 con 0x7f4558136e40 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 0x7f4558135b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4558071b20 con 0x7f4558135760 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4554013470 con 0x7f4558136e40 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.498+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 0x7f4558135b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient: get_auth_request con 0x7f4558135760 auth_method 0 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f454effd640 1 --2- 172.21.15.134:0/1195942099 >> v2:172.21.15.162:6800/980547263 conn(0x7f453c038880 0x7f453c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f455404aa20 con 0x7f4558136e40 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 10 monclient(hunting): handle_auth_done global_id 44225 payload 824 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455da26640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4558061ae0 con 0x7f4558135760 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455ea28640 1 --2- 172.21.15.134:0/1195942099 >> v2:172.21.15.162:6800/980547263 conn(0x7f453c038880 0x7f453c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.499+0000 7f455ea28640 10 monclient: get_auth_request con 0x7f453c038880 auth_method 0 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f455da26640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 msgr2=0x7f4558135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 0x7f4558135b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45580a2840 tx=0x7f454800d950 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4548005cf0 con 0x7f4558135760 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 10 monclient: dump: 2024-09-16T15:05:28.518 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.519 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4548005e50 con 0x7f4558135760 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 msgr2=0x7f4558135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 0x7f4558135b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f45580a2840 tx=0x7f454800d950 comp rx=0 tx=0).stop 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 msgr2=0x7f4558135b40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f454effd640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558135760 0x7f4558135b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.501+0000 7f455ea28640 1 --2- 172.21.15.134:0/1195942099 >> v2:172.21.15.162:6800/980547263 conn(0x7f453c038880 0x7f453c03ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4550034000 tx=0x7f455002f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.507+0000 7f456048b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.507+0000 7f456048b640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 0x7f455809d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.507+0000 7f456048b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.507+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4558121fe0 con 0x7f4558136080 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 0x7f455809d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient: get_auth_request con 0x7f4558136080 auth_method 0 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 10 monclient(hunting): handle_auth_done global_id 44234 payload 824 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.508+0000 7f455da26640 1 -- 172.21.15.134:0/1195942099 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f455807a680 con 0x7f4558136080 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f455da26640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 msgr2=0x7f455809d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f455da26640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 0x7f455809d510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45580a2840 tx=0x7f4548004940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4548021070 con 0x7f4558136080 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 10 monclient: dump: 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.523 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f4548012620 con 0x7f4558136080 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 msgr2=0x7f455809d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 0x7f455809d510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f45580a2840 tx=0x7f4548004940 comp rx=0 tx=0).stop 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 msgr2=0x7f455809d510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.510+0000 7f454effd640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4558136080 0x7f455809d510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.525 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:05:28.526 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.527 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.528 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:28.529 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.530 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.532 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:28.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.542 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.544 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.546 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:28.547 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 >> v2:172.21.15.162:6800/980547263 conn(0x7f453c038880 msgr2=0x7f453c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 --2- 172.21.15.134:0/1195942099 >> v2:172.21.15.162:6800/980547263 conn(0x7f453c038880 0x7f453c03ad40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4550034000 tx=0x7f455002f000 comp rx=0 tx=0).stop 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 10 monclient: shutdown 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 msgr2=0x7f455811fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f455400ebe0 tx=0x7f455400c520 comp rx=0 tx=0).stop 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f455ea28640 1 -- 172.21.15.134:0/1195942099 reap_dead start 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 shutdown_connections 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 --2- 172.21.15.134:0/1195942099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4558136e40 0x7f455811fc60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 >> 172.21.15.134:0/1195942099 conn(0x7f45580a6110 msgr2=0x7f455813a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 shutdown_connections 2024-09-16T15:05:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.511+0000 7f456048b640 1 -- 172.21.15.134:0/1195942099 wait complete. 2024-09-16T15:05:28.550 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:28.550 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:05:28.550 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:05:28.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.646+0000 7ffb5e501640 10 monclient: get_monmap_and_config 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.646+0000 7ffb5e501640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.646+0000 7ffb5e501640 10 monclient: monmap: 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:28.647555+0000 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:28.647555+0000 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.648 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:28.649 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 Processor -- start 2024-09-16T15:05:28.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 -- start start 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient: init 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient: _add_conns ranks=[8,10,7,3,6,5,4,1,2,9,0] 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-e con 0x7ffb58136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 0x7ffb58137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-f con 0x7ffb58137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 0x7ffb58136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-i con 0x7ffb58135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5809abc0 con 0x7ffb58137220 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5809ba50 con 0x7ffb58135dc0 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580a53c0 con 0x7ffb58136e40 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.647+0000 7ffb5e501640 10 monclient(hunting): authenticate will time out at 1065.236919s 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58602/0 (socket says 172.21.15.134:58602) 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 -- 172.21.15.134:0/568263925 learned_addr learned my addr 172.21.15.134:0/568263925 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): get_auth_request con 0x7ffb58136e40 auth_method 0 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb5ca9e640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 0x7ffb58136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb577fe640 1 --2- 172.21.15.134:0/568263925 >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 0x7ffb58137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient(hunting): handle_auth_done global_id 44243 payload 274 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 -- 172.21.15.134:0/568263925 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 msgr2=0x7ffb58136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 0x7ffb58136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 -- 172.21.15.134:0/568263925 >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 msgr2=0x7ffb58137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 --2- 172.21.15.134:0/568263925 >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 0x7ffb58137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 monclient: found mon.noname-e 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:28.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb57fff640 1 -- 172.21.15.134:0/568263925 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58073250 con 0x7ffb58136e40 2024-09-16T15:05:28.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb5ca9e640 10 monclient: get_auth_request con 0x7ffb58135dc0 auth_method 0 2024-09-16T15:05:28.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb577fe640 10 monclient: get_auth_request con 0x7ffb58137220 auth_method 0 2024-09-16T15:05:28.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb5ca9e640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 0x7ffb58136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:28.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.648+0000 7ffb577fe640 1 --2- 172.21.15.134:0/568263925 >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 0x7ffb58137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:28.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb57fff640 1 -- 172.21.15.134:0/568263925 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 msgr2=0x7ffb5809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb57fff640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098a0 tx=0x7ffb4802a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6857cb7f168482fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 1 -- 172.21.15.134:0/568263925 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480307b0 con 0x7ffb58136e40 2024-09-16T15:05:28.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: dump: 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.663 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 5 monclient: authenticate success, global_id 44243 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 1 -- 172.21.15.134:0/568263925 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48030930 con 0x7ffb58136e40 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 4 set_mon_vals no callback set 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 1 -- 172.21.15.134:0/568263925 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb480313a0 con 0x7ffb58136e40 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb56ffd640 10 monclient: dump: 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.665 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 10 monclient: shutdown 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 -- 172.21.15.134:0/568263925 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 msgr2=0x7ffb5809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb480098a0 tx=0x7ffb4802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 -- 172.21.15.134:0/568263925 shutdown_connections 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58135dc0 0x7ffb58136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 --2- 172.21.15.134:0/568263925 >> v2:172.21.15.162:3300/0 conn(0x7ffb58137220 0x7ffb58137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 --2- 172.21.15.134:0/568263925 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb5809c150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 -- 172.21.15.134:0/568263925 >> 172.21.15.134:0/568263925 conn(0x7ffb580a6110 msgr2=0x7ffb580a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 -- 172.21.15.134:0/568263925 shutdown_connections 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.651+0000 7ffb5e501640 1 -- 172.21.15.134:0/568263925 wait complete. 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.652+0000 7ffb5e501640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.652+0000 7ffb5e501640 10 monclient: monmap: 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:28.667 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.652+0000 7ffb5e501640 1 Processor -- start 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.652+0000 7ffb5e501640 1 -- start start 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.652+0000 7ffb5e501640 10 monclient: init 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient: _add_conns ranks=[4,8,7,1,2,6,0,5,3] 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 0x7ffb58134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-c con 0x7ffb58135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb581217f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-e con 0x7ffb58136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58137220 0x7ffb58121d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): picked mon.noname-i con 0x7ffb58137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58125180 con 0x7ffb58135dc0 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580a89e0 con 0x7ffb58137220 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58123fb0 con 0x7ffb58136e40 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5e501640 10 monclient(hunting): authenticate will time out at 1065.242356s 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb581217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 0x7ffb58134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb581217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58618/0 (socket says 172.21.15.134:58618) 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 1 -- 172.21.15.134:0/3452739819 learned_addr learned my addr 172.21.15.134:0/3452739819 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): get_auth_request con 0x7ffb58136e40 auth_method 0 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): get_auth_request con 0x7ffb58135dc0 auth_method 0 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58137220 0x7ffb58121d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): get_auth_request con 0x7ffb58137220 auth_method 0 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb5ca9e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient(hunting): handle_auth_done global_id 4828 payload 824 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 msgr2=0x7ffb581217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb581217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58137220 msgr2=0x7ffb58121d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb58137220 0x7ffb58121d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 monclient: found mon.noname-c 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb57fff640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480094d0 con 0x7ffb58135dc0 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.653+0000 7ffb577fe640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb58136e40 0x7ffb581217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.655+0000 7ffb57fff640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 msgr2=0x7ffb58134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.655+0000 7ffb57fff640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 0x7ffb58134fc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb48031000 tx=0x7ffb480046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb48004460 con 0x7ffb58135dc0 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: dump: 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.673 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48003a80 con 0x7ffb58135dc0 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 5 monclient: authenticate success, global_id 4828 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb48003bf0 con 0x7ffb58135dc0 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 10 monclient: _renew_subs 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb581238f0 con 0x7ffb58135dc0 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb567fc640 4 set_mon_vals no callback set 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb54ff9640 10 monclient: dump: 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.675 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 10 monclient: _renew_subs 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.656+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb58125460 con 0x7ffb58135dc0 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5e501640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5e501640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 0x7ffb58122650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58071b20 con 0x7ffb58122270 2024-09-16T15:05:28.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ffb48039070 con 0x7ffb58135dc0 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb54ff9640 1 --2- 172.21.15.134:0/3452739819 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb44038880 0x7ffb4403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7ffb4806a250 con 0x7ffb58135dc0 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 0x7ffb58122650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient: get_auth_request con 0x7ffb58122270 auth_method 0 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5ca9e640 1 --2- 172.21.15.134:0/3452739819 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb44038880 0x7ffb4403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb5ca9e640 10 monclient: get_auth_request con 0x7ffb44038880 auth_method 0 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 10 monclient(hunting): handle_auth_done global_id 44164 payload 824 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.657+0000 7ffb577fe640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb58061ae0 con 0x7ffb58122270 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb577fe640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 msgr2=0x7ffb58122650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb577fe640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 0x7ffb58122650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c00d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb5ca9e640 1 --2- 172.21.15.134:0/3452739819 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb44038880 0x7ffb4403ad40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d540 tx=0x7ffb50007780 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4c00fec0 con 0x7ffb58122270 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:28.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 10 monclient: dump: 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.679 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ffb4c00c040 con 0x7ffb58122270 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 msgr2=0x7ffb58122650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 0x7ffb58122650 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c00d590 comp rx=0 tx=0).stop 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 msgr2=0x7ffb58122650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.660+0000 7ffb54ff9640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb58122270 0x7ffb58122650 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c00d590 comp rx=0 tx=0).stop 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.666+0000 7ffb5e501640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.666+0000 7ffb5e501640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 0x7ffb580a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.666+0000 7ffb5e501640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.666+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5807a680 con 0x7ffb580a1a20 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 0x7ffb580a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient: get_auth_request con 0x7ffb580a1a20 auth_method 0 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 10 monclient(hunting): handle_auth_done global_id 44173 payload 824 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.667+0000 7ffb577fe640 1 -- 172.21.15.134:0/3452739819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb5809b490 con 0x7ffb580a1a20 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.669+0000 7ffb577fe640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 msgr2=0x7ffb580a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.669+0000 7ffb577fe640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 0x7ffb580a1e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb4c0069c0 con 0x7ffb580a1a20 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 10 monclient: dump: 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.683 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7ffb4c024070 con 0x7ffb580a1a20 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:28.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 msgr2=0x7ffb580a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 0x7ffb580a1e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 msgr2=0x7ffb580a1e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb54ff9640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb580a1a20 0x7ffb580a1e30 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00fbc0 tx=0x7ffb4c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:28.685 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.686 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.687 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:28.688 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:28.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:28.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:28.706 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:28.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb44038880 msgr2=0x7ffb4403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 1 --2- 172.21.15.134:0/3452739819 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb44038880 0x7ffb4403ad40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000d540 tx=0x7ffb50007780 comp rx=0 tx=0).stop 2024-09-16T15:05:28.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 10 monclient: shutdown 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 msgr2=0x7ffb58134fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5e501640 1 --2- 172.21.15.134:0/3452739819 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb58135dc0 0x7ffb58134fc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb48031000 tx=0x7ffb480046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5ca9e640 1 -- 172.21.15.134:0/3452739819 reap_dead start 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.670+0000 7ffb5ca9e640 1 -- 172.21.15.134:0/3452739819 reap_dead start 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.671+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 shutdown_connections 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.671+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 >> 172.21.15.134:0/3452739819 conn(0x7ffb580a6110 msgr2=0x7ffb5806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.671+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 shutdown_connections 2024-09-16T15:05:28.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.671+0000 7ffb5e501640 1 -- 172.21.15.134:0/3452739819 wait complete. 2024-09-16T15:05:28.708 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:28.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:05:28.709 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:05:28.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.804+0000 7f43d582b640 10 monclient: get_monmap_and_config 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.804+0000 7f43d582b640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.804+0000 7f43d582b640 10 monclient: monmap: 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:28.805440+0000 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:28.805440+0000 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.806 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:28.807 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 Processor -- start 2024-09-16T15:05:28.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 -- start start 2024-09-16T15:05:28.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient: init 2024-09-16T15:05:28.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient: _add_conns ranks=[5,10,0,4,7,6,1,9,3,2,8] 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 0x7f43d009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-h con 0x7f43d0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43d0137220 0x7f43d0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-f con 0x7f43d0137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f43d0135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d0073250 con 0x7f43d0135dc0 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d009abc0 con 0x7f43d0137220 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43d009ba50 con 0x7f43d0136e40 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43d582b640 10 monclient(hunting): authenticate will time out at 1065.394739s 2024-09-16T15:05:28.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43cf7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f43d0135dc0 0x7f43d0136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42946/0 (socket says 172.21.15.134:42946) 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43cf7fe640 1 -- 172.21.15.134:0/982551286 learned_addr learned my addr 172.21.15.134:0/982551286 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ceffd640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 0x7f43d009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/982551286 >> v2:172.21.15.162:3300/0 conn(0x7f43d0137220 0x7f43d0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ce7fc640 10 monclient(hunting): get_auth_request con 0x7f43d0137220 auth_method 0 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.805+0000 7f43ceffd640 10 monclient(hunting): get_auth_request con 0x7f43d0136e40 auth_method 0 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_done global_id 44182 payload 274 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 1 -- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 msgr2=0x7f43d009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 0x7f43d009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 1 -- 172.21.15.134:0/982551286 >> v1:172.21.15.134:6789/0 conn(0x7f43d0135dc0 legacy=0x7f43d0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 1 monclient: found mon.noname-f 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ce7fc640 1 -- 172.21.15.134:0/982551286 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00a53c0 con 0x7f43d0137220 2024-09-16T15:05:28.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.806+0000 7f43ceffd640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 0x7f43d009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43ce7fc640 1 -- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0137220 msgr2=0x7f43d0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0137220 0x7f43d0137600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f43c40097a0 tx=0x7f43c402a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8adfab309943a783 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 1 -- 172.21.15.134:0/982551286 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c402ad30 con 0x7f43d0137220 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: dump: 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.815 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 1 -- 172.21.15.134:0/982551286 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40307b0 con 0x7f43d0137220 2024-09-16T15:05:28.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 1 -- 172.21.15.134:0/982551286 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c4030d90 con 0x7f43d0137220 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 5 monclient: authenticate success, global_id 44182 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 4 set_mon_vals no callback set 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43cdffb640 10 monclient: dump: 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.817 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 10 monclient: shutdown 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 -- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0137220 msgr2=0x7f43d0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0137220 0x7f43d0137600 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f43c40097a0 tx=0x7f43c402a010 comp rx=0 tx=0).stop 2024-09-16T15:05:28.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 -- 172.21.15.134:0/982551286 shutdown_connections 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0137220 0x7f43d0137600 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 --2- 172.21.15.134:0/982551286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f43d0136e40 0x7f43d009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 -- 172.21.15.134:0/982551286 >> 172.21.15.134:0/982551286 conn(0x7f43d00a6110 msgr2=0x7f43d00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 -- 172.21.15.134:0/982551286 shutdown_connections 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.809+0000 7f43d582b640 1 -- 172.21.15.134:0/982551286 wait complete. 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.810+0000 7f43d582b640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.810+0000 7f43d582b640 10 monclient: monmap: 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.819 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.820 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.810+0000 7f43d582b640 1 Processor -- start 2024-09-16T15:05:28.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 -- start start 2024-09-16T15:05:28.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient: init 2024-09-16T15:05:28.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient: _add_conns ranks=[1,0,6,3,7,2,8,4,5] 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0135dc0 0x7f43d00a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-f con 0x7f43d0135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 0x7f43d0134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-a con 0x7f43d0136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f43d0137220 0x7f43d0135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): picked mon.noname-d con 0x7f43d0137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01210b0 con 0x7f43d0136e40 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0136950 con 0x7f43d0135dc0 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0136390 con 0x7f43d0137220 2024-09-16T15:05:28.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43d582b640 10 monclient(hunting): authenticate will time out at 1065.400549s 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 0x7f43d0134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 0x7f43d0134ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48992/0 (socket says 172.21.15.134:48992) 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 -- 172.21.15.134:0/972165267 learned_addr learned my addr 172.21.15.134:0/972165267 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): get_auth_request con 0x7f43d0136e40 auth_method 0 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f43d0137220 0x7f43d0135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): get_auth_request con 0x7f43d0137220 auth_method 0 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0135dc0 0x7f43d00a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): get_auth_request con 0x7f43d0135dc0 auth_method 0 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient(hunting): handle_auth_done global_id 4905 payload 824 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f43d0137220 msgr2=0x7f43d0135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f43d0137220 0x7f43d0135430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0135dc0 msgr2=0x7f43d00a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d0135dc0 0x7f43d00a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 monclient: found mon.noname-a 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.811+0000 7f43ce7fc640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009400 con 0x7f43d0136e40 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43ce7fc640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 msgr2=0x7f43d0134ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43ce7fc640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 0x7f43d0134ef0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43c40096f0 tx=0x7f43c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c4030d40 con 0x7f43d0136e40 2024-09-16T15:05:28.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: dump: 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.831 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4003a80 con 0x7f43d0136e40 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 5 monclient: authenticate success, global_id 4905 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 10 monclient: _renew_subs 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c4009400 con 0x7f43d0136e40 2024-09-16T15:05:28.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c4003bf0 con 0x7f43d0136e40 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43b7fff640 10 monclient: dump: 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 10 monclient: _renew_subs 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d01222f0 con 0x7f43d0136e40 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.814+0000 7f43cd7fa640 4 set_mon_vals no callback set 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43d582b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f43c4039040 con 0x7f43d0136e40 2024-09-16T15:05:28.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43d582b640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 0x7f43d002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0071b20 con 0x7f43d01cbb80 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43b7fff640 1 --2- 172.21.15.134:0/972165267 >> v2:172.21.15.162:6800/980547263 conn(0x7f43bc038880 0x7f43bc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f43c406a750 con 0x7f43d0136e40 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 0x7f43d002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient: get_auth_request con 0x7f43d01cbb80 auth_method 0 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43cf7fe640 1 --2- 172.21.15.134:0/972165267 >> v2:172.21.15.162:6800/980547263 conn(0x7f43bc038880 0x7f43bc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43cf7fe640 10 monclient: get_auth_request con 0x7f43bc038880 auth_method 0 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 10 monclient(hunting): handle_auth_done global_id 34419 payload 824 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.815+0000 7f43ceffd640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43d0061ae0 con 0x7f43d01cbb80 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43ceffd640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 msgr2=0x7f43d002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43ceffd640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 0x7f43d002b260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43c0011c00 tx=0x7f43c000f610 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43cf7fe640 1 --2- 172.21.15.134:0/972165267 >> v2:172.21.15.162:6800/980547263 conn(0x7f43bc038880 0x7f43bc03ad40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f43c80055a0 tx=0x7f43c8005530 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c0014ac0 con 0x7f43d01cbb80 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 10 monclient: dump: 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.836 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f43c0014c20 con 0x7f43d01cbb80 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 msgr2=0x7f43d002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 0x7f43d002b260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43c0011c00 tx=0x7f43c000f610 comp rx=0 tx=0).stop 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 msgr2=0x7f43d002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.818+0000 7f43b7fff640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbb80 0x7f43d002b260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.824+0000 7f43d582b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.824+0000 7f43d582b640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 0x7f43d002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.824+0000 7f43d582b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.824+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0122760 con 0x7f43d01cbf60 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 0x7f43d002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient: get_auth_request con 0x7f43d01cbf60 auth_method 0 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 10 monclient(hunting): handle_auth_done global_id 34428 payload 824 2024-09-16T15:05:28.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.825+0000 7f43ceffd640 1 -- 172.21.15.134:0/972165267 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43d007a680 con 0x7f43d01cbf60 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43ceffd640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 msgr2=0x7f43d002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43ceffd640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 0x7f43d002b7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43c0011c00 tx=0x7f43c00046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f43c001d480 con 0x7f43d01cbf60 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43b7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43b7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.827+0000 7f43b7fff640 10 monclient: dump: 2024-09-16T15:05:28.840 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.841 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f43c0026070 con 0x7f43d01cbf60 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 msgr2=0x7f43d002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 0x7f43d002b7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43c0011c00 tx=0x7f43c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 msgr2=0x7f43d002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43b7fff640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43d01cbf60 0x7f43d002b7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:28.842 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:28.843 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:28.846 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:28.847 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.849 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.851 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:28.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:28.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.861 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:28.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:28.863 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:28.864 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:28.865 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:28.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 >> v2:172.21.15.162:6800/980547263 conn(0x7f43bc038880 msgr2=0x7f43bc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 --2- 172.21.15.134:0/972165267 >> v2:172.21.15.162:6800/980547263 conn(0x7f43bc038880 0x7f43bc03ad40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f43c80055a0 tx=0x7f43c8005530 comp rx=0 tx=0).stop 2024-09-16T15:05:28.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 10 monclient: shutdown 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 msgr2=0x7f43d0134ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 --2- 172.21.15.134:0/972165267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43d0136e40 0x7f43d0134ef0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f43c40096f0 tx=0x7f43c40046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43cf7fe640 1 -- 172.21.15.134:0/972165267 reap_dead start 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43cf7fe640 1 -- 172.21.15.134:0/972165267 reap_dead start 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 shutdown_connections 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 >> 172.21.15.134:0/972165267 conn(0x7f43d00a6110 msgr2=0x7f43d009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 shutdown_connections 2024-09-16T15:05:28.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.828+0000 7f43d582b640 1 -- 172.21.15.134:0/972165267 wait complete. 2024-09-16T15:05:28.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:28.868 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:05:28.868 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.963+0000 7f85b0965640 10 monclient: get_monmap_and_config 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.963+0000 7f85b0965640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.963+0000 7f85b0965640 10 monclient: monmap: 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:28.964843+0000 2024-09-16T15:05:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:28.964843+0000 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:28.966 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 1 Processor -- start 2024-09-16T15:05:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 1 -- start start 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient: init 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient: _add_conns ranks=[8,10,3,7,9,2,6,4,0,1,5] 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-e con 0x7f85ac136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 0x7f85ac137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-f con 0x7f85ac137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ac135dc0 0x7f85ac136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.964+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-g con 0x7f85ac135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac09abc0 con 0x7f85ac137220 2024-09-16T15:05:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac09ba50 con 0x7f85ac135dc0 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0a53c0 con 0x7f85ac136e40 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85b0965640 10 monclient(hunting): authenticate will time out at 1065.554025s 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58624/0 (socket says 172.21.15.134:58624) 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 -- 172.21.15.134:0/2184315593 learned_addr learned my addr 172.21.15.134:0/2184315593 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): get_auth_request con 0x7f85ac136e40 auth_method 0 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2184315593 >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 0x7f85ac137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 10 monclient(hunting): get_auth_request con 0x7f85ac137220 auth_method 0 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ac135dc0 0x7f85ac136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aad9d640 10 monclient(hunting): get_auth_request con 0x7f85ac135dc0 auth_method 0 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient(hunting): handle_auth_done global_id 44261 payload 274 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 -- 172.21.15.134:0/2184315593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ac135dc0 msgr2=0x7f85ac136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ac135dc0 0x7f85ac136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 -- 172.21.15.134:0/2184315593 >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 msgr2=0x7f85ac137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2184315593 >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 0x7f85ac137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 monclient: found mon.noname-e 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85aa59c640 1 -- 172.21.15.134:0/2184315593 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac073250 con 0x7f85ac136e40 2024-09-16T15:05:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.965+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2184315593 >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 0x7f85ac137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85aa59c640 1 -- 172.21.15.134:0/2184315593 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 msgr2=0x7f85ac09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f859c0098a0 tx=0x7f859c02a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=324adf14d9ccf1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 1 -- 172.21.15.134:0/2184315593 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f859c02adf0 con 0x7f85ac136e40 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: dump: 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 5 monclient: authenticate success, global_id 44261 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 1 -- 172.21.15.134:0/2184315593 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c0307b0 con 0x7f85ac136e40 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 4 set_mon_vals no callback set 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 1 -- 172.21.15.134:0/2184315593 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f859c030dd0 con 0x7f85ac136e40 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85a9d9b640 10 monclient: dump: 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.982 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 10 monclient: shutdown 2024-09-16T15:05:28.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 -- 172.21.15.134:0/2184315593 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 msgr2=0x7f85ac09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f859c0098a0 tx=0x7f859c02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 -- 172.21.15.134:0/2184315593 shutdown_connections 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ac135dc0 0x7f85ac136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 --2- 172.21.15.134:0/2184315593 >> v2:172.21.15.162:3300/0 conn(0x7f85ac137220 0x7f85ac137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 --2- 172.21.15.134:0/2184315593 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f85ac136e40 0x7f85ac09c150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 -- 172.21.15.134:0/2184315593 >> 172.21.15.134:0/2184315593 conn(0x7f85ac0a6110 msgr2=0x7f85ac0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 -- 172.21.15.134:0/2184315593 shutdown_connections 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.968+0000 7f85b0965640 1 -- 172.21.15.134:0/2184315593 wait complete. 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.969+0000 7f85b0965640 10 monclient: build_initial_monmap 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.969+0000 7f85b0965640 10 monclient: monmap: 2024-09-16T15:05:28.984 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:28.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 Processor -- start 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 -- start start 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient: init 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient: _add_conns ranks=[7,6,1,8,2,4,5,3,0] 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85ac135dc0 0x7f85ac134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-i con 0x7f85ac135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-d con 0x7f85ac136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac137220 0x7f85ac121a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): picked mon.noname-f con 0x7f85ac137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:28.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1cc260 con 0x7f85ac137220 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1254e0 con 0x7f85ac136e40 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac1259c0 con 0x7f85ac135dc0 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85b0965640 10 monclient(hunting): authenticate will time out at 1065.559582s 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:47074/0 (socket says 172.21.15.134:47074) 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 -- 172.21.15.134:0/2437743582 learned_addr learned my addr 172.21.15.134:0/2437743582 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): get_auth_request con 0x7f85ac136e40 auth_method 0 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac137220 0x7f85ac121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): get_auth_request con 0x7f85ac137220 auth_method 0 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85ac135dc0 0x7f85ac134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): get_auth_request con 0x7f85ac135dc0 auth_method 0 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85aa59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient(hunting): handle_auth_done global_id 34449 payload 824 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85ac135dc0 msgr2=0x7f85ac134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85ac135dc0 0x7f85ac134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac137220 msgr2=0x7f85ac121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac137220 0x7f85ac121a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:05:28.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 1 monclient: found mon.noname-d 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.970+0000 7f85a1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.971+0000 7f85a1d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.971+0000 7f85a1d9b640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c0094d0 con 0x7f85ac136e40 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a1d9b640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 msgr2=0x7f85ac135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f859400d7d0 tx=0x7f859400dc90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85940044a0 con 0x7f85ac136e40 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: dump: 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.990 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:28.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400b8d0 con 0x7f85ac136e40 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 5 monclient: authenticate success, global_id 34449 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8594003610 con 0x7f85ac136e40 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 10 monclient: _renew_subs 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac136890 con 0x7f85ac136e40 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a959a640 4 set_mon_vals no callback set 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85a37fe640 10 monclient: dump: 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.992 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.993 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 10 monclient: _renew_subs 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.973+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac136340 con 0x7f85ac136e40 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85a1d9b640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f859400d7d0 tx=0x7f859400dc90 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f859404a310 osd_map(221..221 src has 1..221) v4 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8594004030 con 0x7f85ac136e40 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85a37fe640 1 --2- 172.21.15.134:0/2437743582 >> v2:172.21.15.162:6800/980547263 conn(0x7f858c038880 0x7f858c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85b0965640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85b0965640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 0x7f85ac121fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac075c90 con 0x7f85ac1cb550 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2437743582 >> v2:172.21.15.162:6800/980547263 conn(0x7f858c038880 0x7f858c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 0x7f85ac121fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aa59c640 10 monclient: get_auth_request con 0x7f858c038880 auth_method 0 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient: get_auth_request con 0x7f85ac1cb550 auth_method 0 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 10 monclient(hunting): handle_auth_done global_id 44204 payload 824 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.974+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85ac061ae0 con 0x7f85ac1cb550 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85aa59c640 1 --2- 172.21.15.134:0/2437743582 >> v2:172.21.15.162:6800/980547263 conn(0x7f858c038880 0x7f858c03ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f859c034000 tx=0x7f859c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 msgr2=0x7f85ac121fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 0x7f85ac121fa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85a40045f0 tx=0x7f85a4007700 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85a4014a00 con 0x7f85ac1cb550 2024-09-16T15:05:28.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 10 monclient: dump: 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:28.996 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f85a4014590 con 0x7f85ac1cb550 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 msgr2=0x7f85ac121fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 0x7f85ac121fa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85a40045f0 tx=0x7f85a4007700 comp rx=0 tx=0).stop 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 msgr2=0x7f85ac121fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:28.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.977+0000 7f85a37fe640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1cb550 0x7f85ac121fa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.983+0000 7f85b0965640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.983+0000 7f85b0965640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 0x7f85ac0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.983+0000 7f85b0965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.983+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac071b20 con 0x7f85ac1224e0 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 0x7f85ac0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient: get_auth_request con 0x7f85ac1224e0 auth_method 0 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:28.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 10 monclient(hunting): handle_auth_done global_id 44213 payload 824 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.984+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f85ac07a680 con 0x7f85ac1224e0 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.986+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 msgr2=0x7f85ac0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.986+0000 7f85aad9d640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 0x7f85ac0a1a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85a40045f0 tx=0x7f85a4004a40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85a40288c0 con 0x7f85ac1224e0 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:29.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 10 monclient: dump: 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.001 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f85a4028450 con 0x7f85ac1224e0 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 msgr2=0x7f85ac0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 0x7f85ac0a1a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85a40045f0 tx=0x7f85a4004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 msgr2=0x7f85ac0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f85ac1224e0 0x7f85ac0a1a20 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85a40045f0 tx=0x7f85a4004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:29.002 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:29.003 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 11, 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.004 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:29.006 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.007 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:29.008 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:29.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:29.012 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:29.014 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:29.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.017 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:29.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:29.020 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.024 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:29.024 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 >> v2:172.21.15.162:6800/980547263 conn(0x7f858c038880 msgr2=0x7f858c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 --2- 172.21.15.134:0/2437743582 >> v2:172.21.15.162:6800/980547263 conn(0x7f858c038880 0x7f858c03ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f859c034000 tx=0x7f859c02e000 comp rx=0 tx=0).stop 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 10 monclient: shutdown 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 msgr2=0x7f85ac135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 --2- 172.21.15.134:0/2437743582 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f85ac136e40 0x7f85ac135510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f859400d7d0 tx=0x7f859400dc90 comp rx=0 tx=0).stop 2024-09-16T15:05:29.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 reap_dead start 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85aad9d640 1 -- 172.21.15.134:0/2437743582 reap_dead start 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85a37fe640 1 -- 172.21.15.134:0/2437743582 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7f859404a310 con 0x7f85ac136e40 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 shutdown_connections 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 >> 172.21.15.134:0/2437743582 conn(0x7f85ac0a6110 msgr2=0x7f85ac06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 shutdown_connections 2024-09-16T15:05:29.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:28.987+0000 7f85b0965640 1 -- 172.21.15.134:0/2437743582 wait complete. 2024-09-16T15:05:29.026 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:29.026 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:05:29.026 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.122+0000 7fb008c20640 10 monclient: get_monmap_and_config 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.122+0000 7fb008c20640 10 monclient: build_initial_monmap 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.122+0000 7fb008c20640 10 monclient: monmap: 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:29.123723+0000 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:29.123723+0000 2024-09-16T15:05:29.124 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:29.125 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.123+0000 7fb008c20640 1 Processor -- start 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.123+0000 7fb008c20640 1 -- start start 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.123+0000 7fb008c20640 10 monclient: init 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient: _add_conns ranks=[7,1,2,6,3,4,8,5,10,9,0] 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb00409f390 0x7fb00409e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-i con 0x7fb00409f390 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb004066070 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-b con 0x7fb004066b80 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb004073250 con 0x7fb004066070 2024-09-16T15:05:29.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00409d870 con 0x7fb004066b80 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb00409f770 con 0x7fb00409f390 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb008c20640 10 monclient(hunting): authenticate will time out at 1065.713200s 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41158/0 (socket says 172.21.15.134:41158) 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 -- 172.21.15.134:0/3922525320 learned_addr learned my addr 172.21.15.134:0/3922525320 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): get_auth_request con 0x7fb004066b80 auth_method 0 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb00409f390 0x7fb00409e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): get_auth_request con 0x7fb00409f390 auth_method 0 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient(hunting): handle_auth_done global_id 44210 payload 274 2024-09-16T15:05:29.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 -- 172.21.15.134:0/3922525320 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb00409f390 msgr2=0x7fb00409e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb00409f390 0x7fb00409e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 -- 172.21.15.134:0/3922525320 >> v1:172.21.15.162:6789/0 conn(0x7fb004066070 legacy=0x7fb004066470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 monclient: found mon.noname-b 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:29.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.124+0000 7fb002d9d640 1 -- 172.21.15.134:0/3922525320 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00409dda0 con 0x7fb004066b80 2024-09-16T15:05:29.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb002d9d640 1 -- 172.21.15.134:0/3922525320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 msgr2=0x7fb00413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb002d9d640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faffc009870 tx=0x7faffc02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7560cee877de895a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 1 -- 172.21.15.134:0/3922525320 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faffc02adc0 con 0x7fb004066b80 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: dump: 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.137 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: _finish_auth 0 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: _check_auth_tickets 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 1 -- 172.21.15.134:0/3922525320 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc0307b0 con 0x7fb004066b80 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb008c20640 5 monclient: authenticate success, global_id 44210 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb008c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:29.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 1 -- 172.21.15.134:0/3922525320 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faffc030d90 con 0x7fb004066b80 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb008c20640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb008c20640 4 set_mon_vals no callback set 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.127+0000 7fb00159a640 10 monclient: dump: 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.139 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 10 monclient: shutdown 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 -- 172.21.15.134:0/3922525320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 msgr2=0x7fb00413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faffc009870 tx=0x7faffc02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 -- 172.21.15.134:0/3922525320 shutdown_connections 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb004066b80 0x7fb00413c530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 --2- 172.21.15.134:0/3922525320 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb00409f390 0x7fb00409e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 -- 172.21.15.134:0/3922525320 >> 172.21.15.134:0/3922525320 conn(0x7fb0040a57a0 msgr2=0x7fb0040a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 -- 172.21.15.134:0/3922525320 shutdown_connections 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 1 -- 172.21.15.134:0/3922525320 wait complete. 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 10 monclient: build_initial_monmap 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.128+0000 7fb008c20640 10 monclient: monmap: 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:29.141 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 Processor -- start 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 -- start start 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient: init 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient: _add_conns ranks=[4,5,2,8,7,0,3,1,6] 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 0x7fb004094b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-c con 0x7fb004066070 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb004066b80 0x7fb0041c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-h con 0x7fb004066b80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:29.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb00409f390 0x7fb0041c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): picked mon.noname-b con 0x7fb00409f390 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb004094270 con 0x7fb00409f390 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb004094830 con 0x7fb004066070 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040950c0 con 0x7fb004066b80 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb008c20640 10 monclient(hunting): authenticate will time out at 1065.718795s 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 0x7fb004094b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 0x7fb004094b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:57784/0 (socket says 172.21.15.134:57784) 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 1 -- 172.21.15.134:0/460322842 learned_addr learned my addr 172.21.15.134:0/460322842 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb00409f390 0x7fb0041c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): get_auth_request con 0x7fb004066070 auth_method 0 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): get_auth_request con 0x7fb00409f390 auth_method 0 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb00259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.129+0000 7fb002d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 10 monclient(hunting): handle_auth_done global_id 4837 payload 824 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb004066b80 0x7fb0041c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb004066b80 msgr2=0x7fb0041c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb004066b80 0x7fb0041c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb00409f390 msgr2=0x7fb0041c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb00409f390 0x7fb0041c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 monclient: found mon.noname-c 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb00259c640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0094d0 con 0x7fb004066070 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb001d9b640 10 monclient: get_auth_request con 0x7fb004066b80 auth_method 0 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb002d9d640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb00409f390 0x7fb0041c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.130+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb004066b80 0x7fb0041c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:29.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb00259c640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 msgr2=0x7fb004094b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb00259c640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 0x7fb004094b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faff400efc0 tx=0x7faff400c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faff4009b80 con 0x7fb004066070 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: dump: 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.146 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400f040 con 0x7fb004066070 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 5 monclient: authenticate success, global_id 4837 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 10 monclient: _renew_subs 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb004092d40 con 0x7fb004066070 2024-09-16T15:05:29.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faff4014d60 con 0x7fb004066070 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb000d99640 4 set_mon_vals no callback set 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7faff2ffd640 10 monclient: dump: 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.148 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 10 monclient: _renew_subs 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.132+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041c9fc0 con 0x7fb004066070 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb008c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb008c20640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 0x7fb004091090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040928c0 con 0x7fb004090cb0 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7faff4014480 con 0x7fb004066070 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7faff2ffd640 1 --2- 172.21.15.134:0/460322842 >> v2:172.21.15.162:6800/980547263 conn(0x7fafe4038880 0x7fafe403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 3931+0+0 (secure 0 0 0) 0x7faff404a970 con 0x7fb004066070 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 0x7fb004091090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb001d9b640 10 monclient: get_auth_request con 0x7fb004090cb0 auth_method 0 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb001d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb001d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb001d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.133+0000 7fb002d9d640 1 --2- 172.21.15.134:0/460322842 >> v2:172.21.15.162:6800/980547263 conn(0x7fafe4038880 0x7fafe403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb002d9d640 10 monclient: get_auth_request con 0x7fafe4038880 auth_method 0 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb001d9b640 10 monclient(hunting): handle_auth_done global_id 4741 payload 824 2024-09-16T15:05:29.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.134+0000 7fb001d9b640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0040943e0 con 0x7fb004090cb0 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7fb001d9b640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 msgr2=0x7fb004091090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 0x7fb004091090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff800d950 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7fb002d9d640 1 --2- 172.21.15.134:0/460322842 >> v2:172.21.15.162:6800/980547263 conn(0x7fafe4038880 0x7fafe403ad40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb00409ba70 tx=0x7faffc02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faff8012c80 con 0x7fb004090cb0 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 10 monclient: dump: 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.153 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.154 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faff8012de0 con 0x7fb004090cb0 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 msgr2=0x7fb004091090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 0x7fb004091090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff800d950 comp rx=0 tx=0).stop 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 msgr2=0x7fb004091090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.136+0000 7faff2ffd640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb004090cb0 0x7fb004091090 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff800d950 comp rx=0 tx=0).stop 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.142+0000 7fb008c20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.142+0000 7fb008c20640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 0x7fb00408cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb008c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb004073880 con 0x7fb0040915d0 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 0x7fb00408cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient: get_auth_request con 0x7fb0040915d0 auth_method 0 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 10 monclient(hunting): handle_auth_done global_id 4750 payload 824 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.143+0000 7fb001d9b640 1 -- 172.21.15.134:0/460322842 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb0040949a0 con 0x7fb0040915d0 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.145+0000 7fb001d9b640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 msgr2=0x7fb00408cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.145+0000 7fb001d9b640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 0x7fb00408cfb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff8003760 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faff8028070 con 0x7fb0040915d0 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 10 monclient: dump: 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7faff8020040 con 0x7fb0040915d0 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 msgr2=0x7fb00408cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 0x7fb00408cfb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff8003760 comp rx=0 tx=0).stop 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 msgr2=0x7fb00408cfb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:29.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7faff2ffd640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0040915d0 0x7fb00408cfb0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0040a2100 tx=0x7faff8003760 comp rx=0 tx=0).stop 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 42, 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:29.159 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 11, 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.160 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.161 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:29.162 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:29.163 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:29.164 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.165 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:29.167 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:29.172 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:29.177 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:29.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:29.180 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:29.181 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 >> v2:172.21.15.162:6800/980547263 conn(0x7fafe4038880 msgr2=0x7fafe403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 --2- 172.21.15.134:0/460322842 >> v2:172.21.15.162:6800/980547263 conn(0x7fafe4038880 0x7fafe403ad40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb00409ba70 tx=0x7faffc02e000 comp rx=0 tx=0).stop 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 10 monclient: shutdown 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 msgr2=0x7fb004094b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 --2- 172.21.15.134:0/460322842 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb004066070 0x7fb004094b80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faff400efc0 tx=0x7faff400c490 comp rx=0 tx=0).stop 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb002d9d640 1 -- 172.21.15.134:0/460322842 reap_dead start 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb002d9d640 1 -- 172.21.15.134:0/460322842 reap_dead start 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 shutdown_connections 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 >> 172.21.15.134:0/460322842 conn(0x7fb0040a57a0 msgr2=0x7fb00406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:29.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 shutdown_connections 2024-09-16T15:05:29.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.146+0000 7fb008c20640 1 -- 172.21.15.134:0/460322842 wait complete. 2024-09-16T15:05:29.184 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:05:29.184 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'i'] 2024-09-16T15:05:29.184 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:05:29.184 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-16T15:05:29.184 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:05:29.185 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-16T15:05:29.185 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:05:29.199 INFO:tasks.ceph.mon.g:Stopped 2024-09-16T15:05:29.199 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-16T15:05:29.199 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:05:29.199 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-16T15:05:29.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:05:29.241 INFO:tasks.rados.rados.0.smithi162.stdout:1103: done (1 left) 2024-09-16T15:05:29.241 INFO:tasks.rados.rados.0.smithi162.stdout:1104: write oid 1 current snap is 99 2024-09-16T15:05:29.241 INFO:tasks.rados.rados.0.smithi162.stdout:1104: seq_num 424 ranges {726168=704991,1895071=683474} 2024-09-16T15:05:29.250 INFO:tasks.rados.rados.0.smithi162.stdout:1104: writing smithi16234619-1 from 726168 to 1431159 tid 1 2024-09-16T15:05:29.256 INFO:tasks.ceph.mon.i:Stopped 2024-09-16T15:05:29.257 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-16T15:05:29.257 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:1104: writing smithi16234619-1 from 1895071 to 2578545 tid 2 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:1105: read oid 27 snap -1 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:1105: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:1106: rollback oid 5 current snap is 99 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 96 2024-09-16T15:05:29.258 INFO:tasks.rados.rados.0.smithi162.stdout:1102: finishing write tid 1 to smithi16234619-2 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: finishing write tid 2 to smithi16234619-2 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: finishing write tid 3 to smithi16234619-2 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: finishing write tid 4 to smithi16234619-2 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: finishing write tid 5 to smithi16234619-2 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: oid 2 updating version 0 to 406 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: oid 2 updating version 406 to 409 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: oid 2 version 409 is already newer than 407 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: oid 2 version 409 is already newer than 408 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 409 (ObjNum 423 snap 99 seq_num 423) dirty exists 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: left oid 2 (ObjNum 423 snap 99 seq_num 423) 2024-09-16T15:05:29.259 INFO:tasks.rados.rados.0.smithi162.stdout:1102: done (3 left) 2024-09-16T15:05:29.260 INFO:tasks.rados.rados.0.smithi162.stdout:1107: snap_create 2024-09-16T15:05:29.260 INFO:tasks.rados.rados.0.smithi162.stdout:1106: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:05:29.260 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:05:29.284 INFO:tasks.rados.rados.0.smithi162.stdout:1104: finishing write tid 1 to smithi16234619-1 2024-09-16T15:05:29.290 INFO:tasks.rados.rados.0.smithi162.stdout:1104: finishing write tid 2 to smithi16234619-1 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: finishing write tid 3 to smithi16234619-1 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: finishing write tid 4 to smithi16234619-1 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: oid 1 updating version 0 to 364 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: oid 1 updating version 364 to 365 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: oid 1 updating version 365 to 366 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 366 (ObjNum 424 snap 99 seq_num 424) dirty exists 2024-09-16T15:05:29.291 INFO:tasks.rados.rados.0.smithi162.stdout:1104: left oid 1 (ObjNum 424 snap 99 seq_num 424) 2024-09-16T15:05:29.292 INFO:tasks.rados.rados.0.smithi162.stdout:1105: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:05:29.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.353+0000 7f1789b5e640 10 monclient: get_monmap_and_config 2024-09-16T15:05:29.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.353+0000 7f1789b5e640 10 monclient: build_initial_monmap 2024-09-16T15:05:29.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.353+0000 7f1789b5e640 10 monclient: monmap: 2024-09-16T15:05:29.354 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:29.354 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:29.354070+0000 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:29.354070+0000 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:29.355 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:29.356 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:29.356 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:29.356 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 Processor -- start 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 -- start start 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient: init 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient: _add_conns ranks=[7,3,1,6,2,10,4,8,5,9,0] 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1784136030 0x7f1784136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-i con 0x7f1784136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f178409bfe0 0x7f178409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-g con 0x7f178409bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1784136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1784073250 con 0x7f1784136c60 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178409b600 con 0x7f178409bfe0 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17840a5740 con 0x7f1784136030 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1789b5e640 10 monclient(hunting): authenticate will time out at 1065.943625s 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f177bfff640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f178409bfe0 msgr2=0x7f178409c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f17837fe640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1784136030 msgr2=0x7f1784136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f177bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f178409bfe0 0x7f178409c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f17837fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1784136030 0x7f1784136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1783fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1784136c60 0x7f17841370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:57124/0 (socket says 172.21.15.134:57124) 2024-09-16T15:05:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.354+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 learned_addr learned my addr 172.21.15.134:0/2866047998 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 >> v1:172.21.15.162:6789/0 conn(0x7f1784136c60 legacy=0x7f17841370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597973948 0 0) 0x7f1784073250 con 0x7f1784136c60 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): _negotiate 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): my global_id is 44218 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1770003520 con 0x7f1784136c60 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2191720504 0 0) 0x7f1770003520 con 0x7f1784136c60 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1784136030 msgr2=0x7f1784136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1784136030 0x7f1784136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f178409bfe0 msgr2=0x7f178409c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f178409bfe0 0x7f178409c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840a52c0 con 0x7f1784136c60 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:29.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f177c001fd0 con 0x7f1784136c60 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: dump: 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.368 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 >> v1:172.21.15.162:6789/0 conn(0x7f1784136c60 legacy=0x7f17841370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient: _add_conns ranks=[2,1,4,7,5,8,0,3,6] 2024-09-16T15:05:29.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17700041c0 0x7f17700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): picked mon.b con 0x7f17700041c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1770004eb0 0x7f1770009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): picked mon.f con 0x7f1770004eb0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 0x7f177000c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): picked mon.c con 0x7f1770009eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178409b600 con 0x7f1770004eb0 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840a5740 con 0x7f17700041c0 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f177000cb10 con 0x7f1770009eb0 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:29.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1782ffd640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 0x7f177000c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f17837fe640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17700041c0 0x7f17700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f177bfff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1770004eb0 0x7f1770009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 10 monclient(hunting): get_auth_request con 0x7f1770009eb0 auth_method 0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f1783fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f177bfff640 10 monclient(hunting): get_auth_request con 0x7f1770004eb0 auth_method 0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f177bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f177bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f177bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f17837fe640 10 monclient(hunting): get_auth_request con 0x7f17700041c0 auth_method 0 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f17837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f17837fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.357+0000 7f17837fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient(hunting): handle_auth_done global_id 44218 payload 293 2024-09-16T15:05:29.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17700041c0 msgr2=0x7f17700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17700041c0 0x7f17700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1770004eb0 msgr2=0x7f1770009970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1770004eb0 0x7f1770009970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 monclient: found mon.c 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770003520 con 0x7f1770009eb0 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1789b5e640 5 monclient: authenticate success, global_id 44218 2024-09-16T15:05:29.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1789b5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f177bfff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1770004eb0 0x7f1770009970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.358+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 reap_dead start 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.360+0000 7f1783fff640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 msgr2=0x7f177000c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.360+0000 7f1783fff640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 0x7f177000c2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f177c002230 tx=0x7f17700057f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7f06d92ab91d91f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f177c02bc00 con 0x7f1770009eb0 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: dump: 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c02bda0 con 0x7f1770009eb0 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 1 -- 172.21.15.134:0/2866047998 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f177c008540 con 0x7f1770009eb0 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 4 set_mon_vals no callback set 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1782ffd640 10 monclient: dump: 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.376 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 10 monclient: shutdown 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 -- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 msgr2=0x7f177000c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 0x7f177000c2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f177c002230 tx=0x7f17700057f0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 -- 172.21.15.134:0/2866047998 shutdown_connections 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 --2- 172.21.15.134:0/2866047998 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1770009eb0 0x7f177000c2a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 -- 172.21.15.134:0/2866047998 >> 172.21.15.134:0/2866047998 conn(0x7f17840a9ea0 msgr2=0x7f17840a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:29.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 -- 172.21.15.134:0/2866047998 shutdown_connections 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.361+0000 7f1789b5e640 1 -- 172.21.15.134:0/2866047998 wait complete. 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.362+0000 7f1789b5e640 10 monclient: build_initial_monmap 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.362+0000 7f1789b5e640 10 monclient: monmap: 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:29.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.362+0000 7f1789b5e640 1 Processor -- start 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.362+0000 7f1789b5e640 1 -- start start 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.362+0000 7f1789b5e640 10 monclient: init 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient: _add_conns ranks=[2,6,5,1,3,4,8,0,7] 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-b con 0x7f1770009eb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 0x7f17841227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-d con 0x7f17841cbac0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1784122d00 0x7f1784123180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): picked mon.noname-h con 0x7f1784122d00 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840a7a70 con 0x7f1770009eb0 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1784125de0 con 0x7f1784122d00 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f178409f610 con 0x7f17841cbac0 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1789b5e640 10 monclient(hunting): authenticate will time out at 1065.952209s 2024-09-16T15:05:29.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 0x7f17841227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41188/0 (socket says 172.21.15.134:41188) 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 1 -- 172.21.15.134:0/1841394050 learned_addr learned my addr 172.21.15.134:0/1841394050 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): get_auth_request con 0x7f1770009eb0 auth_method 0 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): get_auth_request con 0x7f17841cbac0 auth_method 0 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1784122d00 0x7f1784123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): get_auth_request con 0x7f1784122d00 auth_method 0 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f1783fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient(hunting): handle_auth_done global_id 34458 payload 824 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 -- 172.21.15.134:0/1841394050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1784122d00 msgr2=0x7f1784123180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1784122d00 0x7f1784123180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 -- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 msgr2=0x7f17841cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 monclient: found mon.noname-d 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f177bfff640 1 -- 172.21.15.134:0/1841394050 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784125080 con 0x7f17841cbac0 2024-09-16T15:05:29.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.363+0000 7f17837fe640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f177bfff640 1 -- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 msgr2=0x7f17841227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f177bfff640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 0x7f17841227c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17840a5950 tx=0x7f176c00eba0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f176c003bb0 con 0x7f17841cbac0 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 10 monclient: dump: 2024-09-16T15:05:29.385 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.386 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 10 monclient: _finish_auth 0 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c003d10 con 0x7f17841cbac0 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1789b5e640 5 monclient: authenticate success, global_id 34458 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1789b5e640 10 monclient: _renew_subs 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1789b5e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.365+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17841242b0 con 0x7f17841cbac0 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f176c003e80 con 0x7f17841cbac0 2024-09-16T15:05:29.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 10 monclient: dump: 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:29.388 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 10 monclient: _renew_subs 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1781ffb640 4 set_mon_vals no callback set 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17841263a0 con 0x7f17841cbac0 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f176c00c730 con 0x7f17841cbac0 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:29.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.366+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17840008d0 con 0x7f17841cbac0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.367+0000 7f1780ff9640 1 --2- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 0x7f176403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.367+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f176c04a0d0 con 0x7f17841cbac0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.367+0000 7f17837fe640 1 --2- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 0x7f176403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.367+0000 7f17837fe640 10 monclient: get_auth_request con 0x7f1764038880 auth_method 0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.369+0000 7f17837fe640 1 --2- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 0x7f176403ad40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1774006f20 tx=0x7f1774006f50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.371+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f176c022300 con 0x7f17841cbac0 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.371+0000 7f1780ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:29.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.371+0000 7f1780ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1789b5e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1789b5e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1789b5e640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1784072070 con 0x7f17841cbac0 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1780ff9640 1 -- 172.21.15.134:0/1841394050 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f176c016020 con 0x7f17841cbac0 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1780ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.596+0000 7f1780ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:29.598 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":42,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 msgr2=0x7f176403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 0x7f176403ad40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1774006f20 tx=0x7f1774006f50 comp rx=0 tx=0).stop 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 10 monclient: shutdown 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 msgr2=0x7f17841227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:29.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 0x7f17841227c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17840a5950 tx=0x7f176c00eba0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 shutdown_connections 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> v2:172.21.15.162:6800/980547263 conn(0x7f1764038880 0x7f176403ad40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1784122d00 0x7f1784123180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f17841cbac0 0x7f17841227c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 --2- 172.21.15.134:0/1841394050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1770009eb0 0x7f17841cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 >> 172.21.15.134:0/1841394050 conn(0x7f17840a9ea0 msgr2=0x7f178406b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 shutdown_connections 2024-09-16T15:05:29.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:29.598+0000 7f1789b5e640 1 -- 172.21.15.134:0/1841394050 wait complete. 2024-09-16T15:05:32.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:32.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.717+0000 7fcfccb33640 10 monclient: get_monmap_and_config 2024-09-16T15:05:32.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.717+0000 7fcfccb33640 10 monclient: build_initial_monmap 2024-09-16T15:05:32.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.718+0000 7fcfccb33640 10 monclient: monmap: 2024-09-16T15:05:32.719 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:32.719 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:32.718949+0000 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:32.718949+0000 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:32.720 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:32.721 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:32.721 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:32.721 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.718+0000 7fcfccb33640 1 Processor -- start 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.718+0000 7fcfccb33640 1 -- start start 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.718+0000 7fcfccb33640 10 monclient: init 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient: _add_conns ranks=[0,6,3,10,1,2,4,8,7,9,5] 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcfc8136030 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 0x7fcfc809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-d con 0x7fcfc809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 0x7fcfc813e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-g con 0x7fcfc813bf80 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:32.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc8073250 con 0x7fcfc8136030 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc809b600 con 0x7fcfc813bf80 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80a5740 con 0x7fcfc809bfe0 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfccb33640 10 monclient(hunting): authenticate will time out at 1069.308390s 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfc659c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcfc8136030 0x7fcfc8136410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42950/0 (socket says 172.21.15.134:42950) 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfc659c640 1 -- 172.21.15.134:0/2498948094 learned_addr learned my addr 172.21.15.134:0/2498948094 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfc6d9d640 1 -- 172.21.15.134:0/2498948094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 msgr2=0x7fcfc813e370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 0x7fcfc809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfc6d9d640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 0x7fcfc813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): get_auth_request con 0x7fcfc809bfe0 auth_method 0 2024-09-16T15:05:32.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient(hunting): handle_auth_done global_id 34467 payload 274 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 -- 172.21.15.134:0/2498948094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 msgr2=0x7fcfc813e370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 0x7fcfc813e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 -- 172.21.15.134:0/2498948094 >> v1:172.21.15.134:6789/0 conn(0x7fcfc8136030 legacy=0x7fcfc8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 monclient: found mon.noname-d 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:32.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.719+0000 7fcfbffff640 1 -- 172.21.15.134:0/2498948094 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80a52c0 con 0x7fcfc809bfe0 2024-09-16T15:05:32.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfbffff640 1 -- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 msgr2=0x7fcfc809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:32.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 0x7fcfc809c400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00097d0 tx=0x7fcfb002a010 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1eec2194457113cc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:32.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 1 -- 172.21.15.134:0/2498948094 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfb002ae10 con 0x7fcfc809bfe0 2024-09-16T15:05:32.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:32.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: dump: 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:32.732 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: _finish_auth 0 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfccb33640 5 monclient: authenticate success, global_id 34467 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 1 -- 172.21.15.134:0/2498948094 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb00307b0 con 0x7fcfc809bfe0 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfccb33640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:32.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfccb33640 4 set_mon_vals no callback set 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 1 -- 172.21.15.134:0/2498948094 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfb0030d90 con 0x7fcfc809bfe0 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.722+0000 7fcfc5d9b640 10 monclient: dump: 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:32.734 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 10 monclient: shutdown 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 -- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 msgr2=0x7fcfc809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 0x7fcfc809c400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00097d0 tx=0x7fcfb002a010 comp rx=0 tx=0).stop 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 -- 172.21.15.134:0/2498948094 shutdown_connections 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc813bf80 0x7fcfc813e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2498948094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcfc809bfe0 0x7fcfc809c400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 -- 172.21.15.134:0/2498948094 >> 172.21.15.134:0/2498948094 conn(0x7fcfc80a9ea0 msgr2=0x7fcfc80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:32.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 -- 172.21.15.134:0/2498948094 shutdown_connections 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.723+0000 7fcfccb33640 1 -- 172.21.15.134:0/2498948094 wait complete. 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.724+0000 7fcfccb33640 10 monclient: build_initial_monmap 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.724+0000 7fcfccb33640 10 monclient: monmap: 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:32.736 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.724+0000 7fcfccb33640 1 Processor -- start 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.724+0000 7fcfccb33640 1 -- start start 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.724+0000 7fcfccb33640 10 monclient: init 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient: _add_conns ranks=[7,8,5,6,0,1,3,4,2] 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 0x7fcfc81c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-i con 0x7fcfc809bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:32.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-e con 0x7fcfc8136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 0x7fcfc81cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): picked mon.noname-h con 0x7fcfc813bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8120710 con 0x7fcfc813bf80 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc81c90a0 con 0x7fcfc809bfe0 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc809fe30 con 0x7fcfc8136030 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfccb33640 10 monclient(hunting): authenticate will time out at 1069.314229s 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc659c640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 msgr2=0x7fcfc81c8150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc659c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 0x7fcfc81c8150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:32.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58652/0 (socket says 172.21.15.134:58652) 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 -- 172.21.15.134:0/2489966398 learned_addr learned my addr 172.21.15.134:0/2489966398 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): get_auth_request con 0x7fcfc8136030 auth_method 0 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 0x7fcfc81cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 10 monclient(hunting): get_auth_request con 0x7fcfc813bf80 auth_method 0 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:32.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient(hunting): handle_auth_done global_id 44270 payload 824 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 -- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 msgr2=0x7fcfc81c8150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 0x7fcfc81c8150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 -- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 msgr2=0x7fcfc81cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 0x7fcfc81cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 monclient: found mon.noname-e 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfbffff640 1 -- 172.21.15.134:0/2489966398 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009400 con 0x7fcfc8136030 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.725+0000 7fcfc6d9d640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 0x7fcfc81cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.727+0000 7fcfbffff640 1 -- 172.21.15.134:0/2489966398 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 msgr2=0x7fcfc8134290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.727+0000 7fcfbffff640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00096f0 tx=0x7fcfb00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:32.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfb0030d40 con 0x7fcfc8136030 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: dump: 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:32.741 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 5 monclient: authenticate success, global_id 44270 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0003a80 con 0x7fcfc8136030 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 10 monclient: _renew_subs 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:32.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc80a49f0 con 0x7fcfc8136030 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfb0003bf0 con 0x7fcfc8136030 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfbeffd640 10 monclient: dump: 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:32.743 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfc559a640 4 set_mon_vals no callback set 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 10 monclient: _renew_subs 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.728+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc8120c60 con 0x7fcfc8136030 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcfb0039040 con 0x7fcfc8136030 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfccb33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:32.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfccb33640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfccb33640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc8120850 con 0x7fcfc8136030 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfbeffd640 1 --2- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 0x7fcfa803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fcfb006aaa0 con 0x7fcfc8136030 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfc659c640 1 --2- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 0x7fcfa803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.729+0000 7fcfc659c640 10 monclient: get_auth_request con 0x7fcfa8038880 auth_method 0 2024-09-16T15:05:32.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.732+0000 7fcfc659c640 1 --2- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 0x7fcfa803ad40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8007fd0 tx=0x7fcfb8007b20 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:32.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.732+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:05:32.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.734+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcfc8120850 con 0x7fcfc8136030 2024-09-16T15:05:32.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.734+0000 7fcfbeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:32.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.734+0000 7fcfbeffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfccb33640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfccb33640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfccb33640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcfc8120850 con 0x7fcfc8136030 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfbeffd640 1 -- 172.21.15.134:0/2489966398 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fcfc8120850 con 0x7fcfc8136030 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfbeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.959+0000 7fcfbeffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:32.961 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":42,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 msgr2=0x7fcfa803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 0x7fcfa803ad40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8007fd0 tx=0x7fcfb8007b20 comp rx=0 tx=0).stop 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 10 monclient: shutdown 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 msgr2=0x7fcfc8134290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00096f0 tx=0x7fcfb00046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 shutdown_connections 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfa8038880 0x7fcfa803ad40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcfc813bf80 0x7fcfc81cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcfc8136030 0x7fcfc8134290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 --2- 172.21.15.134:0/2489966398 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcfc809bfe0 0x7fcfc81c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:32.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 >> 172.21.15.134:0/2489966398 conn(0x7fcfc80a9ea0 msgr2=0x7fcfc80a8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:32.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 shutdown_connections 2024-09-16T15:05:32.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:32.961+0000 7fcfccb33640 1 -- 172.21.15.134:0/2489966398 wait complete. 2024-09-16T15:05:35.975 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.083+0000 7f19c5576640 10 monclient: get_monmap_and_config 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.083+0000 7f19c5576640 10 monclient: build_initial_monmap 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.083+0000 7f19c5576640 10 monclient: monmap: 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:36.084761+0000 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:36.084761+0000 2024-09-16T15:05:36.085 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:36.086 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 Processor -- start 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 -- start start 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient: init 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient: _add_conns ranks=[10,3,7,2,0,9,5,8,1,6,4] 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f19c0136030 0x7f19c0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-f con 0x7f19c0136030 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 0x7f19c009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-g con 0x7f19c009bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:36.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 0x7f19c01370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-i con 0x7f19c0136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c009b600 con 0x7f19c0136030 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.084+0000 7f19c5576640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a5740 con 0x7f19c009bfe0 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19c5576640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a52c0 con 0x7f19c0136c60 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19c5576640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19c5576640 10 monclient(hunting): authenticate will time out at 1072.674012s 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19bf7fe640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 msgr2=0x7f19c01370e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19be7fc640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 msgr2=0x7f19c009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19be7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 0x7f19c009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:36.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19bf7fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 0x7f19c01370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f19c0136030 0x7f19c0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f19c0136030 0x7f19c0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:33474/0 (socket says 172.21.15.134:33474) 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 -- 172.21.15.134:0/1582102030 learned_addr learned my addr 172.21.15.134:0/1582102030 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): get_auth_request con 0x7f19c0136030 auth_method 0 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient(hunting): handle_auth_done global_id 44227 payload 274 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 -- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 msgr2=0x7f19c01370e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 0x7f19c01370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 -- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 msgr2=0x7f19c009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 0x7f19c009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 monclient: found mon.noname-f 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:05:36.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.085+0000 7f19beffd640 1 -- 172.21.15.134:0/1582102030 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0073250 con 0x7f19c0136030 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.088+0000 7f19beffd640 1 -- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19c0136030 msgr2=0x7f19c0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.088+0000 7f19beffd640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19c0136030 0x7f19c0136410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009870 tx=0x7f19b002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bfe0bf457a004d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.088+0000 7f19bdffb640 1 -- 172.21.15.134:0/1582102030 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19b00307b0 con 0x7f19c0136030 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: dump: 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:36.098 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:36.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 1 -- 172.21.15.134:0/1582102030 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b0030930 con 0x7f19c0136030 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 5 monclient: authenticate success, global_id 44227 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 1 -- 172.21.15.134:0/1582102030 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19b00313a0 con 0x7f19c0136030 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 4 set_mon_vals no callback set 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19bdffb640 10 monclient: dump: 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:36.100 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 10 monclient: shutdown 2024-09-16T15:05:36.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 -- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19c0136030 msgr2=0x7f19c0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19c0136030 0x7f19c0136410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009870 tx=0x7f19b002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 -- 172.21.15.134:0/1582102030 shutdown_connections 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c0136c60 0x7f19c01370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19c009bfe0 0x7f19c009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 --2- 172.21.15.134:0/1582102030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19c0136030 0x7f19c0136410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 -- 172.21.15.134:0/1582102030 >> 172.21.15.134:0/1582102030 conn(0x7f19c00a9ea0 msgr2=0x7f19c00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 -- 172.21.15.134:0/1582102030 shutdown_connections 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.089+0000 7f19c5576640 1 -- 172.21.15.134:0/1582102030 wait complete. 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.090+0000 7f19c5576640 10 monclient: build_initial_monmap 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.090+0000 7f19c5576640 10 monclient: monmap: 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:36.102 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.090+0000 7f19c5576640 1 Processor -- start 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.090+0000 7f19c5576640 1 -- start start 2024-09-16T15:05:36.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.090+0000 7f19c5576640 10 monclient: init 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient: _add_conns ranks=[7,0,2,4,1,6,8,3,5] 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 0x7f19c0134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-i con 0x7f19c009bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-a con 0x7f19c0136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 0x7f19c011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): picked mon.noname-b con 0x7f19c0136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00a7ac0 con 0x7f19c0136030 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c009ff20 con 0x7f19c0136c60 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0122580 con 0x7f19c009bfe0 2024-09-16T15:05:36.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19c5576640 10 monclient(hunting): authenticate will time out at 1072.680287s 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19beffd640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 msgr2=0x7f19c0134bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19beffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 0x7f19c0134bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 0x7f19c011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48994/0 (socket says 172.21.15.134:48994) 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 -- 172.21.15.134:0/1315023383 learned_addr learned my addr 172.21.15.134:0/1315023383 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): get_auth_request con 0x7f19c0136030 auth_method 0 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): get_auth_request con 0x7f19c0136c60 auth_method 0 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:36.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient(hunting): handle_auth_done global_id 4914 payload 824 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 -- 172.21.15.134:0/1315023383 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 msgr2=0x7f19c0134bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 0x7f19c0134bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 -- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 msgr2=0x7f19c011fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 0x7f19c011fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 monclient: found mon.noname-a 2024-09-16T15:05:36.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19be7fc640 1 -- 172.21.15.134:0/1315023383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00094d0 con 0x7f19c0136030 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.091+0000 7f19bf7fe640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 0x7f19c011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19be7fc640 1 -- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 msgr2=0x7f19c0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19be7fc640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b550 tx=0x7f19b400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19b4003cd0 con 0x7f19c0136030 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: dump: 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.107 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:36.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 5 monclient: authenticate success, global_id 4914 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b4003e50 con 0x7f19c0136030 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19b40054e0 con 0x7f19c0136030 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 10 monclient: _renew_subs 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c0121ce0 con 0x7f19c0136030 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19a77fe640 10 monclient: dump: 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:36.109 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19bd7fa640 4 set_mon_vals no callback set 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 10 monclient: _renew_subs 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.094+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c00a3050 con 0x7f19c0136030 2024-09-16T15:05:36.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f19b4005070 con 0x7f19c0136030 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19c5576640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19c5576640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19c5576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c009c3e0 con 0x7f19c0136030 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19a77fe640 1 --2- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 0x7f19ac03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f19b4049e80 con 0x7f19c0136030 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.095+0000 7f19beffd640 1 --2- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 0x7f19ac03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.096+0000 7f19beffd640 10 monclient: get_auth_request con 0x7f19ac038880 auth_method 0 2024-09-16T15:05:36.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.098+0000 7f19beffd640 1 --2- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 0x7f19ac03ad40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b0031000 tx=0x7f19b002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:36.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.100+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f19b4005320 con 0x7f19c0136030 2024-09-16T15:05:36.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.100+0000 7f19a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:36.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.100+0000 7f19a77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.322+0000 7f19c5576640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.322+0000 7f19c5576640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.322+0000 7f19c5576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:36.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.322+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f19c0122200 con 0x7f19c0136030 2024-09-16T15:05:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.323+0000 7f19a77fe640 1 -- 172.21.15.134:0/1315023383 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f19b4018b50 con 0x7f19c0136030 2024-09-16T15:05:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.323+0000 7f19a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:36.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.323+0000 7f19a77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:36.325 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":42,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:36.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 msgr2=0x7f19ac03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 0x7f19ac03ad40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19b0031000 tx=0x7f19b002e000 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 10 monclient: shutdown 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 msgr2=0x7f19c0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b550 tx=0x7f19b400ba10 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 shutdown_connections 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> v2:172.21.15.162:6800/980547263 conn(0x7f19ac038880 0x7f19ac03ad40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19c0136c60 0x7f19c011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19c0136030 0x7f19c0135100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 --2- 172.21.15.134:0/1315023383 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f19c009bfe0 0x7f19c0134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 >> 172.21.15.134:0/1315023383 conn(0x7f19c00a9ea0 msgr2=0x7f19c01bbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 shutdown_connections 2024-09-16T15:05:36.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:36.325+0000 7f19c5576640 1 -- 172.21.15.134:0/1315023383 wait complete. 2024-09-16T15:05:39.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:05:39.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.444+0000 7f1239be2640 10 monclient: get_monmap_and_config 2024-09-16T15:05:39.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.444+0000 7f1239be2640 10 monclient: build_initial_monmap 2024-09-16T15:05:39.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.444+0000 7f1239be2640 10 monclient: monmap: 2024-09-16T15:05:39.446 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:39.446071+0000 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:39.446071+0000 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:39.447 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:39.448 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 Processor -- start 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 -- start start 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient: init 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient: _add_conns ranks=[4,0,1,7,9,8,6,10,5,2,3] 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:39.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-c con 0x7f1234136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f123409bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f123409c400 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1234073250 con 0x7f123409bfe0 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123409b600 con 0x7f123409c400 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12340a5740 con 0x7f1234136030 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f1239be2640 10 monclient(hunting): authenticate will time out at 1076.035700s 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f12337fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f122bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f123409bfe0 0x7f1234136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41262/0 (socket says 172.21.15.134:41262) 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 learned_addr learned my addr 172.21.15.134:0/3037262696 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1234136030 auth_method 0 2024-09-16T15:05:39.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.445+0000 7f12337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.446+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.446+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.446+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.446+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.448+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v1:172.21.15.134:6789/0 conn(0x7f123409bfe0 legacy=0x7f1234136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:39.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.448+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> v1:172.21.15.162:6789/0 conn(0x7f123409c400 legacy=0x7f1234137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1234136030 auth_method 0 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:39.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:39.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:39.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:39.647+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:40.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1234136030 auth_method 0 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:40.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.048+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:40.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:40.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1234136030 auth_method 0 2024-09-16T15:05:40.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:40.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:40.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:40.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:40.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:40.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:40.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:40.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:42.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient(hunting): tick 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 msgr2=0x7f1234136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1234136030 0x7f1234136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> v1:172.21.15.162:6789/0 conn(0x7f123409c400 legacy=0x7f1234137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> v1:172.21.15.134:6789/0 conn(0x7f123409bfe0 legacy=0x7f1234136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient: _add_conns ranks=[2,10,6,7,3,8,5,9] 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f1214003940 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:42.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-f con 0x7f1214006410 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.446+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-d con 0x7f1214008f30 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340a5740 con 0x7f1214006410 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bb30 con 0x7f1214003940 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bcf0 con 0x7f1214008f30 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1231ffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1214003940 auth_method 0 2024-09-16T15:05:42.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 10 monclient(hunting): get_auth_request con 0x7f1214008f30 auth_method 0 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 10 monclient(hunting): get_auth_request con 0x7f1214006410 auth_method 0 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:42.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.447+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:42.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1214003940 auth_method 0 2024-09-16T15:05:42.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 10 monclient(hunting): get_auth_request con 0x7f1214008f30 auth_method 0 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:42.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 10 monclient(hunting): get_auth_request con 0x7f1214006410 auth_method 0 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:42.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:42.648+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:05:43.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 10 monclient(hunting): get_auth_request con 0x7f1214008f30 auth_method 0 2024-09-16T15:05:43.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1214003940 auth_method 0 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 10 monclient(hunting): get_auth_request con 0x7f1214006410 auth_method 0 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.049+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:05:43.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:43.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 10 monclient(hunting): get_auth_request con 0x7f1214008f30 auth_method 0 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1214003940 auth_method 0 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 10 monclient(hunting): get_auth_request con 0x7f1214006410 auth_method 0 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:05:43.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:43.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:43.850+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:05:45.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): tick 2024-09-16T15:05:45.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:05:45.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:05:45.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1214008f30 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 msgr2=0x7f1214005d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1214003940 0x7f1214005d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 msgr2=0x7f1214008880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.162:3300/0 conn(0x7f1214006410 0x7f1214008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 1 -- 172.21.15.134:0/3037262696 reap_dead start 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient: _add_conns ranks=[9,3,5,8,7] 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.134:3300/0 conn(0x7f1214006870 0x7f1214003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f1214006870 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 0x7f121400b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-g con 0x7f12140042a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:45.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f12140046e0 0x7f1214004b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): picked mon.noname-h con 0x7f12140046e0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340a5740 con 0x7f1214006870 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bb30 con 0x7f12140042a0 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 1 -- 172.21.15.134:0/3037262696 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bcf0 con 0x7f12140046e0 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1231ffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> v2:172.21.15.134:3300/0 conn(0x7f1214006870 0x7f1214003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f122bfff640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f1214006870 auth_method 0 2024-09-16T15:05:45.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f122bfff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f12140046e0 0x7f1214004b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 10 monclient(hunting): get_auth_request con 0x7f12140046e0 auth_method 0 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f1233fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient(hunting): handle_auth_done global_id 4923 payload 274 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f12140046e0 msgr2=0x7f1214004b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:45.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f12140046e0 0x7f1214004b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 msgr2=0x7f121400b3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 0x7f121400b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient: _finish_hunting 0 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 monclient: found mon.noname-a 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:45.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.447+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1234137a80 con 0x7f1214006870 2024-09-16T15:05:45.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.450+0000 7f12337fe640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214006870 msgr2=0x7f1214003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.450+0000 7f12337fe640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214006870 0x7f1214003d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f122402c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93754b5d257ac2ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 1 -- 172.21.15.134:0/3037262696 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f122402ce30 con 0x7f1214006870 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: dump: 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.458 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 1 -- 172.21.15.134:0/3037262696 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224030070 con 0x7f1214006870 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1239be2640 5 monclient: authenticate success, global_id 4923 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1239be2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:45.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1239be2640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 1 -- 172.21.15.134:0/3037262696 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1224036ce0 con 0x7f1214006870 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1239be2640 4 set_mon_vals no callback set 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.451+0000 7f1232ffd640 10 monclient: dump: 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.460 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 10 monclient: shutdown 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 -- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214006870 msgr2=0x7f1214003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214006870 0x7f1214003d60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f122402c320 comp rx=0 tx=0).stop 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 -- 172.21.15.134:0/3037262696 shutdown_connections 2024-09-16T15:05:45.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f12140046e0 0x7f1214004b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140042a0 0x7f121400b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 --2- 172.21.15.134:0/3037262696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1214006870 0x7f1214003d60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 -- 172.21.15.134:0/3037262696 >> 172.21.15.134:0/3037262696 conn(0x7f12340a9ea0 msgr2=0x7f12340a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 -- 172.21.15.134:0/3037262696 shutdown_connections 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.452+0000 7f1239be2640 1 -- 172.21.15.134:0/3037262696 wait complete. 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.453+0000 7f1239be2640 10 monclient: build_initial_monmap 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.453+0000 7f1239be2640 10 monclient: monmap: 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.462 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.453+0000 7f1239be2640 1 Processor -- start 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.453+0000 7f1239be2640 1 -- start start 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.453+0000 7f1239be2640 10 monclient: init 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:45.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient: _add_conns ranks=[4,0,7,1,6,2,5,8,3] 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-c con 0x7f12140042a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 0x7f1234122830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-a con 0x7f12140046e0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 0x7f1234122d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): picked mon.noname-i con 0x7f1214006870 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234134e10 con 0x7f12140046e0 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1234125760 con 0x7f12140042a0 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1234124a20 con 0x7f1214006870 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:45.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1239be2640 10 monclient(hunting): authenticate will time out at 1082.043130s 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1233fff640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 msgr2=0x7f1234122d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 0x7f1234122830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:60264/0 (socket says 172.21.15.134:60264) 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f1233fff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 0x7f1234122d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 -- 172.21.15.134:0/2055555629 learned_addr learned my addr 172.21.15.134:0/2055555629 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): get_auth_request con 0x7f12140042a0 auth_method 0 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): get_auth_request con 0x7f12140046e0 auth_method 0 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient(hunting): handle_auth_done global_id 4846 payload 824 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 -- 172.21.15.134:0/2055555629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 msgr2=0x7f1234122d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 0x7f1234122d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 -- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 msgr2=0x7f1234122830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 0x7f1234122830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient: _finish_hunting 0 2024-09-16T15:05:45.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 monclient: found mon.noname-c 2024-09-16T15:05:45.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:45.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f12337fe640 1 -- 172.21.15.134:0/2055555629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122400e040 con 0x7f12140042a0 2024-09-16T15:05:45.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.454+0000 7f122bfff640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 0x7f1234122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.456+0000 7f12337fe640 1 -- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 msgr2=0x7f12341222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.456+0000 7f12337fe640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f122402c840 tx=0x7f12240046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f122402ce30 con 0x7f12140042a0 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: dump: 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: _finish_auth 0 2024-09-16T15:05:45.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1224036e80 con 0x7f12140042a0 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 5 monclient: authenticate success, global_id 4846 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 10 monclient: _renew_subs 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12340a91c0 con 0x7f12140042a0 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f122403fcb0 con 0x7f12140042a0 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1230ff9640 10 monclient: dump: 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.471 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f12327fc640 4 set_mon_vals no callback set 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 10 monclient: _renew_subs 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.457+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123406bed0 con 0x7f12140042a0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f122403f840 con 0x7f12140042a0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1239be2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1239be2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1230ff9640 1 --2- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 0x7f121403d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1239be2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12340a7e90 con 0x7f12140042a0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.458+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f122406c600 con 0x7f12140042a0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.459+0000 7f122bfff640 1 --2- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 0x7f121403d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.459+0000 7f122bfff640 10 monclient: get_auth_request con 0x7f121403ad00 auth_method 0 2024-09-16T15:05:45.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.461+0000 7f122bfff640 1 --2- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 0x7f121403d1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f121c000c00 tx=0x7f121c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.464+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1224046020 con 0x7f12140042a0 2024-09-16T15:05:45.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.464+0000 7f1230ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:45.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.464+0000 7f1230ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:45.483 INFO:tasks.rados.rados.0.smithi162.stdout:1104: done (3 left) 2024-09-16T15:05:45.483 INFO:tasks.rados.rados.0.smithi162.stdout:1105: done (2 left) 2024-09-16T15:05:45.483 INFO:tasks.rados.rados.0.smithi162.stdout:1106: done (1 left) 2024-09-16T15:05:45.483 INFO:tasks.rados.rados.0.smithi162.stdout:1107: done (0 left) 2024-09-16T15:05:45.484 INFO:tasks.rados.rados.0.smithi162.stdout:1108: write oid 26 current snap is 100 2024-09-16T15:05:45.484 INFO:tasks.rados.rados.0.smithi162.stdout:1108: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2024-09-16T15:05:45.491 INFO:tasks.rados.rados.0.smithi162.stdout:1108: writing smithi16234619-26 from 742975 to 1523215 tid 1 2024-09-16T15:05:45.497 INFO:tasks.rados.rados.0.smithi162.stdout:1108: writing smithi16234619-26 from 1953553 to 2380685 tid 2 2024-09-16T15:05:45.501 INFO:tasks.rados.rados.0.smithi162.stdout:1108: writing smithi16234619-26 from 2971713 to 3142975 tid 3 2024-09-16T15:05:45.501 INFO:tasks.rados.rados.0.smithi162.stdout:1109: rollback oid 13 current snap is 100 2024-09-16T15:05:45.502 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 96 2024-09-16T15:05:45.502 INFO:tasks.rados.rados.0.smithi162.stdout:1110: copy_from oid 28 from oid 24 current snap is 100 2024-09-16T15:05:45.502 INFO:tasks.rados.rados.0.smithi162.stdout:1111: snap_remove snap 99 2024-09-16T15:05:45.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1239be2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:05:45.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1239be2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:45.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1239be2640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1234125960 con 0x7f12140042a0 2024-09-16T15:05:45.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1230ff9640 1 -- 172.21.15.134:0/2055555629 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f1224058c80 con 0x7f12140042a0 2024-09-16T15:05:45.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1230ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:05:45.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.712+0000 7f1230ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:45.714 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":46,"quorum":[0,1,2,4,5,6,8],"quorum_names":["a","f","b","c","h","d","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:05:45.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.714+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 msgr2=0x7f121403d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 0x7f121403d1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f121c000c00 tx=0x7f121c009040 comp rx=0 tx=0).stop 2024-09-16T15:05:45.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 10 monclient: shutdown 2024-09-16T15:05:45.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:45.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 msgr2=0x7f12341222f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f122402c840 tx=0x7f12240046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 shutdown_connections 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> v2:172.21.15.162:6800/980547263 conn(0x7f121403ad00 0x7f121403d1c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1214006870 0x7f1234122d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12140046e0 0x7f1234122830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 --2- 172.21.15.134:0/2055555629 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f12140042a0 0x7f12341222f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 >> 172.21.15.134:0/2055555629 conn(0x7f12340a9ea0 msgr2=0x7f12340a2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 shutdown_connections 2024-09-16T15:05:45.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.715+0000 7f1239be2640 1 -- 172.21.15.134:0/2055555629 wait complete. 2024-09-16T15:05:45.727 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-16T15:05:45.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-16T15:05:45.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 10 monclient: get_monmap_and_config 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 10 monclient: build_initial_monmap 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 10 monclient: monmap: 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:45.824077+0000 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:45.824077+0000 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:45.825 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 1 Processor -- start 2024-09-16T15:05:45.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 1 -- start start 2024-09-16T15:05:45.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.823+0000 7f61c8a19640 10 monclient: init 2024-09-16T15:05:45.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:45.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient: _add_conns ranks=[1,5,10,6,7,4,9,8,0,2,3] 2024-09-16T15:05:45.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f61c4136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-h con 0x7f61c4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 0x7f61c413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-f con 0x7f61c413c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c4073250 con 0x7f61c4136e40 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40a56e0 con 0x7f61c413c370 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61c409abc0 con 0x7f61c4137220 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c8a19640 10 monclient(hunting): authenticate will time out at 1082.413476s 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 0x7f61c413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:55954/0 (socket says 172.21.15.134:55954) 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 1 -- 172.21.15.134:0/1914682550 learned_addr learned my addr 172.21.15.134:0/1914682550 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request con 0x7f61c4137220 auth_method 0 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): get_auth_request con 0x7f61c413c370 auth_method 0 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.824+0000 7f61c2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_done global_id 44249 payload 274 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 1 -- 172.21.15.134:0/1914682550 >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 msgr2=0x7f61c413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 1 --2- 172.21.15.134:0/1914682550 >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 0x7f61c413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 1 -- 172.21.15.134:0/1914682550 >> v1:172.21.15.162:6789/0 conn(0x7f61c4136e40 legacy=0x7f61c409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 1 monclient: found mon.noname-h 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c1d9b640 1 -- 172.21.15.134:0/1914682550 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c409ba50 con 0x7f61c4137220 2024-09-16T15:05:45.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.825+0000 7f61c2d9d640 1 --2- 172.21.15.134:0/1914682550 >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 0x7f61c413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c1d9b640 1 -- 172.21.15.134:0/1914682550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 msgr2=0x7f61c4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c1d9b640 1 --2- 172.21.15.134:0/1914682550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009870 tx=0x7f61b802a100 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e5772a38cf10db57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 1 -- 172.21.15.134:0/1914682550 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b802ae20 con 0x7f61c4137220 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: dump: 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: _finish_auth 0 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: _check_auth_tickets 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 1 -- 172.21.15.134:0/1914682550 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80307b0 con 0x7f61c4137220 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c8a19640 5 monclient: authenticate success, global_id 44249 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c8a19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c8a19640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 1 -- 172.21.15.134:0/1914682550 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b8030d90 con 0x7f61c4137220 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c8a19640 4 set_mon_vals no callback set 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.828+0000 7f61c159a640 10 monclient: dump: 2024-09-16T15:05:45.839 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.840 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 10 monclient: shutdown 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 -- 172.21.15.134:0/1914682550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 msgr2=0x7f61c4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 --2- 172.21.15.134:0/1914682550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009870 tx=0x7f61b802a100 comp rx=0 tx=0).stop 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 -- 172.21.15.134:0/1914682550 shutdown_connections 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 --2- 172.21.15.134:0/1914682550 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f61c4137220 0x7f61c4137600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 --2- 172.21.15.134:0/1914682550 >> v2:172.21.15.162:3300/0 conn(0x7f61c413c370 0x7f61c413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 -- 172.21.15.134:0/1914682550 >> 172.21.15.134:0/1914682550 conn(0x7f61c40a6110 msgr2=0x7f61c40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 -- 172.21.15.134:0/1914682550 shutdown_connections 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 1 -- 172.21.15.134:0/1914682550 wait complete. 2024-09-16T15:05:45.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 10 monclient: build_initial_monmap 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.829+0000 7f61c8a19640 10 monclient: monmap: 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:45.842 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 Processor -- start 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 -- start start 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient: init 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient: _add_conns ranks=[1,6,7,4,0,3,5,2,8] 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4136e40 0x7f61c411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-f con 0x7f61c4136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-d con 0x7f61c4137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:45.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61c413c370 0x7f61c41206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): picked mon.noname-i con 0x7f61c413c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4121ec0 con 0x7f61c4136e40 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4120d90 con 0x7f61c4137220 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40a7510 con 0x7f61c413c370 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c8a19640 10 monclient(hunting): authenticate will time out at 1082.419719s 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c2d9d640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61c413c370 msgr2=0x7f61c41206e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:45.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:59658/0 (socket says 172.21.15.134:59658) 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c2d9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61c413c370 0x7f61c41206e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 1 -- 172.21.15.134:0/2051742188 learned_addr learned my addr 172.21.15.134:0/2051742188 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request con 0x7f61c4137220 auth_method 0 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4136e40 0x7f61c411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c259c640 10 monclient(hunting): get_auth_request con 0x7f61c4136e40 auth_method 0 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.830+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 10 monclient(hunting): handle_auth_done global_id 34476 payload 824 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61c413c370 msgr2=0x7f61c41206e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f61c413c370 0x7f61c41206e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4136e40 msgr2=0x7f61c411fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4136e40 0x7f61c411fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 monclient: found mon.noname-d 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c1d9b640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc009a70 con 0x7f61c4137220 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.831+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4136e40 0x7f61c411fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:05:45.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c1d9b640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 msgr2=0x7f61c41201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c1d9b640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61b80090b0 tx=0x7f61b80046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b8030d40 con 0x7f61c4137220 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: dump: 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.847 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 5 monclient: authenticate success, global_id 34476 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b8003a80 con 0x7f61c4137220 2024-09-16T15:05:45.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b8003bf0 con 0x7f61c4137220 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 10 monclient: _renew_subs 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b80094d0 con 0x7f61c4137220 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61b2ffd640 10 monclient: dump: 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.849 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c0d99640 4 set_mon_vals no callback set 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 10 monclient: _renew_subs 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.833+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c4122760 con 0x7f61c4137220 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c8a19640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:45.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c8a19640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 0x7f61c4135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4071b20 con 0x7f61c41357d0 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f61b8039070 con 0x7f61c4137220 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 0x7f61c4135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient: get_auth_request con 0x7f61c41357d0 auth_method 0 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61b2ffd640 1 --2- 172.21.15.134:0/2051742188 >> v2:172.21.15.162:6800/980547263 conn(0x7f61a4038880 0x7f61a403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f61b806ab70 con 0x7f61c4137220 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 10 monclient(hunting): handle_auth_done global_id 4941 payload 824 2024-09-16T15:05:45.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c259c640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61c4061ae0 con 0x7f61c41357d0 2024-09-16T15:05:45.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c2d9d640 1 --2- 172.21.15.134:0/2051742188 >> v2:172.21.15.162:6800/980547263 conn(0x7f61a4038880 0x7f61a403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.834+0000 7f61c2d9d640 10 monclient: get_auth_request con 0x7f61a4038880 auth_method 0 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.836+0000 7f61c259c640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 msgr2=0x7f61c4135bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.836+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 0x7f61c4135bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e500 tx=0x7f61b400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b4013c70 con 0x7f61c41357d0 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 10 monclient: dump: 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.854 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.855 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61c2d9d640 1 --2- 172.21.15.134:0/2051742188 >> v2:172.21.15.162:6800/980547263 conn(0x7f61a4038880 0x7f61a403ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61c413a7f0 tx=0x7f61bc00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f61b4013dd0 con 0x7f61c41357d0 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 msgr2=0x7f61c4135bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 0x7f61c4135bb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e500 tx=0x7f61b400e9c0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 msgr2=0x7f61c4135bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.837+0000 7f61b2ffd640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41357d0 0x7f61c4135bb0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e500 tx=0x7f61b400e9c0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c8a19640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c8a19640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 0x7f61c409d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c8a19640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c407a680 con 0x7f61c41360f0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 0x7f61c409d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient: get_auth_request con 0x7f61c41360f0 auth_method 0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.843+0000 7f61c259c640 10 monclient(hunting): handle_auth_done global_id 4950 payload 824 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.844+0000 7f61c259c640 1 -- 172.21.15.134:0/2051742188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f61c40a53c0 con 0x7f61c41360f0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c259c640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 msgr2=0x7f61c409d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c259c640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 0x7f61c409d4f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e500 tx=0x7f61b4003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61b4006bf0 con 0x7f61c41360f0 2024-09-16T15:05:45.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 10 monclient: dump: 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.858 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9341 (secure 0 0 0) 0x7f61b4006d50 con 0x7f61c41360f0 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 msgr2=0x7f61c409d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 0x7f61c409d4f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e500 tx=0x7f61b4003760 comp rx=0 tx=0).stop 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 msgr2=0x7f61c409d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61b2ffd640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61c41360f0 0x7f61c409d4f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.859 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:45.860 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:45.861 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:45.862 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:45.863 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:45.864 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.866 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.868 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.869 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.870 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.871 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:45.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.876 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:45.877 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.880 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.881 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:45.882 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:45.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 >> v2:172.21.15.162:6800/980547263 conn(0x7f61a4038880 msgr2=0x7f61a403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 --2- 172.21.15.134:0/2051742188 >> v2:172.21.15.162:6800/980547263 conn(0x7f61a4038880 0x7f61a403ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61c413a7f0 tx=0x7f61bc00b040 comp rx=0 tx=0).stop 2024-09-16T15:05:45.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 10 monclient: shutdown 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 msgr2=0x7f61c41201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61b80090b0 tx=0x7f61b80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c2d9d640 1 -- 172.21.15.134:0/2051742188 reap_dead start 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 shutdown_connections 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.846+0000 7f61c8a19640 1 --2- 172.21.15.134:0/2051742188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f61c4137220 0x7f61c41201a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.847+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 >> 172.21.15.134:0/2051742188 conn(0x7f61c40a6110 msgr2=0x7f61c413b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.847+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 shutdown_connections 2024-09-16T15:05:45.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.847+0000 7f61c8a19640 1 -- 172.21.15.134:0/2051742188 wait complete. 2024-09-16T15:05:45.884 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:45.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-16T15:05:45.885 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.980+0000 7f9857ed7640 10 monclient: get_monmap_and_config 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.980+0000 7f9857ed7640 10 monclient: build_initial_monmap 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.980+0000 7f9857ed7640 10 monclient: monmap: 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:45.981795+0000 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:45.981795+0000 2024-09-16T15:05:45.982 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:45.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.981+0000 7f9857ed7640 1 Processor -- start 2024-09-16T15:05:45.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.981+0000 7f9857ed7640 1 -- start start 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.981+0000 7f9857ed7640 10 monclient: init 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient: _add_conns ranks=[1,4,0,9,3,8,2,7,10,5,6] 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9850136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 0x7f9850137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-c con 0x7f9850137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f985013c370 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9850073250 con 0x7f985013c370 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985009ba50 con 0x7f9850136e40 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98500a56e0 con 0x7f9850137220 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:45.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9857ed7640 10 monclient(hunting): authenticate will time out at 1082.571219s 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 0x7f9850137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9856474640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f985013c370 0x7f985013e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41274/0 (socket says 172.21.15.134:41274) 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9856474640 1 -- 172.21.15.134:0/2408818132 learned_addr learned my addr 172.21.15.134:0/2408818132 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): get_auth_request con 0x7f9850137220 auth_method 0 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient(hunting): handle_auth_done global_id 4855 payload 274 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 1 -- 172.21.15.134:0/2408818132 >> v1:172.21.15.162:6789/0 conn(0x7f9850136e40 legacy=0x7f985009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:45.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 1 -- 172.21.15.134:0/2408818132 >> v1:172.21.15.134:6789/0 conn(0x7f985013c370 legacy=0x7f985013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:45.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient: _finish_hunting 0 2024-09-16T15:05:45.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 1 monclient: found mon.noname-c 2024-09-16T15:05:45.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:45.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:45.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.982+0000 7f9855472640 1 -- 172.21.15.134:0/2408818132 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500a53c0 con 0x7f9850137220 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9855472640 1 -- 172.21.15.134:0/2408818132 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 msgr2=0x7f9850137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9855472640 1 --2- 172.21.15.134:0/2408818132 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 0x7f9850137600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f98400097d0 tx=0x7f984002a040 comp rx=0 tx=0).ready entity=mon.4 client_cookie=39c02d5d88c4b410 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 1 -- 172.21.15.134:0/2408818132 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f984002ae40 con 0x7f9850137220 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: dump: 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.994 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: _finish_auth 0 2024-09-16T15:05:45.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: _check_auth_tickets 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 1 -- 172.21.15.134:0/2408818132 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98400307b0 con 0x7f9850137220 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9857ed7640 5 monclient: authenticate success, global_id 4855 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9857ed7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9857ed7640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 1 -- 172.21.15.134:0/2408818132 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9840030d90 con 0x7f9850137220 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9857ed7640 4 set_mon_vals no callback set 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.985+0000 7f9854c71640 10 monclient: dump: 2024-09-16T15:05:45.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:45.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 10 monclient: shutdown 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 -- 172.21.15.134:0/2408818132 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 msgr2=0x7f9850137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 --2- 172.21.15.134:0/2408818132 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 0x7f9850137600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f98400097d0 tx=0x7f984002a040 comp rx=0 tx=0).stop 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 -- 172.21.15.134:0/2408818132 shutdown_connections 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 --2- 172.21.15.134:0/2408818132 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9850137220 0x7f9850137600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 -- 172.21.15.134:0/2408818132 >> 172.21.15.134:0/2408818132 conn(0x7f98500a6110 msgr2=0x7f98500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 -- 172.21.15.134:0/2408818132 shutdown_connections 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 -- 172.21.15.134:0/2408818132 wait complete. 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 10 monclient: build_initial_monmap 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 10 monclient: monmap: 2024-09-16T15:05:45.998 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:45.999 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.986+0000 7f9857ed7640 1 Processor -- start 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 -- start start 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient: init 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient: _add_conns ranks=[0,2,4,7,3,5,6,8,1] 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-a con 0x7f9850136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9850137220 0x7f9850135d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-b con 0x7f9850137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f985013c370 0x7f9850136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): picked mon.noname-c con 0x7f985013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98500a33b0 con 0x7f9850136e40 2024-09-16T15:05:46.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985013b5a0 con 0x7f9850137220 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9850120ef0 con 0x7f985013c370 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9857ed7640 10 monclient(hunting): authenticate will time out at 1082.576480s 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f985013c370 0x7f9850136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35844/0 (socket says 172.21.15.134:35844) 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 -- 172.21.15.134:0/966068774 learned_addr learned my addr 172.21.15.134:0/966068774 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): get_auth_request con 0x7f9850136e40 auth_method 0 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9850137220 0x7f9850135d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): get_auth_request con 0x7f985013c370 auth_method 0 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): get_auth_request con 0x7f9850137220 auth_method 0 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient(hunting): handle_auth_done global_id 4959 payload 824 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f985013c370 msgr2=0x7f9850136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f985013c370 0x7f9850136250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9850137220 msgr2=0x7f9850135d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9850137220 0x7f9850135d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 monclient: found mon.noname-a 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855c73640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840009400 con 0x7f9850136e40 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9850137220 0x7f9850135d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.987+0000 7f9856474640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f985013c370 0x7f9850136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9855c73640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 msgr2=0x7f98501357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9855c73640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c430 tx=0x7f984c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f984c003860 con 0x7f9850136e40 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: dump: 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.004 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c011c10 con 0x7f9850136e40 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 5 monclient: authenticate success, global_id 4959 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 10 monclient: _renew_subs 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98501226f0 con 0x7f9850136e40 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f984c011dc0 con 0x7f9850136e40 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f98467fc640 10 monclient: dump: 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.006 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.007 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9847fff640 4 set_mon_vals no callback set 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 10 monclient: _renew_subs 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.990+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850122410 con 0x7f9850136e40 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9857ed7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9857ed7640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 0x7f9850061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98500736c0 con 0x7f98500610b0 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f984c0117a0 con 0x7f9850136e40 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 0x7f9850061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient: get_auth_request con 0x7f98500610b0 auth_method 0 2024-09-16T15:05:46.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f98467fc640 1 --2- 172.21.15.134:0/966068774 >> v2:172.21.15.162:6800/980547263 conn(0x7f98340388b0 0x7f983403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 10 monclient(hunting): handle_auth_done global_id 44255 payload 824 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f984c049f00 con 0x7f9850136e40 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9855472640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98500a7710 con 0x7f98500610b0 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9856474640 1 --2- 172.21.15.134:0/966068774 >> v2:172.21.15.162:6800/980547263 conn(0x7f98340388b0 0x7f983403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.991+0000 7f9856474640 10 monclient: get_auth_request con 0x7f98340388b0 auth_method 0 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f9855472640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 msgr2=0x7f9850061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 0x7f9850061490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9840034000 tx=0x7f98400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9840003ea0 con 0x7f98500610b0 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f98467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f98467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.993+0000 7f98467fc640 10 monclient: dump: 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.012 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f98400044e0 con 0x7f98500610b0 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 msgr2=0x7f9850061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 0x7f9850061490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9840034000 tx=0x7f98400046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 msgr2=0x7f9850061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f98467fc640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500610b0 0x7f9850061490 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9840034000 tx=0x7f98400046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:45.994+0000 7f9856474640 1 --2- 172.21.15.134:0/966068774 >> v2:172.21.15.162:6800/980547263 conn(0x7f98340388b0 0x7f983403ad70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9848009400 tx=0x7f9848009350 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9857ed7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9857ed7640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 0x7f985002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9857ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9850067a80 con 0x7f985002b260 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 0x7f985002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient: get_auth_request con 0x7f985002b260 auth_method 0 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 10 monclient(hunting): handle_auth_done global_id 44264 payload 824 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.000+0000 7f9855472640 1 -- 172.21.15.134:0/966068774 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9850061ae0 con 0x7f985002b260 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f9855472640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 msgr2=0x7f985002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f9855472640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 0x7f985002b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f984002a080 tx=0x7f9840009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f984004e070 con 0x7f985002b260 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f98467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f98467fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.002+0000 7f98467fc640 10 monclient: dump: 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.015 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.016 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f9840048660 con 0x7f985002b260 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 msgr2=0x7f985002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 0x7f985002b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f984002a080 tx=0x7f9840009190 comp rx=0 tx=0).stop 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 msgr2=0x7f985002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f98467fc640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985002b260 0x7f985002b670 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f984002a080 tx=0x7f9840009190 comp rx=0 tx=0).stop 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.017 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.019 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.020 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.021 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.022 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.025 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.027 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.033 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.035 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.038 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.039 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.040 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 >> v2:172.21.15.162:6800/980547263 conn(0x7f98340388b0 msgr2=0x7f983403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 --2- 172.21.15.134:0/966068774 >> v2:172.21.15.162:6800/980547263 conn(0x7f98340388b0 0x7f983403ad70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9848009400 tx=0x7f9848009350 comp rx=0 tx=0).stop 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 10 monclient: shutdown 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 msgr2=0x7f98501357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c430 tx=0x7f984c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9856474640 1 -- 172.21.15.134:0/966068774 reap_dead start 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 shutdown_connections 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 --2- 172.21.15.134:0/966068774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850136e40 0x7f98501357d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 >> 172.21.15.134:0/966068774 conn(0x7f98500a6110 msgr2=0x7f985013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 shutdown_connections 2024-09-16T15:05:46.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.003+0000 7f9857ed7640 1 -- 172.21.15.134:0/966068774 wait complete. 2024-09-16T15:05:46.043 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.043 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-16T15:05:46.043 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.140+0000 7f7ff911f640 10 monclient: get_monmap_and_config 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.140+0000 7f7ff911f640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.140+0000 7f7ff911f640 10 monclient: monmap: 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.142 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:46.141522+0000 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:46.141522+0000 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:46.143 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 Processor -- start 2024-09-16T15:05:46.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 -- start start 2024-09-16T15:05:46.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient: init 2024-09-16T15:05:46.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient: _add_conns ranks=[6,2,4,8,9,0,5,1,7,3,10] 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 0x7f7ff409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-d con 0x7f7ff4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-b con 0x7f7ff4137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 0x7f7ff4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-c con 0x7f7ff4135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff409abc0 con 0x7f7ff4137220 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff409ba50 con 0x7f7ff4135dc0 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40a53c0 con 0x7f7ff4136e40 2024-09-16T15:05:46.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff911f640 10 monclient(hunting): authenticate will time out at 1082.730711s 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54670/0 (socket says 172.21.15.134:54670) 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 1 -- 172.21.15.134:0/1597233023 learned_addr learned my addr 172.21.15.134:0/1597233023 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): get_auth_request con 0x7f7ff4137220 auth_method 0 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 0x7f7ff4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff2d9d640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 0x7f7ff409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): get_auth_request con 0x7f7ff4135dc0 auth_method 0 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff2d9d640 10 monclient(hunting): get_auth_request con 0x7f7ff4136e40 auth_method 0 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.141+0000 7f7ff359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 10 monclient(hunting): handle_auth_done global_id 44273 payload 274 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 -- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 msgr2=0x7f7ff409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 0x7f7ff409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 -- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 msgr2=0x7f7ff4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 0x7f7ff4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 monclient: found mon.noname-b 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff259c640 1 -- 172.21.15.134:0/1597233023 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4073250 con 0x7f7ff4137220 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff2d9d640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 0x7f7ff409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.142+0000 7f7ff359e640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 0x7f7ff4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.144+0000 7f7ff259c640 1 -- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 msgr2=0x7f7ff4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.144+0000 7f7ff259c640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ed40575684237e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 1 -- 172.21.15.134:0/1597233023 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe802ade0 con 0x7f7ff4137220 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: dump: 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.154 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 1 -- 172.21.15.134:0/1597233023 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80307b0 con 0x7f7ff4137220 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 5 monclient: authenticate success, global_id 44273 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 1 -- 172.21.15.134:0/1597233023 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe8030d90 con 0x7f7ff4137220 2024-09-16T15:05:46.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 4 set_mon_vals no callback set 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff1d9b640 10 monclient: dump: 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.156 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 10 monclient: shutdown 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 -- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 msgr2=0x7f7ff4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe802a080 comp rx=0 tx=0).stop 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 -- 172.21.15.134:0/1597233023 shutdown_connections 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4135dc0 0x7f7ff4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4137220 0x7f7ff4137600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 --2- 172.21.15.134:0/1597233023 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7ff4136e40 0x7f7ff409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 -- 172.21.15.134:0/1597233023 >> 172.21.15.134:0/1597233023 conn(0x7f7ff40a6110 msgr2=0x7f7ff40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 -- 172.21.15.134:0/1597233023 shutdown_connections 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.145+0000 7f7ff911f640 1 -- 172.21.15.134:0/1597233023 wait complete. 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.146+0000 7f7ff911f640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.146+0000 7f7ff911f640 10 monclient: monmap: 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.158 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.146+0000 7f7ff911f640 1 Processor -- start 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.146+0000 7f7ff911f640 1 -- start start 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.146+0000 7f7ff911f640 10 monclient: init 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient: _add_conns ranks=[2,1,5,3,8,0,4,7,6] 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 0x7f7ff41cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-b con 0x7f7ff4135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ff4136e40 0x7f7ff4135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-f con 0x7f7ff4136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7ff4137220 0x7f7ff4135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): picked mon.noname-h con 0x7f7ff4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff41cb860 con 0x7f7ff4136e40 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4136390 con 0x7f7ff4135dc0 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4123bc0 con 0x7f7ff4137220 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff911f640 10 monclient(hunting): authenticate will time out at 1082.736369s 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 0x7f7ff41cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 0x7f7ff41cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54686/0 (socket says 172.21.15.134:54686) 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 -- 172.21.15.134:0/3174086617 learned_addr learned my addr 172.21.15.134:0/3174086617 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): get_auth_request con 0x7f7ff4135dc0 auth_method 0 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient(hunting): handle_auth_done global_id 44282 payload 824 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7ff4137220 msgr2=0x7f7ff4135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff359e640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7ff4137220 0x7f7ff4135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7ff4137220 0x7f7ff4135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ff4136e40 msgr2=0x7f7ff4135020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff259c640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ff4136e40 0x7f7ff4135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ff4136e40 0x7f7ff4135020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 monclient: found mon.noname-b 2024-09-16T15:05:46.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.147+0000 7f7ff2d9d640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec009a70 con 0x7f7ff4135dc0 2024-09-16T15:05:46.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.149+0000 7f7ff2d9d640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 msgr2=0x7f7ff41cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.149+0000 7f7ff2d9d640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 0x7f7ff41cb320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400ebe0 tx=0x7f7fe400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe400eee0 con 0x7f7ff4135dc0 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: dump: 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.167 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400ce40 con 0x7f7ff4135dc0 2024-09-16T15:05:46.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 5 monclient: authenticate success, global_id 44282 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 10 monclient: _renew_subs 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe4009400 con 0x7f7ff4135dc0 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe4013d50 con 0x7f7ff4135dc0 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7fdb7fe640 10 monclient: dump: 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.169 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 10 monclient: _renew_subs 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe80094d0 con 0x7f7ff4135dc0 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.150+0000 7f7ff159a640 4 set_mon_vals no callback set 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff911f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff911f640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 0x7f7ff4121a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4071b20 con 0x7f7ff4125690 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7fe4013470 con 0x7f7ff4135dc0 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 0x7f7ff4121a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient: get_auth_request con 0x7f7ff4125690 auth_method 0 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7fdb7fe640 1 --2- 172.21.15.134:0/3174086617 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fe0038880 0x7f7fe003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f7fe404a620 con 0x7f7ff4135dc0 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 10 monclient(hunting): handle_auth_done global_id 4882 payload 824 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff259c640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ff4061ae0 con 0x7f7ff4125690 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff359e640 1 --2- 172.21.15.134:0/3174086617 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fe0038880 0x7f7fe003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.151+0000 7f7ff359e640 10 monclient: get_auth_request con 0x7f7fe0038880 auth_method 0 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7ff259c640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 msgr2=0x7f7ff4121a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7ff259c640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 0x7f7ff4121a30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8034000 tx=0x7f7fe80046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe8003610 con 0x7f7ff4125690 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7fdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7fdb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.153+0000 7f7fdb7fe640 10 monclient: dump: 2024-09-16T15:05:46.172 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.173 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7fe8030b30 con 0x7f7ff4125690 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 msgr2=0x7f7ff4121a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 0x7f7ff4121a30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8034000 tx=0x7f7fe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 msgr2=0x7f7ff4121a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7fdb7fe640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4125690 0x7f7ff4121a30 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8034000 tx=0x7f7fe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.154+0000 7f7ff359e640 1 --2- 172.21.15.134:0/3174086617 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fe0038880 0x7f7fe003ad40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406b550 tx=0x7f7fec00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff911f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff911f640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 0x7f7ff4122380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff911f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff407a680 con 0x7f7ff4121f70 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 0x7f7ff4122380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient: get_auth_request con 0x7f7ff4121f70 auth_method 0 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 10 monclient(hunting): handle_auth_done global_id 4891 payload 824 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.160+0000 7f7ff259c640 1 -- 172.21.15.134:0/3174086617 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ff409b490 con 0x7f7ff4121f70 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.162+0000 7f7ff259c640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 msgr2=0x7f7ff4122380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.162+0000 7f7ff259c640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 0x7f7ff4122380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009240 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fe8003be0 con 0x7f7ff4121f70 2024-09-16T15:05:46.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 10 monclient: dump: 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.176 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9505 (secure 0 0 0) 0x7f7fe8003d40 con 0x7f7ff4121f70 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 msgr2=0x7f7ff4122380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 0x7f7ff4122380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009240 comp rx=0 tx=0).stop 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 msgr2=0x7f7ff4122380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7fdb7fe640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7ff4121f70 0x7f7ff4122380 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009240 comp rx=0 tx=0).stop 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.178 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.179 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.181 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.182 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.184 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.189 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.192 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.193 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.198 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.199 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.200 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fe0038880 msgr2=0x7f7fe003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 --2- 172.21.15.134:0/3174086617 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fe0038880 0x7f7fe003ad40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406b550 tx=0x7f7fec00b040 comp rx=0 tx=0).stop 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 10 monclient: shutdown 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 msgr2=0x7f7ff41cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 --2- 172.21.15.134:0/3174086617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7ff4135dc0 0x7f7ff41cb320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400ebe0 tx=0x7f7fe400c520 comp rx=0 tx=0).stop 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff359e640 1 -- 172.21.15.134:0/3174086617 reap_dead start 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff359e640 1 -- 172.21.15.134:0/3174086617 reap_dead start 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 shutdown_connections 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 >> 172.21.15.134:0/3174086617 conn(0x7f7ff40a6110 msgr2=0x7f7ff406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 shutdown_connections 2024-09-16T15:05:46.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.163+0000 7f7ff911f640 1 -- 172.21.15.134:0/3174086617 wait complete. 2024-09-16T15:05:46.203 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.203 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-16T15:05:46.203 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:05:46.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.300+0000 7fb31bc09640 10 monclient: get_monmap_and_config 2024-09-16T15:05:46.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.300+0000 7fb31bc09640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.300+0000 7fb31bc09640 10 monclient: monmap: 2024-09-16T15:05:46.301 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:46.301229+0000 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:46.301229+0000 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.302 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.303 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:46.303 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:46.303 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.300+0000 7fb31bc09640 1 Processor -- start 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 -- start start 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient: init 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient: _add_conns ranks=[0,2,5,8,1,4,9,6,7,3,10] 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb314136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 0x7fb314137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-b con 0x7fb314137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb31413c370 0x7fb31413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-h con 0x7fb31413c370 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb314073250 con 0x7fb314136e40 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140a56e0 con 0x7fb314137220 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb31409abc0 con 0x7fb31413c370 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb31bc09640 10 monclient(hunting): authenticate will time out at 1082.890601s 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 0x7fb314137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3199a5640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb314136e40 0x7fb31409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:41716/0 (socket says 172.21.15.134:41716) 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3199a5640 1 -- 172.21.15.134:0/99612885 learned_addr learned my addr 172.21.15.134:0/99612885 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): get_auth_request con 0x7fb314137220 auth_method 0 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient(hunting): handle_auth_done global_id 44291 payload 274 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 -- 172.21.15.134:0/99612885 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb31413c370 msgr2=0x7fb31413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 --2- 172.21.15.134:0/99612885 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb31413c370 0x7fb31413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 -- 172.21.15.134:0/99612885 >> v1:172.21.15.134:6789/0 conn(0x7fb314136e40 legacy=0x7fb31409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 monclient: found mon.noname-b 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.301+0000 7fb3191a4640 1 -- 172.21.15.134:0/99612885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31409ba50 con 0x7fb314137220 2024-09-16T15:05:46.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.304+0000 7fb3191a4640 1 -- 172.21.15.134:0/99612885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 msgr2=0x7fb314137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.304+0000 7fb3191a4640 1 --2- 172.21.15.134:0/99612885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 0x7fb314137600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3100097d0 tx=0x7fb31002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ac1a71df4ac385 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 1 -- 172.21.15.134:0/99612885 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb31002ae00 con 0x7fb314137220 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: dump: 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.312 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 5 monclient: authenticate success, global_id 44291 2024-09-16T15:05:46.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 1 -- 172.21.15.134:0/99612885 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3100307b0 con 0x7fb314137220 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 4 set_mon_vals no callback set 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 1 -- 172.21.15.134:0/99612885 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb310030d90 con 0x7fb314137220 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb3189a3640 10 monclient: dump: 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.314 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 10 monclient: shutdown 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 -- 172.21.15.134:0/99612885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 msgr2=0x7fb314137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 --2- 172.21.15.134:0/99612885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 0x7fb314137600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3100097d0 tx=0x7fb31002a000 comp rx=0 tx=0).stop 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 -- 172.21.15.134:0/99612885 shutdown_connections 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 --2- 172.21.15.134:0/99612885 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb31413c370 0x7fb31413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 --2- 172.21.15.134:0/99612885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb314137220 0x7fb314137600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 -- 172.21.15.134:0/99612885 >> 172.21.15.134:0/99612885 conn(0x7fb3140a6110 msgr2=0x7fb3140a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 -- 172.21.15.134:0/99612885 shutdown_connections 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.305+0000 7fb31bc09640 1 -- 172.21.15.134:0/99612885 wait complete. 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient: monmap: 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:46.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 1 Processor -- start 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 1 -- start start 2024-09-16T15:05:46.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient: init 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient: _add_conns ranks=[1,5,3,8,7,6,0,4,2] 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 0x7fb3140a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-f con 0x7fb314136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 0x7fb314135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.306+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-h con 0x7fb314137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb31413c370 0x7fb3141363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): picked mon.noname-g con 0x7fb31413c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:46.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140a75c0 con 0x7fb314136e40 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31413b530 con 0x7fb31413c370 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140a7b10 con 0x7fb314137220 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31bc09640 10 monclient(hunting): authenticate will time out at 1082.896035s 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31a1a6640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb31413c370 msgr2=0x7fb3141363d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb31a1a6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb31413c370 0x7fb3141363d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 0x7fb314135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 0x7fb3140a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 0x7fb314135e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:42744/0 (socket says 172.21.15.134:42744) 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 1 -- 172.21.15.134:0/2667888965 learned_addr learned my addr 172.21.15.134:0/2667888965 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): get_auth_request con 0x7fb314137220 auth_method 0 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): get_auth_request con 0x7fb314136e40 auth_method 0 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient(hunting): handle_auth_done global_id 44254 payload 824 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 msgr2=0x7fb314135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 0x7fb314135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb31413c370 msgr2=0x7fb3141363d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb31413c370 0x7fb3141363d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 monclient: found mon.noname-f 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3199a5640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310009400 con 0x7fb314136e40 2024-09-16T15:05:46.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.307+0000 7fb3191a4640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb314137220 0x7fb314135e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.309+0000 7fb3199a5640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 msgr2=0x7fb3140a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.309+0000 7fb3199a5640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 0x7fb3140a2950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c3e0 tx=0x7fb30800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.309+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:05:46.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb308004a90 con 0x7fb314136e40 2024-09-16T15:05:46.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:05:46.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: dump: 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.327 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308005cf0 con 0x7fb314136e40 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 5 monclient: authenticate success, global_id 44254 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 10 monclient: _renew_subs 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb314122560 con 0x7fb314136e40 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb308005ea0 con 0x7fb314136e40 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:46.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb3067fc640 10 monclient: dump: 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.329 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb307fff640 4 set_mon_vals no callback set 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 10 monclient: _renew_subs 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.312+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb314120dc0 con 0x7fb314136e40 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb31bc09640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb31bc09640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 0x7fb314120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb314071b20 con 0x7fb31411fc60 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 0x7fb314120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient: get_auth_request con 0x7fb31411fc60 auth_method 0 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb308007070 con 0x7fb314136e40 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 10 monclient(hunting): handle_auth_done global_id 34494 payload 824 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3191a4640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb314121fa0 con 0x7fb31411fc60 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3067fc640 1 --2- 172.21.15.134:0/2667888965 >> v2:172.21.15.162:6800/980547263 conn(0x7fb2f8038880 0x7fb2f803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.313+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fb3080492d0 con 0x7fb314136e40 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.314+0000 7fb31a1a6640 1 --2- 172.21.15.134:0/2667888965 >> v2:172.21.15.162:6800/980547263 conn(0x7fb2f8038880 0x7fb2f803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.314+0000 7fb31a1a6640 10 monclient: get_auth_request con 0x7fb2f8038880 auth_method 0 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3191a4640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 msgr2=0x7fb314120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3191a4640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 0x7fb314120040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb310035040 tx=0x7fb3100046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb310003ea0 con 0x7fb31411fc60 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3067fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.315+0000 7fb3067fc640 10 monclient: dump: 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.332 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb310030e10 con 0x7fb31411fc60 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 msgr2=0x7fb314120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 0x7fb314120040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb310035040 tx=0x7fb3100046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 msgr2=0x7fb314120040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb3067fc640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb31411fc60 0x7fb314120040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.316+0000 7fb31a1a6640 1 --2- 172.21.15.134:0/2667888965 >> v2:172.21.15.162:6800/980547263 conn(0x7fb2f8038880 0x7fb2f803ad40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb31409cec0 tx=0x7fb30c007db0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb31bc09640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb31bc09640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 0x7fb31409d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb31bc09640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb314061ae0 con 0x7fb314120580 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 0x7fb31409d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient: get_auth_request con 0x7fb314120580 auth_method 0 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 10 monclient(hunting): handle_auth_done global_id 34503 payload 824 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.322+0000 7fb3191a4640 1 -- 172.21.15.134:0/2667888965 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb31407a680 con 0x7fb314120580 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3191a4640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 msgr2=0x7fb31409d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3191a4640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 0x7fb31409d4f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb31002a040 tx=0x7fb310009190 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb31004d040 con 0x7fb314120580 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3067fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.324+0000 7fb3067fc640 10 monclient: dump: 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7fb31003de00 con 0x7fb314120580 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 msgr2=0x7fb31409d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 0x7fb31409d4f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb31002a040 tx=0x7fb310009190 comp rx=0 tx=0).stop 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 msgr2=0x7fb31409d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb3067fc640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb314120580 0x7fb31409d4f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.337 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.338 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.339 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.340 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.341 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.342 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.344 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.348 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.349 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.350 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.351 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.352 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.353 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.354 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.356 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.357 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.358 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 >> v2:172.21.15.162:6800/980547263 conn(0x7fb2f8038880 msgr2=0x7fb2f803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 --2- 172.21.15.134:0/2667888965 >> v2:172.21.15.162:6800/980547263 conn(0x7fb2f8038880 0x7fb2f803ad40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb31409cec0 tx=0x7fb30c007db0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 10 monclient: shutdown 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 msgr2=0x7fb3140a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 --2- 172.21.15.134:0/2667888965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb314136e40 0x7fb3140a2950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c3e0 tx=0x7fb30800c8a0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31a1a6640 1 -- 172.21.15.134:0/2667888965 reap_dead start 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31a1a6640 1 -- 172.21.15.134:0/2667888965 reap_dead start 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 shutdown_connections 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 >> 172.21.15.134:0/2667888965 conn(0x7fb3140a6110 msgr2=0x7fb31413b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 shutdown_connections 2024-09-16T15:05:46.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.325+0000 7fb31bc09640 1 -- 172.21.15.134:0/2667888965 wait complete. 2024-09-16T15:05:46.361 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.361 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-16T15:05:46.361 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 10 monclient: get_monmap_and_config 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 10 monclient: monmap: 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.458 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:46.457980+0000 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:46.457980+0000 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:46.459 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 1 Processor -- start 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 1 -- start start 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.457+0000 7fd78db65640 10 monclient: init 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient: _add_conns ranks=[3,4,7,5,8,10,6,1,2,9,0] 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 0x7fd7881342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-g con 0x7fd788133f10 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-c con 0x7fd788134830 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:46.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 0x7fd7881351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-i con 0x7fd788071bd0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a3d50 con 0x7fd788133f10 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd78806f630 con 0x7fd788134830 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a1e10 con 0x7fd788071bd0 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd78db65640 10 monclient(hunting): authenticate will time out at 1083.047368s 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd787fff640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 msgr2=0x7fd7881351f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd7877fe640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 msgr2=0x7fd7881342f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40358/0 (socket says 172.21.15.134:40358) 2024-09-16T15:05:46.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 -- 172.21.15.134:0/946687432 learned_addr learned my addr 172.21.15.134:0/946687432 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): get_auth_request con 0x7fd788134830 auth_method 0 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd787fff640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 0x7fd7881351f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd7877fe640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 0x7fd7881342f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient(hunting): handle_auth_done global_id 4900 payload 274 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 -- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 msgr2=0x7fd7881351f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 0x7fd7881351f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 -- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 msgr2=0x7fd7881342f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:05:46.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 0x7fd7881342f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 monclient: found mon.noname-c 2024-09-16T15:05:46.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:05:46.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.458+0000 7fd77effd640 1 -- 172.21.15.134:0/946687432 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788073250 con 0x7fd788134830 2024-09-16T15:05:46.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd77effd640 1 -- 172.21.15.134:0/946687432 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 msgr2=0x7fd788134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd77effd640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd77002a0b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=10d19c4a1f915e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 1 -- 172.21.15.134:0/946687432 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7700307b0 con 0x7fd788134830 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: dump: 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.474 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 1 -- 172.21.15.134:0/946687432 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770030930 con 0x7fd788134830 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd78db65640 5 monclient: authenticate success, global_id 4900 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd78db65640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:46.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd78db65640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 1 -- 172.21.15.134:0/946687432 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd7700313a0 con 0x7fd788134830 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd78db65640 4 set_mon_vals no callback set 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.461+0000 7fd786ffd640 10 monclient: dump: 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.476 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 10 monclient: shutdown 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 -- 172.21.15.134:0/946687432 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 msgr2=0x7fd788134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd77002a0b0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 -- 172.21.15.134:0/946687432 shutdown_connections 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd788071bd0 0x7fd7881351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd788134830 0x7fd788134cb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 --2- 172.21.15.134:0/946687432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788133f10 0x7fd7881342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 -- 172.21.15.134:0/946687432 >> 172.21.15.134:0/946687432 conn(0x7fd7880a7410 msgr2=0x7fd7880a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 -- 172.21.15.134:0/946687432 shutdown_connections 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 1 -- 172.21.15.134:0/946687432 wait complete. 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.462+0000 7fd78db65640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 10 monclient: monmap: 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 1 Processor -- start 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 1 -- start start 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 10 monclient: init 2024-09-16T15:05:46.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 10 monclient: _add_conns ranks=[3,0,8,1,4,5,2,6,7] 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788071bd0 0x7fd788121040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.463+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-g con 0x7fd788071bd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-a con 0x7fd788133f10 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd788121ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): picked mon.noname-e con 0x7fd788134830 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a66c0 con 0x7fd788133f10 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880a6320 con 0x7fd788071bd0 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd78809df60 con 0x7fd788134830 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd78db65640 10 monclient(hunting): authenticate will time out at 1083.053060s 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd7877fe640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788071bd0 msgr2=0x7fd788121040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd788121ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd7877fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788071bd0 0x7fd788121040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47376/0 (socket says 172.21.15.134:47376) 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 learned_addr learned my addr 172.21.15.134:0/1734163147 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd788121ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:42440/0 (socket says 172.21.15.134:42440) 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): get_auth_request con 0x7fd788133f10 auth_method 0 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): get_auth_request con 0x7fd788134830 auth_method 0 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient(hunting): handle_auth_done global_id 4968 payload 824 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 msgr2=0x7fd788121ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd788121ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788071bd0 msgr2=0x7fd788121040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:46.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd788071bd0 0x7fd788121040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 monclient: found mon.noname-a 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.464+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700094d0 con 0x7fd788133f10 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 msgr2=0x7fd788121580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd770035040 tx=0x7fd7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd770030770 con 0x7fd788133f10 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: dump: 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.483 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 5 monclient: authenticate success, global_id 4968 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770004460 con 0x7fd788133f10 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 10 monclient: _renew_subs 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788122c10 con 0x7fd788133f10 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd77003d440 con 0x7fd788133f10 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd7867fc640 4 set_mon_vals no callback set 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd784ff9640 10 monclient: dump: 2024-09-16T15:05:46.485 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.486 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 10 monclient: _renew_subs 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.466+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd788124e90 con 0x7fd788133f10 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 msgr2=0x7fd788121580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 msgr2=0x7fd788121580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd770035040 tx=0x7fd7700046c0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd770035040 tx=0x7fd7700046c0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 msgr2=0x7fd788121580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd788133f10 0x7fd788121580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient: _add_conns ranks=[5,2,8,1,4,3,7,0,6] 2024-09-16T15:05:46.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd768004d50 0x7fd768007140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): picked mon.h con 0x7fd768004d50 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd768007890 0x7fd768009cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): picked mon.b con 0x7fd768007890 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd76800c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): picked mon.e con 0x7fd788134830 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800d1f0 con 0x7fd768007890 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800d330 con 0x7fd768004d50 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800d4a0 con 0x7fd788134830 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd784ff9640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd76800c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): get_auth_request con 0x7fd788134830 auth_method 0 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd768007890 0x7fd768009cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): get_auth_request con 0x7fd768007890 auth_method 0 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd77effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient(hunting): handle_auth_done global_id 4968 payload 293 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd768004d50 msgr2=0x7fd768007140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd768004d50 0x7fd768007140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd768007890 msgr2=0x7fd768009cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd768007890 0x7fd768009cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 monclient: found mon.e 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd78db65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) v3 -- 0x7fd768003f40 con 0x7fd788134830 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd787fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 0x7fd788121ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd788073880 con 0x7fd788124830 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.467+0000 7fd7877fe640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 0x7fd788121ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient: get_auth_request con 0x7fd788124830 auth_method 0 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 10 monclient(hunting): handle_auth_done global_id 44288 payload 824 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.468+0000 7fd7877fe640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd788061ae0 con 0x7fd788124830 2024-09-16T15:05:46.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd787fff640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 msgr2=0x7fd76800c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd787fff640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd76800c7e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7880a84d0 tx=0x7fd77002e000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd780011070 con 0x7fd788134830 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 10 monclient: dump: 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780006880 con 0x7fd788134830 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd785ffb640 4 set_mon_vals no callback set 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd7877fe640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 msgr2=0x7fd788121ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd7877fe640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 0x7fd788121ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd778009fd0 tx=0x7fd77800b810 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.470+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd7800069f0 con 0x7fd788134830 2024-09-16T15:05:46.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 0x7fd768044240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 4 ==== osd_map(223..223 src has 1..223) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fd780048900 con 0x7fd788134830 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd778003de0 con 0x7fd788124830 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 10 monclient: dump: 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.494 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd7780054e0 con 0x7fd788124830 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 msgr2=0x7fd788121ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 0x7fd788121ac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd778009fd0 tx=0x7fd77800b810 comp rx=0 tx=0).stop 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 msgr2=0x7fd788121ac0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd787fff640 1 -- 172.21.15.134:0/1734163147 reap_dead start 2024-09-16T15:05:46.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788124830 0x7fd788121ac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 0x7fd768044240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.471+0000 7fd77effd640 10 monclient: get_auth_request con 0x7fd768041d80 auth_method 0 2024-09-16T15:05:46.497 INFO:tasks.rados.rados.0.smithi162.stdout:1111: done (3 left) 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1112: delete oid 22 current snap is 100 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1109: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1109: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1109: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 467 (ObjNum 413 snap 95 seq_num 413) dirty exists 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: finishing write tid 1 to smithi16234619-26 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: finishing write tid 2 to smithi16234619-26 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: finishing write tid 3 to smithi16234619-26 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: finishing write tid 4 to smithi16234619-26 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: finishing write tid 5 to smithi16234619-26 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: oid 26 updating version 0 to 413 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: oid 26 updating version 413 to 414 2024-09-16T15:05:46.498 INFO:tasks.rados.rados.0.smithi162.stdout:1108: oid 26 updating version 414 to 415 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1108: oid 26 updating version 415 to 416 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 416 (ObjNum 425 snap 100 seq_num 425) dirty exists 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1108: left oid 26 (ObjNum 425 snap 100 seq_num 425) 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1110: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1110: finishing copy_from to smithi16234619-28 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 275 (ObjNum 321 snap 72 seq_num 321) dirty exists 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1108: done (3 left) 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1109: done (2 left) 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1110: done (1 left) 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1112: done (0 left) 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1113: rollback oid 42 current snap is 100 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 96 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1114: copy_from oid 24 from oid 45 current snap is 100 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1115: write oid 27 current snap is 100 2024-09-16T15:05:46.499 INFO:tasks.rados.rados.0.smithi162.stdout:1115: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2024-09-16T15:05:46.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.473+0000 7fd77effd640 1 --2- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 0x7fd768044240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd770035040 tx=0x7fd770052040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd78db65640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd78db65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd788125020 con 0x7fd788122000 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient: get_auth_request con 0x7fd788122000 auth_method 0 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 10 monclient(hunting): handle_auth_done global_id 44297 payload 824 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.477+0000 7fd7877fe640 1 -- 172.21.15.134:0/1734163147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd78807a680 con 0x7fd788122000 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.480+0000 7fd7877fe640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 msgr2=0x7fd788121040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.480+0000 7fd7877fe640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd778013000 tx=0x7fd7780059b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd778011870 con 0x7fd788122000 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 10 monclient: dump: 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.502 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.504 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7fd778011400 con 0x7fd788122000 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 msgr2=0x7fd788121040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd778013000 tx=0x7fd7780059b0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 msgr2=0x7fd788121040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd784ff9640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd778013000 tx=0x7fd7780059b0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.505 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.506 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.507 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.508 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.509 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.510 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.511 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.512 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.513 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.514 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.515 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.519 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.526 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.528 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.528 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 msgr2=0x7fd768044240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 0x7fd768044240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd770035040 tx=0x7fd770052040 comp rx=0 tx=0).stop 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 10 monclient: shutdown 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 msgr2=0x7fd76800c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd76800c7e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7880a84d0 tx=0x7fd77002e000 comp rx=0 tx=0).stop 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 shutdown_connections 2024-09-16T15:05:46.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788134830 0x7fd76800c7e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> v2:172.21.15.162:6800/980547263 conn(0x7fd768041d80 0x7fd768044240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 --2- 172.21.15.134:0/1734163147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd788122000 0x7fd788121040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.481+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 >> 172.21.15.134:0/1734163147 conn(0x7fd7880a7410 msgr2=0x7fd78809bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.482+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 shutdown_connections 2024-09-16T15:05:46.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.482+0000 7fd78db65640 1 -- 172.21.15.134:0/1734163147 wait complete. 2024-09-16T15:05:46.531 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-16T15:05:46.532 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1115: writing smithi16234619-27 from 776589 to 1423680 tid 1 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1115: writing smithi16234619-27 from 2070517 to 2501318 tid 2 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1115: writing smithi16234619-27 from 3026559 to 3176589 tid 3 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1116: copy_from oid 46 from oid 18 current snap is 100 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1113: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1113: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1113: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 471 (ObjNum 410 snap 95 seq_num 410) dirty exists 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1113: done (3 left) 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1117: rollback oid 40 current snap is 100 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 40 to 96 2024-09-16T15:05:46.533 INFO:tasks.rados.rados.0.smithi162.stdout:1118: rollback oid 37 current snap is 100 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 96 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1119: copy_from oid 17 from oid 31 current snap is 100 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1120: write (excl) oid 39 current snap is 100 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1120: seq_num 428 ranges {793396=722340,2128999=690812} 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1120: writing smithi16234619-39 from 793396 to 1515736 tid 1 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1120: writing smithi16234619-39 from 2128999 to 2819811 tid 2 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1121: rollback oid 36 current snap is 100 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 96 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1117: finishing rollback tid 1 to smithi16234619-40 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 0 (ObjNum 33 snap 0 seq_num 866404984) dirty dne 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1118: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1118: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:1118: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:05:46.534 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 477 (ObjNum 405 snap 94 seq_num 405) dirty exists 2024-09-16T15:05:46.535 INFO:tasks.rados.rados.0.smithi162.stdout:1117: done (7 left) 2024-09-16T15:05:46.535 INFO:tasks.rados.rados.0.smithi162.stdout:1118: done (6 left) 2024-09-16T15:05:46.535 INFO:tasks.rados.rados.0.smithi162.stdout:1122: write oid 11 current snap is 100 2024-09-16T15:05:46.535 INFO:tasks.rados.rados.0.smithi162.stdout:1122: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2024-09-16T15:05:46.537 INFO:tasks.rados.rados.0.smithi162.stdout:1122: writing smithi16234619-11 from 410203 to 1207792 tid 1 2024-09-16T15:05:46.543 INFO:tasks.rados.rados.0.smithi162.stdout:1122: writing smithi16234619-11 from 1787481 to 2221951 tid 2 2024-09-16T15:05:46.550 INFO:tasks.rados.rados.0.smithi162.stdout:1122: writing smithi16234619-11 from 2681405 to 3210203 tid 3 2024-09-16T15:05:46.550 INFO:tasks.rados.rados.0.smithi162.stdout:1123: delete oid 6 current snap is 100 2024-09-16T15:05:46.550 INFO:tasks.rados.rados.0.smithi162.stdout:1121: finishing rollback tid 0 to smithi16234619-36 2024-09-16T15:05:46.550 INFO:tasks.rados.rados.0.smithi162.stdout:1115: finishing write tid 1 to smithi16234619-27 2024-09-16T15:05:46.551 INFO:tasks.rados.rados.0.smithi162.stdout:1123: done (7 left) 2024-09-16T15:05:46.551 INFO:tasks.rados.rados.0.smithi162.stdout:1124: snap_create 2024-09-16T15:05:46.561 INFO:tasks.rados.rados.0.smithi162.stdout:1115: finishing write tid 2 to smithi16234619-27 2024-09-16T15:05:46.561 INFO:tasks.rados.rados.0.smithi162.stdout:1121: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1121: finishing rollback tid 2 to smithi16234619-36 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 359 (ObjNum 127 snap 24 seq_num 127) dirty exists 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: finishing write tid 3 to smithi16234619-27 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1114: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: finishing write tid 4 to smithi16234619-27 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: finishing write tid 5 to smithi16234619-27 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: oid 27 updating version 0 to 370 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: oid 27 version 370 is already newer than 369 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: oid 27 version 370 is already newer than 368 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:1115: oid 27 version 370 is already newer than 367 2024-09-16T15:05:46.562 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 370 (ObjNum 427 snap 100 seq_num 427) dirty exists 2024-09-16T15:05:46.563 INFO:tasks.rados.rados.0.smithi162.stdout:1115: left oid 27 (ObjNum 427 snap 100 seq_num 427) 2024-09-16T15:05:46.565 INFO:tasks.rados.rados.0.smithi162.stdout:1120: finishing write tid 1 to smithi16234619-39 2024-09-16T15:05:46.565 INFO:tasks.rados.rados.0.smithi162.stdout:1114: finishing copy_from to smithi16234619-24 2024-09-16T15:05:46.565 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:05:46.566 INFO:tasks.rados.rados.0.smithi162.stdout:1120: finishing write tid 2 to smithi16234619-39 2024-09-16T15:05:46.566 INFO:tasks.rados.rados.0.smithi162.stdout:1120: finishing write tid 3 to smithi16234619-39 2024-09-16T15:05:46.567 INFO:tasks.rados.rados.0.smithi162.stdout:1120: finishing write tid 4 to smithi16234619-39 2024-09-16T15:05:46.567 INFO:tasks.rados.rados.0.smithi162.stdout:1120: oid 39 updating version 0 to 480 2024-09-16T15:05:46.567 INFO:tasks.rados.rados.0.smithi162.stdout:1120: oid 39 updating version 480 to 481 2024-09-16T15:05:46.567 INFO:tasks.rados.rados.0.smithi162.stdout:1120: oid 39 version 481 is already newer than 479 2024-09-16T15:05:46.568 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 481 (ObjNum 428 snap 100 seq_num 428) dirty exists 2024-09-16T15:05:46.568 INFO:tasks.rados.rados.0.smithi162.stdout:1120: left oid 39 (ObjNum 428 snap 100 seq_num 428) 2024-09-16T15:05:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:1116: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:05:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:1116: finishing copy_from to smithi16234619-46 2024-09-16T15:05:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 245 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:05:46.587 INFO:tasks.rados.rados.0.smithi162.stdout:1122: finishing write tid 1 to smithi16234619-11 2024-09-16T15:05:46.588 INFO:tasks.rados.rados.0.smithi162.stdout:1122: finishing write tid 2 to smithi16234619-11 2024-09-16T15:05:46.590 INFO:tasks.rados.rados.0.smithi162.stdout:1122: finishing write tid 3 to smithi16234619-11 2024-09-16T15:05:46.590 INFO:tasks.rados.rados.0.smithi162.stdout:1119: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:05:46.593 INFO:tasks.rados.rados.0.smithi162.stdout:1122: finishing write tid 4 to smithi16234619-11 2024-09-16T15:05:46.593 INFO:tasks.rados.rados.0.smithi162.stdout:1122: finishing write tid 5 to smithi16234619-11 2024-09-16T15:05:46.593 INFO:tasks.rados.rados.0.smithi162.stdout:1122: oid 11 updating version 0 to 54 2024-09-16T15:05:46.593 INFO:tasks.rados.rados.0.smithi162.stdout:1122: oid 11 updating version 54 to 57 2024-09-16T15:05:46.593 INFO:tasks.rados.rados.0.smithi162.stdout:1122: oid 11 version 57 is already newer than 55 2024-09-16T15:05:46.594 INFO:tasks.rados.rados.0.smithi162.stdout:1122: oid 11 version 57 is already newer than 56 2024-09-16T15:05:46.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 57 (ObjNum 429 snap 100 seq_num 429) dirty exists 2024-09-16T15:05:46.594 INFO:tasks.rados.rados.0.smithi162.stdout:1122: left oid 11 (ObjNum 429 snap 100 seq_num 429) 2024-09-16T15:05:46.594 INFO:tasks.rados.rados.0.smithi162.stdout:1119: finishing copy_from to smithi16234619-17 2024-09-16T15:05:46.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.643+0000 7f4877a27640 10 monclient: get_monmap_and_config 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.643+0000 7f4877a27640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.643+0000 7f4877a27640 10 monclient: monmap: 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:46.644809+0000 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:46.644809+0000 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:46.645 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:46.646 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.644+0000 7f4877a27640 1 Processor -- start 2024-09-16T15:05:46.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.644+0000 7f4877a27640 1 -- start start 2024-09-16T15:05:46.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.644+0000 7f4877a27640 10 monclient: init 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient: _add_conns ranks=[9,10,2,8,3,7,5,0,6,4,1] 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-a con 0x7f4870136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4870137220 0x7f4870137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-f con 0x7f4870137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 0x7f4870136250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-b con 0x7f4870135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487009abc0 con 0x7f4870136e40 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487009ba50 con 0x7f4870137220 2024-09-16T15:05:46.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48700a53c0 con 0x7f4870135dc0 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4877a27640 10 monclient(hunting): authenticate will time out at 1083.234192s 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 0x7f4870136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47388/0 (socket says 172.21.15.134:47388) 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 1 -- 172.21.15.134:0/602310730 learned_addr learned my addr 172.21.15.134:0/602310730 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): get_auth_request con 0x7f4870136e40 auth_method 0 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): get_auth_request con 0x7f4870135dc0 auth_method 0 2024-09-16T15:05:46.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient(hunting): handle_auth_done global_id 44300 payload 274 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 -- 172.21.15.134:0/602310730 >> v2:172.21.15.162:3300/0 conn(0x7f4870137220 msgr2=0x7f4870137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 --2- 172.21.15.134:0/602310730 >> v2:172.21.15.162:3300/0 conn(0x7f4870137220 0x7f4870137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 -- 172.21.15.134:0/602310730 >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 msgr2=0x7f487009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 --2- 172.21.15.134:0/602310730 >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 monclient: found mon.noname-b 2024-09-16T15:05:46.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f4875fc4640 1 -- 172.21.15.134:0/602310730 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870073250 con 0x7f4870135dc0 2024-09-16T15:05:46.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.645+0000 7f48757c3640 1 --2- 172.21.15.134:0/602310730 >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4875fc4640 1 -- 172.21.15.134:0/602310730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 msgr2=0x7f4870136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4875fc4640 1 --2- 172.21.15.134:0/602310730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 0x7f4870136250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d90 tx=0x7f486802a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f92a34d89fbe00 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 1 -- 172.21.15.134:0/602310730 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4868007bc0 con 0x7f4870135dc0 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: dump: 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.658 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4877a27640 5 monclient: authenticate success, global_id 44300 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 1 -- 172.21.15.134:0/602310730 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486802d050 con 0x7f4870135dc0 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4877a27640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4877a27640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 1 -- 172.21.15.134:0/602310730 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4868007d40 con 0x7f4870135dc0 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4877a27640 4 set_mon_vals no callback set 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.648+0000 7f4867fff640 10 monclient: dump: 2024-09-16T15:05:46.660 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.661 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 10 monclient: shutdown 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 -- 172.21.15.134:0/602310730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 msgr2=0x7f4870136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 --2- 172.21.15.134:0/602310730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 0x7f4870136250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d90 tx=0x7f486802a4e0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 -- 172.21.15.134:0/602310730 shutdown_connections 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 --2- 172.21.15.134:0/602310730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4870135dc0 0x7f4870136250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 --2- 172.21.15.134:0/602310730 >> v2:172.21.15.162:3300/0 conn(0x7f4870137220 0x7f4870137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 --2- 172.21.15.134:0/602310730 >> v2:172.21.15.134:3300/0 conn(0x7f4870136e40 0x7f487009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 -- 172.21.15.134:0/602310730 >> 172.21.15.134:0/602310730 conn(0x7f48700a6110 msgr2=0x7f48700a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 -- 172.21.15.134:0/602310730 shutdown_connections 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 1 -- 172.21.15.134:0/602310730 wait complete. 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.649+0000 7f4877a27640 10 monclient: monmap: 2024-09-16T15:05:46.662 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.663 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 Processor -- start 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 -- start start 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient: init 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient: _add_conns ranks=[0,8,1,5,3,2,6,7,4] 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 0x7f487011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-a con 0x7f4870135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4870136e40 0x7f48701201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-e con 0x7f4870136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870137220 0x7f48701206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): picked mon.noname-f con 0x7f4870137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48701363c0 con 0x7f4870135dc0 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4870122130 con 0x7f4870137220 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f48700a3a60 con 0x7f4870136e40 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4877a27640 10 monclient(hunting): authenticate will time out at 1083.239713s 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 0x7f487011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 0x7f487011fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47396/0 (socket says 172.21.15.134:47396) 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4870136e40 0x7f48701201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 1 -- 172.21.15.134:0/1485983578 learned_addr learned my addr 172.21.15.134:0/1485983578 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): get_auth_request con 0x7f4870135dc0 auth_method 0 2024-09-16T15:05:46.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): get_auth_request con 0x7f4870136e40 auth_method 0 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f48757c3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.650+0000 7f4874fc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 10 monclient(hunting): handle_auth_done global_id 4986 payload 824 2024-09-16T15:05:46.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870137220 0x7f48701206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4870136e40 msgr2=0x7f48701201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4870136e40 0x7f48701201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870137220 msgr2=0x7f48701206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870137220 0x7f48701206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 monclient: found mon.noname-a 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f48757c3640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0094d0 con 0x7f4870135dc0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f4875fc4640 10 monclient: get_auth_request con 0x7f4870137220 auth_method 0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.651+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870137220 0x7f48701206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f48757c3640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 msgr2=0x7f487011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f48757c3640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 0x7f487011fc60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9d0 tx=0x7f486c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f486c00cc10 con 0x7f4870135dc0 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: dump: 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c00cd90 con 0x7f4870135dc0 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 5 monclient: authenticate success, global_id 4986 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 10 monclient: _renew_subs 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f486c013d50 con 0x7f4870135dc0 2024-09-16T15:05:46.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4868009a70 con 0x7f4870135dc0 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f48677fe640 4 set_mon_vals no callback set 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4865ffb640 10 monclient: dump: 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 10 monclient: _renew_subs 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.653+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4870122410 con 0x7f4870135dc0 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f486c013470 con 0x7f4870135dc0 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4877a27640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4865ffb640 1 --2- 172.21.15.134:0/1485983578 >> v2:172.21.15.162:6800/980547263 conn(0x7f4854038880 0x7f485403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4877a27640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 0x7f487002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4870071b20 con 0x7f4870134d60 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f486c04a530 con 0x7f4870135dc0 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 0x7f487002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4874fc2640 1 --2- 172.21.15.134:0/1485983578 >> v2:172.21.15.162:6800/980547263 conn(0x7f4854038880 0x7f485403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4875fc4640 10 monclient: get_auth_request con 0x7f4870134d60 auth_method 0 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4875fc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4875fc4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4875fc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.654+0000 7f4874fc2640 10 monclient: get_auth_request con 0x7f4854038880 auth_method 0 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.655+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.655+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.655+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.655+0000 7f4875fc4640 10 monclient(hunting): handle_auth_done global_id 44263 payload 824 2024-09-16T15:05:46.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.655+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4870061ae0 con 0x7f4870134d60 2024-09-16T15:05:46.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4874fc2640 1 --2- 172.21.15.134:0/1485983578 >> v2:172.21.15.162:6800/980547263 conn(0x7f4854038880 0x7f485403ad40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f48700a2880 tx=0x7f4860009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 msgr2=0x7f487002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 0x7f487002b260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4868034000 tx=0x7f4868004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48680044b0 con 0x7f4870134d60 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4865ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4865ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.657+0000 7f4865ffb640 10 monclient: dump: 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.676 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4868004610 con 0x7f4870134d60 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 msgr2=0x7f487002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 0x7f487002b260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4868034000 tx=0x7f4868004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:46.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 msgr2=0x7f487002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.658+0000 7f4865ffb640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870134d60 0x7f487002b260 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4868034000 tx=0x7f4868004a40 comp rx=0 tx=0).stop 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.664+0000 7f4877a27640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.664+0000 7f4877a27640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 0x7f487002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.664+0000 7f4877a27640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.664+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4870135700 con 0x7f4870135140 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.664+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 0x7f487002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient: get_auth_request con 0x7f4870135140 auth_method 0 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 10 monclient(hunting): handle_auth_done global_id 44272 payload 824 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.665+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f487007a680 con 0x7f4870135140 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.667+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 msgr2=0x7f487002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.667+0000 7f4875fc4640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 0x7f487002b7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d90 tx=0x7f4868009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4868004300 con 0x7f4870135140 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 10 monclient: dump: 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.679 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f48680499a0 con 0x7f4870135140 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 msgr2=0x7f487002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 0x7f487002b7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d90 tx=0x7f4868009750 comp rx=0 tx=0).stop 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 msgr2=0x7f487002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4865ffb640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4870135140 0x7f487002b7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.681 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.682 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.683 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.684 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.685 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.687 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.689 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.691 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.692 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.694 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.702 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.703 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 >> v2:172.21.15.162:6800/980547263 conn(0x7f4854038880 msgr2=0x7f485403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 1 --2- 172.21.15.134:0/1485983578 >> v2:172.21.15.162:6800/980547263 conn(0x7f4854038880 0x7f485403ad40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f48700a2880 tx=0x7f4860009110 comp rx=0 tx=0).stop 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 10 monclient: shutdown 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 msgr2=0x7f487011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4877a27640 1 --2- 172.21.15.134:0/1485983578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4870135dc0 0x7f487011fc60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9d0 tx=0x7f486c00edd0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 reap_dead start 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.668+0000 7f4875fc4640 1 -- 172.21.15.134:0/1485983578 reap_dead start 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.669+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 shutdown_connections 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.669+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 >> 172.21.15.134:0/1485983578 conn(0x7f48700a6110 msgr2=0x7f487009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.669+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 shutdown_connections 2024-09-16T15:05:46.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.669+0000 7f4877a27640 1 -- 172.21.15.134:0/1485983578 wait complete. 2024-09-16T15:05:46.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-16T15:05:46.706 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.801+0000 7f47bb383640 10 monclient: get_monmap_and_config 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.801+0000 7f47bb383640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.801+0000 7f47bb383640 10 monclient: monmap: 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:05:46.802719+0000 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:05:46.802719+0000 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.803 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:05:46.804 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.802+0000 7f47bb383640 1 Processor -- start 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.802+0000 7f47bb383640 1 -- start start 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.802+0000 7f47bb383640 10 monclient: init 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient: _add_conns ranks=[1,9,2,5,10,7,0,8,3,6,4] 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f47b4136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47b4137220 0x7f47b4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-a con 0x7f47b4137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-b con 0x7f47b413c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b4073250 con 0x7f47b4136e40 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b40a56e0 con 0x7f47b4137220 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b409abc0 con 0x7f47b413c370 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47bb383640 10 monclient(hunting): authenticate will time out at 1083.392181s 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47b4137220 0x7f47b4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47b4137220 0x7f47b4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47408/0 (socket says 172.21.15.134:47408) 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:50536/0 (socket says 172.21.15.134:50536) 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 -- 172.21.15.134:0/3615728778 learned_addr learned my addr 172.21.15.134:0/3615728778 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): get_auth_request con 0x7f47b4137220 auth_method 0 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): get_auth_request con 0x7f47b413c370 auth_method 0 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient(hunting): handle_auth_done global_id 4995 payload 274 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 -- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 msgr2=0x7f47b413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 -- 172.21.15.134:0/3615728778 >> v1:172.21.15.162:6789/0 conn(0x7f47b4136e40 legacy=0x7f47b409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 monclient: found mon.noname-a 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b891e640 1 -- 172.21.15.134:0/3615728778 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b409ba50 con 0x7f47b4137220 2024-09-16T15:05:46.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.803+0000 7f47b9920640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47b891e640 1 -- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4137220 msgr2=0x7f47b4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47b891e640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4137220 0x7f47b4137600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009870 tx=0x7f47a402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e911d6f782958a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 1 -- 172.21.15.134:0/3615728778 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47a402ae20 con 0x7f47b4137220 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: dump: 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.817 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 1 -- 172.21.15.134:0/3615728778 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a40307b0 con 0x7f47b4137220 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47bb383640 5 monclient: authenticate success, global_id 4995 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47bb383640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47bb383640 10 monclient: get_monmap_and_config success 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 1 -- 172.21.15.134:0/3615728778 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47a4030d90 con 0x7f47b4137220 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47bb383640 4 set_mon_vals no callback set 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.806+0000 7f47abfff640 10 monclient: dump: 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.819 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 10 monclient: shutdown 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 -- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4137220 msgr2=0x7f47b4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4137220 0x7f47b4137600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009870 tx=0x7f47a402a100 comp rx=0 tx=0).stop 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 -- 172.21.15.134:0/3615728778 shutdown_connections 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 --2- 172.21.15.134:0/3615728778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4137220 0x7f47b4137600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 -- 172.21.15.134:0/3615728778 >> 172.21.15.134:0/3615728778 conn(0x7f47b40a6110 msgr2=0x7f47b40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 -- 172.21.15.134:0/3615728778 shutdown_connections 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 1 -- 172.21.15.134:0/3615728778 wait complete. 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 10 monclient: build_initial_monmap 2024-09-16T15:05:46.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.807+0000 7f47bb383640 10 monclient: monmap: 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:05:46.822 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 Processor -- start 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 -- start start 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient: init 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient: _reopen_session rank -1 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient: _add_conns ranks=[7,4,2,3,1,5,8,0,6] 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47b4136e40 0x7f47b4135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-i con 0x7f47b4136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f47b4137220 0x7f47b41363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-c con 0x7f47b4137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): picked mon.noname-b con 0x7f47b413c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:05:46.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4122920 con 0x7f47b413c370 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47b40a7a60 con 0x7f47b4137220 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f47b40a74a0 con 0x7f47b4136e40 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): _renew_subs 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47bb383640 10 monclient(hunting): authenticate will time out at 1083.397587s 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b911f640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47b4136e40 msgr2=0x7f47b4135e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f47b4137220 0x7f47b41363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b911f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47b4136e40 0x7f47b4135e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:50542/0 (socket says 172.21.15.134:50542) 2024-09-16T15:05:46.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 learned_addr learned my addr 172.21.15.134:0/2535874563 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): get_auth_request con 0x7f47b413c370 auth_method 0 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): get_auth_request con 0x7f47b4137220 auth_method 0 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient(hunting): handle_auth_done global_id 44318 payload 824 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47b4136e40 msgr2=0x7f47b4135e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47b4136e40 0x7f47b4135e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f47b4137220 msgr2=0x7f47b41363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f47b4137220 0x7f47b41363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient: _finish_hunting 0 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 monclient: found mon.noname-b 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac009a70 con 0x7f47b413c370 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.808+0000 7f47b891e640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f47b4137220 0x7f47b41363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:05:46.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 msgr2=0x7f47b41cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47b9920640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00efc0 tx=0x7f47ac00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47ac0148f0 con 0x7f47b413c370 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: dump: 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.827 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac00f040 con 0x7f47b413c370 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 5 monclient: authenticate success, global_id 44318 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 10 monclient: _renew_subs 2024-09-16T15:05:46.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47ac0133e0 con 0x7f47b413c370 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a40094d0 con 0x7f47b413c370 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47a9ffb640 10 monclient: dump: 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.829 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47ab7fe640 4 set_mon_vals no callback set 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 10 monclient: _renew_subs 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.811+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b4121fa0 con 0x7f47b413c370 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f47ac014480 con 0x7f47b413c370 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47bb383640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47bb383640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 0x7f47b41cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4073f90 con 0x7f47b40a2c20 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47a9ffb640 1 --2- 172.21.15.134:0/2535874563 >> v2:172.21.15.162:6800/980547263 conn(0x7f4798038880 0x7f479803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f47ac04a780 con 0x7f47b413c370 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 0x7f47b41cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b891e640 1 --2- 172.21.15.134:0/2535874563 >> v2:172.21.15.162:6800/980547263 conn(0x7f4798038880 0x7f479803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient: get_auth_request con 0x7f47b40a2c20 auth_method 0 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b891e640 10 monclient: get_auth_request con 0x7f4798038880 auth_method 0 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 10 monclient(hunting): handle_auth_done global_id 44267 payload 824 2024-09-16T15:05:46.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.812+0000 7f47b911f640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4061ae0 con 0x7f47b40a2c20 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47b911f640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 msgr2=0x7f47b41cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47b911f640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 0x7f47b41cbab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c380 tx=0x7f47b000c840 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47b891e640 1 --2- 172.21.15.134:0/2535874563 >> v2:172.21.15.162:6800/980547263 conn(0x7f4798038880 0x7f479803ad40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47b4137f00 tx=0x7f47a402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47b00075d0 con 0x7f47b40a2c20 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 10 monclient: dump: 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.832 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f47b00057c0 con 0x7f47b40a2c20 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 msgr2=0x7f47b41cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 0x7f47b41cbab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c380 tx=0x7f47b000c840 comp rx=0 tx=0).stop 2024-09-16T15:05:46.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 msgr2=0x7f47b41cbab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.815+0000 7f47a9ffb640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b40a2c20 0x7f47b41cbab0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c380 tx=0x7f47b000c840 comp rx=0 tx=0).stop 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.821+0000 7f47bb383640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.821+0000 7f47bb383640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 0x7f47b41cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.821+0000 7f47bb383640 10 monclient(hunting): start opening mon connection 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.821+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4122620 con 0x7f47b4121040 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 0x7f47b41cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient: get_auth_request con 0x7f47b4121040 auth_method 0 2024-09-16T15:05:46.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 10 monclient(hunting): handle_auth_done global_id 44276 payload 824 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.822+0000 7f47b911f640 1 -- 172.21.15.134:0/2535874563 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f47b407a680 con 0x7f47b4121040 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47b911f640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 msgr2=0x7f47b41cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47b911f640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 0x7f47b41cbff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b00037e0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47b001f070 con 0x7f47b4121040 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47a9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47a9ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:05:46.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.824+0000 7f47a9ffb640 10 monclient: dump: 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:05:46.837 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f47b0023410 con 0x7f47b4121040 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 msgr2=0x7f47b41cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 0x7f47b41cbff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b00037e0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 msgr2=0x7f47b41cbff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47a9ffb640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f47b4121040 0x7f47b41cbff0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47b0000c00 tx=0x7f47b00037e0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:05:46.838 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 46, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:05:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.841 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:05:46.842 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:05:46.843 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.844 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.845 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:05:46.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:05:46.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:05:46.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:05:46.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.859 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:05:46.860 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 >> v2:172.21.15.162:6800/980547263 conn(0x7f4798038880 msgr2=0x7f479803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 --2- 172.21.15.134:0/2535874563 >> v2:172.21.15.162:6800/980547263 conn(0x7f4798038880 0x7f479803ad40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47b4137f00 tx=0x7f47a402e000 comp rx=0 tx=0).stop 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 10 monclient: shutdown 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 msgr2=0x7f47b41cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47b9920640 1 -- 172.21.15.134:0/2535874563 reap_dead start 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00efc0 tx=0x7f47ac00c490 comp rx=0 tx=0).stop 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 shutdown_connections 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 --2- 172.21.15.134:0/2535874563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47b413c370 0x7f47b41cb570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 >> 172.21.15.134:0/2535874563 conn(0x7f47b40a6110 msgr2=0x7f47b413a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:05:46.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 shutdown_connections 2024-09-16T15:05:46.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:05:46.825+0000 7f47bb383640 1 -- 172.21.15.134:0/2535874563 wait complete. 2024-09-16T15:05:46.863 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:05:46.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1114: done (7 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1115: done (6 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1116: done (5 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1119: done (4 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1120: done (3 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1121: done (2 left) 2024-09-16T15:05:47.610 INFO:tasks.rados.rados.0.smithi162.stdout:1122: done (1 left) 2024-09-16T15:05:47.611 INFO:tasks.rados.rados.0.smithi162.stdout:1124: done (0 left) 2024-09-16T15:05:47.611 INFO:tasks.rados.rados.0.smithi162.stdout:1125: snap_create 2024-09-16T15:05:49.245 INFO:tasks.rados.rados.0.smithi162.stdout:1125: done (0 left) 2024-09-16T15:05:49.245 INFO:tasks.rados.rados.0.smithi162.stdout:1126: rollback oid 4 current snap is 102 2024-09-16T15:05:49.245 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 96 2024-09-16T15:05:49.245 INFO:tasks.rados.rados.0.smithi162.stdout:1127: snap_remove snap 100 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1127: done (1 left) 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1128: copy_from oid 27 from oid 17 current snap is 102 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1126: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1126: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1126: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 374 (ObjNum 353 snap 81 seq_num 353) dirty exists 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1126: done (1 left) 2024-09-16T15:05:50.253 INFO:tasks.rados.rados.0.smithi162.stdout:1129: snap_remove snap 96 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1129: done (1 left) 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1130: read oid 23 snap -1 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1130: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1131: copy_from oid 5 from oid 44 current snap is 102 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1128: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:1128: finishing copy_from to smithi16234619-27 2024-09-16T15:05:50.469 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 92 (ObjNum 281 snap 59 seq_num 281) dirty exists 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1128: done (2 left) 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1132: copy_from oid 48 from oid 24 current snap is 102 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1133: read oid 15 snap 101 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1133: expect deleted 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1134: write (excl) oid 35 current snap is 102 2024-09-16T15:05:50.470 INFO:tasks.rados.rados.0.smithi162.stdout:1134: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2024-09-16T15:05:50.474 INFO:tasks.rados.rados.0.smithi162.stdout:1134: writing smithi16234619-35 from 443817 to 991904 tid 1 2024-09-16T15:05:50.482 INFO:tasks.rados.rados.0.smithi162.stdout:1134: writing smithi16234619-35 from 1788092 to 2226231 tid 2 2024-09-16T15:05:50.488 INFO:tasks.rados.rados.0.smithi162.stdout:1134: writing smithi16234619-35 from 3019898 to 3243817 tid 3 2024-09-16T15:05:50.488 INFO:tasks.rados.rados.0.smithi162.stdout:1135: copy_from oid 38 from oid 43 current snap is 102 2024-09-16T15:05:50.488 INFO:tasks.rados.rados.0.smithi162.stdout:1136: read oid 45 snap 101 2024-09-16T15:05:50.488 INFO:tasks.rados.rados.0.smithi162.stdout:1136: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:05:50.489 INFO:tasks.rados.rados.0.smithi162.stdout:1137: read oid 42 snap -1 2024-09-16T15:05:50.489 INFO:tasks.rados.rados.0.smithi162.stdout:1137: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:05:50.489 INFO:tasks.rados.rados.0.smithi162.stdout:1138: snap_create 2024-09-16T15:05:50.489 INFO:tasks.rados.rados.0.smithi162.stdout:1130: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:05:50.670 INFO:tasks.rados.rados.0.smithi162.stdout:1136: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:05:50.776 INFO:tasks.rados.rados.0.smithi162.stdout:1137: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:05:50.866 INFO:tasks.rados.rados.0.smithi162.stdout:1131: finishing copy_from to smithi16234619-5 2024-09-16T15:05:50.866 INFO:tasks.rados.rados.0.smithi162.stdout:1131: got expected ENOENT (src dne) 2024-09-16T15:05:50.866 INFO:tasks.rados.rados.0.smithi162.stdout:1131: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:05:50.866 INFO:tasks.rados.rados.0.smithi162.stdout:1134: finishing write tid 1 to smithi16234619-35 2024-09-16T15:05:50.866 INFO:tasks.rados.rados.0.smithi162.stdout:1134: finishing write tid 2 to smithi16234619-35 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: finishing write tid 3 to smithi16234619-35 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: finishing write tid 4 to smithi16234619-35 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: finishing write tid 5 to smithi16234619-35 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: oid 35 updating version 0 to 309 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: oid 35 version 309 is already newer than 308 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: oid 35 version 309 is already newer than 307 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: oid 35 updating version 309 to 310 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 310 (ObjNum 431 snap 102 seq_num 431) dirty exists 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1134: left oid 35 (ObjNum 431 snap 102 seq_num 431) 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1132: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1132: finishing copy_from to smithi16234619-48 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 161 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1135: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:05:50.867 INFO:tasks.rados.rados.0.smithi162.stdout:1135: finishing copy_from to smithi16234619-38 2024-09-16T15:05:50.868 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 106 (ObjNum 419 snap 98 seq_num 419) dirty exists 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1130: done (8 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1131: done (7 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1132: done (6 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1133: done (5 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1134: done (4 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1135: done (3 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1136: done (2 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1137: done (1 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1138: done (0 left) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1139: read oid 50 snap 102 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1139: expect (ObjNum 394 snap 93 seq_num 394) 2024-09-16T15:05:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:1140: snap_remove snap 101 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:1140: done (1 left) 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:1141: read oid 49 snap -1 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:1141: expect deleted 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:1142: rollback oid 21 current snap is 103 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 102 2024-09-16T15:05:52.694 INFO:tasks.rados.rados.0.smithi162.stdout:1139: expect (ObjNum 394 snap 93 seq_num 394) 2024-09-16T15:05:52.778 INFO:tasks.rados.rados.0.smithi162.stdout:1142: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 0 (ObjNum 49 snap 0 seq_num 866426112) dirty dne 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:1139: done (2 left) 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:1141: done (1 left) 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:1142: done (0 left) 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:1143: write oid 15 current snap is 103 2024-09-16T15:05:52.779 INFO:tasks.rados.rados.0.smithi162.stdout:1143: seq_num 432 ranges {460624=623336,1846574=698150,3163674=96950} 2024-09-16T15:05:52.784 INFO:tasks.rados.rados.0.smithi162.stdout:1143: writing smithi16234619-15 from 460624 to 1083960 tid 1 2024-09-16T15:05:52.791 INFO:tasks.rados.rados.0.smithi162.stdout:1143: writing smithi16234619-15 from 1846574 to 2544724 tid 2 2024-09-16T15:05:52.795 INFO:tasks.rados.rados.0.smithi162.stdout:1143: writing smithi16234619-15 from 3163674 to 3260624 tid 3 2024-09-16T15:05:52.796 INFO:tasks.rados.rados.0.smithi162.stdout:1144: rollback oid 38 current snap is 103 2024-09-16T15:05:52.796 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 102 2024-09-16T15:05:52.796 INFO:tasks.rados.rados.0.smithi162.stdout:1145: delete oid 29 current snap is 103 2024-09-16T15:05:52.799 INFO:tasks.rados.rados.0.smithi162.stdout:1145: done (2 left) 2024-09-16T15:05:52.799 INFO:tasks.rados.rados.0.smithi162.stdout:1146: write oid 18 current snap is 103 2024-09-16T15:05:52.799 INFO:tasks.rados.rados.0.smithi162.stdout:1146: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2024-09-16T15:05:52.804 INFO:tasks.rados.rados.0.smithi162.stdout:1146: writing smithi16234619-18 from 494238 to 984425 tid 1 2024-09-16T15:05:52.811 INFO:tasks.rados.rados.0.smithi162.stdout:1146: writing smithi16234619-18 from 1563538 to 2265357 tid 2 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:1146: writing smithi16234619-18 from 2818520 to 3294238 tid 3 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:1147: delete oid 29 current snap is 103 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:1144: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:1144: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:1144: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:05:52.817 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 317 (ObjNum 419 snap 98 seq_num 419) dirty exists 2024-09-16T15:05:52.818 INFO:tasks.rados.rados.0.smithi162.stdout:1144: done (3 left) 2024-09-16T15:05:52.818 INFO:tasks.rados.rados.0.smithi162.stdout:1147: done (2 left) 2024-09-16T15:05:52.818 INFO:tasks.rados.rados.0.smithi162.stdout:1148: snap_create 2024-09-16T15:05:52.834 INFO:tasks.rados.rados.0.smithi162.stdout:1143: finishing write tid 1 to smithi16234619-15 2024-09-16T15:05:52.834 INFO:tasks.rados.rados.0.smithi162.stdout:1143: finishing write tid 2 to smithi16234619-15 2024-09-16T15:05:52.836 INFO:tasks.rados.rados.0.smithi162.stdout:1143: finishing write tid 3 to smithi16234619-15 2024-09-16T15:05:52.836 INFO:tasks.rados.rados.0.smithi162.stdout:1143: finishing write tid 4 to smithi16234619-15 2024-09-16T15:05:52.836 INFO:tasks.rados.rados.0.smithi162.stdout:1143: finishing write tid 5 to smithi16234619-15 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1143: oid 15 updating version 0 to 277 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1143: oid 15 version 277 is already newer than 276 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1143: oid 15 updating version 277 to 279 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1143: oid 15 version 279 is already newer than 278 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 279 (ObjNum 432 snap 103 seq_num 432) dirty exists 2024-09-16T15:05:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1143: left oid 15 (ObjNum 432 snap 103 seq_num 432) 2024-09-16T15:05:52.854 INFO:tasks.rados.rados.0.smithi162.stdout:1146: finishing write tid 1 to smithi16234619-18 2024-09-16T15:05:52.854 INFO:tasks.rados.rados.0.smithi162.stdout:1146: finishing write tid 2 to smithi16234619-18 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: finishing write tid 3 to smithi16234619-18 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: finishing write tid 4 to smithi16234619-18 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: finishing write tid 5 to smithi16234619-18 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: oid 18 updating version 0 to 247 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: oid 18 updating version 247 to 249 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: oid 18 version 249 is already newer than 248 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: oid 18 version 249 is already newer than 246 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 249 (ObjNum 434 snap 103 seq_num 434) dirty exists 2024-09-16T15:05:52.855 INFO:tasks.rados.rados.0.smithi162.stdout:1146: left oid 18 (ObjNum 434 snap 103 seq_num 434) 2024-09-16T15:05:53.491 INFO:tasks.rados.rados.0.smithi162.stdout:1143: done (2 left) 2024-09-16T15:05:53.492 INFO:tasks.rados.rados.0.smithi162.stdout:1146: done (1 left) 2024-09-16T15:05:53.492 INFO:tasks.rados.rados.0.smithi162.stdout:1148: done (0 left) 2024-09-16T15:05:53.492 INFO:tasks.rados.rados.0.smithi162.stdout:1149: read oid 49 snap -1 2024-09-16T15:05:53.492 INFO:tasks.rados.rados.0.smithi162.stdout:1149: expect deleted 2024-09-16T15:05:53.492 INFO:tasks.rados.rados.0.smithi162.stdout:1150: snap_remove snap 102 2024-09-16T15:05:54.501 INFO:tasks.rados.rados.0.smithi162.stdout:1150: done (1 left) 2024-09-16T15:05:54.501 INFO:tasks.rados.rados.0.smithi162.stdout:1151: read oid 19 snap 103 2024-09-16T15:05:54.501 INFO:tasks.rados.rados.0.smithi162.stdout:1151: expect deleted 2024-09-16T15:05:54.501 INFO:tasks.rados.rados.0.smithi162.stdout:1149: done (1 left) 2024-09-16T15:05:54.502 INFO:tasks.rados.rados.0.smithi162.stdout:1152: write oid 50 current snap is 104 2024-09-16T15:05:54.502 INFO:tasks.rados.rados.0.smithi162.stdout:1152: seq_num 436 ranges {527852=640685,1680502=589135,2757013=570839} 2024-09-16T15:05:54.508 INFO:tasks.rados.rados.0.smithi162.stdout:1152: writing smithi16234619-50 from 527852 to 1168537 tid 1 2024-09-16T15:05:54.513 INFO:tasks.rados.rados.0.smithi162.stdout:1152: writing smithi16234619-50 from 1680502 to 2269637 tid 2 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1152: writing smithi16234619-50 from 2757013 to 3327852 tid 3 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1153: copy_from oid 14 from oid 12 current snap is 104 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1151: done (2 left) 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1154: read oid 17 snap -1 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1154: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:05:54.520 INFO:tasks.rados.rados.0.smithi162.stdout:1155: read oid 15 snap -1 2024-09-16T15:05:54.521 INFO:tasks.rados.rados.0.smithi162.stdout:1155: expect (ObjNum 432 snap 103 seq_num 432) 2024-09-16T15:05:54.521 INFO:tasks.rados.rados.0.smithi162.stdout:1156: copy_from oid 34 from oid 21 current snap is 104 2024-09-16T15:05:54.521 INFO:tasks.rados.rados.0.smithi162.stdout:1157: copy_from oid 27 from oid 46 current snap is 104 2024-09-16T15:05:54.521 INFO:tasks.rados.rados.0.smithi162.stdout:1158: snap_remove snap 103 2024-09-16T15:05:55.456 INFO:tasks.rados.rados.0.smithi162.stdout:1158: done (6 left) 2024-09-16T15:05:55.456 INFO:tasks.rados.rados.0.smithi162.stdout:1154: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:05:55.548 INFO:tasks.rados.rados.0.smithi162.stdout:1155: expect (ObjNum 432 snap 103 seq_num 432) 2024-09-16T15:05:55.653 INFO:tasks.rados.rados.0.smithi162.stdout:1152: finishing write tid 1 to smithi16234619-50 2024-09-16T15:05:55.653 INFO:tasks.rados.rados.0.smithi162.stdout:1156: finishing copy_from to smithi16234619-34 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1156: got expected ENOENT (src dne) 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1156: finishing copy_from racing read to smithi16234619-34 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1157: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1153: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1152: finishing write tid 2 to smithi16234619-50 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1157: finishing copy_from to smithi16234619-27 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 245 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1152: finishing write tid 3 to smithi16234619-50 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1152: finishing write tid 4 to smithi16234619-50 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1159: read oid 42 snap -1 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1159: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1154: done (6 left) 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1155: done (5 left) 2024-09-16T15:05:55.654 INFO:tasks.rados.rados.0.smithi162.stdout:1156: done (4 left) 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1157: done (3 left) 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1160: write oid 13 current snap is 104 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: finishing write tid 5 to smithi16234619-50 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: oid 50 updating version 0 to 368 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: oid 50 version 368 is already newer than 366 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: oid 50 version 368 is already newer than 365 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: oid 50 version 368 is already newer than 367 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 368 (ObjNum 436 snap 104 seq_num 436) dirty exists 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1152: left oid 50 (ObjNum 436 snap 104 seq_num 436) 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1153: finishing copy_from to smithi16234619-14 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 209 (ObjNum 415 snap 98 seq_num 415) dirty exists 2024-09-16T15:05:55.655 INFO:tasks.rados.rados.0.smithi162.stdout:1160: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2024-09-16T15:05:55.659 INFO:tasks.rados.rados.0.smithi162.stdout:1160: writing smithi16234619-13 from 544659 to 1260593 tid 1 2024-09-16T15:05:55.665 INFO:tasks.rados.rados.0.smithi162.stdout:1160: writing smithi16234619-13 from 1738984 to 2188130 tid 2 2024-09-16T15:05:55.671 INFO:tasks.rados.rados.0.smithi162.stdout:1160: writing smithi16234619-13 from 2784436 to 3344659 tid 3 2024-09-16T15:05:55.671 INFO:tasks.rados.rados.0.smithi162.stdout:1152: done (3 left) 2024-09-16T15:05:55.671 INFO:tasks.rados.rados.0.smithi162.stdout:1153: done (2 left) 2024-09-16T15:05:55.671 INFO:tasks.rados.rados.0.smithi162.stdout:1161: delete oid 27 current snap is 104 2024-09-16T15:05:55.671 INFO:tasks.rados.rados.0.smithi162.stdout:1159: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: finishing write tid 1 to smithi16234619-13 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: finishing write tid 2 to smithi16234619-13 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: finishing write tid 3 to smithi16234619-13 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: finishing write tid 4 to smithi16234619-13 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: finishing write tid 5 to smithi16234619-13 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: oid 13 updating version 0 to 495 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: oid 13 updating version 495 to 498 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: oid 13 version 498 is already newer than 497 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:1160: oid 13 version 498 is already newer than 496 2024-09-16T15:05:55.766 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 498 (ObjNum 437 snap 104 seq_num 437) dirty exists 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1160: left oid 13 (ObjNum 437 snap 104 seq_num 437) 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1159: done (2 left) 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1160: done (1 left) 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1161: done (0 left) 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1162: write oid 18 current snap is 104 2024-09-16T15:05:55.767 INFO:tasks.rados.rados.0.smithi162.stdout:1162: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2024-09-16T15:05:55.771 INFO:tasks.rados.rados.0.smithi162.stdout:1162: writing smithi16234619-18 from 578273 to 1044705 tid 1 2024-09-16T15:05:55.777 INFO:tasks.rados.rados.0.smithi162.stdout:1162: writing smithi16234619-18 from 1739595 to 2192410 tid 2 2024-09-16T15:05:55.783 INFO:tasks.rados.rados.0.smithi162.stdout:1162: writing smithi16234619-18 from 2722929 to 3378273 tid 3 2024-09-16T15:05:55.783 INFO:tasks.rados.rados.0.smithi162.stdout:1163: delete oid 29 current snap is 104 2024-09-16T15:05:55.787 INFO:tasks.rados.rados.0.smithi162.stdout:1163: done (1 left) 2024-09-16T15:05:55.787 INFO:tasks.rados.rados.0.smithi162.stdout:1164: write oid 5 current snap is 104 2024-09-16T15:05:55.787 INFO:tasks.rados.rados.0.smithi162.stdout:1164: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2024-09-16T15:05:55.794 INFO:tasks.rados.rados.0.smithi162.stdout:1164: writing smithi16234619-5 from 611887 to 1345170 tid 1 2024-09-16T15:05:55.800 INFO:tasks.rados.rados.0.smithi162.stdout:1164: writing smithi16234619-5 from 1972912 to 2429396 tid 2 2024-09-16T15:05:55.805 INFO:tasks.rados.rados.0.smithi162.stdout:1164: writing smithi16234619-5 from 2894128 to 3411887 tid 3 2024-09-16T15:05:55.806 INFO:tasks.rados.rados.0.smithi162.stdout:1165: copy_from oid 29 from oid 46 current snap is 104 2024-09-16T15:05:55.806 INFO:tasks.rados.rados.0.smithi162.stdout:1166: snap_create 2024-09-16T15:05:55.809 INFO:tasks.rados.rados.0.smithi162.stdout:1162: finishing write tid 1 to smithi16234619-18 2024-09-16T15:05:55.811 INFO:tasks.rados.rados.0.smithi162.stdout:1162: finishing write tid 2 to smithi16234619-18 2024-09-16T15:05:55.811 INFO:tasks.rados.rados.0.smithi162.stdout:1162: finishing write tid 3 to smithi16234619-18 2024-09-16T15:05:55.812 INFO:tasks.rados.rados.0.smithi162.stdout:1162: finishing write tid 4 to smithi16234619-18 2024-09-16T15:05:55.812 INFO:tasks.rados.rados.0.smithi162.stdout:1162: finishing write tid 5 to smithi16234619-18 2024-09-16T15:05:55.812 INFO:tasks.rados.rados.0.smithi162.stdout:1162: oid 18 updating version 0 to 252 2024-09-16T15:05:55.812 INFO:tasks.rados.rados.0.smithi162.stdout:1162: oid 18 version 252 is already newer than 250 2024-09-16T15:05:55.812 INFO:tasks.rados.rados.0.smithi162.stdout:1162: oid 18 version 252 is already newer than 251 2024-09-16T15:05:55.813 INFO:tasks.rados.rados.0.smithi162.stdout:1162: oid 18 updating version 252 to 253 2024-09-16T15:05:55.813 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 253 (ObjNum 439 snap 104 seq_num 439) dirty exists 2024-09-16T15:05:55.813 INFO:tasks.rados.rados.0.smithi162.stdout:1162: left oid 18 (ObjNum 439 snap 104 seq_num 439) 2024-09-16T15:05:55.830 INFO:tasks.rados.rados.0.smithi162.stdout:1164: finishing write tid 1 to smithi16234619-5 2024-09-16T15:05:55.831 INFO:tasks.rados.rados.0.smithi162.stdout:1164: finishing write tid 2 to smithi16234619-5 2024-09-16T15:05:55.832 INFO:tasks.rados.rados.0.smithi162.stdout:1164: finishing write tid 3 to smithi16234619-5 2024-09-16T15:05:55.834 INFO:tasks.rados.rados.0.smithi162.stdout:1164: finishing write tid 4 to smithi16234619-5 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: finishing write tid 5 to smithi16234619-5 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: oid 5 updating version 0 to 502 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: oid 5 updating version 502 to 503 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: oid 5 version 503 is already newer than 500 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: oid 5 version 503 is already newer than 501 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 503 (ObjNum 441 snap 104 seq_num 441) dirty exists 2024-09-16T15:05:55.835 INFO:tasks.rados.rados.0.smithi162.stdout:1164: left oid 5 (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:05:55.871 INFO:tasks.rados.rados.0.smithi162.stdout:1165: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:05:55.874 INFO:tasks.rados.rados.0.smithi162.stdout:1165: finishing copy_from to smithi16234619-29 2024-09-16T15:05:55.874 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 245 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:05:56.509 INFO:tasks.rados.rados.0.smithi162.stdout:1162: done (3 left) 2024-09-16T15:05:56.509 INFO:tasks.rados.rados.0.smithi162.stdout:1164: done (2 left) 2024-09-16T15:05:56.509 INFO:tasks.rados.rados.0.smithi162.stdout:1165: done (1 left) 2024-09-16T15:05:56.509 INFO:tasks.rados.rados.0.smithi162.stdout:1166: done (0 left) 2024-09-16T15:05:56.509 INFO:tasks.rados.rados.0.smithi162.stdout:1167: snap_create 2024-09-16T15:05:57.515 INFO:tasks.rados.rados.0.smithi162.stdout:1167: done (0 left) 2024-09-16T15:05:57.515 INFO:tasks.rados.rados.0.smithi162.stdout:1168: write oid 27 current snap is 106 2024-09-16T15:05:57.515 INFO:tasks.rados.rados.0.smithi162.stdout:1168: seq_num 442 ranges {628694=408532,1515041=600142,2805198=552112} 2024-09-16T15:05:57.520 INFO:tasks.rados.rados.0.smithi162.stdout:1168: writing smithi16234619-27 from 628694 to 1037226 tid 1 2024-09-16T15:05:57.526 INFO:tasks.rados.rados.0.smithi162.stdout:1168: writing smithi16234619-27 from 1515041 to 2115183 tid 2 2024-09-16T15:05:57.533 INFO:tasks.rados.rados.0.smithi162.stdout:1168: writing smithi16234619-27 from 2805198 to 3357310 tid 3 2024-09-16T15:05:57.533 INFO:tasks.rados.rados.0.smithi162.stdout:1169: write (excl) oid 45 current snap is 106 2024-09-16T15:05:57.533 INFO:tasks.rados.rados.0.smithi162.stdout:1169: seq_num 443 ranges {645501=483781,1573523=460153,2548974=763384} 2024-09-16T15:05:57.539 INFO:tasks.rados.rados.0.smithi162.stdout:1169: writing smithi16234619-45 from 645501 to 1129282 tid 1 2024-09-16T15:05:57.545 INFO:tasks.rados.rados.0.smithi162.stdout:1169: writing smithi16234619-45 from 1573523 to 2033676 tid 2 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1169: writing smithi16234619-45 from 2548974 to 3312358 tid 3 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1170: read oid 46 snap -1 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1170: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1171: read oid 49 snap -1 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1171: expect deleted 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1172: write (excl) oid 16 current snap is 106 2024-09-16T15:05:57.554 INFO:tasks.rados.rados.0.smithi162.stdout:1172: seq_num 444 ranges {662308=559030,1632005=603811,2860044=602264} 2024-09-16T15:05:57.560 INFO:tasks.rados.rados.0.smithi162.stdout:1172: writing smithi16234619-16 from 662308 to 1221338 tid 1 2024-09-16T15:05:57.566 INFO:tasks.rados.rados.0.smithi162.stdout:1172: writing smithi16234619-16 from 1632005 to 2235816 tid 2 2024-09-16T15:05:57.573 INFO:tasks.rados.rados.0.smithi162.stdout:1172: writing smithi16234619-16 from 2860044 to 3462308 tid 3 2024-09-16T15:05:57.573 INFO:tasks.rados.rados.0.smithi162.stdout:1173: write (excl) oid 9 current snap is 106 2024-09-16T15:05:57.573 INFO:tasks.rados.rados.0.smithi162.stdout:1173: seq_num 445 ranges {679115=634279,2090487=463822,3003820=475295} 2024-09-16T15:05:57.581 INFO:tasks.rados.rados.0.smithi162.stdout:1173: writing smithi16234619-9 from 679115 to 1313394 tid 1 2024-09-16T15:05:57.589 INFO:tasks.rados.rados.0.smithi162.stdout:1173: writing smithi16234619-9 from 2090487 to 2554309 tid 2 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1173: writing smithi16234619-9 from 3003820 to 3479115 tid 3 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1174: delete oid 24 current snap is 106 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1168: finishing write tid 1 to smithi16234619-27 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1168: finishing write tid 2 to smithi16234619-27 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1168: finishing write tid 3 to smithi16234619-27 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1168: finishing write tid 4 to smithi16234619-27 2024-09-16T15:05:57.596 INFO:tasks.rados.rados.0.smithi162.stdout:1168: finishing write tid 5 to smithi16234619-27 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1168: oid 27 updating version 0 to 396 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1168: oid 27 updating version 396 to 398 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1168: oid 27 updating version 398 to 399 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1168: oid 27 version 399 is already newer than 397 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 399 (ObjNum 442 snap 106 seq_num 442) dirty exists 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1168: left oid 27 (ObjNum 442 snap 106 seq_num 442) 2024-09-16T15:05:57.597 INFO:tasks.rados.rados.0.smithi162.stdout:1170: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:05:57.700 INFO:tasks.rados.rados.0.smithi162.stdout:1169: finishing write tid 1 to smithi16234619-45 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: finishing write tid 2 to smithi16234619-45 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: finishing write tid 3 to smithi16234619-45 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: finishing write tid 4 to smithi16234619-45 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: finishing write tid 5 to smithi16234619-45 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: oid 45 updating version 0 to 436 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: oid 45 updating version 436 to 437 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: oid 45 version 437 is already newer than 435 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: oid 45 version 437 is already newer than 434 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 437 (ObjNum 443 snap 106 seq_num 443) dirty exists 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1169: left oid 45 (ObjNum 443 snap 106 seq_num 443) 2024-09-16T15:05:57.701 INFO:tasks.rados.rados.0.smithi162.stdout:1168: done (6 left) 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1169: done (5 left) 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1170: done (4 left) 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1171: done (3 left) 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1174: done (2 left) 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1175: snap_create 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: finishing write tid 1 to smithi16234619-16 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: finishing write tid 2 to smithi16234619-16 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: finishing write tid 3 to smithi16234619-16 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: finishing write tid 4 to smithi16234619-16 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: finishing write tid 5 to smithi16234619-16 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: oid 16 updating version 0 to 323 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: oid 16 version 323 is already newer than 322 2024-09-16T15:05:57.702 INFO:tasks.rados.rados.0.smithi162.stdout:1172: oid 16 updating version 323 to 324 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1172: oid 16 version 324 is already newer than 321 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 324 (ObjNum 444 snap 106 seq_num 444) dirty exists 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1172: left oid 16 (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1173: finishing write tid 1 to smithi16234619-9 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1173: finishing write tid 2 to smithi16234619-9 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1173: finishing write tid 3 to smithi16234619-9 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1173: finishing write tid 4 to smithi16234619-9 2024-09-16T15:05:57.703 INFO:tasks.rados.rados.0.smithi162.stdout:1173: finishing write tid 5 to smithi16234619-9 2024-09-16T15:05:57.704 INFO:tasks.rados.rados.0.smithi162.stdout:1173: oid 9 updating version 0 to 403 2024-09-16T15:05:57.704 INFO:tasks.rados.rados.0.smithi162.stdout:1173: oid 9 version 403 is already newer than 402 2024-09-16T15:05:57.704 INFO:tasks.rados.rados.0.smithi162.stdout:1173: oid 9 updating version 403 to 404 2024-09-16T15:05:57.704 INFO:tasks.rados.rados.0.smithi162.stdout:1173: oid 9 version 404 is already newer than 401 2024-09-16T15:05:57.705 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 404 (ObjNum 445 snap 106 seq_num 445) dirty exists 2024-09-16T15:05:57.705 INFO:tasks.rados.rados.0.smithi162.stdout:1173: left oid 9 (ObjNum 445 snap 106 seq_num 445) 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1172: done (2 left) 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1173: done (1 left) 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1175: done (0 left) 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1176: rollback oid 45 current snap is 107 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 105 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1177: rollback oid 38 current snap is 107 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 104 2024-09-16T15:05:58.521 INFO:tasks.rados.rados.0.smithi162.stdout:1178: read oid 7 snap 104 2024-09-16T15:05:58.522 INFO:tasks.rados.rados.0.smithi162.stdout:1178: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:05:58.522 INFO:tasks.rados.rados.0.smithi162.stdout:1179: delete oid 39 current snap is 107 2024-09-16T15:05:58.529 INFO:tasks.rados.rados.0.smithi162.stdout:1176: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:05:58.531 INFO:tasks.rados.rados.0.smithi162.stdout:1177: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:05:58.532 INFO:tasks.rados.rados.0.smithi162.stdout:1178: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:05:58.778 INFO:tasks.rados.rados.0.smithi162.stdout:1176: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:05:58.778 INFO:tasks.rados.rados.0.smithi162.stdout:1176: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:05:58.778 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 443 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1177: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1177: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 328 (ObjNum 419 snap 98 seq_num 419) dirty exists 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1176: done (3 left) 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1177: done (2 left) 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1178: done (1 left) 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1179: done (0 left) 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1180: read oid 5 snap 104 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1180: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1181: write (excl) oid 48 current snap is 107 2024-09-16T15:05:58.779 INFO:tasks.rados.rados.0.smithi162.stdout:1181: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2024-09-16T15:05:58.784 INFO:tasks.rados.rados.0.smithi162.stdout:1181: writing smithi16234619-48 from 729536 to 1189562 tid 1 2024-09-16T15:05:58.791 INFO:tasks.rados.rados.0.smithi162.stdout:1181: writing smithi16234619-48 from 1749580 to 2360729 tid 2 2024-09-16T15:05:58.796 INFO:tasks.rados.rados.0.smithi162.stdout:1181: writing smithi16234619-48 from 2853383 to 3305833 tid 3 2024-09-16T15:05:58.796 INFO:tasks.rados.rados.0.smithi162.stdout:1182: copy_from oid 38 from oid 22 current snap is 107 2024-09-16T15:05:58.796 INFO:tasks.rados.rados.0.smithi162.stdout:1180: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:05:58.906 INFO:tasks.rados.rados.0.smithi162.stdout:1182: finishing copy_from to smithi16234619-38 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1182: got expected ENOENT (src dne) 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1182: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: finishing write tid 1 to smithi16234619-48 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: finishing write tid 2 to smithi16234619-48 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: finishing write tid 3 to smithi16234619-48 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: finishing write tid 4 to smithi16234619-48 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: finishing write tid 5 to smithi16234619-48 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: oid 48 updating version 0 to 408 2024-09-16T15:05:58.907 INFO:tasks.rados.rados.0.smithi162.stdout:1181: oid 48 version 408 is already newer than 407 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1181: oid 48 updating version 408 to 409 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1181: oid 48 version 409 is already newer than 406 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 409 (ObjNum 448 snap 107 seq_num 448) dirty exists 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1181: left oid 48 (ObjNum 448 snap 107 seq_num 448) 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1180: done (2 left) 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1181: done (1 left) 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1182: done (0 left) 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1183: write oid 48 current snap is 107 2024-09-16T15:05:58.908 INFO:tasks.rados.rados.0.smithi162.stdout:1183: seq_num 449 ranges {746343=651628,1924415=754807,3397159=149184} 2024-09-16T15:05:58.913 INFO:tasks.rados.rados.0.smithi162.stdout:1183: writing smithi16234619-48 from 746343 to 1397971 tid 1 2024-09-16T15:05:58.920 INFO:tasks.rados.rados.0.smithi162.stdout:1183: writing smithi16234619-48 from 1924415 to 2679222 tid 2 2024-09-16T15:05:58.925 INFO:tasks.rados.rados.0.smithi162.stdout:1183: writing smithi16234619-48 from 3397159 to 3546343 tid 3 2024-09-16T15:05:58.925 INFO:tasks.rados.rados.0.smithi162.stdout:1184: write (excl) oid 38 current snap is 107 2024-09-16T15:05:58.925 INFO:tasks.rados.rados.0.smithi162.stdout:1184: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2024-09-16T15:05:58.933 INFO:tasks.rados.rados.0.smithi162.stdout:1184: writing smithi16234619-38 from 763150 to 1490027 tid 1 2024-09-16T15:05:58.939 INFO:tasks.rados.rados.0.smithi162.stdout:1184: writing smithi16234619-38 from 1982897 to 2597715 tid 2 2024-09-16T15:05:58.944 INFO:tasks.rados.rados.0.smithi162.stdout:1184: writing smithi16234619-38 from 3024582 to 3499576 tid 3 2024-09-16T15:05:58.944 INFO:tasks.rados.rados.0.smithi162.stdout:1185: read oid 8 snap -1 2024-09-16T15:05:58.944 INFO:tasks.rados.rados.0.smithi162.stdout:1185: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:05:58.944 INFO:tasks.rados.rados.0.smithi162.stdout:1186: read oid 17 snap 106 2024-09-16T15:05:58.944 INFO:tasks.rados.rados.0.smithi162.stdout:1186: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:05:58.945 INFO:tasks.rados.rados.0.smithi162.stdout:1187: snap_create 2024-09-16T15:05:58.947 INFO:tasks.rados.rados.0.smithi162.stdout:1186: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:05:59.041 INFO:tasks.rados.rados.0.smithi162.stdout:1185: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:05:59.112 INFO:tasks.rados.rados.0.smithi162.stdout:1183: finishing write tid 1 to smithi16234619-48 2024-09-16T15:05:59.112 INFO:tasks.rados.rados.0.smithi162.stdout:1183: finishing write tid 2 to smithi16234619-48 2024-09-16T15:05:59.112 INFO:tasks.rados.rados.0.smithi162.stdout:1183: finishing write tid 3 to smithi16234619-48 2024-09-16T15:05:59.112 INFO:tasks.rados.rados.0.smithi162.stdout:1183: finishing write tid 4 to smithi16234619-48 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: finishing write tid 5 to smithi16234619-48 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: oid 48 updating version 0 to 413 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: oid 48 version 413 is already newer than 412 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: oid 48 version 413 is already newer than 411 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: oid 48 version 413 is already newer than 410 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 413 (ObjNum 449 snap 107 seq_num 449) dirty exists 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1183: left oid 48 (ObjNum 449 snap 107 seq_num 449) 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1184: finishing write tid 1 to smithi16234619-38 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1184: finishing write tid 2 to smithi16234619-38 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1184: finishing write tid 3 to smithi16234619-38 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1184: finishing write tid 4 to smithi16234619-38 2024-09-16T15:05:59.113 INFO:tasks.rados.rados.0.smithi162.stdout:1184: finishing write tid 5 to smithi16234619-38 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:1184: oid 38 updating version 0 to 331 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:1184: oid 38 version 331 is already newer than 329 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:1184: oid 38 updating version 331 to 332 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:1184: oid 38 version 332 is already newer than 330 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 332 (ObjNum 450 snap 107 seq_num 450) dirty exists 2024-09-16T15:05:59.114 INFO:tasks.rados.rados.0.smithi162.stdout:1184: left oid 38 (ObjNum 450 snap 107 seq_num 450) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1183: done (4 left) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1184: done (3 left) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1185: done (2 left) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1186: done (1 left) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1187: done (0 left) 2024-09-16T15:05:59.530 INFO:tasks.rados.rados.0.smithi162.stdout:1188: read oid 41 snap -1 2024-09-16T15:05:59.531 INFO:tasks.rados.rados.0.smithi162.stdout:1188: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:05:59.531 INFO:tasks.rados.rados.0.smithi162.stdout:1189: delete oid 9 current snap is 108 2024-09-16T15:05:59.536 INFO:tasks.rados.rados.0.smithi162.stdout:1189: done (1 left) 2024-09-16T15:05:59.536 INFO:tasks.rados.rados.0.smithi162.stdout:1190: delete oid 14 current snap is 108 2024-09-16T15:05:59.539 INFO:tasks.rados.rados.0.smithi162.stdout:1188: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:05:59.706 INFO:tasks.rados.rados.0.smithi162.stdout:1188: done (1 left) 2024-09-16T15:05:59.706 INFO:tasks.rados.rados.0.smithi162.stdout:1190: done (0 left) 2024-09-16T15:05:59.706 INFO:tasks.rados.rados.0.smithi162.stdout:1191: read oid 25 snap -1 2024-09-16T15:05:59.706 INFO:tasks.rados.rados.0.smithi162.stdout:1191: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:05:59.707 INFO:tasks.rados.rados.0.smithi162.stdout:1192: delete oid 35 current snap is 108 2024-09-16T15:05:59.709 INFO:tasks.rados.rados.0.smithi162.stdout:1192: done (1 left) 2024-09-16T15:05:59.709 INFO:tasks.rados.rados.0.smithi162.stdout:1193: read oid 26 snap -1 2024-09-16T15:05:59.709 INFO:tasks.rados.rados.0.smithi162.stdout:1193: expect (ObjNum 425 snap 100 seq_num 425) 2024-09-16T15:05:59.709 INFO:tasks.rados.rados.0.smithi162.stdout:1194: rollback oid 27 current snap is 108 2024-09-16T15:05:59.710 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 27 to 104 2024-09-16T15:05:59.710 INFO:tasks.rados.rados.0.smithi162.stdout:1195: read oid 37 snap 106 2024-09-16T15:05:59.710 INFO:tasks.rados.rados.0.smithi162.stdout:1195: expect (ObjNum 405 snap 94 seq_num 405) 2024-09-16T15:05:59.710 INFO:tasks.rados.rados.0.smithi162.stdout:1196: delete oid 2 current snap is 108 2024-09-16T15:05:59.712 INFO:tasks.rados.rados.0.smithi162.stdout:1194: finishing rollback tid 0 to smithi16234619-27 2024-09-16T15:05:59.712 INFO:tasks.rados.rados.0.smithi162.stdout:1194: finishing rollback tid 1 to smithi16234619-27 2024-09-16T15:05:59.712 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 418 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:05:59.717 INFO:tasks.rados.rados.0.smithi162.stdout:1193: expect (ObjNum 425 snap 100 seq_num 425) 2024-09-16T15:05:59.879 INFO:tasks.rados.rados.0.smithi162.stdout:1195: expect (ObjNum 405 snap 94 seq_num 405) 2024-09-16T15:06:00.117 INFO:tasks.rados.rados.0.smithi162.stdout:1191: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1191: done (4 left) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1193: done (3 left) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1194: done (2 left) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1195: done (1 left) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1196: done (0 left) 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:1197: rollback oid 32 current snap is 108 2024-09-16T15:06:00.292 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 105 2024-09-16T15:06:00.293 INFO:tasks.rados.rados.0.smithi162.stdout:1198: delete oid 8 current snap is 108 2024-09-16T15:06:00.293 INFO:tasks.rados.rados.0.smithi162.stdout:1197: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:06:00.293 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 0 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-16T15:06:00.295 INFO:tasks.rados.rados.0.smithi162.stdout:1197: done (1 left) 2024-09-16T15:06:00.295 INFO:tasks.rados.rados.0.smithi162.stdout:1198: done (0 left) 2024-09-16T15:06:00.295 INFO:tasks.rados.rados.0.smithi162.stdout:1199: rollback oid 8 current snap is 108 2024-09-16T15:06:00.295 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 106 2024-09-16T15:06:00.295 INFO:tasks.rados.rados.0.smithi162.stdout:1200: snap_create 2024-09-16T15:06:00.297 INFO:tasks.rados.rados.0.smithi162.stdout:1199: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:06:00.297 INFO:tasks.rados.rados.0.smithi162.stdout:1199: finishing rollback tid 2 to smithi16234619-8 2024-09-16T15:06:00.297 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 256 (ObjNum 372 snap 85 seq_num 372) dirty exists 2024-09-16T15:06:00.403 INFO:tasks.rados.rados.0.smithi162.stdout:1199: done (1 left) 2024-09-16T15:06:00.403 INFO:tasks.rados.rados.0.smithi162.stdout:1200: done (0 left) 2024-09-16T15:06:00.404 INFO:tasks.rados.rados.0.smithi162.stdout:1201: write oid 11 current snap is 109 2024-09-16T15:06:00.404 INFO:tasks.rados.rados.0.smithi162.stdout:1201: seq_num 456 ranges {463992=778371,1701083=625825,2956414=658979} 2024-09-16T15:06:00.410 INFO:tasks.rados.rados.0.smithi162.stdout:1201: writing smithi16234619-11 from 463992 to 1242363 tid 1 2024-09-16T15:06:00.416 INFO:tasks.rados.rados.0.smithi162.stdout:1201: writing smithi16234619-11 from 1701083 to 2326908 tid 2 2024-09-16T15:06:00.423 INFO:tasks.rados.rados.0.smithi162.stdout:1201: writing smithi16234619-11 from 2956414 to 3615393 tid 3 2024-09-16T15:06:00.423 INFO:tasks.rados.rados.0.smithi162.stdout:1202: write (excl) oid 26 current snap is 109 2024-09-16T15:06:00.423 INFO:tasks.rados.rados.0.smithi162.stdout:1202: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2024-09-16T15:06:00.428 INFO:tasks.rados.rados.0.smithi162.stdout:1202: writing smithi16234619-26 from 480799 to 1050772 tid 1 2024-09-16T15:06:00.435 INFO:tasks.rados.rados.0.smithi162.stdout:1202: writing smithi16234619-26 from 1475918 to 2245401 tid 2 2024-09-16T15:06:00.441 INFO:tasks.rados.rados.0.smithi162.stdout:1202: writing smithi16234619-26 from 2700190 to 3286794 tid 3 2024-09-16T15:06:00.441 INFO:tasks.rados.rados.0.smithi162.stdout:1203: write oid 43 current snap is 109 2024-09-16T15:06:00.441 INFO:tasks.rados.rados.0.smithi162.stdout:1203: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2024-09-16T15:06:00.447 INFO:tasks.rados.rados.0.smithi162.stdout:1203: writing smithi16234619-43 from 497606 to 1142828 tid 1 2024-09-16T15:06:00.456 INFO:tasks.rados.rados.0.smithi162.stdout:1203: writing smithi16234619-43 from 1934400 to 2563894 tid 2 2024-09-16T15:06:00.463 INFO:tasks.rados.rados.0.smithi162.stdout:1203: writing smithi16234619-43 from 3243966 to 3697606 tid 3 2024-09-16T15:06:00.463 INFO:tasks.rados.rados.0.smithi162.stdout:1204: read oid 32 snap -1 2024-09-16T15:06:00.464 INFO:tasks.rados.rados.0.smithi162.stdout:1204: expect deleted 2024-09-16T15:06:00.464 INFO:tasks.rados.rados.0.smithi162.stdout:1205: delete oid 22 current snap is 109 2024-09-16T15:06:00.465 INFO:tasks.rados.rados.0.smithi162.stdout:1204: done (4 left) 2024-09-16T15:06:00.465 INFO:tasks.rados.rados.0.smithi162.stdout:1205: done (3 left) 2024-09-16T15:06:00.465 INFO:tasks.rados.rados.0.smithi162.stdout:1206: write oid 2 current snap is 109 2024-09-16T15:06:00.465 INFO:tasks.rados.rados.0.smithi162.stdout:1206: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2024-09-16T15:06:00.472 INFO:tasks.rados.rados.0.smithi162.stdout:1206: writing smithi16234619-2 from 531220 to 1326940 tid 1 2024-09-16T15:06:00.480 INFO:tasks.rados.rados.0.smithi162.stdout:1206: writing smithi16234619-2 from 1935011 to 2451821 tid 2 2024-09-16T15:06:00.486 INFO:tasks.rados.rados.0.smithi162.stdout:1206: writing smithi16234619-2 from 3066106 to 3486526 tid 3 2024-09-16T15:06:00.486 INFO:tasks.rados.rados.0.smithi162.stdout:1207: snap_create 2024-09-16T15:06:00.487 INFO:tasks.rados.rados.0.smithi162.stdout:1201: finishing write tid 1 to smithi16234619-11 2024-09-16T15:06:00.489 INFO:tasks.rados.rados.0.smithi162.stdout:1202: finishing write tid 1 to smithi16234619-26 2024-09-16T15:06:00.490 INFO:tasks.rados.rados.0.smithi162.stdout:1202: finishing write tid 2 to smithi16234619-26 2024-09-16T15:06:00.492 INFO:tasks.rados.rados.0.smithi162.stdout:1201: finishing write tid 2 to smithi16234619-11 2024-09-16T15:06:00.493 INFO:tasks.rados.rados.0.smithi162.stdout:1203: finishing write tid 1 to smithi16234619-43 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: finishing write tid 3 to smithi16234619-11 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: finishing write tid 4 to smithi16234619-11 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: finishing write tid 5 to smithi16234619-11 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: oid 11 updating version 0 to 62 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: oid 11 version 62 is already newer than 61 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: oid 11 updating version 62 to 63 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: oid 11 updating version 63 to 64 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 64 (ObjNum 456 snap 109 seq_num 456) dirty exists 2024-09-16T15:06:00.496 INFO:tasks.rados.rados.0.smithi162.stdout:1201: left oid 11 (ObjNum 456 snap 109 seq_num 456) 2024-09-16T15:06:00.500 INFO:tasks.rados.rados.0.smithi162.stdout:1203: finishing write tid 2 to smithi16234619-43 2024-09-16T15:06:00.500 INFO:tasks.rados.rados.0.smithi162.stdout:1203: finishing write tid 3 to smithi16234619-43 2024-09-16T15:06:00.500 INFO:tasks.rados.rados.0.smithi162.stdout:1203: finishing write tid 4 to smithi16234619-43 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: finishing write tid 5 to smithi16234619-43 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: oid 43 updating version 0 to 118 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: oid 43 updating version 118 to 120 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: oid 43 version 120 is already newer than 119 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: oid 43 updating version 120 to 121 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 121 (ObjNum 458 snap 109 seq_num 458) dirty exists 2024-09-16T15:06:00.501 INFO:tasks.rados.rados.0.smithi162.stdout:1203: left oid 43 (ObjNum 458 snap 109 seq_num 458) 2024-09-16T15:06:00.518 INFO:tasks.rados.rados.0.smithi162.stdout:1202: finishing write tid 3 to smithi16234619-26 2024-09-16T15:06:00.528 INFO:tasks.rados.rados.0.smithi162.stdout:1202: finishing write tid 4 to smithi16234619-26 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1206: finishing write tid 1 to smithi16234619-2 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: finishing write tid 5 to smithi16234619-26 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: oid 26 updating version 0 to 450 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: oid 26 version 450 is already newer than 448 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: oid 26 version 450 is already newer than 447 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: oid 26 version 450 is already newer than 449 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 450 (ObjNum 457 snap 109 seq_num 457) dirty exists 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1202: left oid 26 (ObjNum 457 snap 109 seq_num 457) 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1206: finishing write tid 2 to smithi16234619-2 2024-09-16T15:06:00.529 INFO:tasks.rados.rados.0.smithi162.stdout:1206: finishing write tid 3 to smithi16234619-2 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: finishing write tid 4 to smithi16234619-2 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: finishing write tid 5 to smithi16234619-2 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: oid 2 updating version 0 to 454 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: oid 2 version 454 is already newer than 453 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: oid 2 version 454 is already newer than 452 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:1206: oid 2 version 454 is already newer than 451 2024-09-16T15:06:00.531 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 454 (ObjNum 460 snap 109 seq_num 460) dirty exists 2024-09-16T15:06:00.532 INFO:tasks.rados.rados.0.smithi162.stdout:1206: left oid 2 (ObjNum 460 snap 109 seq_num 460) 2024-09-16T15:06:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:1201: done (4 left) 2024-09-16T15:06:01.408 INFO:tasks.rados.rados.0.smithi162.stdout:1202: done (3 left) 2024-09-16T15:06:01.408 INFO:tasks.rados.rados.0.smithi162.stdout:1203: done (2 left) 2024-09-16T15:06:01.408 INFO:tasks.rados.rados.0.smithi162.stdout:1206: done (1 left) 2024-09-16T15:06:01.408 INFO:tasks.rados.rados.0.smithi162.stdout:1207: done (0 left) 2024-09-16T15:06:01.408 INFO:tasks.rados.rados.0.smithi162.stdout:1208: snap_remove snap 106 2024-09-16T15:06:02.416 INFO:tasks.rados.rados.0.smithi162.stdout:1208: done (0 left) 2024-09-16T15:06:02.416 INFO:tasks.rados.rados.0.smithi162.stdout:1209: write (excl) oid 9 current snap is 110 2024-09-16T15:06:02.416 INFO:tasks.rados.rados.0.smithi162.stdout:1209: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2024-09-16T15:06:02.421 INFO:tasks.rados.rados.0.smithi162.stdout:1209: writing smithi16234619-9 from 548027 to 1018996 tid 1 2024-09-16T15:06:02.430 INFO:tasks.rados.rados.0.smithi162.stdout:1209: writing smithi16234619-9 from 1593493 to 2370314 tid 2 2024-09-16T15:06:02.439 INFO:tasks.rados.rados.0.smithi162.stdout:1209: writing smithi16234619-9 from 3093529 to 3725221 tid 3 2024-09-16T15:06:02.439 INFO:tasks.rados.rados.0.smithi162.stdout:1210: read oid 2 snap -1 2024-09-16T15:06:02.439 INFO:tasks.rados.rados.0.smithi162.stdout:1210: expect (ObjNum 460 snap 109 seq_num 460) 2024-09-16T15:06:02.439 INFO:tasks.rados.rados.0.smithi162.stdout:1211: write (excl) oid 34 current snap is 110 2024-09-16T15:06:02.439 INFO:tasks.rados.rados.0.smithi162.stdout:1211: seq_num 462 ranges {564834=546218,1651975=520479,2720952=559317} 2024-09-16T15:06:02.446 INFO:tasks.rados.rados.0.smithi162.stdout:1211: writing smithi16234619-34 from 564834 to 1111052 tid 1 2024-09-16T15:06:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1211: writing smithi16234619-34 from 1651975 to 2172454 tid 2 2024-09-16T15:06:02.460 INFO:tasks.rados.rados.0.smithi162.stdout:1211: writing smithi16234619-34 from 2720952 to 3280269 tid 3 2024-09-16T15:06:02.460 INFO:tasks.rados.rados.0.smithi162.stdout:1212: snap_remove snap 107 2024-09-16T15:06:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:1212: done (3 left) 2024-09-16T15:06:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:1213: rollback oid 12 current snap is 110 2024-09-16T15:06:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 104 2024-09-16T15:06:03.419 INFO:tasks.rados.rados.0.smithi162.stdout:1210: expect (ObjNum 460 snap 109 seq_num 460) 2024-09-16T15:06:03.540 INFO:tasks.rados.rados.0.smithi162.stdout:1209: finishing write tid 1 to smithi16234619-9 2024-09-16T15:06:03.540 INFO:tasks.rados.rados.0.smithi162.stdout:1209: finishing write tid 2 to smithi16234619-9 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: finishing write tid 3 to smithi16234619-9 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: finishing write tid 4 to smithi16234619-9 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1211: finishing write tid 1 to smithi16234619-34 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1211: finishing write tid 2 to smithi16234619-34 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: finishing write tid 5 to smithi16234619-9 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: oid 9 updating version 0 to 425 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: oid 9 version 425 is already newer than 424 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: oid 9 updating version 425 to 426 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: oid 9 version 426 is already newer than 423 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 426 (ObjNum 461 snap 110 seq_num 461) dirty exists 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1209: left oid 9 (ObjNum 461 snap 110 seq_num 461) 2024-09-16T15:06:03.541 INFO:tasks.rados.rados.0.smithi162.stdout:1211: finishing write tid 3 to smithi16234619-34 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1209: done (3 left) 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1210: done (2 left) 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1214: rollback oid 37 current snap is 110 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 104 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: finishing write tid 4 to smithi16234619-34 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: finishing write tid 5 to smithi16234619-34 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: oid 34 updating version 0 to 380 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: oid 34 version 380 is already newer than 378 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: oid 34 version 380 is already newer than 377 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: oid 34 version 380 is already newer than 379 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 380 (ObjNum 462 snap 110 seq_num 462) dirty exists 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1211: left oid 34 (ObjNum 462 snap 110 seq_num 462) 2024-09-16T15:06:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:1213: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1213: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1213: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 226 (ObjNum 415 snap 98 seq_num 415) dirty exists 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1211: done (2 left) 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1213: done (1 left) 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1215: write oid 11 current snap is 110 2024-09-16T15:06:03.543 INFO:tasks.rados.rados.0.smithi162.stdout:1215: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2024-09-16T15:06:03.547 INFO:tasks.rados.rados.0.smithi162.stdout:1215: writing smithi16234619-11 from 581641 to 1203108 tid 1 2024-09-16T15:06:03.556 INFO:tasks.rados.rados.0.smithi162.stdout:1215: writing smithi16234619-11 from 1994104 to 2774594 tid 2 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:1215: writing smithi16234619-11 from 3432022 to 3781641 tid 3 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:1216: snap_create 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:1214: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:1214: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:1214: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:06:03.562 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 512 (ObjNum 405 snap 94 seq_num 405) dirty exists 2024-09-16T15:06:03.592 INFO:tasks.rados.rados.0.smithi162.stdout:1215: finishing write tid 1 to smithi16234619-11 2024-09-16T15:06:03.592 INFO:tasks.rados.rados.0.smithi162.stdout:1215: finishing write tid 2 to smithi16234619-11 2024-09-16T15:06:03.592 INFO:tasks.rados.rados.0.smithi162.stdout:1215: finishing write tid 3 to smithi16234619-11 2024-09-16T15:06:03.592 INFO:tasks.rados.rados.0.smithi162.stdout:1215: finishing write tid 4 to smithi16234619-11 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: finishing write tid 5 to smithi16234619-11 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: oid 11 updating version 0 to 73 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: oid 11 version 73 is already newer than 72 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: oid 11 version 73 is already newer than 71 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: oid 11 version 73 is already newer than 70 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 73 (ObjNum 463 snap 110 seq_num 463) dirty exists 2024-09-16T15:06:03.593 INFO:tasks.rados.rados.0.smithi162.stdout:1215: left oid 11 (ObjNum 463 snap 110 seq_num 463) 2024-09-16T15:06:04.466 INFO:tasks.rados.rados.0.smithi162.stdout:1214: done (2 left) 2024-09-16T15:06:04.466 INFO:tasks.rados.rados.0.smithi162.stdout:1215: done (1 left) 2024-09-16T15:06:04.467 INFO:tasks.rados.rados.0.smithi162.stdout:1216: done (0 left) 2024-09-16T15:06:04.467 INFO:tasks.rados.rados.0.smithi162.stdout:1217: rollback oid 6 current snap is 111 2024-09-16T15:06:04.467 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 105 2024-09-16T15:06:04.467 INFO:tasks.rados.rados.0.smithi162.stdout:1218: snap_create 2024-09-16T15:06:04.468 INFO:tasks.rados.rados.0.smithi162.stdout:1217: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:06:04.468 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:06:06.178 INFO:tasks.rados.rados.0.smithi162.stdout:1217: done (1 left) 2024-09-16T15:06:06.178 INFO:tasks.rados.rados.0.smithi162.stdout:1218: done (0 left) 2024-09-16T15:06:06.178 INFO:tasks.rados.rados.0.smithi162.stdout:1219: snap_create 2024-09-16T15:06:07.180 INFO:tasks.rados.rados.0.smithi162.stdout:1219: done (0 left) 2024-09-16T15:06:07.180 INFO:tasks.rados.rados.0.smithi162.stdout:1220: write (excl) oid 30 current snap is 113 2024-09-16T15:06:07.180 INFO:tasks.rados.rados.0.smithi162.stdout:1220: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2024-09-16T15:06:07.185 INFO:tasks.rados.rados.0.smithi162.stdout:1220: writing smithi16234619-30 from 598448 to 1011517 tid 1 2024-09-16T15:06:07.192 INFO:tasks.rados.rados.0.smithi162.stdout:1220: writing smithi16234619-30 from 1768939 to 2293087 tid 2 2024-09-16T15:06:07.198 INFO:tasks.rados.rados.0.smithi162.stdout:1220: writing smithi16234619-30 from 2775798 to 3357659 tid 3 2024-09-16T15:06:07.200 INFO:tasks.rados.rados.0.smithi162.stdout:1220: writing smithi16234619-30 from 3781488 to 3798448 tid 4 2024-09-16T15:06:07.201 INFO:tasks.rados.rados.0.smithi162.stdout:1221: rollback oid 3 current snap is 113 2024-09-16T15:06:07.201 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 108 2024-09-16T15:06:07.201 INFO:tasks.rados.rados.0.smithi162.stdout:1222: read oid 40 snap -1 2024-09-16T15:06:07.201 INFO:tasks.rados.rados.0.smithi162.stdout:1222: expect deleted 2024-09-16T15:06:07.201 INFO:tasks.rados.rados.0.smithi162.stdout:1223: snap_remove snap 109 2024-09-16T15:06:08.451 INFO:tasks.rados.rados.0.smithi162.stdout:1223: done (3 left) 2024-09-16T15:06:08.451 INFO:tasks.rados.rados.0.smithi162.stdout:1224: write (excl) oid 22 current snap is 113 2024-09-16T15:06:08.451 INFO:tasks.rados.rados.0.smithi162.stdout:1224: seq_num 465 ranges {615255=488318,1827421=784159,3203221=612034} 2024-09-16T15:06:08.457 INFO:tasks.rados.rados.0.smithi162.stdout:1224: writing smithi16234619-22 from 615255 to 1103573 tid 1 2024-09-16T15:06:08.465 INFO:tasks.rados.rados.0.smithi162.stdout:1224: writing smithi16234619-22 from 1827421 to 2611580 tid 2 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1224: writing smithi16234619-22 from 3203221 to 3815255 tid 3 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1225: snap_create 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1221: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 0 (ObjNum 33 snap 0 seq_num 851383544) dirty dne 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 1 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 2 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 3 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 4 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 5 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: finishing write tid 6 to smithi16234619-30 2024-09-16T15:06:08.472 INFO:tasks.rados.rados.0.smithi162.stdout:1220: oid 30 updating version 0 to 284 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:1220: oid 30 version 284 is already newer than 281 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:1220: oid 30 version 284 is already newer than 282 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:1220: oid 30 version 284 is already newer than 280 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:1220: oid 30 version 284 is already newer than 283 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 284 (ObjNum 464 snap 113 seq_num 464) dirty exists 2024-09-16T15:06:08.473 INFO:tasks.rados.rados.0.smithi162.stdout:1220: left oid 30 (ObjNum 464 snap 113 seq_num 464) 2024-09-16T15:06:08.501 INFO:tasks.rados.rados.0.smithi162.stdout:1224: finishing write tid 1 to smithi16234619-22 2024-09-16T15:06:08.502 INFO:tasks.rados.rados.0.smithi162.stdout:1224: finishing write tid 2 to smithi16234619-22 2024-09-16T15:06:08.503 INFO:tasks.rados.rados.0.smithi162.stdout:1224: finishing write tid 3 to smithi16234619-22 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: finishing write tid 4 to smithi16234619-22 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: finishing write tid 5 to smithi16234619-22 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: oid 22 updating version 0 to 276 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: oid 22 updating version 276 to 278 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: oid 22 version 278 is already newer than 275 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: oid 22 version 278 is already newer than 277 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 278 (ObjNum 465 snap 113 seq_num 465) dirty exists 2024-09-16T15:06:08.505 INFO:tasks.rados.rados.0.smithi162.stdout:1224: left oid 22 (ObjNum 465 snap 113 seq_num 465) 2024-09-16T15:06:09.464 INFO:tasks.rados.rados.0.smithi162.stdout:1220: done (4 left) 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1221: done (3 left) 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1222: done (2 left) 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1224: done (1 left) 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1225: done (0 left) 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1226: rollback oid 26 current snap is 114 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 111 2024-09-16T15:06:09.465 INFO:tasks.rados.rados.0.smithi162.stdout:1227: snap_remove snap 110 2024-09-16T15:06:10.408 INFO:tasks.rados.rados.0.smithi162.stdout:1227: done (1 left) 2024-09-16T15:06:10.408 INFO:tasks.rados.rados.0.smithi162.stdout:1228: snap_create 2024-09-16T15:06:10.408 INFO:tasks.rados.rados.0.smithi162.stdout:1226: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:06:10.408 INFO:tasks.rados.rados.0.smithi162.stdout:1226: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:06:10.408 INFO:tasks.rados.rados.0.smithi162.stdout:1226: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:06:10.409 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 468 (ObjNum 457 snap 109 seq_num 457) dirty exists 2024-09-16T15:06:11.409 INFO:tasks.rados.rados.0.smithi162.stdout:1226: done (1 left) 2024-09-16T15:06:11.410 INFO:tasks.rados.rados.0.smithi162.stdout:1228: done (0 left) 2024-09-16T15:06:11.410 INFO:tasks.rados.rados.0.smithi162.stdout:1229: read oid 41 snap -1 2024-09-16T15:06:11.410 INFO:tasks.rados.rados.0.smithi162.stdout:1229: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:06:11.410 INFO:tasks.rados.rados.0.smithi162.stdout:1230: write (excl) oid 49 current snap is 115 2024-09-16T15:06:11.410 INFO:tasks.rados.rados.0.smithi162.stdout:1230: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2024-09-16T15:06:11.416 INFO:tasks.rados.rados.0.smithi162.stdout:1230: writing smithi16234619-49 from 632062 to 1195629 tid 1 2024-09-16T15:06:11.423 INFO:tasks.rados.rados.0.smithi162.stdout:1230: writing smithi16234619-49 from 1885903 to 2413720 tid 2 2024-09-16T15:06:11.429 INFO:tasks.rados.rados.0.smithi162.stdout:1230: writing smithi16234619-49 from 2830644 to 3551402 tid 3 2024-09-16T15:06:11.429 INFO:tasks.rados.rados.0.smithi162.stdout:1231: copy_from oid 15 from oid 20 current snap is 115 2024-09-16T15:06:11.429 INFO:tasks.rados.rados.0.smithi162.stdout:1229: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: finishing write tid 1 to smithi16234619-49 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: finishing write tid 2 to smithi16234619-49 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: finishing write tid 3 to smithi16234619-49 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: finishing write tid 4 to smithi16234619-49 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: finishing write tid 5 to smithi16234619-49 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: oid 49 updating version 0 to 281 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: oid 49 version 281 is already newer than 279 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: oid 49 version 281 is already newer than 280 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: oid 49 updating version 281 to 282 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 282 (ObjNum 466 snap 115 seq_num 466) dirty exists 2024-09-16T15:06:11.599 INFO:tasks.rados.rados.0.smithi162.stdout:1230: left oid 49 (ObjNum 466 snap 115 seq_num 466) 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1231: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1231: finishing copy_from to smithi16234619-15 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 178 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1229: done (2 left) 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1230: done (1 left) 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1231: done (0 left) 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1232: rollback oid 39 current snap is 115 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 108 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1233: rollback oid 12 current snap is 115 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 108 2024-09-16T15:06:11.600 INFO:tasks.rados.rados.0.smithi162.stdout:1234: snap_remove snap 104 2024-09-16T15:06:12.476 INFO:tasks.rados.rados.0.smithi162.stdout:1234: done (2 left) 2024-09-16T15:06:12.476 INFO:tasks.rados.rados.0.smithi162.stdout:1235: delete oid 33 current snap is 115 2024-09-16T15:06:12.476 INFO:tasks.rados.rados.0.smithi162.stdout:1232: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:06:12.476 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 506 (ObjNum 129 snap 0 seq_num 864863056) dirty dne 2024-09-16T15:06:12.477 INFO:tasks.rados.rados.0.smithi162.stdout:1233: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:06:12.477 INFO:tasks.rados.rados.0.smithi162.stdout:1233: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:06:12.477 INFO:tasks.rados.rados.0.smithi162.stdout:1233: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:06:12.477 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 232 (ObjNum 415 snap 98 seq_num 415) dirty exists 2024-09-16T15:06:12.484 INFO:tasks.rados.rados.0.smithi162.stdout:1232: done (2 left) 2024-09-16T15:06:12.484 INFO:tasks.rados.rados.0.smithi162.stdout:1233: done (1 left) 2024-09-16T15:06:12.484 INFO:tasks.rados.rados.0.smithi162.stdout:1235: done (0 left) 2024-09-16T15:06:12.484 INFO:tasks.rados.rados.0.smithi162.stdout:1236: snap_create 2024-09-16T15:06:14.001 INFO:tasks.rados.rados.0.smithi162.stdout:1236: done (0 left) 2024-09-16T15:06:14.001 INFO:tasks.rados.rados.0.smithi162.stdout:1237: write (excl) oid 11 current snap is 116 2024-09-16T15:06:14.001 INFO:tasks.rados.rados.0.smithi162.stdout:1237: seq_num 468 ranges {665676=714065,1886514=531486,3169137=696539} 2024-09-16T15:06:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:1237: writing smithi16234619-11 from 665676 to 1379741 tid 1 2024-09-16T15:06:14.013 INFO:tasks.rados.rados.0.smithi162.stdout:1237: writing smithi16234619-11 from 1886514 to 2418000 tid 2 2024-09-16T15:06:14.022 INFO:tasks.rados.rados.0.smithi162.stdout:1237: writing smithi16234619-11 from 3169137 to 3865676 tid 3 2024-09-16T15:06:14.022 INFO:tasks.rados.rados.0.smithi162.stdout:1238: read oid 33 snap -1 2024-09-16T15:06:14.022 INFO:tasks.rados.rados.0.smithi162.stdout:1238: expect deleted 2024-09-16T15:06:14.022 INFO:tasks.rados.rados.0.smithi162.stdout:1239: write (excl) oid 1 current snap is 116 2024-09-16T15:06:14.022 INFO:tasks.rados.rados.0.smithi162.stdout:1239: seq_num 469 ranges {682483=789314,1944996=791497,3196560=554574} 2024-09-16T15:06:14.029 INFO:tasks.rados.rados.0.smithi162.stdout:1239: writing smithi16234619-1 from 682483 to 1471797 tid 1 2024-09-16T15:06:14.036 INFO:tasks.rados.rados.0.smithi162.stdout:1239: writing smithi16234619-1 from 1944996 to 2736493 tid 2 2024-09-16T15:06:14.042 INFO:tasks.rados.rados.0.smithi162.stdout:1239: writing smithi16234619-1 from 3196560 to 3751134 tid 3 2024-09-16T15:06:14.042 INFO:tasks.rados.rados.0.smithi162.stdout:1240: snap_remove snap 112 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1240: done (3 left) 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1241: read oid 20 snap -1 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1241: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1242: snap_create 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: finishing write tid 1 to smithi16234619-11 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: finishing write tid 2 to smithi16234619-11 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: finishing write tid 3 to smithi16234619-11 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: finishing write tid 4 to smithi16234619-11 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: finishing write tid 5 to smithi16234619-11 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: oid 11 updating version 0 to 80 2024-09-16T15:06:15.009 INFO:tasks.rados.rados.0.smithi162.stdout:1237: oid 11 updating version 80 to 82 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1237: oid 11 version 82 is already newer than 81 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1237: oid 11 version 82 is already newer than 79 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 82 (ObjNum 468 snap 116 seq_num 468) dirty exists 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1237: left oid 11 (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: finishing write tid 1 to smithi16234619-1 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: finishing write tid 2 to smithi16234619-1 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: finishing write tid 3 to smithi16234619-1 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: finishing write tid 4 to smithi16234619-1 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: finishing write tid 5 to smithi16234619-1 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: oid 1 updating version 0 to 440 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: oid 1 updating version 440 to 441 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: oid 1 version 441 is already newer than 439 2024-09-16T15:06:15.010 INFO:tasks.rados.rados.0.smithi162.stdout:1239: oid 1 version 441 is already newer than 438 2024-09-16T15:06:15.011 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 441 (ObjNum 469 snap 116 seq_num 469) dirty exists 2024-09-16T15:06:15.011 INFO:tasks.rados.rados.0.smithi162.stdout:1239: left oid 1 (ObjNum 469 snap 116 seq_num 469) 2024-09-16T15:06:15.016 INFO:tasks.rados.rados.0.smithi162.stdout:1241: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1237: done (4 left) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1238: done (3 left) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1239: done (2 left) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1241: done (1 left) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1242: done (0 left) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1243: read oid 11 snap -1 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1243: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:06:15.408 INFO:tasks.rados.rados.0.smithi162.stdout:1244: snap_create 2024-09-16T15:06:15.423 INFO:tasks.rados.rados.0.smithi162.stdout:1243: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:06:16.410 INFO:tasks.rados.rados.0.smithi162.stdout:1243: done (1 left) 2024-09-16T15:06:16.410 INFO:tasks.rados.rados.0.smithi162.stdout:1244: done (0 left) 2024-09-16T15:06:16.410 INFO:tasks.rados.rados.0.smithi162.stdout:1245: write oid 37 current snap is 118 2024-09-16T15:06:16.410 INFO:tasks.rados.rados.0.smithi162.stdout:1245: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2024-09-16T15:06:16.415 INFO:tasks.rados.rados.0.smithi162.stdout:1245: writing smithi16234619-37 from 699290 to 1163853 tid 1 2024-09-16T15:06:16.421 INFO:tasks.rados.rados.0.smithi162.stdout:1245: writing smithi16234619-37 from 1603478 to 2138633 tid 2 2024-09-16T15:06:16.429 INFO:tasks.rados.rados.0.smithi162.stdout:1245: writing smithi16234619-37 from 2823983 to 3589829 tid 3 2024-09-16T15:06:16.429 INFO:tasks.rados.rados.0.smithi162.stdout:1246: delete oid 32 current snap is 118 2024-09-16T15:06:16.432 INFO:tasks.rados.rados.0.smithi162.stdout:1246: done (1 left) 2024-09-16T15:06:16.432 INFO:tasks.rados.rados.0.smithi162.stdout:1247: write oid 34 current snap is 118 2024-09-16T15:06:16.433 INFO:tasks.rados.rados.0.smithi162.stdout:1247: seq_num 472 ranges {732904=731414,2120442=538824,3278829=504743} 2024-09-16T15:06:16.442 INFO:tasks.rados.rados.0.smithi162.stdout:1247: writing smithi16234619-34 from 732904 to 1464318 tid 1 2024-09-16T15:06:16.448 INFO:tasks.rados.rados.0.smithi162.stdout:1247: writing smithi16234619-34 from 2120442 to 2659266 tid 2 2024-09-16T15:06:16.455 INFO:tasks.rados.rados.0.smithi162.stdout:1247: writing smithi16234619-34 from 3278829 to 3783572 tid 3 2024-09-16T15:06:16.455 INFO:tasks.rados.rados.0.smithi162.stdout:1248: read oid 8 snap -1 2024-09-16T15:06:16.456 INFO:tasks.rados.rados.0.smithi162.stdout:1248: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:06:16.456 INFO:tasks.rados.rados.0.smithi162.stdout:1249: copy_from oid 50 from oid 15 current snap is 118 2024-09-16T15:06:16.456 INFO:tasks.rados.rados.0.smithi162.stdout:1250: snap_remove snap 105 2024-09-16T15:06:17.417 INFO:tasks.rados.rados.0.smithi162.stdout:1250: done (4 left) 2024-09-16T15:06:17.417 INFO:tasks.rados.rados.0.smithi162.stdout:1251: write (excl) oid 26 current snap is 118 2024-09-16T15:06:17.417 INFO:tasks.rados.rados.0.smithi162.stdout:1251: seq_num 473 ranges {749711=406663,1778924=682482,3189899=716015} 2024-09-16T15:06:17.424 INFO:tasks.rados.rados.0.smithi162.stdout:1251: writing smithi16234619-26 from 749711 to 1156374 tid 1 2024-09-16T15:06:17.431 INFO:tasks.rados.rados.0.smithi162.stdout:1251: writing smithi16234619-26 from 1778924 to 2461406 tid 2 2024-09-16T15:06:17.440 INFO:tasks.rados.rados.0.smithi162.stdout:1251: writing smithi16234619-26 from 3189899 to 3905914 tid 3 2024-09-16T15:06:17.440 INFO:tasks.rados.rados.0.smithi162.stdout:1252: read oid 12 snap 114 2024-09-16T15:06:17.440 INFO:tasks.rados.rados.0.smithi162.stdout:1252: expect (ObjNum 415 snap 98 seq_num 415) 2024-09-16T15:06:17.440 INFO:tasks.rados.rados.0.smithi162.stdout:1253: delete oid 38 current snap is 118 2024-09-16T15:06:17.440 INFO:tasks.rados.rados.0.smithi162.stdout:1248: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:06:17.517 INFO:tasks.rados.rados.0.smithi162.stdout:1245: finishing write tid 1 to smithi16234619-37 2024-09-16T15:06:17.517 INFO:tasks.rados.rados.0.smithi162.stdout:1245: finishing write tid 2 to smithi16234619-37 2024-09-16T15:06:17.517 INFO:tasks.rados.rados.0.smithi162.stdout:1245: finishing write tid 3 to smithi16234619-37 2024-09-16T15:06:17.517 INFO:tasks.rados.rados.0.smithi162.stdout:1245: finishing write tid 4 to smithi16234619-37 2024-09-16T15:06:17.517 INFO:tasks.rados.rados.0.smithi162.stdout:1245: finishing write tid 5 to smithi16234619-37 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: oid 37 updating version 0 to 523 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: oid 37 version 523 is already newer than 522 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: oid 37 updating version 523 to 524 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: oid 37 version 524 is already newer than 521 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 524 (ObjNum 470 snap 118 seq_num 470) dirty exists 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: left oid 37 (ObjNum 470 snap 118 seq_num 470) 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1247: finishing write tid 1 to smithi16234619-34 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1247: finishing write tid 2 to smithi16234619-34 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1245: done (6 left) 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1248: done (5 left) 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1253: done (4 left) 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1254: read oid 17 snap -1 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1254: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:06:17.518 INFO:tasks.rados.rados.0.smithi162.stdout:1255: read oid 4 snap 108 2024-09-16T15:06:17.519 INFO:tasks.rados.rados.0.smithi162.stdout:1255: expect (ObjNum 353 snap 81 seq_num 353) 2024-09-16T15:06:17.519 INFO:tasks.rados.rados.0.smithi162.stdout:1256: snap_remove snap 111 2024-09-16T15:06:18.427 INFO:tasks.rados.rados.0.smithi162.stdout:1256: done (6 left) 2024-09-16T15:06:18.427 INFO:tasks.rados.rados.0.smithi162.stdout:1257: rollback oid 18 current snap is 118 2024-09-16T15:06:18.427 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 115 2024-09-16T15:06:18.427 INFO:tasks.rados.rados.0.smithi162.stdout:1247: finishing write tid 3 to smithi16234619-34 2024-09-16T15:06:18.427 INFO:tasks.rados.rados.0.smithi162.stdout:1249: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: finishing write tid 4 to smithi16234619-34 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: finishing write tid 5 to smithi16234619-34 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: oid 34 updating version 0 to 395 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: oid 34 version 395 is already newer than 392 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: oid 34 version 395 is already newer than 394 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: oid 34 version 395 is already newer than 393 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 395 (ObjNum 472 snap 118 seq_num 472) dirty exists 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1247: left oid 34 (ObjNum 472 snap 118 seq_num 472) 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1249: finishing copy_from to smithi16234619-50 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 178 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:06:18.428 INFO:tasks.rados.rados.0.smithi162.stdout:1252: expect (ObjNum 415 snap 98 seq_num 415) 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: finishing write tid 1 to smithi16234619-26 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: finishing write tid 2 to smithi16234619-26 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: finishing write tid 3 to smithi16234619-26 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: finishing write tid 4 to smithi16234619-26 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: finishing write tid 5 to smithi16234619-26 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: oid 26 updating version 0 to 483 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: oid 26 version 483 is already newer than 482 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: oid 26 updating version 483 to 485 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:1251: oid 26 version 485 is already newer than 484 2024-09-16T15:06:18.561 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 485 (ObjNum 473 snap 118 seq_num 473) dirty exists 2024-09-16T15:06:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:1251: left oid 26 (ObjNum 473 snap 118 seq_num 473) 2024-09-16T15:06:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:1254: expect (ObjNum 281 snap 59 seq_num 281) 2024-09-16T15:06:18.649 INFO:tasks.rados.rados.0.smithi162.stdout:1255: expect (ObjNum 353 snap 81 seq_num 353) 2024-09-16T15:06:18.809 INFO:tasks.rados.rados.0.smithi162.stdout:1257: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:06:18.809 INFO:tasks.rados.rados.0.smithi162.stdout:1257: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:06:18.809 INFO:tasks.rados.rados.0.smithi162.stdout:1247: done (6 left) 2024-09-16T15:06:18.809 INFO:tasks.rados.rados.0.smithi162.stdout:1249: done (5 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1251: done (4 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1252: done (3 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1254: done (2 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1255: done (1 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1258: rollback oid 39 current snap is 118 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 116 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1257: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 267 (ObjNum 439 snap 104 seq_num 439) dirty exists 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1257: done (1 left) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1259: read oid 20 snap -1 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1259: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:18.810 INFO:tasks.rados.rados.0.smithi162.stdout:1260: snap_remove snap 115 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1260: done (2 left) 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1261: read oid 27 snap 116 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1261: expect deleted 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1262: read oid 7 snap -1 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1262: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1263: read oid 23 snap -1 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1263: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1264: read oid 42 snap -1 2024-09-16T15:06:19.991 INFO:tasks.rados.rados.0.smithi162.stdout:1264: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:19.992 INFO:tasks.rados.rados.0.smithi162.stdout:1265: delete oid 4 current snap is 118 2024-09-16T15:06:19.992 INFO:tasks.rados.rados.0.smithi162.stdout:1259: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:20.121 INFO:tasks.rados.rados.0.smithi162.stdout:1258: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:06:20.121 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 0 (ObjNum 385 snap 0 seq_num 2027920976) dirty dne 2024-09-16T15:06:20.121 INFO:tasks.rados.rados.0.smithi162.stdout:1264: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:20.210 INFO:tasks.rados.rados.0.smithi162.stdout:1262: expect (ObjNum 232 snap 46 seq_num 232) 2024-09-16T15:06:20.456 INFO:tasks.rados.rados.0.smithi162.stdout:1263: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1258: done (6 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1259: done (5 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1261: done (4 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1262: done (3 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1263: done (2 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1264: done (1 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1265: done (0 left) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1266: read oid 41 snap -1 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1266: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:06:20.608 INFO:tasks.rados.rados.0.smithi162.stdout:1267: delete oid 49 current snap is 118 2024-09-16T15:06:20.613 INFO:tasks.rados.rados.0.smithi162.stdout:1266: expect (ObjNum 376 snap 87 seq_num 376) 2024-09-16T15:06:20.781 INFO:tasks.rados.rados.0.smithi162.stdout:1266: done (1 left) 2024-09-16T15:06:20.782 INFO:tasks.rados.rados.0.smithi162.stdout:1267: done (0 left) 2024-09-16T15:06:20.782 INFO:tasks.rados.rados.0.smithi162.stdout:1268: snap_remove snap 108 2024-09-16T15:06:22.174 INFO:tasks.rados.rados.0.smithi162.stdout:1268: done (0 left) 2024-09-16T15:06:22.174 INFO:tasks.rados.rados.0.smithi162.stdout:1269: delete oid 7 current snap is 118 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1269: done (0 left) 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1270: rollback oid 18 current snap is 118 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 114 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1271: read oid 42 snap -1 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1271: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1272: read oid 23 snap -1 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1272: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1273: rollback oid 3 current snap is 118 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 117 2024-09-16T15:06:22.185 INFO:tasks.rados.rados.0.smithi162.stdout:1274: snap_remove snap 116 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1274: done (4 left) 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1275: read oid 21 snap -1 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1275: expect deleted 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1276: delete oid 40 current snap is 118 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1270: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:06:23.181 INFO:tasks.rados.rados.0.smithi162.stdout:1271: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:23.274 INFO:tasks.rados.rados.0.smithi162.stdout:1273: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:06:23.274 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 0 (ObjNum 129 snap 0 seq_num 866377216) dirty dne 2024-09-16T15:06:23.274 INFO:tasks.rados.rados.0.smithi162.stdout:1270: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:06:23.274 INFO:tasks.rados.rados.0.smithi162.stdout:1270: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:06:23.274 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 276 (ObjNum 439 snap 104 seq_num 439) dirty exists 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1270: done (5 left) 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1271: done (4 left) 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1273: done (3 left) 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1276: done (2 left) 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1277: delete oid 18 current snap is 118 2024-09-16T15:06:23.275 INFO:tasks.rados.rados.0.smithi162.stdout:1272: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:23.427 INFO:tasks.rados.rados.0.smithi162.stdout:1272: done (2 left) 2024-09-16T15:06:23.427 INFO:tasks.rados.rados.0.smithi162.stdout:1275: done (1 left) 2024-09-16T15:06:23.427 INFO:tasks.rados.rados.0.smithi162.stdout:1277: done (0 left) 2024-09-16T15:06:23.428 INFO:tasks.rados.rados.0.smithi162.stdout:1278: delete oid 1 current snap is 118 2024-09-16T15:06:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:1278: done (0 left) 2024-09-16T15:06:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:1279: write oid 12 current snap is 118 2024-09-16T15:06:23.430 INFO:tasks.rados.rados.0.smithi162.stdout:1279: seq_num 481 ranges {484167=725008,1730427=353740} 2024-09-16T15:06:23.436 INFO:tasks.rados.rados.0.smithi162.stdout:1279: writing smithi16234619-12 from 484167 to 1209175 tid 1 2024-09-16T15:06:23.441 INFO:tasks.rados.rados.0.smithi162.stdout:1279: writing smithi16234619-12 from 1730427 to 2084167 tid 2 2024-09-16T15:06:23.441 INFO:tasks.rados.rados.0.smithi162.stdout:1280: snap_remove snap 114 2024-09-16T15:06:24.188 INFO:tasks.rados.rados.0.smithi162.stdout:1280: done (1 left) 2024-09-16T15:06:24.188 INFO:tasks.rados.rados.0.smithi162.stdout:1281: read oid 36 snap -1 2024-09-16T15:06:24.188 INFO:tasks.rados.rados.0.smithi162.stdout:1281: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:06:24.188 INFO:tasks.rados.rados.0.smithi162.stdout:1282: snap_remove snap 117 2024-09-16T15:06:25.897 INFO:tasks.rados.rados.0.smithi162.stdout:1282: done (2 left) 2024-09-16T15:06:25.897 INFO:tasks.rados.rados.0.smithi162.stdout:1283: rollback oid 21 current snap is 118 2024-09-16T15:06:25.897 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 113 2024-09-16T15:06:25.897 INFO:tasks.rados.rados.0.smithi162.stdout:1279: finishing write tid 1 to smithi16234619-12 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: finishing write tid 2 to smithi16234619-12 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: finishing write tid 3 to smithi16234619-12 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: finishing write tid 4 to smithi16234619-12 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: oid 12 updating version 0 to 248 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: oid 12 version 248 is already newer than 246 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: oid 12 version 248 is already newer than 247 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 248 (ObjNum 481 snap 118 seq_num 481) dirty exists 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1279: left oid 12 (ObjNum 481 snap 118 seq_num 481) 2024-09-16T15:06:25.898 INFO:tasks.rados.rados.0.smithi162.stdout:1281: expect (ObjNum 127 snap 24 seq_num 127) 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:1283: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 0 (ObjNum 4385 snap 0 seq_num 865337568) dirty dne 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:1279: done (2 left) 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:1281: done (1 left) 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:1283: done (0 left) 2024-09-16T15:06:25.984 INFO:tasks.rados.rados.0.smithi162.stdout:1284: snap_create 2024-09-16T15:06:26.911 INFO:tasks.rados.rados.0.smithi162.stdout:1284: done (0 left) 2024-09-16T15:06:26.912 INFO:tasks.rados.rados.0.smithi162.stdout:1285: read oid 7 snap -1 2024-09-16T15:06:26.912 INFO:tasks.rados.rados.0.smithi162.stdout:1285: expect deleted 2024-09-16T15:06:26.912 INFO:tasks.rados.rados.0.smithi162.stdout:1286: write oid 47 current snap is 119 2024-09-16T15:06:26.912 INFO:tasks.rados.rados.0.smithi162.stdout:1286: seq_num 482 ranges {500974=400257,1388909=557169} 2024-09-16T15:06:26.916 INFO:tasks.rados.rados.0.smithi162.stdout:1286: writing smithi16234619-47 from 500974 to 901231 tid 1 2024-09-16T15:06:26.922 INFO:tasks.rados.rados.0.smithi162.stdout:1286: writing smithi16234619-47 from 1388909 to 1946078 tid 2 2024-09-16T15:06:26.922 INFO:tasks.rados.rados.0.smithi162.stdout:1287: read oid 5 snap -1 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1287: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1288: read oid 46 snap -1 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1288: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1289: rollback oid 1 current snap is 119 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 113 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1285: done (4 left) 2024-09-16T15:06:26.923 INFO:tasks.rados.rados.0.smithi162.stdout:1290: snap_create 2024-09-16T15:06:26.930 INFO:tasks.rados.rados.0.smithi162.stdout:1289: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:06:26.930 INFO:tasks.rados.rados.0.smithi162.stdout:1289: finishing rollback tid 2 to smithi16234619-1 2024-09-16T15:06:26.930 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 473 (ObjNum 424 snap 99 seq_num 424) dirty exists 2024-09-16T15:06:26.931 INFO:tasks.rados.rados.0.smithi162.stdout:1287: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:06:27.042 INFO:tasks.rados.rados.0.smithi162.stdout:1288: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:27.140 INFO:tasks.rados.rados.0.smithi162.stdout:1286: finishing write tid 1 to smithi16234619-47 2024-09-16T15:06:27.140 INFO:tasks.rados.rados.0.smithi162.stdout:1286: finishing write tid 2 to smithi16234619-47 2024-09-16T15:06:27.140 INFO:tasks.rados.rados.0.smithi162.stdout:1286: finishing write tid 3 to smithi16234619-47 2024-09-16T15:06:27.140 INFO:tasks.rados.rados.0.smithi162.stdout:1286: finishing write tid 4 to smithi16234619-47 2024-09-16T15:06:27.141 INFO:tasks.rados.rados.0.smithi162.stdout:1286: oid 47 updating version 0 to 134 2024-09-16T15:06:27.141 INFO:tasks.rados.rados.0.smithi162.stdout:1286: oid 47 version 134 is already newer than 132 2024-09-16T15:06:27.141 INFO:tasks.rados.rados.0.smithi162.stdout:1286: oid 47 version 134 is already newer than 133 2024-09-16T15:06:27.141 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 134 (ObjNum 482 snap 119 seq_num 482) dirty exists 2024-09-16T15:06:27.141 INFO:tasks.rados.rados.0.smithi162.stdout:1286: left oid 47 (ObjNum 482 snap 119 seq_num 482) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1286: done (4 left) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1287: done (3 left) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1288: done (2 left) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1289: done (1 left) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1290: done (0 left) 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1291: copy_from oid 13 from oid 45 current snap is 120 2024-09-16T15:06:27.937 INFO:tasks.rados.rados.0.smithi162.stdout:1292: rollback oid 21 current snap is 120 2024-09-16T15:06:27.938 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 113 2024-09-16T15:06:27.938 INFO:tasks.rados.rados.0.smithi162.stdout:1293: read oid 26 snap -1 2024-09-16T15:06:27.938 INFO:tasks.rados.rados.0.smithi162.stdout:1293: expect (ObjNum 473 snap 118 seq_num 473) 2024-09-16T15:06:27.938 INFO:tasks.rados.rados.0.smithi162.stdout:1294: write (excl) oid 41 current snap is 120 2024-09-16T15:06:27.938 INFO:tasks.rados.rados.0.smithi162.stdout:1294: seq_num 483 ranges {517781=475506,1731038=386743} 2024-09-16T15:06:27.942 INFO:tasks.rados.rados.0.smithi162.stdout:1294: writing smithi16234619-41 from 517781 to 993287 tid 1 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:1294: writing smithi16234619-41 from 1731038 to 2117781 tid 2 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:1295: read oid 32 snap -1 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:1295: expect deleted 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:1296: rollback oid 34 current snap is 120 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 113 2024-09-16T15:06:27.949 INFO:tasks.rados.rados.0.smithi162.stdout:1293: expect (ObjNum 473 snap 118 seq_num 473) 2024-09-16T15:06:28.183 INFO:tasks.rados.rados.0.smithi162.stdout:1292: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:06:28.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:06:28.183 INFO:tasks.rados.rados.0.smithi162.stdout:1292: done (5 left) 2024-09-16T15:06:28.183 INFO:tasks.rados.rados.0.smithi162.stdout:1293: done (4 left) 2024-09-16T15:06:28.183 INFO:tasks.rados.rados.0.smithi162.stdout:1297: delete oid 1 current snap is 120 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1296: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1296: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1296: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 431 (ObjNum 462 snap 110 seq_num 462) dirty exists 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: finishing write tid 1 to smithi16234619-41 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: finishing write tid 2 to smithi16234619-41 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: finishing write tid 3 to smithi16234619-41 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: finishing write tid 4 to smithi16234619-41 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: oid 41 updating version 0 to 286 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: oid 41 version 286 is already newer than 284 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: oid 41 version 286 is already newer than 285 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 286 (ObjNum 483 snap 120 seq_num 483) dirty exists 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1294: left oid 41 (ObjNum 483 snap 120 seq_num 483) 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1291: finishing copy_from racing read to smithi16234619-13 2024-09-16T15:06:28.184 INFO:tasks.rados.rados.0.smithi162.stdout:1291: finishing copy_from to smithi16234619-13 2024-09-16T15:06:28.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 443 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1291: done (4 left) 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1294: done (3 left) 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1295: done (2 left) 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1296: done (1 left) 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1297: done (0 left) 2024-09-16T15:06:28.190 INFO:tasks.rados.rados.0.smithi162.stdout:1298: delete oid 22 current snap is 120 2024-09-16T15:06:28.193 INFO:tasks.rados.rados.0.smithi162.stdout:1298: done (0 left) 2024-09-16T15:06:28.193 INFO:tasks.rados.rados.0.smithi162.stdout:1299: snap_remove snap 113 2024-09-16T15:06:29.190 INFO:tasks.rados.rados.0.smithi162.stdout:1299: done (0 left) 2024-09-16T15:06:29.190 INFO:tasks.rados.rados.0.smithi162.stdout:1300: copy_from oid 20 from oid 14 current snap is 120 2024-09-16T15:06:29.190 INFO:tasks.rados.rados.0.smithi162.stdout:1301: delete oid 36 current snap is 120 2024-09-16T15:06:29.194 INFO:tasks.rados.rados.0.smithi162.stdout:1300: finishing copy_from to smithi16234619-20 2024-09-16T15:06:29.194 INFO:tasks.rados.rados.0.smithi162.stdout:1300: got expected ENOENT (src dne) 2024-09-16T15:06:29.194 INFO:tasks.rados.rados.0.smithi162.stdout:1300: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:06:29.203 INFO:tasks.rados.rados.0.smithi162.stdout:1300: done (1 left) 2024-09-16T15:06:29.203 INFO:tasks.rados.rados.0.smithi162.stdout:1301: done (0 left) 2024-09-16T15:06:29.203 INFO:tasks.rados.rados.0.smithi162.stdout:1302: snap_remove snap 118 2024-09-16T15:06:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:1302: done (0 left) 2024-09-16T15:06:30.190 INFO:tasks.rados.rados.0.smithi162.stdout:1303: delete oid 17 current snap is 120 2024-09-16T15:06:30.199 INFO:tasks.rados.rados.0.smithi162.stdout:1303: done (0 left) 2024-09-16T15:06:30.199 INFO:tasks.rados.rados.0.smithi162.stdout:1304: read oid 16 snap -1 2024-09-16T15:06:30.199 INFO:tasks.rados.rados.0.smithi162.stdout:1304: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:30.200 INFO:tasks.rados.rados.0.smithi162.stdout:1305: snap_create 2024-09-16T15:06:30.208 INFO:tasks.rados.rados.0.smithi162.stdout:1304: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:30.415 INFO:tasks.rados.rados.0.smithi162.stdout:1304: done (1 left) 2024-09-16T15:06:30.415 INFO:tasks.rados.rados.0.smithi162.stdout:1305: done (0 left) 2024-09-16T15:06:30.415 INFO:tasks.rados.rados.0.smithi162.stdout:1306: write oid 39 current snap is 121 2024-09-16T15:06:30.416 INFO:tasks.rados.rados.0.smithi162.stdout:1306: seq_num 488 ranges {601816=568104,1623448=451823} 2024-09-16T15:06:30.421 INFO:tasks.rados.rados.0.smithi162.stdout:1306: writing smithi16234619-39 from 601816 to 1169920 tid 1 2024-09-16T15:06:30.426 INFO:tasks.rados.rados.0.smithi162.stdout:1306: writing smithi16234619-39 from 1623448 to 2075271 tid 2 2024-09-16T15:06:30.426 INFO:tasks.rados.rados.0.smithi162.stdout:1307: write oid 7 current snap is 121 2024-09-16T15:06:30.427 INFO:tasks.rados.rados.0.smithi162.stdout:1307: seq_num 489 ranges {618623=643353,1681930=536693} 2024-09-16T15:06:30.433 INFO:tasks.rados.rados.0.smithi162.stdout:1307: writing smithi16234619-7 from 618623 to 1261976 tid 1 2024-09-16T15:06:30.438 INFO:tasks.rados.rados.0.smithi162.stdout:1307: writing smithi16234619-7 from 1681930 to 2218623 tid 2 2024-09-16T15:06:30.439 INFO:tasks.rados.rados.0.smithi162.stdout:1308: snap_remove snap 120 2024-09-16T15:06:31.415 INFO:tasks.rados.rados.0.smithi162.stdout:1308: done (2 left) 2024-09-16T15:06:31.416 INFO:tasks.rados.rados.0.smithi162.stdout:1309: snap_remove snap 119 2024-09-16T15:06:32.419 INFO:tasks.rados.rados.0.smithi162.stdout:1309: done (2 left) 2024-09-16T15:06:32.419 INFO:tasks.rados.rados.0.smithi162.stdout:1310: write oid 2 current snap is 121 2024-09-16T15:06:32.420 INFO:tasks.rados.rados.0.smithi162.stdout:1306: finishing write tid 1 to smithi16234619-39 2024-09-16T15:06:32.420 INFO:tasks.rados.rados.0.smithi162.stdout:1306: finishing write tid 2 to smithi16234619-39 2024-09-16T15:06:32.420 INFO:tasks.rados.rados.0.smithi162.stdout:1306: finishing write tid 3 to smithi16234619-39 2024-09-16T15:06:32.420 INFO:tasks.rados.rados.0.smithi162.stdout:1310: seq_num 490 ranges {635430=718602,2140412=95018} 2024-09-16T15:06:32.426 INFO:tasks.rados.rados.0.smithi162.stdout:1310: writing smithi16234619-2 from 635430 to 1354032 tid 1 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1310: writing smithi16234619-2 from 2140412 to 2235430 tid 2 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1311: read oid 19 snap -1 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1311: expect deleted 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1312: read oid 5 snap -1 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1312: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1313: read oid 25 snap -1 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1313: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1314: read oid 20 snap -1 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1314: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1315: copy_from oid 45 from oid 29 current snap is 121 2024-09-16T15:06:32.433 INFO:tasks.rados.rados.0.smithi162.stdout:1306: finishing write tid 4 to smithi16234619-39 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1306: oid 39 updating version 0 to 551 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1306: oid 39 version 551 is already newer than 549 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1306: oid 39 version 551 is already newer than 550 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 551 (ObjNum 488 snap 121 seq_num 488) dirty exists 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1306: left oid 39 (ObjNum 488 snap 121 seq_num 488) 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: finishing write tid 1 to smithi16234619-7 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: finishing write tid 2 to smithi16234619-7 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: finishing write tid 3 to smithi16234619-7 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: finishing write tid 4 to smithi16234619-7 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: oid 7 updating version 0 to 295 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: oid 7 updating version 295 to 296 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:1307: oid 7 version 296 is already newer than 294 2024-09-16T15:06:32.434 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 296 (ObjNum 489 snap 121 seq_num 489) dirty exists 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1307: left oid 7 (ObjNum 489 snap 121 seq_num 489) 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1306: done (7 left) 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1307: done (6 left) 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1316: copy_from oid 1 from oid 47 current snap is 121 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1317: write (excl) oid 34 current snap is 121 2024-09-16T15:06:32.435 INFO:tasks.rados.rados.0.smithi162.stdout:1317: seq_num 491 ranges {652237=793851,2198894=53343} 2024-09-16T15:06:32.441 INFO:tasks.rados.rados.0.smithi162.stdout:1317: writing smithi16234619-34 from 652237 to 1446088 tid 1 2024-09-16T15:06:32.446 INFO:tasks.rados.rados.0.smithi162.stdout:1317: writing smithi16234619-34 from 2198894 to 2252237 tid 2 2024-09-16T15:06:32.446 INFO:tasks.rados.rados.0.smithi162.stdout:1318: write (excl) oid 40 current snap is 121 2024-09-16T15:06:32.446 INFO:tasks.rados.rados.0.smithi162.stdout:1314: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:32.578 INFO:tasks.rados.rados.0.smithi162.stdout:1312: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:06:32.683 INFO:tasks.rados.rados.0.smithi162.stdout:1313: expect (ObjNum 311 snap 66 seq_num 311) 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: finishing write tid 1 to smithi16234619-2 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: finishing write tid 2 to smithi16234619-2 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: finishing write tid 3 to smithi16234619-2 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: finishing write tid 4 to smithi16234619-2 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: oid 2 updating version 0 to 514 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: oid 2 version 514 is already newer than 513 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: oid 2 version 514 is already newer than 512 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 514 (ObjNum 490 snap 121 seq_num 490) dirty exists 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1310: left oid 2 (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:06:32.856 INFO:tasks.rados.rados.0.smithi162.stdout:1316: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:06:32.857 INFO:tasks.rados.rados.0.smithi162.stdout:1317: finishing write tid 1 to smithi16234619-34 2024-09-16T15:06:32.857 INFO:tasks.rados.rados.0.smithi162.stdout:1317: finishing write tid 2 to smithi16234619-34 2024-09-16T15:06:32.857 INFO:tasks.rados.rados.0.smithi162.stdout:1317: finishing write tid 3 to smithi16234619-34 2024-09-16T15:06:32.857 INFO:tasks.rados.rados.0.smithi162.stdout:1318: seq_num 492 ranges {669044=469100,1741023=459161} 2024-09-16T15:06:32.862 INFO:tasks.rados.rados.0.smithi162.stdout:1318: writing smithi16234619-40 from 669044 to 1138144 tid 1 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1318: writing smithi16234619-40 from 1741023 to 2200184 tid 2 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1310: done (8 left) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1311: done (7 left) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1312: done (6 left) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1313: done (5 left) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1314: done (4 left) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1319: write oid 7 current snap is 121 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1317: finishing write tid 4 to smithi16234619-34 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1317: oid 34 updating version 0 to 448 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1317: oid 34 version 448 is already newer than 446 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1317: oid 34 version 448 is already newer than 447 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 448 (ObjNum 491 snap 121 seq_num 491) dirty exists 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1317: left oid 34 (ObjNum 491 snap 121 seq_num 491) 2024-09-16T15:06:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1316: finishing copy_from to smithi16234619-1 2024-09-16T15:06:32.869 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 134 (ObjNum 482 snap 119 seq_num 482) dirty exists 2024-09-16T15:06:32.869 INFO:tasks.rados.rados.0.smithi162.stdout:1315: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:06:32.869 INFO:tasks.rados.rados.0.smithi162.stdout:1319: seq_num 493 ranges {685851=544349,1799505=486346} 2024-09-16T15:06:32.874 INFO:tasks.rados.rados.0.smithi162.stdout:1319: writing smithi16234619-7 from 685851 to 1230200 tid 1 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:1319: writing smithi16234619-7 from 1799505 to 2285851 tid 2 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:1316: done (4 left) 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:1317: done (3 left) 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:1320: snap_create 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:1315: finishing copy_from to smithi16234619-45 2024-09-16T15:06:32.880 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 245 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:06:32.895 INFO:tasks.rados.rados.0.smithi162.stdout:1318: finishing write tid 1 to smithi16234619-40 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: finishing write tid 2 to smithi16234619-40 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1319: finishing write tid 1 to smithi16234619-7 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: finishing write tid 3 to smithi16234619-40 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: finishing write tid 4 to smithi16234619-40 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: oid 40 updating version 0 to 256 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: oid 40 version 256 is already newer than 255 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: oid 40 updating version 256 to 257 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 257 (ObjNum 492 snap 121 seq_num 492) dirty exists 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1318: left oid 40 (ObjNum 492 snap 121 seq_num 492) 2024-09-16T15:06:32.907 INFO:tasks.rados.rados.0.smithi162.stdout:1319: finishing write tid 2 to smithi16234619-7 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: finishing write tid 3 to smithi16234619-7 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: finishing write tid 4 to smithi16234619-7 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: oid 7 updating version 0 to 298 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: oid 7 updating version 298 to 299 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: oid 7 version 299 is already newer than 297 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 299 (ObjNum 493 snap 121 seq_num 493) dirty exists 2024-09-16T15:06:32.908 INFO:tasks.rados.rados.0.smithi162.stdout:1319: left oid 7 (ObjNum 493 snap 121 seq_num 493) 2024-09-16T15:06:33.577 INFO:tasks.rados.rados.0.smithi162.stdout:1315: done (3 left) 2024-09-16T15:06:33.578 INFO:tasks.rados.rados.0.smithi162.stdout:1318: done (2 left) 2024-09-16T15:06:33.578 INFO:tasks.rados.rados.0.smithi162.stdout:1319: done (1 left) 2024-09-16T15:06:33.578 INFO:tasks.rados.rados.0.smithi162.stdout:1320: done (0 left) 2024-09-16T15:06:33.578 INFO:tasks.rados.rados.0.smithi162.stdout:1321: delete oid 36 current snap is 122 2024-09-16T15:06:33.579 INFO:tasks.rados.rados.0.smithi162.stdout:1321: done (0 left) 2024-09-16T15:06:33.579 INFO:tasks.rados.rados.0.smithi162.stdout:1322: copy_from oid 44 from oid 11 current snap is 122 2024-09-16T15:06:33.579 INFO:tasks.rados.rados.0.smithi162.stdout:1323: delete oid 9 current snap is 122 2024-09-16T15:06:33.584 INFO:tasks.rados.rados.0.smithi162.stdout:1323: done (1 left) 2024-09-16T15:06:33.585 INFO:tasks.rados.rados.0.smithi162.stdout:1324: write (excl) oid 17 current snap is 122 2024-09-16T15:06:33.585 INFO:tasks.rados.rados.0.smithi162.stdout:1324: seq_num 496 ranges {736272=486449,1974951=361321} 2024-09-16T15:06:33.591 INFO:tasks.rados.rados.0.smithi162.stdout:1324: writing smithi16234619-17 from 736272 to 1222721 tid 1 2024-09-16T15:06:33.598 INFO:tasks.rados.rados.0.smithi162.stdout:1324: writing smithi16234619-17 from 1974951 to 2336272 tid 2 2024-09-16T15:06:33.598 INFO:tasks.rados.rados.0.smithi162.stdout:1325: rollback oid 25 current snap is 122 2024-09-16T15:06:33.598 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 121 2024-09-16T15:06:33.598 INFO:tasks.rados.rados.0.smithi162.stdout:1326: write (excl) oid 1 current snap is 122 2024-09-16T15:06:33.598 INFO:tasks.rados.rados.0.smithi162.stdout:1326: seq_num 497 ranges {753079=561698,2033433=319646} 2024-09-16T15:06:33.604 INFO:tasks.rados.rados.0.smithi162.stdout:1326: writing smithi16234619-1 from 753079 to 1314777 tid 1 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:1326: writing smithi16234619-1 from 2033433 to 2353079 tid 2 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:1327: delete oid 14 current snap is 122 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:1325: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:1325: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:1325: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:06:33.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 453 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:06:33.618 INFO:tasks.rados.rados.0.smithi162.stdout:1325: done (4 left) 2024-09-16T15:06:33.618 INFO:tasks.rados.rados.0.smithi162.stdout:1327: done (3 left) 2024-09-16T15:06:33.619 INFO:tasks.rados.rados.0.smithi162.stdout:1328: delete oid 38 current snap is 122 2024-09-16T15:06:33.619 INFO:tasks.rados.rados.0.smithi162.stdout:1324: finishing write tid 1 to smithi16234619-17 2024-09-16T15:06:33.621 INFO:tasks.rados.rados.0.smithi162.stdout:1324: finishing write tid 2 to smithi16234619-17 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: finishing write tid 3 to smithi16234619-17 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: finishing write tid 4 to smithi16234619-17 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: oid 17 updating version 0 to 519 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: oid 17 version 519 is already newer than 518 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: oid 17 updating version 519 to 520 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 520 (ObjNum 496 snap 122 seq_num 496) dirty exists 2024-09-16T15:06:33.622 INFO:tasks.rados.rados.0.smithi162.stdout:1324: left oid 17 (ObjNum 496 snap 122 seq_num 496) 2024-09-16T15:06:33.632 INFO:tasks.rados.rados.0.smithi162.stdout:1326: finishing write tid 1 to smithi16234619-1 2024-09-16T15:06:33.632 INFO:tasks.rados.rados.0.smithi162.stdout:1326: finishing write tid 2 to smithi16234619-1 2024-09-16T15:06:33.632 INFO:tasks.rados.rados.0.smithi162.stdout:1326: finishing write tid 3 to smithi16234619-1 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:1326: finishing write tid 4 to smithi16234619-1 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:1326: oid 1 updating version 0 to 487 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:1326: oid 1 version 487 is already newer than 486 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:1326: oid 1 updating version 487 to 488 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 488 (ObjNum 497 snap 122 seq_num 497) dirty exists 2024-09-16T15:06:33.633 INFO:tasks.rados.rados.0.smithi162.stdout:1326: left oid 1 (ObjNum 497 snap 122 seq_num 497) 2024-09-16T15:06:33.661 INFO:tasks.rados.rados.0.smithi162.stdout:1322: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:06:33.666 INFO:tasks.rados.rados.0.smithi162.stdout:1322: finishing copy_from to smithi16234619-44 2024-09-16T15:06:33.666 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 82 (ObjNum 468 snap 116 seq_num 468) dirty exists 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1322: done (3 left) 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1324: done (2 left) 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1326: done (1 left) 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1328: done (0 left) 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1329: read oid 46 snap -1 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1329: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1330: read oid 49 snap -1 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1330: expect deleted 2024-09-16T15:06:33.667 INFO:tasks.rados.rados.0.smithi162.stdout:1331: delete oid 37 current snap is 122 2024-09-16T15:06:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:1330: done (2 left) 2024-09-16T15:06:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:1331: done (1 left) 2024-09-16T15:06:33.671 INFO:tasks.rados.rados.0.smithi162.stdout:1332: snap_create 2024-09-16T15:06:33.677 INFO:tasks.rados.rados.0.smithi162.stdout:1329: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:34.582 INFO:tasks.rados.rados.0.smithi162.stdout:1329: done (1 left) 2024-09-16T15:06:34.583 INFO:tasks.rados.rados.0.smithi162.stdout:1332: done (0 left) 2024-09-16T15:06:34.583 INFO:tasks.rados.rados.0.smithi162.stdout:1333: snap_create 2024-09-16T15:06:35.413 INFO:tasks.rados.rados.0.smithi162.stdout:1333: done (0 left) 2024-09-16T15:06:35.413 INFO:tasks.rados.rados.0.smithi162.stdout:1334: copy_from oid 35 from oid 2 current snap is 124 2024-09-16T15:06:35.413 INFO:tasks.rados.rados.0.smithi162.stdout:1335: read oid 15 snap -1 2024-09-16T15:06:35.413 INFO:tasks.rados.rados.0.smithi162.stdout:1335: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:35.414 INFO:tasks.rados.rados.0.smithi162.stdout:1336: delete oid 41 current snap is 124 2024-09-16T15:06:35.420 INFO:tasks.rados.rados.0.smithi162.stdout:1336: done (2 left) 2024-09-16T15:06:35.420 INFO:tasks.rados.rados.0.smithi162.stdout:1337: snap_create 2024-09-16T15:06:35.425 INFO:tasks.rados.rados.0.smithi162.stdout:1335: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:35.554 INFO:tasks.rados.rados.0.smithi162.stdout:1334: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:06:35.554 INFO:tasks.rados.rados.0.smithi162.stdout:1334: finishing copy_from to smithi16234619-35 2024-09-16T15:06:35.554 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 514 (ObjNum 490 snap 121 seq_num 490) dirty exists 2024-09-16T15:06:36.416 INFO:tasks.rados.rados.0.smithi162.stdout:1334: done (2 left) 2024-09-16T15:06:36.417 INFO:tasks.rados.rados.0.smithi162.stdout:1335: done (1 left) 2024-09-16T15:06:36.417 INFO:tasks.rados.rados.0.smithi162.stdout:1337: done (0 left) 2024-09-16T15:06:36.417 INFO:tasks.rados.rados.0.smithi162.stdout:1338: read oid 16 snap -1 2024-09-16T15:06:36.417 INFO:tasks.rados.rados.0.smithi162.stdout:1338: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:36.417 INFO:tasks.rados.rados.0.smithi162.stdout:1339: snap_create 2024-09-16T15:06:36.424 INFO:tasks.rados.rados.0.smithi162.stdout:1338: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:37.416 INFO:tasks.rados.rados.0.smithi162.stdout:1338: done (1 left) 2024-09-16T15:06:37.416 INFO:tasks.rados.rados.0.smithi162.stdout:1339: done (0 left) 2024-09-16T15:06:37.416 INFO:tasks.rados.rados.0.smithi162.stdout:1340: read oid 19 snap 125 2024-09-16T15:06:37.416 INFO:tasks.rados.rados.0.smithi162.stdout:1340: expect deleted 2024-09-16T15:06:37.416 INFO:tasks.rados.rados.0.smithi162.stdout:1341: snap_remove snap 123 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1341: done (1 left) 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1342: read oid 46 snap -1 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1342: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1343: read oid 20 snap -1 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1343: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:38.418 INFO:tasks.rados.rados.0.smithi162.stdout:1344: snap_create 2024-09-16T15:06:38.422 INFO:tasks.rados.rados.0.smithi162.stdout:1343: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:38.552 INFO:tasks.rados.rados.0.smithi162.stdout:1342: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:39.420 INFO:tasks.rados.rados.0.smithi162.stdout:1340: done (3 left) 2024-09-16T15:06:39.421 INFO:tasks.rados.rados.0.smithi162.stdout:1342: done (2 left) 2024-09-16T15:06:39.421 INFO:tasks.rados.rados.0.smithi162.stdout:1343: done (1 left) 2024-09-16T15:06:39.421 INFO:tasks.rados.rados.0.smithi162.stdout:1344: done (0 left) 2024-09-16T15:06:39.421 INFO:tasks.rados.rados.0.smithi162.stdout:1345: snap_create 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1345: done (0 left) 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1346: read oid 26 snap -1 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1346: expect (ObjNum 473 snap 118 seq_num 473) 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1347: read oid 42 snap -1 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1347: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:40.415 INFO:tasks.rados.rados.0.smithi162.stdout:1348: delete oid 47 current snap is 128 2024-09-16T15:06:40.421 INFO:tasks.rados.rados.0.smithi162.stdout:1347: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:06:40.512 INFO:tasks.rados.rados.0.smithi162.stdout:1346: expect (ObjNum 473 snap 118 seq_num 473) 2024-09-16T15:06:40.743 INFO:tasks.rados.rados.0.smithi162.stdout:1346: done (2 left) 2024-09-16T15:06:40.743 INFO:tasks.rados.rados.0.smithi162.stdout:1347: done (1 left) 2024-09-16T15:06:40.743 INFO:tasks.rados.rados.0.smithi162.stdout:1348: done (0 left) 2024-09-16T15:06:40.744 INFO:tasks.rados.rados.0.smithi162.stdout:1349: snap_remove snap 121 2024-09-16T15:06:41.422 INFO:tasks.rados.rados.0.smithi162.stdout:1349: done (0 left) 2024-09-16T15:06:41.422 INFO:tasks.rados.rados.0.smithi162.stdout:1350: write oid 22 current snap is 128 2024-09-16T15:06:41.422 INFO:tasks.rados.rados.0.smithi162.stdout:1350: seq_num 503 ranges {453921=729545,1584325=621164} 2024-09-16T15:06:41.428 INFO:tasks.rados.rados.0.smithi162.stdout:1350: writing smithi16234619-22 from 453921 to 1183466 tid 1 2024-09-16T15:06:41.433 INFO:tasks.rados.rados.0.smithi162.stdout:1350: writing smithi16234619-22 from 1584325 to 2205489 tid 2 2024-09-16T15:06:41.433 INFO:tasks.rados.rados.0.smithi162.stdout:1351: write (excl) oid 45 current snap is 128 2024-09-16T15:06:41.434 INFO:tasks.rados.rados.0.smithi162.stdout:1351: seq_num 504 ranges {470728=404794,1526454=481175} 2024-09-16T15:06:41.438 INFO:tasks.rados.rados.0.smithi162.stdout:1351: writing smithi16234619-45 from 470728 to 875522 tid 1 2024-09-16T15:06:41.444 INFO:tasks.rados.rados.0.smithi162.stdout:1351: writing smithi16234619-45 from 1526454 to 2007629 tid 2 2024-09-16T15:06:41.445 INFO:tasks.rados.rados.0.smithi162.stdout:1352: snap_create 2024-09-16T15:06:41.463 INFO:tasks.rados.rados.0.smithi162.stdout:1350: finishing write tid 1 to smithi16234619-22 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: finishing write tid 2 to smithi16234619-22 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: finishing write tid 3 to smithi16234619-22 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1351: finishing write tid 1 to smithi16234619-45 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: finishing write tid 4 to smithi16234619-22 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: oid 22 updating version 0 to 290 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: oid 22 updating version 290 to 291 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: oid 22 version 291 is already newer than 289 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 291 (ObjNum 503 snap 128 seq_num 503) dirty exists 2024-09-16T15:06:41.469 INFO:tasks.rados.rados.0.smithi162.stdout:1350: left oid 22 (ObjNum 503 snap 128 seq_num 503) 2024-09-16T15:06:41.471 INFO:tasks.rados.rados.0.smithi162.stdout:1351: finishing write tid 2 to smithi16234619-45 2024-09-16T15:06:41.471 INFO:tasks.rados.rados.0.smithi162.stdout:1351: finishing write tid 3 to smithi16234619-45 2024-09-16T15:06:41.471 INFO:tasks.rados.rados.0.smithi162.stdout:1351: finishing write tid 4 to smithi16234619-45 2024-09-16T15:06:41.471 INFO:tasks.rados.rados.0.smithi162.stdout:1351: oid 45 updating version 0 to 524 2024-09-16T15:06:41.471 INFO:tasks.rados.rados.0.smithi162.stdout:1351: oid 45 version 524 is already newer than 522 2024-09-16T15:06:41.472 INFO:tasks.rados.rados.0.smithi162.stdout:1351: oid 45 version 524 is already newer than 523 2024-09-16T15:06:41.472 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 524 (ObjNum 504 snap 128 seq_num 504) dirty exists 2024-09-16T15:06:41.472 INFO:tasks.rados.rados.0.smithi162.stdout:1351: left oid 45 (ObjNum 504 snap 128 seq_num 504) 2024-09-16T15:06:43.230 INFO:tasks.rados.rados.0.smithi162.stdout:1350: done (2 left) 2024-09-16T15:06:43.230 INFO:tasks.rados.rados.0.smithi162.stdout:1351: done (1 left) 2024-09-16T15:06:43.230 INFO:tasks.rados.rados.0.smithi162.stdout:1352: done (0 left) 2024-09-16T15:06:43.230 INFO:tasks.rados.rados.0.smithi162.stdout:1353: snap_create 2024-09-16T15:06:44.229 INFO:tasks.rados.rados.0.smithi162.stdout:1353: done (0 left) 2024-09-16T15:06:44.229 INFO:tasks.rados.rados.0.smithi162.stdout:1354: rollback oid 39 current snap is 130 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 129 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1355: copy_from oid 25 from oid 33 current snap is 130 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1356: read oid 20 snap -1 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1356: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1357: read oid 24 snap -1 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1357: expect deleted 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1358: write oid 26 current snap is 130 2024-09-16T15:06:44.230 INFO:tasks.rados.rados.0.smithi162.stdout:1358: seq_num 505 ranges {487535=480043,1584936=624833} 2024-09-16T15:06:44.235 INFO:tasks.rados.rados.0.smithi162.stdout:1358: writing smithi16234619-26 from 487535 to 967578 tid 1 2024-09-16T15:06:44.244 INFO:tasks.rados.rados.0.smithi162.stdout:1358: writing smithi16234619-26 from 1584936 to 2209769 tid 2 2024-09-16T15:06:44.244 INFO:tasks.rados.rados.0.smithi162.stdout:1359: snap_create 2024-09-16T15:06:44.244 INFO:tasks.rados.rados.0.smithi162.stdout:1356: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:44.372 INFO:tasks.rados.rados.0.smithi162.stdout:1355: finishing copy_from to smithi16234619-25 2024-09-16T15:06:44.372 INFO:tasks.rados.rados.0.smithi162.stdout:1355: got expected ENOENT (src dne) 2024-09-16T15:06:44.372 INFO:tasks.rados.rados.0.smithi162.stdout:1355: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:06:44.372 INFO:tasks.rados.rados.0.smithi162.stdout:1354: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:06:44.372 INFO:tasks.rados.rados.0.smithi162.stdout:1354: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1354: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 561 (ObjNum 488 snap 121 seq_num 488) dirty exists 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: finishing write tid 1 to smithi16234619-26 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: finishing write tid 2 to smithi16234619-26 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: finishing write tid 3 to smithi16234619-26 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: finishing write tid 4 to smithi16234619-26 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: oid 26 updating version 0 to 528 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: oid 26 version 528 is already newer than 527 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: oid 26 version 528 is already newer than 526 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 528 (ObjNum 505 snap 130 seq_num 505) dirty exists 2024-09-16T15:06:44.373 INFO:tasks.rados.rados.0.smithi162.stdout:1358: left oid 26 (ObjNum 505 snap 130 seq_num 505) 2024-09-16T15:06:45.242 INFO:tasks.rados.rados.0.smithi162.stdout:1354: done (5 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1355: done (4 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1356: done (3 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1357: done (2 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1358: done (1 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1359: done (0 left) 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1360: write (excl) oid 7 current snap is 131 2024-09-16T15:06:45.243 INFO:tasks.rados.rados.0.smithi162.stdout:1360: seq_num 506 ranges {504342=555292,1643418=484844} 2024-09-16T15:06:45.248 INFO:tasks.rados.rados.0.smithi162.stdout:1360: writing smithi16234619-7 from 504342 to 1059634 tid 1 2024-09-16T15:06:45.254 INFO:tasks.rados.rados.0.smithi162.stdout:1360: writing smithi16234619-7 from 1643418 to 2128262 tid 2 2024-09-16T15:06:45.254 INFO:tasks.rados.rados.0.smithi162.stdout:1361: snap_create 2024-09-16T15:06:45.275 INFO:tasks.rados.rados.0.smithi162.stdout:1360: finishing write tid 1 to smithi16234619-7 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: finishing write tid 2 to smithi16234619-7 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: finishing write tid 3 to smithi16234619-7 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: finishing write tid 4 to smithi16234619-7 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: oid 7 updating version 0 to 303 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: oid 7 version 303 is already newer than 302 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: oid 7 version 303 is already newer than 301 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 303 (ObjNum 506 snap 131 seq_num 506) dirty exists 2024-09-16T15:06:45.278 INFO:tasks.rados.rados.0.smithi162.stdout:1360: left oid 7 (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:06:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:1360: done (1 left) 2024-09-16T15:06:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:1361: done (0 left) 2024-09-16T15:06:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:1362: write (excl) oid 22 current snap is 132 2024-09-16T15:06:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:1362: seq_num 507 ranges {521149=630541,1701900=628502} 2024-09-16T15:06:45.422 INFO:tasks.rados.rados.0.smithi162.stdout:1362: writing smithi16234619-22 from 521149 to 1151690 tid 1 2024-09-16T15:06:45.429 INFO:tasks.rados.rados.0.smithi162.stdout:1362: writing smithi16234619-22 from 1701900 to 2330402 tid 2 2024-09-16T15:06:45.429 INFO:tasks.rados.rados.0.smithi162.stdout:1363: delete oid 32 current snap is 132 2024-09-16T15:06:45.430 INFO:tasks.rados.rados.0.smithi162.stdout:1363: done (1 left) 2024-09-16T15:06:45.430 INFO:tasks.rados.rados.0.smithi162.stdout:1364: read oid 7 snap -1 2024-09-16T15:06:45.431 INFO:tasks.rados.rados.0.smithi162.stdout:1364: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:06:45.431 INFO:tasks.rados.rados.0.smithi162.stdout:1365: snap_create 2024-09-16T15:06:45.439 INFO:tasks.rados.rados.0.smithi162.stdout:1364: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:06:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:1362: finishing write tid 1 to smithi16234619-22 2024-09-16T15:06:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:1362: finishing write tid 2 to smithi16234619-22 2024-09-16T15:06:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:1362: finishing write tid 3 to smithi16234619-22 2024-09-16T15:06:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:1362: finishing write tid 4 to smithi16234619-22 2024-09-16T15:06:45.572 INFO:tasks.rados.rados.0.smithi162.stdout:1362: oid 22 updating version 0 to 293 2024-09-16T15:06:45.572 INFO:tasks.rados.rados.0.smithi162.stdout:1362: oid 22 version 293 is already newer than 292 2024-09-16T15:06:45.572 INFO:tasks.rados.rados.0.smithi162.stdout:1362: oid 22 updating version 293 to 294 2024-09-16T15:06:45.572 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 294 (ObjNum 507 snap 132 seq_num 507) dirty exists 2024-09-16T15:06:45.572 INFO:tasks.rados.rados.0.smithi162.stdout:1362: left oid 22 (ObjNum 507 snap 132 seq_num 507) 2024-09-16T15:06:46.420 INFO:tasks.rados.rados.0.smithi162.stdout:1362: done (2 left) 2024-09-16T15:06:46.420 INFO:tasks.rados.rados.0.smithi162.stdout:1364: done (1 left) 2024-09-16T15:06:46.420 INFO:tasks.rados.rados.0.smithi162.stdout:1365: done (0 left) 2024-09-16T15:06:46.420 INFO:tasks.rados.rados.0.smithi162.stdout:1366: delete oid 32 current snap is 133 2024-09-16T15:06:46.423 INFO:tasks.rados.rados.0.smithi162.stdout:1366: done (0 left) 2024-09-16T15:06:46.423 INFO:tasks.rados.rados.0.smithi162.stdout:1367: read oid 46 snap -1 2024-09-16T15:06:46.423 INFO:tasks.rados.rados.0.smithi162.stdout:1367: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:46.424 INFO:tasks.rados.rados.0.smithi162.stdout:1368: snap_remove snap 125 2024-09-16T15:06:47.419 INFO:tasks.rados.rados.0.smithi162.stdout:1368: done (1 left) 2024-09-16T15:06:47.419 INFO:tasks.rados.rados.0.smithi162.stdout:1369: read oid 16 snap -1 2024-09-16T15:06:47.419 INFO:tasks.rados.rados.0.smithi162.stdout:1369: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:47.419 INFO:tasks.rados.rados.0.smithi162.stdout:1370: write (excl) oid 39 current snap is 133 2024-09-16T15:06:47.419 INFO:tasks.rados.rados.0.smithi162.stdout:1370: seq_num 510 ranges {571570=572641,1877346=694224} 2024-09-16T15:06:47.426 INFO:tasks.rados.rados.0.smithi162.stdout:1370: writing smithi16234619-39 from 571570 to 1144211 tid 1 2024-09-16T15:06:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:1370: writing smithi16234619-39 from 1877346 to 2571570 tid 2 2024-09-16T15:06:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:1371: rollback oid 38 current snap is 133 2024-09-16T15:06:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 126 2024-09-16T15:06:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:1367: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:47.543 INFO:tasks.rados.rados.0.smithi162.stdout:1369: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:47.691 INFO:tasks.rados.rados.0.smithi162.stdout:1371: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: finishing write tid 1 to smithi16234619-39 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: finishing write tid 2 to smithi16234619-39 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: finishing write tid 3 to smithi16234619-39 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: finishing write tid 4 to smithi16234619-39 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: oid 39 updating version 0 to 565 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: oid 39 updating version 565 to 566 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: oid 39 updating version 566 to 567 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 567 (ObjNum 510 snap 133 seq_num 510) dirty exists 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: left oid 39 (ObjNum 510 snap 133 seq_num 510) 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1367: done (3 left) 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1369: done (2 left) 2024-09-16T15:06:47.692 INFO:tasks.rados.rados.0.smithi162.stdout:1370: done (1 left) 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1371: done (0 left) 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1372: rollback oid 39 current snap is 133 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 129 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1373: read oid 24 snap -1 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1373: expect deleted 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1374: copy_from oid 45 from oid 27 current snap is 133 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1375: read oid 43 snap -1 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1375: expect (ObjNum 458 snap 109 seq_num 458) 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1376: rollback oid 49 current snap is 133 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 131 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1377: read oid 32 snap -1 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1377: expect deleted 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1378: snap_create 2024-09-16T15:06:47.693 INFO:tasks.rados.rados.0.smithi162.stdout:1376: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:06:47.694 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:06:47.694 INFO:tasks.rados.rados.0.smithi162.stdout:1374: finishing copy_from to smithi16234619-45 2024-09-16T15:06:47.694 INFO:tasks.rados.rados.0.smithi162.stdout:1374: got expected ENOENT (src dne) 2024-09-16T15:06:47.694 INFO:tasks.rados.rados.0.smithi162.stdout:1374: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:06:47.695 INFO:tasks.rados.rados.0.smithi162.stdout:1372: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:06:47.695 INFO:tasks.rados.rados.0.smithi162.stdout:1372: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:06:47.696 INFO:tasks.rados.rados.0.smithi162.stdout:1375: expect (ObjNum 458 snap 109 seq_num 458) 2024-09-16T15:06:47.972 INFO:tasks.rados.rados.0.smithi162.stdout:1372: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:06:47.972 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 570 (ObjNum 488 snap 121 seq_num 488) dirty exists 2024-09-16T15:06:48.659 INFO:tasks.rados.rados.0.smithi162.stdout:1372: done (6 left) 2024-09-16T15:06:48.659 INFO:tasks.rados.rados.0.smithi162.stdout:1373: done (5 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1374: done (4 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1375: done (3 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1376: done (2 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1377: done (1 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1378: done (0 left) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1379: read oid 2 snap -1 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1379: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1380: write (excl) oid 22 current snap is 134 2024-09-16T15:06:48.660 INFO:tasks.rados.rados.0.smithi162.stdout:1380: seq_num 511 ranges {588377=647890,1935828=635840} 2024-09-16T15:06:48.666 INFO:tasks.rados.rados.0.smithi162.stdout:1380: writing smithi16234619-22 from 588377 to 1236267 tid 1 2024-09-16T15:06:48.673 INFO:tasks.rados.rados.0.smithi162.stdout:1380: writing smithi16234619-22 from 1935828 to 2571668 tid 2 2024-09-16T15:06:48.673 INFO:tasks.rados.rados.0.smithi162.stdout:1381: write oid 11 current snap is 134 2024-09-16T15:06:48.673 INFO:tasks.rados.rados.0.smithi162.stdout:1381: seq_num 512 ranges {605184=723139,1877957=727227} 2024-09-16T15:06:48.680 INFO:tasks.rados.rados.0.smithi162.stdout:1381: writing smithi16234619-11 from 605184 to 1328323 tid 1 2024-09-16T15:06:48.688 INFO:tasks.rados.rados.0.smithi162.stdout:1381: writing smithi16234619-11 from 1877957 to 2605184 tid 2 2024-09-16T15:06:48.689 INFO:tasks.rados.rados.0.smithi162.stdout:1382: snap_create 2024-09-16T15:06:48.689 INFO:tasks.rados.rados.0.smithi162.stdout:1379: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:06:48.794 INFO:tasks.rados.rados.0.smithi162.stdout:1380: finishing write tid 1 to smithi16234619-22 2024-09-16T15:06:48.794 INFO:tasks.rados.rados.0.smithi162.stdout:1380: finishing write tid 2 to smithi16234619-22 2024-09-16T15:06:48.794 INFO:tasks.rados.rados.0.smithi162.stdout:1380: finishing write tid 3 to smithi16234619-22 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1380: finishing write tid 4 to smithi16234619-22 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1380: oid 22 updating version 0 to 296 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1380: oid 22 updating version 296 to 297 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1380: oid 22 version 297 is already newer than 295 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 297 (ObjNum 511 snap 134 seq_num 511) dirty exists 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1380: left oid 22 (ObjNum 511 snap 134 seq_num 511) 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: finishing write tid 1 to smithi16234619-11 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: finishing write tid 2 to smithi16234619-11 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: finishing write tid 3 to smithi16234619-11 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: finishing write tid 4 to smithi16234619-11 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: oid 11 updating version 0 to 100 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: oid 11 version 100 is already newer than 98 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: oid 11 version 100 is already newer than 99 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 100 (ObjNum 512 snap 134 seq_num 512) dirty exists 2024-09-16T15:06:48.795 INFO:tasks.rados.rados.0.smithi162.stdout:1381: left oid 11 (ObjNum 512 snap 134 seq_num 512) 2024-09-16T15:06:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:1379: done (3 left) 2024-09-16T15:06:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:1380: done (2 left) 2024-09-16T15:06:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:1381: done (1 left) 2024-09-16T15:06:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:1382: done (0 left) 2024-09-16T15:06:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:1383: rollback oid 50 current snap is 135 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 133 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1384: read oid 40 snap 128 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1384: expect (ObjNum 492 snap 121 seq_num 492) 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1385: read oid 20 snap -1 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1385: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1386: read oid 16 snap 129 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1386: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1387: copy_from oid 43 from oid 5 current snap is 135 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1388: write (excl) oid 21 current snap is 135 2024-09-16T15:06:50.659 INFO:tasks.rados.rados.0.smithi162.stdout:1388: seq_num 513 ranges {621991=798388,1936439=639509} 2024-09-16T15:06:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1388: writing smithi16234619-21 from 621991 to 1420379 tid 1 2024-09-16T15:06:50.674 INFO:tasks.rados.rados.0.smithi162.stdout:1388: writing smithi16234619-21 from 1936439 to 2575948 tid 2 2024-09-16T15:06:50.674 INFO:tasks.rados.rados.0.smithi162.stdout:1389: rollback oid 42 current snap is 135 2024-09-16T15:06:50.674 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 133 2024-09-16T15:06:50.674 INFO:tasks.rados.rados.0.smithi162.stdout:1385: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:50.808 INFO:tasks.rados.rados.0.smithi162.stdout:1383: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:06:50.808 INFO:tasks.rados.rados.0.smithi162.stdout:1386: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:06:50.954 INFO:tasks.rados.rados.0.smithi162.stdout:1383: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:06:50.954 INFO:tasks.rados.rados.0.smithi162.stdout:1384: expect (ObjNum 492 snap 121 seq_num 492) 2024-09-16T15:06:51.030 INFO:tasks.rados.rados.0.smithi162.stdout:1383: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:06:51.030 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 460 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:06:51.030 INFO:tasks.rados.rados.0.smithi162.stdout:1389: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1389: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1389: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 574 (ObjNum 410 snap 95 seq_num 410) dirty exists 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1388: finishing write tid 1 to smithi16234619-21 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1388: finishing write tid 2 to smithi16234619-21 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1383: done (6 left) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1384: done (5 left) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1385: done (4 left) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1386: done (3 left) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1389: done (2 left) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1390: read oid 7 snap -1 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1390: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1391: rollback oid 46 current snap is 135 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 131 2024-09-16T15:06:51.031 INFO:tasks.rados.rados.0.smithi162.stdout:1388: finishing write tid 3 to smithi16234619-21 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: finishing write tid 4 to smithi16234619-21 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: oid 21 updating version 0 to 534 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: oid 21 version 534 is already newer than 533 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: oid 21 updating version 534 to 535 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 535 (ObjNum 513 snap 135 seq_num 513) dirty exists 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: left oid 21 (ObjNum 513 snap 135 seq_num 513) 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1387: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1387: finishing copy_from to smithi16234619-43 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 503 (ObjNum 441 snap 104 seq_num 441) dirty exists 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1387: done (3 left) 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1388: done (2 left) 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:1392: rollback oid 1 current snap is 135 2024-09-16T15:06:51.032 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 127 2024-09-16T15:06:51.033 INFO:tasks.rados.rados.0.smithi162.stdout:1393: delete oid 5 current snap is 135 2024-09-16T15:06:51.034 INFO:tasks.rados.rados.0.smithi162.stdout:1393: done (3 left) 2024-09-16T15:06:51.034 INFO:tasks.rados.rados.0.smithi162.stdout:1394: copy_from oid 11 from oid 41 current snap is 135 2024-09-16T15:06:51.034 INFO:tasks.rados.rados.0.smithi162.stdout:1395: write oid 34 current snap is 135 2024-09-16T15:06:51.034 INFO:tasks.rados.rados.0.smithi162.stdout:1395: seq_num 515 ranges {655605=548886,1653403=643178} 2024-09-16T15:06:51.041 INFO:tasks.rados.rados.0.smithi162.stdout:1395: writing smithi16234619-34 from 655605 to 1204491 tid 1 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1395: writing smithi16234619-34 from 1653403 to 2296581 tid 2 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1396: read oid 17 snap -1 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1396: expect (ObjNum 496 snap 122 seq_num 496) 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1391: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1392: finishing rollback tid 0 to smithi16234619-1 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1394: finishing copy_from to smithi16234619-11 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1394: got expected ENOENT (src dne) 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1394: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1391: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:1391: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:06:51.048 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 248 (ObjNum 422 snap 98 seq_num 422) dirty exists 2024-09-16T15:06:51.049 INFO:tasks.rados.rados.0.smithi162.stdout:1392: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:06:51.049 INFO:tasks.rados.rados.0.smithi162.stdout:1392: finishing rollback tid 2 to smithi16234619-1 2024-09-16T15:06:51.049 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 496 (ObjNum 497 snap 122 seq_num 497) dirty exists 2024-09-16T15:06:51.049 INFO:tasks.rados.rados.0.smithi162.stdout:1390: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:06:51.179 INFO:tasks.rados.rados.0.smithi162.stdout:1395: finishing write tid 1 to smithi16234619-34 2024-09-16T15:06:51.179 INFO:tasks.rados.rados.0.smithi162.stdout:1395: finishing write tid 2 to smithi16234619-34 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: finishing write tid 3 to smithi16234619-34 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: finishing write tid 4 to smithi16234619-34 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: oid 34 updating version 0 to 463 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: oid 34 version 463 is already newer than 462 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: oid 34 updating version 463 to 464 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 464 (ObjNum 515 snap 135 seq_num 515) dirty exists 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: left oid 34 (ObjNum 515 snap 135 seq_num 515) 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1390: done (5 left) 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1391: done (4 left) 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1392: done (3 left) 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1394: done (2 left) 2024-09-16T15:06:51.180 INFO:tasks.rados.rados.0.smithi162.stdout:1395: done (1 left) 2024-09-16T15:06:51.181 INFO:tasks.rados.rados.0.smithi162.stdout:1397: write (excl) oid 34 current snap is 135 2024-09-16T15:06:51.181 INFO:tasks.rados.rados.0.smithi162.stdout:1397: seq_num 516 ranges {672412=624135,1995532=676880} 2024-09-16T15:06:51.186 INFO:tasks.rados.rados.0.smithi162.stdout:1397: writing smithi16234619-34 from 672412 to 1296547 tid 1 2024-09-16T15:06:51.193 INFO:tasks.rados.rados.0.smithi162.stdout:1397: writing smithi16234619-34 from 1995532 to 2672412 tid 2 2024-09-16T15:06:51.193 INFO:tasks.rados.rados.0.smithi162.stdout:1398: delete oid 42 current snap is 135 2024-09-16T15:06:51.194 INFO:tasks.rados.rados.0.smithi162.stdout:1396: expect (ObjNum 496 snap 122 seq_num 496) 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: finishing write tid 1 to smithi16234619-34 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: finishing write tid 2 to smithi16234619-34 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: finishing write tid 3 to smithi16234619-34 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: finishing write tid 4 to smithi16234619-34 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: oid 34 updating version 0 to 465 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: oid 34 updating version 465 to 466 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:1397: oid 34 updating version 466 to 467 2024-09-16T15:06:51.273 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 467 (ObjNum 516 snap 135 seq_num 516) dirty exists 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1397: left oid 34 (ObjNum 516 snap 135 seq_num 516) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1396: done (2 left) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1397: done (1 left) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1398: done (0 left) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1399: read oid 45 snap 129 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1399: expect (ObjNum 504 snap 128 seq_num 504) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1400: read oid 12 snap -1 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1400: expect (ObjNum 481 snap 118 seq_num 481) 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1401: rollback oid 14 current snap is 135 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 129 2024-09-16T15:06:51.274 INFO:tasks.rados.rados.0.smithi162.stdout:1402: rollback oid 42 current snap is 135 2024-09-16T15:06:51.275 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 128 2024-09-16T15:06:51.275 INFO:tasks.rados.rados.0.smithi162.stdout:1403: copy_from oid 35 from oid 5 current snap is 135 2024-09-16T15:06:51.275 INFO:tasks.rados.rados.0.smithi162.stdout:1404: write (excl) oid 27 current snap is 135 2024-09-16T15:06:51.275 INFO:tasks.rados.rados.0.smithi162.stdout:1404: seq_num 518 ranges {706026=490986,1828849=790505} 2024-09-16T15:06:51.279 INFO:tasks.rados.rados.0.smithi162.stdout:1404: writing smithi16234619-27 from 706026 to 1197012 tid 1 2024-09-16T15:06:51.287 INFO:tasks.rados.rados.0.smithi162.stdout:1404: writing smithi16234619-27 from 1828849 to 2619354 tid 2 2024-09-16T15:06:51.287 INFO:tasks.rados.rados.0.smithi162.stdout:1405: snap_remove snap 132 2024-09-16T15:06:52.018 INFO:tasks.rados.rados.0.smithi162.stdout:1405: done (6 left) 2024-09-16T15:06:52.018 INFO:tasks.rados.rados.0.smithi162.stdout:1406: snap_remove snap 130 2024-09-16T15:06:53.025 INFO:tasks.rados.rados.0.smithi162.stdout:1406: done (6 left) 2024-09-16T15:06:53.025 INFO:tasks.rados.rados.0.smithi162.stdout:1407: read oid 23 snap 128 2024-09-16T15:06:53.025 INFO:tasks.rados.rados.0.smithi162.stdout:1407: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:53.025 INFO:tasks.rados.rados.0.smithi162.stdout:1408: write (excl) oid 17 current snap is 135 2024-09-16T15:06:53.025 INFO:tasks.rados.rados.0.smithi162.stdout:1408: seq_num 519 ranges {722833=566235,1887331=650516} 2024-09-16T15:06:53.032 INFO:tasks.rados.rados.0.smithi162.stdout:1408: writing smithi16234619-17 from 722833 to 1289068 tid 1 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1408: writing smithi16234619-17 from 1887331 to 2537847 tid 2 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1409: write (excl) oid 25 current snap is 135 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1401: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 0 (ObjNum 33 snap 0 seq_num 851383544) dirty dne 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1403: finishing copy_from to smithi16234619-35 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1403: got expected ENOENT (src dne) 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1403: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1402: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1402: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 579 (ObjNum 410 snap 95 seq_num 410) dirty exists 2024-09-16T15:06:53.040 INFO:tasks.rados.rados.0.smithi162.stdout:1399: expect (ObjNum 504 snap 128 seq_num 504) 2024-09-16T15:06:53.151 INFO:tasks.rados.rados.0.smithi162.stdout:1400: expect (ObjNum 481 snap 118 seq_num 481) 2024-09-16T15:06:53.234 INFO:tasks.rados.rados.0.smithi162.stdout:1404: finishing write tid 1 to smithi16234619-27 2024-09-16T15:06:53.234 INFO:tasks.rados.rados.0.smithi162.stdout:1404: finishing write tid 2 to smithi16234619-27 2024-09-16T15:06:53.234 INFO:tasks.rados.rados.0.smithi162.stdout:1404: finishing write tid 3 to smithi16234619-27 2024-09-16T15:06:53.234 INFO:tasks.rados.rados.0.smithi162.stdout:1404: finishing write tid 4 to smithi16234619-27 2024-09-16T15:06:53.234 INFO:tasks.rados.rados.0.smithi162.stdout:1404: oid 27 updating version 0 to 499 2024-09-16T15:06:53.235 INFO:tasks.rados.rados.0.smithi162.stdout:1404: oid 27 version 499 is already newer than 497 2024-09-16T15:06:53.235 INFO:tasks.rados.rados.0.smithi162.stdout:1404: oid 27 version 499 is already newer than 498 2024-09-16T15:06:53.235 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 499 (ObjNum 518 snap 135 seq_num 518) dirty exists 2024-09-16T15:06:53.235 INFO:tasks.rados.rados.0.smithi162.stdout:1404: left oid 27 (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:06:53.235 INFO:tasks.rados.rados.0.smithi162.stdout:1409: seq_num 520 ranges {739640=641484,1829460=794174} 2024-09-16T15:06:53.241 INFO:tasks.rados.rados.0.smithi162.stdout:1409: writing smithi16234619-25 from 739640 to 1381124 tid 1 2024-09-16T15:06:53.248 INFO:tasks.rados.rados.0.smithi162.stdout:1409: writing smithi16234619-25 from 1829460 to 2623634 tid 2 2024-09-16T15:06:53.248 INFO:tasks.rados.rados.0.smithi162.stdout:1399: done (8 left) 2024-09-16T15:06:53.248 INFO:tasks.rados.rados.0.smithi162.stdout:1400: done (7 left) 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1401: done (6 left) 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1402: done (5 left) 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1403: done (4 left) 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1404: done (3 left) 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1410: write (excl) oid 14 current snap is 135 2024-09-16T15:06:53.249 INFO:tasks.rados.rados.0.smithi162.stdout:1410: seq_num 521 ranges {756447=716733,1887942=537832} 2024-09-16T15:06:53.256 INFO:tasks.rados.rados.0.smithi162.stdout:1410: writing smithi16234619-14 from 756447 to 1473180 tid 1 2024-09-16T15:06:53.261 INFO:tasks.rados.rados.0.smithi162.stdout:1410: writing smithi16234619-14 from 1887942 to 2425774 tid 2 2024-09-16T15:06:53.261 INFO:tasks.rados.rados.0.smithi162.stdout:1411: snap_remove snap 131 2024-09-16T15:06:54.025 INFO:tasks.rados.rados.0.smithi162.stdout:1411: done (4 left) 2024-09-16T15:06:54.026 INFO:tasks.rados.rados.0.smithi162.stdout:1412: rollback oid 38 current snap is 135 2024-09-16T15:06:54.026 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 122 2024-09-16T15:06:54.026 INFO:tasks.rados.rados.0.smithi162.stdout:1407: expect (ObjNum 411 snap 95 seq_num 411) 2024-09-16T15:06:54.180 INFO:tasks.rados.rados.0.smithi162.stdout:1408: finishing write tid 1 to smithi16234619-17 2024-09-16T15:06:54.180 INFO:tasks.rados.rados.0.smithi162.stdout:1408: finishing write tid 2 to smithi16234619-17 2024-09-16T15:06:54.180 INFO:tasks.rados.rados.0.smithi162.stdout:1408: finishing write tid 3 to smithi16234619-17 2024-09-16T15:06:54.180 INFO:tasks.rados.rados.0.smithi162.stdout:1408: finishing write tid 4 to smithi16234619-17 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1408: oid 17 updating version 0 to 537 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1408: oid 17 updating version 537 to 539 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1408: oid 17 version 539 is already newer than 538 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 539 (ObjNum 519 snap 135 seq_num 519) dirty exists 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1408: left oid 17 (ObjNum 519 snap 135 seq_num 519) 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1409: finishing write tid 1 to smithi16234619-25 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1409: finishing write tid 2 to smithi16234619-25 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1409: finishing write tid 3 to smithi16234619-25 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1407: done (4 left) 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1408: done (3 left) 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1413: write (excl) oid 23 current snap is 135 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1409: finishing write tid 4 to smithi16234619-25 2024-09-16T15:06:54.181 INFO:tasks.rados.rados.0.smithi162.stdout:1409: oid 25 updating version 0 to 478 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:1409: oid 25 version 478 is already newer than 477 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:1409: oid 25 updating version 478 to 479 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 479 (ObjNum 520 snap 135 seq_num 520) dirty exists 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:1409: left oid 25 (ObjNum 520 snap 135 seq_num 520) 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:1410: finishing write tid 1 to smithi16234619-14 2024-09-16T15:06:54.182 INFO:tasks.rados.rados.0.smithi162.stdout:1413: seq_num 522 ranges {773254=791982,2346424=426830} 2024-09-16T15:06:54.188 INFO:tasks.rados.rados.0.smithi162.stdout:1413: writing smithi16234619-23 from 773254 to 1565236 tid 1 2024-09-16T15:06:54.195 INFO:tasks.rados.rados.0.smithi162.stdout:1413: writing smithi16234619-23 from 2346424 to 2773254 tid 2 2024-09-16T15:06:54.195 INFO:tasks.rados.rados.0.smithi162.stdout:1409: done (3 left) 2024-09-16T15:06:54.195 INFO:tasks.rados.rados.0.smithi162.stdout:1414: snap_create 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: finishing write tid 2 to smithi16234619-14 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: finishing write tid 3 to smithi16234619-14 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: finishing write tid 4 to smithi16234619-14 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: oid 14 updating version 0 to 481 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: oid 14 updating version 481 to 482 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: oid 14 version 482 is already newer than 480 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 482 (ObjNum 521 snap 135 seq_num 521) dirty exists 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1410: left oid 14 (ObjNum 521 snap 135 seq_num 521) 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:1412: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:06:54.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 0 (ObjNum 129 snap 0 seq_num 864873064) dirty dne 2024-09-16T15:06:54.219 INFO:tasks.rados.rados.0.smithi162.stdout:1413: finishing write tid 1 to smithi16234619-23 2024-09-16T15:06:54.226 INFO:tasks.rados.rados.0.smithi162.stdout:1413: finishing write tid 2 to smithi16234619-23 2024-09-16T15:06:54.228 INFO:tasks.rados.rados.0.smithi162.stdout:1413: finishing write tid 3 to smithi16234619-23 2024-09-16T15:06:54.228 INFO:tasks.rados.rados.0.smithi162.stdout:1413: finishing write tid 4 to smithi16234619-23 2024-09-16T15:06:54.228 INFO:tasks.rados.rados.0.smithi162.stdout:1413: oid 23 updating version 0 to 110 2024-09-16T15:06:54.228 INFO:tasks.rados.rados.0.smithi162.stdout:1413: oid 23 updating version 110 to 111 2024-09-16T15:06:54.228 INFO:tasks.rados.rados.0.smithi162.stdout:1413: oid 23 updating version 111 to 112 2024-09-16T15:06:54.229 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 112 (ObjNum 522 snap 135 seq_num 522) dirty exists 2024-09-16T15:06:54.229 INFO:tasks.rados.rados.0.smithi162.stdout:1413: left oid 23 (ObjNum 522 snap 135 seq_num 522) 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1410: done (3 left) 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1412: done (2 left) 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1413: done (1 left) 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1414: done (0 left) 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1415: write (excl) oid 1 current snap is 136 2024-09-16T15:06:55.028 INFO:tasks.rados.rados.0.smithi162.stdout:1415: seq_num 523 ranges {790061=467231,2004906=541501} 2024-09-16T15:06:55.034 INFO:tasks.rados.rados.0.smithi162.stdout:1415: writing smithi16234619-1 from 790061 to 1257292 tid 1 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1415: writing smithi16234619-1 from 2004906 to 2546407 tid 2 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1416: read oid 25 snap -1 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1416: expect (ObjNum 520 snap 135 seq_num 520) 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1417: read oid 18 snap -1 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1417: expect deleted 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1418: write oid 2 current snap is 136 2024-09-16T15:06:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:1418: seq_num 524 ranges {406868=542480,1547035=401512,2690354=116514} 2024-09-16T15:06:55.046 INFO:tasks.rados.rados.0.smithi162.stdout:1418: writing smithi16234619-2 from 406868 to 949348 tid 1 2024-09-16T15:06:55.052 INFO:tasks.rados.rados.0.smithi162.stdout:1418: writing smithi16234619-2 from 1547035 to 1948547 tid 2 2024-09-16T15:06:55.057 INFO:tasks.rados.rados.0.smithi162.stdout:1418: writing smithi16234619-2 from 2690354 to 2806868 tid 3 2024-09-16T15:06:55.058 INFO:tasks.rados.rados.0.smithi162.stdout:1419: delete oid 10 current snap is 136 2024-09-16T15:06:55.058 INFO:tasks.rados.rados.0.smithi162.stdout:1416: expect (ObjNum 520 snap 135 seq_num 520) 2024-09-16T15:06:55.209 INFO:tasks.rados.rados.0.smithi162.stdout:1415: finishing write tid 1 to smithi16234619-1 2024-09-16T15:06:55.209 INFO:tasks.rados.rados.0.smithi162.stdout:1415: finishing write tid 2 to smithi16234619-1 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: finishing write tid 3 to smithi16234619-1 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: finishing write tid 4 to smithi16234619-1 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: oid 1 updating version 0 to 508 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: oid 1 updating version 508 to 509 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: oid 1 version 509 is already newer than 507 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 509 (ObjNum 523 snap 136 seq_num 523) dirty exists 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: left oid 1 (ObjNum 523 snap 136 seq_num 523) 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1418: finishing write tid 1 to smithi16234619-2 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1418: finishing write tid 2 to smithi16234619-2 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1418: finishing write tid 3 to smithi16234619-2 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1415: done (4 left) 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1416: done (3 left) 2024-09-16T15:06:55.210 INFO:tasks.rados.rados.0.smithi162.stdout:1417: done (2 left) 2024-09-16T15:06:55.211 INFO:tasks.rados.rados.0.smithi162.stdout:1419: done (1 left) 2024-09-16T15:06:55.211 INFO:tasks.rados.rados.0.smithi162.stdout:1420: read oid 20 snap -1 2024-09-16T15:06:55.211 INFO:tasks.rados.rados.0.smithi162.stdout:1420: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:55.211 INFO:tasks.rados.rados.0.smithi162.stdout:1421: snap_remove snap 135 2024-09-16T15:06:55.418 INFO:tasks.rados.rados.0.smithi162.stdout:1421: done (2 left) 2024-09-16T15:06:55.418 INFO:tasks.rados.rados.0.smithi162.stdout:1422: snap_create 2024-09-16T15:06:55.418 INFO:tasks.rados.rados.0.smithi162.stdout:1418: finishing write tid 4 to smithi16234619-2 2024-09-16T15:06:55.418 INFO:tasks.rados.rados.0.smithi162.stdout:1418: finishing write tid 5 to smithi16234619-2 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1418: oid 2 updating version 0 to 546 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1418: oid 2 version 546 is already newer than 544 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1418: oid 2 version 546 is already newer than 543 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1418: oid 2 version 546 is already newer than 545 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 546 (ObjNum 524 snap 136 seq_num 524) dirty exists 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1418: left oid 2 (ObjNum 524 snap 136 seq_num 524) 2024-09-16T15:06:55.419 INFO:tasks.rados.rados.0.smithi162.stdout:1420: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:06:56.429 INFO:tasks.rados.rados.0.smithi162.stdout:1418: done (2 left) 2024-09-16T15:06:56.430 INFO:tasks.rados.rados.0.smithi162.stdout:1420: done (1 left) 2024-09-16T15:06:56.430 INFO:tasks.rados.rados.0.smithi162.stdout:1422: done (0 left) 2024-09-16T15:06:56.430 INFO:tasks.rados.rados.0.smithi162.stdout:1423: write oid 49 current snap is 137 2024-09-16T15:06:56.430 INFO:tasks.rados.rados.0.smithi162.stdout:1423: seq_num 526 ranges {440482=409331,1380352=405181,2461553=378929} 2024-09-16T15:06:56.434 INFO:tasks.rados.rados.0.smithi162.stdout:1423: writing smithi16234619-49 from 440482 to 849813 tid 1 2024-09-16T15:06:56.439 INFO:tasks.rados.rados.0.smithi162.stdout:1423: writing smithi16234619-49 from 1380352 to 1785533 tid 2 2024-09-16T15:06:56.445 INFO:tasks.rados.rados.0.smithi162.stdout:1423: writing smithi16234619-49 from 2461553 to 2840482 tid 3 2024-09-16T15:06:56.445 INFO:tasks.rados.rados.0.smithi162.stdout:1424: read oid 22 snap -1 2024-09-16T15:06:56.445 INFO:tasks.rados.rados.0.smithi162.stdout:1424: expect (ObjNum 511 snap 134 seq_num 511) 2024-09-16T15:06:56.445 INFO:tasks.rados.rados.0.smithi162.stdout:1425: delete oid 26 current snap is 137 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1425: done (2 left) 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1426: copy_from oid 24 from oid 26 current snap is 137 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1427: read oid 30 snap -1 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1427: expect (ObjNum 464 snap 113 seq_num 464) 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1428: rollback oid 41 current snap is 137 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 41 to 128 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1429: snap_create 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1426: finishing copy_from to smithi16234619-24 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1426: got expected ENOENT (src dne) 2024-09-16T15:06:56.449 INFO:tasks.rados.rados.0.smithi162.stdout:1426: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:06:56.463 INFO:tasks.rados.rados.0.smithi162.stdout:1427: expect (ObjNum 464 snap 113 seq_num 464) 2024-09-16T15:06:56.668 INFO:tasks.rados.rados.0.smithi162.stdout:1423: finishing write tid 1 to smithi16234619-49 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: finishing write tid 2 to smithi16234619-49 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: finishing write tid 3 to smithi16234619-49 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: finishing write tid 4 to smithi16234619-49 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: finishing write tid 5 to smithi16234619-49 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: oid 49 updating version 0 to 301 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: oid 49 version 301 is already newer than 300 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: oid 49 version 301 is already newer than 299 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: oid 49 updating version 301 to 302 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 302 (ObjNum 526 snap 137 seq_num 526) dirty exists 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1423: left oid 49 (ObjNum 526 snap 137 seq_num 526) 2024-09-16T15:06:56.669 INFO:tasks.rados.rados.0.smithi162.stdout:1424: expect (ObjNum 511 snap 134 seq_num 511) 2024-09-16T15:06:56.809 INFO:tasks.rados.rados.0.smithi162.stdout:1428: finishing rollback tid 1 to smithi16234619-41 2024-09-16T15:06:56.809 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 288 (ObjNum 129 snap 0 seq_num 865798272) dirty dne 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1423: done (5 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1424: done (4 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1426: done (3 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1427: done (2 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1428: done (1 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1429: done (0 left) 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1430: copy_from oid 18 from oid 2 current snap is 138 2024-09-16T15:06:57.443 INFO:tasks.rados.rados.0.smithi162.stdout:1431: write (excl) oid 9 current snap is 138 2024-09-16T15:06:57.444 INFO:tasks.rados.rados.0.smithi162.stdout:1431: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2024-09-16T15:06:57.448 INFO:tasks.rados.rados.0.smithi162.stdout:1431: writing smithi16234619-9 from 474096 to 1033925 tid 1 2024-09-16T15:06:57.455 INFO:tasks.rados.rados.0.smithi162.stdout:1431: writing smithi16234619-9 from 1780963 to 2189813 tid 2 2024-09-16T15:06:57.459 INFO:tasks.rados.rados.0.smithi162.stdout:1431: writing smithi16234619-9 from 2800046 to 2874096 tid 3 2024-09-16T15:06:57.459 INFO:tasks.rados.rados.0.smithi162.stdout:1432: read oid 46 snap -1 2024-09-16T15:06:57.459 INFO:tasks.rados.rados.0.smithi162.stdout:1432: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:57.460 INFO:tasks.rados.rados.0.smithi162.stdout:1433: snap_remove snap 126 2024-09-16T15:06:58.450 INFO:tasks.rados.rados.0.smithi162.stdout:1433: done (3 left) 2024-09-16T15:06:58.451 INFO:tasks.rados.rados.0.smithi162.stdout:1434: copy_from oid 29 from oid 34 current snap is 138 2024-09-16T15:06:58.451 INFO:tasks.rados.rados.0.smithi162.stdout:1432: expect (ObjNum 422 snap 98 seq_num 422) 2024-09-16T15:06:58.553 INFO:tasks.rados.rados.0.smithi162.stdout:1431: finishing write tid 1 to smithi16234619-9 2024-09-16T15:06:58.553 INFO:tasks.rados.rados.0.smithi162.stdout:1431: finishing write tid 2 to smithi16234619-9 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: finishing write tid 3 to smithi16234619-9 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: finishing write tid 4 to smithi16234619-9 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: finishing write tid 5 to smithi16234619-9 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: oid 9 updating version 0 to 515 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: oid 9 version 515 is already newer than 514 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: oid 9 version 515 is already newer than 513 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: oid 9 version 515 is already newer than 512 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 515 (ObjNum 528 snap 138 seq_num 528) dirty exists 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: left oid 9 (ObjNum 528 snap 138 seq_num 528) 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1430: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1430: finishing copy_from to smithi16234619-18 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 546 (ObjNum 524 snap 136 seq_num 524) dirty exists 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1430: done (3 left) 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1431: done (2 left) 2024-09-16T15:06:58.554 INFO:tasks.rados.rados.0.smithi162.stdout:1432: done (1 left) 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1435: rollback oid 16 current snap is 138 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 133 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1434: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1434: finishing copy_from to smithi16234619-29 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 467 (ObjNum 516 snap 135 seq_num 516) dirty exists 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1434: done (1 left) 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1436: delete oid 29 current snap is 138 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1436: done (1 left) 2024-09-16T15:06:58.555 INFO:tasks.rados.rados.0.smithi162.stdout:1437: read oid 2 snap -1 2024-09-16T15:06:58.556 INFO:tasks.rados.rados.0.smithi162.stdout:1437: expect (ObjNum 524 snap 136 seq_num 524) 2024-09-16T15:06:58.556 INFO:tasks.rados.rados.0.smithi162.stdout:1438: snap_create 2024-09-16T15:06:58.557 INFO:tasks.rados.rados.0.smithi162.stdout:1435: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:06:58.557 INFO:tasks.rados.rados.0.smithi162.stdout:1435: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:06:58.558 INFO:tasks.rados.rados.0.smithi162.stdout:1437: expect (ObjNum 524 snap 136 seq_num 524) 2024-09-16T15:06:58.709 INFO:tasks.rados.rados.0.smithi162.stdout:1435: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:06:58.709 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 517 (ObjNum 444 snap 106 seq_num 444) dirty exists 2024-09-16T15:06:59.454 INFO:tasks.rados.rados.0.smithi162.stdout:1435: done (2 left) 2024-09-16T15:06:59.454 INFO:tasks.rados.rados.0.smithi162.stdout:1437: done (1 left) 2024-09-16T15:06:59.455 INFO:tasks.rados.rados.0.smithi162.stdout:1438: done (0 left) 2024-09-16T15:06:59.455 INFO:tasks.rados.rados.0.smithi162.stdout:1439: read oid 27 snap 124 2024-09-16T15:06:59.455 INFO:tasks.rados.rados.0.smithi162.stdout:1439: expect deleted 2024-09-16T15:06:59.455 INFO:tasks.rados.rados.0.smithi162.stdout:1440: snap_remove snap 129 2024-09-16T15:07:00.426 INFO:tasks.rados.rados.0.smithi162.stdout:1440: done (1 left) 2024-09-16T15:07:00.427 INFO:tasks.rados.rados.0.smithi162.stdout:1441: write oid 47 current snap is 139 2024-09-16T15:07:00.427 INFO:tasks.rados.rados.0.smithi162.stdout:1441: seq_num 530 ranges {507710=710327,1897927=412519,2854892=52818} 2024-09-16T15:07:00.434 INFO:tasks.rados.rados.0.smithi162.stdout:1441: writing smithi16234619-47 from 507710 to 1218037 tid 1 2024-09-16T15:07:00.441 INFO:tasks.rados.rados.0.smithi162.stdout:1441: writing smithi16234619-47 from 1897927 to 2310446 tid 2 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1441: writing smithi16234619-47 from 2854892 to 2907710 tid 3 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1442: rollback oid 11 current snap is 139 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 134 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1439: done (2 left) 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1443: read oid 13 snap -1 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1443: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1444: write oid 37 current snap is 139 2024-09-16T15:07:00.445 INFO:tasks.rados.rados.0.smithi162.stdout:1444: seq_num 531 ranges {524517=785576,1956409=556177} 2024-09-16T15:07:00.453 INFO:tasks.rados.rados.0.smithi162.stdout:1444: writing smithi16234619-37 from 524517 to 1310093 tid 1 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1444: writing smithi16234619-37 from 1956409 to 2512586 tid 2 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1445: read oid 21 snap -1 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1445: expect (ObjNum 513 snap 135 seq_num 513) 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1446: write oid 32 current snap is 139 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1442: finishing rollback tid 0 to smithi16234619-11 2024-09-16T15:07:00.461 INFO:tasks.rados.rados.0.smithi162.stdout:1446: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2024-09-16T15:07:00.466 INFO:tasks.rados.rados.0.smithi162.stdout:1446: writing smithi16234619-32 from 541324 to 1002149 tid 1 2024-09-16T15:07:00.471 INFO:tasks.rados.rados.0.smithi162.stdout:1446: writing smithi16234619-32 from 1498538 to 1914726 tid 2 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1446: writing smithi16234619-32 from 2393385 to 2941324 tid 3 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1447: read oid 20 snap -1 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1447: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1448: read oid 26 snap -1 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1448: expect deleted 2024-09-16T15:07:00.478 INFO:tasks.rados.rados.0.smithi162.stdout:1449: write oid 3 current snap is 139 2024-09-16T15:07:00.479 INFO:tasks.rados.rados.0.smithi162.stdout:1442: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:07:00.479 INFO:tasks.rados.rados.0.smithi162.stdout:1442: finishing rollback tid 2 to smithi16234619-11 2024-09-16T15:07:00.479 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 114 (ObjNum 512 snap 134 seq_num 512) dirty exists 2024-09-16T15:07:00.479 INFO:tasks.rados.rados.0.smithi162.stdout:1443: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:00.587 INFO:tasks.rados.rados.0.smithi162.stdout:1445: expect (ObjNum 513 snap 135 seq_num 513) 2024-09-16T15:07:00.667 INFO:tasks.rados.rados.0.smithi162.stdout:1441: finishing write tid 1 to smithi16234619-47 2024-09-16T15:07:00.667 INFO:tasks.rados.rados.0.smithi162.stdout:1441: finishing write tid 2 to smithi16234619-47 2024-09-16T15:07:00.667 INFO:tasks.rados.rados.0.smithi162.stdout:1441: finishing write tid 3 to smithi16234619-47 2024-09-16T15:07:00.667 INFO:tasks.rados.rados.0.smithi162.stdout:1447: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: finishing write tid 4 to smithi16234619-47 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: finishing write tid 5 to smithi16234619-47 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: oid 47 updating version 0 to 505 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: oid 47 updating version 505 to 506 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: oid 47 updating version 506 to 507 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: oid 47 version 507 is already newer than 504 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 507 (ObjNum 530 snap 139 seq_num 530) dirty exists 2024-09-16T15:07:00.795 INFO:tasks.rados.rados.0.smithi162.stdout:1441: left oid 47 (ObjNum 530 snap 139 seq_num 530) 2024-09-16T15:07:00.796 INFO:tasks.rados.rados.0.smithi162.stdout:1444: finishing write tid 1 to smithi16234619-37 2024-09-16T15:07:00.796 INFO:tasks.rados.rados.0.smithi162.stdout:1444: finishing write tid 2 to smithi16234619-37 2024-09-16T15:07:00.796 INFO:tasks.rados.rados.0.smithi162.stdout:1444: finishing write tid 3 to smithi16234619-37 2024-09-16T15:07:00.796 INFO:tasks.rados.rados.0.smithi162.stdout:1449: seq_num 533 ranges {558131=652427,1673373=559846,2820808=137323} 2024-09-16T15:07:00.801 INFO:tasks.rados.rados.0.smithi162.stdout:1449: writing smithi16234619-3 from 558131 to 1210558 tid 1 2024-09-16T15:07:00.806 INFO:tasks.rados.rados.0.smithi162.stdout:1449: writing smithi16234619-3 from 1673373 to 2233219 tid 2 2024-09-16T15:07:00.811 INFO:tasks.rados.rados.0.smithi162.stdout:1449: writing smithi16234619-3 from 2820808 to 2958131 tid 3 2024-09-16T15:07:00.811 INFO:tasks.rados.rados.0.smithi162.stdout:1441: done (8 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1442: done (7 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1443: done (6 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1445: done (5 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1447: done (4 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1448: done (3 left) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1450: read oid 8 snap 138 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1450: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1451: read oid 17 snap 134 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1451: expect (ObjNum 496 snap 122 seq_num 496) 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1452: delete oid 39 current snap is 139 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1444: finishing write tid 4 to smithi16234619-37 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1444: oid 37 updating version 0 to 615 2024-09-16T15:07:00.812 INFO:tasks.rados.rados.0.smithi162.stdout:1444: oid 37 updating version 615 to 617 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1444: oid 37 version 617 is already newer than 616 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 617 (ObjNum 531 snap 139 seq_num 531) dirty exists 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1444: left oid 37 (ObjNum 531 snap 139 seq_num 531) 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: finishing write tid 1 to smithi16234619-32 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: finishing write tid 2 to smithi16234619-32 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: finishing write tid 3 to smithi16234619-32 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: finishing write tid 4 to smithi16234619-32 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: finishing write tid 5 to smithi16234619-32 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: oid 32 updating version 0 to 263 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: oid 32 version 263 is already newer than 260 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: oid 32 version 263 is already newer than 261 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: oid 32 version 263 is already newer than 262 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 263 (ObjNum 532 snap 139 seq_num 532) dirty exists 2024-09-16T15:07:00.813 INFO:tasks.rados.rados.0.smithi162.stdout:1446: left oid 32 (ObjNum 532 snap 139 seq_num 532) 2024-09-16T15:07:00.814 INFO:tasks.rados.rados.0.smithi162.stdout:1451: expect (ObjNum 496 snap 122 seq_num 496) 2024-09-16T15:07:00.891 INFO:tasks.rados.rados.0.smithi162.stdout:1450: expect (ObjNum 372 snap 85 seq_num 372) 2024-09-16T15:07:00.962 INFO:tasks.rados.rados.0.smithi162.stdout:1449: finishing write tid 1 to smithi16234619-3 2024-09-16T15:07:00.962 INFO:tasks.rados.rados.0.smithi162.stdout:1449: finishing write tid 2 to smithi16234619-3 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: finishing write tid 3 to smithi16234619-3 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: finishing write tid 4 to smithi16234619-3 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: finishing write tid 5 to smithi16234619-3 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: oid 3 updating version 0 to 62 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: oid 3 version 62 is already newer than 60 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: oid 3 version 62 is already newer than 61 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: oid 3 version 62 is already newer than 59 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 62 (ObjNum 533 snap 139 seq_num 533) dirty exists 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: left oid 3 (ObjNum 533 snap 139 seq_num 533) 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1444: done (5 left) 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1446: done (4 left) 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1449: done (3 left) 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1450: done (2 left) 2024-09-16T15:07:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:1451: done (1 left) 2024-09-16T15:07:00.964 INFO:tasks.rados.rados.0.smithi162.stdout:1452: done (0 left) 2024-09-16T15:07:00.964 INFO:tasks.rados.rados.0.smithi162.stdout:1453: read oid 47 snap -1 2024-09-16T15:07:00.964 INFO:tasks.rados.rados.0.smithi162.stdout:1453: expect (ObjNum 530 snap 139 seq_num 530) 2024-09-16T15:07:00.964 INFO:tasks.rados.rados.0.smithi162.stdout:1454: snap_remove snap 136 2024-09-16T15:07:01.422 INFO:tasks.rados.rados.0.smithi162.stdout:1454: done (1 left) 2024-09-16T15:07:01.427 INFO:tasks.rados.rados.0.smithi162.stdout:1455: copy_from oid 25 from oid 14 current snap is 139 2024-09-16T15:07:01.427 INFO:tasks.rados.rados.0.smithi162.stdout:1453: expect (ObjNum 530 snap 139 seq_num 530) 2024-09-16T15:07:01.518 INFO:tasks.rados.rados.0.smithi162.stdout:1455: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:07:01.518 INFO:tasks.rados.rados.0.smithi162.stdout:1455: finishing copy_from to smithi16234619-25 2024-09-16T15:07:01.518 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 482 (ObjNum 521 snap 135 seq_num 521) dirty exists 2024-09-16T15:07:01.518 INFO:tasks.rados.rados.0.smithi162.stdout:1453: done (1 left) 2024-09-16T15:07:01.518 INFO:tasks.rados.rados.0.smithi162.stdout:1455: done (0 left) 2024-09-16T15:07:01.519 INFO:tasks.rados.rados.0.smithi162.stdout:1456: write (excl) oid 10 current snap is 139 2024-09-16T15:07:01.519 INFO:tasks.rados.rados.0.smithi162.stdout:1456: seq_num 535 ranges {591745=402925,1790337=563515} 2024-09-16T15:07:01.523 INFO:tasks.rados.rados.0.smithi162.stdout:1456: writing smithi16234619-10 from 591745 to 994670 tid 1 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1456: writing smithi16234619-10 from 1790337 to 2353852 tid 2 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1457: read oid 5 snap -1 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1457: expect deleted 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1458: read oid 13 snap -1 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1458: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:01.532 INFO:tasks.rados.rados.0.smithi162.stdout:1459: write oid 31 current snap is 139 2024-09-16T15:07:01.533 INFO:tasks.rados.rados.0.smithi162.stdout:1459: seq_num 536 ranges {608552=478174,1848819=707173} 2024-09-16T15:07:01.539 INFO:tasks.rados.rados.0.smithi162.stdout:1459: writing smithi16234619-31 from 608552 to 1086726 tid 1 2024-09-16T15:07:01.548 INFO:tasks.rados.rados.0.smithi162.stdout:1459: writing smithi16234619-31 from 1848819 to 2555992 tid 2 2024-09-16T15:07:01.548 INFO:tasks.rados.rados.0.smithi162.stdout:1460: read oid 35 snap -1 2024-09-16T15:07:01.548 INFO:tasks.rados.rados.0.smithi162.stdout:1460: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:01.548 INFO:tasks.rados.rados.0.smithi162.stdout:1461: delete oid 11 current snap is 139 2024-09-16T15:07:01.548 INFO:tasks.rados.rados.0.smithi162.stdout:1458: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:01.657 INFO:tasks.rados.rados.0.smithi162.stdout:1460: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:01.760 INFO:tasks.rados.rados.0.smithi162.stdout:1456: finishing write tid 1 to smithi16234619-10 2024-09-16T15:07:01.760 INFO:tasks.rados.rados.0.smithi162.stdout:1456: finishing write tid 2 to smithi16234619-10 2024-09-16T15:07:01.760 INFO:tasks.rados.rados.0.smithi162.stdout:1456: finishing write tid 3 to smithi16234619-10 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: finishing write tid 4 to smithi16234619-10 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: oid 10 updating version 0 to 575 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: oid 10 version 575 is already newer than 574 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: oid 10 version 575 is already newer than 573 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 575 (ObjNum 535 snap 139 seq_num 535) dirty exists 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: left oid 10 (ObjNum 535 snap 139 seq_num 535) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1459: finishing write tid 1 to smithi16234619-31 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1459: finishing write tid 2 to smithi16234619-31 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1456: done (5 left) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1457: done (4 left) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1458: done (3 left) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1460: done (2 left) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1461: done (1 left) 2024-09-16T15:07:01.761 INFO:tasks.rados.rados.0.smithi162.stdout:1462: write oid 28 current snap is 139 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: finishing write tid 3 to smithi16234619-31 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: finishing write tid 4 to smithi16234619-31 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: oid 31 updating version 0 to 548 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: oid 31 updating version 548 to 549 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: oid 31 version 549 is already newer than 547 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1459: left oid 31 (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:01.762 INFO:tasks.rados.rados.0.smithi162.stdout:1462: seq_num 538 ranges {642166=628672,1849430=710842} 2024-09-16T15:07:01.767 INFO:tasks.rados.rados.0.smithi162.stdout:1462: writing smithi16234619-28 from 642166 to 1270838 tid 1 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:1462: writing smithi16234619-28 from 1849430 to 2560272 tid 2 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:1459: done (1 left) 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:1463: rollback oid 4 current snap is 139 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 134 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:1464: rollback oid 42 current snap is 139 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 122 2024-09-16T15:07:01.774 INFO:tasks.rados.rados.0.smithi162.stdout:1465: snap_create 2024-09-16T15:07:01.775 INFO:tasks.rados.rados.0.smithi162.stdout:1463: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:07:01.775 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 0 (ObjNum 33 snap 0 seq_num 1953066355) dirty dne 2024-09-16T15:07:01.777 INFO:tasks.rados.rados.0.smithi162.stdout:1464: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:07:01.778 INFO:tasks.rados.rados.0.smithi162.stdout:1464: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:07:01.778 INFO:tasks.rados.rados.0.smithi162.stdout:1464: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:07:01.778 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 627 (ObjNum 410 snap 95 seq_num 410) dirty exists 2024-09-16T15:07:01.799 INFO:tasks.rados.rados.0.smithi162.stdout:1462: finishing write tid 1 to smithi16234619-28 2024-09-16T15:07:01.802 INFO:tasks.rados.rados.0.smithi162.stdout:1462: finishing write tid 2 to smithi16234619-28 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: finishing write tid 3 to smithi16234619-28 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: finishing write tid 4 to smithi16234619-28 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: oid 28 updating version 0 to 290 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: oid 28 updating version 290 to 291 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: oid 28 version 291 is already newer than 289 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 291 (ObjNum 538 snap 139 seq_num 538) dirty exists 2024-09-16T15:07:01.803 INFO:tasks.rados.rados.0.smithi162.stdout:1462: left oid 28 (ObjNum 538 snap 139 seq_num 538) 2024-09-16T15:07:02.568 INFO:tasks.rados.rados.0.smithi162.stdout:1462: done (3 left) 2024-09-16T15:07:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1463: done (2 left) 2024-09-16T15:07:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1464: done (1 left) 2024-09-16T15:07:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1465: done (0 left) 2024-09-16T15:07:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1466: write oid 30 current snap is 140 2024-09-16T15:07:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1466: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2024-09-16T15:07:02.575 INFO:tasks.rados.rados.0.smithi162.stdout:1466: writing smithi16234619-30 from 658973 to 1362894 tid 1 2024-09-16T15:07:02.581 INFO:tasks.rados.rados.0.smithi162.stdout:1466: writing smithi16234619-30 from 1907912 to 2478765 tid 2 2024-09-16T15:07:02.584 INFO:tasks.rados.rados.0.smithi162.stdout:1466: writing smithi16234619-30 from 2985346 to 3058973 tid 3 2024-09-16T15:07:02.584 INFO:tasks.rados.rados.0.smithi162.stdout:1467: delete oid 22 current snap is 140 2024-09-16T15:07:02.587 INFO:tasks.rados.rados.0.smithi162.stdout:1467: done (1 left) 2024-09-16T15:07:02.588 INFO:tasks.rados.rados.0.smithi162.stdout:1468: write (excl) oid 42 current snap is 140 2024-09-16T15:07:02.588 INFO:tasks.rados.rados.0.smithi162.stdout:1468: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2024-09-16T15:07:02.594 INFO:tasks.rados.rados.0.smithi162.stdout:1468: writing smithi16234619-42 from 692587 to 1263359 tid 1 2024-09-16T15:07:02.602 INFO:tasks.rados.rados.0.smithi162.stdout:1468: writing smithi16234619-42 from 2024876 to 2599398 tid 2 2024-09-16T15:07:02.605 INFO:tasks.rados.rados.0.smithi162.stdout:1468: writing smithi16234619-42 from 3040192 to 3092587 tid 3 2024-09-16T15:07:02.606 INFO:tasks.rados.rados.0.smithi162.stdout:1469: delete oid 22 current snap is 140 2024-09-16T15:07:02.607 INFO:tasks.rados.rados.0.smithi162.stdout:1469: done (2 left) 2024-09-16T15:07:02.607 INFO:tasks.rados.rados.0.smithi162.stdout:1470: copy_from oid 17 from oid 31 current snap is 140 2024-09-16T15:07:02.607 INFO:tasks.rados.rados.0.smithi162.stdout:1471: write (excl) oid 37 current snap is 140 2024-09-16T15:07:02.607 INFO:tasks.rados.rados.0.smithi162.stdout:1471: seq_num 543 ranges {726201=721270,2141840=578191} 2024-09-16T15:07:02.616 INFO:tasks.rados.rados.0.smithi162.stdout:1471: writing smithi16234619-37 from 726201 to 1447471 tid 1 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1471: writing smithi16234619-37 from 2141840 to 2720031 tid 2 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1472: snap_remove snap 124 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1466: finishing write tid 1 to smithi16234619-30 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1466: finishing write tid 2 to smithi16234619-30 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1466: finishing write tid 3 to smithi16234619-30 2024-09-16T15:07:02.623 INFO:tasks.rados.rados.0.smithi162.stdout:1466: finishing write tid 4 to smithi16234619-30 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: finishing write tid 5 to smithi16234619-30 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: oid 30 updating version 0 to 296 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: oid 30 version 296 is already newer than 294 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: oid 30 version 296 is already newer than 295 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: oid 30 version 296 is already newer than 293 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 296 (ObjNum 539 snap 140 seq_num 539) dirty exists 2024-09-16T15:07:02.624 INFO:tasks.rados.rados.0.smithi162.stdout:1466: left oid 30 (ObjNum 539 snap 140 seq_num 539) 2024-09-16T15:07:03.579 INFO:tasks.rados.rados.0.smithi162.stdout:1466: done (4 left) 2024-09-16T15:07:03.579 INFO:tasks.rados.rados.0.smithi162.stdout:1472: done (3 left) 2024-09-16T15:07:03.579 INFO:tasks.rados.rados.0.smithi162.stdout:1473: write oid 35 current snap is 140 2024-09-16T15:07:03.579 INFO:tasks.rados.rados.0.smithi162.stdout:1468: finishing write tid 1 to smithi16234619-42 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1470: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1470: finishing copy_from to smithi16234619-17 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: finishing write tid 2 to smithi16234619-42 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: finishing write tid 3 to smithi16234619-42 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: finishing write tid 4 to smithi16234619-42 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: finishing write tid 5 to smithi16234619-42 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: oid 42 updating version 0 to 630 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: oid 42 updating version 630 to 632 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: oid 42 version 632 is already newer than 629 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: oid 42 version 632 is already newer than 631 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 632 (ObjNum 541 snap 140 seq_num 541) dirty exists 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1468: left oid 42 (ObjNum 541 snap 140 seq_num 541) 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1471: finishing write tid 1 to smithi16234619-37 2024-09-16T15:07:03.580 INFO:tasks.rados.rados.0.smithi162.stdout:1471: finishing write tid 2 to smithi16234619-37 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: finishing write tid 3 to smithi16234619-37 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: finishing write tid 4 to smithi16234619-37 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: oid 37 updating version 0 to 636 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: oid 37 version 636 is already newer than 635 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: oid 37 version 636 is already newer than 634 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 636 (ObjNum 543 snap 140 seq_num 543) dirty exists 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1471: left oid 37 (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:03.581 INFO:tasks.rados.rados.0.smithi162.stdout:1473: seq_num 544 ranges {743008=796519,2200322=721849} 2024-09-16T15:07:03.588 INFO:tasks.rados.rados.0.smithi162.stdout:1473: writing smithi16234619-35 from 743008 to 1539527 tid 1 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1473: writing smithi16234619-35 from 2200322 to 2922171 tid 2 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1468: done (3 left) 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1470: done (2 left) 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1471: done (1 left) 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1474: read oid 50 snap 133 2024-09-16T15:07:03.596 INFO:tasks.rados.rados.0.smithi162.stdout:1474: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:03.597 INFO:tasks.rados.rados.0.smithi162.stdout:1475: rollback oid 38 current snap is 140 2024-09-16T15:07:03.597 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 137 2024-09-16T15:07:03.597 INFO:tasks.rados.rados.0.smithi162.stdout:1476: write (excl) oid 19 current snap is 140 2024-09-16T15:07:03.597 INFO:tasks.rados.rados.0.smithi162.stdout:1476: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2024-09-16T15:07:03.603 INFO:tasks.rados.rados.0.smithi162.stdout:1476: writing smithi16234619-19 from 759815 to 1231583 tid 1 2024-09-16T15:07:03.608 INFO:tasks.rados.rados.0.smithi162.stdout:1476: writing smithi16234619-19 from 1742451 to 2207958 tid 2 2024-09-16T15:07:03.614 INFO:tasks.rados.rados.0.smithi162.stdout:1476: writing smithi16234619-19 from 2917178 to 3159815 tid 3 2024-09-16T15:07:03.614 INFO:tasks.rados.rados.0.smithi162.stdout:1477: copy_from oid 23 from oid 26 current snap is 140 2024-09-16T15:07:03.614 INFO:tasks.rados.rados.0.smithi162.stdout:1474: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1477: finishing copy_from to smithi16234619-23 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1477: got expected ENOENT (src dne) 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1477: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: finishing write tid 1 to smithi16234619-35 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: finishing write tid 2 to smithi16234619-35 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1474: done (4 left) 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1477: done (3 left) 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1478: write (excl) oid 8 current snap is 140 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: finishing write tid 3 to smithi16234619-35 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: finishing write tid 4 to smithi16234619-35 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: oid 35 updating version 0 to 519 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: oid 35 version 519 is already newer than 518 2024-09-16T15:07:03.743 INFO:tasks.rados.rados.0.smithi162.stdout:1473: oid 35 updating version 519 to 520 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 520 (ObjNum 544 snap 140 seq_num 544) dirty exists 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:1473: left oid 35 (ObjNum 544 snap 140 seq_num 544) 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:1475: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:1476: finishing write tid 1 to smithi16234619-19 2024-09-16T15:07:03.744 INFO:tasks.rados.rados.0.smithi162.stdout:1478: seq_num 546 ranges {776622=547017,1800933=725518,2944601=232021} 2024-09-16T15:07:03.749 INFO:tasks.rados.rados.0.smithi162.stdout:1478: writing smithi16234619-8 from 776622 to 1323639 tid 1 2024-09-16T15:07:03.756 INFO:tasks.rados.rados.0.smithi162.stdout:1478: writing smithi16234619-8 from 1800933 to 2526451 tid 2 2024-09-16T15:07:03.759 INFO:tasks.rados.rados.0.smithi162.stdout:1478: writing smithi16234619-8 from 2944601 to 3176622 tid 3 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1473: done (3 left) 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1475: done (2 left) 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1479: rollback oid 32 current snap is 140 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 138 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: finishing write tid 2 to smithi16234619-19 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: finishing write tid 3 to smithi16234619-19 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: finishing write tid 4 to smithi16234619-19 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: finishing write tid 5 to smithi16234619-19 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: oid 19 updating version 0 to 509 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: oid 19 updating version 509 to 511 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: oid 19 updating version 511 to 512 2024-09-16T15:07:03.760 INFO:tasks.rados.rados.0.smithi162.stdout:1476: oid 19 version 512 is already newer than 510 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 512 (ObjNum 545 snap 140 seq_num 545) dirty exists 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:1476: left oid 19 (ObjNum 545 snap 140 seq_num 545) 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:1476: done (2 left) 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:1480: delete oid 4 current snap is 140 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:1480: done (2 left) 2024-09-16T15:07:03.761 INFO:tasks.rados.rados.0.smithi162.stdout:1481: delete oid 26 current snap is 140 2024-09-16T15:07:03.762 INFO:tasks.rados.rados.0.smithi162.stdout:1481: done (2 left) 2024-09-16T15:07:03.762 INFO:tasks.rados.rados.0.smithi162.stdout:1482: copy_from oid 26 from oid 28 current snap is 140 2024-09-16T15:07:03.763 INFO:tasks.rados.rados.0.smithi162.stdout:1483: rollback oid 14 current snap is 140 2024-09-16T15:07:03.763 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 127 2024-09-16T15:07:03.763 INFO:tasks.rados.rados.0.smithi162.stdout:1484: write oid 49 current snap is 140 2024-09-16T15:07:03.763 INFO:tasks.rados.rados.0.smithi162.stdout:1484: seq_num 549 ranges {427043=489117,1576379=472845,2626870=600173} 2024-09-16T15:07:03.767 INFO:tasks.rados.rados.0.smithi162.stdout:1484: writing smithi16234619-49 from 427043 to 916160 tid 1 2024-09-16T15:07:03.774 INFO:tasks.rados.rados.0.smithi162.stdout:1484: writing smithi16234619-49 from 1576379 to 2049224 tid 2 2024-09-16T15:07:03.781 INFO:tasks.rados.rados.0.smithi162.stdout:1484: writing smithi16234619-49 from 2626870 to 3227043 tid 3 2024-09-16T15:07:03.781 INFO:tasks.rados.rados.0.smithi162.stdout:1485: delete oid 46 current snap is 140 2024-09-16T15:07:03.781 INFO:tasks.rados.rados.0.smithi162.stdout:1483: finishing rollback tid 0 to smithi16234619-14 2024-09-16T15:07:03.781 INFO:tasks.rados.rados.0.smithi162.stdout:1483: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:07:03.781 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 515 (ObjNum 33 snap 0 seq_num 1580415402) dirty dne 2024-09-16T15:07:03.785 INFO:tasks.rados.rados.0.smithi162.stdout:1483: done (5 left) 2024-09-16T15:07:03.785 INFO:tasks.rados.rados.0.smithi162.stdout:1485: done (4 left) 2024-09-16T15:07:03.785 INFO:tasks.rados.rados.0.smithi162.stdout:1486: copy_from oid 11 from oid 39 current snap is 140 2024-09-16T15:07:03.785 INFO:tasks.rados.rados.0.smithi162.stdout:1487: write oid 22 current snap is 140 2024-09-16T15:07:03.785 INFO:tasks.rados.rados.0.smithi162.stdout:1487: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2024-09-16T15:07:03.791 INFO:tasks.rados.rados.0.smithi162.stdout:1487: writing smithi16234619-22 from 460657 to 1100272 tid 1 2024-09-16T15:07:03.797 INFO:tasks.rados.rados.0.smithi162.stdout:1487: writing smithi16234619-22 from 1693343 to 2169857 tid 2 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1487: writing smithi16234619-22 from 2681716 to 3260657 tid 3 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1488: read oid 4 snap -1 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1488: expect deleted 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1489: snap_create 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1478: finishing write tid 1 to smithi16234619-8 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1479: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:07:03.803 INFO:tasks.rados.rados.0.smithi162.stdout:1479: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 266 (ObjNum 257 snap 0 seq_num 865760592) dirty dne 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1486: finishing copy_from to smithi16234619-11 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1486: got expected ENOENT (src dne) 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1486: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: finishing write tid 2 to smithi16234619-8 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: finishing write tid 3 to smithi16234619-8 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: finishing write tid 4 to smithi16234619-8 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: finishing write tid 5 to smithi16234619-8 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: oid 8 updating version 0 to 550 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: oid 8 updating version 550 to 553 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: oid 8 version 553 is already newer than 551 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:1478: oid 8 version 553 is already newer than 552 2024-09-16T15:07:03.804 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 553 (ObjNum 546 snap 140 seq_num 546) dirty exists 2024-09-16T15:07:03.805 INFO:tasks.rados.rados.0.smithi162.stdout:1478: left oid 8 (ObjNum 546 snap 140 seq_num 546) 2024-09-16T15:07:03.809 INFO:tasks.rados.rados.0.smithi162.stdout:1484: finishing write tid 1 to smithi16234619-49 2024-09-16T15:07:03.827 INFO:tasks.rados.rados.0.smithi162.stdout:1482: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:07:03.832 INFO:tasks.rados.rados.0.smithi162.stdout:1484: finishing write tid 2 to smithi16234619-49 2024-09-16T15:07:03.846 INFO:tasks.rados.rados.0.smithi162.stdout:1484: finishing write tid 3 to smithi16234619-49 2024-09-16T15:07:03.857 INFO:tasks.rados.rados.0.smithi162.stdout:1482: finishing copy_from to smithi16234619-26 2024-09-16T15:07:03.857 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 291 (ObjNum 538 snap 139 seq_num 538) dirty exists 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: finishing write tid 4 to smithi16234619-49 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: finishing write tid 5 to smithi16234619-49 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: oid 49 updating version 0 to 313 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: oid 49 version 313 is already newer than 311 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: oid 49 version 313 is already newer than 310 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: oid 49 version 313 is already newer than 312 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 313 (ObjNum 549 snap 140 seq_num 549) dirty exists 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1484: left oid 49 (ObjNum 549 snap 140 seq_num 549) 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1487: finishing write tid 1 to smithi16234619-22 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1487: finishing write tid 2 to smithi16234619-22 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1487: finishing write tid 3 to smithi16234619-22 2024-09-16T15:07:03.863 INFO:tasks.rados.rados.0.smithi162.stdout:1487: finishing write tid 4 to smithi16234619-22 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: finishing write tid 5 to smithi16234619-22 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: oid 22 updating version 0 to 316 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: oid 22 updating version 316 to 317 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: oid 22 version 317 is already newer than 315 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: oid 22 version 317 is already newer than 314 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 317 (ObjNum 551 snap 140 seq_num 551) dirty exists 2024-09-16T15:07:03.864 INFO:tasks.rados.rados.0.smithi162.stdout:1487: left oid 22 (ObjNum 551 snap 140 seq_num 551) 2024-09-16T15:07:04.592 INFO:tasks.rados.rados.0.smithi162.stdout:1478: done (7 left) 2024-09-16T15:07:04.592 INFO:tasks.rados.rados.0.smithi162.stdout:1479: done (6 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1482: done (5 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1484: done (4 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1486: done (3 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1487: done (2 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1488: done (1 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1489: done (0 left) 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1490: write (excl) oid 24 current snap is 141 2024-09-16T15:07:04.593 INFO:tasks.rados.rados.0.smithi162.stdout:1490: seq_num 552 ranges {477464=714864,1751825=736525,3225492=51972} 2024-09-16T15:07:04.598 INFO:tasks.rados.rados.0.smithi162.stdout:1490: writing smithi16234619-24 from 477464 to 1192328 tid 1 2024-09-16T15:07:04.606 INFO:tasks.rados.rados.0.smithi162.stdout:1490: writing smithi16234619-24 from 1751825 to 2488350 tid 2 2024-09-16T15:07:04.610 INFO:tasks.rados.rados.0.smithi162.stdout:1490: writing smithi16234619-24 from 3225492 to 3277464 tid 3 2024-09-16T15:07:04.610 INFO:tasks.rados.rados.0.smithi162.stdout:1491: read oid 20 snap 127 2024-09-16T15:07:04.610 INFO:tasks.rados.rados.0.smithi162.stdout:1491: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:04.611 INFO:tasks.rados.rados.0.smithi162.stdout:1492: write oid 41 current snap is 141 2024-09-16T15:07:04.611 INFO:tasks.rados.rados.0.smithi162.stdout:1492: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2024-09-16T15:07:04.618 INFO:tasks.rados.rados.0.smithi162.stdout:1492: writing smithi16234619-41 from 494271 to 1284384 tid 1 2024-09-16T15:07:04.622 INFO:tasks.rados.rados.0.smithi162.stdout:1492: writing smithi16234619-41 from 1693954 to 2174137 tid 2 2024-09-16T15:07:04.628 INFO:tasks.rados.rados.0.smithi162.stdout:1492: writing smithi16234619-41 from 2620209 to 3132338 tid 3 2024-09-16T15:07:04.628 INFO:tasks.rados.rados.0.smithi162.stdout:1493: write oid 9 current snap is 141 2024-09-16T15:07:04.628 INFO:tasks.rados.rados.0.smithi162.stdout:1493: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2024-09-16T15:07:04.634 INFO:tasks.rados.rados.0.smithi162.stdout:1493: writing smithi16234619-9 from 511078 to 976440 tid 1 2024-09-16T15:07:04.644 INFO:tasks.rados.rados.0.smithi162.stdout:1493: writing smithi16234619-9 from 1752436 to 2492630 tid 2 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:1493: writing smithi16234619-9 from 3163985 to 3311078 tid 3 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:1494: read oid 23 snap -1 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:1494: expect (ObjNum 522 snap 135 seq_num 522) 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:1495: rollback oid 45 current snap is 141 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 128 2024-09-16T15:07:04.650 INFO:tasks.rados.rados.0.smithi162.stdout:1491: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: finishing write tid 1 to smithi16234619-24 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: finishing write tid 2 to smithi16234619-24 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: finishing write tid 3 to smithi16234619-24 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: finishing write tid 4 to smithi16234619-24 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: finishing write tid 5 to smithi16234619-24 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: oid 24 updating version 0 to 593 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: oid 24 updating version 593 to 594 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: oid 24 updating version 594 to 595 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: oid 24 version 595 is already newer than 592 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 595 (ObjNum 552 snap 141 seq_num 552) dirty exists 2024-09-16T15:07:04.781 INFO:tasks.rados.rados.0.smithi162.stdout:1490: left oid 24 (ObjNum 552 snap 141 seq_num 552) 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1490: done (5 left) 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1491: done (4 left) 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1496: delete oid 26 current snap is 141 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1495: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1495: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1495: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 599 (ObjNum 504 snap 128 seq_num 504) dirty exists 2024-09-16T15:07:04.782 INFO:tasks.rados.rados.0.smithi162.stdout:1494: expect (ObjNum 522 snap 135 seq_num 522) 2024-09-16T15:07:04.957 INFO:tasks.rados.rados.0.smithi162.stdout:1492: finishing write tid 1 to smithi16234619-41 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: finishing write tid 1 to smithi16234619-9 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: finishing write tid 2 to smithi16234619-9 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1492: finishing write tid 2 to smithi16234619-41 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: finishing write tid 3 to smithi16234619-9 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: finishing write tid 4 to smithi16234619-9 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: finishing write tid 5 to smithi16234619-9 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: oid 9 updating version 0 to 526 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: oid 9 updating version 526 to 527 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: oid 9 version 527 is already newer than 524 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: oid 9 version 527 is already newer than 525 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 527 (ObjNum 554 snap 141 seq_num 554) dirty exists 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1493: left oid 9 (ObjNum 554 snap 141 seq_num 554) 2024-09-16T15:07:04.958 INFO:tasks.rados.rados.0.smithi162.stdout:1492: finishing write tid 3 to smithi16234619-41 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: finishing write tid 4 to smithi16234619-41 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: finishing write tid 5 to smithi16234619-41 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: oid 41 updating version 0 to 321 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: oid 41 version 321 is already newer than 319 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: oid 41 version 321 is already newer than 320 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: oid 41 version 321 is already newer than 318 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 321 (ObjNum 553 snap 141 seq_num 553) dirty exists 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: left oid 41 (ObjNum 553 snap 141 seq_num 553) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1492: done (4 left) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1493: done (3 left) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1494: done (2 left) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1495: done (1 left) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1496: done (0 left) 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1497: write oid 15 current snap is 141 2024-09-16T15:07:04.959 INFO:tasks.rados.rados.0.smithi162.stdout:1497: seq_num 556 ranges {544692=732213,1985753=743863,3335184=9508} 2024-09-16T15:07:04.964 INFO:tasks.rados.rados.0.smithi162.stdout:1497: writing smithi16234619-15 from 544692 to 1276905 tid 1 2024-09-16T15:07:04.971 INFO:tasks.rados.rados.0.smithi162.stdout:1497: writing smithi16234619-15 from 1985753 to 2729616 tid 2 2024-09-16T15:07:04.975 INFO:tasks.rados.rados.0.smithi162.stdout:1497: writing smithi16234619-15 from 3335184 to 3344692 tid 3 2024-09-16T15:07:04.975 INFO:tasks.rados.rados.0.smithi162.stdout:1498: write oid 33 current snap is 141 2024-09-16T15:07:04.975 INFO:tasks.rados.rados.0.smithi162.stdout:1498: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2024-09-16T15:07:04.980 INFO:tasks.rados.rados.0.smithi162.stdout:1498: writing smithi16234619-33 from 561499 to 968961 tid 1 2024-09-16T15:07:04.986 INFO:tasks.rados.rados.0.smithi162.stdout:1498: writing smithi16234619-33 from 1527882 to 2015403 tid 2 2024-09-16T15:07:04.994 INFO:tasks.rados.rados.0.smithi162.stdout:1498: writing smithi16234619-33 from 2729901 to 3287118 tid 3 2024-09-16T15:07:04.994 INFO:tasks.rados.rados.0.smithi162.stdout:1499: write (excl) oid 18 current snap is 141 2024-09-16T15:07:04.994 INFO:tasks.rados.rados.0.smithi162.stdout:1499: seq_num 558 ranges {578306=482711,1586364=631179,2757324=484842} 2024-09-16T15:07:05.000 INFO:tasks.rados.rados.0.smithi162.stdout:1499: writing smithi16234619-18 from 578306 to 1061017 tid 1 2024-09-16T15:07:05.006 INFO:tasks.rados.rados.0.smithi162.stdout:1499: writing smithi16234619-18 from 1586364 to 2217543 tid 2 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1499: writing smithi16234619-18 from 2757324 to 3242166 tid 3 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1500: snap_create 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: finishing write tid 1 to smithi16234619-15 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: finishing write tid 2 to smithi16234619-15 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: finishing write tid 3 to smithi16234619-15 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: finishing write tid 4 to smithi16234619-15 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: finishing write tid 5 to smithi16234619-15 2024-09-16T15:07:05.012 INFO:tasks.rados.rados.0.smithi162.stdout:1497: oid 15 updating version 0 to 302 2024-09-16T15:07:05.013 INFO:tasks.rados.rados.0.smithi162.stdout:1497: oid 15 updating version 302 to 304 2024-09-16T15:07:05.013 INFO:tasks.rados.rados.0.smithi162.stdout:1497: oid 15 version 304 is already newer than 303 2024-09-16T15:07:05.013 INFO:tasks.rados.rados.0.smithi162.stdout:1497: oid 15 updating version 304 to 305 2024-09-16T15:07:05.013 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 305 (ObjNum 556 snap 141 seq_num 556) dirty exists 2024-09-16T15:07:05.013 INFO:tasks.rados.rados.0.smithi162.stdout:1497: left oid 15 (ObjNum 556 snap 141 seq_num 556) 2024-09-16T15:07:05.015 INFO:tasks.rados.rados.0.smithi162.stdout:1498: finishing write tid 1 to smithi16234619-33 2024-09-16T15:07:05.015 INFO:tasks.rados.rados.0.smithi162.stdout:1498: finishing write tid 2 to smithi16234619-33 2024-09-16T15:07:05.018 INFO:tasks.rados.rados.0.smithi162.stdout:1498: finishing write tid 3 to smithi16234619-33 2024-09-16T15:07:05.018 INFO:tasks.rados.rados.0.smithi162.stdout:1498: finishing write tid 4 to smithi16234619-33 2024-09-16T15:07:05.018 INFO:tasks.rados.rados.0.smithi162.stdout:1498: finishing write tid 5 to smithi16234619-33 2024-09-16T15:07:05.018 INFO:tasks.rados.rados.0.smithi162.stdout:1498: oid 33 updating version 0 to 182 2024-09-16T15:07:05.019 INFO:tasks.rados.rados.0.smithi162.stdout:1498: oid 33 version 182 is already newer than 179 2024-09-16T15:07:05.019 INFO:tasks.rados.rados.0.smithi162.stdout:1498: oid 33 version 182 is already newer than 181 2024-09-16T15:07:05.019 INFO:tasks.rados.rados.0.smithi162.stdout:1498: oid 33 version 182 is already newer than 180 2024-09-16T15:07:05.019 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 182 (ObjNum 557 snap 141 seq_num 557) dirty exists 2024-09-16T15:07:05.019 INFO:tasks.rados.rados.0.smithi162.stdout:1498: left oid 33 (ObjNum 557 snap 141 seq_num 557) 2024-09-16T15:07:05.037 INFO:tasks.rados.rados.0.smithi162.stdout:1499: finishing write tid 1 to smithi16234619-18 2024-09-16T15:07:05.037 INFO:tasks.rados.rados.0.smithi162.stdout:1499: finishing write tid 2 to smithi16234619-18 2024-09-16T15:07:05.037 INFO:tasks.rados.rados.0.smithi162.stdout:1499: finishing write tid 3 to smithi16234619-18 2024-09-16T15:07:05.037 INFO:tasks.rados.rados.0.smithi162.stdout:1499: finishing write tid 4 to smithi16234619-18 2024-09-16T15:07:05.037 INFO:tasks.rados.rados.0.smithi162.stdout:1499: finishing write tid 5 to smithi16234619-18 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:1499: oid 18 updating version 0 to 554 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:1499: oid 18 updating version 554 to 556 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:1499: oid 18 version 556 is already newer than 555 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:1499: oid 18 updating version 556 to 557 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 557 (ObjNum 558 snap 141 seq_num 558) dirty exists 2024-09-16T15:07:05.038 INFO:tasks.rados.rados.0.smithi162.stdout:1499: left oid 18 (ObjNum 558 snap 141 seq_num 558) 2024-09-16T15:07:05.427 INFO:tasks.rados.rados.0.smithi162.stdout:1497: done (3 left) 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1498: done (2 left) 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1499: done (1 left) 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1500: done (0 left) 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1501: read oid 46 snap -1 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1501: expect deleted 2024-09-16T15:07:05.428 INFO:tasks.rados.rados.0.smithi162.stdout:1502: delete oid 36 current snap is 142 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:1501: done (1 left) 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:1502: done (0 left) 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:1503: rollback oid 44 current snap is 142 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 137 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:1504: read oid 9 snap -1 2024-09-16T15:07:05.429 INFO:tasks.rados.rados.0.smithi162.stdout:1504: expect (ObjNum 554 snap 141 seq_num 554) 2024-09-16T15:07:05.430 INFO:tasks.rados.rados.0.smithi162.stdout:1505: read oid 26 snap -1 2024-09-16T15:07:05.430 INFO:tasks.rados.rados.0.smithi162.stdout:1505: expect deleted 2024-09-16T15:07:05.430 INFO:tasks.rados.rados.0.smithi162.stdout:1506: snap_remove snap 139 2024-09-16T15:07:06.899 INFO:tasks.rados.rados.0.smithi162.stdout:1506: done (3 left) 2024-09-16T15:07:06.900 INFO:tasks.rados.rados.0.smithi162.stdout:1507: snap_remove snap 140 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1507: done (3 left) 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1508: snap_create 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1503: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1503: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1503: finishing rollback tid 2 to smithi16234619-44 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 523 (ObjNum 468 snap 116 seq_num 468) dirty exists 2024-09-16T15:07:07.906 INFO:tasks.rados.rados.0.smithi162.stdout:1504: expect (ObjNum 554 snap 141 seq_num 554) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1503: done (3 left) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1504: done (2 left) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1505: done (1 left) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1508: done (0 left) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1509: read oid 25 snap 137 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1509: expect (ObjNum 520 snap 135 seq_num 520) 2024-09-16T15:07:08.911 INFO:tasks.rados.rados.0.smithi162.stdout:1510: delete oid 39 current snap is 143 2024-09-16T15:07:08.914 INFO:tasks.rados.rados.0.smithi162.stdout:1510: done (1 left) 2024-09-16T15:07:08.914 INFO:tasks.rados.rados.0.smithi162.stdout:1511: read oid 19 snap 141 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1511: expect (ObjNum 545 snap 140 seq_num 545) 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1512: rollback oid 2 current snap is 143 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 134 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1513: copy_from oid 15 from oid 17 current snap is 143 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1514: read oid 20 snap -1 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1514: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:08.915 INFO:tasks.rados.rados.0.smithi162.stdout:1515: delete oid 24 current snap is 143 2024-09-16T15:07:08.920 INFO:tasks.rados.rados.0.smithi162.stdout:1509: expect (ObjNum 520 snap 135 seq_num 520) 2024-09-16T15:07:09.068 INFO:tasks.rados.rados.0.smithi162.stdout:1514: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:09.195 INFO:tasks.rados.rados.0.smithi162.stdout:1511: expect (ObjNum 545 snap 140 seq_num 545) 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:1512: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:1512: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:1512: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 613 (ObjNum 490 snap 121 seq_num 490) dirty exists 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:1513: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:07:09.300 INFO:tasks.rados.rados.0.smithi162.stdout:1513: finishing copy_from to smithi16234619-15 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1509: done (5 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1511: done (4 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1512: done (3 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1513: done (2 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1514: done (1 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1515: done (0 left) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1516: copy_from oid 29 from oid 38 current snap is 143 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1517: read oid 37 snap -1 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1517: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:09.301 INFO:tasks.rados.rados.0.smithi162.stdout:1518: snap_remove snap 138 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1518: done (2 left) 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1519: rollback oid 44 current snap is 143 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 142 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1516: finishing copy_from to smithi16234619-29 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1516: got expected ENOENT (src dne) 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1516: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:07:09.924 INFO:tasks.rados.rados.0.smithi162.stdout:1517: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1519: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1519: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1519: finishing rollback tid 2 to smithi16234619-44 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 526 (ObjNum 468 snap 116 seq_num 468) dirty exists 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1516: done (2 left) 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1517: done (1 left) 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1519: done (0 left) 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1520: write oid 9 current snap is 143 2024-09-16T15:07:10.070 INFO:tasks.rados.rados.0.smithi162.stdout:1520: seq_num 562 ranges {645534=783707,2103939=638517,3150663=294871} 2024-09-16T15:07:10.077 INFO:tasks.rados.rados.0.smithi162.stdout:1520: writing smithi16234619-9 from 645534 to 1429241 tid 1 2024-09-16T15:07:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:1520: writing smithi16234619-9 from 2103939 to 2742456 tid 2 2024-09-16T15:07:10.088 INFO:tasks.rados.rados.0.smithi162.stdout:1520: writing smithi16234619-9 from 3150663 to 3445534 tid 3 2024-09-16T15:07:10.088 INFO:tasks.rados.rados.0.smithi162.stdout:1521: read oid 6 snap 122 2024-09-16T15:07:10.088 INFO:tasks.rados.rados.0.smithi162.stdout:1521: expect deleted 2024-09-16T15:07:10.088 INFO:tasks.rados.rados.0.smithi162.stdout:1522: write oid 36 current snap is 143 2024-09-16T15:07:10.089 INFO:tasks.rados.rados.0.smithi162.stdout:1522: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2024-09-16T15:07:10.095 INFO:tasks.rados.rados.0.smithi162.stdout:1522: writing smithi16234619-36 from 662341 to 1237650 tid 1 2024-09-16T15:07:10.101 INFO:tasks.rados.rados.0.smithi162.stdout:1522: writing smithi16234619-36 from 1878774 to 2377302 tid 2 2024-09-16T15:07:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:1522: writing smithi16234619-36 from 2894439 to 3351994 tid 3 2024-09-16T15:07:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:1523: delete oid 39 current snap is 143 2024-09-16T15:07:10.108 INFO:tasks.rados.rados.0.smithi162.stdout:1521: done (3 left) 2024-09-16T15:07:10.108 INFO:tasks.rados.rados.0.smithi162.stdout:1523: done (2 left) 2024-09-16T15:07:10.108 INFO:tasks.rados.rados.0.smithi162.stdout:1524: read oid 8 snap -1 2024-09-16T15:07:10.109 INFO:tasks.rados.rados.0.smithi162.stdout:1524: expect (ObjNum 546 snap 140 seq_num 546) 2024-09-16T15:07:10.109 INFO:tasks.rados.rados.0.smithi162.stdout:1525: write oid 31 current snap is 143 2024-09-16T15:07:10.109 INFO:tasks.rados.rados.0.smithi162.stdout:1525: seq_num 565 ranges {695955=725807,1879385=502197,2949285=480099} 2024-09-16T15:07:10.116 INFO:tasks.rados.rados.0.smithi162.stdout:1525: writing smithi16234619-31 from 695955 to 1421762 tid 1 2024-09-16T15:07:10.121 INFO:tasks.rados.rados.0.smithi162.stdout:1525: writing smithi16234619-31 from 1879385 to 2381582 tid 2 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1525: writing smithi16234619-31 from 2949285 to 3429384 tid 3 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1526: read oid 42 snap 134 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1526: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1527: read oid 16 snap -1 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1527: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:07:10.127 INFO:tasks.rados.rados.0.smithi162.stdout:1528: snap_remove snap 128 2024-09-16T15:07:10.423 INFO:tasks.rados.rados.0.smithi162.stdout:1528: done (6 left) 2024-09-16T15:07:10.423 INFO:tasks.rados.rados.0.smithi162.stdout:1529: delete oid 10 current snap is 143 2024-09-16T15:07:10.423 INFO:tasks.rados.rados.0.smithi162.stdout:1520: finishing write tid 1 to smithi16234619-9 2024-09-16T15:07:10.423 INFO:tasks.rados.rados.0.smithi162.stdout:1524: expect (ObjNum 546 snap 140 seq_num 546) 2024-09-16T15:07:10.563 INFO:tasks.rados.rados.0.smithi162.stdout:1527: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:07:10.710 INFO:tasks.rados.rados.0.smithi162.stdout:1526: expect (ObjNum 410 snap 95 seq_num 410) 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: finishing write tid 2 to smithi16234619-9 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: finishing write tid 3 to smithi16234619-9 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1522: finishing write tid 1 to smithi16234619-36 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: finishing write tid 4 to smithi16234619-9 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1522: finishing write tid 2 to smithi16234619-36 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1522: finishing write tid 3 to smithi16234619-36 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: finishing write tid 5 to smithi16234619-9 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: oid 9 updating version 0 to 537 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: oid 9 version 537 is already newer than 535 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: oid 9 updating version 537 to 538 2024-09-16T15:07:10.801 INFO:tasks.rados.rados.0.smithi162.stdout:1520: oid 9 version 538 is already newer than 536 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 538 (ObjNum 562 snap 143 seq_num 562) dirty exists 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1520: left oid 9 (ObjNum 562 snap 143 seq_num 562) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1522: finishing write tid 4 to smithi16234619-36 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1520: done (6 left) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1524: done (5 left) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1526: done (4 left) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1527: done (3 left) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1529: done (2 left) 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1530: rollback oid 34 current snap is 143 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 137 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1522: finishing write tid 5 to smithi16234619-36 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1522: oid 36 updating version 0 to 526 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1522: oid 36 version 526 is already newer than 523 2024-09-16T15:07:10.802 INFO:tasks.rados.rados.0.smithi162.stdout:1522: oid 36 version 526 is already newer than 525 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1522: oid 36 version 526 is already newer than 524 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 526 (ObjNum 563 snap 143 seq_num 563) dirty exists 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1522: left oid 36 (ObjNum 563 snap 143 seq_num 563) 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: finishing write tid 1 to smithi16234619-31 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: finishing write tid 2 to smithi16234619-31 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: finishing write tid 3 to smithi16234619-31 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: finishing write tid 4 to smithi16234619-31 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: finishing write tid 5 to smithi16234619-31 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: oid 31 updating version 0 to 559 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: oid 31 updating version 559 to 561 2024-09-16T15:07:10.803 INFO:tasks.rados.rados.0.smithi162.stdout:1525: oid 31 version 561 is already newer than 558 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1525: oid 31 version 561 is already newer than 560 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 561 (ObjNum 565 snap 143 seq_num 565) dirty exists 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1525: left oid 31 (ObjNum 565 snap 143 seq_num 565) 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1522: done (2 left) 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1525: done (1 left) 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1531: rollback oid 25 current snap is 143 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 127 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1532: copy_from oid 30 from oid 32 current snap is 143 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1533: rollback oid 16 current snap is 143 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 127 2024-09-16T15:07:10.804 INFO:tasks.rados.rados.0.smithi162.stdout:1534: snap_remove snap 122 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1534: done (4 left) 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1535: read oid 43 snap -1 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1535: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1536: write (excl) oid 42 current snap is 143 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1532: finishing copy_from to smithi16234619-30 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1532: got expected ENOENT (src dne) 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1532: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1533: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:07:11.425 INFO:tasks.rados.rados.0.smithi162.stdout:1533: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1533: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 529 (ObjNum 444 snap 106 seq_num 444) dirty exists 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1530: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1530: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1530: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 536 (ObjNum 516 snap 135 seq_num 516) dirty exists 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1531: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1531: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1531: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 540 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:07:11.426 INFO:tasks.rados.rados.0.smithi162.stdout:1536: seq_num 567 ranges {729569=476305,1996349=505866,3004131=525438} 2024-09-16T15:07:11.431 INFO:tasks.rados.rados.0.smithi162.stdout:1536: writing smithi16234619-42 from 729569 to 1205874 tid 1 2024-09-16T15:07:11.439 INFO:tasks.rados.rados.0.smithi162.stdout:1536: writing smithi16234619-42 from 1996349 to 2502215 tid 2 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1536: writing smithi16234619-42 from 3004131 to 3529569 tid 3 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1530: done (5 left) 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1531: done (4 left) 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1532: done (3 left) 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1533: done (2 left) 2024-09-16T15:07:11.446 INFO:tasks.rados.rados.0.smithi162.stdout:1537: snap_remove snap 127 2024-09-16T15:07:12.438 INFO:tasks.rados.rados.0.smithi162.stdout:1537: done (2 left) 2024-09-16T15:07:12.438 INFO:tasks.rados.rados.0.smithi162.stdout:1538: rollback oid 9 current snap is 143 2024-09-16T15:07:12.438 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 9 to 133 2024-09-16T15:07:12.438 INFO:tasks.rados.rados.0.smithi162.stdout:1535: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: finishing write tid 1 to smithi16234619-42 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: finishing write tid 2 to smithi16234619-42 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: finishing write tid 3 to smithi16234619-42 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: finishing write tid 4 to smithi16234619-42 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: finishing write tid 5 to smithi16234619-42 2024-09-16T15:07:12.548 INFO:tasks.rados.rados.0.smithi162.stdout:1536: oid 42 updating version 0 to 664 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1536: oid 42 updating version 664 to 667 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1536: oid 42 version 667 is already newer than 665 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1536: oid 42 version 667 is already newer than 666 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 667 (ObjNum 567 snap 143 seq_num 567) dirty exists 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1536: left oid 42 (ObjNum 567 snap 143 seq_num 567) 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1535: done (2 left) 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1536: done (1 left) 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1539: write oid 43 current snap is 143 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1538: finishing rollback tid 0 to smithi16234619-9 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1538: finishing rollback tid 1 to smithi16234619-9 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 546 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:07:12.549 INFO:tasks.rados.rados.0.smithi162.stdout:1539: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2024-09-16T15:07:12.554 INFO:tasks.rados.rados.0.smithi162.stdout:1539: writing smithi16234619-43 from 746376 to 1297930 tid 1 2024-09-16T15:07:12.562 INFO:tasks.rados.rados.0.smithi162.stdout:1539: writing smithi16234619-43 from 2054831 to 2704355 tid 2 2024-09-16T15:07:12.567 INFO:tasks.rados.rados.0.smithi162.stdout:1539: writing smithi16234619-43 from 3315201 to 3546376 tid 3 2024-09-16T15:07:12.567 INFO:tasks.rados.rados.0.smithi162.stdout:1538: done (1 left) 2024-09-16T15:07:12.567 INFO:tasks.rados.rados.0.smithi162.stdout:1540: write oid 1 current snap is 143 2024-09-16T15:07:12.567 INFO:tasks.rados.rados.0.smithi162.stdout:1540: seq_num 569 ranges {763183=626803,1996960=793182,3226271=336912} 2024-09-16T15:07:12.574 INFO:tasks.rados.rados.0.smithi162.stdout:1540: writing smithi16234619-1 from 763183 to 1389986 tid 1 2024-09-16T15:07:12.582 INFO:tasks.rados.rados.0.smithi162.stdout:1540: writing smithi16234619-1 from 1996960 to 2790142 tid 2 2024-09-16T15:07:12.587 INFO:tasks.rados.rados.0.smithi162.stdout:1540: writing smithi16234619-1 from 3226271 to 3563183 tid 3 2024-09-16T15:07:12.587 INFO:tasks.rados.rados.0.smithi162.stdout:1541: snap_remove snap 141 2024-09-16T15:07:13.435 INFO:tasks.rados.rados.0.smithi162.stdout:1539: finishing write tid 1 to smithi16234619-43 2024-09-16T15:07:13.435 INFO:tasks.rados.rados.0.smithi162.stdout:1539: finishing write tid 2 to smithi16234619-43 2024-09-16T15:07:13.435 INFO:tasks.rados.rados.0.smithi162.stdout:1539: finishing write tid 3 to smithi16234619-43 2024-09-16T15:07:13.435 INFO:tasks.rados.rados.0.smithi162.stdout:1539: finishing write tid 4 to smithi16234619-43 2024-09-16T15:07:13.435 INFO:tasks.rados.rados.0.smithi162.stdout:1539: finishing write tid 5 to smithi16234619-43 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1539: oid 43 updating version 0 to 509 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1539: oid 43 updating version 509 to 511 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1539: oid 43 version 511 is already newer than 510 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1539: oid 43 version 511 is already newer than 508 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 511 (ObjNum 568 snap 143 seq_num 568) dirty exists 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1539: left oid 43 (ObjNum 568 snap 143 seq_num 568) 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: finishing write tid 1 to smithi16234619-1 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: finishing write tid 2 to smithi16234619-1 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: finishing write tid 3 to smithi16234619-1 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: finishing write tid 4 to smithi16234619-1 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: finishing write tid 5 to smithi16234619-1 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: oid 1 updating version 0 to 548 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: oid 1 updating version 548 to 549 2024-09-16T15:07:13.436 INFO:tasks.rados.rados.0.smithi162.stdout:1540: oid 1 updating version 549 to 550 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1540: oid 1 updating version 550 to 551 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 551 (ObjNum 569 snap 143 seq_num 569) dirty exists 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1540: left oid 1 (ObjNum 569 snap 143 seq_num 569) 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1539: done (2 left) 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1540: done (1 left) 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1541: done (0 left) 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1542: read oid 44 snap -1 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1542: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:07:13.437 INFO:tasks.rados.rados.0.smithi162.stdout:1543: snap_remove snap 133 2024-09-16T15:07:14.438 INFO:tasks.rados.rados.0.smithi162.stdout:1543: done (1 left) 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1544: read oid 7 snap 142 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1544: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1545: read oid 21 snap -1 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1545: expect (ObjNum 513 snap 135 seq_num 513) 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1546: read oid 49 snap -1 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1546: expect (ObjNum 549 snap 140 seq_num 549) 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1547: write oid 23 current snap is 143 2024-09-16T15:07:14.439 INFO:tasks.rados.rados.0.smithi162.stdout:1547: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2024-09-16T15:07:14.447 INFO:tasks.rados.rados.0.smithi162.stdout:1547: writing smithi16234619-23 from 779990 to 1482042 tid 1 2024-09-16T15:07:14.456 INFO:tasks.rados.rados.0.smithi162.stdout:1547: writing smithi16234619-23 from 2055442 to 2708635 tid 2 2024-09-16T15:07:14.463 INFO:tasks.rados.rados.0.smithi162.stdout:1547: writing smithi16234619-23 from 3253694 to 3579990 tid 3 2024-09-16T15:07:14.463 INFO:tasks.rados.rados.0.smithi162.stdout:1548: rollback oid 34 current snap is 143 2024-09-16T15:07:14.463 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 142 2024-09-16T15:07:14.463 INFO:tasks.rados.rados.0.smithi162.stdout:1542: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:07:14.709 INFO:tasks.rados.rados.0.smithi162.stdout:1544: expect (ObjNum 506 snap 131 seq_num 506) 2024-09-16T15:07:14.841 INFO:tasks.rados.rados.0.smithi162.stdout:1545: expect (ObjNum 513 snap 135 seq_num 513) 2024-09-16T15:07:14.921 INFO:tasks.rados.rados.0.smithi162.stdout:1546: expect (ObjNum 549 snap 140 seq_num 549) 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1548: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1548: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1548: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 565 (ObjNum 516 snap 135 seq_num 516) dirty exists 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1547: finishing write tid 1 to smithi16234619-23 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1542: done (5 left) 2024-09-16T15:07:15.059 INFO:tasks.rados.rados.0.smithi162.stdout:1544: done (4 left) 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1545: done (3 left) 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1546: done (2 left) 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1548: done (1 left) 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1549: rollback oid 29 current snap is 143 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 142 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: finishing write tid 2 to smithi16234619-23 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: finishing write tid 3 to smithi16234619-23 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: finishing write tid 4 to smithi16234619-23 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: finishing write tid 5 to smithi16234619-23 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: oid 23 updating version 0 to 131 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: oid 23 version 131 is already newer than 128 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: oid 23 version 131 is already newer than 129 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:1547: oid 23 version 131 is already newer than 130 2024-09-16T15:07:15.060 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 131 (ObjNum 570 snap 143 seq_num 570) dirty exists 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1547: left oid 23 (ObjNum 570 snap 143 seq_num 570) 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1547: done (1 left) 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1550: read oid 12 snap 142 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1550: expect (ObjNum 481 snap 118 seq_num 481) 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1551: rollback oid 15 current snap is 143 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 134 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1552: read oid 6 snap -1 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1552: expect deleted 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1553: delete oid 7 current snap is 143 2024-09-16T15:07:15.061 INFO:tasks.rados.rados.0.smithi162.stdout:1549: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:07:15.062 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 606 (ObjNum 305 snap 0 seq_num 2027920896) dirty dne 2024-09-16T15:07:15.063 INFO:tasks.rados.rados.0.smithi162.stdout:1549: done (4 left) 2024-09-16T15:07:15.063 INFO:tasks.rados.rados.0.smithi162.stdout:1552: done (3 left) 2024-09-16T15:07:15.063 INFO:tasks.rados.rados.0.smithi162.stdout:1553: done (2 left) 2024-09-16T15:07:15.064 INFO:tasks.rados.rados.0.smithi162.stdout:1554: snap_remove snap 137 2024-09-16T15:07:15.422 INFO:tasks.rados.rados.0.smithi162.stdout:1554: done (2 left) 2024-09-16T15:07:15.422 INFO:tasks.rados.rados.0.smithi162.stdout:1555: copy_from oid 20 from oid 17 current snap is 143 2024-09-16T15:07:15.422 INFO:tasks.rados.rados.0.smithi162.stdout:1550: expect (ObjNum 481 snap 118 seq_num 481) 2024-09-16T15:07:15.512 INFO:tasks.rados.rados.0.smithi162.stdout:1551: finishing rollback tid 0 to smithi16234619-15 2024-09-16T15:07:15.512 INFO:tasks.rados.rados.0.smithi162.stdout:1551: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:07:15.512 INFO:tasks.rados.rados.0.smithi162.stdout:1551: finishing rollback tid 2 to smithi16234619-15 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 552 (ObjNum 377 snap 89 seq_num 377) dirty exists 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1555: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1555: finishing copy_from to smithi16234619-20 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1550: done (2 left) 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1551: done (1 left) 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1555: done (0 left) 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1556: write (excl) oid 42 current snap is 143 2024-09-16T15:07:15.513 INFO:tasks.rados.rados.0.smithi162.stdout:1556: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2024-09-16T15:07:15.517 INFO:tasks.rados.rados.0.smithi162.stdout:1556: writing smithi16234619-42 from 413604 to 982507 tid 1 2024-09-16T15:07:15.523 INFO:tasks.rados.rados.0.smithi162.stdout:1556: writing smithi16234619-42 from 1488759 to 2145621 tid 2 2024-09-16T15:07:15.530 INFO:tasks.rados.rados.0.smithi162.stdout:1556: writing smithi16234619-42 from 2624893 to 3216602 tid 3 2024-09-16T15:07:15.530 INFO:tasks.rados.rados.0.smithi162.stdout:1557: rollback oid 18 current snap is 143 2024-09-16T15:07:15.530 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 134 2024-09-16T15:07:15.530 INFO:tasks.rados.rados.0.smithi162.stdout:1558: write oid 1 current snap is 143 2024-09-16T15:07:15.530 INFO:tasks.rados.rados.0.smithi162.stdout:1558: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2024-09-16T15:07:15.535 INFO:tasks.rados.rados.0.smithi162.stdout:1558: writing smithi16234619-1 from 430411 to 1074563 tid 1 2024-09-16T15:07:15.541 INFO:tasks.rados.rados.0.smithi162.stdout:1558: writing smithi16234619-1 from 1830888 to 2231408 tid 2 2024-09-16T15:07:15.548 INFO:tasks.rados.rados.0.smithi162.stdout:1558: writing smithi16234619-1 from 2935963 to 3338944 tid 3 2024-09-16T15:07:15.548 INFO:tasks.rados.rados.0.smithi162.stdout:1559: snap_create 2024-09-16T15:07:15.548 INFO:tasks.rados.rados.0.smithi162.stdout:1557: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:07:15.548 INFO:tasks.rados.rados.0.smithi162.stdout:1557: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:07:15.548 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 564 (ObjNum 129 snap 0 seq_num 1839915345) dirty dne 2024-09-16T15:07:15.558 INFO:tasks.rados.rados.0.smithi162.stdout:1556: finishing write tid 1 to smithi16234619-42 2024-09-16T15:07:15.558 INFO:tasks.rados.rados.0.smithi162.stdout:1556: finishing write tid 2 to smithi16234619-42 2024-09-16T15:07:15.559 INFO:tasks.rados.rados.0.smithi162.stdout:1556: finishing write tid 3 to smithi16234619-42 2024-09-16T15:07:15.559 INFO:tasks.rados.rados.0.smithi162.stdout:1556: finishing write tid 4 to smithi16234619-42 2024-09-16T15:07:15.559 INFO:tasks.rados.rados.0.smithi162.stdout:1556: finishing write tid 5 to smithi16234619-42 2024-09-16T15:07:15.559 INFO:tasks.rados.rados.0.smithi162.stdout:1556: oid 42 updating version 0 to 704 2024-09-16T15:07:15.559 INFO:tasks.rados.rados.0.smithi162.stdout:1556: oid 42 version 704 is already newer than 702 2024-09-16T15:07:15.560 INFO:tasks.rados.rados.0.smithi162.stdout:1556: oid 42 version 704 is already newer than 701 2024-09-16T15:07:15.560 INFO:tasks.rados.rados.0.smithi162.stdout:1556: oid 42 version 704 is already newer than 703 2024-09-16T15:07:15.560 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 704 (ObjNum 572 snap 143 seq_num 572) dirty exists 2024-09-16T15:07:15.560 INFO:tasks.rados.rados.0.smithi162.stdout:1556: left oid 42 (ObjNum 572 snap 143 seq_num 572) 2024-09-16T15:07:15.571 INFO:tasks.rados.rados.0.smithi162.stdout:1558: finishing write tid 1 to smithi16234619-1 2024-09-16T15:07:15.571 INFO:tasks.rados.rados.0.smithi162.stdout:1558: finishing write tid 2 to smithi16234619-1 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: finishing write tid 3 to smithi16234619-1 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: finishing write tid 4 to smithi16234619-1 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: finishing write tid 5 to smithi16234619-1 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: oid 1 updating version 0 to 560 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: oid 1 updating version 560 to 563 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: oid 1 version 563 is already newer than 562 2024-09-16T15:07:15.572 INFO:tasks.rados.rados.0.smithi162.stdout:1558: oid 1 version 563 is already newer than 561 2024-09-16T15:07:15.573 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 563 (ObjNum 573 snap 143 seq_num 573) dirty exists 2024-09-16T15:07:15.573 INFO:tasks.rados.rados.0.smithi162.stdout:1558: left oid 1 (ObjNum 573 snap 143 seq_num 573) 2024-09-16T15:07:16.864 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:07:16.864 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-16T15:07:16.864 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-16T15:07:16.865 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-16T15:07:16.868 INFO:tasks.ceph.mon.g:Started 2024-09-16T15:07:16.868 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:07:16.868 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-16T15:07:16.868 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-16T15:07:16.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-16T15:07:16.870 INFO:tasks.ceph.mon.i:Started 2024-09-16T15:07:16.870 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:07:16.871 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 10 monclient: get_monmap_and_config 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 10 monclient: build_initial_monmap 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 10 monclient: monmap: 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:16.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:16.979001+0000 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:16.979001+0000 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:16.980 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:16.981 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:16.981 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 1 Processor -- start 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 1 -- start start 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.978+0000 7f89f1e32640 10 monclient: init 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient: _add_conns ranks=[7,0,4,8,2,6,3,9,5,10,1] 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-i con 0x7f89ec136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:16.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f89ec09bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 0x7f89ec137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-c con 0x7f89ec09c400 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec073250 con 0x7f89ec09bfe0 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec09c880 con 0x7f89ec09c400 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec09b600 con 0x7f89ec136030 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89f1e32640 10 monclient(hunting): authenticate will time out at 1173.568377s 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eaffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f89ec09bfe0 0x7f89ec136c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:53258/0 (socket says 172.21.15.134:53258) 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eaffd640 1 -- 172.21.15.134:0/773957468 learned_addr learned my addr 172.21.15.134:0/773957468 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:16.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 0x7f89ec137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): get_auth_request con 0x7f89ec09c400 auth_method 0 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 10 monclient(hunting): get_auth_request con 0x7f89ec136030 auth_method 0 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:16.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 1 -- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 msgr2=0x7f89ec136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 1 -- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 msgr2=0x7f89ec136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient(hunting): handle_auth_done global_id 4918 payload 274 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 -- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 msgr2=0x7f89ec136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 -- 172.21.15.134:0/773957468 >> v1:172.21.15.134:6789/0 conn(0x7f89ec09bfe0 legacy=0x7f89ec136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 monclient: found mon.noname-c 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:16.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:16.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.979+0000 7f89ebfff640 1 -- 172.21.15.134:0/773957468 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0a5740 con 0x7f89ec09c400 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.982+0000 7f89ebfff640 1 -- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 msgr2=0x7f89ec137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.982+0000 7f89ebfff640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 0x7f89ec137370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009dc0 tx=0x7f89e402a4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1d30f83b7f2e934b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 1 -- 172.21.15.134:0/773957468 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89e4007bc0 con 0x7f89ec09c400 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:16.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: dump: 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:16.994 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 5 monclient: authenticate success, global_id 4918 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 1 -- 172.21.15.134:0/773957468 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e402d050 con 0x7f89ec09c400 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 4 set_mon_vals no callback set 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 1 -- 172.21.15.134:0/773957468 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89e4007d40 con 0x7f89ec09c400 2024-09-16T15:07:16.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89ea7fc640 10 monclient: dump: 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:16.996 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 10 monclient: shutdown 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 -- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 msgr2=0x7f89ec137370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 0x7f89ec137370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009dc0 tx=0x7f89e402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 -- 172.21.15.134:0/773957468 shutdown_connections 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f89ec09c400 0x7f89ec137370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 --2- 172.21.15.134:0/773957468 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f89ec136030 0x7f89ec136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:16.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 -- 172.21.15.134:0/773957468 >> 172.21.15.134:0/773957468 conn(0x7f89ec0a9ea0 msgr2=0x7f89ec0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 -- 172.21.15.134:0/773957468 shutdown_connections 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.983+0000 7f89f1e32640 1 -- 172.21.15.134:0/773957468 wait complete. 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.984+0000 7f89f1e32640 10 monclient: build_initial_monmap 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.984+0000 7f89f1e32640 10 monclient: monmap: 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:16.998 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.984+0000 7f89f1e32640 1 Processor -- start 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.984+0000 7f89f1e32640 1 -- start start 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.984+0000 7f89f1e32640 10 monclient: init 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient: _add_conns ranks=[6,5,2,0,8,1,3,7,4] 2024-09-16T15:07:16.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-d con 0x7f89ec09bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ec09c400 0x7f89ec1cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-h con 0x7f89ec09c400 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): picked mon.noname-b con 0x7f89ec136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0a8e90 con 0x7f89ec136030 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec1272d0 con 0x7f89ec09c400 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec125760 con 0x7f89ec09bfe0 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89f1e32640 10 monclient(hunting): authenticate will time out at 1173.574392s 2024-09-16T15:07:17.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:37398/0 (socket says 172.21.15.134:37398) 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:39304/0 (socket says 172.21.15.134:39304) 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 1 -- 172.21.15.134:0/2518878539 learned_addr learned my addr 172.21.15.134:0/2518878539 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): get_auth_request con 0x7f89ec136030 auth_method 0 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): get_auth_request con 0x7f89ec09bfe0 auth_method 0 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:17.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eaffd640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ec09c400 0x7f89ec1cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eaffd640 10 monclient(hunting): get_auth_request con 0x7f89ec09c400 auth_method 0 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient(hunting): handle_auth_done global_id 34512 payload 824 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 -- 172.21.15.134:0/2518878539 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ec09c400 msgr2=0x7f89ec1cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ec09c400 0x7f89ec1cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 -- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 msgr2=0x7f89ec123580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 monclient: found mon.noname-d 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89eb7fe640 1 -- 172.21.15.134:0/2518878539 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009a70 con 0x7f89ec09bfe0 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.985+0000 7f89ebfff640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89eb7fe640 1 -- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 msgr2=0x7f89ec1cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89eb7fe640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d5a0 tx=0x7f89dc00da60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89dc012c70 con 0x7f89ec09bfe0 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: dump: 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:17.003 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:17.004 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: _finish_auth 0 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc012df0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 5 monclient: authenticate success, global_id 34512 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 10 monclient: _renew_subs 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec09f8f0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f89dc0148a0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:17.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89d3fff640 10 monclient: dump: 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:17.006 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 10 monclient: _renew_subs 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89e9ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.988+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec125250 con 0x7f89ec09bfe0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f89dc014430 con 0x7f89ec09bfe0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89f1e32640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89f1e32640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89f1e32640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89d3fff640 1 --2- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 0x7f89d803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ec061ae0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(311..311 src has 1..311) v4 ==== 4067+0+0 (secure 0 0 0) 0x7f89dc04a6d0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89eaffd640 1 --2- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 0x7f89d803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.989+0000 7f89eaffd640 10 monclient: get_auth_request con 0x7f89d80388b0 auth_method 0 2024-09-16T15:07:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.992+0000 7f89eaffd640 1 --2- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 0x7f89d803ad70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89ec124ce0 tx=0x7f89e0009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.994+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f89ec061ae0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.994+0000 7f89d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:17.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:16.994+0000 7f89d3fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:17.050 INFO:tasks.rados.rados.0.smithi162.stdout:1556: done (3 left) 2024-09-16T15:07:17.050 INFO:tasks.rados.rados.0.smithi162.stdout:1557: done (2 left) 2024-09-16T15:07:17.050 INFO:tasks.rados.rados.0.smithi162.stdout:1558: done (1 left) 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1559: done (0 left) 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1560: read oid 5 snap -1 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1560: expect deleted 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1561: read oid 27 snap -1 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1561: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1562: read oid 15 snap -1 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1562: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1563: read oid 41 snap 143 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1563: expect (ObjNum 553 snap 141 seq_num 553) 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1564: write (excl) oid 22 current snap is 144 2024-09-16T15:07:17.051 INFO:tasks.rados.rados.0.smithi162.stdout:1564: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2024-09-16T15:07:17.057 INFO:tasks.rados.rados.0.smithi162.stdout:1564: writing smithi16234619-22 from 447218 to 1166619 tid 1 2024-09-16T15:07:17.065 INFO:tasks.rados.rados.0.smithi162.stdout:1564: writing smithi16234619-22 from 1889370 to 2549901 tid 2 2024-09-16T15:07:17.071 INFO:tasks.rados.rados.0.smithi162.stdout:1564: writing smithi16234619-22 from 2963386 to 3577639 tid 3 2024-09-16T15:07:17.071 INFO:tasks.rados.rados.0.smithi162.stdout:1565: delete oid 39 current snap is 144 2024-09-16T15:07:17.071 INFO:tasks.rados.rados.0.smithi162.stdout:1561: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:07:17.156 INFO:tasks.rados.rados.0.smithi162.stdout:1562: expect (ObjNum 377 snap 89 seq_num 377) 2024-09-16T15:07:17.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:17.250+0000 7f89f1e32640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:17.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:17.250+0000 7f89f1e32640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:17.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:17.250+0000 7f89f1e32640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:17.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:17.250+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f89ec061ae0 con 0x7f89ec09bfe0 2024-09-16T15:07:17.290 INFO:tasks.rados.rados.0.smithi162.stdout:1563: expect (ObjNum 553 snap 141 seq_num 553) 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: finishing write tid 1 to smithi16234619-22 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: finishing write tid 2 to smithi16234619-22 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: finishing write tid 3 to smithi16234619-22 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: finishing write tid 4 to smithi16234619-22 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: finishing write tid 5 to smithi16234619-22 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: oid 22 updating version 0 to 343 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: oid 22 version 343 is already newer than 341 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: oid 22 updating version 343 to 344 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:1564: oid 22 version 344 is already newer than 342 2024-09-16T15:07:17.392 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 344 (ObjNum 574 snap 144 seq_num 574) dirty exists 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1564: left oid 22 (ObjNum 574 snap 144 seq_num 574) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1560: done (5 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1561: done (4 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1562: done (3 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1563: done (2 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1564: done (1 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1565: done (0 left) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1566: read oid 3 snap 142 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1566: expect (ObjNum 533 snap 139 seq_num 533) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1567: read oid 17 snap -1 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1567: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1568: read oid 6 snap -1 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1568: expect deleted 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1569: write (excl) oid 47 current snap is 144 2024-09-16T15:07:17.393 INFO:tasks.rados.rados.0.smithi162.stdout:1569: seq_num 576 ranges {480832=469899,1606334=664200,3018232=662600} 2024-09-16T15:07:17.397 INFO:tasks.rados.rados.0.smithi162.stdout:1569: writing smithi16234619-47 from 480832 to 950731 tid 1 2024-09-16T15:07:17.405 INFO:tasks.rados.rados.0.smithi162.stdout:1569: writing smithi16234619-47 from 1606334 to 2270534 tid 2 2024-09-16T15:07:17.413 INFO:tasks.rados.rados.0.smithi162.stdout:1569: writing smithi16234619-47 from 3018232 to 3680832 tid 3 2024-09-16T15:07:17.413 INFO:tasks.rados.rados.0.smithi162.stdout:1570: copy_from oid 29 from oid 33 current snap is 144 2024-09-16T15:07:17.413 INFO:tasks.rados.rados.0.smithi162.stdout:1567: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:17.553 INFO:tasks.rados.rados.0.smithi162.stdout:1566: expect (ObjNum 533 snap 139 seq_num 533) 2024-09-16T15:07:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:1569: finishing write tid 1 to smithi16234619-47 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: finishing write tid 2 to smithi16234619-47 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: finishing write tid 3 to smithi16234619-47 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: finishing write tid 4 to smithi16234619-47 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: finishing write tid 5 to smithi16234619-47 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: oid 47 updating version 0 to 513 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: oid 47 updating version 513 to 514 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: oid 47 version 514 is already newer than 512 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: oid 47 updating version 514 to 515 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 515 (ObjNum 576 snap 144 seq_num 576) dirty exists 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1569: left oid 47 (ObjNum 576 snap 144 seq_num 576) 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1570: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1570: finishing copy_from to smithi16234619-29 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 182 (ObjNum 557 snap 141 seq_num 557) dirty exists 2024-09-16T15:07:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:1566: done (4 left) 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1567: done (3 left) 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1568: done (2 left) 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1569: done (1 left) 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1570: done (0 left) 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1571: write (excl) oid 45 current snap is 144 2024-09-16T15:07:17.650 INFO:tasks.rados.rados.0.smithi162.stdout:1571: seq_num 577 ranges {497639=545148,1548463=407858,2529302=564422,3528143=169496} 2024-09-16T15:07:17.653 INFO:tasks.rados.rados.0.smithi162.stdout:1571: writing smithi16234619-45 from 497639 to 1042787 tid 1 2024-09-16T15:07:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:1571: writing smithi16234619-45 from 1548463 to 1956321 tid 2 2024-09-16T15:07:17.665 INFO:tasks.rados.rados.0.smithi162.stdout:1571: writing smithi16234619-45 from 2529302 to 3093724 tid 3 2024-09-16T15:07:17.668 INFO:tasks.rados.rados.0.smithi162.stdout:1571: writing smithi16234619-45 from 3528143 to 3697639 tid 4 2024-09-16T15:07:17.668 INFO:tasks.rados.rados.0.smithi162.stdout:1572: write oid 46 current snap is 144 2024-09-16T15:07:17.668 INFO:tasks.rados.rados.0.smithi162.stdout:1572: seq_num 578 ranges {514446=736750,1723298=667869,3073078=641368} 2024-09-16T15:07:17.675 INFO:tasks.rados.rados.0.smithi162.stdout:1572: writing smithi16234619-46 from 514446 to 1251196 tid 1 2024-09-16T15:07:17.681 INFO:tasks.rados.rados.0.smithi162.stdout:1572: writing smithi16234619-46 from 1723298 to 2391167 tid 2 2024-09-16T15:07:17.689 INFO:tasks.rados.rados.0.smithi162.stdout:1572: writing smithi16234619-46 from 3073078 to 3714446 tid 3 2024-09-16T15:07:17.689 INFO:tasks.rados.rados.0.smithi162.stdout:1573: write (excl) oid 13 current snap is 144 2024-09-16T15:07:17.689 INFO:tasks.rados.rados.0.smithi162.stdout:1573: seq_num 579 ranges {531253=411999,1381780=411527,2300501=586966,3376974=354279} 2024-09-16T15:07:17.694 INFO:tasks.rados.rados.0.smithi162.stdout:1573: writing smithi16234619-13 from 531253 to 943252 tid 1 2024-09-16T15:07:17.698 INFO:tasks.rados.rados.0.smithi162.stdout:1573: writing smithi16234619-13 from 1381780 to 1793307 tid 2 2024-09-16T15:07:17.704 INFO:tasks.rados.rados.0.smithi162.stdout:1573: writing smithi16234619-13 from 2300501 to 2887467 tid 3 2024-09-16T15:07:17.710 INFO:tasks.rados.rados.0.smithi162.stdout:1573: writing smithi16234619-13 from 3376974 to 3731253 tid 4 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1574: copy_from oid 21 from oid 24 current snap is 144 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 1 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 2 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 3 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 4 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 5 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: finishing write tid 6 to smithi16234619-45 2024-09-16T15:07:17.711 INFO:tasks.rados.rados.0.smithi162.stdout:1571: oid 45 updating version 0 to 679 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: oid 45 version 679 is already newer than 677 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: oid 45 updating version 679 to 680 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: oid 45 updating version 680 to 681 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: oid 45 version 681 is already newer than 678 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 681 (ObjNum 577 snap 144 seq_num 577) dirty exists 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: left oid 45 (ObjNum 577 snap 144 seq_num 577) 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1571: done (3 left) 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1575: snap_create 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1574: finishing copy_from to smithi16234619-21 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1574: got expected ENOENT (src dne) 2024-09-16T15:07:17.712 INFO:tasks.rados.rados.0.smithi162.stdout:1574: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:07:17.721 INFO:tasks.rados.rados.0.smithi162.stdout:1572: finishing write tid 1 to smithi16234619-46 2024-09-16T15:07:17.721 INFO:tasks.rados.rados.0.smithi162.stdout:1572: finishing write tid 2 to smithi16234619-46 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: finishing write tid 3 to smithi16234619-46 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: finishing write tid 4 to smithi16234619-46 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: finishing write tid 5 to smithi16234619-46 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: oid 46 updating version 0 to 251 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: oid 46 updating version 251 to 252 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: oid 46 version 252 is already newer than 250 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: oid 46 updating version 252 to 253 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 253 (ObjNum 578 snap 144 seq_num 578) dirty exists 2024-09-16T15:07:17.722 INFO:tasks.rados.rados.0.smithi162.stdout:1572: left oid 46 (ObjNum 578 snap 144 seq_num 578) 2024-09-16T15:07:17.732 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 1 to smithi16234619-13 2024-09-16T15:07:17.735 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 2 to smithi16234619-13 2024-09-16T15:07:17.735 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 3 to smithi16234619-13 2024-09-16T15:07:17.735 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 4 to smithi16234619-13 2024-09-16T15:07:17.737 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 5 to smithi16234619-13 2024-09-16T15:07:17.738 INFO:tasks.rados.rados.0.smithi162.stdout:1573: finishing write tid 6 to smithi16234619-13 2024-09-16T15:07:17.738 INFO:tasks.rados.rados.0.smithi162.stdout:1573: oid 13 updating version 0 to 710 2024-09-16T15:07:17.738 INFO:tasks.rados.rados.0.smithi162.stdout:1573: oid 13 updating version 710 to 712 2024-09-16T15:07:17.738 INFO:tasks.rados.rados.0.smithi162.stdout:1573: oid 13 version 712 is already newer than 708 2024-09-16T15:07:17.738 INFO:tasks.rados.rados.0.smithi162.stdout:1573: oid 13 version 712 is already newer than 709 2024-09-16T15:07:17.739 INFO:tasks.rados.rados.0.smithi162.stdout:1573: oid 13 version 712 is already newer than 711 2024-09-16T15:07:17.739 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 712 (ObjNum 579 snap 144 seq_num 579) dirty exists 2024-09-16T15:07:17.739 INFO:tasks.rados.rados.0.smithi162.stdout:1573: left oid 13 (ObjNum 579 snap 144 seq_num 579) 2024-09-16T15:07:19.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:19.985+0000 7f89d2ffd640 10 monclient: tick 2024-09-16T15:07:19.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:19.985+0000 7f89d2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:19.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:19.985+0000 7f89d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:19.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:19.985+0000 7f89d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:20.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:20.985+0000 7f89d2ffd640 10 monclient: tick 2024-09-16T15:07:20.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:20.985+0000 7f89d2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:20.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:20.985+0000 7f89d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:21.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:21.986+0000 7f89d2ffd640 10 monclient: tick 2024-09-16T15:07:21.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:21.986+0000 7f89d2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:21.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:21.986+0000 7f89d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:22.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.216+0000 7f89d3fff640 1 -- 172.21.15.134:0/2518878539 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f89ec061ae0 con 0x7f89ec09bfe0 2024-09-16T15:07:22.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.216+0000 7f89d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:22.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.216+0000 7f89d3fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:22.218 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":50,"quorum":[0,1,2,4,5,6,8],"quorum_names":["a","f","b","c","h","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:22.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 msgr2=0x7f89d803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 0x7f89d803ad70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89ec124ce0 tx=0x7f89e0009040 comp rx=0 tx=0).stop 2024-09-16T15:07:22.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 10 monclient: shutdown 2024-09-16T15:07:22.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:22.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 msgr2=0x7f89ec1cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d5a0 tx=0x7f89dc00da60 comp rx=0 tx=0).stop 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 shutdown_connections 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> v2:172.21.15.162:6800/980547263 conn(0x7f89d80388b0 0x7f89d803ad70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89ec136030 0x7f89ec123580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f89ec09c400 0x7f89ec1cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 --2- 172.21.15.134:0/2518878539 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f89ec09bfe0 0x7f89ec1cb4c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.219+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 >> 172.21.15.134:0/2518878539 conn(0x7f89ec0a9ea0 msgr2=0x7f89ec09e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.220+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 shutdown_connections 2024-09-16T15:07:22.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:22.220+0000 7f89f1e32640 1 -- 172.21.15.134:0/2518878539 wait complete. 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1572: done (3 left) 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1573: done (2 left) 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1574: done (1 left) 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1575: done (0 left) 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1576: copy_from oid 22 from oid 40 current snap is 145 2024-09-16T15:07:22.566 INFO:tasks.rados.rados.0.smithi162.stdout:1577: read oid 5 snap -1 2024-09-16T15:07:22.567 INFO:tasks.rados.rados.0.smithi162.stdout:1577: expect deleted 2024-09-16T15:07:22.567 INFO:tasks.rados.rados.0.smithi162.stdout:1578: read oid 13 snap -1 2024-09-16T15:07:22.567 INFO:tasks.rados.rados.0.smithi162.stdout:1578: expect (ObjNum 579 snap 144 seq_num 579) 2024-09-16T15:07:22.567 INFO:tasks.rados.rados.0.smithi162.stdout:1579: snap_remove snap 142 2024-09-16T15:07:25.235 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:25.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.342+0000 7fe8e8b0f640 10 monclient: get_monmap_and_config 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.342+0000 7fe8e8b0f640 10 monclient: build_initial_monmap 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.342+0000 7fe8e8b0f640 10 monclient: monmap: 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:25.343620+0000 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:25.343620+0000 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:25.344 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:25.345 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:25.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 1 Processor -- start 2024-09-16T15:07:25.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 1 -- start start 2024-09-16T15:07:25.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient: init 2024-09-16T15:07:25.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient: _add_conns ranks=[0,8,1,5,4,3,10,7,2,9,6] 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe8e4136080 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-e con 0x7fe8e40a5700 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.343+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe8e413c610 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e4073250 con 0x7fe8e4136080 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e409bb00 con 0x7fe8e413c610 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e409b4b0 con 0x7fe8e40a5700 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:25.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e8b0f640 10 monclient(hunting): authenticate will time out at 1181.933036s 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e259c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe8e4136080 0x7fe8e4136460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:53268/0 (socket says 172.21.15.134:53268) 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 learned_addr learned my addr 172.21.15.134:0/2144321976 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8e40a5700 auth_method 0 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.344+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.346+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 >> v1:172.21.15.134:6789/0 conn(0x7fe8e4136080 legacy=0x7fe8e4136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:25.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.346+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 >> v1:172.21.15.162:6789/0 conn(0x7fe8e413c610 legacy=0x7fe8e413ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:25.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8e40a5700 auth_method 0 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:25.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:25.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8e40a5700 auth_method 0 2024-09-16T15:07:25.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:25.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:25.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:25.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8e40a5700 auth_method 0 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:26.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:26.747+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): tick 2024-09-16T15:07:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:07:28.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 monclient(hunting): continuing hunt 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 msgr2=0x7fe8e413c0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e40a5700 0x7fe8e413c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 >> v1:172.21.15.162:6789/0 conn(0x7fe8e413c610 legacy=0x7fe8e413ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 >> v1:172.21.15.134:6789/0 conn(0x7fe8e4136080 legacy=0x7fe8e4136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient: _add_conns ranks=[2,6,9,7,10,3,5,4] 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): picked mon.noname-b con 0x7fe8c4003940 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): picked mon.noname-d con 0x7fe8c4006460 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): picked mon.noname-a con 0x7fe8c4008fc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:28.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e409b4b0 con 0x7fe8c4008fc0 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c400bb50 con 0x7fe8c4003940 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 1 -- 172.21.15.134:0/2144321976 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c400bd10 con 0x7fe8c4006460 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e0d99640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8c4006460 auth_method 0 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request con 0x7fe8c4008fc0 auth_method 0 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 10 monclient(hunting): get_auth_request con 0x7fe8c4003940 auth_method 0 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 msgr2=0x7fe8c400b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 msgr2=0x7fe8c400b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 msgr2=0x7fe8c4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 msgr2=0x7fe8c4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:28.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.344+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request con 0x7fe8c4008fc0 auth_method 0 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8c4006460 auth_method 0 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 10 monclient(hunting): get_auth_request con 0x7fe8c4003940 auth_method 0 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 msgr2=0x7fe8c400b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 msgr2=0x7fe8c400b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 msgr2=0x7fe8c4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 msgr2=0x7fe8c4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:28.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.545+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:28.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): get_auth_request con 0x7fe8c4003940 auth_method 0 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8c4006460 auth_method 0 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request con 0x7fe8c4008fc0 auth_method 0 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_done global_id 34521 payload 274 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 msgr2=0x7fe8c4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 msgr2=0x7fe8c400b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:28.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 monclient: found mon.noname-d 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e4136b70 con 0x7fe8c4006460 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e259c640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8c4003940 0x7fe8c4005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/2144321976 >> v2:172.21.15.134:3300/0 conn(0x7fe8c4008fc0 0x7fe8c400b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.946+0000 7fe8e2d9d640 1 -- 172.21.15.134:0/2144321976 reap_dead start 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8dbfff640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc006fd0 tx=0x7fe8cc02bad0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=4a46b74d283eb60b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 1 -- 172.21.15.134:0/2144321976 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe8cc02fae0 con 0x7fe8c4006460 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: dump: 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:28.953 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: _finish_auth 0 2024-09-16T15:07:28.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e8b0f640 5 monclient: authenticate success, global_id 34521 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e8b0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 1 -- 172.21.15.134:0/2144321976 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc02fc80 con 0x7fe8c4006460 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e8b0f640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e8b0f640 4 set_mon_vals no callback set 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 1 -- 172.21.15.134:0/2144321976 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe8cc02fdf0 con 0x7fe8c4006460 2024-09-16T15:07:28.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.949+0000 7fe8e1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:28.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e1d9b640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:28.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e1d9b640 10 monclient: dump: 2024-09-16T15:07:28.956 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:28.956 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:28.956 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 10 monclient: shutdown 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 msgr2=0x7fe8c40088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc006fd0 tx=0x7fe8cc02bad0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/2144321976 shutdown_connections 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/2144321976 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe8c4006460 0x7fe8c40088f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/2144321976 >> 172.21.15.134:0/2144321976 conn(0x7fe8e40a99a0 msgr2=0x7fe8e40a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/2144321976 shutdown_connections 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.950+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/2144321976 wait complete. 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.951+0000 7fe8e8b0f640 10 monclient: build_initial_monmap 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient: monmap: 2024-09-16T15:07:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:28.960 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 Processor -- start 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 -- start start 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient: init 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient: _add_conns ranks=[4,8,1,2,3,0,6,5,7] 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-c con 0x7fe8c4006460 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 0x7fe8e4139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-e con 0x7fe8e4134cd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e4139f90 0x7fe8e413a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): picked mon.noname-f con 0x7fe8e4139f90 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4135560 con 0x7fe8e4139f90 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e413bdd0 con 0x7fe8c4006460 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4137ad0 con 0x7fe8e4134cd0 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.952+0000 7fe8e8b0f640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e8b0f640 10 monclient(hunting): authenticate will time out at 1185.542011s 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 0x7fe8e4139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:35134/0 (socket says 172.21.15.134:35134) 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 -- 172.21.15.134:0/496692512 learned_addr learned my addr 172.21.15.134:0/496692512 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request con 0x7fe8e4134cd0 auth_method 0 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): get_auth_request con 0x7fe8c4006460 auth_method 0 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e4139f90 0x7fe8e413a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request con 0x7fe8e4139f90 auth_method 0 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient(hunting): handle_auth_done global_id 4927 payload 824 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 -- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 msgr2=0x7fe8e4139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 0x7fe8e4139a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 -- 172.21.15.134:0/496692512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e4139f90 msgr2=0x7fe8e413a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e4139f90 0x7fe8e413a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 monclient: found mon.noname-c 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8e259c640 1 -- 172.21.15.134:0/496692512 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc00e040 con 0x7fe8c4006460 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.953+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 0x7fe8e4139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.955+0000 7fe8e259c640 1 -- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 msgr2=0x7fe8e4134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.955+0000 7fe8e259c640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007440 tx=0x7fe8d40015a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe8d401b070 con 0x7fe8c4006460 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: dump: 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:28.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 5 monclient: authenticate success, global_id 4927 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4006530 con 0x7fe8c4006460 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe8d4004c50 con 0x7fe8c4006460 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 10 monclient: _renew_subs 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e413c350 con 0x7fe8c4006460 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8daffd640 10 monclient: dump: 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:28.971 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:28.972 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e159a640 4 set_mon_vals no callback set 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 10 monclient: _renew_subs 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.956+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e4120d40 con 0x7fe8c4006460 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe8d40047e0 con 0x7fe8c4006460 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8e8b0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8e8b0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8e8b0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e41211b0 con 0x7fe8c4006460 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8daffd640 1 --2- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 0x7fe8c403c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(313..313 src has 1..313) v4 ==== 3995+0+0 (secure 0 0 0) 0x7fe8d404a450 con 0x7fe8c4006460 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 0x7fe8c403c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.957+0000 7fe8dbfff640 10 monclient: get_auth_request con 0x7fe8c4039e90 auth_method 0 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.960+0000 7fe8dbfff640 1 --2- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 0x7fe8c403c350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4008f90 tx=0x7fe8cc02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.962+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe8d4004a90 con 0x7fe8c4006460 2024-09-16T15:07:28.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.962+0000 7fe8daffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:28.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:28.962+0000 7fe8daffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.085 INFO:tasks.rados.rados.0.smithi162.stdout:1579: done (3 left) 2024-09-16T15:07:29.085 INFO:tasks.rados.rados.0.smithi162.stdout:1580: snap_create 2024-09-16T15:07:29.085 INFO:tasks.rados.rados.0.smithi162.stdout:1578: expect (ObjNum 579 snap 144 seq_num 579) 2024-09-16T15:07:29.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8e8b0f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:29.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8e8b0f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:29.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8e8b0f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:29.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe8e409b790 con 0x7fe8c4006460 2024-09-16T15:07:29.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8daffd640 1 -- 172.21.15.134:0/496692512 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7fe8d4014b30 con 0x7fe8c4006460 2024-09-16T15:07:29.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8daffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:29.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.201+0000 7fe8daffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:29.203 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 msgr2=0x7fe8c403c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 0x7fe8c403c350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4008f90 tx=0x7fe8cc02f000 comp rx=0 tx=0).stop 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 10 monclient: shutdown 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 msgr2=0x7fe8e4134790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007440 tx=0x7fe8d40015a0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 shutdown_connections 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> v2:172.21.15.162:6800/980547263 conn(0x7fe8c4039e90 0x7fe8c403c350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8e4139f90 0x7fe8e413a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe8e4134cd0 0x7fe8e4139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 --2- 172.21.15.134:0/496692512 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe8c4006460 0x7fe8e4134790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 >> 172.21.15.134:0/496692512 conn(0x7fe8e40a99a0 msgr2=0x7fe8e40a8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 shutdown_connections 2024-09-16T15:07:29.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.203+0000 7fe8e8b0f640 1 -- 172.21.15.134:0/496692512 wait complete. 2024-09-16T15:07:29.215 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:07:29.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:07:29.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:07:29.247 INFO:tasks.rados.rados.0.smithi162.stdout:1576: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:07:29.247 INFO:tasks.rados.rados.0.smithi162.stdout:1576: finishing copy_from to smithi16234619-22 2024-09-16T15:07:29.247 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 257 (ObjNum 492 snap 121 seq_num 492) dirty exists 2024-09-16T15:07:29.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.314+0000 7f4248565640 10 monclient: get_monmap_and_config 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.314+0000 7f4248565640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.314+0000 7f4248565640 10 monclient: monmap: 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:29.315520+0000 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:29.315520+0000 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.316 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:29.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 1 Processor -- start 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 1 -- start start 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 10 monclient: init 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 10 monclient: _add_conns ranks=[8,6,1,7,9,10,4,0,2,5,3] 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.315+0000 7f4248565640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): picked mon.noname-e con 0x7f42400a3260 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 0x7f42400a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): picked mon.noname-d con 0x7f424009bbd0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f424009fcb0 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4240073250 con 0x7f424009fcb0 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f424009dae0 con 0x7f424009bbd0 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f424009de70 con 0x7f42400a3260 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4248565640 10 monclient(hunting): authenticate will time out at 1185.905202s 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:51282/0 (socket says 172.21.15.134:51282) 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 1 -- 172.21.15.134:0/2247983405 learned_addr learned my addr 172.21.15.134:0/2247983405 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): get_auth_request con 0x7f42400a3260 auth_method 0 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 0x7f42400a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): get_auth_request con 0x7f424009bbd0 auth_method 0 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient(hunting): handle_auth_done global_id 34530 payload 274 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 -- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 msgr2=0x7f42400a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 -- 172.21.15.134:0/2247983405 >> v1:172.21.15.162:6789/0 conn(0x7f424009fcb0 legacy=0x7f424013a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 monclient: found mon.noname-d 2024-09-16T15:07:29.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:29.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4245b00640 1 -- 172.21.15.134:0/2247983405 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400a3640 con 0x7f424009bbd0 2024-09-16T15:07:29.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.316+0000 7f4246301640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4245b00640 1 -- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 msgr2=0x7f42400a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4245b00640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 0x7f42400a1b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4230010bc0 tx=0x7f423002e5b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=10bd28272239e318 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 1 -- 172.21.15.134:0/2247983405 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4230033b90 con 0x7f424009bbd0 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: dump: 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.326 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 1 -- 172.21.15.134:0/2247983405 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4230031050 con 0x7f424009bbd0 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4248565640 5 monclient: authenticate success, global_id 34530 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4248565640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 1 -- 172.21.15.134:0/2247983405 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4230033d10 con 0x7f424009bbd0 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4248565640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f4248565640 4 set_mon_vals no callback set 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.319+0000 7f42452ff640 10 monclient: dump: 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.328 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 10 monclient: shutdown 2024-09-16T15:07:29.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 -- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 msgr2=0x7f42400a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 0x7f42400a1b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4230010bc0 tx=0x7f423002e5b0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 -- 172.21.15.134:0/2247983405 shutdown_connections 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f424009bbd0 0x7f42400a1b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 --2- 172.21.15.134:0/2247983405 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f42400a3260 0x7f42400a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 -- 172.21.15.134:0/2247983405 >> 172.21.15.134:0/2247983405 conn(0x7f42400a2740 msgr2=0x7f42400a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 -- 172.21.15.134:0/2247983405 shutdown_connections 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.320+0000 7f4248565640 1 -- 172.21.15.134:0/2247983405 wait complete. 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.321+0000 7f4248565640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.321+0000 7f4248565640 10 monclient: monmap: 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:29.330 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.321+0000 7f4248565640 1 Processor -- start 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 -- start start 2024-09-16T15:07:29.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient: init 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient: _add_conns ranks=[3,5,1,0,4,6,8,2,7] 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): picked mon.noname-g con 0x7f424009bbd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 0x7f42401c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): picked mon.noname-h con 0x7f424009fcb0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): picked mon.noname-f con 0x7f42400a3260 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a8a70 con 0x7f42400a3260 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a6650 con 0x7f424009bbd0 2024-09-16T15:07:29.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42400a1da0 con 0x7f424009fcb0 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4248565640 10 monclient(hunting): authenticate will time out at 1185.911722s 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 0x7f42401c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 0x7f42401c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:53426/0 (socket says 172.21.15.134:53426) 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:58862/0 (socket says 172.21.15.134:58862) 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 1 -- 172.21.15.134:0/1481834670 learned_addr learned my addr 172.21.15.134:0/1481834670 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:36420/0 (socket says 172.21.15.134:36420) 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 10 monclient(hunting): get_auth_request con 0x7f424009fcb0 auth_method 0 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4245b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 10 monclient(hunting): get_auth_request con 0x7f424009bbd0 auth_method 0 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246301640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 10 monclient(hunting): get_auth_request con 0x7f42400a3260 auth_method 0 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.322+0000 7f4246b02640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246301640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 msgr2=0x7f42401c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246301640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 msgr2=0x7f42401c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246301640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246301640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient(hunting): handle_auth_done global_id 44290 payload 824 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 msgr2=0x7f42401c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 0x7f42401c3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 msgr2=0x7f42401c6fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f424009bbd0 0x7f42401c6fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 monclient: found mon.noname-f 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c009b30 con 0x7f42400a3260 2024-09-16T15:07:29.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.323+0000 7f4245b00640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f424009fcb0 0x7f42401c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.325+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 msgr2=0x7f42401c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.325+0000 7f4246b02640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f423800c410 tx=0x7f423800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4238011c10 con 0x7f42400a3260 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: dump: 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.339 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238011dd0 con 0x7f42400a3260 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4238012890 con 0x7f42400a3260 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4236ffd640 10 monclient: dump: 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 5 monclient: authenticate success, global_id 44290 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 10 monclient: _renew_subs 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4244afe640 4 set_mon_vals no callback set 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42300107f0 con 0x7f42400a3260 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 10 monclient: _renew_subs 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.326+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42401c7a90 con 0x7f42400a3260 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.327+0000 7f4246b02640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f423800c410 tx=0x7f423800c8d0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f4238049c70 osd_map(314..314 src has 1..314) v4 2024-09-16T15:07:29.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.327+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4238012420 con 0x7f42400a3260 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.327+0000 7f4236ffd640 1 --2- 172.21.15.134:0/1481834670 >> v2:172.21.15.162:6800/980547263 conn(0x7f4224038880 0x7f422403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.327+0000 7f4246301640 1 --2- 172.21.15.134:0/1481834670 >> v2:172.21.15.162:6800/980547263 conn(0x7f4224038880 0x7f422403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.327+0000 7f4246301640 10 monclient: get_auth_request con 0x7f4224038880 auth_method 0 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4248565640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4248565640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 0x7f42401c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424009ba10 con 0x7f42401a8140 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 0x7f42401c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient: get_auth_request con 0x7f42401a8140 auth_method 0 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 10 monclient(hunting): handle_auth_done global_id 5013 payload 824 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.329+0000 7f4245b00640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4240061ae0 con 0x7f42401a8140 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.330+0000 7f4246301640 1 --2- 172.21.15.134:0/1481834670 >> v2:172.21.15.162:6800/980547263 conn(0x7f4224038880 0x7f422403ad40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f423c009280 tx=0x7f423c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.331+0000 7f4245b00640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 msgr2=0x7f42401c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.331+0000 7f4245b00640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 0x7f42401c3fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42401a6480 tx=0x7f42300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4230003ea0 con 0x7f42401a8140 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 10 monclient: dump: 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.344 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.345 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4230003a30 con 0x7f42401a8140 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 msgr2=0x7f42401c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 0x7f42401c3fa0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42401a6480 tx=0x7f42300046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 msgr2=0x7f42401c3fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.332+0000 7f4236ffd640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401a8140 0x7f42401c3fa0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f42401a6480 tx=0x7f42300046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4248565640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4248565640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 0x7f42401c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4248565640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240073880 con 0x7f42401c35c0 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 0x7f42401c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient: get_auth_request con 0x7f42401c35c0 auth_method 0 2024-09-16T15:07:29.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.338+0000 7f4245b00640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.339+0000 7f4245b00640 10 monclient(hunting): handle_auth_done global_id 5022 payload 824 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.339+0000 7f4245b00640 1 -- 172.21.15.134:0/1481834670 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f424007a680 con 0x7f42401c35c0 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4245b00640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 msgr2=0x7f42401c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4245b00640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 0x7f42401c44e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42400a1a30 tx=0x7f4230034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f423004ae90 con 0x7f42401c35c0 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:29.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 10 monclient: dump: 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.350 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f4230040630 con 0x7f42401c35c0 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 msgr2=0x7f42401c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 0x7f42401c44e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42400a1a30 tx=0x7f4230034000 comp rx=0 tx=0).stop 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 msgr2=0x7f42401c44e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.341+0000 7f4236ffd640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42401c35c0 0x7f42401c44e0 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f42400a1a30 tx=0x7f4230034000 comp rx=0 tx=0).stop 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:29.352 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.353 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.355 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.358 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:29.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:29.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.369 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:29.371 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.372 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:29.373 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:29.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 >> v2:172.21.15.162:6800/980547263 conn(0x7f4224038880 msgr2=0x7f422403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 --2- 172.21.15.134:0/1481834670 >> v2:172.21.15.162:6800/980547263 conn(0x7f4224038880 0x7f422403ad40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f423c009280 tx=0x7f423c00b040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 10 monclient: shutdown 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 msgr2=0x7f42401c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 --2- 172.21.15.134:0/1481834670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42400a3260 0x7f42401c3a60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f423800c410 tx=0x7f423800c8d0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 reap_dead start 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4246b02640 1 -- 172.21.15.134:0/1481834670 reap_dead start 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4236ffd640 1 -- 172.21.15.134:0/1481834670 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(314..314 src has 1..314) v4 ==== 4039+0+0 (unknown 0 0 0) 0x7f4238049c70 con 0x7f42400a3260 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 shutdown_connections 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 >> 172.21.15.134:0/1481834670 conn(0x7f42400a2740 msgr2=0x7f42400a0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 shutdown_connections 2024-09-16T15:07:29.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.342+0000 7f4248565640 1 -- 172.21.15.134:0/1481834670 wait complete. 2024-09-16T15:07:29.376 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:29.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:07:29.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:07:29.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.472+0000 7fe61a9aa640 10 monclient: get_monmap_and_config 2024-09-16T15:07:29.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.472+0000 7fe61a9aa640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.472+0000 7fe61a9aa640 10 monclient: monmap: 2024-09-16T15:07:29.473 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:29.473053+0000 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:29.473053+0000 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.474 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.475 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:29.475 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:29.475 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.472+0000 7fe61a9aa640 1 Processor -- start 2024-09-16T15:07:29.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 1 -- start start 2024-09-16T15:07:29.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient: init 2024-09-16T15:07:29.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient: _add_conns ranks=[0,7,1,8,2,4,10,3,5,6,9] 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe614136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 0x7fe614137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-i con 0x7fe614137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe61413c370 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe614073250 con 0x7fe614136e40 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61409ba50 con 0x7fe61413c370 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140a56e0 con 0x7fe614137220 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe61a9aa640 10 monclient(hunting): authenticate will time out at 1186.062579s 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe613fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe614136e40 0x7fe61409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59316/0 (socket says 172.21.15.134:59316) 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe613fff640 1 -- 172.21.15.134:0/577414912 learned_addr learned my addr 172.21.15.134:0/577414912 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:29.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 0x7fe614137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 10 monclient(hunting): get_auth_request con 0x7fe614137220 auth_method 0 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 msgr2=0x7fe614137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 msgr2=0x7fe614137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 0x7fe614137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.473+0000 7fe60bfff640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 0x7fe614137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.475+0000 7fe613fff640 1 -- 172.21.15.134:0/577414912 >> v1:172.21.15.134:6789/0 conn(0x7fe614136e40 legacy=0x7fe61409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.475+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 >> v1:172.21.15.162:6789/0 conn(0x7fe61413c370 legacy=0x7fe61413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250675034 0 0) 0x7fe614073250 con 0x7fe614136e40 2024-09-16T15:07:29.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _negotiate 2024-09-16T15:07:29.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): my global_id is 5031 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe600003520 con 0x7fe614136e40 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713949489 0 0) 0x7fe61409ba50 con 0x7fe61413c370 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _negotiate 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): my global_id is 44299 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe614073250 con 0x7fe61413c370 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1201360785 0 0) 0x7fe600003520 con 0x7fe614136e40 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 msgr2=0x7fe614137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe614137220 0x7fe614137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 >> v1:172.21.15.162:6789/0 conn(0x7fe61413c370 legacy=0x7fe61413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:07:29.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:07:29.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140a53c0 con 0x7fe614136e40 2024-09-16T15:07:29.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe61a9aa640 5 monclient: authenticate success, global_id 5031 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fe604001b90 con 0x7fe614136e40 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe61a9aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: dump: 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.491 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 >> v1:172.21.15.134:6789/0 conn(0x7fe614136e40 legacy=0x7fe61409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient: _add_conns ranks=[3,5,4,0,1,7,6,2,8] 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe600003760 0x7fe600003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): picked mon.g con 0x7fe600003760 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe600004080 0x7fe600009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): picked mon.h con 0x7fe600004080 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe60000c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): picked mon.c con 0x7fe600009eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c001970 con 0x7fe600003760 2024-09-16T15:07:29.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140a56e0 con 0x7fe600009eb0 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe60000caa0 con 0x7fe600004080 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe61a9aa640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fe60c001fa0 con 0x7fe61413c370 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe6137fe640 10 monclient(hunting): discarding stray monitor message mon_map magic: 0 v1 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe60000c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): get_auth_request con 0x7fe600009eb0 auth_method 0 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe600003760 0x7fe600003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe600004080 0x7fe600009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): get_auth_request con 0x7fe600003760 auth_method 0 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): get_auth_request con 0x7fe600004080 auth_method 0 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:07:29.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient(hunting): handle_auth_done global_id 5031 payload 293 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe600004080 msgr2=0x7fe600009970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe600004080 0x7fe600009970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe600003760 msgr2=0x7fe600003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe600003760 0x7fe600003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 monclient: found mon.c 2024-09-16T15:07:29.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600003520 con 0x7fe600009eb0 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.476+0000 7fe618f47640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.477+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 reap_dead start 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe618f47640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 msgr2=0x7fe60000c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe618f47640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe60000c2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe60c001860 tx=0x7fe6000057f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d26e5166478e6378 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe60c005c00 con 0x7fe600009eb0 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: dump: 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.497 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.498 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe61a9aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c005da0 con 0x7fe600009eb0 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe61a9aa640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe61a9aa640 4 set_mon_vals no callback set 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 1 -- 172.21.15.134:0/577414912 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe60c007950 con 0x7fe600009eb0 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.479+0000 7fe6137fe640 10 monclient: dump: 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.499 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 10 monclient: shutdown 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 -- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 msgr2=0x7fe60000c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe60000c2a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe60c001860 tx=0x7fe6000057f0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 -- 172.21.15.134:0/577414912 shutdown_connections 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/577414912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe60000c2a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 -- 172.21.15.134:0/577414912 >> 172.21.15.134:0/577414912 conn(0x7fe6140a6110 msgr2=0x7fe6140a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 -- 172.21.15.134:0/577414912 shutdown_connections 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 1 -- 172.21.15.134:0/577414912 wait complete. 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.480+0000 7fe61a9aa640 10 monclient: monmap: 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:29.501 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.502 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 Processor -- start 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 -- start start 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient: init 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient: _add_conns ranks=[4,8,3,0,2,7,1,5,6] 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-c con 0x7fe600009eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe61409f6d0 0x7fe61409fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-e con 0x7fe61409f6d0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 0x7fe6141d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): picked mon.noname-g con 0x7fe61409fff0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140a7410 con 0x7fe61409fff0 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140a90a0 con 0x7fe600009eb0 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe614121b40 con 0x7fe61409f6d0 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe61a9aa640 10 monclient(hunting): authenticate will time out at 1186.070753s 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:35164/0 (socket says 172.21.15.134:35164) 2024-09-16T15:07:29.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 1 -- 172.21.15.134:0/141569716 learned_addr learned my addr 172.21.15.134:0/141569716 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): get_auth_request con 0x7fe600009eb0 auth_method 0 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe61409f6d0 0x7fe61409fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): get_auth_request con 0x7fe61409f6d0 auth_method 0 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe613fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe618f47640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 0x7fe6141d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe618f47640 10 monclient(hunting): get_auth_request con 0x7fe61409fff0 auth_method 0 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe618f47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe618f47640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe618f47640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.481+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 10 monclient(hunting): handle_auth_done global_id 4936 payload 824 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe61409f6d0 msgr2=0x7fe61409fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe618f47640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 msgr2=0x7fe6141d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe61409f6d0 0x7fe61409fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe618f47640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 msgr2=0x7fe6141d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:29.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 msgr2=0x7fe6141d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe618f47640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 0x7fe6141d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe618f47640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 0x7fe6141d1530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe61409fff0 0x7fe6141d1530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 monclient: found mon.noname-c 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.482+0000 7fe613fff640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614122100 con 0x7fe600009eb0 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe613fff640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 msgr2=0x7fe61409f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe613fff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe6040010e0 tx=0x7fe6040084a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe60400fc30 con 0x7fe600009eb0 2024-09-16T15:07:29.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: dump: 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.508 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60400fdd0 con 0x7fe600009eb0 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 5 monclient: authenticate success, global_id 4936 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe604012750 con 0x7fe600009eb0 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe6117fa640 10 monclient: dump: 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.509 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe612ffd640 4 set_mon_vals no callback set 2024-09-16T15:07:29.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 10 monclient: _renew_subs 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6141223e0 con 0x7fe600009eb0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 10 monclient: _renew_subs 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.484+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140a7850 con 0x7fe600009eb0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe61a9aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 0x7fe614138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe614073620 con 0x7fe614138350 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe6040122e0 con 0x7fe600009eb0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 0x7fe614138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient: get_auth_request con 0x7fe614138350 auth_method 0 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe6117fa640 1 --2- 172.21.15.134:0/141569716 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5f40388b0 0x7fe5f403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(314..314 src has 1..314) v4 ==== 4039+0+0 (secure 0 0 0) 0x7fe604049df0 con 0x7fe600009eb0 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 10 monclient(hunting): handle_auth_done global_id 44345 payload 824 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe60bfff640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe614061ae0 con 0x7fe614138350 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe618f47640 1 --2- 172.21.15.134:0/141569716 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5f40388b0 0x7fe5f403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.485+0000 7fe618f47640 10 monclient: get_auth_request con 0x7fe5f40388b0 auth_method 0 2024-09-16T15:07:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.487+0000 7fe60bfff640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 msgr2=0x7fe614138730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe60bfff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 0x7fe614138730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc006fd0 tx=0x7fe5fc00bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5fc00be60 con 0x7fe614138350 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 10 monclient: dump: 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.515 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe618f47640 1 --2- 172.21.15.134:0/141569716 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5f40388b0 0x7fe5f403ad70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009030 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe5fc013d50 con 0x7fe614138350 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 msgr2=0x7fe614138730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 0x7fe614138730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc006fd0 tx=0x7fe5fc00bc70 comp rx=0 tx=0).stop 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 msgr2=0x7fe614138730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.488+0000 7fe6117fa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138350 0x7fe614138730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe61a9aa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 0x7fe6141d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe61a9aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe614071b20 con 0x7fe614138c70 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 0x7fe6141d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient: get_auth_request con 0x7fe614138c70 auth_method 0 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 10 monclient(hunting): handle_auth_done global_id 44354 payload 824 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.494+0000 7fe60bfff640 1 -- 172.21.15.134:0/141569716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe61407a680 con 0x7fe614138c70 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe60bfff640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 msgr2=0x7fe6141d5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe60bfff640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 0x7fe6141d5a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc008010 tx=0x7fe5fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5fc023c00 con 0x7fe614138c70 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 10 monclient: dump: 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fe5fc027070 con 0x7fe614138c70 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 msgr2=0x7fe6141d5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 0x7fe6141d5a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc008010 tx=0x7fe5fc004730 comp rx=0 tx=0).stop 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 msgr2=0x7fe6141d5a90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe6117fa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe614138c70 0x7fe6141d5a90 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc008010 tx=0x7fe5fc004730 comp rx=0 tx=0).stop 2024-09-16T15:07:29.519 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:29.520 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.521 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.522 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:29.523 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.524 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.526 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.532 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.534 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:29.536 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.537 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.540 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:29.541 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5f40388b0 msgr2=0x7fe5f403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/141569716 >> v2:172.21.15.162:6800/980547263 conn(0x7fe5f40388b0 0x7fe5f403ad70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009030 comp rx=0 tx=0).stop 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 10 monclient: shutdown 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 msgr2=0x7fe61409f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe6040010e0 tx=0x7fe6040084a0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe618f47640 1 -- 172.21.15.134:0/141569716 reap_dead start 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 shutdown_connections 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 --2- 172.21.15.134:0/141569716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe600009eb0 0x7fe61409f190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 >> 172.21.15.134:0/141569716 conn(0x7fe6140a6110 msgr2=0x7fe61413bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 shutdown_connections 2024-09-16T15:07:29.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.497+0000 7fe61a9aa640 1 -- 172.21.15.134:0/141569716 wait complete. 2024-09-16T15:07:29.544 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:29.544 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:07:29.544 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:07:29.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.640+0000 7f43026b7640 10 monclient: get_monmap_and_config 2024-09-16T15:07:29.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.640+0000 7f43026b7640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.640+0000 7f43026b7640 10 monclient: monmap: 2024-09-16T15:07:29.641 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:29.641101+0000 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:29.641101+0000 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.642 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.643 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:29.643 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:29.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 Processor -- start 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 -- start start 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient: init 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient: _add_conns ranks=[6,10,1,2,9,5,8,0,7,3,4] 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-d con 0x7f42fc136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:29.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42fc137220 0x7f42fc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-f con 0x7f42fc137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f42fc135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc073250 con 0x7f42fc135dc0 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc09abc0 con 0x7f42fc137220 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc09ba50 con 0x7f42fc136e40 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f43026b7640 10 monclient(hunting): authenticate will time out at 1186.230609s 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58044/0 (socket says 172.21.15.134:58044) 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 1 -- 172.21.15.134:0/2559449710 learned_addr learned my addr 172.21.15.134:0/2559449710 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): get_auth_request con 0x7f42fc136e40 auth_method 0 2024-09-16T15:07:29.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 1 --2- 172.21.15.134:0/2559449710 >> v2:172.21.15.162:3300/0 conn(0x7f42fc137220 0x7f42fc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): get_auth_request con 0x7f42fc137220 auth_method 0 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.641+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 10 monclient(hunting): handle_auth_done global_id 34539 payload 274 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 1 -- 172.21.15.134:0/2559449710 >> v2:172.21.15.162:3300/0 conn(0x7f42fc137220 msgr2=0x7f42fc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 1 --2- 172.21.15.134:0/2559449710 >> v2:172.21.15.162:3300/0 conn(0x7f42fc137220 0x7f42fc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 1 -- 172.21.15.134:0/2559449710 >> v1:172.21.15.162:6789/0 conn(0x7f42fc135dc0 legacy=0x7f42fc136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 1 monclient: found mon.noname-d 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:29.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.642+0000 7f42fbfff640 1 -- 172.21.15.134:0/2559449710 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0a53c0 con 0x7f42fc136e40 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fbfff640 1 -- 172.21.15.134:0/2559449710 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 msgr2=0x7f42fc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fbfff640 1 --2- 172.21.15.134:0/2559449710 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0098a0 tx=0x7f42ec02a110 comp rx=0 tx=0).ready entity=mon.6 client_cookie=582436cc0284a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fb7fe640 1 -- 172.21.15.134:0/2559449710 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ec02ae30 con 0x7f42fc136e40 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.644+0000 7f42fb7fe640 10 monclient: dump: 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.655 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 5 monclient: authenticate success, global_id 34539 2024-09-16T15:07:29.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 1 -- 172.21.15.134:0/2559449710 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0307b0 con 0x7f42fc136e40 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 4 set_mon_vals no callback set 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 1 -- 172.21.15.134:0/2559449710 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ec030d90 con 0x7f42fc136e40 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f42fb7fe640 10 monclient: dump: 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.657 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 10 monclient: shutdown 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 -- 172.21.15.134:0/2559449710 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 msgr2=0x7f42fc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 --2- 172.21.15.134:0/2559449710 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0098a0 tx=0x7f42ec02a110 comp rx=0 tx=0).stop 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 -- 172.21.15.134:0/2559449710 shutdown_connections 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 --2- 172.21.15.134:0/2559449710 >> v2:172.21.15.162:3300/0 conn(0x7f42fc137220 0x7f42fc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 --2- 172.21.15.134:0/2559449710 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f42fc136e40 0x7f42fc09c150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 -- 172.21.15.134:0/2559449710 >> 172.21.15.134:0/2559449710 conn(0x7f42fc0a6110 msgr2=0x7f42fc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 -- 172.21.15.134:0/2559449710 shutdown_connections 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.645+0000 7f43026b7640 1 -- 172.21.15.134:0/2559449710 wait complete. 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient: monmap: 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.659 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 Processor -- start 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 -- start start 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient: init 2024-09-16T15:07:29.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient: _add_conns ranks=[0,7,3,6,4,2,5,1,8] 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 0x7f42fc134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-a con 0x7f42fc135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 0x7f42fc135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-i con 0x7f42fc136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 0x7f42fc1cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): picked mon.noname-g con 0x7f42fc137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc120d20 con 0x7f42fc135dc0 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc1225f0 con 0x7f42fc137220 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc1364e0 con 0x7f42fc136e40 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.646+0000 7f43026b7640 10 monclient(hunting): authenticate will time out at 1186.235996s 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 0x7f42fc134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 0x7f42fc134f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42782/0 (socket says 172.21.15.134:42782) 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 -- 172.21.15.134:0/928689023 learned_addr learned my addr 172.21.15.134:0/928689023 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): get_auth_request con 0x7f42fc135dc0 auth_method 0 2024-09-16T15:07:29.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 0x7f42fc135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 10 monclient(hunting): get_auth_request con 0x7f42fc136e40 auth_method 0 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 0x7f42fc1cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 msgr2=0x7f42fc135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 msgr2=0x7f42fc135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 10 monclient(hunting): get_auth_request con 0x7f42fc137220 auth_method 0 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 0x7f42fc135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 0x7f42fc135450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 msgr2=0x7f42fc1cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 msgr2=0x7f42fc1cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 0x7f42fc1cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient(hunting): handle_auth_done global_id 5040 payload 824 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f4300c54640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 0x7f42fc1cb3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 msgr2=0x7f42fc135450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f42fc136e40 0x7f42fc135450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 msgr2=0x7f42fc1cb3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42fc137220 0x7f42fc1cb3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 monclient: found mon.noname-a 2024-09-16T15:07:29.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.647+0000 7f42fbfff640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0094d0 con 0x7f42fc135dc0 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42fbfff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 msgr2=0x7f42fc134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42fbfff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 0x7f42fc134f10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0090b0 tx=0x7f42ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ec003610 con 0x7f42fc135dc0 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: dump: 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.666 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0037b0 con 0x7f42fc135dc0 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 5 monclient: authenticate success, global_id 5040 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 10 monclient: _renew_subs 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc0a3560 con 0x7f42fc135dc0 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42ec003d90 con 0x7f42fc135dc0 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42f97fa640 10 monclient: dump: 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.668 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f42faffd640 4 set_mon_vals no callback set 2024-09-16T15:07:29.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 10 monclient: _renew_subs 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.649+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc122310 con 0x7f42fc135dc0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f42ec039070 con 0x7f42fc135dc0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f43026b7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f43026b7640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 0x7f42fc02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc071b20 con 0x7f42fc120130 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 0x7f42fc02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient: get_auth_request con 0x7f42fc120130 auth_method 0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f97fa640 1 --2- 172.21.15.134:0/928689023 >> v2:172.21.15.162:6800/980547263 conn(0x7f42dc0388e0 0x7f42dc03ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(314..314 src has 1..314) v4 ==== 4039+0+0 (secure 0 0 0) 0x7f42ec06ac70 con 0x7f42fc135dc0 2024-09-16T15:07:29.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.650+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.651+0000 7f4300c54640 1 --2- 172.21.15.134:0/928689023 >> v2:172.21.15.162:6800/980547263 conn(0x7f42dc0388e0 0x7f42dc03ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.651+0000 7f4300c54640 10 monclient: get_auth_request con 0x7f42dc0388e0 auth_method 0 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.651+0000 7f42f3fff640 10 monclient(hunting): handle_auth_done global_id 4945 payload 824 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.651+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42fc061ae0 con 0x7f42fc120130 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 msgr2=0x7f42fc02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 0x7f42fc02b260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0a3160 tx=0x7f42e4011970 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42e4015dd0 con 0x7f42fc120130 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f4300c54640 1 --2- 172.21.15.134:0/928689023 >> v2:172.21.15.162:6800/980547263 conn(0x7f42dc0388e0 0x7f42dc03ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0a30e0 tx=0x7f42f4009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 10 monclient: dump: 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.671 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f42e4015960 con 0x7f42fc120130 2024-09-16T15:07:29.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 msgr2=0x7f42fc02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 0x7f42fc02b260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0a3160 tx=0x7f42e4011970 comp rx=0 tx=0).stop 2024-09-16T15:07:29.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 msgr2=0x7f42fc02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.653+0000 7f42f97fa640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc120130 0x7f42fc02b260 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0a3160 tx=0x7f42e4011970 comp rx=0 tx=0).stop 2024-09-16T15:07:29.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f43026b7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f43026b7640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 0x7f42fc02bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f43026b7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc07a680 con 0x7f42fc02b7a0 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 0x7f42fc02bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f42f3fff640 10 monclient: get_auth_request con 0x7f42fc02b7a0 auth_method 0 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f42f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f42f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.659+0000 7f42f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.660+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.660+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.660+0000 7f42f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.660+0000 7f42f3fff640 10 monclient(hunting): handle_auth_done global_id 4954 payload 824 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.660+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42fc09b490 con 0x7f42fc02b7a0 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f3fff640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 msgr2=0x7f42fc02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f3fff640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 0x7f42fc02bbb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42e4000c00 tx=0x7f42e40046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42e4022580 con 0x7f42fc02b7a0 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 10 monclient: dump: 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.676 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9527 (secure 0 0 0) 0x7f42e40064d0 con 0x7f42fc02b7a0 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 msgr2=0x7f42fc02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 0x7f42fc02bbb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42e4000c00 tx=0x7f42e40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 msgr2=0x7f42fc02bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.662+0000 7f42f97fa640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f42fc02b7a0 0x7f42fc02bbb0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42e4000c00 tx=0x7f42e40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:29.678 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.679 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.680 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:29.681 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.682 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:29.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:29.685 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:29.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.688 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:29.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.696 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.699 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.700 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:29.701 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:29.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 >> v2:172.21.15.162:6800/980547263 conn(0x7f42dc0388e0 msgr2=0x7f42dc03ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 --2- 172.21.15.134:0/928689023 >> v2:172.21.15.162:6800/980547263 conn(0x7f42dc0388e0 0x7f42dc03ada0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0a30e0 tx=0x7f42f4009040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 10 monclient: shutdown 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 msgr2=0x7f42fc134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 --2- 172.21.15.134:0/928689023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42fc135dc0 0x7f42fc134f10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0090b0 tx=0x7f42ec0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f4300c54640 1 -- 172.21.15.134:0/928689023 reap_dead start 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f4300c54640 1 -- 172.21.15.134:0/928689023 reap_dead start 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 shutdown_connections 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 >> 172.21.15.134:0/928689023 conn(0x7f42fc0a6110 msgr2=0x7f42fc09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 shutdown_connections 2024-09-16T15:07:29.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.663+0000 7f43026b7640 1 -- 172.21.15.134:0/928689023 wait complete. 2024-09-16T15:07:29.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:29.704 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:07:29.704 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:07:29.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.800+0000 7f4ce46d7640 10 monclient: get_monmap_and_config 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.800+0000 7f4ce46d7640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.800+0000 7f4ce46d7640 10 monclient: monmap: 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:29.801286+0000 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:29.801286+0000 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.802 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:29.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.800+0000 7f4ce46d7640 1 Processor -- start 2024-09-16T15:07:29.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 -- start start 2024-09-16T15:07:29.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient: init 2024-09-16T15:07:29.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient: _add_conns ranks=[2,7,5,0,4,3,8,1,10,9,6] 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-b con 0x7f4cdc136eb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-i con 0x7f4cdc137290 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4cdc133d10 0x7f4cdc134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-h con 0x7f4cdc133d10 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0a5560 con 0x7f4cdc136eb0 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc09b240 con 0x7f4cdc133d10 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc09b520 con 0x7f4cdc137290 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce46d7640 10 monclient(hunting): authenticate will time out at 1186.390725s 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40198/0 (socket says 172.21.15.134:40198) 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 1 -- 172.21.15.134:0/2324293296 learned_addr learned my addr 172.21.15.134:0/2324293296 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): get_auth_request con 0x7f4cdc136eb0 auth_method 0 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce2c74640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4cdc133d10 0x7f4cdc134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.801+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): get_auth_request con 0x7f4cdc133d10 auth_method 0 2024-09-16T15:07:29.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 10 monclient(hunting): get_auth_request con 0x7f4cdc137290 auth_method 0 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 10 monclient(hunting): handle_auth_done global_id 44363 payload 274 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 msgr2=0x7f4cdc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 msgr2=0x7f4cdc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 msgr2=0x7f4cdc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4cdc133d10 msgr2=0x7f4cdc134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4cdc133d10 0x7f4cdc134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 monclient: found mon.noname-b 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.802+0000 7f4ce2473640 1 -- 172.21.15.134:0/2324293296 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc073250 con 0x7f4cdc136eb0 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.804+0000 7f4ce2473640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 msgr2=0x7f4cdc09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.804+0000 7f4ce2473640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40098a0 tx=0x7f4cd402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a8baeb926d3066 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 1 -- 172.21.15.134:0/2324293296 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4cd402ade0 con 0x7f4cdc136eb0 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: dump: 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.815 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 1 -- 172.21.15.134:0/2324293296 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd40307b0 con 0x7f4cdc136eb0 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 1 -- 172.21.15.134:0/2324293296 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4cd4030d90 con 0x7f4cdc136eb0 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 5 monclient: authenticate success, global_id 44363 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 4 set_mon_vals no callback set 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce1471640 10 monclient: dump: 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 10 monclient: shutdown 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 -- 172.21.15.134:0/2324293296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 msgr2=0x7f4cdc09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40098a0 tx=0x7f4cd402a080 comp rx=0 tx=0).stop 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 -- 172.21.15.134:0/2324293296 shutdown_connections 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4cdc133d10 0x7f4cdc134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc137290 0x7f4cdc137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/2324293296 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cdc136eb0 0x7f4cdc09bf00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 -- 172.21.15.134:0/2324293296 >> 172.21.15.134:0/2324293296 conn(0x7f4cdc0a99a0 msgr2=0x7f4cdc0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 -- 172.21.15.134:0/2324293296 shutdown_connections 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.805+0000 7f4ce46d7640 1 -- 172.21.15.134:0/2324293296 wait complete. 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.806+0000 7f4ce46d7640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.806+0000 7f4ce46d7640 10 monclient: monmap: 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:29.819 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.806+0000 7f4ce46d7640 1 Processor -- start 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.806+0000 7f4ce46d7640 1 -- start start 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.806+0000 7f4ce46d7640 10 monclient: init 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient: _add_conns ranks=[1,7,6,8,5,3,2,4,0] 2024-09-16T15:07:29.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cdc133d10 0x7f4cdc1cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-f con 0x7f4cdc133d10 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 0x7f4cdc1cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-i con 0x7f4cdc136eb0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): picked mon.noname-d con 0x7f4cdc137290 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1364b0 con 0x7f4cdc133d10 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc11fbd0 con 0x7f4cdc137290 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0a7cd0 con 0x7f4cdc136eb0 2024-09-16T15:07:29.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce46d7640 10 monclient(hunting): authenticate will time out at 1186.396257s 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cdc133d10 0x7f4cdc1cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58048/0 (socket says 172.21.15.134:58048) 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 0x7f4cdc1cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 learned_addr learned my addr 172.21.15.134:0/1048352534 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): get_auth_request con 0x7f4cdc137290 auth_method 0 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): get_auth_request con 0x7f4cdc133d10 auth_method 0 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 10 monclient(hunting): get_auth_request con 0x7f4cdc136eb0 auth_method 0 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 msgr2=0x7f4cdc1cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 msgr2=0x7f4cdc1cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 0x7f4cdc1cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 0x7f4cdc1cbc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient(hunting): handle_auth_done global_id 34548 payload 824 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 msgr2=0x7f4cdc1cbc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4cdc136eb0 0x7f4cdc1cbc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cdc133d10 msgr2=0x7f4cdc1cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cdc133d10 0x7f4cdc1cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 monclient: found mon.noname-d 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40094d0 con 0x7f4cdc137290 2024-09-16T15:07:29.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.807+0000 7f4ce2473640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cdc133d10 0x7f4cdc1cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.809+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 msgr2=0x7f4cdc0a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.809+0000 7f4ce2c74640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e850 tx=0x7f4ccc00ed10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ccc009630 con 0x7f4cdc137290 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: dump: 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.829 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc013470 con 0x7f4cdc137290 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 5 monclient: authenticate success, global_id 34548 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 monclient: _renew_subs 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc120570 con 0x7f4cdc137290 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ccc013a50 con 0x7f4cdc137290 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4cdaffd640 10 monclient: dump: 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.831 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 monclient: _renew_subs 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce0c70640 4 set_mon_vals no callback set 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc0a2ca0 con 0x7f4cdc137290 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 0x7f4cdc0a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc073880 con 0x7f4cdc1cc180 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.810+0000 7f4ce2c74640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e850 tx=0x7f4ccc00ed10 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f4ccc04a4c0 osd_map(314..314 src has 1..314) v4 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4ccc013bb0 con 0x7f4cdc137290 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 0x7f4cdc0a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient: get_auth_request con 0x7f4cdc1cc180 auth_method 0 2024-09-16T15:07:29.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4cdaffd640 1 --2- 172.21.15.134:0/1048352534 >> v2:172.21.15.162:6800/980547263 conn(0x7f4cc8038880 0x7f4cc803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 10 monclient(hunting): handle_auth_done global_id 34557 payload 824 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce2473640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cdc061ae0 con 0x7f4cdc1cc180 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/1048352534 >> v2:172.21.15.162:6800/980547263 conn(0x7f4cc8038880 0x7f4cc803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.811+0000 7f4ce1c72640 10 monclient: get_auth_request con 0x7f4cc8038880 auth_method 0 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4ce2473640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 msgr2=0x7f4cdc0a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4ce2473640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 0x7f4cdc0a0ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4034000 tx=0x7f4cd40046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4cd4030e90 con 0x7f4cdc1cc180 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4cdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4cdaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4cdaffd640 10 monclient: dump: 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.813+0000 7f4ce1c72640 1 --2- 172.21.15.134:0/1048352534 >> v2:172.21.15.162:6800/980547263 conn(0x7f4cc8038880 0x7f4cc803ad40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a6df0 tx=0x7f4cd000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4cd402ade0 con 0x7f4cdc1cc180 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 msgr2=0x7f4cdc0a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 0x7f4cdc0a0ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4034000 tx=0x7f4cd40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 msgr2=0x7f4cdc0a0ec0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.814+0000 7f4cdaffd640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc1cc180 0x7f4cdc0a0ec0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce46d7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 0x7f4cdc134320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce46d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc071b20 con 0x7f4cdc0a1400 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 0x7f4cdc134320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient: get_auth_request con 0x7f4cdc0a1400 auth_method 0 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 10 monclient(hunting): handle_auth_done global_id 34566 payload 824 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.820+0000 7f4ce2473640 1 -- 172.21.15.134:0/1048352534 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4cdc07a680 con 0x7f4cdc0a1400 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4ce2473640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 msgr2=0x7f4cdc134320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4ce2473640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 0x7f4cdc134320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40097c0 tx=0x7f4cd4008030 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4cd404c300 con 0x7f4cdc0a1400 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4cdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4cdaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.822+0000 7f4cdaffd640 10 monclient: dump: 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f4cd4003e40 con 0x7f4cdc0a1400 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 msgr2=0x7f4cdc134320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 0x7f4cdc134320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40097c0 tx=0x7f4cd4008030 comp rx=0 tx=0).stop 2024-09-16T15:07:29.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 msgr2=0x7f4cdc134320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc0a1400 0x7f4cdc134320 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40097c0 tx=0x7f4cd4008030 comp rx=0 tx=0).stop 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:29.840 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.842 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.843 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:29.844 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:29.845 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:29.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.851 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:29.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:29.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:29.856 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:29.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:29.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:29.861 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 >> v2:172.21.15.162:6800/980547263 conn(0x7f4cc8038880 msgr2=0x7f4cc803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/1048352534 >> v2:172.21.15.162:6800/980547263 conn(0x7f4cc8038880 0x7f4cc803ad40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0a6df0 tx=0x7f4cd000b040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 10 monclient: shutdown 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 msgr2=0x7f4cdc0a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 --2- 172.21.15.134:0/1048352534 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cdc137290 0x7f4cdc0a0980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e850 tx=0x7f4ccc00ed10 comp rx=0 tx=0).stop 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 reap_dead start 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce2c74640 1 -- 172.21.15.134:0/1048352534 reap_dead start 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4cdaffd640 1 -- 172.21.15.134:0/1048352534 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(314..314 src has 1..314) v4 ==== 4039+0+0 (unknown 0 0 0) 0x7f4ccc04a4c0 con 0x7f4cdc137290 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 shutdown_connections 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 >> 172.21.15.134:0/1048352534 conn(0x7f4cdc0a99a0 msgr2=0x7f4cdc1366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 shutdown_connections 2024-09-16T15:07:29.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.823+0000 7f4ce46d7640 1 -- 172.21.15.134:0/1048352534 wait complete. 2024-09-16T15:07:29.864 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:29.864 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:07:29.864 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:07:29.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.960+0000 7fd08089c640 10 monclient: get_monmap_and_config 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.960+0000 7fd08089c640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.960+0000 7fd08089c640 10 monclient: monmap: 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:29.961617+0000 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:29.961617+0000 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:29.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:29.963 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 Processor -- start 2024-09-16T15:07:29.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 -- start start 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient: init 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient: _add_conns ranks=[9,3,5,0,4,1,2,7,8,10,6] 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd07c136e40 0x7fd07c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-a con 0x7fd07c136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-g con 0x7fd07c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd07c135dc0 0x7fd07c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-h con 0x7fd07c135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07c09abc0 con 0x7fd07c136e40 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c09ba50 con 0x7fd07c137220 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0a53c0 con 0x7fd07c135dc0 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.961+0000 7fd08089c640 10 monclient(hunting): authenticate will time out at 1186.550908s 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd07c136e40 0x7fd07c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd07c136e40 0x7fd07c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42786/0 (socket says 172.21.15.134:42786) 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 -- 172.21.15.134:0/2695026431 learned_addr learned my addr 172.21.15.134:0/2695026431 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): get_auth_request con 0x7fd07c136e40 auth_method 0 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd07c135dc0 0x7fd07c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): get_auth_request con 0x7fd07c135dc0 auth_method 0 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 10 monclient(hunting): get_auth_request con 0x7fd07c137220 auth_method 0 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 msgr2=0x7fd07c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 msgr2=0x7fd07c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd073fff640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient(hunting): handle_auth_done global_id 5049 payload 274 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd07c135dc0 msgr2=0x7fd07c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd07c135dc0 0x7fd07c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 msgr2=0x7fd07c137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 monclient: found mon.noname-a 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.962+0000 7fd07a59c640 1 -- 172.21.15.134:0/2695026431 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c073250 con 0x7fd07c136e40 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd07a59c640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c136e40 msgr2=0x7fd07c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd07a59c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c136e40 0x7fd07c09c150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b5b25140c7a52b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 1 -- 172.21.15.134:0/2695026431 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c02ae00 con 0x7fd07c136e40 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: dump: 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 5 monclient: authenticate success, global_id 5049 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 1 -- 172.21.15.134:0/2695026431 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0307b0 con 0x7fd07c136e40 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 1 -- 172.21.15.134:0/2695026431 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c030d90 con 0x7fd07c136e40 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 4 set_mon_vals no callback set 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd079d9b640 10 monclient: dump: 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.979 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.980 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 10 monclient: shutdown 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 -- 172.21.15.134:0/2695026431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c136e40 msgr2=0x7fd07c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c136e40 0x7fd07c09c150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009870 tx=0x7fd06c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 -- 172.21.15.134:0/2695026431 shutdown_connections 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd07c135dc0 0x7fd07c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c137220 0x7fd07c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 --2- 172.21.15.134:0/2695026431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c136e40 0x7fd07c09c150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 -- 172.21.15.134:0/2695026431 >> 172.21.15.134:0/2695026431 conn(0x7fd07c0a6110 msgr2=0x7fd07c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 -- 172.21.15.134:0/2695026431 shutdown_connections 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.965+0000 7fd08089c640 1 -- 172.21.15.134:0/2695026431 wait complete. 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.966+0000 7fd08089c640 10 monclient: build_initial_monmap 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.966+0000 7fd08089c640 10 monclient: monmap: 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:29.982 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.966+0000 7fd08089c640 1 Processor -- start 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.966+0000 7fd08089c640 1 -- start start 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.966+0000 7fd08089c640 10 monclient: init 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient: _add_conns ranks=[2,1,0,3,4,6,8,7,5] 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 0x7fd07c134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-b con 0x7fd07c135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c136e40 0x7fd07c1352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-f con 0x7fd07c136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:29.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c137220 0x7fd07c1217f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): picked mon.noname-a con 0x7fd07c137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0a8980 con 0x7fd07c137220 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c136800 con 0x7fd07c136e40 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c125480 con 0x7fd07c135dc0 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd08089c640 10 monclient(hunting): authenticate will time out at 1186.556320s 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 0x7fd07c134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c137220 0x7fd07c1217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 0x7fd07c134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40208/0 (socket says 172.21.15.134:40208) 2024-09-16T15:07:29.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 -- 172.21.15.134:0/3072293690 learned_addr learned my addr 172.21.15.134:0/3072293690 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): get_auth_request con 0x7fd07c135dc0 auth_method 0 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): get_auth_request con 0x7fd07c137220 auth_method 0 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c136e40 0x7fd07c1352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): get_auth_request con 0x7fd07c136e40 auth_method 0 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient(hunting): handle_auth_done global_id 44372 payload 824 2024-09-16T15:07:29.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c136e40 msgr2=0x7fd07c1352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c136e40 0x7fd07c1352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c137220 msgr2=0x7fd07c1217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c137220 0x7fd07c1217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 monclient: found mon.noname-b 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07a59c640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0094d0 con 0x7fd07c135dc0 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.967+0000 7fd07ad9d640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c137220 0x7fd07c1217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.969+0000 7fd07a59c640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 msgr2=0x7fd07c134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.969+0000 7fd07a59c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 0x7fd07c134d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097c0 tx=0x7fd06c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c030d40 con 0x7fd07c135dc0 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: dump: 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.988 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 5 monclient: authenticate success, global_id 44372 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c003a80 con 0x7fd07c135dc0 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:29.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 monclient: _renew_subs 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd06c003bf0 con 0x7fd07c135dc0 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c124c20 con 0x7fd07c135dc0 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd07959a640 4 set_mon_vals no callback set 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 10 monclient: dump: 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.990 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 monclient: _renew_subs 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c1362b0 con 0x7fd07c135dc0 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd06c039070 con 0x7fd07c135dc0 2024-09-16T15:07:29.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 0x7fd07c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.970+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c071b20 con 0x7fd07c1224f0 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 0x7fd07c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient: get_auth_request con 0x7fd07c1224f0 auth_method 0 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd072ffd640 1 --2- 172.21.15.134:0/3072293690 >> v2:172.21.15.162:6800/980547263 conn(0x7fd05c038880 0x7fd05c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(314..314 src has 1..314) v4 ==== 4039+0+0 (secure 0 0 0) 0x7fd06c06ab10 con 0x7fd07c135dc0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 10 monclient(hunting): handle_auth_done global_id 44342 payload 824 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd073fff640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd07c061ae0 con 0x7fd07c1224f0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd07ad9d640 1 --2- 172.21.15.134:0/3072293690 >> v2:172.21.15.162:6800/980547263 conn(0x7fd05c038880 0x7fd05c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.971+0000 7fd07ad9d640 10 monclient: get_auth_request con 0x7fd05c038880 auth_method 0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd073fff640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 msgr2=0x7fd07c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd073fff640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 0x7fd07c02b260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd064006fd0 tx=0x7fd064010bc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd06400eea0 con 0x7fd07c1224f0 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 10 monclient: dump: 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:29.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd06400ea30 con 0x7fd07c1224f0 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 msgr2=0x7fd07c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 0x7fd07c02b260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd064006fd0 tx=0x7fd064010bc0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd07ad9d640 1 --2- 172.21.15.134:0/3072293690 >> v2:172.21.15.162:6800/980547263 conn(0x7fd05c038880 0x7fd05c03ad40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd07c122f90 tx=0x7fd07400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 msgr2=0x7fd07c02b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:29.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.973+0000 7fd072ffd640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1224f0 0x7fd07c02b260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.979+0000 7fd08089c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.979+0000 7fd08089c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 0x7fd07c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd08089c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c07a680 con 0x7fd07c1228d0 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 0x7fd07c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient: get_auth_request con 0x7fd07c1228d0 auth_method 0 2024-09-16T15:07:29.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 10 monclient(hunting): handle_auth_done global_id 44351 payload 824 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.980+0000 7fd073fff640 1 -- 172.21.15.134:0/3072293690 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd07c09b490 con 0x7fd07c1228d0 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd073fff640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 msgr2=0x7fd07c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd073fff640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 0x7fd07c02b7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd064006fd0 tx=0x7fd0640046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd064004490 con 0x7fd07c1228d0 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:29.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 10 monclient: dump: 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:29.999 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fd064023040 con 0x7fd07c1228d0 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 msgr2=0x7fd07c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 0x7fd07c02b7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd064006fd0 tx=0x7fd0640046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 msgr2=0x7fd07c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.982+0000 7fd072ffd640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd07c1228d0 0x7fd07c02b7a0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd064006fd0 tx=0x7fd0640046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:30.000 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:30.001 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.002 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.003 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:30.004 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.005 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.006 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:30.007 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.009 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:30.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:30.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.015 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:30.017 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:30.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.023 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:30.023 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 >> v2:172.21.15.162:6800/980547263 conn(0x7fd05c038880 msgr2=0x7fd05c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 --2- 172.21.15.134:0/3072293690 >> v2:172.21.15.162:6800/980547263 conn(0x7fd05c038880 0x7fd05c03ad40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd07c122f90 tx=0x7fd07400b040 comp rx=0 tx=0).stop 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 10 monclient: shutdown 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 msgr2=0x7fd07c134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 --2- 172.21.15.134:0/3072293690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c135dc0 0x7fd07c134d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097c0 tx=0x7fd06c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd07ad9d640 1 -- 172.21.15.134:0/3072293690 reap_dead start 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd07ad9d640 1 -- 172.21.15.134:0/3072293690 reap_dead start 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 shutdown_connections 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 >> 172.21.15.134:0/3072293690 conn(0x7fd07c0a6110 msgr2=0x7fd07c09f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 shutdown_connections 2024-09-16T15:07:30.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:29.983+0000 7fd08089c640 1 -- 172.21.15.134:0/3072293690 wait complete. 2024-09-16T15:07:30.025 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:30.025 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:07:30.026 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:07:30.085 INFO:tasks.rados.rados.0.smithi162.stdout:1576: done (3 left) 2024-09-16T15:07:30.085 INFO:tasks.rados.rados.0.smithi162.stdout:1577: done (2 left) 2024-09-16T15:07:30.085 INFO:tasks.rados.rados.0.smithi162.stdout:1578: done (1 left) 2024-09-16T15:07:30.085 INFO:tasks.rados.rados.0.smithi162.stdout:1580: done (0 left) 2024-09-16T15:07:30.085 INFO:tasks.rados.rados.0.smithi162.stdout:1581: delete oid 3 current snap is 146 2024-09-16T15:07:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:1581: done (0 left) 2024-09-16T15:07:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:1582: read oid 32 snap -1 2024-09-16T15:07:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:1582: expect deleted 2024-09-16T15:07:30.089 INFO:tasks.rados.rados.0.smithi162.stdout:1583: read oid 24 snap -1 2024-09-16T15:07:30.089 INFO:tasks.rados.rados.0.smithi162.stdout:1583: expect deleted 2024-09-16T15:07:30.089 INFO:tasks.rados.rados.0.smithi162.stdout:1584: delete oid 18 current snap is 146 2024-09-16T15:07:30.090 INFO:tasks.rados.rados.0.smithi162.stdout:1583: done (2 left) 2024-09-16T15:07:30.090 INFO:tasks.rados.rados.0.smithi162.stdout:1584: done (1 left) 2024-09-16T15:07:30.090 INFO:tasks.rados.rados.0.smithi162.stdout:1585: snap_remove snap 145 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.125+0000 7f1fef5aa640 10 monclient: get_monmap_and_config 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.125+0000 7f1fef5aa640 10 monclient: build_initial_monmap 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.125+0000 7f1fef5aa640 10 monclient: monmap: 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:30.126818+0000 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:30.126818+0000 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:30.128 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.126+0000 7f1fef5aa640 1 Processor -- start 2024-09-16T15:07:30.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.126+0000 7f1fef5aa640 1 -- start start 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.126+0000 7f1fef5aa640 10 monclient: init 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient: _add_conns ranks=[2,5,8,3,9,7,6,0,1,10,4] 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-b con 0x7f1fe8136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 0x7f1fe8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-h con 0x7f1fe8137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-e con 0x7f1fe8135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe809abc0 con 0x7f1fe8136e40 2024-09-16T15:07:30.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe809ba50 con 0x7f1fe8137220 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80a53c0 con 0x7f1fe8135dc0 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fef5aa640 10 monclient(hunting): authenticate will time out at 1186.716254s 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:51328/0 (socket says 172.21.15.134:51328) 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40210/0 (socket says 172.21.15.134:40210) 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 1 -- 172.21.15.134:0/2753300024 learned_addr learned my addr 172.21.15.134:0/2753300024 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): get_auth_request con 0x7f1fe8135dc0 auth_method 0 2024-09-16T15:07:30.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): get_auth_request con 0x7f1fe8136e40 auth_method 0 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fecb45640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 0x7f1fe8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient(hunting): handle_auth_done global_id 44381 payload 274 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 -- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 msgr2=0x7f1fe8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 -- 172.21.15.134:0/2753300024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 msgr2=0x7f1fe8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 0x7f1fe8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient: _finish_hunting 0 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 monclient: found mon.noname-b 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fed346640 1 -- 172.21.15.134:0/2753300024 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8073250 con 0x7f1fe8136e40 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fecb45640 10 monclient: get_auth_request con 0x7f1fe8137220 auth_method 0 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fedb47640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.127+0000 7f1fecb45640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 0x7f1fe8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fed346640 1 -- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 msgr2=0x7f1fe809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fed346640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40097d0 tx=0x7f1fe4029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6042a918efce1590 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 1 -- 172.21.15.134:0/2753300024 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe402ad10 con 0x7f1fe8136e40 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: dump: 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.142 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: _finish_auth 0 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:30.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fef5aa640 5 monclient: authenticate success, global_id 44381 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 1 -- 172.21.15.134:0/2753300024 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe402ae90 con 0x7f1fe8136e40 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fef5aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fef5aa640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fef5aa640 4 set_mon_vals no callback set 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 1 -- 172.21.15.134:0/2753300024 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe4030c20 con 0x7f1fe8136e40 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.130+0000 7f1fdffff640 10 monclient: dump: 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.144 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 10 monclient: shutdown 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 -- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 msgr2=0x7f1fe809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40097d0 tx=0x7f1fe4029fb0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 -- 172.21.15.134:0/2753300024 shutdown_connections 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1fe8135dc0 0x7f1fe8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8137220 0x7f1fe8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/2753300024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe809c150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 -- 172.21.15.134:0/2753300024 >> 172.21.15.134:0/2753300024 conn(0x7f1fe80a6110 msgr2=0x7f1fe80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 -- 172.21.15.134:0/2753300024 shutdown_connections 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 1 -- 172.21.15.134:0/2753300024 wait complete. 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 10 monclient: build_initial_monmap 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.131+0000 7f1fef5aa640 10 monclient: monmap: 2024-09-16T15:07:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 Processor -- start 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 -- start start 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient: init 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient: _add_conns ranks=[5,2,7,3,0,8,6,1,4] 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8135dc0 0x7f1fe8134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-h con 0x7f1fe8135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-b con 0x7f1fe8136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 0x7f1fe8121a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): picked mon.noname-i con 0x7f1fe8137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81cc260 con 0x7f1fe8136e40 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81254e0 con 0x7f1fe8135dc0 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81363b0 con 0x7f1fe8137220 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fef5aa640 10 monclient(hunting): authenticate will time out at 1186.721696s 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40216/0 (socket says 172.21.15.134:40216) 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 1 -- 172.21.15.134:0/3627680011 learned_addr learned my addr 172.21.15.134:0/3627680011 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): get_auth_request con 0x7f1fe8136e40 auth_method 0 2024-09-16T15:07:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fecb45640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8135dc0 0x7f1fe8134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fedb47640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 0x7f1fe8121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): get_auth_request con 0x7f1fe8135dc0 auth_method 0 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fedb47640 10 monclient(hunting): get_auth_request con 0x7f1fe8137220 auth_method 0 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fedb47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fedb47640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fedb47640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.132+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 10 monclient(hunting): handle_auth_done global_id 44390 payload 824 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fedb47640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 msgr2=0x7f1fe8121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fedb47640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 msgr2=0x7f1fe8121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 msgr2=0x7f1fe8121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fedb47640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 0x7f1fe8121a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fedb47640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 0x7f1fe8121a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1fe8137220 0x7f1fe8121a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8135dc0 msgr2=0x7f1fe8134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1fe8135dc0 0x7f1fe8134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 10 monclient: _finish_hunting 0 2024-09-16T15:07:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 monclient: found mon.noname-b 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.133+0000 7f1fecb45640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe0009b30 con 0x7f1fe8136e40 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fecb45640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 msgr2=0x7f1fe8135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fecb45640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c820 tx=0x7f1fd800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fd800ce10 con 0x7f1fe8136e40 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: dump: 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: _finish_auth 0 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: _check_auth_tickets 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd80113f0 con 0x7f1fe8136e40 2024-09-16T15:07:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 5 monclient: authenticate success, global_id 44390 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fd80119d0 con 0x7f1fe8136e40 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 10 monclient: _renew_subs 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe4009400 con 0x7f1fe8136e40 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fddffb640 10 monclient: dump: 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fdf7fe640 4 set_mon_vals no callback set 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 10 monclient: _renew_subs 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.135+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe8136900 con 0x7f1fe8136e40 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1fd8011560 con 0x7f1fe8136e40 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fef5aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 0x7f1fe802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8071b20 con 0x7f1fe81cb550 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fddffb640 1 --2- 172.21.15.134:0/3627680011 >> v2:172.21.15.162:6800/980547263 conn(0x7f1fcc038880 0x7f1fcc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(315..315 src has 1..315) v4 ==== 4011+0+0 (secure 0 0 0) 0x7f1fd804a130 con 0x7f1fe8136e40 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 0x7f1fe802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fedb47640 1 --2- 172.21.15.134:0/3627680011 >> v2:172.21.15.162:6800/980547263 conn(0x7f1fcc038880 0x7f1fcc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fedb47640 10 monclient: get_auth_request con 0x7f1fcc038880 auth_method 0 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient: get_auth_request con 0x7f1fe81cb550 auth_method 0 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.136+0000 7f1fed346640 10 monclient(hunting): handle_auth_done global_id 44335 payload 824 2024-09-16T15:07:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.137+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fe8061ae0 con 0x7f1fe81cb550 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fedb47640 1 --2- 172.21.15.134:0/3627680011 >> v2:172.21.15.162:6800/980547263 conn(0x7f1fcc038880 0x7f1fcc03ad40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b550 tx=0x7f1fe000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 msgr2=0x7f1fe802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 0x7f1fe802b260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe4003e10 con 0x7f1fe81cb550 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 10 monclient: dump: 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1fe4030340 con 0x7f1fe81cb550 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 msgr2=0x7f1fe802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 0x7f1fe802b260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 msgr2=0x7f1fe802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.139+0000 7f1fddffb640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe81cb550 0x7f1fe802b260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fef5aa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fef5aa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe807a680 con 0x7f1fe802b7a0 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient: get_auth_request con 0x7f1fe802b7a0 auth_method 0 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.145+0000 7f1fed346640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 10 monclient(hunting): handle_auth_done global_id 44344 payload 824 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1fe809b490 con 0x7f1fe802b7a0 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 msgr2=0x7f1fe802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 msgr2=0x7f1fe802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40097d0 tx=0x7f1fe4008030 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.146+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:30.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: get_auth_request con 0x7f1fe802b7a0 auth_method 0 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: _init_auth method 2 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: _init_auth already have auth, reseting 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: handle_auth_reply_more payload 9 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-16T15:07:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.347+0000 7f1fed346640 10 monclient: handle_auth_done global_id 44344 payload 293 2024-09-16T15:07:30.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.349+0000 7f1fed346640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 msgr2=0x7f1fe802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.349+0000 7f1fed346640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe4040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1fe4030cd0 con 0x7f1fe802b7a0 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 10 monclient: dump: 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.352 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f1fe404c790 con 0x7f1fe802b7a0 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 msgr2=0x7f1fe802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe4040000 comp rx=0 tx=0).stop 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 msgr2=0x7f1fe802bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:30.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.350+0000 7f1fddffb640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fe802b7a0 0x7f1fe802bbb0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe4040000 comp rx=0 tx=0).stop 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:30.354 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.355 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.356 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:30.357 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:30.358 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:30.363 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:30.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:30.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:30.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:30.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:30.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:30.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:30.375 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:30.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 >> v2:172.21.15.162:6800/980547263 conn(0x7f1fcc038880 msgr2=0x7f1fcc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/3627680011 >> v2:172.21.15.162:6800/980547263 conn(0x7f1fcc038880 0x7f1fcc03ad40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1fe806b550 tx=0x7f1fe000b040 comp rx=0 tx=0).stop 2024-09-16T15:07:30.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 10 monclient: shutdown 2024-09-16T15:07:30.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:30.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 msgr2=0x7f1fe8135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800c820 tx=0x7f1fd800cce0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fedb47640 1 -- 172.21.15.134:0/3627680011 reap_dead start 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 shutdown_connections 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 --2- 172.21.15.134:0/3627680011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe8136e40 0x7f1fe8135510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 >> 172.21.15.134:0/3627680011 conn(0x7f1fe80a6110 msgr2=0x7f1fe809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 shutdown_connections 2024-09-16T15:07:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.351+0000 7f1fef5aa640 1 -- 172.21.15.134:0/3627680011 wait complete. 2024-09-16T15:07:30.377 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:30.377 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:07:30.378 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:1582: done (1 left) 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:1585: done (0 left) 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:1586: rollback oid 13 current snap is 146 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 134 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:1587: copy_from oid 39 from oid 9 current snap is 146 2024-09-16T15:07:30.424 INFO:tasks.rados.rados.0.smithi162.stdout:1588: delete oid 30 current snap is 146 2024-09-16T15:07:30.430 INFO:tasks.rados.rados.0.smithi162.stdout:1586: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:07:30.430 INFO:tasks.rados.rados.0.smithi162.stdout:1586: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:07:30.430 INFO:tasks.rados.rados.0.smithi162.stdout:1586: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:07:30.430 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 720 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1586: done (2 left) 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1588: done (1 left) 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1589: read oid 20 snap -1 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1589: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1590: read oid 5 snap 134 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1590: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:07:30.431 INFO:tasks.rados.rados.0.smithi162.stdout:1591: delete oid 47 current snap is 146 2024-09-16T15:07:30.432 INFO:tasks.rados.rados.0.smithi162.stdout:1587: finishing copy_from to smithi16234619-39 2024-09-16T15:07:30.432 INFO:tasks.rados.rados.0.smithi162.stdout:1587: got expected ENOENT (src dne) 2024-09-16T15:07:30.432 INFO:tasks.rados.rados.0.smithi162.stdout:1587: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:07:30.434 INFO:tasks.rados.rados.0.smithi162.stdout:1587: done (3 left) 2024-09-16T15:07:30.434 INFO:tasks.rados.rados.0.smithi162.stdout:1591: done (2 left) 2024-09-16T15:07:30.434 INFO:tasks.rados.rados.0.smithi162.stdout:1592: snap_remove snap 144 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.479+0000 7f25d5672640 10 monclient: get_monmap_and_config 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.479+0000 7f25d5672640 10 monclient: build_initial_monmap 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.480+0000 7f25d5672640 10 monclient: monmap: 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:30.480916+0000 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:30.480916+0000 2024-09-16T15:07:30.481 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:30.482 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.480+0000 7f25d5672640 1 Processor -- start 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.480+0000 7f25d5672640 1 -- start start 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.480+0000 7f25d5672640 10 monclient: init 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient: _add_conns ranks=[2,0,9,10,4,7,1,6,5,3,8] 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 0x7f25d009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-b con 0x7f25d0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f25d0137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 0x7f25d013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-a con 0x7f25d013c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0073250 con 0x7f25d0137220 2024-09-16T15:07:30.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00a56e0 con 0x7f25d013c370 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d009abc0 con 0x7f25d0136e40 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25d5672640 10 monclient(hunting): authenticate will time out at 1187.070244s 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 0x7f25d013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 0x7f25d009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 0x7f25d013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42806/0 (socket says 172.21.15.134:42806) 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 1 -- 172.21.15.134:0/4091232610 learned_addr learned my addr 172.21.15.134:0/4091232610 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): get_auth_request con 0x7f25d013c370 auth_method 0 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): get_auth_request con 0x7f25d0136e40 auth_method 0 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient(hunting): handle_auth_done global_id 44399 payload 274 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 -- 172.21.15.134:0/4091232610 >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 msgr2=0x7f25d013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 --2- 172.21.15.134:0/4091232610 >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 0x7f25d013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 -- 172.21.15.134:0/4091232610 >> v1:172.21.15.134:6789/0 conn(0x7f25d0137220 legacy=0x7f25d0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient: _finish_hunting 0 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 monclient: found mon.noname-b 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.481+0000 7f25ceffd640 1 -- 172.21.15.134:0/4091232610 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d009ba50 con 0x7f25d0136e40 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ceffd640 1 -- 172.21.15.134:0/4091232610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 msgr2=0x7f25d009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ceffd640 1 --2- 172.21.15.134:0/4091232610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 0x7f25d009c150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097a0 tx=0x7f25c002a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5ab33f39f21324 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 1 -- 172.21.15.134:0/4091232610 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25c002ad30 con 0x7f25d0136e40 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: dump: 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 1 -- 172.21.15.134:0/4091232610 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c00307b0 con 0x7f25d0136e40 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25d5672640 5 monclient: authenticate success, global_id 44399 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25d5672640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25d5672640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25d5672640 4 set_mon_vals no callback set 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 1 -- 172.21.15.134:0/4091232610 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25c0030d90 con 0x7f25d0136e40 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.484+0000 7f25ce7fc640 10 monclient: dump: 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 10 monclient: shutdown 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 -- 172.21.15.134:0/4091232610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 msgr2=0x7f25d009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 --2- 172.21.15.134:0/4091232610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 0x7f25d009c150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097a0 tx=0x7f25c002a010 comp rx=0 tx=0).stop 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 -- 172.21.15.134:0/4091232610 shutdown_connections 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 --2- 172.21.15.134:0/4091232610 >> v2:172.21.15.134:3300/0 conn(0x7f25d013c370 0x7f25d013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 --2- 172.21.15.134:0/4091232610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d0136e40 0x7f25d009c150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 -- 172.21.15.134:0/4091232610 >> 172.21.15.134:0/4091232610 conn(0x7f25d00a6110 msgr2=0x7f25d00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 -- 172.21.15.134:0/4091232610 shutdown_connections 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 1 -- 172.21.15.134:0/4091232610 wait complete. 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 10 monclient: build_initial_monmap 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.485+0000 7f25d5672640 10 monclient: monmap: 2024-09-16T15:07:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:30.495 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 Processor -- start 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 -- start start 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient: init 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient: _add_conns ranks=[5,1,2,3,4,0,8,6,7] 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f25d0136e40 0x7f25d00a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-h con 0x7f25d0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d0137220 0x7f25d0135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-f con 0x7f25d0137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): picked mon.noname-b con 0x7f25d013c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0122370 con 0x7f25d0137220 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d01225c0 con 0x7f25d013c370 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00a7a80 con 0x7f25d0136e40 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25d5672640 10 monclient(hunting): authenticate will time out at 1187.075742s 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40240/0 (socket says 172.21.15.134:40240) 2024-09-16T15:07:30.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 learned_addr learned my addr 172.21.15.134:0/2549193565 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): get_auth_request con 0x7f25d013c370 auth_method 0 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.486+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d0137220 0x7f25d0135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 10 monclient(hunting): handle_auth_done global_id 44408 payload 824 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25ceffd640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f25d0136e40 0x7f25d00a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f25d0136e40 msgr2=0x7f25d00a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f25d0136e40 0x7f25d00a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d0137220 msgr2=0x7f25d0135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d0137220 0x7f25d0135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 monclient: found mon.noname-b 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009400 con 0x7f25d013c370 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d0137220 auth_method 0 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.487+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25d0137220 0x7f25d0135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 msgr2=0x7f25d0136150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25cf7fe640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009cb0 tx=0x7f25c800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25c8009b20 con 0x7f25d013c370 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:30.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: dump: 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.504 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: _finish_auth 0 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800f040 con 0x7f25d013c370 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 5 monclient: authenticate success, global_id 44408 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 10 monclient: _renew_subs 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d01cc310 con 0x7f25d013c370 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25c8003e00 con 0x7f25d013c370 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25c7fff640 10 monclient: dump: 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25cdffb640 4 set_mon_vals no callback set 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 10 monclient: _renew_subs 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.489+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0121d20 con 0x7f25d013c370 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f25c800cd30 con 0x7f25d013c370 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c7fff640 1 --2- 172.21.15.134:0/2549193565 >> v2:172.21.15.162:6800/980547263 conn(0x7f25b0038880 0x7f25b003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25d5672640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25d5672640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0073670 con 0x7f25d011fc60 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(316..316 src has 1..316) v4 ==== 4067+0+0 (secure 0 0 0) 0x7f25c804a710 con 0x7f25d013c370 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25ceffd640 1 --2- 172.21.15.134:0/2549193565 >> v2:172.21.15.162:6800/980547263 conn(0x7f25b0038880 0x7f25b003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25ceffd640 10 monclient: get_auth_request con 0x7f25b0038880 auth_method 0 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d011fc60 auth_method 0 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.490+0000 7f25c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.491+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.491+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.491+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.491+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.493+0000 7f25ceffd640 1 --2- 172.21.15.134:0/2549193565 >> v2:172.21.15.162:6800/980547263 conn(0x7f25b0038880 0x7f25b003ad40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097a0 tx=0x7f25c002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d011fc60 auth_method 0 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:30.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:30.692+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d011fc60 auth_method 0 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:31.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.093+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:31.426 INFO:tasks.rados.rados.0.smithi162.stdout:1592: done (2 left) 2024-09-16T15:07:31.426 INFO:tasks.rados.rados.0.smithi162.stdout:1593: rollback oid 13 current snap is 146 2024-09-16T15:07:31.427 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 134 2024-09-16T15:07:31.427 INFO:tasks.rados.rados.0.smithi162.stdout:1589: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:31.566 INFO:tasks.rados.rados.0.smithi162.stdout:1590: expect (ObjNum 441 snap 104 seq_num 441) 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1593: finishing rollback tid 0 to smithi16234619-13 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1593: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1593: finishing rollback tid 2 to smithi16234619-13 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 723 (ObjNum 200 snap 42 seq_num 200) dirty exists 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1589: done (2 left) 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1590: done (1 left) 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1593: done (0 left) 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1594: write (excl) oid 29 current snap is 146 2024-09-16T15:07:31.674 INFO:tasks.rados.rados.0.smithi162.stdout:1594: seq_num 584 ranges {615288=788244,1957837=562523,3121263=559679} 2024-09-16T15:07:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:1594: writing smithi16234619-29 from 615288 to 1403532 tid 1 2024-09-16T15:07:31.687 INFO:tasks.rados.rados.0.smithi162.stdout:1594: writing smithi16234619-29 from 1957837 to 2520360 tid 2 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1594: writing smithi16234619-29 from 3121263 to 3680942 tid 3 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1595: read oid 13 snap 134 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1595: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1596: copy_from oid 15 from oid 19 current snap is 146 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1597: read oid 34 snap -1 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1597: expect (ObjNum 516 snap 135 seq_num 516) 2024-09-16T15:07:31.694 INFO:tasks.rados.rados.0.smithi162.stdout:1598: snap_remove snap 143 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d011fc60 auth_method 0 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:07:31.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:07:31.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:31.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:31.894+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:32.431 INFO:tasks.rados.rados.0.smithi162.stdout:1598: done (4 left) 2024-09-16T15:07:32.432 INFO:tasks.rados.rados.0.smithi162.stdout:1599: write oid 12 current snap is 146 2024-09-16T15:07:32.432 INFO:tasks.rados.rados.0.smithi162.stdout:1599: seq_num 585 ranges {632095=463493,1616319=422534,2748686=487304} 2024-09-16T15:07:32.438 INFO:tasks.rados.rados.0.smithi162.stdout:1599: writing smithi16234619-12 from 632095 to 1095588 tid 1 2024-09-16T15:07:32.443 INFO:tasks.rados.rados.0.smithi162.stdout:1599: writing smithi16234619-12 from 1616319 to 2038853 tid 2 2024-09-16T15:07:32.449 INFO:tasks.rados.rados.0.smithi162.stdout:1599: writing smithi16234619-12 from 2748686 to 3235990 tid 3 2024-09-16T15:07:32.449 INFO:tasks.rados.rados.0.smithi162.stdout:1600: snap_create 2024-09-16T15:07:32.449 INFO:tasks.rados.rados.0.smithi162.stdout:1597: expect (ObjNum 516 snap 135 seq_num 516) 2024-09-16T15:07:32.606 INFO:tasks.rados.rados.0.smithi162.stdout:1594: finishing write tid 1 to smithi16234619-29 2024-09-16T15:07:32.607 INFO:tasks.rados.rados.0.smithi162.stdout:1596: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:07:32.607 INFO:tasks.rados.rados.0.smithi162.stdout:1595: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:07:32.717 INFO:tasks.rados.rados.0.smithi162.stdout:1594: finishing write tid 2 to smithi16234619-29 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: finishing write tid 3 to smithi16234619-29 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: finishing write tid 4 to smithi16234619-29 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: finishing write tid 5 to smithi16234619-29 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: oid 29 updating version 0 to 729 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: oid 29 version 729 is already newer than 728 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: oid 29 version 729 is already newer than 727 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: oid 29 version 729 is already newer than 726 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 729 (ObjNum 584 snap 146 seq_num 584) dirty exists 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1594: left oid 29 (ObjNum 584 snap 146 seq_num 584) 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1596: finishing copy_from to smithi16234619-15 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 512 (ObjNum 545 snap 140 seq_num 545) dirty exists 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1599: finishing write tid 1 to smithi16234619-12 2024-09-16T15:07:32.718 INFO:tasks.rados.rados.0.smithi162.stdout:1599: finishing write tid 2 to smithi16234619-12 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: finishing write tid 3 to smithi16234619-12 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: finishing write tid 4 to smithi16234619-12 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: finishing write tid 5 to smithi16234619-12 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: oid 12 updating version 0 to 271 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: oid 12 version 271 is already newer than 270 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: oid 12 updating version 271 to 272 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: oid 12 updating version 272 to 273 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 273 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:07:32.719 INFO:tasks.rados.rados.0.smithi162.stdout:1599: left oid 12 (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:07:33.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.486+0000 7f25c6ffd640 10 monclient: tick 2024-09-16T15:07:33.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.486+0000 7f25c6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.486+0000 7f25c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.486+0000 7f25c6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d011fc60 auth_method 0 2024-09-16T15:07:33.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:33.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 10 monclient(hunting): handle_auth_done global_id 54102 payload 824 2024-09-16T15:07:33.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.497+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25d0061ae0 con 0x7f25d011fc60 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.499+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.499+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25b80158f0 tx=0x7f25b8015db0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25b8005490 con 0x7f25d011fc60 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:33.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 10 monclient: dump: 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.502 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f25b800d030 con 0x7f25d011fc60 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.500+0000 7f25c7fff640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25b80158f0 tx=0x7f25b8015db0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.501+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 msgr2=0x7f25d0120040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.501+0000 7f25c7fff640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d011fc60 0x7f25d0120040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.507+0000 7f25d5672640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.507+0000 7f25d5672640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 0x7f25d009e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.507+0000 7f25d5672640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.507+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0122750 con 0x7f25d0120580 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 0x7f25d009e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient: get_auth_request con 0x7f25d0120580 auth_method 0 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 10 monclient(hunting): handle_auth_done global_id 54111 payload 824 2024-09-16T15:07:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.508+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f25d007a680 con 0x7f25d0120580 2024-09-16T15:07:33.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.510+0000 7f25c67fc640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 msgr2=0x7f25d009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.510+0000 7f25c67fc640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 0x7f25d009e520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25b80158f0 tx=0x7f25b80036f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25b8024040 con 0x7f25d0120580 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 10 monclient: dump: 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f25b8007450 con 0x7f25d0120580 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 msgr2=0x7f25d009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 0x7f25d009e520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25b80158f0 tx=0x7f25b80036f0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 msgr2=0x7f25d009e520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.511+0000 7f25c7fff640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25d0120580 0x7f25d009e520 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25b80158f0 tx=0x7f25b80036f0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.517 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:33.518 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.519 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:33.520 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:33.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.526 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:33.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:33.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.537 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:33.537 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 >> v2:172.21.15.162:6800/980547263 conn(0x7f25b0038880 msgr2=0x7f25b003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 --2- 172.21.15.134:0/2549193565 >> v2:172.21.15.162:6800/980547263 conn(0x7f25b0038880 0x7f25b003ad40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25c00097a0 tx=0x7f25c002e000 comp rx=0 tx=0).stop 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 10 monclient: shutdown 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 msgr2=0x7f25d0136150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25cf7fe640 1 -- 172.21.15.134:0/2549193565 reap_dead start 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009cb0 tx=0x7f25c800c490 comp rx=0 tx=0).stop 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 shutdown_connections 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 --2- 172.21.15.134:0/2549193565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25d013c370 0x7f25d0136150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 >> 172.21.15.134:0/2549193565 conn(0x7f25d00a6110 msgr2=0x7f25d006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 shutdown_connections 2024-09-16T15:07:33.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.512+0000 7f25d5672640 1 -- 172.21.15.134:0/2549193565 wait complete. 2024-09-16T15:07:33.539 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:33.539 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:07:33.540 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1594: done (5 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1595: done (4 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1596: done (3 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1597: done (2 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1599: done (1 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1600: done (0 left) 2024-09-16T15:07:33.561 INFO:tasks.rados.rados.0.smithi162.stdout:1601: snap_remove snap 146 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.645+0000 7fabfe357640 10 monclient: get_monmap_and_config 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.645+0000 7fabfe357640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.645+0000 7fabfe357640 10 monclient: monmap: 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:33.646559+0000 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:33.646559+0000 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.647 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.646+0000 7fabfe357640 1 Processor -- start 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.646+0000 7fabfe357640 1 -- start start 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.646+0000 7fabfe357640 10 monclient: init 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient: _add_conns ranks=[9,2,4,7,6,8,0,1,3,5,10] 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fabf8133f10 0x7fabf81342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-a con 0x7fabf8133f10 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-b con 0x7fabf8134830 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-c con 0x7fabf8071bd0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80a3d50 con 0x7fabf8133f10 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806f630 con 0x7fabf8134830 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80a1e10 con 0x7fabf8071bd0 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfe357640 10 monclient(hunting): authenticate will time out at 1190.236186s 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fabf8133f10 0x7fabf81342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fabf8133f10 0x7fabf81342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42822/0 (socket says 172.21.15.134:42822) 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40256/0 (socket says 172.21.15.134:40256) 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:35194/0 (socket says 172.21.15.134:35194) 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 -- 172.21.15.134:0/743071600 learned_addr learned my addr 172.21.15.134:0/743071600 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): get_auth_request con 0x7fabf8133f10 auth_method 0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): get_auth_request con 0x7fabf8134830 auth_method 0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): get_auth_request con 0x7fabf8071bd0 auth_method 0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient(hunting): handle_auth_done global_id 5076 payload 274 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 -- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 msgr2=0x7fabf81351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 -- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 msgr2=0x7fabf8134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 monclient: found mon.noname-a 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabf7fff640 1 -- 172.21.15.134:0/743071600 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8073250 con 0x7fabf8133f10 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabeffff640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:33.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.647+0000 7fabfc8f4640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf7fff640 1 -- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 msgr2=0x7fabf81342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf7fff640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81342f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009870 tx=0x7fabe802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbb4b26bb9bce18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 1 -- 172.21.15.134:0/743071600 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe802ae00 con 0x7fabf8133f10 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: dump: 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 1 -- 172.21.15.134:0/743071600 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe80307b0 con 0x7fabf8133f10 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabfe357640 5 monclient: authenticate success, global_id 5076 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabfe357640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabfe357640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 1 -- 172.21.15.134:0/743071600 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe8030dd0 con 0x7fabf8133f10 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabfe357640 4 set_mon_vals no callback set 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.650+0000 7fabf77fe640 10 monclient: dump: 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.664 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 10 monclient: shutdown 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 -- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 msgr2=0x7fabf81342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81342f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009870 tx=0x7fabe802a0a0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 -- 172.21.15.134:0/743071600 shutdown_connections 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fabf8071bd0 0x7fabf81351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf8134830 0x7fabf8134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 --2- 172.21.15.134:0/743071600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81342f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 -- 172.21.15.134:0/743071600 >> 172.21.15.134:0/743071600 conn(0x7fabf80a7410 msgr2=0x7fabf80a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 -- 172.21.15.134:0/743071600 shutdown_connections 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 1 -- 172.21.15.134:0/743071600 wait complete. 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.651+0000 7fabfe357640 10 monclient: monmap: 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:33.666 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 Processor -- start 2024-09-16T15:07:33.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 -- start start 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient: init 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient: _add_conns ranks=[3,0,6,5,8,2,4,1,7] 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabf8071bd0 0x7fabf80a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-g con 0x7fabf8071bd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-a con 0x7fabf8133f10 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fabf8134830 0x7fabf81cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): picked mon.noname-d con 0x7fabf8134830 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806caa0 con 0x7fabf8133f10 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8122ab0 con 0x7fabf8071bd0 2024-09-16T15:07:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806c5f0 con 0x7fabf8134830 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfe357640 10 monclient(hunting): authenticate will time out at 1190.241667s 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42828/0 (socket says 172.21.15.134:42828) 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 -- 172.21.15.134:0/425946771 learned_addr learned my addr 172.21.15.134:0/425946771 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): get_auth_request con 0x7fabf8133f10 auth_method 0 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fabf8134830 0x7fabf81cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): get_auth_request con 0x7fabf8134830 auth_method 0 2024-09-16T15:07:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabfc8f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 10 monclient(hunting): handle_auth_done global_id 5085 payload 824 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabf8071bd0 0x7fabf80a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fabf8134830 msgr2=0x7fabf81cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fabf8134830 0x7fabf81cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.652+0000 7fabeffff640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabf8071bd0 msgr2=0x7fabf80a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabf8071bd0 0x7fabf80a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 1 monclient: found mon.noname-a 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabeffff640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0009a70 con 0x7fabf8133f10 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabf7fff640 10 monclient: get_auth_request con 0x7fabf8071bd0 auth_method 0 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.653+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabf8071bd0 0x7fabf80a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabeffff640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 msgr2=0x7fabf81330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabeffff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e940 tx=0x7fabe000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe0013470 con 0x7fabf8133f10 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:33.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: dump: 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe00135d0 con 0x7fabf8133f10 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 5 monclient: authenticate success, global_id 5085 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 10 monclient: _renew_subs 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe0009400 con 0x7fabf8133f10 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe0013bb0 con 0x7fabf8133f10 2024-09-16T15:07:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf57fa640 10 monclient: dump: 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabf6ffd640 4 set_mon_vals no callback set 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 10 monclient: _renew_subs 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.655+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe80094d0 con 0x7fabf8133f10 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabeffff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e940 tx=0x7fabe000ed40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fabe0013d10 mgrmap(e 5) v1 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfe357640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fabe0013d10 con 0x7fabf8133f10 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfe357640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 0x7fabf81335e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806a3f0 con 0x7fabf81225e0 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf57fa640 1 --2- 172.21.15.134:0/425946771 >> v2:172.21.15.162:6800/980547263 conn(0x7fabd80388b0 0x7fabd803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 0x7fabf81335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient: get_auth_request con 0x7fabf81225e0 auth_method 0 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfc8f4640 1 --2- 172.21.15.134:0/425946771 >> v2:172.21.15.162:6800/980547263 conn(0x7fabd80388b0 0x7fabd803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabfc8f4640 10 monclient: get_auth_request con 0x7fabd80388b0 auth_method 0 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 10 monclient(hunting): handle_auth_done global_id 44330 payload 824 2024-09-16T15:07:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.656+0000 7fabf7fff640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabf8061ae0 con 0x7fabf81225e0 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf7fff640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 msgr2=0x7fabf81335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 0x7fabf81335e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fabf8135d70 tx=0x7fabe80046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabfc8f4640 1 --2- 172.21.15.134:0/425946771 >> v2:172.21.15.162:6800/980547263 conn(0x7fabd80388b0 0x7fabd803ad70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fabf806c4a0 tx=0x7fabf000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe8003610 con 0x7fabf81225e0 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 10 monclient: dump: 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.677 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fabe8030b30 con 0x7fabf81225e0 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 msgr2=0x7fabf81335e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 0x7fabf81335e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fabf8135d70 tx=0x7fabe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 msgr2=0x7fabf81335e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.659+0000 7fabf57fa640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81225e0 0x7fabf81335e0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fabf8135d70 tx=0x7fabe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.665+0000 7fabfe357640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:33.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.665+0000 7fabfe357640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 0x7fabf81cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.665+0000 7fabfe357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.665+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabf807a680 con 0x7fabf81cb8b0 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 0x7fabf81cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient: get_auth_request con 0x7fabf81cb8b0 auth_method 0 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 10 monclient(hunting): handle_auth_done global_id 44339 payload 824 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.666+0000 7fabf7fff640 1 -- 172.21.15.134:0/425946771 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fabf81cc200 con 0x7fabf81cb8b0 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.668+0000 7fabf7fff640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 msgr2=0x7fabf81cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.668+0000 7fabf7fff640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 0x7fabf81cbcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097c0 tx=0x7fabe8009240 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fabe8047cd0 con 0x7fabf81cb8b0 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 10 monclient: dump: 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.682 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fabe804b070 con 0x7fabf81cb8b0 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 msgr2=0x7fabf81cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 0x7fabf81cbcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097c0 tx=0x7fabe8009240 comp rx=0 tx=0).stop 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 msgr2=0x7fabf81cbcc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fabf81cb8b0 0x7fabf81cbcc0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097c0 tx=0x7fabe8009240 comp rx=0 tx=0).stop 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:33.684 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.686 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:33.687 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.688 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.690 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:33.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.694 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:33.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:33.706 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 >> v2:172.21.15.162:6800/980547263 conn(0x7fabd80388b0 msgr2=0x7fabd803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 1 --2- 172.21.15.134:0/425946771 >> v2:172.21.15.162:6800/980547263 conn(0x7fabd80388b0 0x7fabd803ad70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fabf806c4a0 tx=0x7fabf000b040 comp rx=0 tx=0).stop 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 10 monclient: shutdown 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 msgr2=0x7fabf81330a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfe357640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e940 tx=0x7fabe000ed40 comp rx=0 tx=0).stop 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabfc8f4640 1 -- 172.21.15.134:0/425946771 reap_dead start 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.669+0000 7fabf57fa640 1 -- 172.21.15.134:0/425946771 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(319..319 src has 1..319) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7fabe004a3c0 con 0x7fabf8133f10 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.670+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 shutdown_connections 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.670+0000 7fabfe357640 1 --2- 172.21.15.134:0/425946771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf8133f10 0x7fabf81330a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.670+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 >> 172.21.15.134:0/425946771 conn(0x7fabf80a7410 msgr2=0x7fabf8134c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.670+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 shutdown_connections 2024-09-16T15:07:33.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.670+0000 7fabfe357640 1 -- 172.21.15.134:0/425946771 wait complete. 2024-09-16T15:07:33.709 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:33.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:07:33.709 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.805+0000 7f8c1a62b640 10 monclient: get_monmap_and_config 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.805+0000 7f8c1a62b640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.806+0000 7f8c1a62b640 10 monclient: monmap: 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:33.806926+0000 2024-09-16T15:07:33.807 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:33.806926+0000 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:33.808 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.806+0000 7f8c1a62b640 1 Processor -- start 2024-09-16T15:07:33.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.806+0000 7f8c1a62b640 1 -- start start 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.806+0000 7f8c1a62b640 10 monclient: init 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient: _add_conns ranks=[2,0,4,7,6,5,10,3,9,8,1] 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-b con 0x7f8c14136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8c14137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c1413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-c con 0x7f8c1413c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c14073250 con 0x7f8c14137220 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140a56e0 con 0x7f8c14136e40 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1409abc0 con 0x7f8c1413c370 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c1a62b640 10 monclient(hunting): authenticate will time out at 1190.396301s 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c137fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8c14137220 0x7f8c14137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59332/0 (socket says 172.21.15.134:59332) 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c1413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c137fe640 1 -- 172.21.15.134:0/4086490363 learned_addr learned my addr 172.21.15.134:0/4086490363 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request con 0x7f8c1413c370 auth_method 0 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c13fff640 1 -- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 msgr2=0x7f8c1409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c13fff640 1 -- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 msgr2=0x7f8c1409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c13fff640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:33.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c13fff640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_done global_id 4972 payload 274 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 -- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 msgr2=0x7f8c1409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 -- 172.21.15.134:0/4086490363 >> v1:172.21.15.134:6789/0 conn(0x7f8c14137220 legacy=0x7f8c14137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 monclient: found mon.noname-c 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:33.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.807+0000 7f8c18bc8640 1 -- 172.21.15.134:0/4086490363 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1409ba50 con 0x7f8c1413c370 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c18bc8640 1 -- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 msgr2=0x7f8c1413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c1413e760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009dc0 tx=0x7f8c0c02a4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=eb8befe3b2388602 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 1 -- 172.21.15.134:0/4086490363 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c007ca0 con 0x7f8c1413c370 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: dump: 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.820 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.821 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 1 -- 172.21.15.134:0/4086490363 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c02d050 con 0x7f8c1413c370 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c1a62b640 5 monclient: authenticate success, global_id 4972 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c1a62b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c1a62b640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 1 -- 172.21.15.134:0/4086490363 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c007e20 con 0x7f8c1413c370 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c1a62b640 4 set_mon_vals no callback set 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.810+0000 7f8c12ffd640 10 monclient: dump: 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.822 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 10 monclient: shutdown 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 -- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 msgr2=0x7f8c1413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c1413e760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009dc0 tx=0x7f8c0c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 -- 172.21.15.134:0/4086490363 shutdown_connections 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c1413e760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/4086490363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 -- 172.21.15.134:0/4086490363 >> 172.21.15.134:0/4086490363 conn(0x7f8c140a6110 msgr2=0x7f8c140a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 -- 172.21.15.134:0/4086490363 shutdown_connections 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 1 -- 172.21.15.134:0/4086490363 wait complete. 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.811+0000 7f8c1a62b640 10 monclient: monmap: 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 Processor -- start 2024-09-16T15:07:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 -- start start 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient: init 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient: _add_conns ranks=[2,3,4,6,0,1,7,5,8] 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-b con 0x7f8c14136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c14137220 0x7f8c141201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-g con 0x7f8c14137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c141206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): picked mon.noname-c con 0x7f8c1413c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1413b5a0 con 0x7f8c14136e40 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141227b0 con 0x7f8c14137220 2024-09-16T15:07:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140a7af0 con 0x7f8c1413c370 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c1a62b640 10 monclient(hunting): authenticate will time out at 1190.401884s 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1411fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40270/0 (socket says 172.21.15.134:40270) 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 1 -- 172.21.15.134:0/3884069645 learned_addr learned my addr 172.21.15.134:0/3884069645 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 10 monclient(hunting): get_auth_request con 0x7f8c14136e40 auth_method 0 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c13fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.812+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c141206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request con 0x7f8c1413c370 auth_method 0 2024-09-16T15:07:33.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c137fe640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c14137220 0x7f8c141201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c137fe640 10 monclient(hunting): get_auth_request con 0x7f8c14137220 auth_method 0 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient(hunting): handle_auth_done global_id 44426 payload 824 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 msgr2=0x7f8c141206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c141206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c14137220 msgr2=0x7f8c141201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c14137220 0x7f8c141201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c13fff640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c009a70 con 0x7f8c14136e40 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.813+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8c1413c370 0x7f8c141206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c13fff640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 msgr2=0x7f8c1411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c13fff640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1411fc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c7f0 tx=0x7f8c0400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c04007ca0 con 0x7f8c14136e40 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: dump: 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c04007e40 con 0x7f8c14136e40 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 5 monclient: authenticate success, global_id 44426 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 10 monclient: _renew_subs 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c14120fb0 con 0x7f8c14136e40 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c04011860 con 0x7f8c14136e40 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c10ff9640 10 monclient: dump: 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.832 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c127fc640 4 set_mon_vals no callback set 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 10 monclient: _renew_subs 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.815+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140a3470 con 0x7f8c14136e40 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8c040113f0 con 0x7f8c14136e40 2024-09-16T15:07:33.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c1a62b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c10ff9640 1 --2- 172.21.15.134:0/3884069645 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bf4038880 0x7f8bf403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 0x7f8c14136050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14071b20 con 0x7f8c140a2b40 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.816+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(319..319 src has 1..319) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f8c04049e10 con 0x7f8c14136e40 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c137fe640 1 --2- 172.21.15.134:0/3884069645 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bf4038880 0x7f8bf403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 0x7f8c14136050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c137fe640 10 monclient: get_auth_request con 0x7f8bf4038880 auth_method 0 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient: get_auth_request con 0x7f8c140a2b40 auth_method 0 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_done global_id 54106 payload 824 2024-09-16T15:07:33.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.817+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c14122920 con 0x7f8c140a2b40 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c137fe640 1 --2- 172.21.15.134:0/3884069645 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bf4038880 0x7f8bf403ad40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c140a28a0 tx=0x7f8bfc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 msgr2=0x7f8c14136050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 0x7f8c14136050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c1406cb00 tx=0x7f8c0c004a40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c003e00 con 0x7f8c140a2b40 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c10ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.819+0000 7f8c10ff9640 10 monclient: dump: 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8c0c004690 con 0x7f8c140a2b40 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 msgr2=0x7f8c14136050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 0x7f8c14136050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c1406cb00 tx=0x7f8c0c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 msgr2=0x7f8c14136050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.820+0000 7f8c10ff9640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c140a2b40 0x7f8c14136050 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c1406cb00 tx=0x7f8c0c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c1a62b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 0x7f8c1402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c1a62b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14061ae0 con 0x7f8c1402b260 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 0x7f8c1402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient: get_auth_request con 0x7f8c1402b260 auth_method 0 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 10 monclient(hunting): handle_auth_done global_id 54115 payload 824 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.826+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8c1407a680 con 0x7f8c1402b260 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 msgr2=0x7f8c1402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c18bc8640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 0x7f8c1402b670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c14135790 tx=0x7f8c0c030000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0c04a330 con 0x7f8c1402b260 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 10 monclient: dump: 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f8c0c045440 con 0x7f8c1402b260 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 msgr2=0x7f8c1402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 0x7f8c1402b670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c14135790 tx=0x7f8c0c030000 comp rx=0 tx=0).stop 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 msgr2=0x7f8c1402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.829+0000 7f8c10ff9640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8c1402b260 0x7f8c1402b670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:33.842 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.843 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.844 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:33.845 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:33.846 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:33.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.848 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:33.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:33.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:33.863 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:33.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bf4038880 msgr2=0x7f8bf403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/3884069645 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bf4038880 0x7f8bf403ad40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c140a28a0 tx=0x7f8bfc009040 comp rx=0 tx=0).stop 2024-09-16T15:07:33.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 10 monclient: shutdown 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 msgr2=0x7f8c1411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 --2- 172.21.15.134:0/3884069645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c14136e40 0x7f8c1411fc60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c7f0 tx=0x7f8c0400ccb0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 reap_dead start 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c18bc8640 1 -- 172.21.15.134:0/3884069645 reap_dead start 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 shutdown_connections 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 >> 172.21.15.134:0/3884069645 conn(0x7f8c140a6110 msgr2=0x7f8c1413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 shutdown_connections 2024-09-16T15:07:33.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.830+0000 7f8c1a62b640 1 -- 172.21.15.134:0/3884069645 wait complete. 2024-09-16T15:07:33.865 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:33.865 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:07:33.865 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 10 monclient: get_monmap_and_config 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 10 monclient: monmap: 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:33.961105+0000 2024-09-16T15:07:33.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:33.961105+0000 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:33.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 1 Processor -- start 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 1 -- start start 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.960+0000 7f9118cd7640 10 monclient: init 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient: _add_conns ranks=[10,3,9,8,2,7,6,4,5,0,1] 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91140a1d30 0x7f9114071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-f con 0x7f91140a1d30 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91140717c0 0x7f91140a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-g con 0x7f91140717c0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91140a2650 0x7f9114134af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-a con 0x7f91140a2650 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140704d0 con 0x7f91140a2650 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140a37f0 con 0x7f91140a1d30 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91140a39f0 con 0x7f91140717c0 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9118cd7640 10 monclient(hunting): authenticate will time out at 1190.550538s 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91140a2650 0x7f9114134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91140a2650 0x7f9114134af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42836/0 (socket says 172.21.15.134:42836) 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 -- 172.21.15.134:0/1139693797 learned_addr learned my addr 172.21.15.134:0/1139693797 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:33.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): get_auth_request con 0x7f91140a2650 auth_method 0 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9111d9b640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91140717c0 0x7f91140a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f911259c640 1 --2- 172.21.15.134:0/1139693797 >> v2:172.21.15.162:3300/0 conn(0x7f91140a1d30 0x7f9114071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9111d9b640 10 monclient(hunting): get_auth_request con 0x7f91140717c0 auth_method 0 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9111d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9111d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9111d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f911259c640 10 monclient(hunting): get_auth_request con 0x7f91140a1d30 auth_method 0 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f911259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f911259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f911259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient(hunting): handle_auth_done global_id 5094 payload 274 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 -- 172.21.15.134:0/1139693797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91140717c0 msgr2=0x7f91140a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91140717c0 0x7f91140a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 -- 172.21.15.134:0/1139693797 >> v2:172.21.15.162:3300/0 conn(0x7f91140a1d30 msgr2=0x7f9114071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 --2- 172.21.15.134:0/1139693797 >> v2:172.21.15.162:3300/0 conn(0x7f91140a1d30 0x7f9114071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 monclient: found mon.noname-a 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:33.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.961+0000 7f9112d9d640 1 -- 172.21.15.134:0/1139693797 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114073250 con 0x7f91140a2650 2024-09-16T15:07:33.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9112d9d640 1 -- 172.21.15.134:0/1139693797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140a2650 msgr2=0x7f9114134af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9112d9d640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140a2650 0x7f9114134af0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9114072350 tx=0x7f910402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd941c3ad766a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 1 -- 172.21.15.134:0/1139693797 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f910402ae10 con 0x7f91140a2650 2024-09-16T15:07:33.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: dump: 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.974 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9118cd7640 5 monclient: authenticate success, global_id 5094 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9118cd7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 1 -- 172.21.15.134:0/1139693797 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91040307b0 con 0x7f91140a2650 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9118cd7640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f9118cd7640 4 set_mon_vals no callback set 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 1 -- 172.21.15.134:0/1139693797 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9104030dd0 con 0x7f91140a2650 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.964+0000 7f911159a640 10 monclient: dump: 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 10 monclient: shutdown 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 -- 172.21.15.134:0/1139693797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140a2650 msgr2=0x7f9114134af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140a2650 0x7f9114134af0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9114072350 tx=0x7f910402a0b0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 -- 172.21.15.134:0/1139693797 shutdown_connections 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140a2650 0x7f9114134af0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 --2- 172.21.15.134:0/1139693797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91140717c0 0x7f91140a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 --2- 172.21.15.134:0/1139693797 >> v2:172.21.15.162:3300/0 conn(0x7f91140a1d30 0x7f9114071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 -- 172.21.15.134:0/1139693797 >> 172.21.15.134:0/1139693797 conn(0x7f911409abf0 msgr2=0x7f91140a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 -- 172.21.15.134:0/1139693797 shutdown_connections 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.965+0000 7f9118cd7640 1 -- 172.21.15.134:0/1139693797 wait complete. 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient: build_initial_monmap 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient: monmap: 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 Processor -- start 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 -- start start 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient: init 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient: _add_conns ranks=[6,5,1,7,4,2,3,0,8] 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-d con 0x7f91140717c0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f91140a1d30 0x7f91141350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-h con 0x7f91140a1d30 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 0x7f9114121590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): picked mon.noname-f con 0x7f91140a2650 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140a7340 con 0x7f91140a2650 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9114123720 con 0x7f91140a1d30 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f911409d7d0 con 0x7f91140717c0 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.966+0000 7f9118cd7640 10 monclient(hunting): authenticate will time out at 1190.555986s 2024-09-16T15:07:33.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58094/0 (socket says 172.21.15.134:58094) 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 -- 172.21.15.134:0/375069082 learned_addr learned my addr 172.21.15.134:0/375069082 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): get_auth_request con 0x7f91140717c0 auth_method 0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 0x7f9114121590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 10 monclient(hunting): get_auth_request con 0x7f91140a2650 auth_method 0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9111d9b640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f91140a1d30 0x7f91141350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9111d9b640 10 monclient(hunting): get_auth_request con 0x7f91140a1d30 auth_method 0 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9111d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9111d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9111d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient(hunting): handle_auth_done global_id 34584 payload 824 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 -- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f91140a1d30 msgr2=0x7f91141350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f91140a1d30 0x7f91141350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 -- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 msgr2=0x7f9114121590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 0x7f9114121590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 monclient: found mon.noname-d 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f911259c640 1 -- 172.21.15.134:0/375069082 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040094d0 con 0x7f91140717c0 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.967+0000 7f9112d9d640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 0x7f9114121590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f911259c640 1 -- 172.21.15.134:0/375069082 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 msgr2=0x7f9114134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f911259c640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f910c00d800 tx=0x7f910c00dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f910c00bd40 con 0x7f91140717c0 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: dump: 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 5 monclient: authenticate success, global_id 34584 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c014430 con 0x7f91140717c0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 10 monclient: _renew_subs 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911409db00 con 0x7f91140717c0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f910c014a10 con 0x7f91140717c0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9110d99640 4 set_mon_vals no callback set 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9102ffd640 10 monclient: dump: 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 10 monclient: _renew_subs 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.969+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9114125570 con 0x7f91140717c0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.970+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f910c014b90 con 0x7f91140717c0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.970+0000 7f9118cd7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.970+0000 7f9118cd7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.970+0000 7f9118cd7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.970+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91140a2110 con 0x7f91140717c0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.971+0000 7f9102ffd640 1 --2- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 0x7f90f403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.971+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(319..319 src has 1..319) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f910c04a190 con 0x7f91140717c0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.971+0000 7f9111d9b640 1 --2- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 0x7f90f403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.971+0000 7f9111d9b640 10 monclient: get_auth_request con 0x7f90f40388b0 auth_method 0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.973+0000 7f9111d9b640 1 --2- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 0x7f90f403ad70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f91140a4e90 tx=0x7f9108003990 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.975+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f910c014e40 con 0x7f91140717c0 2024-09-16T15:07:33.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.975+0000 7f9102ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:33.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:33.975+0000 7f9102ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:34.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.199+0000 7f9118cd7640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:07:34.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.199+0000 7f9118cd7640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:07:34.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.199+0000 7f9118cd7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:34.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.199+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f9114124e60 con 0x7f91140717c0 2024-09-16T15:07:34.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.201+0000 7f9102ffd640 1 -- 172.21.15.134:0/375069082 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f910c011340 con 0x7f91140717c0 2024-09-16T15:07:34.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.201+0000 7f9102ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:07:34.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.201+0000 7f9102ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=13,config=2,health=11,kv=12,logm=62} crc {auth=2594611666,config=2586702516,health=3819404006,kv=1090497770,logm=3161208282}) 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=3552294427}) 2024-09-16T15:07:34.204 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=2594611666,config=2586702516,health=1361790972,kv=174400417,logm=1293655992}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=2594611666,config=2586702516,health=585825456,kv=3324725102,logm=2466321607}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=2594611666,config=2586702516,health=1361790972,kv=174400417,logm=1990624297}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=2283116316}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.205 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=13,config=2,health=12,kv=11,logm=62} crc {auth=2594611666,config=2586702516,health=1361790972,kv=318701324,logm=2471250392}) 2024-09-16T15:07:34.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=2594611666,config=2586702516,health=1361790972,kv=1090497770,logm=4014103205}) 2024-09-16T15:07:34.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=13,config=2,health=11,kv=11,logm=63} crc {auth=2594611666,config=2586702516,health=3649345650,kv=1671578860,logm=2692689408}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1494837424}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=138552186}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2594543571}) 2024-09-16T15:07:34.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2994300542}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1956220330}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1726574492}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=556636576}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643146681}) 2024-09-16T15:07:34.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=410312458}) 2024-09-16T15:07:34.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 msgr2=0x7f90f403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 0x7f90f403ad70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f91140a4e90 tx=0x7f9108003990 comp rx=0 tx=0).stop 2024-09-16T15:07:34.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 10 monclient: shutdown 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 msgr2=0x7f9114134b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f910c00d800 tx=0x7f910c00dcc0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 shutdown_connections 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> v2:172.21.15.162:6800/980547263 conn(0x7f90f40388b0 0x7f90f403ad70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91140a2650 0x7f9114121590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f91140a1d30 0x7f91141350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 --2- 172.21.15.134:0/375069082 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f91140717c0 0x7f9114134b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.204+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 >> 172.21.15.134:0/375069082 conn(0x7f911409abf0 msgr2=0x7f91140a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 shutdown_connections 2024-09-16T15:07:34.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f9118cd7640 1 -- 172.21.15.134:0/375069082 wait complete. 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4223099359}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1999330293}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1584825943}) 2024-09-16T15:07:34.213 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4187706076}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4245129979}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1874139770}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=626596104}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520049867}) 2024-09-16T15:07:34.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2583898898}) 2024-09-16T15:07:34.217 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4076226874}) 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1573405906}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1883535481}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2148920607}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3904188794}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1296458471}) 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=569211677}) 2024-09-16T15:07:34.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557083972}) 2024-09-16T15:07:34.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.553+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1778891373}) 2024-09-16T15:07:34.560 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.560 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1171968511}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=954320974}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3217327230}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4081930188}) 2024-09-16T15:07:34.561 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.558+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3961270065}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=410716396}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3415403856}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=754951623}) 2024-09-16T15:07:34.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.559+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1433412867}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3944822635}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2553801319}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2740372503}) 2024-09-16T15:07:34.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.571+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2259687070}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1746112560}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2741211629}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=805443458}) 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311013725}) 2024-09-16T15:07:34.577 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.572+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=705704900}) 2024-09-16T15:07:34.577 INFO:tasks.rados.rados.0.smithi162.stdout:1601: done (0 left) 2024-09-16T15:07:34.577 INFO:tasks.rados.rados.0.smithi162.stdout:1602: write (excl) oid 19 current snap is 147 2024-09-16T15:07:34.577 INFO:tasks.rados.rados.0.smithi162.stdout:1602: seq_num 586 ranges {648902=655095,2074801=566192,3176109=672793} 2024-09-16T15:07:34.582 INFO:tasks.rados.rados.0.smithi162.stdout:1602: writing smithi16234619-19 from 648902 to 1303997 tid 1 2024-09-16T15:07:34.589 INFO:tasks.rados.rados.0.smithi162.stdout:1602: writing smithi16234619-19 from 2074801 to 2640993 tid 2 2024-09-16T15:07:34.596 INFO:tasks.rados.rados.0.smithi162.stdout:1602: writing smithi16234619-19 from 3176109 to 3848902 tid 3 2024-09-16T15:07:34.596 INFO:tasks.rados.rados.0.smithi162.stdout:1603: snap_create 2024-09-16T15:07:34.628 INFO:tasks.rados.rados.0.smithi162.stdout:1602: finishing write tid 1 to smithi16234619-19 2024-09-16T15:07:34.628 INFO:tasks.rados.rados.0.smithi162.stdout:1602: finishing write tid 2 to smithi16234619-19 2024-09-16T15:07:34.630 INFO:tasks.rados.rados.0.smithi162.stdout:1602: finishing write tid 3 to smithi16234619-19 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: finishing write tid 4 to smithi16234619-19 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: finishing write tid 5 to smithi16234619-19 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: oid 19 updating version 0 to 577 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: oid 19 updating version 577 to 578 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: oid 19 version 578 is already newer than 576 2024-09-16T15:07:34.633 INFO:tasks.rados.rados.0.smithi162.stdout:1602: oid 19 updating version 578 to 579 2024-09-16T15:07:34.634 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 579 (ObjNum 586 snap 147 seq_num 586) dirty exists 2024-09-16T15:07:34.634 INFO:tasks.rados.rados.0.smithi162.stdout:1602: left oid 19 (ObjNum 586 snap 147 seq_num 586) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3425850156}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=551901450}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1952227658}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1555081528}) 2024-09-16T15:07:34.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2988141794}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3618944812}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2764195536}) 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:34.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=316001962}) 2024-09-16T15:07:34.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:34.660+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4291331357}) 2024-09-16T15:07:35.218 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:07:35.219 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.314+0000 7f766bf18640 10 monclient: get_monmap_and_config 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.314+0000 7f766bf18640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.314+0000 7f766bf18640 10 monclient: monmap: 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:35.315652+0000 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:35.315652+0000 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:35.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 Processor -- start 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 -- start start 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient: init 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient: _add_conns ranks=[1,7,6,9,3,8,5,4,10,0,2] 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7664136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f766409bfe0 0x7f766409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-i con 0x7f766409bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-d con 0x7f766413bf80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7664073250 con 0x7f7664136030 2024-09-16T15:07:35.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f766409b600 con 0x7f766413bf80 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f76640a5740 con 0x7f766409bfe0 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.315+0000 7f766bf18640 10 monclient(hunting): authenticate will time out at 1191.904980s 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58096/0 (socket says 172.21.15.134:58096) 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 -- 172.21.15.134:0/1754843816 learned_addr learned my addr 172.21.15.134:0/1754843816 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): get_auth_request con 0x7f766413bf80 auth_method 0 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f76694b3640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f766409bfe0 0x7f766409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f76694b3640 10 monclient(hunting): get_auth_request con 0x7f766409bfe0 auth_method 0 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f76694b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f76694b3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f76694b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient(hunting): handle_auth_done global_id 34593 payload 274 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 -- 172.21.15.134:0/1754843816 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f766409bfe0 msgr2=0x7f766409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f766409bfe0 0x7f766409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 -- 172.21.15.134:0/1754843816 >> v1:172.21.15.162:6789/0 conn(0x7f7664136030 legacy=0x7f7664136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 monclient: found mon.noname-d 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:35.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.316+0000 7f766a4b5640 1 -- 172.21.15.134:0/1754843816 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640a52c0 con 0x7f766413bf80 2024-09-16T15:07:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766a4b5640 1 -- 172.21.15.134:0/1754843816 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 msgr2=0x7f766413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766a4b5640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f765c009e80 tx=0x7f765c02a580 comp rx=0 tx=0).ready entity=mon.6 client_cookie=372675ed9f775b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 1 -- 172.21.15.134:0/1754843816 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f765c007a70 con 0x7f766413bf80 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: dump: 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.329 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 1 -- 172.21.15.134:0/1754843816 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c02d050 con 0x7f766413bf80 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 5 monclient: authenticate success, global_id 34593 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 1 -- 172.21.15.134:0/1754843816 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f765c007bf0 con 0x7f766413bf80 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 4 set_mon_vals no callback set 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f7668cb2640 10 monclient: dump: 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 10 monclient: shutdown 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 -- 172.21.15.134:0/1754843816 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 msgr2=0x7f766413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f765c009e80 tx=0x7f765c02a580 comp rx=0 tx=0).stop 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 -- 172.21.15.134:0/1754843816 shutdown_connections 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f766413bf80 0x7f766413e370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 --2- 172.21.15.134:0/1754843816 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f766409bfe0 0x7f766409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 -- 172.21.15.134:0/1754843816 >> 172.21.15.134:0/1754843816 conn(0x7f76640a9ea0 msgr2=0x7f76640a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 -- 172.21.15.134:0/1754843816 shutdown_connections 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.319+0000 7f766bf18640 1 -- 172.21.15.134:0/1754843816 wait complete. 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.320+0000 7f766bf18640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.320+0000 7f766bf18640 10 monclient: monmap: 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.320+0000 7f766bf18640 1 Processor -- start 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.320+0000 7f766bf18640 1 -- start start 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.320+0000 7f766bf18640 10 monclient: init 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient: _add_conns ranks=[3,2,5,1,0,7,6,8,4] 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 0x7f7664133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-g con 0x7f766409bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-b con 0x7f7664136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 0x7f7664138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): picked mon.noname-h con 0x7f766413bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7664120020 con 0x7f7664136030 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76640a31c0 con 0x7f766409bfe0 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76640a4830 con 0x7f766413bf80 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766bf18640 10 monclient(hunting): authenticate will time out at 1191.910277s 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40272/0 (socket says 172.21.15.134:40272) 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 -- 172.21.15.134:0/360288534 learned_addr learned my addr 172.21.15.134:0/360288534 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): get_auth_request con 0x7f7664136030 auth_method 0 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient(hunting): handle_auth_done global_id 44435 payload 824 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f7669cb4640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 0x7f7664133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766a4b5640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 0x7f7664138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 -- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 msgr2=0x7f7664138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 0x7f7664138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 -- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 msgr2=0x7f7664133e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 0x7f7664133e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 monclient: found mon.noname-b 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f76694b3640 1 -- 172.21.15.134:0/360288534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c009b30 con 0x7f7664136030 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f7669cb4640 10 monclient: get_auth_request con 0x7f766409bfe0 auth_method 0 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f7669cb4640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 0x7f7664133e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766a4b5640 10 monclient: get_auth_request con 0x7f766413bf80 auth_method 0 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.321+0000 7f766a4b5640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 0x7f7664138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.323+0000 7f76694b3640 1 -- 172.21.15.134:0/360288534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 msgr2=0x7f76641343a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.323+0000 7f76694b3640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f766000d7d0 tx=0x7f766000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.323+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7660018070 con 0x7f7664136030 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.323+0000 7f765a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: dump: 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766000bd40 con 0x7f7664136030 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 5 monclient: authenticate success, global_id 44435 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _renew_subs 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76640a82d0 con 0x7f7664136030 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76600148a0 con 0x7f7664136030 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 10 monclient: dump: 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _renew_subs 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765bfff640 4 set_mon_vals no callback set 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766413a700 con 0x7f7664136030 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7660014430 con 0x7f7664136030 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.324+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766409c400 con 0x7f7664136030 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.325+0000 7f765a7fc640 1 --2- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 0x7f764803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.325+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(320..320 src has 1..320) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f766004a0e0 con 0x7f7664136030 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.325+0000 7f7669cb4640 1 --2- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 0x7f764803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.325+0000 7f7669cb4640 10 monclient: get_auth_request con 0x7f7648038880 auth_method 0 2024-09-16T15:07:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.327+0000 7f7669cb4640 1 --2- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 0x7f764803ad40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7654003eb0 tx=0x7f7654009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.329+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f76600146e0 con 0x7f7664136030 2024-09-16T15:07:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.329+0000 7f765a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.329+0000 7f765a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2926428478}) 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.380 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1097344698}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1794379371}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1101010940}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1577068244}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=104473916}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.381 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=556491844}) 2024-09-16T15:07:35.382 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.382 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3947130628}) 2024-09-16T15:07:35.382 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.378+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2627036393}) 2024-09-16T15:07:35.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3410784810}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3655995353}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=333009597}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3231133452}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=245735392}) 2024-09-16T15:07:35.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3471650204}) 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2565023107}) 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3916157449}) 2024-09-16T15:07:35.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.419+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1376611324}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4122650593}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3816572162}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3402382470}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2108550891}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2741567136}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1324532419}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1321700317}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3115269158}) 2024-09-16T15:07:35.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.425+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2733756351}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2435352227}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1314051291}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2419764222}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3354899884}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.431+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=889406714}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3758557939}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3321289576}) 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2941647615}) 2024-09-16T15:07:35.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.432+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=367019834}) 2024-09-16T15:07:35.438 INFO:tasks.rados.rados.0.smithi162.stdout:1602: done (1 left) 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1603: done (0 left) 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1604: read oid 44 snap -1 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1604: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1605: read oid 43 snap -1 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1605: expect (ObjNum 568 snap 143 seq_num 568) 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1606: rollback oid 2 current snap is 148 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 134 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1607: rollback oid 18 current snap is 148 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 147 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1608: read oid 28 snap -1 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1608: expect (ObjNum 538 snap 139 seq_num 538) 2024-09-16T15:07:35.439 INFO:tasks.rados.rados.0.smithi162.stdout:1609: snap_create 2024-09-16T15:07:35.444 INFO:tasks.rados.rados.0.smithi162.stdout:1604: expect (ObjNum 468 snap 116 seq_num 468) 2024-09-16T15:07:35.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.563+0000 7f766bf18640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:35.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.563+0000 7f766bf18640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:35.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.563+0000 7f766bf18640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.563+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f766413b020 con 0x7f7664136030 2024-09-16T15:07:35.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.564+0000 7f765a7fc640 1 -- 172.21.15.134:0/360288534 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f7660011340 con 0x7f7664136030 2024-09-16T15:07:35.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.564+0000 7f765a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:35.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.564+0000 7f765a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:35.566 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:35.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 msgr2=0x7f764803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 0x7f764803ad40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7654003eb0 tx=0x7f7654009040 comp rx=0 tx=0).stop 2024-09-16T15:07:35.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 10 monclient: shutdown 2024-09-16T15:07:35.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 msgr2=0x7f76641343a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f766000d7d0 tx=0x7f766000dc90 comp rx=0 tx=0).stop 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 shutdown_connections 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f766413bf80 0x7f7664138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7664136030 0x7f76641343a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> v2:172.21.15.162:6800/980547263 conn(0x7f7648038880 0x7f764803ad40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 --2- 172.21.15.134:0/360288534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f766409bfe0 0x7f7664133e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4230985060}) 2024-09-16T15:07:35.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 >> 172.21.15.134:0/360288534 conn(0x7f76640a9ea0 msgr2=0x7f76640a7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 shutdown_connections 2024-09-16T15:07:35.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3049297249}) 2024-09-16T15:07:35.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.566+0000 7f766bf18640 1 -- 172.21.15.134:0/360288534 wait complete. 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3783582399}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3006337928}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2324539453}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3686167551}) 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3073447971}) 2024-09-16T15:07:35.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4181019938}) 2024-09-16T15:07:35.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.567+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1013530153}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3680765746}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2213062951}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=500443809}) 2024-09-16T15:07:35.581 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=316238422}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3082976960}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1312833522}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1200957298}) 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:35.582 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2223184145}) 2024-09-16T15:07:35.583 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:35.578+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1263042676}) 2024-09-16T15:07:35.585 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:07:35.585 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:07:35.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:07:35.585 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:07:35.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.680+0000 7f5d56853640 10 monclient: get_monmap_and_config 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.680+0000 7f5d56853640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.680+0000 7f5d56853640 10 monclient: monmap: 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:35.681509+0000 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:35.681509+0000 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.682 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:35.683 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.681+0000 7f5d56853640 1 Processor -- start 2024-09-16T15:07:35.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.681+0000 7f5d56853640 1 -- start start 2024-09-16T15:07:35.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.681+0000 7f5d56853640 10 monclient: init 2024-09-16T15:07:35.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.681+0000 7f5d56853640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient: _add_conns ranks=[2,1,6,3,7,0,9,5,8,10,4] 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-b con 0x7f5d50136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5d50137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 0x7f5d5013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-d con 0x7f5d5013c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d50073250 con 0x7f5d50137220 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500a56e0 con 0x7f5d50136e40 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5009abc0 con 0x7f5d5013c370 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d56853640 10 monclient(hunting): authenticate will time out at 1192.271110s 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 0x7f5d5013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40280/0 (socket says 172.21.15.134:40280) 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 -- 172.21.15.134:0/2170212813 learned_addr learned my addr 172.21.15.134:0/2170212813 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): get_auth_request con 0x7f5d5013c370 auth_method 0 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request con 0x7f5d50136e40 auth_method 0 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_done global_id 44444 payload 274 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 -- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 msgr2=0x7f5d5013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 0x7f5d5013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 -- 172.21.15.134:0/2170212813 >> v1:172.21.15.162:6789/0 conn(0x7f5d50137220 legacy=0x7f5d50137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 monclient: found mon.noname-b 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d4ffff640 1 -- 172.21.15.134:0/2170212813 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5009ba50 con 0x7f5d50136e40 2024-09-16T15:07:35.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.682+0000 7f5d54df0640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 0x7f5d5013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:35.696 INFO:tasks.rados.rados.0.smithi162.stdout:1607: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:07:35.696 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 0 (ObjNum 289 snap 0 seq_num 1840527439) dirty dne 2024-09-16T15:07:35.696 INFO:tasks.rados.rados.0.smithi162.stdout:1605: expect (ObjNum 568 snap 143 seq_num 568) 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4ffff640 1 -- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 msgr2=0x7f5d5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098a0 tx=0x7f5d4002a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754e1141db6783a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 1 -- 172.21.15.134:0/2170212813 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4002adf0 con 0x7f5d50136e40 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: dump: 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.697 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 1 -- 172.21.15.134:0/2170212813 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d400307b0 con 0x7f5d50136e40 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d56853640 5 monclient: authenticate success, global_id 44444 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d56853640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d56853640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 1 -- 172.21.15.134:0/2170212813 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d40030d90 con 0x7f5d50136e40 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d56853640 4 set_mon_vals no callback set 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.685+0000 7f5d4f7fe640 10 monclient: dump: 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.699 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.700 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 10 monclient: shutdown 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 -- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 msgr2=0x7f5d5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098a0 tx=0x7f5d4002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 -- 172.21.15.134:0/2170212813 shutdown_connections 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d5013c370 0x7f5d5013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 --2- 172.21.15.134:0/2170212813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50136e40 0x7f5d5009c150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 -- 172.21.15.134:0/2170212813 >> 172.21.15.134:0/2170212813 conn(0x7f5d500a6110 msgr2=0x7f5d500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 -- 172.21.15.134:0/2170212813 shutdown_connections 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 1 -- 172.21.15.134:0/2170212813 wait complete. 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.686+0000 7f5d56853640 10 monclient: monmap: 2024-09-16T15:07:35.701 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.702 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 Processor -- start 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 -- start start 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient: init 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient: _add_conns ranks=[5,2,1,7,8,4,6,0,3] 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5d50136e40 0x7f5d50135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-h con 0x7f5d50136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 0x7f5d501363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-b con 0x7f5d50137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5013c370 0x7f5d5011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): picked mon.noname-f con 0x7f5d5013c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d501228b0 con 0x7f5d5013c370 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500a7c40 con 0x7f5d50137220 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500a7710 con 0x7f5d50136e40 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d56853640 10 monclient(hunting): authenticate will time out at 1192.276572s 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 0x7f5d501363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 0x7f5d501363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40286/0 (socket says 172.21.15.134:40286) 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 -- 172.21.15.134:0/3545130935 learned_addr learned my addr 172.21.15.134:0/3545130935 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): get_auth_request con 0x7f5d50137220 auth_method 0 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5013c370 0x7f5d5011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5d50136e40 0x7f5d50135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): get_auth_request con 0x7f5d5013c370 auth_method 0 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request con 0x7f5d50136e40 auth_method 0 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d54df0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient(hunting): handle_auth_done global_id 44453 payload 824 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5d50136e40 msgr2=0x7f5d50135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5d50136e40 0x7f5d50135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5013c370 msgr2=0x7f5d5011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5013c370 0x7f5d5011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.687+0000 7f5d47fff640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48009a70 con 0x7f5d50137220 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d47fff640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 msgr2=0x7f5d501363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d47fff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 0x7f5d501363d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c460 tx=0x7f5d3800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d38011c10 con 0x7f5d50137220 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: dump: 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.707 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38011dd0 con 0x7f5d50137220 2024-09-16T15:07:35.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d38012890 con 0x7f5d50137220 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 5 monclient: authenticate success, global_id 44453 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4effd640 4 set_mon_vals no callback set 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d4d7fa640 10 monclient: dump: 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.709 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 10 monclient: _renew_subs 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400094d0 con 0x7f5d50137220 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 10 monclient: _renew_subs 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.690+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d50121fa0 con 0x7f5d50137220 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5d38012420 con 0x7f5d50137220 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d56853640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d56853640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 0x7f5d501201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50071b20 con 0x7f5d501221c0 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4d7fa640 1 --2- 172.21.15.134:0/3545130935 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d30038880 0x7f5d3003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(321..321 src has 1..321) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f5d38049ea0 con 0x7f5d50137220 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 0x7f5d501201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient: get_auth_request con 0x7f5d501221c0 auth_method 0 2024-09-16T15:07:35.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d54df0640 1 --2- 172.21.15.134:0/3545130935 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d30038880 0x7f5d3003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d54df0640 10 monclient: get_auth_request con 0x7f5d30038880 auth_method 0 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_done global_id 5103 payload 824 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.691+0000 7f5d4ffff640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d50061ae0 con 0x7f5d501221c0 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.693+0000 7f5d4ffff640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 msgr2=0x7f5d501201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.693+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 0x7f5d501201a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5d40034000 tx=0x7f5d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d40003610 con 0x7f5d501221c0 2024-09-16T15:07:35.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d54df0640 1 --2- 172.21.15.134:0/3545130935 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d30038880 0x7f5d3003ad40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d5006b7d0 tx=0x7f5d4800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 10 monclient: dump: 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.713 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5d40030b30 con 0x7f5d501221c0 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 msgr2=0x7f5d501201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 0x7f5d501201a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5d40034000 tx=0x7f5d400046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 msgr2=0x7f5d501201a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:35.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.694+0000 7f5d4d7fa640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501221c0 0x7f5d501201a0 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5d40034000 tx=0x7f5d400046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d56853640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:35.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d56853640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 0x7f5d5009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d56853640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5007a680 con 0x7f5d501206e0 2024-09-16T15:07:35.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 0x7f5d5009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient: get_auth_request con 0x7f5d501206e0 auth_method 0 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 10 monclient(hunting): handle_auth_done global_id 5112 payload 824 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.700+0000 7f5d4ffff640 1 -- 172.21.15.134:0/3545130935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5d500a53c0 con 0x7f5d501206e0 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4ffff640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 msgr2=0x7f5d5009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4ffff640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 0x7f5d5009d4f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002a110 tx=0x7f5d40009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d4003cd00 con 0x7f5d501206e0 2024-09-16T15:07:35.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 10 monclient: dump: 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.718 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f5d4003ce60 con 0x7f5d501206e0 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 msgr2=0x7f5d5009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 0x7f5d5009d4f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002a110 tx=0x7f5d40009240 comp rx=0 tx=0).stop 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 msgr2=0x7f5d5009d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d4d7fa640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d501206e0 0x7f5d5009d4f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.719 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:35.720 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.721 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.722 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:35.723 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.724 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:35.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:35.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:35.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:35.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.734 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:35.736 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.738 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.739 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:35.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:35.742 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:35.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d30038880 msgr2=0x7f5d3003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 --2- 172.21.15.134:0/3545130935 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d30038880 0x7f5d3003ad40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5d5006b7d0 tx=0x7f5d4800b040 comp rx=0 tx=0).stop 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 10 monclient: shutdown 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 msgr2=0x7f5d501363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 --2- 172.21.15.134:0/3545130935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d50137220 0x7f5d501363d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c460 tx=0x7f5d3800c860 comp rx=0 tx=0).stop 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d54df0640 1 -- 172.21.15.134:0/3545130935 reap_dead start 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d54df0640 1 -- 172.21.15.134:0/3545130935 reap_dead start 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 shutdown_connections 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 >> 172.21.15.134:0/3545130935 conn(0x7f5d500a6110 msgr2=0x7f5d5013b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 shutdown_connections 2024-09-16T15:07:35.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.703+0000 7f5d56853640 1 -- 172.21.15.134:0/3545130935 wait complete. 2024-09-16T15:07:35.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:35.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:07:35.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:07:35.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 10 monclient: get_monmap_and_config 2024-09-16T15:07:35.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 10 monclient: monmap: 2024-09-16T15:07:35.842 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.842 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:35.842089+0000 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:35.842089+0000 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.843 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.844 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:35.844 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:35.844 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 1 Processor -- start 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 1 -- start start 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.841+0000 7f95d6e9c640 10 monclient: init 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient: _add_conns ranks=[7,1,6,4,9,2,8,3,0,5,10] 2024-09-16T15:07:35.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d0136e40 0x7f95d009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-i con 0x7f95d0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f95d0137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-d con 0x7f95d013c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d0073250 con 0x7f95d0137220 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00a56e0 con 0x7f95d013c370 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95d009abc0 con 0x7f95d0136e40 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d6e9c640 10 monclient(hunting): authenticate will time out at 1192.431482s 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58110/0 (socket says 172.21.15.134:58110) 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 -- 172.21.15.134:0/1568943711 learned_addr learned my addr 172.21.15.134:0/1568943711 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): get_auth_request con 0x7f95d013c370 auth_method 0 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d0136e40 0x7f95d009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): get_auth_request con 0x7f95d0136e40 auth_method 0 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient(hunting): handle_auth_done global_id 34611 payload 274 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 -- 172.21.15.134:0/1568943711 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d0136e40 msgr2=0x7f95d009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d0136e40 0x7f95d009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 -- 172.21.15.134:0/1568943711 >> v1:172.21.15.162:6789/0 conn(0x7f95d0137220 legacy=0x7f95d0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 monclient: found mon.noname-d 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:35.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.842+0000 7f95d5439640 1 -- 172.21.15.134:0/1568943711 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d009ba50 con 0x7f95d013c370 2024-09-16T15:07:35.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.845+0000 7f95d5439640 1 -- 172.21.15.134:0/1568943711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 msgr2=0x7f95d013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.845+0000 7f95d5439640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009e80 tx=0x7f95c802a580 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3c1486eedefc0f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 1 -- 172.21.15.134:0/1568943711 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c8007a70 con 0x7f95d013c370 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: dump: 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.857 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 1 -- 172.21.15.134:0/1568943711 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c802d050 con 0x7f95d013c370 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 5 monclient: authenticate success, global_id 34611 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:35.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 1 -- 172.21.15.134:0/1568943711 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c8007bf0 con 0x7f95d013c370 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95cf7fe640 10 monclient: dump: 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.859 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 4 set_mon_vals no callback set 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 10 monclient: shutdown 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 -- 172.21.15.134:0/1568943711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 msgr2=0x7f95d013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009e80 tx=0x7f95c802a580 comp rx=0 tx=0).stop 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 -- 172.21.15.134:0/1568943711 shutdown_connections 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f95d013c370 0x7f95d013e760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/1568943711 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d0136e40 0x7f95d009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 -- 172.21.15.134:0/1568943711 >> 172.21.15.134:0/1568943711 conn(0x7f95d00a6110 msgr2=0x7f95d00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 -- 172.21.15.134:0/1568943711 shutdown_connections 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.846+0000 7f95d6e9c640 1 -- 172.21.15.134:0/1568943711 wait complete. 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.847+0000 7f95d6e9c640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.847+0000 7f95d6e9c640 10 monclient: monmap: 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.861 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.847+0000 7f95d6e9c640 1 Processor -- start 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.847+0000 7f95d6e9c640 1 -- start start 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.847+0000 7f95d6e9c640 10 monclient: init 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient: _add_conns ranks=[3,0,7,4,1,5,6,2,8] 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0136e40 0x7f95d0135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-g con 0x7f95d0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-a con 0x7f95d0137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:35.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d013c370 0x7f95d01362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): picked mon.noname-i con 0x7f95d013c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00a7610 con 0x7f95d0137220 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d01221d0 con 0x7f95d0136e40 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00a3400 con 0x7f95d013c370 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d6e9c640 10 monclient(hunting): authenticate will time out at 1192.437304s 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42872/0 (socket says 172.21.15.134:42872) 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 -- 172.21.15.134:0/2028013879 learned_addr learned my addr 172.21.15.134:0/2028013879 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): get_auth_request con 0x7f95d0137220 auth_method 0 2024-09-16T15:07:35.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d013c370 0x7f95d01362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0136e40 0x7f95d0135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): get_auth_request con 0x7f95d013c370 auth_method 0 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): get_auth_request con 0x7f95d0136e40 auth_method 0 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d5439640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient(hunting): handle_auth_done global_id 5121 payload 824 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d013c370 msgr2=0x7f95d01362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f95d013c370 0x7f95d01362f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0136e40 msgr2=0x7f95d0135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95d0136e40 0x7f95d0135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 monclient: found mon.noname-a 2024-09-16T15:07:35.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.848+0000 7f95cffff640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009b30 con 0x7f95d0137220 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.850+0000 7f95cffff640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 msgr2=0x7f95d0135db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.850+0000 7f95cffff640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c430 tx=0x7f95c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c4003860 con 0x7f95d0137220 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: dump: 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.866 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4011c10 con 0x7f95d0137220 2024-09-16T15:07:35.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 5 monclient: authenticate success, global_id 5121 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 monclient: _renew_subs 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d0121ef0 con 0x7f95d0137220 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c4011e00 con 0x7f95d0137220 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 10 monclient: dump: 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.868 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95ceffd640 4 set_mon_vals no callback set 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 monclient: _renew_subs 2024-09-16T15:07:35.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00a2aa0 con 0x7f95d0137220 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 0x7f95d009e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00a2c30 con 0x7f95d009e520 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.851+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f95c40117a0 con 0x7f95d0137220 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 0x7f95d009e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient: get_auth_request con 0x7f95d009e520 auth_method 0 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95cd7fa640 1 --2- 172.21.15.134:0/2028013879 >> v2:172.21.15.162:6800/980547263 conn(0x7f95bc0388b0 0x7f95bc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(321..321 src has 1..321) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f95c4049f30 con 0x7f95d0137220 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 10 monclient(hunting): handle_auth_done global_id 44462 payload 824 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d4c38640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95d0122820 con 0x7f95d009e520 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d5439640 1 --2- 172.21.15.134:0/2028013879 >> v2:172.21.15.162:6800/980547263 conn(0x7f95bc0388b0 0x7f95bc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.852+0000 7f95d5439640 10 monclient: get_auth_request con 0x7f95bc0388b0 auth_method 0 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95d4c38640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 msgr2=0x7f95d009e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95d4c38640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 0x7f95d009e900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c0011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c000fdc0 con 0x7f95d009e520 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 10 monclient: dump: 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.871 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95d5439640 1 --2- 172.21.15.134:0/2028013879 >> v2:172.21.15.162:6800/980547263 conn(0x7f95bc0388b0 0x7f95bc03ad70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009e80 tx=0x7f95c8009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f95c000f950 con 0x7f95d009e520 2024-09-16T15:07:35.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:35.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:35.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 msgr2=0x7f95d009e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 0x7f95d009e900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c0011da0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 msgr2=0x7f95d009e900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:35.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.854+0000 7f95cd7fa640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009e520 0x7f95d009e900 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c0011da0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.873 INFO:tasks.rados.rados.0.smithi162.stdout:1606: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:07:35.873 INFO:tasks.rados.rados.0.smithi162.stdout:1606: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:07:35.873 INFO:tasks.rados.rados.0.smithi162.stdout:1606: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:07:35.873 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 695 (ObjNum 490 snap 121 seq_num 490) dirty exists 2024-09-16T15:07:35.873 INFO:tasks.rados.rados.0.smithi162.stdout:1608: expect (ObjNum 538 snap 139 seq_num 538) 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.860+0000 7f95d6e9c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.860+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 0x7f95d011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d6e9c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0071b20 con 0x7f95d009ee40 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 0x7f95d011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient: get_auth_request con 0x7f95d009ee40 auth_method 0 2024-09-16T15:07:35.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 10 monclient(hunting): handle_auth_done global_id 44471 payload 824 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.861+0000 7f95d4c38640 1 -- 172.21.15.134:0/2028013879 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95d0061ae0 con 0x7f95d009ee40 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95d4c38640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 msgr2=0x7f95d011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95d4c38640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 0x7f95d011fc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c002c070 con 0x7f95d009ee40 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 10 monclient: dump: 2024-09-16T15:07:35.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:35.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f95c0024040 con 0x7f95d009ee40 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 msgr2=0x7f95d011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 0x7f95d011fc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 msgr2=0x7f95d011fc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.863+0000 7f95cd7fa640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95d009ee40 0x7f95d011fc60 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95c00118e0 tx=0x7f95c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:35.878 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:35.879 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.880 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:35.881 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:35.882 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.883 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.884 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.886 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:35.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.891 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.892 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.895 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:35.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:35.897 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:35.898 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:35.899 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:35.899 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:35.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:35.899 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:35.899 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 >> v2:172.21.15.162:6800/980547263 conn(0x7f95bc0388b0 msgr2=0x7f95bc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/2028013879 >> v2:172.21.15.162:6800/980547263 conn(0x7f95bc0388b0 0x7f95bc03ad70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009e80 tx=0x7f95c8009000 comp rx=0 tx=0).stop 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 10 monclient: shutdown 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 msgr2=0x7f95d0135db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c430 tx=0x7f95c400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d5439640 1 -- 172.21.15.134:0/2028013879 reap_dead start 2024-09-16T15:07:35.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 shutdown_connections 2024-09-16T15:07:35.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 --2- 172.21.15.134:0/2028013879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95d0137220 0x7f95d0135db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:35.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 >> 172.21.15.134:0/2028013879 conn(0x7f95d00a6110 msgr2=0x7f95d006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:35.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 shutdown_connections 2024-09-16T15:07:35.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.864+0000 7f95d6e9c640 1 -- 172.21.15.134:0/2028013879 wait complete. 2024-09-16T15:07:35.901 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:35.901 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:07:35.901 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.997+0000 7f28db176640 10 monclient: get_monmap_and_config 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.997+0000 7f28db176640 10 monclient: build_initial_monmap 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.997+0000 7f28db176640 10 monclient: monmap: 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:35.998153+0000 2024-09-16T15:07:35.998 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:35.998153+0000 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:35.999 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.997+0000 7f28db176640 1 Processor -- start 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.997+0000 7f28db176640 1 -- start start 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient: init 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient: _add_conns ranks=[5,9,4,10,6,7,3,1,0,8,2] 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4136e40 0x7f28d409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): picked mon.noname-h con 0x7f28d4136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f28d4137220 0x7f28d4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): picked mon.noname-a con 0x7f28d4137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:36.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): picked mon.noname-c con 0x7f28d4135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d409abc0 con 0x7f28d4137220 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d409ba50 con 0x7f28d4135dc0 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40a53c0 con 0x7f28d4136e40 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28db176640 10 monclient(hunting): authenticate will time out at 1192.587483s 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f28d4137220 0x7f28d4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f28d4137220 0x7f28d4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:42878/0 (socket says 172.21.15.134:42878) 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:35208/0 (socket says 172.21.15.134:35208) 2024-09-16T15:07:36.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 -- 172.21.15.134:0/1405228452 learned_addr learned my addr 172.21.15.134:0/1405228452 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): get_auth_request con 0x7f28d4137220 auth_method 0 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): get_auth_request con 0x7f28d4135dc0 auth_method 0 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4136e40 0x7f28d409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): get_auth_request con 0x7f28d4136e40 auth_method 0 2024-09-16T15:07:36.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient(hunting): handle_auth_done global_id 5130 payload 274 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 -- 172.21.15.134:0/1405228452 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4136e40 msgr2=0x7f28d409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4136e40 0x7f28d409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 -- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 msgr2=0x7f28d4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 monclient: found mon.noname-a 2024-09-16T15:07:36.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:36.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d3fff640 1 -- 172.21.15.134:0/1405228452 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d4073250 con 0x7f28d4137220 2024-09-16T15:07:36.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:35.998+0000 7f28d9713640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:36.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d3fff640 1 -- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d4137220 msgr2=0x7f28d4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d3fff640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d4137220 0x7f28d4137600 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44be4ecb4155f4ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 1 -- 172.21.15.134:0/1405228452 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c002ae00 con 0x7f28d4137220 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: dump: 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.013 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 1 -- 172.21.15.134:0/1405228452 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00307b0 con 0x7f28d4137220 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28db176640 5 monclient: authenticate success, global_id 5130 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28db176640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28db176640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 1 -- 172.21.15.134:0/1405228452 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c0030d90 con 0x7f28d4137220 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28db176640 4 set_mon_vals no callback set 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.001+0000 7f28d37fe640 10 monclient: dump: 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.015 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 10 monclient: shutdown 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 -- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d4137220 msgr2=0x7f28d4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d4137220 0x7f28d4137600 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 -- 172.21.15.134:0/1405228452 shutdown_connections 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4135dc0 0x7f28d4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d4137220 0x7f28d4137600 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 --2- 172.21.15.134:0/1405228452 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4136e40 0x7f28d409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 -- 172.21.15.134:0/1405228452 >> 172.21.15.134:0/1405228452 conn(0x7f28d40a6110 msgr2=0x7f28d40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 -- 172.21.15.134:0/1405228452 shutdown_connections 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 1 -- 172.21.15.134:0/1405228452 wait complete. 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.002+0000 7f28db176640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient: monmap: 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.017 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 Processor -- start 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 -- start start 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient: init 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient: _add_conns ranks=[1,4,5,2,7,8,0,3,6] 2024-09-16T15:07:36.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d4135dc0 0x7f28d4134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): picked mon.noname-f con 0x7f28d4135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 0x7f28d41354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): picked mon.noname-c con 0x7f28d4136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4137220 0x7f28d411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): picked mon.noname-h con 0x7f28d4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4120db0 con 0x7f28d4135dc0 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41cc470 con 0x7f28d4136e40 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.003+0000 7f28db176640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41cb9f0 con 0x7f28d4137220 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28db176640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28db176640 10 monclient(hunting): authenticate will time out at 1192.593008s 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 0x7f28d41354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 0x7f28d41354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:35222/0 (socket says 172.21.15.134:35222) 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 -- 172.21.15.134:0/3693408040 learned_addr learned my addr 172.21.15.134:0/3693408040 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): get_auth_request con 0x7f28d4136e40 auth_method 0 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d4135dc0 0x7f28d4134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): get_auth_request con 0x7f28d4135dc0 auth_method 0 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4137220 0x7f28d411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): get_auth_request con 0x7f28d4137220 auth_method 0 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d9713640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient(hunting): handle_auth_done global_id 4999 payload 824 2024-09-16T15:07:36.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4137220 msgr2=0x7f28d411fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f28d4137220 0x7f28d411fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d4135dc0 msgr2=0x7f28d4134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28d4135dc0 0x7f28d4134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 monclient: found mon.noname-c 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.004+0000 7f28d3fff640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009a70 con 0x7f28d4136e40 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d3fff640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 msgr2=0x7f28d41354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d3fff640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 0x7f28d41354b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28c00097c0 tx=0x7f28c00046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c0003610 con 0x7f28d4136e40 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: dump: 2024-09-16T15:07:36.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00037b0 con 0x7f28d4136e40 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 5 monclient: authenticate success, global_id 4999 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c0003d90 con 0x7f28d4136e40 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 10 monclient: _renew_subs 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c00094d0 con 0x7f28d4136e40 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d17fa640 10 monclient: dump: 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.024 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28d2ffd640 4 set_mon_vals no callback set 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 10 monclient: _renew_subs 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.006+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d41368e0 con 0x7f28d4136e40 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f28c0039070 con 0x7f28d4136e40 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28db176640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28db176640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 0x7f28d402b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4071b20 con 0x7f28d402b260 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 0x7f28d402b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient: get_auth_request con 0x7f28d402b260 auth_method 0 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d17fa640 1 --2- 172.21.15.134:0/3693408040 >> v2:172.21.15.162:6800/980547263 conn(0x7f28b8038880 0x7f28b803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.007+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(321..321 src has 1..321) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f28c006a7f0 con 0x7f28d4136e40 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.008+0000 7f28d8f12640 10 monclient(hunting): handle_auth_done global_id 5008 payload 824 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.008+0000 7f28d8f12640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28d4061ae0 con 0x7f28d402b260 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.008+0000 7f28d9713640 1 --2- 172.21.15.134:0/3693408040 >> v2:172.21.15.162:6800/980547263 conn(0x7f28b8038880 0x7f28b803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.008+0000 7f28d9713640 10 monclient: get_auth_request con 0x7f28b8038880 auth_method 0 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d8f12640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 msgr2=0x7f28d402b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d8f12640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 0x7f28d402b640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f28c8011860 tx=0x7f28c8011d20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c800fd00 con 0x7f28d402b260 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 10 monclient: dump: 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.027 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d9713640 1 --2- 172.21.15.134:0/3693408040 >> v2:172.21.15.162:6800/980547263 conn(0x7f28b8038880 0x7f28b803ad40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f28d4121f80 tx=0x7f28cc00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f28c800fe60 con 0x7f28d402b260 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 msgr2=0x7f28d402b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 0x7f28d402b640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f28c8011860 tx=0x7f28c8011d20 comp rx=0 tx=0).stop 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 msgr2=0x7f28d402b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.010+0000 7f28d17fa640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402b260 0x7f28d402b640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28db176640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28db176640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 0x7f28d402bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28db176640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d407a680 con 0x7f28d402bb80 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 0x7f28d402bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient: get_auth_request con 0x7f28d402bb80 auth_method 0 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.016+0000 7f28d8f12640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.017+0000 7f28d8f12640 10 monclient(hunting): handle_auth_done global_id 5017 payload 824 2024-09-16T15:07:36.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.017+0000 7f28d8f12640 1 -- 172.21.15.134:0/3693408040 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f28d409b490 con 0x7f28d402bb80 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d8f12640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 msgr2=0x7f28d402bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d8f12640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 0x7f28d402bf90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28c80096a0 tx=0x7f28c80046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28c802c070 con 0x7f28d402bb80 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 10 monclient: dump: 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.032 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9527 (secure 0 0 0) 0x7f28c8024040 con 0x7f28d402bb80 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 msgr2=0x7f28d402bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 0x7f28d402bf90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28c80096a0 tx=0x7f28c80046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 msgr2=0x7f28d402bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d17fa640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d402bb80 0x7f28d402bf90 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28c80096a0 tx=0x7f28c80046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.034 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.036 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.037 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.041 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.043 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.045 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.046 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.049 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.053 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.054 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:07:36.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.056 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 >> v2:172.21.15.162:6800/980547263 conn(0x7f28b8038880 msgr2=0x7f28b803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 1 --2- 172.21.15.134:0/3693408040 >> v2:172.21.15.162:6800/980547263 conn(0x7f28b8038880 0x7f28b803ad40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f28d4121f80 tx=0x7f28cc00b040 comp rx=0 tx=0).stop 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 10 monclient: shutdown 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 msgr2=0x7f28d41354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28db176640 1 --2- 172.21.15.134:0/3693408040 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f28d4136e40 0x7f28d41354b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28c00097c0 tx=0x7f28c00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.019+0000 7f28d9713640 1 -- 172.21.15.134:0/3693408040 reap_dead start 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.020+0000 7f28d9713640 1 -- 172.21.15.134:0/3693408040 reap_dead start 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.020+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 shutdown_connections 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.020+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 >> 172.21.15.134:0/3693408040 conn(0x7f28d40a6110 msgr2=0x7f28d409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.020+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 shutdown_connections 2024-09-16T15:07:36.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.020+0000 7f28db176640 1 -- 172.21.15.134:0/3693408040 wait complete. 2024-09-16T15:07:36.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:07:36.059 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4110467410}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3428942210}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=469019725}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1480787837}) 2024-09-16T15:07:36.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.048+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1656354946}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1276468096}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=913547425}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=991906419}) 2024-09-16T15:07:36.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.049+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2108671575}) 2024-09-16T15:07:36.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.157+0000 7fd8ea183640 10 monclient: get_monmap_and_config 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.157+0000 7fd8ea183640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.157+0000 7fd8ea183640 10 monclient: monmap: 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:36.158460+0000 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:36.158460+0000 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:36.160 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 Processor -- start 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 -- start start 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient: init 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient: _add_conns ranks=[2,9,6,3,8,0,1,5,10,4,7] 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-b con 0x7fd8e4136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8e4137220 0x7fd8e4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-a con 0x7fd8e4137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 0x7fd8e4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-d con 0x7fd8e4135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e409abc0 con 0x7fd8e4137220 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e409ba50 con 0x7fd8e4136e40 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40a53c0 con 0x7fd8e4135dc0 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8ea183640 10 monclient(hunting): authenticate will time out at 1192.747776s 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e2ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd8e4137220 0x7fd8e4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40322/0 (socket says 172.21.15.134:40322) 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 1 -- 172.21.15.134:0/1577505676 learned_addr learned my addr 172.21.15.134:0/1577505676 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request con 0x7fd8e4136e40 auth_method 0 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 0x7fd8e4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e3fff640 10 monclient(hunting): get_auth_request con 0x7fd8e4135dc0 auth_method 0 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.158+0000 7fd8e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd8e4137220 auth_method 0 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_done global_id 34620 payload 274 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 -- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 msgr2=0x7fd8e409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 -- 172.21.15.134:0/1577505676 >> v2:172.21.15.134:3300/0 conn(0x7fd8e4137220 msgr2=0x7fd8e4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/1577505676 >> v2:172.21.15.134:3300/0 conn(0x7fd8e4137220 0x7fd8e4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 monclient: found mon.noname-d 2024-09-16T15:07:36.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:36.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e3fff640 1 -- 172.21.15.134:0/1577505676 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4073250 con 0x7fd8e4135dc0 2024-09-16T15:07:36.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.159+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.161+0000 7fd8e3fff640 1 -- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 msgr2=0x7fd8e4136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 0x7fd8e4136220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009dc0 tx=0x7fd8dc02a4d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=63dee491ed66353e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 1 -- 172.21.15.134:0/1577505676 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8dc007bc0 con 0x7fd8e4135dc0 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: dump: 2024-09-16T15:07:36.172 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.173 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 5 monclient: authenticate success, global_id 34620 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 1 -- 172.21.15.134:0/1577505676 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc02d050 con 0x7fd8e4135dc0 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 4 set_mon_vals no callback set 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 1 -- 172.21.15.134:0/1577505676 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8dc007d40 con 0x7fd8e4135dc0 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8e27fc640 10 monclient: dump: 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.174 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 10 monclient: shutdown 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 -- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 msgr2=0x7fd8e4136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 0x7fd8e4136220 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009dc0 tx=0x7fd8dc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 -- 172.21.15.134:0/1577505676 shutdown_connections 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4135dc0 0x7fd8e4136220 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 --2- 172.21.15.134:0/1577505676 >> v2:172.21.15.134:3300/0 conn(0x7fd8e4137220 0x7fd8e4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 --2- 172.21.15.134:0/1577505676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4136e40 0x7fd8e409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 -- 172.21.15.134:0/1577505676 >> 172.21.15.134:0/1577505676 conn(0x7fd8e40a6110 msgr2=0x7fd8e40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 -- 172.21.15.134:0/1577505676 shutdown_connections 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.162+0000 7fd8ea183640 1 -- 172.21.15.134:0/1577505676 wait complete. 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.163+0000 7fd8ea183640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.163+0000 7fd8ea183640 10 monclient: monmap: 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.176 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.163+0000 7fd8ea183640 1 Processor -- start 2024-09-16T15:07:36.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.163+0000 7fd8ea183640 1 -- start start 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.163+0000 7fd8ea183640 10 monclient: init 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient: _add_conns ranks=[1,7,2,5,4,3,8,6,0] 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e4135dc0 0x7fd8e4134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-f con 0x7fd8e4135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd8e4136e40 0x7fd8e4135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-i con 0x7fd8e4136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): picked mon.noname-b con 0x7fd8e4137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e41cc090 con 0x7fd8e4135dc0 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4125350 con 0x7fd8e4137220 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e41362a0 con 0x7fd8e4136e40 2024-09-16T15:07:36.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8ea183640 10 monclient(hunting): authenticate will time out at 1192.753310s 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:40330/0 (socket says 172.21.15.134:40330) 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 learned_addr learned my addr 172.21.15.134:0/2720464830 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): get_auth_request con 0x7fd8e4137220 auth_method 0 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e2ffd640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd8e4136e40 0x7fd8e4135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e4135dc0 0x7fd8e4134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e2ffd640 10 monclient(hunting): get_auth_request con 0x7fd8e4136e40 auth_method 0 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request con 0x7fd8e4135dc0 auth_method 0 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient(hunting): handle_auth_done global_id 44489 payload 824 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd8e4136e40 msgr2=0x7fd8e4135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd8e4136e40 0x7fd8e4135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e4135dc0 msgr2=0x7fd8e4134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e4135dc0 0x7fd8e4134e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:36.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.164+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40094d0 con 0x7fd8e4137220 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.166+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 msgr2=0x7fd8e4121990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.166+0000 7fd8e3fff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc02a9f0 tx=0x7fd8dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8dc007ea0 con 0x7fd8e4137220 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: dump: 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.181 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0044b0 con 0x7fd8e4137220 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 5 monclient: authenticate success, global_id 44489 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 10 monclient: _renew_subs 2024-09-16T15:07:36.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc009a70 con 0x7fd8e4137220 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8dc004620 con 0x7fd8e4137220 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 10 monclient: dump: 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.183 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 10 monclient: _renew_subs 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8e1ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e41367f0 con 0x7fd8e4137220 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.167+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd8dc004040 con 0x7fd8e4137220 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8ea183640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8ea183640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 0x7fd8e41222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4071b20 con 0x7fd8e4121ed0 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8dbfff640 1 --2- 172.21.15.134:0/2720464830 >> v2:172.21.15.162:6800/980547263 conn(0x7fd8c40388b0 0x7fd8c403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(321..321 src has 1..321) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fd8dc06b8c0 con 0x7fd8e4137220 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 0x7fd8e41222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient: get_auth_request con 0x7fd8e4121ed0 auth_method 0 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e2ffd640 1 --2- 172.21.15.134:0/2720464830 >> v2:172.21.15.162:6800/980547263 conn(0x7fd8c40388b0 0x7fd8c403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e2ffd640 10 monclient: get_auth_request con 0x7fd8c40388b0 auth_method 0 2024-09-16T15:07:36.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_done global_id 34629 payload 824 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.168+0000 7fd8e37fe640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8e4061ae0 con 0x7fd8e4121ed0 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8e37fe640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 msgr2=0x7fd8e41222b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 0x7fd8e41222b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d4011cf0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8e2ffd640 1 --2- 172.21.15.134:0/2720464830 >> v2:172.21.15.162:6800/980547263 conn(0x7fd8c40388b0 0x7fd8c403ad70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc0099b0 tx=0x7fd8cc008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8d4011ee0 con 0x7fd8e4121ed0 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8dbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.170+0000 7fd8dbfff640 10 monclient: dump: 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd8d400fc50 con 0x7fd8e4121ed0 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 msgr2=0x7fd8e41222b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 0x7fd8e41222b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d4011cf0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 msgr2=0x7fd8e41222b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.171+0000 7fd8dbfff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e4121ed0 0x7fd8e41222b0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d4011cf0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8ea183640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8ea183640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 0x7fd8e40a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8ea183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e407a680 con 0x7fd8e41227f0 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 0x7fd8e40a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient: get_auth_request con 0x7fd8e41227f0 auth_method 0 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 10 monclient(hunting): handle_auth_done global_id 34638 payload 824 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.177+0000 7fd8e37fe640 1 -- 172.21.15.134:0/2720464830 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd8e409b490 con 0x7fd8e41227f0 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8e37fe640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 msgr2=0x7fd8e40a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8e37fe640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 0x7fd8e40a1a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d40046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd8d4022580 con 0x7fd8e41227f0 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8dbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8dbfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.179+0000 7fd8dbfff640 10 monclient: dump: 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fd8d4006460 con 0x7fd8e41227f0 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 msgr2=0x7fd8e40a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 0x7fd8e40a1a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 msgr2=0x7fd8e40a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8dbfff640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd8e41227f0 0x7fd8e40a1a20 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011830 tx=0x7fd8d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.192 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.193 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.194 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.195 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.196 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.197 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.198 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.200 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.202 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.204 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.208 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.210 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.212 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.213 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.214 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 >> v2:172.21.15.162:6800/980547263 conn(0x7fd8c40388b0 msgr2=0x7fd8c403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 --2- 172.21.15.134:0/2720464830 >> v2:172.21.15.162:6800/980547263 conn(0x7fd8c40388b0 0x7fd8c403ad70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc0099b0 tx=0x7fd8cc008040 comp rx=0 tx=0).stop 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 10 monclient: shutdown 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 msgr2=0x7fd8e4121990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc02a9f0 tx=0x7fd8dc004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8e3fff640 1 -- 172.21.15.134:0/2720464830 reap_dead start 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 shutdown_connections 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 --2- 172.21.15.134:0/2720464830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8e4137220 0x7fd8e4121990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 >> 172.21.15.134:0/2720464830 conn(0x7fd8e40a6110 msgr2=0x7fd8e406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 shutdown_connections 2024-09-16T15:07:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.180+0000 7fd8ea183640 1 -- 172.21.15.134:0/2720464830 wait complete. 2024-09-16T15:07:36.217 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.217 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:07:36.217 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3834312969}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2901560674}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3712820801}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=788827251}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1135057679}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=912241255}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.191+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.192+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2576033769}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.192+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.192+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335558906}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.192+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2456262668}) 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3620873685}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=440237454}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2562520256}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1242706728}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1438255673}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1621890233}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1164022742}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2636228049}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.198+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=624977891}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3419784311}) 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3589480377}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2476713068}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=865986421}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2872972955}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1288328254}) 2024-09-16T15:07:36.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963346476}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=160983293}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.199+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3472854945}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=987346909}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1705271884}) 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3531100632}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2233684438}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3050259475}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3394751308}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3513731955}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2965505945}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.200+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1873224607}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4120194812}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=131497525}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2544565380}) 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2100448990}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3554979766}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1872543305}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1436928662}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1556552886}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.201+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1904119843}) 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1254972798}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1137221210}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.202+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=420246023}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1841099302}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3880022186}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2746012573}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4146450727}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=574768613}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.203+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=471417054}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2243619548}) 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=802654775}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=50727391}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1256507171}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3325059846}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2308081219}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=711243964}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2414643035}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.204+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3828928817}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3330312745}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=749846466}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1212179851}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3869640654}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=156397114}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2072649899}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=52450077}) 2024-09-16T15:07:36.252 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761237840}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.205+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2274779764}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=8} crc {logm=2515787297,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=351612246}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=73,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7} crc {logm=2462915722,mdsmap=3140514009,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=2164635949}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=70,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10} crc {logm=914879444,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=595042576}) 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.253 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=10} crc {logm=253999351,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=2117612215}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=72,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=6,mon_config_key=10} crc {logm=752002648,mdsmap=1531151339,mgr=1580275584,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=783959332,mon_config_key=2918577248}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=10} crc {logm=346897994,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=505205405,mon_config_key=2194010160}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=10} crc {logm=4292739199,mdsmap=1531151339,mgr=2345524120,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1515201435,mon_config_key=2117612215}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=69,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=11} crc {logm=1422877425,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3567873725,mon_config_key=3367576244}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.206+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=10} crc {logm=2960211739,mdsmap=1531151339,mgr=2345524120,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1692702949,mon_config_key=2117612215}) 2024-09-16T15:07:36.254 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=64,osdmap=20} crc {mon_config_key=1587122494,monmap=2503974663,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=2975519481,osdmap=14934073}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=22} crc {mon_config_key=975869477,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=3630869179,osdmap=2853084613}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=62,osdmap=22} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=2036797853,osdmap=2596373360}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=66,osdmap=18} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=1186519058,osdmap=2584159431}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.255 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=181165562,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=3451878375,osdmap=3521360118}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.207+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=65,osdmap=19} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=689204729,osdmap=2100868920}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=6,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=63,osdmap=23} crc {mon_config_key=1587122494,monmap=2200900527,osd_metadata=1958636322,osd_pg_creating=3812033758,osd_snap=2356776721,osdmap=3691631287}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=63,osdmap=21} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=107200971,osdmap=2214826950}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.208+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=64,osdmap=20} crc {mon_config_key=1587122494,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3812033758,osd_snap=2935269125,osdmap=2883032771}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=75956793}) 2024-09-16T15:07:36.256 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2312053647}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2186231898}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=547745998}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=599047319}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1392050103}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=965804789}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1999019075}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.209+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3291912406}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2837154632}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=467804364}) 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.258 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2155259814}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=348914}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1214306098}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2653719071}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=466800685}) 2024-09-16T15:07:36.259 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1108833720}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.210+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3009911500}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2890478827}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4105323927}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2280858653}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.260 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4129958388}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2638002052}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3457654274}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3394690748}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=687686677}) 2024-09-16T15:07:36.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.212+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2783145763}) 2024-09-16T15:07:36.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.357+0000 7f32e45b5640 10 monclient: get_monmap_and_config 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.357+0000 7f32e45b5640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.357+0000 7f32e45b5640 10 monclient: monmap: 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:36.358586+0000 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:36.358586+0000 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.359 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:36.360 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 Processor -- start 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 -- start start 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient: init 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient: _add_conns ranks=[7,5,10,2,8,9,1,0,6,3,4] 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-i con 0x7f32dc136eb0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-h con 0x7f32dc137290 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32dc133d10 0x7f32dc1341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-f con 0x7f32dc133d10 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc0a5560 con 0x7f32dc133d10 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc09b240 con 0x7f32dc137290 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc09b520 con 0x7f32dc136eb0 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.358+0000 7f32e45b5640 10 monclient(hunting): authenticate will time out at 1192.947919s 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32dc133d10 0x7f32dc1341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:49282/0 (socket says 172.21.15.134:49282) 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32dc133d10 0x7f32dc1341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:41392/0 (socket says 172.21.15.134:41392) 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:43866/0 (socket says 172.21.15.134:43866) 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 1 -- 172.21.15.134:0/3348746371 learned_addr learned my addr 172.21.15.134:0/3348746371 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): get_auth_request con 0x7f32dc137290 auth_method 0 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): get_auth_request con 0x7f32dc133d10 auth_method 0 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): get_auth_request con 0x7f32dc136eb0 auth_method 0 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient(hunting): handle_auth_done global_id 44398 payload 274 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 -- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 msgr2=0x7f32dc09bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 -- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 msgr2=0x7f32dc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 monclient: found mon.noname-f 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2b52640 1 -- 172.21.15.134:0/3348746371 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc073250 con 0x7f32dc133d10 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e1b50640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:36.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.359+0000 7f32e2351640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:36.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e2b52640 1 -- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32dc133d10 msgr2=0x7f32dc1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e2b52640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32dc133d10 0x7f32dc1341c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097a0 tx=0x7f32d4029fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec02448f61ff14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 1 -- 172.21.15.134:0/3348746371 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32d402ad30 con 0x7f32dc133d10 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: dump: 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.372 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 1 -- 172.21.15.134:0/3348746371 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d40307b0 con 0x7f32dc133d10 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 1 -- 172.21.15.134:0/3348746371 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32d4030d90 con 0x7f32dc133d10 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e134f640 10 monclient: dump: 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.374 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 5 monclient: authenticate success, global_id 44398 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 4 set_mon_vals no callback set 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 10 monclient: shutdown 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 1 -- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32dc133d10 msgr2=0x7f32dc1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.362+0000 7f32e45b5640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32dc133d10 0x7f32dc1341c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097a0 tx=0x7f32d4029fd0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 -- 172.21.15.134:0/3348746371 shutdown_connections 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32dc133d10 0x7f32dc1341c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f32dc137290 0x7f32dc137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 --2- 172.21.15.134:0/3348746371 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f32dc136eb0 0x7f32dc09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 -- 172.21.15.134:0/3348746371 >> 172.21.15.134:0/3348746371 conn(0x7f32dc0a99a0 msgr2=0x7f32dc0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 -- 172.21.15.134:0/3348746371 shutdown_connections 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 -- 172.21.15.134:0/3348746371 wait complete. 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 10 monclient: monmap: 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.376 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.363+0000 7f32e45b5640 1 Processor -- start 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 -- start start 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient: init 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient: _add_conns ranks=[6,3,2,4,7,5,1,0,8] 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 0x7f32dc132cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-d con 0x7f32dc133d10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32dc136eb0 0x7f32dc133200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-g con 0x7f32dc136eb0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 0x7f32dc0a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): picked mon.noname-b con 0x7f32dc137290 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc133af0 con 0x7f32dc137290 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0a2ee0 con 0x7f32dc136eb0 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc1364a0 con 0x7f32dc133d10 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e45b5640 10 monclient(hunting): authenticate will time out at 1192.953470s 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 0x7f32dc132cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 0x7f32dc0a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 0x7f32dc132cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57870/0 (socket says 172.21.15.134:57870) 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 1 -- 172.21.15.134:0/1211311135 learned_addr learned my addr 172.21.15.134:0/1211311135 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): get_auth_request con 0x7f32dc133d10 auth_method 0 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): get_auth_request con 0x7f32dc137290 auth_method 0 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32dc136eb0 0x7f32dc133200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): get_auth_request con 0x7f32dc136eb0 auth_method 0 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient(hunting): handle_auth_done global_id 44498 payload 824 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 msgr2=0x7f32dc132cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 0x7f32dc132cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32dc136eb0 msgr2=0x7f32dc133200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32dc136eb0 0x7f32dc133200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 monclient: found mon.noname-b 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8009a70 con 0x7f32dc137290 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.364+0000 7f32e2351640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f32dc133d10 0x7f32dc132cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 msgr2=0x7f32dc0a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e2b52640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 0x7f32dc0a0980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f32d4031000 tx=0x7f32d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32d4003610 con 0x7f32dc137290 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: dump: 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.381 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d40037b0 con 0x7f32dc137290 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 5 monclient: authenticate success, global_id 44498 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32d4003d90 con 0x7f32dc137290 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 10 monclient: _renew_subs 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc0094d0 con 0x7f32dc137290 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e0b4e640 4 set_mon_vals no callback set 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32d2ffd640 10 monclient: dump: 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.383 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 10 monclient: _renew_subs 2024-09-16T15:07:36.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.367+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d4009400 con 0x7f32dc137290 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f32d4039040 con 0x7f32dc137290 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e45b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32d2ffd640 1 --2- 172.21.15.134:0/1211311135 >> v2:172.21.15.162:6800/980547263 conn(0x7f32c0038880 0x7f32c003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e45b5640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 0x7f32dc0a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc073880 con 0x7f32dc0a0ec0 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(321..321 src has 1..321) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f32d406a840 con 0x7f32dc137290 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 0x7f32dc0a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient: get_auth_request con 0x7f32dc0a0ec0 auth_method 0 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e2351640 1 --2- 172.21.15.134:0/1211311135 >> v2:172.21.15.162:6800/980547263 conn(0x7f32c0038880 0x7f32c003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 10 monclient(hunting): handle_auth_done global_id 44369 payload 824 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e1b50640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32dc061ae0 con 0x7f32dc0a0ec0 2024-09-16T15:07:36.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.368+0000 7f32e2351640 10 monclient: get_auth_request con 0x7f32c0038880 auth_method 0 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.370+0000 7f32e1b50640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 msgr2=0x7f32dc0a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.370+0000 7f32e1b50640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 0x7f32dc0a12a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0a6df0 tx=0x7f32cc011e30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc00fec0 con 0x7f32dc0a0ec0 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:36.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 10 monclient: dump: 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.389 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32e2351640 1 --2- 172.21.15.134:0/1211311135 >> v2:172.21.15.162:6800/980547263 conn(0x7f32c0038880 0x7f32c003ad40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f32d8003e80 tx=0x7f32d800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f32cc014440 con 0x7f32dc0a0ec0 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 msgr2=0x7f32dc0a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 0x7f32dc0a12a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0a6df0 tx=0x7f32cc011e30 comp rx=0 tx=0).stop 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 msgr2=0x7f32dc0a12a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.371+0000 7f32d2ffd640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a0ec0 0x7f32dc0a12a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e45b5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e45b5640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 0x7f32dc1340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e45b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc071b20 con 0x7f32dc0a17e0 2024-09-16T15:07:36.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 0x7f32dc1340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient: get_auth_request con 0x7f32dc0a17e0 auth_method 0 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 10 monclient(hunting): handle_auth_done global_id 44378 payload 824 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.377+0000 7f32e1b50640 1 -- 172.21.15.134:0/1211311135 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f32dc07a680 con 0x7f32dc0a17e0 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e1b50640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 msgr2=0x7f32dc1340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e1b50640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 0x7f32dc1340f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32cc01c000 tx=0x7f32cc0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f32cc004490 con 0x7f32dc0a17e0 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:36.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 10 monclient: dump: 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.392 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f32cc0229a0 con 0x7f32dc0a17e0 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 msgr2=0x7f32dc1340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 0x7f32dc1340f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32cc01c000 tx=0x7f32cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 msgr2=0x7f32dc1340f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32d2ffd640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f32dc0a17e0 0x7f32dc1340f0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32cc01c000 tx=0x7f32cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.393 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:36.394 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.395 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.396 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.397 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.398 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.401 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.402 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.403 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.406 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.407 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.409 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.410 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.412 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.413 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.414 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 >> v2:172.21.15.162:6800/980547263 conn(0x7f32c0038880 msgr2=0x7f32c003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 --2- 172.21.15.134:0/1211311135 >> v2:172.21.15.162:6800/980547263 conn(0x7f32c0038880 0x7f32c003ad40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f32d8003e80 tx=0x7f32d800b040 comp rx=0 tx=0).stop 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 10 monclient: shutdown 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 msgr2=0x7f32dc0a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 --2- 172.21.15.134:0/1211311135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32dc137290 0x7f32dc0a0980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f32d4031000 tx=0x7f32d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 reap_dead start 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e2b52640 1 -- 172.21.15.134:0/1211311135 reap_dead start 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 shutdown_connections 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 >> 172.21.15.134:0/1211311135 conn(0x7f32dc0a99a0 msgr2=0x7f32dc0a8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 shutdown_connections 2024-09-16T15:07:36.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.380+0000 7f32e45b5640 1 -- 172.21.15.134:0/1211311135 wait complete. 2024-09-16T15:07:36.416 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:07:36.417 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2207221584}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4003543686}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1234400115}) 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4225035242}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=151401735}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2131627377}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=228860973}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1226243486}) 2024-09-16T15:07:36.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.429+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=440370754}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=56} crc {osdmap=3421769699}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=55} crc {osdmap=240834922}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=57} crc {osdmap=374018464}) 2024-09-16T15:07:36.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=56} crc {osdmap=2798279730}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=57} crc {osdmap=263230575}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=422431480}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.434+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=55} crc {osdmap=168633940}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.435+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.435+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=58} crc {osdmap=3800125350}) 2024-09-16T15:07:36.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:07:36.435+0000 7f7fc2745640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=56} crc {osdmap=178284088}) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1604: done (5 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1605: done (4 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1606: done (3 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1607: done (2 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1608: done (1 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1609: done (0 left) 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1610: write oid 5 current snap is 149 2024-09-16T15:07:36.438 INFO:tasks.rados.rados.0.smithi162.stdout:1610: seq_num 587 ranges {665709=730344,2133283=426203,3203532=509848} 2024-09-16T15:07:36.445 INFO:tasks.rados.rados.0.smithi162.stdout:1610: writing smithi16234619-5 from 665709 to 1396053 tid 1 2024-09-16T15:07:36.451 INFO:tasks.rados.rados.0.smithi162.stdout:1610: writing smithi16234619-5 from 2133283 to 2559486 tid 2 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:1610: writing smithi16234619-5 from 3203532 to 3713380 tid 3 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:1611: rollback oid 6 current snap is 149 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 148 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:1612: read oid 33 snap -1 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:1612: expect (ObjNum 557 snap 141 seq_num 557) 2024-09-16T15:07:36.458 INFO:tasks.rados.rados.0.smithi162.stdout:1613: snap_create 2024-09-16T15:07:36.460 INFO:tasks.rados.rados.0.smithi162.stdout:1612: expect (ObjNum 557 snap 141 seq_num 557) 2024-09-16T15:07:36.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.517+0000 7fbca719c640 10 monclient: get_monmap_and_config 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.517+0000 7fbca719c640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.517+0000 7fbca719c640 10 monclient: monmap: 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:36.518554+0000 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:36.518554+0000 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.519 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:36.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 Processor -- start 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 -- start start 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient: init 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient: _add_conns ranks=[3,2,10,0,1,7,5,6,8,4,9] 2024-09-16T15:07:36.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbca0136e40 0x7fbca009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-g con 0x7fbca0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-b con 0x7fbca0137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbca0135dc0 0x7fbca0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-f con 0x7fbca0135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca009abc0 con 0x7fbca0135dc0 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca009ba50 con 0x7fbca0137220 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00a53c0 con 0x7fbca0136e40 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.518+0000 7fbca719c640 10 monclient(hunting): authenticate will time out at 1193.107875s 2024-09-16T15:07:36.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:32856/0 (socket says 172.21.15.134:32856) 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 -- 172.21.15.134:0/2170784761 learned_addr learned my addr 172.21.15.134:0/2170784761 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): get_auth_request con 0x7fbca0137220 auth_method 0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbca0136e40 0x7fbca009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 1 --2- 172.21.15.134:0/2170784761 >> v2:172.21.15.162:3300/0 conn(0x7fbca0135dc0 0x7fbca0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): get_auth_request con 0x7fbca0136e40 auth_method 0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): get_auth_request con 0x7fbca0135dc0 auth_method 0 2024-09-16T15:07:36.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient(hunting): handle_auth_done global_id 44507 payload 274 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 -- 172.21.15.134:0/2170784761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbca0136e40 msgr2=0x7fbca009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbca0136e40 0x7fbca009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 -- 172.21.15.134:0/2170784761 >> v2:172.21.15.162:3300/0 conn(0x7fbca0135dc0 msgr2=0x7fbca0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2170784761 >> v2:172.21.15.162:3300/0 conn(0x7fbca0135dc0 0x7fbca0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.519+0000 7fbc97fff640 1 -- 172.21.15.134:0/2170784761 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0073250 con 0x7fbca0137220 2024-09-16T15:07:36.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc97fff640 1 -- 172.21.15.134:0/2170784761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 msgr2=0x7fbca0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098a0 tx=0x7fbc9002a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bf3c8b9b45a96a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 1 -- 172.21.15.134:0/2170784761 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc9002adf0 con 0x7fbca0137220 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: dump: 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.533 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 1 -- 172.21.15.134:0/2170784761 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900307b0 con 0x7fbca0137220 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 5 monclient: authenticate success, global_id 44507 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 1 -- 172.21.15.134:0/2170784761 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc90030d90 con 0x7fbca0137220 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 4 set_mon_vals no callback set 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbc977fe640 10 monclient: dump: 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.535 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 10 monclient: shutdown 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 -- 172.21.15.134:0/2170784761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 msgr2=0x7fbca0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098a0 tx=0x7fbc9002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 -- 172.21.15.134:0/2170784761 shutdown_connections 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 --2- 172.21.15.134:0/2170784761 >> v2:172.21.15.162:3300/0 conn(0x7fbca0135dc0 0x7fbca0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0137220 0x7fbca0137600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 --2- 172.21.15.134:0/2170784761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbca0136e40 0x7fbca009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 -- 172.21.15.134:0/2170784761 >> 172.21.15.134:0/2170784761 conn(0x7fbca00a6110 msgr2=0x7fbca00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 -- 172.21.15.134:0/2170784761 shutdown_connections 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.522+0000 7fbca719c640 1 -- 172.21.15.134:0/2170784761 wait complete. 2024-09-16T15:07:36.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.523+0000 7fbca719c640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.523+0000 7fbca719c640 10 monclient: monmap: 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.537 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.523+0000 7fbca719c640 1 Processor -- start 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.523+0000 7fbca719c640 1 -- start start 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.523+0000 7fbca719c640 10 monclient: init 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient: _add_conns ranks=[8,2,5,0,7,1,6,3,4] 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbca0135dc0 0x7fbca0134e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-e con 0x7fbca0135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-b con 0x7fbca0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbca0137220 0x7fbca0121890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): picked mon.noname-h con 0x7fbca0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0136850 con 0x7fbca0136e40 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0136300 con 0x7fbca0137220 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0125840 con 0x7fbca0135dc0 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca719c640 10 monclient(hunting): authenticate will time out at 1193.113339s 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbca0135dc0 0x7fbca0134e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:32872/0 (socket says 172.21.15.134:32872) 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbca0135dc0 0x7fbca0134e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:58994/0 (socket says 172.21.15.134:58994) 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 -- 172.21.15.134:0/2567144859 learned_addr learned my addr 172.21.15.134:0/2567144859 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): get_auth_request con 0x7fbca0136e40 auth_method 0 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): get_auth_request con 0x7fbca0135dc0 auth_method 0 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbca0137220 0x7fbca0121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): get_auth_request con 0x7fbca0137220 auth_method 0 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbca5739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient(hunting): handle_auth_done global_id 44516 payload 824 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbca0135dc0 msgr2=0x7fbca0134e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbca0135dc0 0x7fbca0134e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbca0137220 msgr2=0x7fbca0121890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbca0137220 0x7fbca0121890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.524+0000 7fbc97fff640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900094d0 con 0x7fbca0136e40 2024-09-16T15:07:36.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.526+0000 7fbc97fff640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 msgr2=0x7fbca0135390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.526+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc90035040 tx=0x7fbc900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc90030d90 con 0x7fbca0136e40 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: dump: 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.542 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc90003610 con 0x7fbca0136e40 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 5 monclient: authenticate success, global_id 44516 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 10 monclient: _renew_subs 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca00a8980 con 0x7fbca0136e40 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc90003bf0 con 0x7fbca0136e40 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 10 monclient: dump: 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.544 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 10 monclient: _renew_subs 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca0125560 con 0x7fbca0136e40 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc96ffd640 4 set_mon_vals no callback set 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc97fff640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc90035040 tx=0x7fbc900046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fbc90039040 mgrmap(e 5) v1 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbc90039040 con 0x7fbca0136e40 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.527+0000 7fbca719c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca719c640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 0x7fbca01221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0071b20 con 0x7fbca0121dd0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbc957fa640 1 --2- 172.21.15.134:0/2567144859 >> v2:172.21.15.162:6800/980547263 conn(0x7fbc8c038880 0x7fbc8c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 0x7fbca01221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient: get_auth_request con 0x7fbca0121dd0 auth_method 0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca5739640 1 --2- 172.21.15.134:0/2567144859 >> v2:172.21.15.162:6800/980547263 conn(0x7fbc8c038880 0x7fbc8c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca5739640 10 monclient: get_auth_request con 0x7fbc8c038880 auth_method 0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 10 monclient(hunting): handle_auth_done global_id 44416 payload 824 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.528+0000 7fbca4f38640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbca0061ae0 con 0x7fbca0121dd0 2024-09-16T15:07:36.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbca4f38640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 msgr2=0x7fbca01221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbca4f38640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 0x7fbca01221b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f410 tx=0x7fbc9c00f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbca5739640 1 --2- 172.21.15.134:0/2567144859 >> v2:172.21.15.162:6800/980547263 conn(0x7fbc8c038880 0x7fbc8c03ad40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbca0123070 tx=0x7fbc98009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc9c00e040 con 0x7fbca0121dd0 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbc957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbc957fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.530+0000 7fbc957fa640 10 monclient: dump: 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.547 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbc9c00f040 con 0x7fbca0121dd0 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 msgr2=0x7fbca01221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 0x7fbca01221b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f410 tx=0x7fbc9c00f8d0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 msgr2=0x7fbca01221b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.531+0000 7fbc957fa640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca0121dd0 0x7fbca01221b0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00f410 tx=0x7fbc9c00f8d0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca719c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca719c640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 0x7fbca00a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca719c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca007a680 con 0x7fbca01226f0 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 0x7fbca00a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient: get_auth_request con 0x7fbca01226f0 auth_method 0 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 10 monclient(hunting): handle_auth_done global_id 44425 payload 824 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.537+0000 7fbca4f38640 1 -- 172.21.15.134:0/2567144859 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbca009b490 con 0x7fbca01226f0 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.539+0000 7fbca4f38640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 msgr2=0x7fbca00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.539+0000 7fbca4f38640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 0x7fbca00a1a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b010 tx=0x7fbc9c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbc9c024070 con 0x7fbca01226f0 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 10 monclient: dump: 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.552 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fbc9c01e660 con 0x7fbca01226f0 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 msgr2=0x7fbca00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 0x7fbca00a1a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b010 tx=0x7fbc9c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 msgr2=0x7fbca00a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbca01226f0 0x7fbca00a1a20 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b010 tx=0x7fbc9c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.554 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.555 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.556 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.557 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.558 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.560 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.561 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.562 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.563 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.564 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.565 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.566 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.569 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.570 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.572 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.574 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.576 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.577 INFO:tasks.rados.rados.0.smithi162.stdout:1611: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:07:36.577 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 0 (ObjNum 289 snap 0 seq_num 1839817040) dirty dne 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: finishing write tid 1 to smithi16234619-5 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: finishing write tid 2 to smithi16234619-5 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: finishing write tid 3 to smithi16234619-5 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: finishing write tid 4 to smithi16234619-5 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: finishing write tid 5 to smithi16234619-5 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: oid 5 updating version 0 to 735 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: oid 5 version 735 is already newer than 733 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: oid 5 version 735 is already newer than 732 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: oid 5 version 735 is already newer than 734 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 735 (ObjNum 587 snap 149 seq_num 587) dirty exists 2024-09-16T15:07:36.578 INFO:tasks.rados.rados.0.smithi162.stdout:1610: left oid 5 (ObjNum 587 snap 149 seq_num 587) 2024-09-16T15:07:36.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 >> v2:172.21.15.162:6800/980547263 conn(0x7fbc8c038880 msgr2=0x7fbc8c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 --2- 172.21.15.134:0/2567144859 >> v2:172.21.15.162:6800/980547263 conn(0x7fbc8c038880 0x7fbc8c03ad40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbca0123070 tx=0x7fbc98009190 comp rx=0 tx=0).stop 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 10 monclient: shutdown 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 msgr2=0x7fbca0135390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc90035040 tx=0x7fbc900046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca5739640 1 -- 172.21.15.134:0/2567144859 reap_dead start 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbc957fa640 1 -- 172.21.15.134:0/2567144859 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7fbc9006a460 con 0x7fbca0136e40 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 shutdown_connections 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 --2- 172.21.15.134:0/2567144859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbca0136e40 0x7fbca0135390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 >> 172.21.15.134:0/2567144859 conn(0x7fbca00a6110 msgr2=0x7fbca006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 shutdown_connections 2024-09-16T15:07:36.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.540+0000 7fbca719c640 1 -- 172.21.15.134:0/2567144859 wait complete. 2024-09-16T15:07:36.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:07:36.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.716+0000 7fefe8d5a640 10 monclient: get_monmap_and_config 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.716+0000 7fefe8d5a640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.716+0000 7fefe8d5a640 10 monclient: monmap: 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:36.717818+0000 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:36.717818+0000 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.718 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:36.719 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.717+0000 7fefe8d5a640 1 Processor -- start 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.717+0000 7fefe8d5a640 1 -- start start 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.717+0000 7fefe8d5a640 10 monclient: init 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient: _add_conns ranks=[0,1,5,4,8,10,2,6,9,7,3] 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fefe4136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fefe4137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 0x7fefe413f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-h con 0x7fefe413d380 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe4073250 con 0x7fefe4136e40 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe409ba50 con 0x7fefe4137220 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefe409c860 con 0x7fefe413d380 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe8d5a640 10 monclient(hunting): authenticate will time out at 1193.307244s 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe259c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fefe4136e40 0x7fefe409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:34414/0 (socket says 172.21.15.134:34414) 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe259c640 1 -- 172.21.15.134:0/3107473787 learned_addr learned my addr 172.21.15.134:0/3107473787 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 1 --2- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 0x7fefe413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): get_auth_request con 0x7fefe413d380 auth_method 0 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_done global_id 44402 payload 274 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 1 -- 172.21.15.134:0/3107473787 >> v1:172.21.15.162:6789/0 conn(0x7fefe4137220 legacy=0x7fefe4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 1 -- 172.21.15.134:0/3107473787 >> v1:172.21.15.134:6789/0 conn(0x7fefe4136e40 legacy=0x7fefe409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 1 monclient: found mon.noname-h 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:07:36.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.718+0000 7fefe2d9d640 1 -- 172.21.15.134:0/3107473787 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40a53c0 con 0x7fefe413d380 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.721+0000 7fefe2d9d640 1 -- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 msgr2=0x7fefe413f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.721+0000 7fefe2d9d640 1 --2- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 0x7fefe413f770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009dc0 tx=0x7fefdc02a510 comp rx=0 tx=0).ready entity=mon.5 client_cookie=76dfd17864e4f883 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 1 -- 172.21.15.134:0/3107473787 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefdc007ca0 con 0x7fefe413d380 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: dump: 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.727 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 5 monclient: authenticate success, global_id 44402 2024-09-16T15:07:36.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 1 -- 172.21.15.134:0/3107473787 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc02d050 con 0x7fefe413d380 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 4 set_mon_vals no callback set 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 1 -- 172.21.15.134:0/3107473787 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefdc007e20 con 0x7fefe413d380 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe159a640 10 monclient: dump: 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.729 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 10 monclient: shutdown 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 1 -- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 msgr2=0x7fefe413f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 0x7fefe413f770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009dc0 tx=0x7fefdc02a510 comp rx=0 tx=0).stop 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 1 -- 172.21.15.134:0/3107473787 shutdown_connections 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.722+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/3107473787 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe413d380 0x7fefe413f770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.723+0000 7fefe8d5a640 1 -- 172.21.15.134:0/3107473787 >> 172.21.15.134:0/3107473787 conn(0x7fefe40a6110 msgr2=0x7fefe40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.723+0000 7fefe8d5a640 1 -- 172.21.15.134:0/3107473787 shutdown_connections 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.723+0000 7fefe8d5a640 1 -- 172.21.15.134:0/3107473787 wait complete. 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.723+0000 7fefe8d5a640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.723+0000 7fefe8d5a640 10 monclient: monmap: 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.731 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 Processor -- start 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 -- start start 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient: init 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient: _add_conns ranks=[6,5,3,8,2,1,4,0,7] 2024-09-16T15:07:36.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-d con 0x7fefe4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe4137220 0x7fefe4139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-h con 0x7fefe4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413d380 0x7fefe413a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): picked mon.noname-g con 0x7fefe413d380 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40a6810 con 0x7fefe413d380 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4137920 con 0x7fefe4137220 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4121190 con 0x7fefe4136e40 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe8d5a640 10 monclient(hunting): authenticate will time out at 1193.313533s 2024-09-16T15:07:36.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57874/0 (socket says 172.21.15.134:57874) 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 -- 172.21.15.134:0/1513278527 learned_addr learned my addr 172.21.15.134:0/1513278527 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): get_auth_request con 0x7fefe4136e40 auth_method 0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe4137220 0x7fefe4139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413d380 0x7fefe413a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): get_auth_request con 0x7fefe413d380 auth_method 0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): get_auth_request con 0x7fefe4137220 auth_method 0 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient(hunting): handle_auth_done global_id 34656 payload 824 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe4137220 msgr2=0x7fefe4139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fefe4137220 0x7fefe4139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413d380 msgr2=0x7fefe413a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413d380 0x7fefe413a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 monclient: found mon.noname-d 2024-09-16T15:07:36.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:36.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.724+0000 7fefe259c640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc009a70 con 0x7fefe4136e40 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe259c640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 msgr2=0x7fefe4136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe259c640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c490 tx=0x7fefd400c950 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefd4005cf0 con 0x7fefe4136e40 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: dump: 2024-09-16T15:07:36.739 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.740 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4005e90 con 0x7fefe4136e40 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 5 monclient: authenticate success, global_id 34656 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 10 monclient: _renew_subs 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe413bad0 con 0x7fefe4136e40 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefd40074e0 con 0x7fefe4136e40 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefd2ffd640 10 monclient: dump: 2024-09-16T15:07:36.741 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.742 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 10 monclient: _renew_subs 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe0d99640 4 set_mon_vals no callback set 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.727+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe4136560 con 0x7fefe4136e40 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fefd4007640 con 0x7fefe4136e40 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe8d5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 0x7fefe413adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4071b20 con 0x7fefe413aa10 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefd2ffd640 1 --2- 172.21.15.134:0/1513278527 >> v2:172.21.15.162:6800/980547263 conn(0x7fefc4038880 0x7fefc403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fefd4049bd0 con 0x7fefe4136e40 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 0x7fefe413adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient: get_auth_request con 0x7fefe413aa10 auth_method 0 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe2d9d640 1 --2- 172.21.15.134:0/1513278527 >> v2:172.21.15.162:6800/980547263 conn(0x7fefc4038880 0x7fefc403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe2d9d640 10 monclient: get_auth_request con 0x7fefc4038880 auth_method 0 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_done global_id 54174 payload 824 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.728+0000 7fefe1d9b640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefe4061ae0 con 0x7fefe413aa10 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.730+0000 7fefe1d9b640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 msgr2=0x7fefe413adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.730+0000 7fefe1d9b640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 0x7fefe413adf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd800fc30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefe2d9d640 1 --2- 172.21.15.134:0/1513278527 >> v2:172.21.15.162:6800/980547263 conn(0x7fefc4038880 0x7fefc403ad40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fefdc031000 tx=0x7fefdc02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefd800fe40 con 0x7fefe413aa10 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:36.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 10 monclient: dump: 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.745 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fefd800dbc0 con 0x7fefe413aa10 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 msgr2=0x7fefe413adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 0x7fefe413adf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd800fc30 comp rx=0 tx=0).stop 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 msgr2=0x7fefe413adf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.731+0000 7fefd2ffd640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe413aa10 0x7fefe413adf0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd800fc30 comp rx=0 tx=0).stop 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe8d5a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 0x7fefe409e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe8d5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe407a680 con 0x7fefe409e520 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 0x7fefe409e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient: get_auth_request con 0x7fefe409e520 auth_method 0 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 10 monclient(hunting): handle_auth_done global_id 54183 payload 824 2024-09-16T15:07:36.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.737+0000 7fefe1d9b640 1 -- 172.21.15.134:0/1513278527 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fefe409abc0 con 0x7fefe409e520 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefe1d9b640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 msgr2=0x7fefe409e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefe1d9b640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 0x7fefe409e930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd8003760 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fefd801ec40 con 0x7fefe409e520 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 10 monclient: dump: 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.748 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fefd801eda0 con 0x7fefe409e520 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 msgr2=0x7fefe409e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 0x7fefe409e930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd8003760 comp rx=0 tx=0).stop 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 msgr2=0x7fefe409e930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.740+0000 7fefd2ffd640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefe409e520 0x7fefe409e930 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefd800f770 tx=0x7fefd8003760 comp rx=0 tx=0).stop 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.750 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.751 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.752 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.753 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.754 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.755 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.756 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.757 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.758 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.759 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.760 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.761 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.763 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.765 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.766 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.767 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.769 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.770 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.771 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 >> v2:172.21.15.162:6800/980547263 conn(0x7fefc4038880 msgr2=0x7fefc403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/1513278527 >> v2:172.21.15.162:6800/980547263 conn(0x7fefc4038880 0x7fefc403ad40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fefdc031000 tx=0x7fefdc02f000 comp rx=0 tx=0).stop 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 10 monclient: shutdown 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 msgr2=0x7fefe4136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fefd400c490 tx=0x7fefd400c950 comp rx=0 tx=0).stop 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe2d9d640 1 -- 172.21.15.134:0/1513278527 reap_dead start 2024-09-16T15:07:36.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 shutdown_connections 2024-09-16T15:07:36.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 --2- 172.21.15.134:0/1513278527 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fefe4136e40 0x7fefe4136020 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 >> 172.21.15.134:0/1513278527 conn(0x7fefe40a6110 msgr2=0x7fefe40a7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 shutdown_connections 2024-09-16T15:07:36.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.741+0000 7fefe8d5a640 1 -- 172.21.15.134:0/1513278527 wait complete. 2024-09-16T15:07:36.773 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:07:36.773 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:07:36.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.874+0000 7f4c6de53640 10 monclient: get_monmap_and_config 2024-09-16T15:07:36.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.874+0000 7f4c6de53640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.874+0000 7f4c6de53640 10 monclient: monmap: 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:36.875117+0000 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:36.875117+0000 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.876 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:36.877 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 Processor -- start 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 -- start start 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient: init 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient: _add_conns ranks=[8,1,3,10,4,9,5,0,6,2,7] 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-e con 0x7f4c68135170 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4c68071450 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c680a2670 0x7f4c6813cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-g con 0x7f4c680a2670 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c68073250 con 0x7f4c68071450 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680a3d40 con 0x7f4c680a2670 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680a3ee0 con 0x7f4c68135170 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c6de53640 10 monclient(hunting): authenticate will time out at 1193.464695s 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:59002/0 (socket says 172.21.15.134:59002) 2024-09-16T15:07:36.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 1 -- 172.21.15.134:0/709287422 learned_addr learned my addr 172.21.15.134:0/709287422 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): get_auth_request con 0x7f4c68135170 auth_method 0 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c67fff640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c680a2670 0x7f4c6813cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c67fff640 10 monclient(hunting): get_auth_request con 0x7f4c680a2670 auth_method 0 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.875+0000 7f4c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 10 monclient(hunting): handle_auth_done global_id 44396 payload 274 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 1 -- 172.21.15.134:0/709287422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c680a2670 msgr2=0x7f4c6813cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c680a2670 0x7f4c6813cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 1 -- 172.21.15.134:0/709287422 >> v1:172.21.15.162:6789/0 conn(0x7f4c68071450 legacy=0x7f4c680a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 1 monclient: found mon.noname-e 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.876+0000 7f4c677fe640 1 -- 172.21.15.134:0/709287422 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680709b0 con 0x7f4c68135170 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.878+0000 7f4c677fe640 1 -- 172.21.15.134:0/709287422 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 msgr2=0x7f4c68135550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.878+0000 7f4c677fe640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098d0 tx=0x7f4c5802a100 comp rx=0 tx=0).ready entity=mon.8 client_cookie=754c5526f7a552d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 1 -- 172.21.15.134:0/709287422 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c5802ae20 con 0x7f4c68135170 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: dump: 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.885 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 5 monclient: authenticate success, global_id 44396 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 1 -- 172.21.15.134:0/709287422 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580307b0 con 0x7f4c68135170 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:36.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 4 set_mon_vals no callback set 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 1 -- 172.21.15.134:0/709287422 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c58030d90 con 0x7f4c68135170 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c66ffd640 10 monclient: dump: 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.887 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 10 monclient: shutdown 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 -- 172.21.15.134:0/709287422 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 msgr2=0x7f4c68135550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098d0 tx=0x7f4c5802a100 comp rx=0 tx=0).stop 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 -- 172.21.15.134:0/709287422 shutdown_connections 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c680a2670 0x7f4c6813cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 --2- 172.21.15.134:0/709287422 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68135170 0x7f4c68135550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 -- 172.21.15.134:0/709287422 >> 172.21.15.134:0/709287422 conn(0x7f4c680a7560 msgr2=0x7f4c680a7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 -- 172.21.15.134:0/709287422 shutdown_connections 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.879+0000 7f4c6de53640 1 -- 172.21.15.134:0/709287422 wait complete. 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.880+0000 7f4c6de53640 10 monclient: build_initial_monmap 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.880+0000 7f4c6de53640 10 monclient: monmap: 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:36.889 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:36.890 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.880+0000 7f4c6de53640 1 Processor -- start 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.880+0000 7f4c6de53640 1 -- start start 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.880+0000 7f4c6de53640 10 monclient: init 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient: _add_conns ranks=[8,2,7,3,6,5,1,0,4] 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-e con 0x7f4c68071450 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c680a2670 0x7f4c68134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-b con 0x7f4c680a2670 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c68135170 0x7f4c681345b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): picked mon.noname-i con 0x7f4c68135170 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68127840 con 0x7f4c680a2670 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68127ab0 con 0x7f4c68135170 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680a4aa0 con 0x7f4c68071450 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c6de53640 10 monclient(hunting): authenticate will time out at 1193.470298s 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c680a2670 0x7f4c68134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:59004/0 (socket says 172.21.15.134:59004) 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 -- 172.21.15.134:0/967913815 learned_addr learned my addr 172.21.15.134:0/967913815 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:36.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): get_auth_request con 0x7f4c68071450 auth_method 0 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request con 0x7f4c680a2670 auth_method 0 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c68135170 0x7f4c681345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): get_auth_request con 0x7f4c68135170 auth_method 0 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c67fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient(hunting): handle_auth_done global_id 44405 payload 824 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c68135170 msgr2=0x7f4c681345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c68135170 0x7f4c681345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c680a2670 msgr2=0x7f4c68134070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c680a2670 0x7f4c68134070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 monclient: found mon.noname-e 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.881+0000 7f4c677fe640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580094d0 con 0x7f4c68071450 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.883+0000 7f4c677fe640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 msgr2=0x7f4c681cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.883+0000 7f4c677fe640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098a0 tx=0x7f4c580046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c58003610 con 0x7f4c68071450 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: dump: 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.901 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.902 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: _finish_auth 0 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580037b0 con 0x7f4c68071450 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 5 monclient: authenticate success, global_id 44405 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 10 monclient: _renew_subs 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680a4770 con 0x7f4c68071450 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c58003d90 con 0x7f4c68071450 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c64ff9640 10 monclient: dump: 2024-09-16T15:07:36.903 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.904 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c667fc640 4 set_mon_vals no callback set 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 10 monclient: _renew_subs 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.884+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68127210 con 0x7f4c68071450 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c6de53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c6de53640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 0x7f4c6806b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68073880 con 0x7f4c6806af60 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4c58003920 con 0x7f4c68071450 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c64ff9640 1 --2- 172.21.15.134:0/967913815 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c480388e0 0x7f4c4803ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 0x7f4c6806b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f4c5806a600 con 0x7f4c68071450 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient: get_auth_request con 0x7f4c6806af60 auth_method 0 2024-09-16T15:07:36.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c67fff640 1 --2- 172.21.15.134:0/967913815 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c480388e0 0x7f4c4803ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c67fff640 10 monclient: get_auth_request con 0x7f4c480388e0 auth_method 0 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_done global_id 44420 payload 824 2024-09-16T15:07:36.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.885+0000 7f4c5ffff640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c68061ae0 con 0x7f4c6806af60 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.887+0000 7f4c5ffff640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 msgr2=0x7f4c6806b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.887+0000 7f4c5ffff640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 0x7f4c6806b340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000f770 tx=0x7f4c5000fc30 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.887+0000 7f4c67fff640 1 --2- 172.21.15.134:0/967913815 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c480388e0 0x7f4c4803ada0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c600113e0 tx=0x7f4c6000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c5000de20 con 0x7f4c6806af60 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 10 monclient: dump: 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.907 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4c5000d9b0 con 0x7f4c6806af60 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:36.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 msgr2=0x7f4c6806b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 0x7f4c6806b340 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000f770 tx=0x7f4c5000fc30 comp rx=0 tx=0).stop 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 msgr2=0x7f4c6806b340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.888+0000 7f4c64ff9640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806af60 0x7f4c6806b340 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000f770 tx=0x7f4c5000fc30 comp rx=0 tx=0).stop 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c6de53640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c6de53640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 0x7f4c6811dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c6de53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6806a3f0 con 0x7f4c6806b880 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 0x7f4c6811dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:36.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient: get_auth_request con 0x7f4c6806b880 auth_method 0 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.894+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.895+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.895+0000 7f4c5ffff640 10 monclient(hunting): handle_auth_done global_id 44429 payload 824 2024-09-16T15:07:36.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.895+0000 7f4c5ffff640 1 -- 172.21.15.134:0/967913815 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4c6807a680 con 0x7f4c6806b880 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c5ffff640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 msgr2=0x7f4c6811dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c5ffff640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 0x7f4c6811dbb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000f770 tx=0x7f4c50003760 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c50022070 con 0x7f4c6806b880 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c64ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c64ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.897+0000 7f4c64ff9640 10 monclient: dump: 2024-09-16T15:07:36.911 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:36.912 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f4c50004000 con 0x7f4c6806b880 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 msgr2=0x7f4c6811dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 0x7f4c6811dbb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000f770 tx=0x7f4c50003760 comp rx=0 tx=0).stop 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 msgr2=0x7f4c6811dbb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c64ff9640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4c6806b880 0x7f4c6811dbb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:36.913 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:36.914 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.915 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.916 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:36.917 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:36.918 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.919 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.920 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:36.921 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.922 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:36.923 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.924 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.925 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.926 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.927 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:36.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.931 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.933 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:36.934 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:36.935 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:36.935 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:36.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:36.935 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:36.935 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c480388e0 msgr2=0x7f4c4803ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 --2- 172.21.15.134:0/967913815 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c480388e0 0x7f4c4803ada0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c600113e0 tx=0x7f4c6000b040 comp rx=0 tx=0).stop 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 10 monclient: shutdown 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 msgr2=0x7f4c681cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:36.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098a0 tx=0x7f4c580046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c67fff640 1 -- 172.21.15.134:0/967913815 reap_dead start 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 shutdown_connections 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 --2- 172.21.15.134:0/967913815 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c68071450 0x7f4c681cb750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.898+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 >> 172.21.15.134:0/967913815 conn(0x7f4c680a7560 msgr2=0x7f4c6809fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.899+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 shutdown_connections 2024-09-16T15:07:36.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:36.899+0000 7f4c6de53640 1 -- 172.21.15.134:0/967913815 wait complete. 2024-09-16T15:07:36.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:36.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:07:36.937 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:07:37.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.033+0000 7fb46b35e640 10 monclient: get_monmap_and_config 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.033+0000 7fb46b35e640 10 monclient: build_initial_monmap 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.033+0000 7fb46b35e640 10 monclient: monmap: 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:37.034479+0000 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:37.034479+0000 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:37.035 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:37.036 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 Processor -- start 2024-09-16T15:07:37.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 -- start start 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient: init 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient: _add_conns ranks=[9,2,3,7,4,0,6,8,10,1,5] 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb464136eb0 0x7fb46409bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-a con 0x7fb464136eb0 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-b con 0x7fb464137290 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb464133d10 0x7fb4641341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-g con 0x7fb464133d10 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640a5560 con 0x7fb464136eb0 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46409b240 con 0x7fb464137290 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46409b520 con 0x7fb464133d10 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb46b35e640 10 monclient(hunting): authenticate will time out at 1193.623867s 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb4688f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.034+0000 7fb4690fa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb464136eb0 0x7fb46409bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:32882/0 (socket says 172.21.15.134:32882) 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 1 -- 172.21.15.134:0/2495867936 learned_addr learned my addr 172.21.15.134:0/2495867936 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): get_auth_request con 0x7fb464137290 auth_method 0 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): get_auth_request con 0x7fb464136eb0 auth_method 0 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4698fb640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb464133d10 0x7fb4641341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4698fb640 10 monclient(hunting): get_auth_request con 0x7fb464133d10 auth_method 0 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4698fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4698fb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4698fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient(hunting): handle_auth_done global_id 5148 payload 274 2024-09-16T15:07:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 -- 172.21.15.134:0/2495867936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb464133d10 msgr2=0x7fb4641341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb464133d10 0x7fb4641341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 -- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 msgr2=0x7fb464137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient: _finish_hunting 0 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 monclient: found mon.noname-a 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4690fa640 1 -- 172.21.15.134:0/2495867936 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464073250 con 0x7fb464136eb0 2024-09-16T15:07:37.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.035+0000 7fb4688f9640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb4690fa640 1 -- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464136eb0 msgr2=0x7fb46409bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb4690fa640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464136eb0 0x7fb46409bf00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb460009d90 tx=0x7fb46002a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb9d0abf781caab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 1 -- 172.21.15.134:0/2495867936 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb460007bc0 con 0x7fb464136eb0 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: dump: 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.050 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: _finish_auth 0 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 1 -- 172.21.15.134:0/2495867936 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46002d050 con 0x7fb464136eb0 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 5 monclient: authenticate success, global_id 5148 2024-09-16T15:07:37.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 1 -- 172.21.15.134:0/2495867936 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb460007d40 con 0x7fb464136eb0 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 4 set_mon_vals no callback set 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb45bfff640 10 monclient: dump: 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.052 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 10 monclient: shutdown 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 -- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464136eb0 msgr2=0x7fb46409bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464136eb0 0x7fb46409bf00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb460009d90 tx=0x7fb46002a490 comp rx=0 tx=0).stop 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 -- 172.21.15.134:0/2495867936 shutdown_connections 2024-09-16T15:07:37.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb464133d10 0x7fb4641341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb464137290 0x7fb464137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 --2- 172.21.15.134:0/2495867936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb464136eb0 0x7fb46409bf00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 -- 172.21.15.134:0/2495867936 >> 172.21.15.134:0/2495867936 conn(0x7fb4640a99a0 msgr2=0x7fb4640a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 -- 172.21.15.134:0/2495867936 shutdown_connections 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.038+0000 7fb46b35e640 1 -- 172.21.15.134:0/2495867936 wait complete. 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.039+0000 7fb46b35e640 10 monclient: build_initial_monmap 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.039+0000 7fb46b35e640 10 monclient: monmap: 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:37.054 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.039+0000 7fb46b35e640 1 Processor -- start 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.039+0000 7fb46b35e640 1 -- start start 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.039+0000 7fb46b35e640 10 monclient: init 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:37.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient: _add_conns ranks=[1,5,7,4,0,2,6,3,8] 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb464133d10 0x7fb464132d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-f con 0x7fb464133d10 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 0x7fb46409d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-h con 0x7fb464136eb0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 0x7fb46409d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): picked mon.noname-i con 0x7fb464137290 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641362e0 con 0x7fb464133d10 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640a7960 con 0x7fb464136eb0 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640a7480 con 0x7fb464137290 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:37.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb46b35e640 10 monclient(hunting): authenticate will time out at 1193.629354s 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 0x7fb46409d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 0x7fb46409d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb464133d10 0x7fb464132d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 0x7fb46409d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:49330/0 (socket says 172.21.15.134:49330) 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 0x7fb46409d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:43894/0 (socket says 172.21.15.134:43894) 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 1 -- 172.21.15.134:0/771071314 learned_addr learned my addr 172.21.15.134:0/771071314 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): get_auth_request con 0x7fb464136eb0 auth_method 0 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): get_auth_request con 0x7fb464133d10 auth_method 0 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): get_auth_request con 0x7fb464137290 auth_method 0 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient(hunting): handle_auth_done global_id 54178 payload 824 2024-09-16T15:07:37.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 msgr2=0x7fb46409d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 0x7fb46409d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb464133d10 msgr2=0x7fb464132d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb464133d10 0x7fb464132d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient: _finish_hunting 0 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 monclient: found mon.noname-i 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540094d0 con 0x7fb464137290 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4690fa640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb464133d10 0x7fb464132d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.040+0000 7fb4688f9640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb464136eb0 0x7fb46409d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 msgr2=0x7fb46409d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb4698fb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 0x7fb46409d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d530 tx=0x7fb45c00d9f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb45c012c70 con 0x7fb464137290 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: dump: 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: _finish_auth 0 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 5 monclient: authenticate success, global_id 54178 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c012dd0 con 0x7fb464137290 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 10 monclient: _renew_subs 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb460009a70 con 0x7fb464137290 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb45c0148a0 con 0x7fb464137290 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb459ffb640 10 monclient: dump: 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.062 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 10 monclient: _renew_subs 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb45b7fe640 4 set_mon_vals no callback set 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.043+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4641205a0 con 0x7fb464137290 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb46b35e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb46b35e640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 0x7fb46409df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb464073880 con 0x7fb46411fd00 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb45c014430 con 0x7fb464137290 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb4690fa640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 0x7fb46409df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb459ffb640 1 --2- 172.21.15.134:0/771071314 >> v2:172.21.15.162:6800/980547263 conn(0x7fb448038880 0x7fb44803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb4690fa640 10 monclient: get_auth_request con 0x7fb46411fd00 auth_method 0 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb4690fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb4690fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb4690fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.044+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fb45c04a6a0 con 0x7fb464137290 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4688f9640 1 --2- 172.21.15.134:0/771071314 >> v2:172.21.15.162:6800/980547263 conn(0x7fb448038880 0x7fb44803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4688f9640 10 monclient: get_auth_request con 0x7fb448038880 auth_method 0 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4690fa640 10 monclient(hunting): handle_auth_done global_id 54187 payload 824 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.045+0000 7fb4690fa640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb464061ae0 con 0x7fb46411fd00 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb4690fa640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 msgr2=0x7fb46409df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb4690fa640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 0x7fb46409df00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb460034000 tx=0x7fb460004a40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb4688f9640 1 --2- 172.21.15.134:0/771071314 >> v2:172.21.15.162:6800/980547263 conn(0x7fb448038880 0x7fb44803ad40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4640a3810 tx=0x7fb454009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb460007ea0 con 0x7fb46411fd00 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 10 monclient: dump: 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.065 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb460004460 con 0x7fb46411fd00 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 msgr2=0x7fb46409df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 0x7fb46409df00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb460034000 tx=0x7fb460004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 msgr2=0x7fb46409df00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:37.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.047+0000 7fb459ffb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb46411fd00 0x7fb46409df00 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb460034000 tx=0x7fb460004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb46b35e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb46b35e640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 0x7fb46409e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb46b35e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb464120730 con 0x7fb4641332a0 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 0x7fb46409e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient: get_auth_request con 0x7fb4641332a0 auth_method 0 2024-09-16T15:07:37.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 10 monclient(hunting): handle_auth_done global_id 54196 payload 824 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.054+0000 7fb4690fa640 1 -- 172.21.15.134:0/771071314 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb4640a5cf0 con 0x7fb4641332a0 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.056+0000 7fb4690fa640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 msgr2=0x7fb46409e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.056+0000 7fb4690fa640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 0x7fb46409e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4640a75c0 tx=0x7fb460030000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb46004b2e0 con 0x7fb4641332a0 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 10 monclient: dump: 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.070 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fb46003c420 con 0x7fb4641332a0 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 msgr2=0x7fb46409e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 0x7fb46409e440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4640a75c0 tx=0x7fb460030000 comp rx=0 tx=0).stop 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 msgr2=0x7fb46409e440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb459ffb640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb4641332a0 0x7fb46409e440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:37.071 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 58, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:37.072 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:37.073 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.074 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:37.075 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.076 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.078 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.079 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:37.080 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.082 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:37.084 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:37.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.087 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.088 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:37.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:37.090 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:37.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:37.092 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:37.092 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:37.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:37.092 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:37.092 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 >> v2:172.21.15.162:6800/980547263 conn(0x7fb448038880 msgr2=0x7fb44803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 --2- 172.21.15.134:0/771071314 >> v2:172.21.15.162:6800/980547263 conn(0x7fb448038880 0x7fb44803ad40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4640a3810 tx=0x7fb454009240 comp rx=0 tx=0).stop 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 10 monclient: shutdown 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 msgr2=0x7fb46409d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 --2- 172.21.15.134:0/771071314 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb464137290 0x7fb46409d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d530 tx=0x7fb45c00d9f0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 reap_dead start 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb4698fb640 1 -- 172.21.15.134:0/771071314 reap_dead start 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 shutdown_connections 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 >> 172.21.15.134:0/771071314 conn(0x7fb4640a99a0 msgr2=0x7fb4640a8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 shutdown_connections 2024-09-16T15:07:37.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.057+0000 7fb46b35e640 1 -- 172.21.15.134:0/771071314 wait complete. 2024-09-16T15:07:37.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:07:37.094 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a', 'e'] 2024-09-16T15:07:37.094 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:07:37.094 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-16T15:07:37.095 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:07:37.095 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:07:37.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:07:37.108 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:07:37.108 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-16T15:07:37.108 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:07:37.108 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-16T15:07:37.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:07:37.159 INFO:tasks.ceph.mon.e:Stopped 2024-09-16T15:07:37.159 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-16T15:07:37.159 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 10 monclient: get_monmap_and_config 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 10 monclient: build_initial_monmap 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 10 monclient: monmap: 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:37.295107+0000 2024-09-16T15:07:37.295 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:37.295107+0000 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:37.296 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 1 Processor -- start 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 1 -- start start 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.294+0000 7f5b81b53640 10 monclient: init 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient: _add_conns ranks=[2,10,5,8,6,7,3,4,0,1,9] 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-b con 0x7f5b7c136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b7c09bfe0 0x7f5b7c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-f con 0x7f5b7c09bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b7c136c60 0x7f5b7c1370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-h con 0x7f5b7c136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c09b600 con 0x7f5b7c09bfe0 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0a5740 con 0x7f5b7c136030 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0a52c0 con 0x7f5b7c136c60 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b81b53640 10 monclient(hunting): authenticate will time out at 1193.884443s 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:32894/0 (socket says 172.21.15.134:32894) 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 -- 172.21.15.134:0/1509734018 learned_addr learned my addr 172.21.15.134:0/1509734018 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b7c136030 auth_method 0 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b7c136c60 0x7f5b7c1370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 1 --2- 172.21.15.134:0/1509734018 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c09bfe0 0x7f5b7c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): get_auth_request con 0x7f5b7c136c60 auth_method 0 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): get_auth_request con 0x7f5b7c09bfe0 auth_method 0 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_done global_id 44543 payload 274 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 -- 172.21.15.134:0/1509734018 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b7c136c60 msgr2=0x7f5b7c1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b7c136c60 0x7f5b7c1370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 -- 172.21.15.134:0/1509734018 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c09bfe0 msgr2=0x7f5b7c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/1509734018 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c09bfe0 0x7f5b7c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 monclient: found mon.noname-b 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:37.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.295+0000 7f5b7b7fe640 1 -- 172.21.15.134:0/1509734018 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c073250 con 0x7f5b7c136030 2024-09-16T15:07:37.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7b7fe640 1 -- 172.21.15.134:0/1509734018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 msgr2=0x7f5b7c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009870 tx=0x7f5b6c02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7c8acc8d0818a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 1 -- 172.21.15.134:0/1509734018 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b6c02ae00 con 0x7f5b7c136030 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: dump: 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.308 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 1 -- 172.21.15.134:0/1509734018 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0307b0 con 0x7f5b7c136030 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b81b53640 5 monclient: authenticate success, global_id 44543 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b81b53640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:37.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b81b53640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b81b53640 4 set_mon_vals no callback set 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 1 -- 172.21.15.134:0/1509734018 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b6c030d90 con 0x7f5b7c136030 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.298+0000 7f5b7a7fc640 10 monclient: dump: 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.310 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 10 monclient: shutdown 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 -- 172.21.15.134:0/1509734018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 msgr2=0x7f5b7c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009870 tx=0x7f5b6c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 -- 172.21.15.134:0/1509734018 shutdown_connections 2024-09-16T15:07:37.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b7c136c60 0x7f5b7c1370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 --2- 172.21.15.134:0/1509734018 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c09bfe0 0x7f5b7c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 --2- 172.21.15.134:0/1509734018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b7c136030 0x7f5b7c136410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 -- 172.21.15.134:0/1509734018 >> 172.21.15.134:0/1509734018 conn(0x7f5b7c0a9ea0 msgr2=0x7f5b7c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 -- 172.21.15.134:0/1509734018 shutdown_connections 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 1 -- 172.21.15.134:0/1509734018 wait complete. 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 10 monclient: build_initial_monmap 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.299+0000 7f5b81b53640 10 monclient: monmap: 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.312 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 Processor -- start 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 -- start start 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient: init 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:37.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient: _add_conns ranks=[7,4,8,0,6,5,3,1,2] 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 0x7f5b7c134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-i con 0x7f5b7c09bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-c con 0x7f5b7c136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 0x7f5b7c11fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): picked mon.noname-e con 0x7f5b7c136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1cc2f0 con 0x7f5b7c136030 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c121eb0 con 0x7f5b7c09bfe0 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c0a7e30 con 0x7f5b7c136c60 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:37.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b81b53640 10 monclient(hunting): authenticate will time out at 1193.889939s 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b7bfff640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 msgr2=0x7f5b7c11fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.300+0000 7f5b7bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 0x7f5b7c11fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 0x7f5b7c134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33008/0 (socket says 172.21.15.134:33008) 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 -- 172.21.15.134:0/690771374 learned_addr learned my addr 172.21.15.134:0/690771374 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): get_auth_request con 0x7f5b7c136030 auth_method 0 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): get_auth_request con 0x7f5b7c09bfe0 auth_method 0 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:37.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient(hunting): handle_auth_done global_id 5026 payload 824 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 -- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 msgr2=0x7f5b7c11fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 0x7f5b7c11fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 -- 172.21.15.134:0/690771374 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 msgr2=0x7f5b7c134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 0x7f5b7c134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient: _finish_hunting 0 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 monclient: found mon.noname-c 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:37.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7affd640 1 -- 172.21.15.134:0/690771374 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c0094d0 con 0x7f5b7c136030 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.301+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 0x7f5b7c134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b7affd640 1 -- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 msgr2=0x7f5b7c1352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b7affd640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c1358d0 tx=0x7f5b7000ee10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b70009670 con 0x7f5b7c136030 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: dump: 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: _finish_auth 0 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:37.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70013470 con 0x7f5b7c136030 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 5 monclient: authenticate success, global_id 5026 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 10 monclient: _renew_subs 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c1cb850 con 0x7f5b7c136030 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b70013a50 con 0x7f5b7c136030 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b63fff640 10 monclient: dump: 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 10 monclient: _renew_subs 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c120df0 con 0x7f5b7c136030 2024-09-16T15:07:37.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.303+0000 7f5b79ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5b70013bb0 con 0x7f5b7c136030 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b81b53640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b81b53640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b81b53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c09c400 con 0x7f5b7c136030 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b63fff640 1 --2- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 0x7f5b6803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.304+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f5b7004a410 con 0x7f5b7c136030 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.305+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 0x7f5b6803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:37.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.305+0000 7f5b7b7fe640 10 monclient: get_auth_request con 0x7f5b68038880 auth_method 0 2024-09-16T15:07:37.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.307+0000 7f5b7b7fe640 1 --2- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 0x7f5b6803ad40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0a68b0 tx=0x7f5b6c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:37.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.309+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5b700232f0 con 0x7f5b7c136030 2024-09-16T15:07:37.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.309+0000 7f5b63fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:37.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.309+0000 7f5b63fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:37.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b81b53640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:37.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b81b53640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b81b53640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5b7c121ff0 con 0x7f5b7c136030 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b63fff640 1 -- 172.21.15.134:0/690771374 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f5b70017020 con 0x7f5b7c136030 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b63fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.531+0000 7f5b63fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:37.533 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 msgr2=0x7f5b6803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 0x7f5b6803ad40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0a68b0 tx=0x7f5b6c02e000 comp rx=0 tx=0).stop 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 10 monclient: shutdown 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 msgr2=0x7f5b7c1352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:37.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c1358d0 tx=0x7f5b7000ee10 comp rx=0 tx=0).stop 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 shutdown_connections 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b68038880 0x7f5b6803ad40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b7c136c60 0x7f5b7c11fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b7c136030 0x7f5b7c1352d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 --2- 172.21.15.134:0/690771374 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5b7c09bfe0 0x7f5b7c134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.533+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 >> 172.21.15.134:0/690771374 conn(0x7f5b7c0a9ea0 msgr2=0x7f5b7c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.534+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 shutdown_connections 2024-09-16T15:07:37.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:37.534+0000 7f5b81b53640 1 -- 172.21.15.134:0/690771374 wait complete. 2024-09-16T15:07:40.546 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.653+0000 7fc4a0333640 10 monclient: get_monmap_and_config 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.653+0000 7fc4a0333640 10 monclient: build_initial_monmap 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.653+0000 7fc4a0333640 10 monclient: monmap: 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:40.654748+0000 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:40.654748+0000 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:40.655 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:40.656 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.654+0000 7fc4a0333640 1 Processor -- start 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.654+0000 7fc4a0333640 1 -- start start 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.654+0000 7fc4a0333640 10 monclient: init 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient: _add_conns ranks=[6,9,3,2,1,4,8,0,7,10,5] 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-d con 0x7fc498136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 0x7fc49809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-a con 0x7fc49809bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc498136c60 0x7fc4981370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-g con 0x7fc498136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49809b600 con 0x7fc49809bfe0 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980a5740 con 0x7fc498136c60 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980a52c0 con 0x7fc498136030 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc4a0333640 10 monclient(hunting): authenticate will time out at 1197.244218s 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49d8ce640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 msgr2=0x7fc49809c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49d8ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 0x7fc49809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57884/0 (socket says 172.21.15.134:57884) 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 -- 172.21.15.134:0/2908627066 learned_addr learned my addr 172.21.15.134:0/2908627066 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): get_auth_request con 0x7fc498136030 auth_method 0 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:40.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e8d0640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc498136c60 0x7fc4981370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e8d0640 10 monclient(hunting): get_auth_request con 0x7fc498136c60 auth_method 0 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e8d0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_done global_id 34665 payload 274 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 -- 172.21.15.134:0/2908627066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc498136c60 msgr2=0x7fc4981370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc498136c60 0x7fc4981370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 -- 172.21.15.134:0/2908627066 >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 msgr2=0x7fc49809c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:40.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/2908627066 >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 0x7fc49809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient: _finish_hunting 0 2024-09-16T15:07:40.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 monclient: found mon.noname-d 2024-09-16T15:07:40.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:40.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:40.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.655+0000 7fc49e0cf640 1 -- 172.21.15.134:0/2908627066 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498073250 con 0x7fc498136030 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49e0cf640 1 -- 172.21.15.134:0/2908627066 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 msgr2=0x7fc498136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc494009870 tx=0x7fc49402a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=778acf1e9621d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 1 -- 172.21.15.134:0/2908627066 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc49402ae30 con 0x7fc498136030 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:40.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: dump: 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:40.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: _finish_auth 0 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 1 -- 172.21.15.134:0/2908627066 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4940307b0 con 0x7fc498136030 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc4a0333640 5 monclient: authenticate success, global_id 34665 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc4a0333640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 1 -- 172.21.15.134:0/2908627066 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc494030d90 con 0x7fc498136030 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:40.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc4a0333640 4 set_mon_vals no callback set 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.658+0000 7fc49d0cd640 10 monclient: dump: 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:40.671 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 10 monclient: shutdown 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 -- 172.21.15.134:0/2908627066 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 msgr2=0x7fc498136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc494009870 tx=0x7fc49402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 -- 172.21.15.134:0/2908627066 shutdown_connections 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc498136c60 0x7fc4981370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 --2- 172.21.15.134:0/2908627066 >> v2:172.21.15.134:3300/0 conn(0x7fc49809bfe0 0x7fc49809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 --2- 172.21.15.134:0/2908627066 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc498136030 0x7fc498136410 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 -- 172.21.15.134:0/2908627066 >> 172.21.15.134:0/2908627066 conn(0x7fc4980a9ea0 msgr2=0x7fc4980a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 -- 172.21.15.134:0/2908627066 shutdown_connections 2024-09-16T15:07:40.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 1 -- 172.21.15.134:0/2908627066 wait complete. 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 10 monclient: build_initial_monmap 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.659+0000 7fc4a0333640 10 monclient: monmap: 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:40.673 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 Processor -- start 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 -- start start 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient: init 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient: _add_conns ranks=[1,0,7,8,4,3,2,5,6] 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-f con 0x7fc49809bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 0x7fc498134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-a con 0x7fc498136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 0x7fc498135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): picked mon.noname-i con 0x7fc498136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4981cb8c0 con 0x7fc498136030 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498122290 con 0x7fc49809bfe0 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980a0330 con 0x7fc498136c60 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc4a0333640 10 monclient(hunting): authenticate will time out at 1197.249847s 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc49d8ce640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 msgr2=0x7fc498134e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.660+0000 7fc49d8ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 0x7fc498134e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 0x7fc498135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:40.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:41418/0 (socket says 172.21.15.134:41418) 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 1 -- 172.21.15.134:0/1620584765 learned_addr learned my addr 172.21.15.134:0/1620584765 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): get_auth_request con 0x7fc49809bfe0 auth_method 0 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): get_auth_request con 0x7fc498136c60 auth_method 0 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:40.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient(hunting): handle_auth_done global_id 54214 payload 824 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 -- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 msgr2=0x7fc4981becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 -- 172.21.15.134:0/1620584765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 msgr2=0x7fc498134e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 0x7fc498134e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient: _finish_hunting 0 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 monclient: found mon.noname-i 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e8d0640 1 -- 172.21.15.134:0/1620584765 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940094d0 con 0x7fc498136c60 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.661+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc49e8d0640 1 -- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 msgr2=0x7fc498135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:40.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc49e8d0640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 0x7fc498135340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc490007cf0 tx=0x7fc49000e4b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc490018070 con 0x7fc498136c60 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: dump: 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:40.678 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: _finish_auth 0 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490014890 con 0x7fc498136c60 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc4a0333640 5 monclient: authenticate success, global_id 54214 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc490014e70 con 0x7fc498136c60 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc4a0333640 10 monclient: _renew_subs 2024-09-16T15:07:40.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc4a0333640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49809fe90 con 0x7fc498136c60 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc48effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc48effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.663+0000 7fc49c8cc640 4 set_mon_vals no callback set 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc48effd640 10 monclient: dump: 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:40.680 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 10 monclient: _renew_subs 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc498121f20 con 0x7fc498136c60 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.664+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc498122400 con 0x7fc498136c60 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.665+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc490014a00 con 0x7fc498136c60 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.665+0000 7fc48effd640 1 --2- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 0x7fc47c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.665+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc49004a0f0 con 0x7fc498136c60 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.665+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 0x7fc47c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.665+0000 7fc49e0cf640 10 monclient: get_auth_request con 0x7fc47c0388b0 auth_method 0 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.667+0000 7fc49e0cf640 1 --2- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 0x7fc47c03ad70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc494035040 tx=0x7fc49402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.670+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc490014cb0 con 0x7fc498136c60 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.670+0000 7fc48effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.670+0000 7fc48effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:40.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.893+0000 7fc4a0333640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.893+0000 7fc4a0333640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.893+0000 7fc4a0333640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.893+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4981bf8e0 con 0x7fc498136c60 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.894+0000 7fc48effd640 1 -- 172.21.15.134:0/1620584765 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fc4900122e0 con 0x7fc498136c60 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.894+0000 7fc48effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:40.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.894+0000 7fc48effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:40.896 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 msgr2=0x7fc47c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 0x7fc47c03ad70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc494035040 tx=0x7fc49402e000 comp rx=0 tx=0).stop 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 10 monclient: shutdown 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 msgr2=0x7fc498135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 0x7fc498135340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc490007cf0 tx=0x7fc49000e4b0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 shutdown_connections 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> v2:172.21.15.162:6800/980547263 conn(0x7fc47c0388b0 0x7fc47c03ad70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc498136c60 0x7fc498135340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc498136030 0x7fc498134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 --2- 172.21.15.134:0/1620584765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc49809bfe0 0x7fc4981becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:40.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 >> 172.21.15.134:0/1620584765 conn(0x7fc4980a9ea0 msgr2=0x7fc49806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:40.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 shutdown_connections 2024-09-16T15:07:40.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:40.896+0000 7fc4a0333640 1 -- 172.21.15.134:0/1620584765 wait complete. 2024-09-16T15:07:43.909 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.020+0000 7fc4fdfbc640 10 monclient: get_monmap_and_config 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.020+0000 7fc4fdfbc640 10 monclient: build_initial_monmap 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.020+0000 7fc4fdfbc640 10 monclient: monmap: 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:44.021565+0000 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:44.021565+0000 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:44.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:44.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 1 Processor -- start 2024-09-16T15:07:44.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 1 -- start start 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 10 monclient: init 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 10 monclient: _add_conns ranks=[7,6,2,10,1,9,3,5,4,8,0] 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.021+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc4f80664e0 0x7fc4f809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-i con 0x7fc4f80664e0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 0x7fc4f809b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-d con 0x7fc4f809af50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-b con 0x7fc4f8137a00 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f809d820 con 0x7fc4f8137a00 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f809b870 con 0x7fc4f809af50 2024-09-16T15:07:44.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f809dde0 con 0x7fc4f80664e0 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4fdfbc640 10 monclient(hunting): authenticate will time out at 1200.611084s 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 0x7fc4f809b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:32908/0 (socket says 172.21.15.134:32908) 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2953190275 learned_addr learned my addr 172.21.15.134:0/2953190275 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): get_auth_request con 0x7fc4f8137a00 auth_method 0 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): get_auth_request con 0x7fc4f809af50 auth_method 0 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:44.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f77fe640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc4f80664e0 0x7fc4f809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f77fe640 10 monclient(hunting): get_auth_request con 0x7fc4f80664e0 auth_method 0 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_done global_id 44552 payload 274 2024-09-16T15:07:44.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2953190275 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc4f80664e0 msgr2=0x7fc4f809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc4f80664e0 0x7fc4f809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 msgr2=0x7fc4f809b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 0x7fc4f809b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 monclient: found mon.noname-b 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2953190275 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8073250 con 0x7fc4f8137a00 2024-09-16T15:07:44.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.022+0000 7fc4f6ffd640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 0x7fc4f809b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 msgr2=0x7fc4f8139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009900 tx=0x7fc4f002b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cb72c5108a580e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 1 -- 172.21.15.134:0/2953190275 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f00317b0 con 0x7fc4f8137a00 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: dump: 2024-09-16T15:07:44.035 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:44.036 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4fdfbc640 5 monclient: authenticate success, global_id 44552 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 1 -- 172.21.15.134:0/2953190275 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f0031930 con 0x7fc4f8137a00 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4fdfbc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 1 -- 172.21.15.134:0/2953190275 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f00323a0 con 0x7fc4f8137a00 2024-09-16T15:07:44.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4fdfbc640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4fdfbc640 4 set_mon_vals no callback set 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.025+0000 7fc4f67fc640 10 monclient: dump: 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:44.038 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 10 monclient: shutdown 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 msgr2=0x7fc4f8139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009900 tx=0x7fc4f002b1b0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2953190275 shutdown_connections 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4f8137a00 0x7fc4f8139df0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc4f809af50 0x7fc4f809b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2953190275 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc4f80664e0 0x7fc4f809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2953190275 >> 172.21.15.134:0/2953190275 conn(0x7fc4f80a5210 msgr2=0x7fc4f80a5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:44.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2953190275 shutdown_connections 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2953190275 wait complete. 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.026+0000 7fc4fdfbc640 10 monclient: build_initial_monmap 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.027+0000 7fc4fdfbc640 10 monclient: monmap: 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:44.040 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.027+0000 7fc4fdfbc640 1 Processor -- start 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.027+0000 7fc4fdfbc640 1 -- start start 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.027+0000 7fc4fdfbc640 10 monclient: init 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient: _add_conns ranks=[8,1,4,0,2,3,7,6,5] 2024-09-16T15:07:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 0x7fc4f81c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-e con 0x7fc4f80664e0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4f809af50 0x7fc4f8135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-f con 0x7fc4f809af50 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): picked mon.noname-c con 0x7fc4f8137a00 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81c73c0 con 0x7fc4f809af50 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f81227c0 con 0x7fc4f8137a00 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8071070 con 0x7fc4f80664e0 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f77fe640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 msgr2=0x7fc4f81c6e80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4fdfbc640 10 monclient(hunting): authenticate will time out at 1200.617124s 2024-09-16T15:07:44.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f77fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 0x7fc4f81c6e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33024/0 (socket says 172.21.15.134:33024) 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2003506122 learned_addr learned my addr 172.21.15.134:0/2003506122 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): get_auth_request con 0x7fc4f8137a00 auth_method 0 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4f809af50 0x7fc4f8135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): get_auth_request con 0x7fc4f809af50 auth_method 0 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient(hunting): handle_auth_done global_id 5035 payload 824 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 msgr2=0x7fc4f81c6e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 0x7fc4f81c6e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2003506122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4f809af50 msgr2=0x7fc4f8135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4f809af50 0x7fc4f8135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 monclient: found mon.noname-c 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:44.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.028+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2003506122 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec009400 con 0x7fc4f8137a00 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4f7fff640 1 -- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 msgr2=0x7fc4f8136350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4f7fff640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009820 tx=0x7fc4f00046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f0004250 con 0x7fc4f8137a00 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 10 monclient: dump: 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:44.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 10 monclient: _finish_auth 0 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f00043b0 con 0x7fc4f8137a00 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4fdfbc640 5 monclient: authenticate success, global_id 5035 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4fdfbc640 10 monclient: _renew_subs 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4fdfbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:44.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.030+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f00094d0 con 0x7fc4f8137a00 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc4f0004520 con 0x7fc4f8137a00 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 10 monclient: dump: 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:44.047 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4fdfbc640 10 monclient: _renew_subs 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4fdfbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f809efb0 con 0x7fc4f8137a00 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4f5ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.031+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc4f003a070 con 0x7fc4f8137a00 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4fdfbc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:44.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4fdfbc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4fdfbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f809d540 con 0x7fc4f8137a00 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4dffff640 1 --2- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 0x7fc4e403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc4f006a950 con 0x7fc4f8137a00 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4f77fe640 1 --2- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 0x7fc4e403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.032+0000 7fc4f77fe640 10 monclient: get_auth_request con 0x7fc4e4038880 auth_method 0 2024-09-16T15:07:44.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.035+0000 7fc4f77fe640 1 --2- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 0x7fc4e403ad40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009590 tx=0x7fc4e8009450 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:44.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.036+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc4f003d7e0 con 0x7fc4f8137a00 2024-09-16T15:07:44.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.036+0000 7fc4dffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:44.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.036+0000 7fc4dffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.261+0000 7fc4fdfbc640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.261+0000 7fc4fdfbc640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.261+0000 7fc4fdfbc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.261+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4f809c0b0 con 0x7fc4f8137a00 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.262+0000 7fc4dffff640 1 -- 172.21.15.134:0/2003506122 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fc4f0038020 con 0x7fc4f8137a00 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.262+0000 7fc4dffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.262+0000 7fc4dffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:44.263 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:44.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 msgr2=0x7fc4e403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:44.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 0x7fc4e403ad40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009590 tx=0x7fc4e8009450 comp rx=0 tx=0).stop 2024-09-16T15:07:44.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 10 monclient: shutdown 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 msgr2=0x7fc4f8136350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009820 tx=0x7fc4f00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 shutdown_connections 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> v2:172.21.15.162:6800/980547263 conn(0x7fc4e4038880 0x7fc4e403ad40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc4f8137a00 0x7fc4f8136350 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4f809af50 0x7fc4f8135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 --2- 172.21.15.134:0/2003506122 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc4f80664e0 0x7fc4f81c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.263+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 >> 172.21.15.134:0/2003506122 conn(0x7fc4f80a5210 msgr2=0x7fc4f8071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.264+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 shutdown_connections 2024-09-16T15:07:44.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:44.264+0000 7fc4fdfbc640 1 -- 172.21.15.134:0/2003506122 wait complete. 2024-09-16T15:07:47.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:07:47.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.382+0000 7f298f9f4640 10 monclient: get_monmap_and_config 2024-09-16T15:07:47.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.382+0000 7f298f9f4640 10 monclient: build_initial_monmap 2024-09-16T15:07:47.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.382+0000 7f298f9f4640 10 monclient: monmap: 2024-09-16T15:07:47.385 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:47.385 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:47.384853+0000 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:47.384853+0000 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:47.386 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:47.387 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:47.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.383+0000 7f298f9f4640 1 Processor -- start 2024-09-16T15:07:47.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.383+0000 7f298f9f4640 1 -- start start 2024-09-16T15:07:47.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.383+0000 7f298f9f4640 10 monclient: init 2024-09-16T15:07:47.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient: _add_conns ranks=[10,6,7,3,5,8,0,4,2,9,1] 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-f con 0x7f2988136030 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-d con 0x7f298809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-i con 0x7f2988136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298809b600 con 0x7f2988136030 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f29880a5740 con 0x7f298809bfe0 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f29880a52c0 con 0x7f2988136c60 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:47.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298f9f4640 10 monclient(hunting): authenticate will time out at 1203.974315s 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:46416/0 (socket says 172.21.15.134:46416) 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 learned_addr learned my addr 172.21.15.134:0/3578592508 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 10 monclient(hunting): get_auth_request con 0x7f298809bfe0 auth_method 0 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2988136030 auth_method 0 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 10 monclient(hunting): get_auth_request con 0x7f2988136c60 auth_method 0 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:47.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.384+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 10 monclient(hunting): get_auth_request con 0x7f298809bfe0 auth_method 0 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 10 monclient(hunting): get_auth_request con 0x7f2988136c60 auth_method 0 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2988136030 auth_method 0 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:47.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:47.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.585+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:47.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 10 monclient(hunting): get_auth_request con 0x7f298809bfe0 auth_method 0 2024-09-16T15:07:47.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:47.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:47.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2988136030 auth_method 0 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 10 monclient(hunting): get_auth_request con 0x7f2988136c60 auth_method 0 2024-09-16T15:07:47.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:47.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:47.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:47.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:47.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:47.986+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 10 monclient(hunting): get_auth_request con 0x7f298809bfe0 auth_method 0 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:48.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 10 monclient(hunting): get_auth_request con 0x7f2988136c60 auth_method 0 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2988136030 auth_method 0 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:07:48.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:48.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:48.787+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): tick 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 monclient(hunting): continuing hunt 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 msgr2=0x7f29881370e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2988136c60 0x7f29881370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 msgr2=0x7f298809c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f298809bfe0 0x7f298809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 msgr2=0x7f2988136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.162:3300/0 conn(0x7f2988136030 0x7f2988136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient: _add_conns ranks=[3,1,9,2,4,5,8,0] 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): picked mon.noname-g con 0x7f2970003810 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2970006220 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f2970009f30 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2988073250 con 0x7f2970006220 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298809b600 con 0x7f2970009f30 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 1 -- 172.21.15.134:0/3578592508 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880a5740 con 0x7f2970003810 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f2986ffd640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 msgr2=0x7f297000c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2970003810 auth_method 0 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.385+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.387+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> v1:172.21.15.162:6789/0 conn(0x7f2970006220 legacy=0x7f2970008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.387+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:07:50.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 msgr2=0x7f297000c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:50.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:50.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:50.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2970003810 auth_method 0 2024-09-16T15:07:50.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:50.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:50.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:50.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:50.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:50.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:50.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.586+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:50.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 msgr2=0x7f297000c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:50.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2970003810 auth_method 0 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:50.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:50.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:50.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:50.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:50.987+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:07:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 msgr2=0x7f297000c3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:51.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2970003810 auth_method 0 2024-09-16T15:07:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:51.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:51.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:51.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:51.788+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663249394 0 0) 0x7f2988073250 con 0x7f2970006220 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 10 monclient(hunting): _negotiate 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 10 monclient(hunting): my global_id is 44470 2024-09-16T15:07:52.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.275+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003520 con 0x7f2970006220 2024-09-16T15:07:52.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300178346 0 0) 0x7f2974003520 con 0x7f2970006220 2024-09-16T15:07:52.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 msgr2=0x7f2970005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:52.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2970003810 0x7f2970005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 msgr2=0x7f297000c3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 --2- 172.21.15.134:0/3578592508 >> v2:172.21.15.134:3300/0 conn(0x7f2970009f30 0x7f297000c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297000dbb0 con 0x7f2970006220 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 reap_dead start 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _finish_auth 0 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _check_auth_tickets 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f297c00caf0 con 0x7f2970006220 2024-09-16T15:07:52.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: dump: 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 >> v1:172.21.15.162:6789/0 conn(0x7f2970006220 legacy=0x7f2970008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient: _add_conns ranks=[5,0,8,2,3,7,1,4,6] 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): picked mon.h con 0x7f29740041c0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2974005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): picked mon.a con 0x7f2974004dd0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 0x7f297400c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): picked mon.e con 0x7f2974009c90 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 msgr2=0x7f297400c120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298809b600 con 0x7f2974004dd0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 msgr2=0x7f2974005250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29880a5740 con 0x7f29740041c0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f297400ca30 con 0x7f2974009c90 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 0x7f297400c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f2987fff640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2974005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f29740041c0 auth_method 0 2024-09-16T15:07:52.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 msgr2=0x7f29740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 msgr2=0x7f29740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:07:52.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.277+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298df91640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 msgr2=0x7f297400c120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:52.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298cf8f640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 msgr2=0x7f2974005250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:52.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298cf8f640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2974005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:52.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298df91640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 0x7f297400c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:07:52.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f29740041c0 auth_method 0 2024-09-16T15:07:52.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:07:52.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:07:52.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient(hunting): handle_auth_done global_id 44470 payload 293 2024-09-16T15:07:52.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 msgr2=0x7f297400c120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:52.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 0x7f297400c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 msgr2=0x7f2974005250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:52.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2974005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient: _finish_hunting 0 2024-09-16T15:07:52.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 monclient: found mon.h 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974003520 con 0x7f29740041c0 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient: _finish_auth 0 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 10 monclient: _check_auth_tickets 2024-09-16T15:07:52.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298d790640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:52.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298f9f4640 5 monclient: authenticate success, global_id 44470 2024-09-16T15:07:52.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.478+0000 7f298f9f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:52.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.481+0000 7f298d790640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 msgr2=0x7f29740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.481+0000 7f298d790640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2978009f80 tx=0x7f29740057f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4bedbb9119b4bf65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2978038070 con 0x7f29740041c0 2024-09-16T15:07:52.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:52.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: dump: 2024-09-16T15:07:52.500 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.501 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.502 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f298f9f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800f040 con 0x7f29740041c0 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 1 -- 172.21.15.134:0/3578592508 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2978034470 con 0x7f29740041c0 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f2987fff640 10 monclient: dump: 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.503 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f298f9f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.482+0000 7f298f9f4640 4 set_mon_vals no callback set 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 10 monclient: shutdown 2024-09-16T15:07:52.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 -- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 msgr2=0x7f29740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2978009f80 tx=0x7f29740057f0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 -- 172.21.15.134:0/3578592508 shutdown_connections 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2974009c90 0x7f297400c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2974005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 --2- 172.21.15.134:0/3578592508 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f29740041c0 0x7f29740047c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 -- 172.21.15.134:0/3578592508 >> 172.21.15.134:0/3578592508 conn(0x7f29880a9ea0 msgr2=0x7f29880a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 -- 172.21.15.134:0/3578592508 shutdown_connections 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.483+0000 7f298f9f4640 1 -- 172.21.15.134:0/3578592508 wait complete. 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient: build_initial_monmap 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient: monmap: 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:52.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 Processor -- start 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 -- start start 2024-09-16T15:07:52.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient: init 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient: _add_conns ranks=[7,2,0,8,1,4,6,5,3] 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2970006220 0x7f29881cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-i con 0x7f2970006220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-b con 0x7f29740041c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2988121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): picked mon.noname-a con 0x7f2974004dd0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988123870 con 0x7f2974004dd0 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2988123f30 con 0x7f29740041c0 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2988134e60 con 0x7f2970006220 2024-09-16T15:07:52.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.484+0000 7f298f9f4640 10 monclient(hunting): authenticate will time out at 1209.073982s 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298df91640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 msgr2=0x7f2988121c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298df91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2988121c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45368/0 (socket says 172.21.15.134:45368) 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 -- 172.21.15.134:0/4224593069 learned_addr learned my addr 172.21.15.134:0/4224593069 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): get_auth_request con 0x7f29740041c0 auth_method 0 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2970006220 0x7f29881cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): get_auth_request con 0x7f2970006220 auth_method 0 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298d790640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient(hunting): handle_auth_done global_id 44561 payload 824 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 -- 172.21.15.134:0/4224593069 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2970006220 msgr2=0x7f29881cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2970006220 0x7f29881cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 -- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 msgr2=0x7f2988121c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:52.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2988121c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient: _finish_hunting 0 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 monclient: found mon.noname-b 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.485+0000 7f298cf8f640 1 -- 172.21.15.134:0/4224593069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988125100 con 0x7f29740041c0 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298cf8f640 1 -- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 msgr2=0x7f2988121730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298cf8f640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f297c013930 tx=0x7f297c013df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f297c011c50 con 0x7f29740041c0 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: dump: 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.510 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: _finish_auth 0 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c006e90 con 0x7f29740041c0 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f297c011dd0 con 0x7f29740041c0 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298f9f4640 5 monclient: authenticate success, global_id 44561 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298f9f4640 10 monclient: _renew_subs 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298f9f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29881253e0 con 0x7f29740041c0 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f2985ffb640 10 monclient: dump: 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.512 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.487+0000 7f29877fe640 4 set_mon_vals no callback set 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.488+0000 7f298f9f4640 10 monclient: _renew_subs 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.488+0000 7f298f9f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.488+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29881cb9b0 con 0x7f29740041c0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f297c004260 con 0x7f29740041c0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298f9f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298f9f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298f9f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298809c400 con 0x7f29740041c0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f2985ffb640 1 --2- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 0x7f297003bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(322..322 src has 1..322) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f297c000e40 con 0x7f29740041c0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298d790640 1 --2- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 0x7f297003bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.489+0000 7f298d790640 10 monclient: get_auth_request con 0x7f2970039a60 auth_method 0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.492+0000 7f298d790640 1 --2- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 0x7f297003bf20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2978035000 tx=0x7f297803f040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.494+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f297c0147c0 con 0x7f29740041c0 2024-09-16T15:07:52.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.494+0000 7f2985ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:52.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.494+0000 7f2985ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f298f9f4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f298f9f4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f298f9f4640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2988072070 con 0x7f29740041c0 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f2985ffb640 1 -- 172.21.15.134:0/4224593069 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f297c0147c0 con 0x7f29740041c0 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f2985ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.717+0000 7f2985ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:52.719 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":62,"quorum":[1,2,3,4,5,6,7],"quorum_names":["f","b","g","c","h","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 msgr2=0x7f297003bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 0x7f297003bf20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2978035000 tx=0x7f297803f040 comp rx=0 tx=0).stop 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 10 monclient: shutdown 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 msgr2=0x7f2988121730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f297c013930 tx=0x7f297c013df0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 shutdown_connections 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> v2:172.21.15.162:6800/980547263 conn(0x7f2970039a60 0x7f297003bf20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2974004dd0 0x7f2988121c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29740041c0 0x7f2988121730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 --2- 172.21.15.134:0/4224593069 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2970006220 0x7f29881cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 >> 172.21.15.134:0/4224593069 conn(0x7f29880a9ea0 msgr2=0x7f29880a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 shutdown_connections 2024-09-16T15:07:52.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.719+0000 7f298f9f4640 1 -- 172.21.15.134:0/4224593069 wait complete. 2024-09-16T15:07:52.731 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-16T15:07:52.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-16T15:07:52.732 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:07:52.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 10 monclient: get_monmap_and_config 2024-09-16T15:07:52.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 10 monclient: build_initial_monmap 2024-09-16T15:07:52.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 10 monclient: monmap: 2024-09-16T15:07:52.829 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:52.829 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:52.829054+0000 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:52.829054+0000 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:52.830 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:52.831 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:52.831 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 1 Processor -- start 2024-09-16T15:07:52.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 1 -- start start 2024-09-16T15:07:52.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.828+0000 7f8210fed640 10 monclient: init 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient: _add_conns ranks=[1,3,10,2,5,6,4,7,0,9,8] 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f820c136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 0x7f820c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-g con 0x7f820c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f820c13c370 0x7f820c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-f con 0x7f820c13c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c073250 con 0x7f820c136e40 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0a56e0 con 0x7f820c13c370 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c09abc0 con 0x7f820c137220 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:52.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8210fed640 10 monclient(hunting): authenticate will time out at 1209.418671s 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820a59c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f820c136e40 0x7f820c09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:50320/0 (socket says 172.21.15.134:50320) 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820a59c640 1 -- 172.21.15.134:0/253524120 learned_addr learned my addr 172.21.15.134:0/253524120 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 1 --2- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 0x7f820c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): get_auth_request con 0x7f820c137220 auth_method 0 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820ad9d640 1 --2- 172.21.15.134:0/253524120 >> v2:172.21.15.162:3300/0 conn(0x7f820c13c370 0x7f820c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820ad9d640 10 monclient(hunting): get_auth_request con 0x7f820c13c370 auth_method 0 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f820ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.829+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f820ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f820ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f820ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 10 monclient(hunting): handle_auth_done global_id 54219 payload 274 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 1 -- 172.21.15.134:0/253524120 >> v2:172.21.15.162:3300/0 conn(0x7f820c13c370 msgr2=0x7f820c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 1 --2- 172.21.15.134:0/253524120 >> v2:172.21.15.162:3300/0 conn(0x7f820c13c370 0x7f820c13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 1 -- 172.21.15.134:0/253524120 >> v1:172.21.15.162:6789/0 conn(0x7f820c136e40 legacy=0x7f820c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 1 monclient: found mon.noname-g 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:07:52.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.830+0000 7f8209d9b640 1 -- 172.21.15.134:0/253524120 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c09ba50 con 0x7f820c137220 2024-09-16T15:07:52.837 INFO:tasks.rados.rados.0.smithi162.stdout:1610: done (3 left) 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1611: done (2 left) 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1612: done (1 left) 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1613: done (0 left) 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1614: copy_from oid 40 from oid 38 current snap is 150 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1615: write oid 22 current snap is 150 2024-09-16T15:07:52.838 INFO:tasks.rados.rados.0.smithi162.stdout:1615: seq_num 588 ranges {682516=405593,1791765=569861,2830955=721120} 2024-09-16T15:07:52.844 INFO:tasks.rados.rados.0.smithi162.stdout:1615: writing smithi16234619-22 from 682516 to 1088109 tid 1 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f8209d9b640 1 -- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 msgr2=0x7f820c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f8209d9b640 1 --2- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 0x7f820c137600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82000097a0 tx=0x7f820002a010 comp rx=0 tx=0).ready entity=mon.3 client_cookie=a0a0a0fda0d2f3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 1 -- 172.21.15.134:0/253524120 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f820002ad30 con 0x7f820c137220 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: dump: 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.847 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.848 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: _finish_auth 0 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: _check_auth_tickets 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 1 -- 172.21.15.134:0/253524120 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000307b0 con 0x7f820c137220 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 1 -- 172.21.15.134:0/253524120 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8200030d90 con 0x7f820c137220 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f820959a640 10 monclient: dump: 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.849 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.850 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f8210fed640 5 monclient: authenticate success, global_id 54219 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f8210fed640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.834+0000 7f8210fed640 4 set_mon_vals no callback set 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 10 monclient: shutdown 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:52.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 -- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 msgr2=0x7f820c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 --2- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 0x7f820c137600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f82000097a0 tx=0x7f820002a010 comp rx=0 tx=0).stop 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 -- 172.21.15.134:0/253524120 shutdown_connections 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 --2- 172.21.15.134:0/253524120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f820c137220 0x7f820c137600 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 --2- 172.21.15.134:0/253524120 >> v2:172.21.15.162:3300/0 conn(0x7f820c13c370 0x7f820c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 -- 172.21.15.134:0/253524120 >> 172.21.15.134:0/253524120 conn(0x7f820c0a6110 msgr2=0x7f820c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 -- 172.21.15.134:0/253524120 shutdown_connections 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.835+0000 7f8210fed640 1 -- 172.21.15.134:0/253524120 wait complete. 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.836+0000 7f8210fed640 10 monclient: build_initial_monmap 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.836+0000 7f8210fed640 10 monclient: monmap: 2024-09-16T15:07:52.852 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:52.853 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 Processor -- start 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 -- start start 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient: init 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient: _add_conns ranks=[8,7,0,3,6,1,5,2,4] 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f820c136e40 0x7f820c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-e con 0x7f820c136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-i con 0x7f820c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c13c370 0x7f820c136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): picked mon.noname-a con 0x7f820c13c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c122430 con 0x7f820c13c370 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f820c122960 con 0x7f820c137220 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0a7af0 con 0x7f820c136e40 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8210fed640 10 monclient(hunting): authenticate will time out at 1209.426554s 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f820a59c640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f820c136e40 msgr2=0x7f820c0a2a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f820a59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f820c136e40 0x7f820c0a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f820ad9d640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c13c370 msgr2=0x7f820c136480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:52.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f820ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c13c370 0x7f820c136480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:36110/0 (socket says 172.21.15.134:36110) 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 1 -- 172.21.15.134:0/1762566639 learned_addr learned my addr 172.21.15.134:0/1762566639 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): get_auth_request con 0x7f820c137220 auth_method 0 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.837+0000 7f8209d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 10 monclient(hunting): handle_auth_done global_id 54241 payload 824 2024-09-16T15:07:52.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f820c136e40 msgr2=0x7f820c0a2a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f820c136e40 0x7f820c0a2a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c13c370 msgr2=0x7f820c136480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f820c13c370 0x7f820c136480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 monclient: found mon.noname-i 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.838+0000 7f8209d9b640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009400 con 0x7f820c137220 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8209d9b640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 msgr2=0x7f820c135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8209d9b640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82000096f0 tx=0x7f82000046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8209d9b640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82000096f0 tx=0x7f82000046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f8200003610 mon_map magic: 0 v1 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8200003610 con 0x7f820c137220 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: dump: 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:52.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200003790 con 0x7f820c137220 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f81faffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 5 monclient: authenticate success, global_id 54241 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 10 monclient: _renew_subs 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0a75a0 con 0x7f820c137220 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8208d99640 4 set_mon_vals no callback set 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 10 monclient: _renew_subs 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.840+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c122050 con 0x7f820c137220 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f8210fed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f8210fed640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 0x7f820c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c071b20 con 0x7f820c1cbd30 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 0x7f820c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient: get_auth_request con 0x7f820c1cbd30 auth_method 0 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8200003d70 con 0x7f820c137220 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 10 monclient: dump: 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.861 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.862 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8200030b30 con 0x7f820c137220 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.841+0000 7f81faffd640 1 --2- 172.21.15.134:0/1762566639 >> v2:172.21.15.162:6800/980547263 conn(0x7f81ec038880 0x7f81ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.842+0000 7f820a59c640 10 monclient(hunting): handle_auth_done global_id 44570 payload 824 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.842+0000 7f820a59c640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f820c061ae0 con 0x7f820c1cbd30 2024-09-16T15:07:52.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.842+0000 7f820ad9d640 1 --2- 172.21.15.134:0/1762566639 >> v2:172.21.15.162:6800/980547263 conn(0x7f81ec038880 0x7f81ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.842+0000 7f820ad9d640 10 monclient: get_auth_request con 0x7f81ec038880 auth_method 0 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f820a59c640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 msgr2=0x7f820c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f820a59c640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 0x7f820c02b260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004bf0 tx=0x7f81fc00cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81fc00ad00 con 0x7f820c1cbd30 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 10 monclient: dump: 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.864 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f820ad9d640 1 --2- 172.21.15.134:0/1762566639 >> v2:172.21.15.162:6800/980547263 conn(0x7f81ec038880 0x7f81ec03ad40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f820c13ad40 tx=0x7f8204009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f81fc00ae60 con 0x7f820c1cbd30 2024-09-16T15:07:52.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:52.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:52.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 msgr2=0x7f820c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 0x7f820c02b260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004bf0 tx=0x7f81fc00cf30 comp rx=0 tx=0).stop 2024-09-16T15:07:52.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 msgr2=0x7f820c02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:52.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.844+0000 7f81faffd640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cbd30 0x7f820c02b260 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004bf0 tx=0x7f81fc00cf30 comp rx=0 tx=0).stop 2024-09-16T15:07:52.868 INFO:tasks.rados.rados.0.smithi162.stdout:1615: writing smithi16234619-22 from 1791765 to 2361626 tid 2 2024-09-16T15:07:52.868 INFO:tasks.rados.rados.0.smithi162.stdout:1615: writing smithi16234619-22 from 2830955 to 3552075 tid 3 2024-09-16T15:07:52.868 INFO:tasks.rados.rados.0.smithi162.stdout:1616: rollback oid 19 current snap is 150 2024-09-16T15:07:52.868 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 148 2024-09-16T15:07:52.868 INFO:tasks.rados.rados.0.smithi162.stdout:1614: finishing copy_from to smithi16234619-40 2024-09-16T15:07:52.869 INFO:tasks.rados.rados.0.smithi162.stdout:1614: got expected ENOENT (src dne) 2024-09-16T15:07:52.869 INFO:tasks.rados.rados.0.smithi162.stdout:1614: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:07:52.869 INFO:tasks.rados.rados.0.smithi162.stdout:1614: done (2 left) 2024-09-16T15:07:52.869 INFO:tasks.rados.rados.0.smithi162.stdout:1617: snap_remove snap 134 2024-09-16T15:07:52.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f8210fed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:52.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f8210fed640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 0x7f820c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f8210fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c1227a0 con 0x7f820c1cc110 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 0x7f820c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient: get_auth_request con 0x7f820c1cc110 auth_method 0 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 10 monclient(hunting): handle_auth_done global_id 44579 payload 824 2024-09-16T15:07:52.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.851+0000 7f820a59c640 1 -- 172.21.15.134:0/1762566639 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f820c07a680 con 0x7f820c1cc110 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f820a59c640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 msgr2=0x7f820c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f820a59c640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 0x7f820c02b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004bf0 tx=0x7f81fc004d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:52.871 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:52.872 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:52.873 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:52.874 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:52.875 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:52.876 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.878 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:52.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:52.880 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.881 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:52.883 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:52.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:52.891 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:52.892 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.893 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:52.894 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:52.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f81fc023410 con 0x7f820c1cc110 2024-09-16T15:07:52.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f81faffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:52.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f81faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.854+0000 7f81faffd640 10 monclient: dump: 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:52.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f81fc023570 con 0x7f820c1cc110 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 msgr2=0x7f820c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 0x7f820c02b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004bf0 tx=0x7f81fc004d90 comp rx=0 tx=0).stop 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 msgr2=0x7f820c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.855+0000 7f81faffd640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f820c1cc110 0x7f820c02b7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 >> v2:172.21.15.162:6800/980547263 conn(0x7f81ec038880 msgr2=0x7f81ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 1 --2- 172.21.15.134:0/1762566639 >> v2:172.21.15.162:6800/980547263 conn(0x7f81ec038880 0x7f81ec03ad40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f820c13ad40 tx=0x7f8204009190 comp rx=0 tx=0).stop 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 10 monclient: shutdown 2024-09-16T15:07:52.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 msgr2=0x7f820c135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f8210fed640 1 --2- 172.21.15.134:0/1762566639 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f820c137220 0x7f820c135f40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82000096f0 tx=0x7f82000046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f820ad9d640 1 -- 172.21.15.134:0/1762566639 reap_dead start 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f820ad9d640 1 -- 172.21.15.134:0/1762566639 reap_dead start 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.856+0000 7f81faffd640 1 -- 172.21.15.134:0/1762566639 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (unknown 0 0 0) 0x7f820006b060 con 0x7f820c137220 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.857+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 shutdown_connections 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.857+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 >> 172.21.15.134:0/1762566639 conn(0x7f820c0a6110 msgr2=0x7f820c13a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.857+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 shutdown_connections 2024-09-16T15:07:52.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.857+0000 7f8210fed640 1 -- 172.21.15.134:0/1762566639 wait complete. 2024-09-16T15:07:52.899 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:52.900 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-16T15:07:52.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:07:52.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 10 monclient: get_monmap_and_config 2024-09-16T15:07:52.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 10 monclient: build_initial_monmap 2024-09-16T15:07:52.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 10 monclient: monmap: 2024-09-16T15:07:52.997 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:52.997 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:52.997040+0000 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:52.997040+0000 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:52.998 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:52.999 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 1 Processor -- start 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 1 -- start start 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.996+0000 7f9c32c6b640 10 monclient: init 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient: _add_conns ranks=[2,8,1,5,4,7,9,6,10,3,0] 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-b con 0x7f9c2c133f10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 0x7f9c2c134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-e con 0x7f9c2c134830 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9c2c071bd0 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c073250 con 0x7f9c2c071bd0 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0a3d50 con 0x7f9c2c133f10 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c06f630 con 0x7f9c2c134830 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c32c6b640 10 monclient(hunting): authenticate will time out at 1209.586314s 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c2bfff640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 msgr2=0x7f9c2c134cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c2bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 0x7f9c2c134cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45410/0 (socket says 172.21.15.134:45410) 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 -- 172.21.15.134:0/3129000059 learned_addr learned my addr 172.21.15.134:0/3129000059 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): get_auth_request con 0x7f9c2c133f10 auth_method 0 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient(hunting): handle_auth_done global_id 44588 payload 274 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 -- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 msgr2=0x7f9c2c134cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 --2- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 0x7f9c2c134cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 -- 172.21.15.134:0/3129000059 >> v1:172.21.15.162:6789/0 conn(0x7f9c2c071bd0 legacy=0x7f9c2c1351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 monclient: found mon.noname-b 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:52.997+0000 7f9c30a07640 1 -- 172.21.15.134:0/3129000059 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0a1e10 con 0x7f9c2c133f10 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c30a07640 1 -- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 msgr2=0x7f9c2c1342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c30a07640 1 --2- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c200098a0 tx=0x7f9c2002a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96443281d5918542 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 1 -- 172.21.15.134:0/3129000059 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c200307b0 con 0x7f9c2c133f10 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c32c6b640 5 monclient: authenticate success, global_id 44588 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c32c6b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 1 -- 172.21.15.134:0/3129000059 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c20030930 con 0x7f9c2c133f10 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 1 -- 172.21.15.134:0/3129000059 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c20030aa0 con 0x7f9c2c133f10 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c32c6b640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c32c6b640 4 set_mon_vals no callback set 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.000+0000 7f9c2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 10 monclient: shutdown 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 -- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 msgr2=0x7f9c2c1342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c200098a0 tx=0x7f9c2002a0e0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 -- 172.21.15.134:0/3129000059 shutdown_connections 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c134830 0x7f9c2c134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/3129000059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c2c133f10 0x7f9c2c1342f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 -- 172.21.15.134:0/3129000059 >> 172.21.15.134:0/3129000059 conn(0x7f9c2c0a7410 msgr2=0x7f9c2c0a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 -- 172.21.15.134:0/3129000059 shutdown_connections 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 1 -- 172.21.15.134:0/3129000059 wait complete. 2024-09-16T15:07:53.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.001+0000 7f9c32c6b640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient: monmap: 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.014 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 1 Processor -- start 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 1 -- start start 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient: init 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient: _add_conns ranks=[8,7,6,0,4,2,3,1,5] 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c071bd0 0x7f9c2c1cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-e con 0x7f9c2c071bd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9c2c133f10 0x7f9c2c1cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-i con 0x7f9c2c133f10 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.002+0000 7f9c32c6b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): picked mon.noname-d con 0x7f9c2c134830 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c126ed0 con 0x7f9c2c134830 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c30a07640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c071bd0 msgr2=0x7f9c2c1cbcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c127120 con 0x7f9c2c133f10 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0a6750 con 0x7f9c2c071bd0 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c30a07640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c071bd0 0x7f9c2c1cbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c32c6b640 10 monclient(hunting): authenticate will time out at 1209.592041s 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:46456/0 (socket says 172.21.15.134:46456) 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9c2c133f10 0x7f9c2c1cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 learned_addr learned my addr 172.21.15.134:0/1469158847 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): get_auth_request con 0x7f9c2c134830 auth_method 0 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): get_auth_request con 0x7f9c2c133f10 auth_method 0 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient(hunting): handle_auth_done global_id 34683 payload 824 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c071bd0 msgr2=0x7f9c2c1cbcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9c2c071bd0 0x7f9c2c1cbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9c2c133f10 msgr2=0x7f9c2c1cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9c2c133f10 0x7f9c2c1cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 monclient: found mon.noname-d 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.003+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200094d0 con 0x7f9c2c134830 2024-09-16T15:07:53.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 msgr2=0x7f9c2c123580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c31208640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c780 tx=0x7f9c2400cc40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c2400ce90 con 0x7f9c2c134830 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: dump: 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.024 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 5 monclient: authenticate success, global_id 34683 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24004960 con 0x7f9c2c134830 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 10 monclient: _renew_subs 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c09dff0 con 0x7f9c2c134830 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c24005030 con 0x7f9c2c134830 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c2affd640 4 set_mon_vals no callback set 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c297fa640 10 monclient: dump: 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.026 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 10 monclient: _renew_subs 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.005+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c127400 con 0x7f9c2c134830 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c32c6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9c24004b10 con 0x7f9c2c134830 2024-09-16T15:07:53.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 0x7f9c2c123ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1272b0 con 0x7f9c2c123ac0 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 0x7f9c2c123ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient: get_auth_request con 0x7f9c2c123ac0 auth_method 0 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c297fa640 1 --2- 172.21.15.134:0/1469158847 >> v2:172.21.15.162:6800/980547263 conn(0x7f9c10038880 0x7f9c1003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f9c2404a110 con 0x7f9c2c134830 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.006+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.007+0000 7f9c30a07640 10 monclient(hunting): handle_auth_done global_id 5044 payload 824 2024-09-16T15:07:53.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.007+0000 7f9c30a07640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c2c061ae0 con 0x7f9c2c123ac0 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.007+0000 7f9c2bfff640 1 --2- 172.21.15.134:0/1469158847 >> v2:172.21.15.162:6800/980547263 conn(0x7f9c10038880 0x7f9c1003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.007+0000 7f9c2bfff640 10 monclient: get_auth_request con 0x7f9c10038880 auth_method 0 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c30a07640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 msgr2=0x7f9c2c123ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c30a07640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 0x7f9c2c123ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9c20034000 tx=0x7f9c200046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c20030aa0 con 0x7f9c2c123ac0 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 10 monclient: dump: 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.029 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c2bfff640 1 --2- 172.21.15.134:0/1469158847 >> v2:172.21.15.162:6800/980547263 conn(0x7f9c10038880 0x7f9c1003ad40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0a81c0 tx=0x7f9c180090e0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9c20004460 con 0x7f9c2c123ac0 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 msgr2=0x7f9c2c123ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 0x7f9c2c123ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9c20034000 tx=0x7f9c200046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 msgr2=0x7f9c2c123ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.009+0000 7f9c297fa640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c123ac0 0x7f9c2c123ea0 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9c20034000 tx=0x7f9c200046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c32c6b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 0x7f9c2c09e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c32c6b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c06a3f0 con 0x7f9c2c1243e0 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 0x7f9c2c09e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient: get_auth_request con 0x7f9c2c1243e0 auth_method 0 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.015+0000 7f9c30a07640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.016+0000 7f9c30a07640 10 monclient(hunting): handle_auth_done global_id 5053 payload 824 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.016+0000 7f9c30a07640 1 -- 172.21.15.134:0/1469158847 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9c2c07a680 con 0x7f9c2c1243e0 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c30a07640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 msgr2=0x7f9c2c09e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c30a07640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 0x7f9c2c09e810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c2003f000 tx=0x7f9c20009240 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9c200482e0 con 0x7f9c2c1243e0 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 10 monclient: dump: 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.032 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.033 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9505 (secure 0 0 0) 0x7f9c20003aa0 con 0x7f9c2c1243e0 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 msgr2=0x7f9c2c09e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 0x7f9c2c09e810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c2003f000 tx=0x7f9c20009240 comp rx=0 tx=0).stop 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 msgr2=0x7f9c2c09e810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.018+0000 7f9c297fa640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9c2c1243e0 0x7f9c2c09e810 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9c2003f000 tx=0x7f9c20009240 comp rx=0 tx=0).stop 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.042 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.044 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.047 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.049 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.052 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.055 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.056 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 >> v2:172.21.15.162:6800/980547263 conn(0x7f9c10038880 msgr2=0x7f9c1003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/1469158847 >> v2:172.21.15.162:6800/980547263 conn(0x7f9c10038880 0x7f9c1003ad40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0a81c0 tx=0x7f9c180090e0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 10 monclient: shutdown 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 msgr2=0x7f9c2c123580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c31208640 1 -- 172.21.15.134:0/1469158847 reap_dead start 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c780 tx=0x7f9c2400cc40 comp rx=0 tx=0).stop 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 shutdown_connections 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 --2- 172.21.15.134:0/1469158847 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9c2c134830 0x7f9c2c123580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 >> 172.21.15.134:0/1469158847 conn(0x7f9c2c0a7410 msgr2=0x7f9c2c09d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 shutdown_connections 2024-09-16T15:07:53.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.019+0000 7f9c32c6b640 1 -- 172.21.15.134:0/1469158847 wait complete. 2024-09-16T15:07:53.058 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-16T15:07:53.059 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 10 monclient: get_monmap_and_config 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 10 monclient: monmap: 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:53.156018+0000 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:53.156018+0000 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.158 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.158 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:53.158 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:53.158 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 1 Processor -- start 2024-09-16T15:07:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 1 -- start start 2024-09-16T15:07:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.155+0000 7fdc390e5640 10 monclient: init 2024-09-16T15:07:53.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient: _add_conns ranks=[4,5,2,8,0,9,10,7,1,6,3] 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-c con 0x7fdc34069e50 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdc34066530 0x7fdc34066910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-h con 0x7fdc34066530 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-b con 0x7fdc3413b5a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3409e4a0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34066e50 con 0x7fdc34069e50 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3406a230 con 0x7fdc34066530 2024-09-16T15:07:53.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc390e5640 10 monclient(hunting): authenticate will time out at 1209.745136s 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:51408/0 (socket says 172.21.15.134:51408) 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45414/0 (socket says 172.21.15.134:45414) 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 1 -- 172.21.15.134:0/1501074693 learned_addr learned my addr 172.21.15.134:0/1501074693 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request con 0x7fdc34069e50 auth_method 0 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): get_auth_request con 0x7fdc3413b5a0 auth_method 0 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdc34066530 0x7fdc34066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): get_auth_request con 0x7fdc34066530 auth_method 0 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient(hunting): handle_auth_done global_id 44597 payload 274 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 -- 172.21.15.134:0/1501074693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdc34066530 msgr2=0x7fdc34066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdc34066530 0x7fdc34066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 -- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 msgr2=0x7fdc34065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 monclient: found mon.noname-b 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc3359e640 1 -- 172.21.15.134:0/1501074693 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34073250 con 0x7fdc3413b5a0 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.156+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:53.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc3359e640 1 -- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 msgr2=0x7fdc3413d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc3359e640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097d0 tx=0x7fdc2402a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4531c57e7a2beb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 1 -- 172.21.15.134:0/1501074693 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2402ae30 con 0x7fdc3413b5a0 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: dump: 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.175 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc390e5640 5 monclient: authenticate success, global_id 44597 2024-09-16T15:07:53.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 1 -- 172.21.15.134:0/1501074693 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc240307b0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc390e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc390e5640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc390e5640 4 set_mon_vals no callback set 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 1 -- 172.21.15.134:0/1501074693 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc24030d90 con 0x7fdc3413b5a0 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.159+0000 7fdc31d9b640 10 monclient: dump: 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.177 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.178 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 10 monclient: shutdown 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 -- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 msgr2=0x7fdc3413d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097d0 tx=0x7fdc2402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 -- 172.21.15.134:0/1501074693 shutdown_connections 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc3413d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdc34066530 0x7fdc34066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 --2- 172.21.15.134:0/1501074693 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdc34069e50 0x7fdc34065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 -- 172.21.15.134:0/1501074693 >> 172.21.15.134:0/1501074693 conn(0x7fdc340a6400 msgr2=0x7fdc340a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 -- 172.21.15.134:0/1501074693 shutdown_connections 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 1 -- 172.21.15.134:0/1501074693 wait complete. 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.160+0000 7fdc390e5640 10 monclient: monmap: 2024-09-16T15:07:53.179 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.180 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 Processor -- start 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 -- start start 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient: init 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient: _add_conns ranks=[7,6,2,8,5,1,3,0,4] 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdc34066530 0x7fdc341cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-i con 0x7fdc34066530 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 0x7fdc34098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-d con 0x7fdc34069e50 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc34098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): picked mon.noname-b con 0x7fdc3413b5a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340996e0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34093a20 con 0x7fdc34069e50 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34093d00 con 0x7fdc34066530 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc390e5640 10 monclient(hunting): authenticate will time out at 1209.750833s 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc34098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 0x7fdc34098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc34098a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45430/0 (socket says 172.21.15.134:45430) 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 0x7fdc34098540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:46468/0 (socket says 172.21.15.134:46468) 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 learned_addr learned my addr 172.21.15.134:0/997723217 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 10 monclient(hunting): get_auth_request con 0x7fdc3413b5a0 auth_method 0 2024-09-16T15:07:53.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 10 monclient(hunting): get_auth_request con 0x7fdc34069e50 auth_method 0 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.161+0000 7fdc3259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdc34066530 0x7fdc341cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request con 0x7fdc34066530 auth_method 0 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient(hunting): handle_auth_done global_id 44606 payload 824 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdc34066530 msgr2=0x7fdc341cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdc34066530 0x7fdc341cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 msgr2=0x7fdc34098540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 0x7fdc34098540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 monclient: found mon.noname-b 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c009b30 con 0x7fdc3413b5a0 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.162+0000 7fdc3259c640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34069e50 0x7fdc34098540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 msgr2=0x7fdc34098a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc3359e640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc34098a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdc24035040 tx=0x7fdc240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc24003610 con 0x7fdc3413b5a0 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc240037b0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 5 monclient: authenticate success, global_id 44606 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 10 monclient: _renew_subs 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc24003d90 con 0x7fdc3413b5a0 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc24009400 con 0x7fdc3413b5a0 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.187 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc3159a640 4 set_mon_vals no callback set 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 10 monclient: _renew_subs 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.164+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc340929a0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdc24039040 con 0x7fdc3413b5a0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc390e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc390e5640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 0x7fdc34090790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34092b30 con 0x7fdc340903b0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 0x7fdc34090790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient: get_auth_request con 0x7fdc340903b0 auth_method 0 2024-09-16T15:07:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc2b7fe640 1 --2- 172.21.15.134:0/997723217 >> v2:172.21.15.162:6800/980547263 conn(0x7fdc14038880 0x7fdc1403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7fdc2406a7e0 con 0x7fdc3413b5a0 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_done global_id 34701 payload 824 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.165+0000 7fdc32d9d640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc34061ae0 con 0x7fdc340903b0 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.166+0000 7fdc3259c640 1 --2- 172.21.15.134:0/997723217 >> v2:172.21.15.162:6800/980547263 conn(0x7fdc14038880 0x7fdc1403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.166+0000 7fdc3259c640 10 monclient: get_auth_request con 0x7fdc14038880 auth_method 0 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc32d9d640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 msgr2=0x7fdc34090790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 0x7fdc34090790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00f3e0 tx=0x7fdc2c00f8a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2c00e040 con 0x7fdc340903b0 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc3259c640 1 --2- 172.21.15.134:0/997723217 >> v2:172.21.15.162:6800/980547263 conn(0x7fdc14038880 0x7fdc1403ad40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc3406ce50 tx=0x7fdc1c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdc2c00f040 con 0x7fdc340903b0 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 msgr2=0x7fdc34090790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 0x7fdc34090790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00f3e0 tx=0x7fdc2c00f8a0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 msgr2=0x7fdc34090790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.168+0000 7fdc2b7fe640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc340903b0 0x7fdc34090790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc390e5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc390e5640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 0x7fdc3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc390e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34073880 con 0x7fdc34090cd0 2024-09-16T15:07:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 0x7fdc3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient: get_auth_request con 0x7fdc34090cd0 auth_method 0 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.174+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.175+0000 7fdc32d9d640 10 monclient(hunting): handle_auth_done global_id 34710 payload 824 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.175+0000 7fdc32d9d640 1 -- 172.21.15.134:0/997723217 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdc3407a680 con 0x7fdc34090cd0 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc32d9d640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 msgr2=0x7fdc3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc32d9d640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 0x7fdc3406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b010 tx=0x7fdc2c004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdc2c023070 con 0x7fdc34090cd0 2024-09-16T15:07:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 10 monclient: dump: 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7fdc2c0073f0 con 0x7fdc34090cd0 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 msgr2=0x7fdc3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 0x7fdc3406e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b010 tx=0x7fdc2c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 msgr2=0x7fdc3406e2f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc2b7fe640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdc34090cd0 0x7fdc3406e2f0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b010 tx=0x7fdc2c004a40 comp rx=0 tx=0).stop 2024-09-16T15:07:53.197 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.198 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.199 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.200 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.201 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.204 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.206 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.218 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:53.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.220 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 >> v2:172.21.15.162:6800/980547263 conn(0x7fdc14038880 msgr2=0x7fdc1403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 --2- 172.21.15.134:0/997723217 >> v2:172.21.15.162:6800/980547263 conn(0x7fdc14038880 0x7fdc1403ad40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc3406ce50 tx=0x7fdc1c009190 comp rx=0 tx=0).stop 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 10 monclient: shutdown 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 msgr2=0x7fdc34098a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 --2- 172.21.15.134:0/997723217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc3413b5a0 0x7fdc34098a80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdc24035040 tx=0x7fdc240046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 reap_dead start 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc3359e640 1 -- 172.21.15.134:0/997723217 reap_dead start 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 shutdown_connections 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.177+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 >> 172.21.15.134:0/997723217 conn(0x7fdc340a6400 msgr2=0x7fdc341397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.178+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 shutdown_connections 2024-09-16T15:07:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.178+0000 7fdc390e5640 1 -- 172.21.15.134:0/997723217 wait complete. 2024-09-16T15:07:53.223 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.223 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-16T15:07:53.223 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.320+0000 7f977bfff640 10 monclient: get_monmap_and_config 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.320+0000 7f977bfff640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.320+0000 7f977bfff640 10 monclient: monmap: 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:53.321529+0000 2024-09-16T15:07:53.322 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:53.321529+0000 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.323 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.324 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:53.324 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:53.324 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 Processor -- start 2024-09-16T15:07:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 -- start start 2024-09-16T15:07:53.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient: init 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient: _add_conns ranks=[9,8,4,6,10,7,1,5,3,0,2] 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 0x7f977c09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f977c135de0 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 0x7f977c1365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-e con 0x7f977c1361c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f977c0a3ce0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c09b620 con 0x7f977c135de0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a5540 con 0x7f977c0a3ce0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a4e80 con 0x7f977c1361c0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977bfff640 10 monclient(hunting): authenticate will time out at 1209.910934s 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977affd640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 msgr2=0x7f977c09bf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f9773fff640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 msgr2=0x7f977c1365c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f977affd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 0x7f977c09bf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.321+0000 7f9773fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 0x7f977c1365c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:51412/0 (socket says 172.21.15.134:51412) 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 -- 172.21.15.134:0/1595297205 learned_addr learned my addr 172.21.15.134:0/1595297205 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): get_auth_request con 0x7f977c0a3ce0 auth_method 0 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient(hunting): handle_auth_done global_id 5071 payload 274 2024-09-16T15:07:53.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 -- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 msgr2=0x7f977c1365c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 --2- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 0x7f977c1365c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 -- 172.21.15.134:0/1595297205 >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 msgr2=0x7f977c09bf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 --2- 172.21.15.134:0/1595297205 >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 0x7f977c09bf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 monclient: found mon.noname-c 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.322+0000 7f977b7fe640 1 -- 172.21.15.134:0/1595297205 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c073250 con 0x7f977c0a3ce0 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977b7fe640 1 -- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 msgr2=0x7f977c0a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977b7fe640 1 --2- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9774009e50 tx=0x7f977402a590 comp rx=0 tx=0).ready entity=mon.4 client_cookie=be33382b308f4f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 1 -- 172.21.15.134:0/1595297205 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9774007a70 con 0x7f977c0a3ce0 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: dump: 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.338 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 5 monclient: authenticate success, global_id 5071 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 1 -- 172.21.15.134:0/1595297205 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977402d050 con 0x7f977c0a3ce0 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 1 -- 172.21.15.134:0/1595297205 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9774007bf0 con 0x7f977c0a3ce0 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 4 set_mon_vals no callback set 2024-09-16T15:07:53.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977a7fc640 10 monclient: dump: 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.340 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 10 monclient: shutdown 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 -- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 msgr2=0x7f977c0a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 --2- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9774009e50 tx=0x7f977402a590 comp rx=0 tx=0).stop 2024-09-16T15:07:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 -- 172.21.15.134:0/1595297205 shutdown_connections 2024-09-16T15:07:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 --2- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f977c0a3ce0 0x7f977c0a4190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 --2- 172.21.15.134:0/1595297205 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f977c1361c0 0x7f977c1365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 --2- 172.21.15.134:0/1595297205 >> v2:172.21.15.134:3300/0 conn(0x7f977c135de0 0x7f977c09bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 -- 172.21.15.134:0/1595297205 >> 172.21.15.134:0/1595297205 conn(0x7f977c066070 msgr2=0x7f977c066480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 -- 172.21.15.134:0/1595297205 shutdown_connections 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.325+0000 7f977bfff640 1 -- 172.21.15.134:0/1595297205 wait complete. 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.326+0000 7f977bfff640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.326+0000 7f977bfff640 10 monclient: monmap: 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.326+0000 7f977bfff640 1 Processor -- start 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.326+0000 7f977bfff640 1 -- start start 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.326+0000 7f977bfff640 10 monclient: init 2024-09-16T15:07:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient: _add_conns ranks=[2,6,1,5,8,4,7,0,3] 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977c0a3ce0 0x7f977c0a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-b con 0x7f977c0a3ce0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-d con 0x7f977c135de0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c1361c0 0x7f977c135350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): picked mon.noname-f con 0x7f977c1361c0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c121f30 con 0x7f977c1361c0 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a4940 con 0x7f977c0a3ce0 2024-09-16T15:07:53.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f977c1cb730 con 0x7f977c135de0 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977bfff640 10 monclient(hunting): authenticate will time out at 1209.916259s 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977c0a3ce0 0x7f977c0a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:46496/0 (socket says 172.21.15.134:46496) 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 -- 172.21.15.134:0/149572942 learned_addr learned my addr 172.21.15.134:0/149572942 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): get_auth_request con 0x7f977c135de0 auth_method 0 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): get_auth_request con 0x7f977c0a3ce0 auth_method 0 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c1361c0 0x7f977c135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): get_auth_request con 0x7f977c1361c0 auth_method 0 2024-09-16T15:07:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f977b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient(hunting): handle_auth_done global_id 34719 payload 824 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977c0a3ce0 msgr2=0x7f977c0a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977c0a3ce0 0x7f977c0a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c1361c0 msgr2=0x7f977c135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c1361c0 0x7f977c135350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 monclient: found mon.noname-d 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.327+0000 7f9773fff640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774009b30 con 0x7f977c135de0 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f9773fff640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 msgr2=0x7f977c134e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f9773fff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f976400b520 tx=0x7f976400b920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9764003da0 con 0x7f977c135de0 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f97737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f97737fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.329+0000 7f97737fe640 10 monclient: dump: 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97640054e0 con 0x7f977c135de0 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 5 monclient: authenticate success, global_id 34719 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 monclient: _renew_subs 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c1227d0 con 0x7f977c135de0 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9764005650 con 0x7f977c135de0 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f97737fe640 10 monclient: dump: 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f9779ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 monclient: _renew_subs 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c0a72e0 con 0x7f977c135de0 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 0x7f977c11fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.330+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c069fe0 con 0x7f977c11f930 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9764005070 con 0x7f977c135de0 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 0x7f977c11fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient: get_auth_request con 0x7f977c11f930 auth_method 0 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f97737fe640 1 --2- 172.21.15.134:0/149572942 >> v2:172.21.15.162:6800/980547263 conn(0x7f975c038880 0x7f975c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 10 monclient(hunting): handle_auth_done global_id 44497 payload 824 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977affd640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f977c061ae0 con 0x7f977c11f930 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f97640491a0 con 0x7f977c135de0 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977b7fe640 1 --2- 172.21.15.134:0/149572942 >> v2:172.21.15.162:6800/980547263 conn(0x7f975c038880 0x7f975c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.331+0000 7f977b7fe640 10 monclient: get_auth_request con 0x7f975c038880 auth_method 0 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f977affd640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 msgr2=0x7f977c11fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f977affd640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 0x7f977c11fd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f976c012d60 con 0x7f977c11f930 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 10 monclient: dump: 2024-09-16T15:07:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f977b7fe640 1 --2- 172.21.15.134:0/149572942 >> v2:172.21.15.162:6800/980547263 conn(0x7f975c038880 0x7f975c03ad40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9774033000 tx=0x7f9774009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f976c0128f0 con 0x7f977c11f930 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.333+0000 7f97737fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.334+0000 7f97737fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.334+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 msgr2=0x7f977c11fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.334+0000 7f97737fe640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 0x7f977c11fd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c009ad0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.334+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 msgr2=0x7f977c11fd10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.334+0000 7f97737fe640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c11f930 0x7f977c11fd10 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c009ad0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977bfff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977bfff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 0x7f977c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c07a680 con 0x7f977c120250 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 0x7f977c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient: get_auth_request con 0x7f977c120250 auth_method 0 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 10 monclient(hunting): handle_auth_done global_id 44506 payload 824 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.340+0000 7f977affd640 1 -- 172.21.15.134:0/149572942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f977c09b240 con 0x7f977c120250 2024-09-16T15:07:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.342+0000 7f977affd640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 msgr2=0x7f977c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.342+0000 7f977affd640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 0x7f977c09db30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f976c005c50 con 0x7f977c120250 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 10 monclient: dump: 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9341 (secure 0 0 0) 0x7f976c005db0 con 0x7f977c120250 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 msgr2=0x7f977c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 0x7f977c09db30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c0059d0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 msgr2=0x7f977c09db30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f97737fe640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f977c120250 0x7f977c09db30 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f976c009610 tx=0x7f976c0059d0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.365 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.366 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.368 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.369 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.388 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 >> v2:172.21.15.162:6800/980547263 conn(0x7f975c038880 msgr2=0x7f975c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 --2- 172.21.15.134:0/149572942 >> v2:172.21.15.162:6800/980547263 conn(0x7f975c038880 0x7f975c03ad40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9774033000 tx=0x7f9774009000 comp rx=0 tx=0).stop 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 10 monclient: shutdown 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 msgr2=0x7f977c134e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f976400b520 tx=0x7f976400b920 comp rx=0 tx=0).stop 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977b7fe640 1 -- 172.21.15.134:0/149572942 reap_dead start 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 shutdown_connections 2024-09-16T15:07:53.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 --2- 172.21.15.134:0/149572942 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f977c135de0 0x7f977c134e10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 >> 172.21.15.134:0/149572942 conn(0x7f977c066070 msgr2=0x7f977c1365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 shutdown_connections 2024-09-16T15:07:53.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.343+0000 7f977bfff640 1 -- 172.21.15.134:0/149572942 wait complete. 2024-09-16T15:07:53.391 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.391 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-16T15:07:53.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:07:53.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.489+0000 7f95bd93e640 10 monclient: get_monmap_and_config 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.489+0000 7f95bd93e640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.489+0000 7f95bd93e640 10 monclient: monmap: 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:53.490306+0000 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:53.490306+0000 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:53.491 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:53.492 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 Processor -- start 2024-09-16T15:07:53.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 -- start start 2024-09-16T15:07:53.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient: init 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient: _add_conns ranks=[3,5,10,9,0,4,8,2,1,7,6] 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-g con 0x7f95b8136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-h con 0x7f95b8137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-f con 0x7f95b8135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b809abc0 con 0x7f95b8135dc0 2024-09-16T15:07:53.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b809ba50 con 0x7f95b8136e40 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80a53c0 con 0x7f95b8137220 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95bd93e640 10 monclient(hunting): authenticate will time out at 1210.079629s 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:37636/0 (socket says 172.21.15.134:37636) 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:48778/0 (socket says 172.21.15.134:48778) 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 1 -- 172.21.15.134:0/2861469458 learned_addr learned my addr 172.21.15.134:0/2861469458 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:36102/0 (socket says 172.21.15.134:36102) 2024-09-16T15:07:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 10 monclient(hunting): get_auth_request con 0x7f95b8137220 auth_method 0 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 10 monclient(hunting): get_auth_request con 0x7f95b8136e40 auth_method 0 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 10 monclient(hunting): get_auth_request con 0x7f95b8135dc0 auth_method 0 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.490+0000 7f95b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_done global_id 54228 payload 274 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 -- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 msgr2=0x7f95b8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 -- 172.21.15.134:0/2861469458 >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 msgr2=0x7f95b8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/2861469458 >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 monclient: found mon.noname-g 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b6ffd640 1 -- 172.21.15.134:0/2861469458 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8073250 con 0x7f95b8136e40 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95b77fe640 1 --2- 172.21.15.134:0/2861469458 >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.491+0000 7f95ae7fc640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b6ffd640 1 -- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 msgr2=0x7f95b809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95a80098a0 tx=0x7f95a802a0d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=bba5e4ae21fc339d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 1 -- 172.21.15.134:0/2861469458 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a802adf0 con 0x7f95b8136e40 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:07:53.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: dump: 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 1 -- 172.21.15.134:0/2861469458 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a80307b0 con 0x7f95b8136e40 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 1 -- 172.21.15.134:0/2861469458 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a8030d90 con 0x7f95b8136e40 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 5 monclient: authenticate success, global_id 54228 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 4 set_mon_vals no callback set 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95b67fc640 10 monclient: dump: 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 10 monclient: shutdown 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 -- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 msgr2=0x7f95b809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f95a80098a0 tx=0x7f95a802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 -- 172.21.15.134:0/2861469458 shutdown_connections 2024-09-16T15:07:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 --2- 172.21.15.134:0/2861469458 >> v2:172.21.15.162:3300/0 conn(0x7f95b8135dc0 0x7f95b8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 --2- 172.21.15.134:0/2861469458 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8136e40 0x7f95b809c150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 -- 172.21.15.134:0/2861469458 >> 172.21.15.134:0/2861469458 conn(0x7f95b80a6110 msgr2=0x7f95b80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 -- 172.21.15.134:0/2861469458 shutdown_connections 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.494+0000 7f95bd93e640 1 -- 172.21.15.134:0/2861469458 wait complete. 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.495+0000 7f95bd93e640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.495+0000 7f95bd93e640 10 monclient: monmap: 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.495+0000 7f95bd93e640 1 Processor -- start 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.495+0000 7f95bd93e640 1 -- start start 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.495+0000 7f95bd93e640 10 monclient: init 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient: _add_conns ranks=[2,8,5,4,0,6,1,3,7] 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 0x7f95b81cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-b con 0x7f95b8135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95b8136e40 0x7f95b8134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-e con 0x7f95b8136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b81354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): picked mon.noname-h con 0x7f95b8137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81cc200 con 0x7f95b8135dc0 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8123e00 con 0x7f95b8137220 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80a8a30 con 0x7f95b8136e40 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95bd93e640 10 monclient(hunting): authenticate will time out at 1210.085387s 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95ae7fc640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95b8136e40 msgr2=0x7f95b8134fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95ae7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95b8136e40 0x7f95b8134fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 0x7f95b81cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 0x7f95b81cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45444/0 (socket says 172.21.15.134:45444) 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 -- 172.21.15.134:0/3642289533 learned_addr learned my addr 172.21.15.134:0/3642289533 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): get_auth_request con 0x7f95b8135dc0 auth_method 0 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b81354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): get_auth_request con 0x7f95b8137220 auth_method 0 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient(hunting): handle_auth_done global_id 44624 payload 824 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95b8136e40 msgr2=0x7f95b8134fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95b8136e40 0x7f95b8134fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 msgr2=0x7f95b81354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f95b8137220 0x7f95b81354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 monclient: found mon.noname-b 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.496+0000 7f95b6ffd640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0009a70 con 0x7f95b8135dc0 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.498+0000 7f95b6ffd640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 msgr2=0x7f95b81cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.498+0000 7f95b6ffd640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 0x7f95b81cb4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097c0 tx=0x7f95a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a8030d90 con 0x7f95b8135dc0 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: dump: 2024-09-16T15:07:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8003610 con 0x7f95b8135dc0 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 5 monclient: authenticate success, global_id 44624 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 monclient: _renew_subs 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a0009400 con 0x7f95b8135dc0 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a8003bf0 con 0x7f95b8135dc0 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 10 monclient: dump: 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b5ffb640 4 set_mon_vals no callback set 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 monclient: _renew_subs 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a80094d0 con 0x7f95b8135dc0 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 0x7f95b81222a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8075ce0 con 0x7f95b8125620 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.499+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f95a8039070 con 0x7f95b8135dc0 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 0x7f95b81222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient: get_auth_request con 0x7f95b8125620 auth_method 0 2024-09-16T15:07:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95b4ff9640 1 --2- 172.21.15.134:0/3642289533 >> v2:172.21.15.162:6800/980547263 conn(0x7f95980388b0 0x7f959803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f95a806a6c0 con 0x7f95b8135dc0 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_done global_id 54237 payload 824 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95b77fe640 1 --2- 172.21.15.134:0/3642289533 >> v2:172.21.15.162:6800/980547263 conn(0x7f95980388b0 0x7f959803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95ae7fc640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95b8061ae0 con 0x7f95b8125620 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.500+0000 7f95b77fe640 10 monclient: get_auth_request con 0x7f95980388b0 auth_method 0 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95ae7fc640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 msgr2=0x7f95b81222a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95ae7fc640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 0x7f95b81222a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a000ec00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a0004030 con 0x7f95b8125620 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95b4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95b77fe640 1 --2- 172.21.15.134:0/3642289533 >> v2:172.21.15.162:6800/980547263 conn(0x7f95980388b0 0x7f959803ad70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95b8125e90 tx=0x7f95b000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.502+0000 7f95b4ff9640 10 monclient: dump: 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f95a000cb60 con 0x7f95b8125620 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 msgr2=0x7f95b81222a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 0x7f95b81222a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a000ec00 comp rx=0 tx=0).stop 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 msgr2=0x7f95b81222a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.503+0000 7f95b4ff9640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b8125620 0x7f95b81222a0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a000ec00 comp rx=0 tx=0).stop 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95bd93e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95bd93e640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 0x7f95b802b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95bd93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8071b20 con 0x7f95b802b260 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 0x7f95b802b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient: get_auth_request con 0x7f95b802b260 auth_method 0 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 10 monclient(hunting): handle_auth_done global_id 54246 payload 824 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.509+0000 7f95ae7fc640 1 -- 172.21.15.134:0/3642289533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95b807a680 con 0x7f95b802b260 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95ae7fc640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 msgr2=0x7f95b802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95ae7fc640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 0x7f95b802b670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a0004730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95a0003e10 con 0x7f95b802b260 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95b4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95b4ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.511+0000 7f95b4ff9640 10 monclient: dump: 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f95a0013470 con 0x7f95b802b260 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 msgr2=0x7f95b802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 0x7f95b802b670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a0004730 comp rx=0 tx=0).stop 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 msgr2=0x7f95b802b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b4ff9640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b802b260 0x7f95b802b670 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95b8137f00 tx=0x7f95a0004730 comp rx=0 tx=0).stop 2024-09-16T15:07:53.529 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.531 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.532 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.533 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.534 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.551 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 >> v2:172.21.15.162:6800/980547263 conn(0x7f95980388b0 msgr2=0x7f959803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 --2- 172.21.15.134:0/3642289533 >> v2:172.21.15.162:6800/980547263 conn(0x7f95980388b0 0x7f959803ad70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95b8125e90 tx=0x7f95b000b040 comp rx=0 tx=0).stop 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 10 monclient: shutdown 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 msgr2=0x7f95b81cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 --2- 172.21.15.134:0/3642289533 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b8135dc0 0x7f95b81cb4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097c0 tx=0x7f95a80046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b77fe640 1 -- 172.21.15.134:0/3642289533 reap_dead start 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95b77fe640 1 -- 172.21.15.134:0/3642289533 reap_dead start 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 shutdown_connections 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 >> 172.21.15.134:0/3642289533 conn(0x7f95b80a6110 msgr2=0x7f95b809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 shutdown_connections 2024-09-16T15:07:53.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.512+0000 7f95bd93e640 1 -- 172.21.15.134:0/3642289533 wait complete. 2024-09-16T15:07:53.554 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.554 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-16T15:07:53.554 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.650+0000 7fc22cb65640 10 monclient: get_monmap_and_config 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.650+0000 7fc22cb65640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.650+0000 7fc22cb65640 10 monclient: monmap: 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.652 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:53.651446+0000 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:53.651446+0000 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:53.653 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:53.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 Processor -- start 2024-09-16T15:07:53.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 -- start start 2024-09-16T15:07:53.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient: init 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient: _add_conns ranks=[9,7,0,4,5,3,10,2,1,8,6] 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 0x7fc22809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-a con 0x7fc228136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-i con 0x7fc228137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc228135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc228073250 con 0x7fc228135dc0 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22809abc0 con 0x7fc228136e40 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc22809ba50 con 0x7fc228137220 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22cb65640 10 monclient(hunting): authenticate will time out at 1210.240723s 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc226d9d640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fc228135dc0 legacy=0x7fc228136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-09-16T15:07:53.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22659c640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 msgr2=0x7fc22809c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc22659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 0x7fc22809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.651+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:36126/0 (socket says 172.21.15.134:36126) 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 -- 172.21.15.134:0/3705354832 learned_addr learned my addr 172.21.15.134:0/3705354832 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): get_auth_request con 0x7fc228137220 auth_method 0 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient(hunting): handle_auth_done global_id 54268 payload 274 2024-09-16T15:07:53.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 -- 172.21.15.134:0/3705354832 >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 msgr2=0x7fc22809c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 --2- 172.21.15.134:0/3705354832 >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 0x7fc22809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 -- 172.21.15.134:0/3705354832 >> v1:172.21.15.134:6789/0 conn(0x7fc228135dc0 legacy=0x7fc228136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 monclient: found mon.noname-i 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.652+0000 7fc225d9b640 1 -- 172.21.15.134:0/3705354832 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280a53c0 con 0x7fc228137220 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc225d9b640 1 -- 172.21.15.134:0/3705354832 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 msgr2=0x7fc228137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc225d9b640 1 --2- 172.21.15.134:0/3705354832 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098a0 tx=0x7fc21c02a130 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4bf268a4ffa00332 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 1 -- 172.21.15.134:0/3705354832 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c0307b0 con 0x7fc228137220 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:07:53.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: dump: 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.665 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 1 -- 172.21.15.134:0/3705354832 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c030930 con 0x7fc228137220 2024-09-16T15:07:53.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22cb65640 5 monclient: authenticate success, global_id 54268 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22cb65640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22cb65640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 1 -- 172.21.15.134:0/3705354832 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c0313a0 con 0x7fc228137220 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22cb65640 4 set_mon_vals no callback set 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.655+0000 7fc22559a640 10 monclient: dump: 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.667 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 10 monclient: shutdown 2024-09-16T15:07:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 -- 172.21.15.134:0/3705354832 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 msgr2=0x7fc228137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 --2- 172.21.15.134:0/3705354832 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098a0 tx=0x7fc21c02a130 comp rx=0 tx=0).stop 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 -- 172.21.15.134:0/3705354832 shutdown_connections 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 --2- 172.21.15.134:0/3705354832 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228137220 0x7fc228137600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 --2- 172.21.15.134:0/3705354832 >> v2:172.21.15.134:3300/0 conn(0x7fc228136e40 0x7fc22809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 -- 172.21.15.134:0/3705354832 >> 172.21.15.134:0/3705354832 conn(0x7fc2280a6110 msgr2=0x7fc2280a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 -- 172.21.15.134:0/3705354832 shutdown_connections 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.656+0000 7fc22cb65640 1 -- 172.21.15.134:0/3705354832 wait complete. 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient: monmap: 2024-09-16T15:07:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 Processor -- start 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 -- start start 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient: init 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient: _add_conns ranks=[7,4,6,3,5,0,8,1,2] 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 0x7fc22809f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-i con 0x7fc228135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:07:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc228136e40 0x7fc228135050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-c con 0x7fc228136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc228137220 0x7fc228135590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): picked mon.noname-d con 0x7fc228137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc228122aa0 con 0x7fc228136e40 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc228136b10 con 0x7fc228137220 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281365e0 con 0x7fc228135dc0 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22cb65640 10 monclient(hunting): authenticate will time out at 1210.246857s 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22659c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 0x7fc22809f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc228136e40 0x7fc228135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc226d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc228137220 0x7fc228135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc225d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc228136e40 0x7fc228135050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:51422/0 (socket says 172.21.15.134:51422) 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc22659c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 0x7fc22809f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:36128/0 (socket says 172.21.15.134:36128) 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc226d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc228137220 0x7fc228135590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:46498/0 (socket says 172.21.15.134:46498) 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.657+0000 7fc225d9b640 1 -- 172.21.15.134:0/2890949760 learned_addr learned my addr 172.21.15.134:0/2890949760 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): get_auth_request con 0x7fc228136e40 auth_method 0 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): get_auth_request con 0x7fc228135dc0 auth_method 0 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): get_auth_request con 0x7fc228137220 auth_method 0 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc226d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient(hunting): handle_auth_done global_id 54277 payload 824 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc228137220 msgr2=0x7fc228135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc228137220 0x7fc228135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc228136e40 msgr2=0x7fc228135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc228136e40 0x7fc228135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 monclient: found mon.noname-i 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:07:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.658+0000 7fc22659c640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0094d0 con 0x7fc228135dc0 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22659c640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 msgr2=0x7fc22809f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22659c640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 0x7fc22809f460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc21800b430 tx=0x7fc21800b8f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc218003d00 con 0x7fc228135dc0 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: dump: 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.682 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180054e0 con 0x7fc228135dc0 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 5 monclient: authenticate success, global_id 54277 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 10 monclient: _renew_subs 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2280a0030 con 0x7fc228135dc0 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc218003ea0 con 0x7fc228135dc0 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc216ffd640 10 monclient: dump: 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.684 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc224d99640 4 set_mon_vals no callback set 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 10 monclient: _renew_subs 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.660+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc228122450 con 0x7fc228135dc0 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc218005650 con 0x7fc228135dc0 2024-09-16T15:07:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc22cb65640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc22cb65640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 0x7fc22802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281225e0 con 0x7fc228120420 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 0x7fc22802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient: get_auth_request con 0x7fc228120420 auth_method 0 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc216ffd640 1 --2- 172.21.15.134:0/2890949760 >> v2:172.21.15.162:6800/980547263 conn(0x7fc208038880 0x7fc20803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7fc218049200 con 0x7fc228135dc0 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.661+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.662+0000 7fc226d9d640 1 --2- 172.21.15.134:0/2890949760 >> v2:172.21.15.162:6800/980547263 conn(0x7fc208038880 0x7fc20803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.662+0000 7fc226d9d640 10 monclient: get_auth_request con 0x7fc208038880 auth_method 0 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.662+0000 7fc225d9b640 10 monclient(hunting): handle_auth_done global_id 44483 payload 824 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.662+0000 7fc225d9b640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc228061ae0 con 0x7fc228120420 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc225d9b640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 msgr2=0x7fc22802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc225d9b640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 0x7fc22802b260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000c00 tx=0x7fc21c0046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc226d9d640 1 --2- 172.21.15.134:0/2890949760 >> v2:172.21.15.162:6800/980547263 conn(0x7fc208038880 0x7fc20803ad40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc220003cb0 tx=0x7fc22000d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c0307c0 con 0x7fc228120420 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 10 monclient: dump: 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc21c004460 con 0x7fc228120420 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 msgr2=0x7fc22802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 0x7fc22802b260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000c00 tx=0x7fc21c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 msgr2=0x7fc22802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.664+0000 7fc216ffd640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120420 0x7fc22802b260 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc21c000c00 tx=0x7fc21c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.670+0000 7fc22cb65640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.670+0000 7fc22cb65640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 0x7fc22802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.670+0000 7fc22cb65640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.670+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc228071b20 con 0x7fc228120800 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 0x7fc22802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient: get_auth_request con 0x7fc228120800 auth_method 0 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 10 monclient(hunting): handle_auth_done global_id 44492 payload 824 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.671+0000 7fc225d9b640 1 -- 172.21.15.134:0/2890949760 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc22807a680 con 0x7fc228120800 2024-09-16T15:07:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc225d9b640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 msgr2=0x7fc22802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc225d9b640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 0x7fc22802b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc21c033000 tx=0x7fc21c009240 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc21c04c040 con 0x7fc228120800 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 10 monclient: dump: 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7fc21c03d960 con 0x7fc228120800 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 msgr2=0x7fc22802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 0x7fc22802b7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc21c033000 tx=0x7fc21c009240 comp rx=0 tx=0).stop 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.673+0000 7fc216ffd640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 msgr2=0x7fc22802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc216ffd640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc228120800 0x7fc22802b7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.693 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.694 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.695 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.696 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.697 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.698 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.715 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 >> v2:172.21.15.162:6800/980547263 conn(0x7fc208038880 msgr2=0x7fc20803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 --2- 172.21.15.134:0/2890949760 >> v2:172.21.15.162:6800/980547263 conn(0x7fc208038880 0x7fc20803ad40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc220003cb0 tx=0x7fc22000d040 comp rx=0 tx=0).stop 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 10 monclient: shutdown 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 msgr2=0x7fc22809f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 --2- 172.21.15.134:0/2890949760 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc228135dc0 0x7fc22809f460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc21800b430 tx=0x7fc21800b8f0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc226d9d640 1 -- 172.21.15.134:0/2890949760 reap_dead start 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc226d9d640 1 -- 172.21.15.134:0/2890949760 reap_dead start 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 shutdown_connections 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 >> 172.21.15.134:0/2890949760 conn(0x7fc2280a6110 msgr2=0x7fc2280a2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 shutdown_connections 2024-09-16T15:07:53.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.674+0000 7fc22cb65640 1 -- 172.21.15.134:0/2890949760 wait complete. 2024-09-16T15:07:53.718 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.718 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-16T15:07:53.718 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.813+0000 7efccdb0f640 10 monclient: get_monmap_and_config 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.813+0000 7efccdb0f640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.813+0000 7efccdb0f640 10 monclient: monmap: 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:07:53.814846+0000 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:07:53.814846+0000 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:07:53.816 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:07:53.817 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.814+0000 7efccdb0f640 1 Processor -- start 2024-09-16T15:07:53.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.814+0000 7efccdb0f640 1 -- start start 2024-09-16T15:07:53.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.814+0000 7efccdb0f640 10 monclient: init 2024-09-16T15:07:53.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient: _add_conns ranks=[1,3,8,6,7,4,9,2,10,0,5] 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7efcc8136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 0x7efcc8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-g con 0x7efcc8137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 0x7efcc813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-e con 0x7efcc813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc8073250 con 0x7efcc8136e40 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a56e0 con 0x7efcc8137220 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7efcc809abc0 con 0x7efcc813c370 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efccdb0f640 10 monclient(hunting): authenticate will time out at 1210.404348s 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcc7fff640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 msgr2=0x7efcc813e760 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 0x7efcc813e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcc77fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efcc8136e40 0x7efcc809c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:50350/0 (socket says 172.21.15.134:50350) 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcc77fe640 1 -- 172.21.15.134:0/348595679 learned_addr learned my addr 172.21.15.134:0/348595679 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 0x7efcc8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): get_auth_request con 0x7efcc8137220 auth_method 0 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient(hunting): handle_auth_done global_id 54255 payload 274 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 -- 172.21.15.134:0/348595679 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 msgr2=0x7efcc813e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 0x7efcc813e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 -- 172.21.15.134:0/348595679 >> v1:172.21.15.162:6789/0 conn(0x7efcc8136e40 legacy=0x7efcc809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 monclient: found mon.noname-g 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:07:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.815+0000 7efcbffff640 1 -- 172.21.15.134:0/348595679 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc809ba50 con 0x7efcc8137220 2024-09-16T15:07:53.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.818+0000 7efcbffff640 1 -- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 msgr2=0x7efcc8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.818+0000 7efcbffff640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 0x7efcc8137600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcb00097d0 tx=0x7efcb002a010 comp rx=0 tx=0).ready entity=mon.3 client_cookie=3f17ced66bba5185 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 1 -- 172.21.15.134:0/348595679 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb002ae10 con 0x7efcc8137220 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: dump: 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 1 -- 172.21.15.134:0/348595679 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb00307b0 con 0x7efcc8137220 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 1 -- 172.21.15.134:0/348595679 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb0030d90 con 0x7efcc8137220 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 5 monclient: authenticate success, global_id 54255 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 10 monclient: get_monmap_and_config success 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 4 set_mon_vals no callback set 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efcc6ffd640 10 monclient: dump: 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 10 monclient: shutdown 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 -- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 msgr2=0x7efcc8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 0x7efcc8137600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efcb00097d0 tx=0x7efcb002a010 comp rx=0 tx=0).stop 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 -- 172.21.15.134:0/348595679 shutdown_connections 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7efcc813c370 0x7efcc813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 --2- 172.21.15.134:0/348595679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcc8137220 0x7efcc8137600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 -- 172.21.15.134:0/348595679 >> 172.21.15.134:0/348595679 conn(0x7efcc80a6110 msgr2=0x7efcc80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 -- 172.21.15.134:0/348595679 shutdown_connections 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.819+0000 7efccdb0f640 1 -- 172.21.15.134:0/348595679 wait complete. 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.820+0000 7efccdb0f640 10 monclient: build_initial_monmap 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.820+0000 7efccdb0f640 10 monclient: monmap: 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:07:53.834 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.820+0000 7efccdb0f640 1 Processor -- start 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.820+0000 7efccdb0f640 1 -- start start 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.820+0000 7efccdb0f640 10 monclient: init 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient: _reopen_session rank -1 2024-09-16T15:07:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient: _add_conns ranks=[5,4,0,8,1,6,2,7,3] 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 0x7efcc8134fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-h con 0x7efcc8136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 0x7efcc80a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-c con 0x7efcc8137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc813c370 0x7efcc80a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): picked mon.noname-a con 0x7efcc813c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc81cc390 con 0x7efcc813c370 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a7e80 con 0x7efcc8137220 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80a3310 con 0x7efcc8136e40 2024-09-16T15:07:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): _renew_subs 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efccdb0f640 10 monclient(hunting): authenticate will time out at 1210.410257s 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc7fff640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc813c370 msgr2=0x7efcc80a1db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc813c370 0x7efcc80a1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 0x7efcc80a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 0x7efcc80a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:51432/0 (socket says 172.21.15.134:51432) 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 1 -- 172.21.15.134:0/3845151418 learned_addr learned my addr 172.21.15.134:0/3845151418 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): get_auth_request con 0x7efcc8137220 auth_method 0 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 0x7efcc8134fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): get_auth_request con 0x7efcc8136e40 auth_method 0 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient(hunting): handle_auth_done global_id 44501 payload 824 2024-09-16T15:07:53.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 msgr2=0x7efcc80a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 0x7efcc80a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc813c370 msgr2=0x7efcc80a1db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc813c370 0x7efcc80a1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 monclient: found mon.noname-h 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcc77fe640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb0009400 con 0x7efcc8136e40 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.821+0000 7efcbffff640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efcc8137220 0x7efcc80a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.823+0000 7efcc77fe640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 msgr2=0x7efcc8134fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.823+0000 7efcc77fe640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 0x7efcc8134fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c410 tx=0x7efcb800c8d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb8005cf0 con 0x7efcc8136e40 2024-09-16T15:07:53.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: dump: 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.840 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb8005e90 con 0x7efcc8136e40 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb8007540 con 0x7efcc8136e40 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 5 monclient: authenticate success, global_id 44501 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 monclient: _renew_subs 2024-09-16T15:07:53.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80a78c0 con 0x7efcc8136e40 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc67fc640 4 set_mon_vals no callback set 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efcc4ff9640 10 monclient: dump: 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.842 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 monclient: _renew_subs 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc8121400 con 0x7efcc8136e40 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 0x7efcc80a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.824+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efcc8071b20 con 0x7efcc80a22f0 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7efcb80076a0 con 0x7efcc8136e40 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 0x7efcc80a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient: get_auth_request con 0x7efcc80a22f0 auth_method 0 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcc4ff9640 1 --2- 172.21.15.134:0/3845151418 >> v2:172.21.15.162:6800/980547263 conn(0x7efca8038880 0x7efca803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(323..323 src has 1..323) v4 ==== 3943+0+0 (secure 0 0 0) 0x7efcb8049300 con 0x7efcc8136e40 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 10 monclient(hunting): handle_auth_done global_id 54286 payload 824 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcbffff640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcc8061ae0 con 0x7efcc80a22f0 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcc7fff640 1 --2- 172.21.15.134:0/3845151418 >> v2:172.21.15.162:6800/980547263 conn(0x7efca8038880 0x7efca803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.825+0000 7efcc7fff640 10 monclient: get_auth_request con 0x7efca8038880 auth_method 0 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcbffff640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 msgr2=0x7efcc80a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcbffff640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 0x7efcc80a26d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb00096f0 tx=0x7efcb00046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb0030e40 con 0x7efcc80a22f0 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcc4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcc4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcc4ff9640 10 monclient: dump: 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.827+0000 7efcc7fff640 1 --2- 172.21.15.134:0/3845151418 >> v2:172.21.15.162:6800/980547263 conn(0x7efca8038880 0x7efca803ad40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcc8120ef0 tx=0x7efcc0009230 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efcb0003820 con 0x7efcc80a22f0 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 msgr2=0x7efcc80a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 0x7efcc80a26d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb00096f0 tx=0x7efcb00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 msgr2=0x7efcc80a26d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.828+0000 7efcc4ff9640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc80a22f0 0x7efcc80a26d0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efcb00096f0 tx=0x7efcb00046c0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efccdb0f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efccdb0f640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 0x7efcc809d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efccdb0f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efcc809bd30 con 0x7efcc809d4f0 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 0x7efcc809d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient: get_auth_request con 0x7efcc809d4f0 auth_method 0 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:07:53.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 10 monclient(hunting): handle_auth_done global_id 54295 payload 824 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.834+0000 7efcbffff640 1 -- 172.21.15.134:0/3845151418 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efcc807a680 con 0x7efcc809d4f0 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.836+0000 7efcbffff640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 msgr2=0x7efcc809d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.836+0000 7efcbffff640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 0x7efcc809d900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcb002a050 tx=0x7efcb0009190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efcb004c040 con 0x7efcc809d4f0 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 10 monclient: dump: 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:07:53.850 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7efcb0003ec0 con 0x7efcc809d4f0 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 msgr2=0x7efcc809d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 0x7efcc809d900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcb002a050 tx=0x7efcb0009190 comp rx=0 tx=0).stop 2024-09-16T15:07:53.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 msgr2=0x7efcc809d900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc4ff9640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7efcc809d4f0 0x7efcc809d900 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efcb002a050 tx=0x7efcb0009190 comp rx=0 tx=0).stop 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 62, 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:07:53.852 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.853 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.854 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:07:53.855 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:07:53.856 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:07:53.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:07:53.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:07:53.868 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.870 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:07:53.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:07:53.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:07:53.873 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 >> v2:172.21.15.162:6800/980547263 conn(0x7efca8038880 msgr2=0x7efca803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 --2- 172.21.15.134:0/3845151418 >> v2:172.21.15.162:6800/980547263 conn(0x7efca8038880 0x7efca803ad40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efcc8120ef0 tx=0x7efcc0009230 comp rx=0 tx=0).stop 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 10 monclient: shutdown 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 msgr2=0x7efcc8134fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 --2- 172.21.15.134:0/3845151418 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efcc8136e40 0x7efcc8134fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c410 tx=0x7efcb800c8d0 comp rx=0 tx=0).stop 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc7fff640 1 -- 172.21.15.134:0/3845151418 reap_dead start 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efcc7fff640 1 -- 172.21.15.134:0/3845151418 reap_dead start 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 shutdown_connections 2024-09-16T15:07:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 >> 172.21.15.134:0/3845151418 conn(0x7efcc80a6110 msgr2=0x7efcc813a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 shutdown_connections 2024-09-16T15:07:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:07:53.837+0000 7efccdb0f640 1 -- 172.21.15.134:0/3845151418 wait complete. 2024-09-16T15:07:53.876 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:07:53.876 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:07:53.899 INFO:tasks.rados.rados.0.smithi162.stdout:1617: done (2 left) 2024-09-16T15:07:53.899 INFO:tasks.rados.rados.0.smithi162.stdout:1618: rollback oid 23 current snap is 150 2024-09-16T15:07:53.899 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 147 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1619: read oid 2 snap -1 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1619: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1620: delete oid 45 current snap is 150 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1616: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1616: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1616: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 583 (ObjNum 586 snap 147 seq_num 586) dirty exists 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: finishing write tid 1 to smithi16234619-22 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: finishing write tid 2 to smithi16234619-22 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: finishing write tid 3 to smithi16234619-22 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: finishing write tid 4 to smithi16234619-22 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: finishing write tid 5 to smithi16234619-22 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: oid 22 updating version 0 to 355 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: oid 22 updating version 355 to 357 2024-09-16T15:07:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:1615: oid 22 version 357 is already newer than 354 2024-09-16T15:07:53.901 INFO:tasks.rados.rados.0.smithi162.stdout:1615: oid 22 version 357 is already newer than 356 2024-09-16T15:07:53.901 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 357 (ObjNum 588 snap 150 seq_num 588) dirty exists 2024-09-16T15:07:53.901 INFO:tasks.rados.rados.0.smithi162.stdout:1615: left oid 22 (ObjNum 588 snap 150 seq_num 588) 2024-09-16T15:07:53.905 INFO:tasks.rados.rados.0.smithi162.stdout:1619: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:1618: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:07:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:1618: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1618: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 141 (ObjNum 570 snap 143 seq_num 570) dirty exists 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1615: done (4 left) 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1616: done (3 left) 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1618: done (2 left) 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1619: done (1 left) 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1620: done (0 left) 2024-09-16T15:07:54.015 INFO:tasks.rados.rados.0.smithi162.stdout:1621: delete oid 49 current snap is 150 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1621: done (0 left) 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1622: rollback oid 33 current snap is 150 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 149 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1623: read oid 12 snap -1 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1623: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1624: read oid 49 snap -1 2024-09-16T15:07:54.018 INFO:tasks.rados.rados.0.smithi162.stdout:1624: expect deleted 2024-09-16T15:07:54.019 INFO:tasks.rados.rados.0.smithi162.stdout:1625: read oid 37 snap -1 2024-09-16T15:07:54.019 INFO:tasks.rados.rados.0.smithi162.stdout:1625: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:54.019 INFO:tasks.rados.rados.0.smithi162.stdout:1626: write (excl) oid 28 current snap is 150 2024-09-16T15:07:54.019 INFO:tasks.rados.rados.0.smithi162.stdout:1626: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2024-09-16T15:07:54.026 INFO:tasks.rados.rados.0.smithi162.stdout:1626: writing smithi16234619-28 from 732937 to 1364277 tid 1 2024-09-16T15:07:54.032 INFO:tasks.rados.rados.0.smithi162.stdout:1626: writing smithi16234619-28 from 1850858 to 2284399 tid 2 2024-09-16T15:07:54.038 INFO:tasks.rados.rados.0.smithi162.stdout:1626: writing smithi16234619-28 from 2796871 to 3468160 tid 3 2024-09-16T15:07:54.041 INFO:tasks.rados.rados.0.smithi162.stdout:1626: writing smithi16234619-28 from 3888195 to 3932937 tid 4 2024-09-16T15:07:54.041 INFO:tasks.rados.rados.0.smithi162.stdout:1627: copy_from oid 23 from oid 19 current snap is 150 2024-09-16T15:07:54.042 INFO:tasks.rados.rados.0.smithi162.stdout:1625: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:54.190 INFO:tasks.rados.rados.0.smithi162.stdout:1622: finishing rollback tid 0 to smithi16234619-33 2024-09-16T15:07:54.191 INFO:tasks.rados.rados.0.smithi162.stdout:1622: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:07:54.191 INFO:tasks.rados.rados.0.smithi162.stdout:1622: finishing rollback tid 2 to smithi16234619-33 2024-09-16T15:07:54.191 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 552 (ObjNum 557 snap 141 seq_num 557) dirty exists 2024-09-16T15:07:54.191 INFO:tasks.rados.rados.0.smithi162.stdout:1623: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 1 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 2 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 3 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 4 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 5 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: finishing write tid 6 to smithi16234619-28 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: oid 28 updating version 0 to 554 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: oid 28 updating version 554 to 558 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: oid 28 version 558 is already newer than 556 2024-09-16T15:07:54.386 INFO:tasks.rados.rados.0.smithi162.stdout:1626: oid 28 version 558 is already newer than 555 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1626: oid 28 version 558 is already newer than 557 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 558 (ObjNum 591 snap 150 seq_num 591) dirty exists 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1626: left oid 28 (ObjNum 591 snap 150 seq_num 591) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1627: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1627: finishing copy_from to smithi16234619-23 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 583 (ObjNum 586 snap 147 seq_num 586) dirty exists 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1622: done (5 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1623: done (4 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1624: done (3 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1625: done (2 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1626: done (1 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1627: done (0 left) 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1628: read oid 4 snap -1 2024-09-16T15:07:54.387 INFO:tasks.rados.rados.0.smithi162.stdout:1628: expect deleted 2024-09-16T15:07:54.388 INFO:tasks.rados.rados.0.smithi162.stdout:1629: read oid 7 snap -1 2024-09-16T15:07:54.388 INFO:tasks.rados.rados.0.smithi162.stdout:1629: expect deleted 2024-09-16T15:07:54.388 INFO:tasks.rados.rados.0.smithi162.stdout:1630: write oid 38 current snap is 150 2024-09-16T15:07:54.388 INFO:tasks.rados.rados.0.smithi162.stdout:1630: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2024-09-16T15:07:54.393 INFO:tasks.rados.rados.0.smithi162.stdout:1630: writing smithi16234619-38 from 749744 to 1456333 tid 1 2024-09-16T15:07:54.398 INFO:tasks.rados.rados.0.smithi162.stdout:1630: writing smithi16234619-38 from 1909340 to 2486539 tid 2 2024-09-16T15:07:54.405 INFO:tasks.rados.rados.0.smithi162.stdout:1630: writing smithi16234619-38 from 3224294 to 3706855 tid 3 2024-09-16T15:07:54.405 INFO:tasks.rados.rados.0.smithi162.stdout:1631: write oid 45 current snap is 150 2024-09-16T15:07:54.406 INFO:tasks.rados.rados.0.smithi162.stdout:1631: seq_num 593 ranges {766551=498191,1684175=720857,2851717=693833} 2024-09-16T15:07:54.412 INFO:tasks.rados.rados.0.smithi162.stdout:1631: writing smithi16234619-45 from 766551 to 1264742 tid 1 2024-09-16T15:07:54.418 INFO:tasks.rados.rados.0.smithi162.stdout:1631: writing smithi16234619-45 from 1684175 to 2405032 tid 2 2024-09-16T15:07:54.425 INFO:tasks.rados.rados.0.smithi162.stdout:1631: writing smithi16234619-45 from 2851717 to 3545550 tid 3 2024-09-16T15:07:54.425 INFO:tasks.rados.rados.0.smithi162.stdout:1632: write oid 35 current snap is 150 2024-09-16T15:07:54.425 INFO:tasks.rados.rados.0.smithi162.stdout:1632: seq_num 594 ranges {783358=573440,2026304=580868,3279140=621458} 2024-09-16T15:07:54.432 INFO:tasks.rados.rados.0.smithi162.stdout:1632: writing smithi16234619-35 from 783358 to 1356798 tid 1 2024-09-16T15:07:54.439 INFO:tasks.rados.rados.0.smithi162.stdout:1632: writing smithi16234619-35 from 2026304 to 2607172 tid 2 2024-09-16T15:07:54.446 INFO:tasks.rados.rados.0.smithi162.stdout:1632: writing smithi16234619-35 from 3279140 to 3900598 tid 3 2024-09-16T15:07:54.446 INFO:tasks.rados.rados.0.smithi162.stdout:1633: snap_create 2024-09-16T15:07:54.446 INFO:tasks.rados.rados.0.smithi162.stdout:1630: finishing write tid 1 to smithi16234619-38 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: finishing write tid 2 to smithi16234619-38 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: finishing write tid 3 to smithi16234619-38 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: finishing write tid 4 to smithi16234619-38 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: finishing write tid 5 to smithi16234619-38 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: oid 38 updating version 0 to 532 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: oid 38 updating version 532 to 533 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: oid 38 version 533 is already newer than 530 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: oid 38 version 533 is already newer than 531 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 533 (ObjNum 592 snap 150 seq_num 592) dirty exists 2024-09-16T15:07:54.451 INFO:tasks.rados.rados.0.smithi162.stdout:1630: left oid 38 (ObjNum 592 snap 150 seq_num 592) 2024-09-16T15:07:54.460 INFO:tasks.rados.rados.0.smithi162.stdout:1631: finishing write tid 1 to smithi16234619-45 2024-09-16T15:07:54.461 INFO:tasks.rados.rados.0.smithi162.stdout:1631: finishing write tid 2 to smithi16234619-45 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: finishing write tid 3 to smithi16234619-45 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: finishing write tid 4 to smithi16234619-45 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: finishing write tid 5 to smithi16234619-45 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: oid 45 updating version 0 to 710 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: oid 45 version 710 is already newer than 708 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: oid 45 version 710 is already newer than 709 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: oid 45 updating version 710 to 711 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 711 (ObjNum 593 snap 150 seq_num 593) dirty exists 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1631: left oid 45 (ObjNum 593 snap 150 seq_num 593) 2024-09-16T15:07:54.464 INFO:tasks.rados.rados.0.smithi162.stdout:1632: finishing write tid 1 to smithi16234619-35 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: finishing write tid 2 to smithi16234619-35 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: finishing write tid 3 to smithi16234619-35 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: finishing write tid 4 to smithi16234619-35 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: finishing write tid 5 to smithi16234619-35 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: oid 35 updating version 0 to 536 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: oid 35 version 536 is already newer than 535 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: oid 35 updating version 536 to 537 2024-09-16T15:07:54.484 INFO:tasks.rados.rados.0.smithi162.stdout:1632: oid 35 version 537 is already newer than 534 2024-09-16T15:07:54.485 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 537 (ObjNum 594 snap 150 seq_num 594) dirty exists 2024-09-16T15:07:54.485 INFO:tasks.rados.rados.0.smithi162.stdout:1632: left oid 35 (ObjNum 594 snap 150 seq_num 594) 2024-09-16T15:07:55.266 INFO:tasks.rados.rados.0.smithi162.stdout:1628: done (5 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1629: done (4 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1630: done (3 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1631: done (2 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1632: done (1 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1633: done (0 left) 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1634: rollback oid 45 current snap is 151 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 148 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1635: read oid 11 snap -1 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1635: expect deleted 2024-09-16T15:07:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:1636: snap_create 2024-09-16T15:07:55.271 INFO:tasks.rados.rados.0.smithi162.stdout:1634: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:07:55.271 INFO:tasks.rados.rados.0.smithi162.stdout:1634: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:07:55.271 INFO:tasks.rados.rados.0.smithi162.stdout:1634: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:07:55.272 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 715 (ObjNum 577 snap 144 seq_num 577) dirty exists 2024-09-16T15:07:56.273 INFO:tasks.rados.rados.0.smithi162.stdout:1634: done (2 left) 2024-09-16T15:07:56.273 INFO:tasks.rados.rados.0.smithi162.stdout:1635: done (1 left) 2024-09-16T15:07:56.273 INFO:tasks.rados.rados.0.smithi162.stdout:1636: done (0 left) 2024-09-16T15:07:56.274 INFO:tasks.rados.rados.0.smithi162.stdout:1637: rollback oid 40 current snap is 152 2024-09-16T15:07:56.274 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 40 to 150 2024-09-16T15:07:56.274 INFO:tasks.rados.rados.0.smithi162.stdout:1638: delete oid 5 current snap is 152 2024-09-16T15:07:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:1637: finishing rollback tid 0 to smithi16234619-40 2024-09-16T15:07:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:1638: done (1 left) 2024-09-16T15:07:56.279 INFO:tasks.rados.rados.0.smithi162.stdout:1639: write (excl) oid 31 current snap is 152 2024-09-16T15:07:56.279 INFO:tasks.rados.rados.0.smithi162.stdout:1639: seq_num 596 ranges {416972=723938,1743268=273704} 2024-09-16T15:07:56.285 INFO:tasks.rados.rados.0.smithi162.stdout:1639: writing smithi16234619-31 from 416972 to 1140910 tid 1 2024-09-16T15:07:56.290 INFO:tasks.rados.rados.0.smithi162.stdout:1639: writing smithi16234619-31 from 1743268 to 2016972 tid 2 2024-09-16T15:07:56.290 INFO:tasks.rados.rados.0.smithi162.stdout:1640: read oid 22 snap -1 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1640: expect (ObjNum 588 snap 150 seq_num 588) 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1641: copy_from oid 4 from oid 17 current snap is 152 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1637: finishing rollback tid 1 to smithi16234619-40 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1637: finishing rollback tid 2 to smithi16234619-40 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 279 (ObjNum 492 snap 121 seq_num 492) dirty exists 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1637: done (3 left) 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1642: copy_from oid 21 from oid 20 current snap is 152 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1643: write oid 26 current snap is 152 2024-09-16T15:07:56.291 INFO:tasks.rados.rados.0.smithi162.stdout:1643: seq_num 597 ranges {433779=799187,1801750=232029} 2024-09-16T15:07:56.297 INFO:tasks.rados.rados.0.smithi162.stdout:1643: writing smithi16234619-26 from 433779 to 1232966 tid 1 2024-09-16T15:07:56.302 INFO:tasks.rados.rados.0.smithi162.stdout:1643: writing smithi16234619-26 from 1801750 to 2033779 tid 2 2024-09-16T15:07:56.303 INFO:tasks.rados.rados.0.smithi162.stdout:1644: rollback oid 25 current snap is 152 2024-09-16T15:07:56.303 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 150 2024-09-16T15:07:56.303 INFO:tasks.rados.rados.0.smithi162.stdout:1645: snap_remove snap 151 2024-09-16T15:07:57.283 INFO:tasks.rados.rados.0.smithi162.stdout:1645: done (6 left) 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:1646: snap_create 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:1644: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:1644: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:1644: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 593 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:07:57.284 INFO:tasks.rados.rados.0.smithi162.stdout:1640: expect (ObjNum 588 snap 150 seq_num 588) 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: finishing write tid 1 to smithi16234619-31 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: finishing write tid 2 to smithi16234619-31 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: finishing write tid 3 to smithi16234619-31 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: finishing write tid 4 to smithi16234619-31 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: oid 31 updating version 0 to 566 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: oid 31 version 566 is already newer than 565 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: oid 31 updating version 566 to 567 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 567 (ObjNum 596 snap 152 seq_num 596) dirty exists 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1639: left oid 31 (ObjNum 596 snap 152 seq_num 596) 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1642: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1641: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:07:57.465 INFO:tasks.rados.rados.0.smithi162.stdout:1642: finishing copy_from to smithi16234619-21 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: finishing write tid 1 to smithi16234619-26 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: finishing write tid 2 to smithi16234619-26 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: finishing write tid 3 to smithi16234619-26 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: finishing write tid 4 to smithi16234619-26 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: oid 26 updating version 0 to 719 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: oid 26 updating version 719 to 720 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: oid 26 version 720 is already newer than 718 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 720 (ObjNum 597 snap 152 seq_num 597) dirty exists 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1643: left oid 26 (ObjNum 597 snap 152 seq_num 597) 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:1641: finishing copy_from to smithi16234619-4 2024-09-16T15:07:57.466 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 549 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:57.841 INFO:tasks.rados.rados.0.smithi162.stdout:1639: done (6 left) 2024-09-16T15:07:57.841 INFO:tasks.rados.rados.0.smithi162.stdout:1640: done (5 left) 2024-09-16T15:07:57.841 INFO:tasks.rados.rados.0.smithi162.stdout:1641: done (4 left) 2024-09-16T15:07:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:1642: done (3 left) 2024-09-16T15:07:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:1643: done (2 left) 2024-09-16T15:07:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:1644: done (1 left) 2024-09-16T15:07:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:1646: done (0 left) 2024-09-16T15:07:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:1647: snap_remove snap 148 2024-09-16T15:07:58.848 INFO:tasks.rados.rados.0.smithi162.stdout:1647: done (0 left) 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1648: read oid 35 snap 152 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1648: expect (ObjNum 594 snap 150 seq_num 594) 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1649: rollback oid 11 current snap is 153 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 149 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1650: read oid 37 snap -1 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1650: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1651: read oid 2 snap -1 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1651: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1652: copy_from oid 30 from oid 27 current snap is 153 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1653: rollback oid 17 current snap is 153 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 147 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1654: read oid 21 snap -1 2024-09-16T15:07:58.849 INFO:tasks.rados.rados.0.smithi162.stdout:1654: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:58.850 INFO:tasks.rados.rados.0.smithi162.stdout:1655: write (excl) oid 1 current snap is 153 2024-09-16T15:07:58.850 INFO:tasks.rados.rados.0.smithi162.stdout:1655: seq_num 598 ranges {450586=474436,1343879=588206} 2024-09-16T15:07:58.853 INFO:tasks.rados.rados.0.smithi162.stdout:1655: writing smithi16234619-1 from 450586 to 925022 tid 1 2024-09-16T15:07:58.860 INFO:tasks.rados.rados.0.smithi162.stdout:1655: writing smithi16234619-1 from 1343879 to 1932085 tid 2 2024-09-16T15:07:58.860 INFO:tasks.rados.rados.0.smithi162.stdout:1656: copy_from oid 24 from oid 49 current snap is 153 2024-09-16T15:07:58.861 INFO:tasks.rados.rados.0.smithi162.stdout:1657: delete oid 48 current snap is 153 2024-09-16T15:07:58.861 INFO:tasks.rados.rados.0.smithi162.stdout:1649: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:07:58.861 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:07:58.861 INFO:tasks.rados.rados.0.smithi162.stdout:1650: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:07:59.005 INFO:tasks.rados.rados.0.smithi162.stdout:1651: expect (ObjNum 490 snap 121 seq_num 490) 2024-09-16T15:07:59.108 INFO:tasks.rados.rados.0.smithi162.stdout:1653: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:07:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:1653: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:07:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:1653: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:07:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 730 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:07:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:1654: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:59.247 INFO:tasks.rados.rados.0.smithi162.stdout:1648: expect (ObjNum 594 snap 150 seq_num 594) 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1656: finishing copy_from to smithi16234619-24 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1656: got expected ENOENT (src dne) 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1656: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: finishing write tid 1 to smithi16234619-1 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: finishing write tid 2 to smithi16234619-1 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: finishing write tid 3 to smithi16234619-1 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: finishing write tid 4 to smithi16234619-1 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: oid 1 updating version 0 to 574 2024-09-16T15:07:59.502 INFO:tasks.rados.rados.0.smithi162.stdout:1655: oid 1 version 574 is already newer than 572 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1655: oid 1 version 574 is already newer than 573 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 574 (ObjNum 598 snap 153 seq_num 598) dirty exists 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1655: left oid 1 (ObjNum 598 snap 153 seq_num 598) 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1652: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1652: finishing copy_from to smithi16234619-30 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 499 (ObjNum 518 snap 135 seq_num 518) dirty exists 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1648: done (9 left) 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1649: done (8 left) 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1650: done (7 left) 2024-09-16T15:07:59.503 INFO:tasks.rados.rados.0.smithi162.stdout:1651: done (6 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1652: done (5 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1653: done (4 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1654: done (3 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1655: done (2 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1656: done (1 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1657: done (0 left) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1658: read oid 21 snap -1 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1658: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1659: rollback oid 24 current snap is 153 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 147 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1660: write (excl) oid 18 current snap is 153 2024-09-16T15:07:59.504 INFO:tasks.rados.rados.0.smithi162.stdout:1660: seq_num 600 ranges {484200=624934,1860843=223357} 2024-09-16T15:07:59.507 INFO:tasks.rados.rados.0.smithi162.stdout:1660: writing smithi16234619-18 from 484200 to 1109134 tid 1 2024-09-16T15:07:59.514 INFO:tasks.rados.rados.0.smithi162.stdout:1660: writing smithi16234619-18 from 1860843 to 2084200 tid 2 2024-09-16T15:07:59.514 INFO:tasks.rados.rados.0.smithi162.stdout:1661: snap_remove snap 150 2024-09-16T15:07:59.857 INFO:tasks.rados.rados.0.smithi162.stdout:1661: done (3 left) 2024-09-16T15:07:59.857 INFO:tasks.rados.rados.0.smithi162.stdout:1662: copy_from oid 44 from oid 49 current snap is 153 2024-09-16T15:07:59.857 INFO:tasks.rados.rados.0.smithi162.stdout:1658: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:07:59.995 INFO:tasks.rados.rados.0.smithi162.stdout:1659: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: finishing write tid 1 to smithi16234619-18 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: finishing write tid 2 to smithi16234619-18 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: finishing write tid 3 to smithi16234619-18 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: finishing write tid 4 to smithi16234619-18 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: oid 18 updating version 0 to 568 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: oid 18 updating version 568 to 569 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:1660: oid 18 updating version 569 to 570 2024-09-16T15:07:59.996 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 570 (ObjNum 600 snap 153 seq_num 600) dirty exists 2024-09-16T15:07:59.997 INFO:tasks.rados.rados.0.smithi162.stdout:1660: left oid 18 (ObjNum 600 snap 153 seq_num 600) 2024-09-16T15:07:59.997 INFO:tasks.rados.rados.0.smithi162.stdout:1658: done (3 left) 2024-09-16T15:07:59.997 INFO:tasks.rados.rados.0.smithi162.stdout:1659: done (2 left) 2024-09-16T15:07:59.997 INFO:tasks.rados.rados.0.smithi162.stdout:1660: done (1 left) 2024-09-16T15:07:59.997 INFO:tasks.rados.rados.0.smithi162.stdout:1663: snap_remove snap 149 2024-09-16T15:08:00.865 INFO:tasks.rados.rados.0.smithi162.stdout:1663: done (1 left) 2024-09-16T15:08:00.865 INFO:tasks.rados.rados.0.smithi162.stdout:1664: write (excl) oid 6 current snap is 153 2024-09-16T15:08:00.866 INFO:tasks.rados.rados.0.smithi162.stdout:1664: seq_num 601 ranges {501007=416536,1635678=465329} 2024-09-16T15:08:00.871 INFO:tasks.rados.rados.0.smithi162.stdout:1664: writing smithi16234619-6 from 501007 to 917543 tid 1 2024-09-16T15:08:00.878 INFO:tasks.rados.rados.0.smithi162.stdout:1664: writing smithi16234619-6 from 1635678 to 2101007 tid 2 2024-09-16T15:08:00.878 INFO:tasks.rados.rados.0.smithi162.stdout:1662: finishing copy_from to smithi16234619-44 2024-09-16T15:08:00.879 INFO:tasks.rados.rados.0.smithi162.stdout:1662: got expected ENOENT (src dne) 2024-09-16T15:08:00.879 INFO:tasks.rados.rados.0.smithi162.stdout:1662: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:08:00.879 INFO:tasks.rados.rados.0.smithi162.stdout:1662: done (1 left) 2024-09-16T15:08:00.879 INFO:tasks.rados.rados.0.smithi162.stdout:1665: read oid 13 snap -1 2024-09-16T15:08:00.879 INFO:tasks.rados.rados.0.smithi162.stdout:1665: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:08:00.880 INFO:tasks.rados.rados.0.smithi162.stdout:1666: write (excl) oid 46 current snap is 153 2024-09-16T15:08:00.880 INFO:tasks.rados.rados.0.smithi162.stdout:1666: seq_num 602 ranges {517814=491785,1577807=540007} 2024-09-16T15:08:00.885 INFO:tasks.rados.rados.0.smithi162.stdout:1666: writing smithi16234619-46 from 517814 to 1009599 tid 1 2024-09-16T15:08:00.892 INFO:tasks.rados.rados.0.smithi162.stdout:1666: writing smithi16234619-46 from 1577807 to 2117814 tid 2 2024-09-16T15:08:00.892 INFO:tasks.rados.rados.0.smithi162.stdout:1667: rollback oid 24 current snap is 153 2024-09-16T15:08:00.892 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 152 2024-09-16T15:08:00.892 INFO:tasks.rados.rados.0.smithi162.stdout:1665: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:08:01.006 INFO:tasks.rados.rados.0.smithi162.stdout:1667: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:08:01.006 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: finishing write tid 1 to smithi16234619-6 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: finishing write tid 2 to smithi16234619-6 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: finishing write tid 3 to smithi16234619-6 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: finishing write tid 4 to smithi16234619-6 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: oid 6 updating version 0 to 538 2024-09-16T15:08:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:1664: oid 6 updating version 538 to 539 2024-09-16T15:08:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:1664: oid 6 updating version 539 to 540 2024-09-16T15:08:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 540 (ObjNum 601 snap 153 seq_num 601) dirty exists 2024-09-16T15:08:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:1664: left oid 6 (ObjNum 601 snap 153 seq_num 601) 2024-09-16T15:08:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:1666: finishing write tid 1 to smithi16234619-46 2024-09-16T15:08:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:1664: done (3 left) 2024-09-16T15:08:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:1665: done (2 left) 2024-09-16T15:08:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:1667: done (1 left) 2024-09-16T15:08:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:1668: snap_create 2024-09-16T15:08:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:1666: finishing write tid 2 to smithi16234619-46 2024-09-16T15:08:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:1666: finishing write tid 3 to smithi16234619-46 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:1666: finishing write tid 4 to smithi16234619-46 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:1666: oid 46 updating version 0 to 255 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:1666: oid 46 version 255 is already newer than 254 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:1666: oid 46 updating version 255 to 256 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 256 (ObjNum 602 snap 153 seq_num 602) dirty exists 2024-09-16T15:08:01.010 INFO:tasks.rados.rados.0.smithi162.stdout:1666: left oid 46 (ObjNum 602 snap 153 seq_num 602) 2024-09-16T15:08:02.137 INFO:tasks.rados.rados.0.smithi162.stdout:1666: done (1 left) 2024-09-16T15:08:02.138 INFO:tasks.rados.rados.0.smithi162.stdout:1668: done (0 left) 2024-09-16T15:08:02.138 INFO:tasks.rados.rados.0.smithi162.stdout:1669: write (excl) oid 34 current snap is 154 2024-09-16T15:08:02.138 INFO:tasks.rados.rados.0.smithi162.stdout:1669: seq_num 603 ranges {534621=567034,1636289=498332} 2024-09-16T15:08:02.143 INFO:tasks.rados.rados.0.smithi162.stdout:1669: writing smithi16234619-34 from 534621 to 1101655 tid 1 2024-09-16T15:08:02.149 INFO:tasks.rados.rados.0.smithi162.stdout:1669: writing smithi16234619-34 from 1636289 to 2134621 tid 2 2024-09-16T15:08:02.149 INFO:tasks.rados.rados.0.smithi162.stdout:1670: copy_from oid 43 from oid 40 current snap is 154 2024-09-16T15:08:02.149 INFO:tasks.rados.rados.0.smithi162.stdout:1671: write oid 36 current snap is 154 2024-09-16T15:08:02.149 INFO:tasks.rados.rados.0.smithi162.stdout:1671: seq_num 604 ranges {551428=642283,1694771=456657} 2024-09-16T15:08:02.156 INFO:tasks.rados.rados.0.smithi162.stdout:1671: writing smithi16234619-36 from 551428 to 1193711 tid 1 2024-09-16T15:08:02.161 INFO:tasks.rados.rados.0.smithi162.stdout:1671: writing smithi16234619-36 from 1694771 to 2151428 tid 2 2024-09-16T15:08:02.161 INFO:tasks.rados.rados.0.smithi162.stdout:1672: delete oid 3 current snap is 154 2024-09-16T15:08:02.163 INFO:tasks.rados.rados.0.smithi162.stdout:1672: done (3 left) 2024-09-16T15:08:02.163 INFO:tasks.rados.rados.0.smithi162.stdout:1673: read oid 15 snap 153 2024-09-16T15:08:02.163 INFO:tasks.rados.rados.0.smithi162.stdout:1673: expect (ObjNum 545 snap 140 seq_num 545) 2024-09-16T15:08:02.164 INFO:tasks.rados.rados.0.smithi162.stdout:1674: copy_from oid 24 from oid 14 current snap is 154 2024-09-16T15:08:02.164 INFO:tasks.rados.rados.0.smithi162.stdout:1675: rollback oid 42 current snap is 154 2024-09-16T15:08:02.164 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 153 2024-09-16T15:08:02.164 INFO:tasks.rados.rados.0.smithi162.stdout:1676: write oid 3 current snap is 154 2024-09-16T15:08:02.164 INFO:tasks.rados.rados.0.smithi162.stdout:1676: seq_num 606 ranges {585042=792781,2095382=89660} 2024-09-16T15:08:02.171 INFO:tasks.rados.rados.0.smithi162.stdout:1676: writing smithi16234619-3 from 585042 to 1377823 tid 1 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1676: writing smithi16234619-3 from 2095382 to 2185042 tid 2 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1677: read oid 10 snap -1 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1677: expect deleted 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1678: copy_from oid 26 from oid 17 current snap is 154 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1675: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1675: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:1675: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:08:02.177 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 757 (ObjNum 572 snap 143 seq_num 572) dirty exists 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1675: done (8 left) 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1679: read oid 16 snap -1 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1679: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1680: read oid 9 snap 147 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1680: expect deleted 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1681: rollback oid 20 current snap is 154 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 153 2024-09-16T15:08:02.178 INFO:tasks.rados.rados.0.smithi162.stdout:1673: expect (ObjNum 545 snap 140 seq_num 545) 2024-09-16T15:08:02.300 INFO:tasks.rados.rados.0.smithi162.stdout:1669: finishing write tid 1 to smithi16234619-34 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:1681: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:1670: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:1681: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:1681: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 555 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:08:02.301 INFO:tasks.rados.rados.0.smithi162.stdout:1679: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:1670: finishing copy_from to smithi16234619-43 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 279 (ObjNum 492 snap 121 seq_num 492) dirty exists 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:1676: finishing write tid 1 to smithi16234619-3 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:1676: finishing write tid 2 to smithi16234619-3 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:1676: finishing write tid 3 to smithi16234619-3 2024-09-16T15:08:02.449 INFO:tasks.rados.rados.0.smithi162.stdout:1676: finishing write tid 4 to smithi16234619-3 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1676: oid 3 updating version 0 to 73 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1676: oid 3 updating version 73 to 74 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1676: oid 3 version 74 is already newer than 72 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 74 (ObjNum 606 snap 154 seq_num 606) dirty exists 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1676: left oid 3 (ObjNum 606 snap 154 seq_num 606) 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: finishing write tid 2 to smithi16234619-34 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: finishing write tid 3 to smithi16234619-34 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: finishing write tid 4 to smithi16234619-34 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: oid 34 updating version 0 to 607 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: oid 34 updating version 607 to 608 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: oid 34 updating version 608 to 609 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 609 (ObjNum 603 snap 154 seq_num 603) dirty exists 2024-09-16T15:08:02.450 INFO:tasks.rados.rados.0.smithi162.stdout:1669: left oid 34 (ObjNum 603 snap 154 seq_num 603) 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: finishing write tid 1 to smithi16234619-36 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: finishing write tid 2 to smithi16234619-36 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: finishing write tid 3 to smithi16234619-36 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: finishing write tid 4 to smithi16234619-36 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: oid 36 updating version 0 to 611 2024-09-16T15:08:02.451 INFO:tasks.rados.rados.0.smithi162.stdout:1671: oid 36 updating version 611 to 612 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1671: oid 36 updating version 612 to 613 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 613 (ObjNum 604 snap 154 seq_num 604) dirty exists 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1671: left oid 36 (ObjNum 604 snap 154 seq_num 604) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1674: finishing copy_from to smithi16234619-24 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1674: got expected ENOENT (src dne) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1674: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1678: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1678: finishing copy_from to smithi16234619-26 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 730 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1669: done (10 left) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1670: done (9 left) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1671: done (8 left) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1673: done (7 left) 2024-09-16T15:08:02.452 INFO:tasks.rados.rados.0.smithi162.stdout:1674: done (6 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1676: done (5 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1677: done (4 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1678: done (3 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1679: done (2 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1680: done (1 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1681: done (0 left) 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1682: rollback oid 25 current snap is 154 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 153 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1683: read oid 48 snap -1 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1683: expect deleted 2024-09-16T15:08:02.453 INFO:tasks.rados.rados.0.smithi162.stdout:1684: snap_remove snap 147 2024-09-16T15:08:04.011 INFO:tasks.rados.rados.0.smithi162.stdout:1684: done (2 left) 2024-09-16T15:08:04.011 INFO:tasks.rados.rados.0.smithi162.stdout:1685: copy_from oid 20 from oid 41 current snap is 154 2024-09-16T15:08:04.011 INFO:tasks.rados.rados.0.smithi162.stdout:1682: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:1682: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:1682: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 617 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:1682: done (2 left) 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:1683: done (1 left) 2024-09-16T15:08:04.012 INFO:tasks.rados.rados.0.smithi162.stdout:1686: delete oid 21 current snap is 154 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1686: done (1 left) 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1687: rollback oid 2 current snap is 154 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 153 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1688: read oid 1 snap -1 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1688: expect (ObjNum 598 snap 153 seq_num 598) 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1689: read oid 34 snap -1 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1689: expect (ObjNum 603 snap 154 seq_num 603) 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1690: write (excl) oid 35 current snap is 154 2024-09-16T15:08:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:1690: seq_num 608 ranges {618656=543279,1812346=406310} 2024-09-16T15:08:04.027 INFO:tasks.rados.rados.0.smithi162.stdout:1690: writing smithi16234619-35 from 618656 to 1161935 tid 1 2024-09-16T15:08:04.033 INFO:tasks.rados.rados.0.smithi162.stdout:1690: writing smithi16234619-35 from 1812346 to 2218656 tid 2 2024-09-16T15:08:04.033 INFO:tasks.rados.rados.0.smithi162.stdout:1691: delete oid 42 current snap is 154 2024-09-16T15:08:04.033 INFO:tasks.rados.rados.0.smithi162.stdout:1688: expect (ObjNum 598 snap 153 seq_num 598) 2024-09-16T15:08:04.149 INFO:tasks.rados.rados.0.smithi162.stdout:1687: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:08:04.149 INFO:tasks.rados.rados.0.smithi162.stdout:1689: expect (ObjNum 603 snap 154 seq_num 603) 2024-09-16T15:08:04.267 INFO:tasks.rados.rados.0.smithi162.stdout:1687: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:08:04.267 INFO:tasks.rados.rados.0.smithi162.stdout:1687: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 755 (ObjNum 490 snap 121 seq_num 490) dirty exists 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: finishing write tid 1 to smithi16234619-35 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: finishing write tid 2 to smithi16234619-35 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: finishing write tid 3 to smithi16234619-35 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: finishing write tid 4 to smithi16234619-35 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: oid 35 updating version 0 to 542 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: oid 35 version 542 is already newer than 541 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: oid 35 updating version 542 to 543 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 543 (ObjNum 608 snap 154 seq_num 608) dirty exists 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1690: left oid 35 (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1685: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:1685: finishing copy_from to smithi16234619-20 2024-09-16T15:08:04.268 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 321 (ObjNum 553 snap 141 seq_num 553) dirty exists 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1685: done (5 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1687: done (4 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1688: done (3 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1689: done (2 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1690: done (1 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1691: done (0 left) 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1692: read oid 39 snap 153 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1692: expect deleted 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1693: write (excl) oid 14 current snap is 154 2024-09-16T15:08:04.269 INFO:tasks.rados.rados.0.smithi162.stdout:1693: seq_num 610 ranges {652270=410130,1529310=493867} 2024-09-16T15:08:04.272 INFO:tasks.rados.rados.0.smithi162.stdout:1693: writing smithi16234619-14 from 652270 to 1062400 tid 1 2024-09-16T15:08:04.278 INFO:tasks.rados.rados.0.smithi162.stdout:1693: writing smithi16234619-14 from 1529310 to 2023177 tid 2 2024-09-16T15:08:04.278 INFO:tasks.rados.rados.0.smithi162.stdout:1694: read oid 37 snap -1 2024-09-16T15:08:04.278 INFO:tasks.rados.rados.0.smithi162.stdout:1694: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:08:04.279 INFO:tasks.rados.rados.0.smithi162.stdout:1692: done (2 left) 2024-09-16T15:08:04.279 INFO:tasks.rados.rados.0.smithi162.stdout:1695: read oid 41 snap -1 2024-09-16T15:08:04.279 INFO:tasks.rados.rados.0.smithi162.stdout:1695: expect (ObjNum 553 snap 141 seq_num 553) 2024-09-16T15:08:04.279 INFO:tasks.rados.rados.0.smithi162.stdout:1696: delete oid 31 current snap is 154 2024-09-16T15:08:04.283 INFO:tasks.rados.rados.0.smithi162.stdout:1694: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:08:04.427 INFO:tasks.rados.rados.0.smithi162.stdout:1695: expect (ObjNum 553 snap 141 seq_num 553) 2024-09-16T15:08:04.528 INFO:tasks.rados.rados.0.smithi162.stdout:1693: finishing write tid 1 to smithi16234619-14 2024-09-16T15:08:04.528 INFO:tasks.rados.rados.0.smithi162.stdout:1693: finishing write tid 2 to smithi16234619-14 2024-09-16T15:08:04.528 INFO:tasks.rados.rados.0.smithi162.stdout:1693: finishing write tid 3 to smithi16234619-14 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: finishing write tid 4 to smithi16234619-14 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: oid 14 updating version 0 to 628 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: oid 14 version 628 is already newer than 627 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: oid 14 version 628 is already newer than 626 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 628 (ObjNum 610 snap 154 seq_num 610) dirty exists 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: left oid 14 (ObjNum 610 snap 154 seq_num 610) 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1693: done (3 left) 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1694: done (2 left) 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1695: done (1 left) 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1696: done (0 left) 2024-09-16T15:08:04.529 INFO:tasks.rados.rados.0.smithi162.stdout:1697: snap_remove snap 152 2024-09-16T15:08:05.062 INFO:tasks.rados.rados.0.smithi162.stdout:1697: done (0 left) 2024-09-16T15:08:05.063 INFO:tasks.rados.rados.0.smithi162.stdout:1698: write oid 2 current snap is 154 2024-09-16T15:08:05.063 INFO:tasks.rados.rados.0.smithi162.stdout:1698: seq_num 612 ranges {685884=560628,2046274=239610} 2024-09-16T15:08:05.070 INFO:tasks.rados.rados.0.smithi162.stdout:1698: writing smithi16234619-2 from 685884 to 1246512 tid 1 2024-09-16T15:08:05.076 INFO:tasks.rados.rados.0.smithi162.stdout:1698: writing smithi16234619-2 from 2046274 to 2285884 tid 2 2024-09-16T15:08:05.076 INFO:tasks.rados.rados.0.smithi162.stdout:1699: snap_remove snap 153 2024-09-16T15:08:06.066 INFO:tasks.rados.rados.0.smithi162.stdout:1699: done (1 left) 2024-09-16T15:08:06.066 INFO:tasks.rados.rados.0.smithi162.stdout:1700: write oid 27 current snap is 154 2024-09-16T15:08:06.066 INFO:tasks.rados.rados.0.smithi162.stdout:1700: seq_num 613 ranges {702691=635877,2104756=197935} 2024-09-16T15:08:06.074 INFO:tasks.rados.rados.0.smithi162.stdout:1700: writing smithi16234619-27 from 702691 to 1338568 tid 1 2024-09-16T15:08:06.080 INFO:tasks.rados.rados.0.smithi162.stdout:1700: writing smithi16234619-27 from 2104756 to 2302691 tid 2 2024-09-16T15:08:06.080 INFO:tasks.rados.rados.0.smithi162.stdout:1701: copy_from oid 29 from oid 6 current snap is 154 2024-09-16T15:08:06.080 INFO:tasks.rados.rados.0.smithi162.stdout:1702: snap_create 2024-09-16T15:08:06.081 INFO:tasks.rados.rados.0.smithi162.stdout:1698: finishing write tid 1 to smithi16234619-2 2024-09-16T15:08:06.081 INFO:tasks.rados.rados.0.smithi162.stdout:1698: finishing write tid 2 to smithi16234619-2 2024-09-16T15:08:06.081 INFO:tasks.rados.rados.0.smithi162.stdout:1698: finishing write tid 3 to smithi16234619-2 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:1698: finishing write tid 4 to smithi16234619-2 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:1698: oid 2 updating version 0 to 770 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:1698: oid 2 version 770 is already newer than 768 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:1698: oid 2 version 770 is already newer than 769 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 770 (ObjNum 612 snap 154 seq_num 612) dirty exists 2024-09-16T15:08:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:1698: left oid 2 (ObjNum 612 snap 154 seq_num 612) 2024-09-16T15:08:06.102 INFO:tasks.rados.rados.0.smithi162.stdout:1700: finishing write tid 1 to smithi16234619-27 2024-09-16T15:08:06.102 INFO:tasks.rados.rados.0.smithi162.stdout:1700: finishing write tid 2 to smithi16234619-27 2024-09-16T15:08:06.104 INFO:tasks.rados.rados.0.smithi162.stdout:1700: finishing write tid 3 to smithi16234619-27 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:1700: finishing write tid 4 to smithi16234619-27 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:1700: oid 27 updating version 0 to 594 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:1700: oid 27 updating version 594 to 595 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:1700: oid 27 version 595 is already newer than 593 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 595 (ObjNum 613 snap 154 seq_num 613) dirty exists 2024-09-16T15:08:06.105 INFO:tasks.rados.rados.0.smithi162.stdout:1700: left oid 27 (ObjNum 613 snap 154 seq_num 613) 2024-09-16T15:08:06.113 INFO:tasks.rados.rados.0.smithi162.stdout:1701: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:08:06.115 INFO:tasks.rados.rados.0.smithi162.stdout:1701: finishing copy_from to smithi16234619-29 2024-09-16T15:08:06.115 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 540 (ObjNum 601 snap 153 seq_num 601) dirty exists 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:1698: done (3 left) 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:1700: done (2 left) 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:1701: done (1 left) 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:1702: done (0 left) 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:1703: rollback oid 24 current snap is 155 2024-09-16T15:08:07.486 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 154 2024-09-16T15:08:07.487 INFO:tasks.rados.rados.0.smithi162.stdout:1704: read oid 33 snap -1 2024-09-16T15:08:07.487 INFO:tasks.rados.rados.0.smithi162.stdout:1704: expect (ObjNum 557 snap 141 seq_num 557) 2024-09-16T15:08:07.487 INFO:tasks.rados.rados.0.smithi162.stdout:1705: delete oid 21 current snap is 155 2024-09-16T15:08:07.488 INFO:tasks.rados.rados.0.smithi162.stdout:1704: expect (ObjNum 557 snap 141 seq_num 557) 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1703: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 0 (ObjNum 193 snap 0 seq_num 862635424) dirty dne 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1703: done (2 left) 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1704: done (1 left) 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1705: done (0 left) 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1706: write oid 1 current snap is 155 2024-09-16T15:08:07.597 INFO:tasks.rados.rados.0.smithi162.stdout:1706: seq_num 615 ranges {736305=786375,2105367=230938} 2024-09-16T15:08:07.605 INFO:tasks.rados.rados.0.smithi162.stdout:1706: writing smithi16234619-1 from 736305 to 1522680 tid 1 2024-09-16T15:08:07.609 INFO:tasks.rados.rados.0.smithi162.stdout:1706: writing smithi16234619-1 from 2105367 to 2336305 tid 2 2024-09-16T15:08:07.610 INFO:tasks.rados.rados.0.smithi162.stdout:1707: rollback oid 31 current snap is 155 2024-09-16T15:08:07.610 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 154 2024-09-16T15:08:07.610 INFO:tasks.rados.rados.0.smithi162.stdout:1708: copy_from oid 24 from oid 21 current snap is 155 2024-09-16T15:08:07.610 INFO:tasks.rados.rados.0.smithi162.stdout:1709: write oid 44 current snap is 155 2024-09-16T15:08:07.610 INFO:tasks.rados.rados.0.smithi162.stdout:1709: seq_num 616 ranges {753112=577977,1880202=472910} 2024-09-16T15:08:07.616 INFO:tasks.rados.rados.0.smithi162.stdout:1709: writing smithi16234619-44 from 753112 to 1331089 tid 1 2024-09-16T15:08:07.622 INFO:tasks.rados.rados.0.smithi162.stdout:1709: writing smithi16234619-44 from 1880202 to 2353112 tid 2 2024-09-16T15:08:07.622 INFO:tasks.rados.rados.0.smithi162.stdout:1710: delete oid 8 current snap is 155 2024-09-16T15:08:07.623 INFO:tasks.rados.rados.0.smithi162.stdout:1707: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:08:07.623 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:08:07.623 INFO:tasks.rados.rados.0.smithi162.stdout:1708: finishing copy_from to smithi16234619-24 2024-09-16T15:08:07.623 INFO:tasks.rados.rados.0.smithi162.stdout:1708: got expected ENOENT (src dne) 2024-09-16T15:08:07.623 INFO:tasks.rados.rados.0.smithi162.stdout:1708: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:08:07.626 INFO:tasks.rados.rados.0.smithi162.stdout:1707: done (4 left) 2024-09-16T15:08:07.626 INFO:tasks.rados.rados.0.smithi162.stdout:1708: done (3 left) 2024-09-16T15:08:07.626 INFO:tasks.rados.rados.0.smithi162.stdout:1710: done (2 left) 2024-09-16T15:08:07.626 INFO:tasks.rados.rados.0.smithi162.stdout:1711: write oid 3 current snap is 155 2024-09-16T15:08:07.626 INFO:tasks.rados.rados.0.smithi162.stdout:1711: seq_num 618 ranges {786726=728475,2280813=105913} 2024-09-16T15:08:07.634 INFO:tasks.rados.rados.0.smithi162.stdout:1711: writing smithi16234619-3 from 786726 to 1515201 tid 1 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1711: writing smithi16234619-3 from 2280813 to 2386726 tid 2 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1712: snap_create 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: finishing write tid 1 to smithi16234619-1 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: finishing write tid 2 to smithi16234619-1 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: finishing write tid 3 to smithi16234619-1 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: finishing write tid 4 to smithi16234619-1 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: oid 1 updating version 0 to 598 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: oid 1 updating version 598 to 599 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: oid 1 version 599 is already newer than 597 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 599 (ObjNum 615 snap 155 seq_num 615) dirty exists 2024-09-16T15:08:07.640 INFO:tasks.rados.rados.0.smithi162.stdout:1706: left oid 1 (ObjNum 615 snap 155 seq_num 615) 2024-09-16T15:08:07.644 INFO:tasks.rados.rados.0.smithi162.stdout:1709: finishing write tid 1 to smithi16234619-44 2024-09-16T15:08:07.644 INFO:tasks.rados.rados.0.smithi162.stdout:1709: finishing write tid 2 to smithi16234619-44 2024-09-16T15:08:07.644 INFO:tasks.rados.rados.0.smithi162.stdout:1709: finishing write tid 3 to smithi16234619-44 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:1709: finishing write tid 4 to smithi16234619-44 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:1709: oid 44 updating version 0 to 544 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:1709: oid 44 updating version 544 to 546 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:1709: oid 44 version 546 is already newer than 545 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 546 (ObjNum 616 snap 155 seq_num 616) dirty exists 2024-09-16T15:08:07.645 INFO:tasks.rados.rados.0.smithi162.stdout:1709: left oid 44 (ObjNum 616 snap 155 seq_num 616) 2024-09-16T15:08:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1711: finishing write tid 1 to smithi16234619-3 2024-09-16T15:08:07.654 INFO:tasks.rados.rados.0.smithi162.stdout:1711: finishing write tid 2 to smithi16234619-3 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: finishing write tid 3 to smithi16234619-3 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: finishing write tid 4 to smithi16234619-3 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: oid 3 updating version 0 to 78 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: oid 3 version 78 is already newer than 77 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: oid 3 version 78 is already newer than 76 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 78 (ObjNum 618 snap 155 seq_num 618) dirty exists 2024-09-16T15:08:07.658 INFO:tasks.rados.rados.0.smithi162.stdout:1711: left oid 3 (ObjNum 618 snap 155 seq_num 618) 2024-09-16T15:08:07.839 INFO:tasks.rados.rados.0.smithi162.stdout:1706: done (3 left) 2024-09-16T15:08:07.839 INFO:tasks.rados.rados.0.smithi162.stdout:1709: done (2 left) 2024-09-16T15:08:07.839 INFO:tasks.rados.rados.0.smithi162.stdout:1711: done (1 left) 2024-09-16T15:08:07.839 INFO:tasks.rados.rados.0.smithi162.stdout:1712: done (0 left) 2024-09-16T15:08:07.839 INFO:tasks.rados.rados.0.smithi162.stdout:1713: snap_remove snap 155 2024-09-16T15:08:09.065 INFO:tasks.rados.rados.0.smithi162.stdout:1713: done (0 left) 2024-09-16T15:08:09.065 INFO:tasks.rados.rados.0.smithi162.stdout:1714: read oid 35 snap -1 2024-09-16T15:08:09.065 INFO:tasks.rados.rados.0.smithi162.stdout:1714: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:09.065 INFO:tasks.rados.rados.0.smithi162.stdout:1715: delete oid 17 current snap is 156 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1715: done (1 left) 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1716: read oid 18 snap 154 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1716: expect (ObjNum 600 snap 153 seq_num 600) 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1717: read oid 32 snap 154 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1717: expect deleted 2024-09-16T15:08:09.069 INFO:tasks.rados.rados.0.smithi162.stdout:1718: read oid 30 snap 154 2024-09-16T15:08:09.070 INFO:tasks.rados.rados.0.smithi162.stdout:1718: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:08:09.070 INFO:tasks.rados.rados.0.smithi162.stdout:1719: write (excl) oid 36 current snap is 156 2024-09-16T15:08:09.070 INFO:tasks.rados.rados.0.smithi162.stdout:1719: seq_num 620 ranges {420340=478973,1597777=512212} 2024-09-16T15:08:09.074 INFO:tasks.rados.rados.0.smithi162.stdout:1719: writing smithi16234619-36 from 420340 to 899313 tid 1 2024-09-16T15:08:09.081 INFO:tasks.rados.rados.0.smithi162.stdout:1719: writing smithi16234619-36 from 1597777 to 2109989 tid 2 2024-09-16T15:08:09.081 INFO:tasks.rados.rados.0.smithi162.stdout:1720: snap_create 2024-09-16T15:08:09.082 INFO:tasks.rados.rados.0.smithi162.stdout:1714: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:09.202 INFO:tasks.rados.rados.0.smithi162.stdout:1716: expect (ObjNum 600 snap 153 seq_num 600) 2024-09-16T15:08:09.270 INFO:tasks.rados.rados.0.smithi162.stdout:1718: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:08:09.356 INFO:tasks.rados.rados.0.smithi162.stdout:1719: finishing write tid 1 to smithi16234619-36 2024-09-16T15:08:09.356 INFO:tasks.rados.rados.0.smithi162.stdout:1719: finishing write tid 2 to smithi16234619-36 2024-09-16T15:08:09.356 INFO:tasks.rados.rados.0.smithi162.stdout:1719: finishing write tid 3 to smithi16234619-36 2024-09-16T15:08:09.356 INFO:tasks.rados.rados.0.smithi162.stdout:1719: finishing write tid 4 to smithi16234619-36 2024-09-16T15:08:09.356 INFO:tasks.rados.rados.0.smithi162.stdout:1719: oid 36 updating version 0 to 644 2024-09-16T15:08:09.357 INFO:tasks.rados.rados.0.smithi162.stdout:1719: oid 36 version 644 is already newer than 643 2024-09-16T15:08:09.357 INFO:tasks.rados.rados.0.smithi162.stdout:1719: oid 36 version 644 is already newer than 642 2024-09-16T15:08:09.357 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 644 (ObjNum 620 snap 156 seq_num 620) dirty exists 2024-09-16T15:08:09.357 INFO:tasks.rados.rados.0.smithi162.stdout:1719: left oid 36 (ObjNum 620 snap 156 seq_num 620) 2024-09-16T15:08:10.074 INFO:tasks.rados.rados.0.smithi162.stdout:1714: done (5 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1716: done (4 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1717: done (3 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1718: done (2 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1719: done (1 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1720: done (0 left) 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1721: rollback oid 12 current snap is 157 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 156 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1722: read oid 39 snap 156 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1722: expect deleted 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1723: read oid 32 snap -1 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1723: expect deleted 2024-09-16T15:08:10.075 INFO:tasks.rados.rados.0.smithi162.stdout:1724: snap_remove snap 154 2024-09-16T15:08:11.081 INFO:tasks.rados.rados.0.smithi162.stdout:1724: done (3 left) 2024-09-16T15:08:11.081 INFO:tasks.rados.rados.0.smithi162.stdout:1725: copy_from oid 20 from oid 37 current snap is 157 2024-09-16T15:08:11.081 INFO:tasks.rados.rados.0.smithi162.stdout:1721: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:08:11.081 INFO:tasks.rados.rados.0.smithi162.stdout:1721: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:08:11.081 INFO:tasks.rados.rados.0.smithi162.stdout:1721: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 293 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:1721: done (3 left) 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:1722: done (2 left) 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:1723: done (1 left) 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:1726: write (excl) oid 24 current snap is 157 2024-09-16T15:08:11.082 INFO:tasks.rados.rados.0.smithi162.stdout:1726: seq_num 621 ranges {437147=554222,1656259=655870} 2024-09-16T15:08:11.086 INFO:tasks.rados.rados.0.smithi162.stdout:1726: writing smithi16234619-24 from 437147 to 991369 tid 1 2024-09-16T15:08:11.094 INFO:tasks.rados.rados.0.smithi162.stdout:1726: writing smithi16234619-24 from 1656259 to 2312129 tid 2 2024-09-16T15:08:11.095 INFO:tasks.rados.rados.0.smithi162.stdout:1727: copy_from oid 41 from oid 34 current snap is 157 2024-09-16T15:08:11.096 INFO:tasks.rados.rados.0.smithi162.stdout:1728: delete oid 28 current snap is 157 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1728: done (3 left) 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1729: rollback oid 5 current snap is 157 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 156 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1730: read oid 48 snap -1 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1730: expect deleted 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1731: write oid 44 current snap is 157 2024-09-16T15:08:11.102 INFO:tasks.rados.rados.0.smithi162.stdout:1731: seq_num 623 ranges {470761=704720,1656870=659539} 2024-09-16T15:08:11.109 INFO:tasks.rados.rados.0.smithi162.stdout:1731: writing smithi16234619-44 from 470761 to 1175481 tid 1 2024-09-16T15:08:11.117 INFO:tasks.rados.rados.0.smithi162.stdout:1731: writing smithi16234619-44 from 1656870 to 2316409 tid 2 2024-09-16T15:08:11.117 INFO:tasks.rados.rados.0.smithi162.stdout:1732: write (excl) oid 9 current snap is 157 2024-09-16T15:08:11.117 INFO:tasks.rados.rados.0.smithi162.stdout:1732: seq_num 624 ranges {487568=496322,1431705=519550} 2024-09-16T15:08:11.123 INFO:tasks.rados.rados.0.smithi162.stdout:1732: writing smithi16234619-9 from 487568 to 983890 tid 1 2024-09-16T15:08:11.130 INFO:tasks.rados.rados.0.smithi162.stdout:1732: writing smithi16234619-9 from 1431705 to 1951255 tid 2 2024-09-16T15:08:11.130 INFO:tasks.rados.rados.0.smithi162.stdout:1733: write oid 39 current snap is 157 2024-09-16T15:08:11.130 INFO:tasks.rados.rados.0.smithi162.stdout:1733: seq_num 625 ranges {504375=571571,1490187=663208} 2024-09-16T15:08:11.137 INFO:tasks.rados.rados.0.smithi162.stdout:1733: writing smithi16234619-39 from 504375 to 1075946 tid 1 2024-09-16T15:08:11.144 INFO:tasks.rados.rados.0.smithi162.stdout:1733: writing smithi16234619-39 from 1490187 to 2153395 tid 2 2024-09-16T15:08:11.144 INFO:tasks.rados.rados.0.smithi162.stdout:1734: write oid 32 current snap is 157 2024-09-16T15:08:11.144 INFO:tasks.rados.rados.0.smithi162.stdout:1729: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:08:11.144 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 0 (ObjNum 65 snap 0 seq_num 1953066355) dirty dne 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: finishing write tid 1 to smithi16234619-24 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: finishing write tid 2 to smithi16234619-24 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: finishing write tid 3 to smithi16234619-24 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: finishing write tid 4 to smithi16234619-24 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: oid 24 updating version 0 to 783 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: oid 24 updating version 783 to 785 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: oid 24 version 785 is already newer than 784 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 785 (ObjNum 621 snap 157 seq_num 621) dirty exists 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1726: left oid 24 (ObjNum 621 snap 157 seq_num 621) 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1727: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1725: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:08:11.145 INFO:tasks.rados.rados.0.smithi162.stdout:1734: seq_num 626 ranges {521182=646820,1948669=523219} 2024-09-16T15:08:11.150 INFO:tasks.rados.rados.0.smithi162.stdout:1734: writing smithi16234619-32 from 521182 to 1168002 tid 1 2024-09-16T15:08:11.159 INFO:tasks.rados.rados.0.smithi162.stdout:1734: writing smithi16234619-32 from 1948669 to 2471888 tid 2 2024-09-16T15:08:11.159 INFO:tasks.rados.rados.0.smithi162.stdout:1727: finishing copy_from to smithi16234619-41 2024-09-16T15:08:11.159 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 609 (ObjNum 603 snap 154 seq_num 603) dirty exists 2024-09-16T15:08:11.159 INFO:tasks.rados.rados.0.smithi162.stdout:1725: finishing copy_from to smithi16234619-20 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 636 (ObjNum 543 snap 140 seq_num 543) dirty exists 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1731: finishing write tid 1 to smithi16234619-44 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1731: finishing write tid 2 to smithi16234619-44 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1725: done (8 left) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1726: done (7 left) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1727: done (6 left) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1729: done (5 left) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1730: done (4 left) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1735: read oid 22 snap -1 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1735: expect (ObjNum 588 snap 150 seq_num 588) 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1736: delete oid 40 current snap is 157 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1731: finishing write tid 3 to smithi16234619-44 2024-09-16T15:08:11.160 INFO:tasks.rados.rados.0.smithi162.stdout:1731: finishing write tid 4 to smithi16234619-44 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1731: oid 44 updating version 0 to 549 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1731: oid 44 version 549 is already newer than 547 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1731: oid 44 version 549 is already newer than 548 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 549 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1731: left oid 44 (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1732: finishing write tid 1 to smithi16234619-9 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1732: finishing write tid 2 to smithi16234619-9 2024-09-16T15:08:11.161 INFO:tasks.rados.rados.0.smithi162.stdout:1732: finishing write tid 3 to smithi16234619-9 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:1732: finishing write tid 4 to smithi16234619-9 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:1732: oid 9 updating version 0 to 603 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:1732: oid 9 version 603 is already newer than 602 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:1732: oid 9 updating version 603 to 604 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 604 (ObjNum 624 snap 157 seq_num 624) dirty exists 2024-09-16T15:08:11.162 INFO:tasks.rados.rados.0.smithi162.stdout:1732: left oid 9 (ObjNum 624 snap 157 seq_num 624) 2024-09-16T15:08:11.167 INFO:tasks.rados.rados.0.smithi162.stdout:1735: expect (ObjNum 588 snap 150 seq_num 588) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1733: finishing write tid 1 to smithi16234619-39 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1733: finishing write tid 2 to smithi16234619-39 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1731: done (5 left) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1732: done (4 left) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1735: done (3 left) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1736: done (2 left) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1737: read oid 12 snap -1 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1737: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1738: write oid 49 current snap is 157 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1733: finishing write tid 3 to smithi16234619-39 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1733: finishing write tid 4 to smithi16234619-39 2024-09-16T15:08:11.348 INFO:tasks.rados.rados.0.smithi162.stdout:1733: oid 39 updating version 0 to 766 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:1733: oid 39 updating version 766 to 768 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:1733: oid 39 version 768 is already newer than 767 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 768 (ObjNum 625 snap 157 seq_num 625) dirty exists 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:1733: left oid 39 (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:1734: finishing write tid 1 to smithi16234619-32 2024-09-16T15:08:11.349 INFO:tasks.rados.rados.0.smithi162.stdout:1738: seq_num 628 ranges {554796=797318,1949280=526888} 2024-09-16T15:08:11.354 INFO:tasks.rados.rados.0.smithi162.stdout:1738: writing smithi16234619-49 from 554796 to 1352114 tid 1 2024-09-16T15:08:11.361 INFO:tasks.rados.rados.0.smithi162.stdout:1738: writing smithi16234619-49 from 1949280 to 2476168 tid 2 2024-09-16T15:08:11.361 INFO:tasks.rados.rados.0.smithi162.stdout:1733: done (3 left) 2024-09-16T15:08:11.361 INFO:tasks.rados.rados.0.smithi162.stdout:1739: read oid 13 snap 156 2024-09-16T15:08:11.361 INFO:tasks.rados.rados.0.smithi162.stdout:1739: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1740: snap_create 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: finishing write tid 2 to smithi16234619-32 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: finishing write tid 3 to smithi16234619-32 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: finishing write tid 4 to smithi16234619-32 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: oid 32 updating version 0 to 296 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: oid 32 updating version 296 to 297 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: oid 32 updating version 297 to 298 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 298 (ObjNum 626 snap 157 seq_num 626) dirty exists 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1734: left oid 32 (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:08:11.362 INFO:tasks.rados.rados.0.smithi162.stdout:1737: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:08:11.554 INFO:tasks.rados.rados.0.smithi162.stdout:1739: expect (ObjNum 200 snap 42 seq_num 200) 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: finishing write tid 1 to smithi16234619-49 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: finishing write tid 2 to smithi16234619-49 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: finishing write tid 3 to smithi16234619-49 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: finishing write tid 4 to smithi16234619-49 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: oid 49 updating version 0 to 612 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: oid 49 version 612 is already newer than 610 2024-09-16T15:08:11.662 INFO:tasks.rados.rados.0.smithi162.stdout:1738: oid 49 version 612 is already newer than 611 2024-09-16T15:08:11.663 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 612 (ObjNum 628 snap 157 seq_num 628) dirty exists 2024-09-16T15:08:11.663 INFO:tasks.rados.rados.0.smithi162.stdout:1738: left oid 49 (ObjNum 628 snap 157 seq_num 628) 2024-09-16T15:08:12.088 INFO:tasks.rados.rados.0.smithi162.stdout:1734: done (4 left) 2024-09-16T15:08:12.088 INFO:tasks.rados.rados.0.smithi162.stdout:1737: done (3 left) 2024-09-16T15:08:12.089 INFO:tasks.rados.rados.0.smithi162.stdout:1738: done (2 left) 2024-09-16T15:08:12.089 INFO:tasks.rados.rados.0.smithi162.stdout:1739: done (1 left) 2024-09-16T15:08:12.089 INFO:tasks.rados.rados.0.smithi162.stdout:1740: done (0 left) 2024-09-16T15:08:12.089 INFO:tasks.rados.rados.0.smithi162.stdout:1741: write (excl) oid 13 current snap is 158 2024-09-16T15:08:12.089 INFO:tasks.rados.rados.0.smithi162.stdout:1741: seq_num 629 ranges {571603=472567,1607762=670546} 2024-09-16T15:08:12.094 INFO:tasks.rados.rados.0.smithi162.stdout:1741: writing smithi16234619-13 from 571603 to 1044170 tid 1 2024-09-16T15:08:12.100 INFO:tasks.rados.rados.0.smithi162.stdout:1741: writing smithi16234619-13 from 1607762 to 2278308 tid 2 2024-09-16T15:08:12.101 INFO:tasks.rados.rados.0.smithi162.stdout:1742: write oid 10 current snap is 158 2024-09-16T15:08:12.101 INFO:tasks.rados.rados.0.smithi162.stdout:1742: seq_num 630 ranges {588410=547816,1666244=414204,2484603=103807} 2024-09-16T15:08:12.106 INFO:tasks.rados.rados.0.smithi162.stdout:1742: writing smithi16234619-10 from 588410 to 1136226 tid 1 2024-09-16T15:08:12.111 INFO:tasks.rados.rados.0.smithi162.stdout:1742: writing smithi16234619-10 from 1666244 to 2080448 tid 2 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1742: writing smithi16234619-10 from 2484603 to 2588410 tid 3 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1743: read oid 1 snap 156 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1743: expect (ObjNum 615 snap 155 seq_num 615) 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1744: read oid 30 snap 157 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1744: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:08:12.115 INFO:tasks.rados.rados.0.smithi162.stdout:1745: delete oid 22 current snap is 158 2024-09-16T15:08:12.118 INFO:tasks.rados.rados.0.smithi162.stdout:1745: done (4 left) 2024-09-16T15:08:12.118 INFO:tasks.rados.rados.0.smithi162.stdout:1746: read oid 21 snap -1 2024-09-16T15:08:12.118 INFO:tasks.rados.rados.0.smithi162.stdout:1746: expect deleted 2024-09-16T15:08:12.119 INFO:tasks.rados.rados.0.smithi162.stdout:1747: snap_create 2024-09-16T15:08:12.120 INFO:tasks.rados.rados.0.smithi162.stdout:1743: expect (ObjNum 615 snap 155 seq_num 615) 2024-09-16T15:08:12.251 INFO:tasks.rados.rados.0.smithi162.stdout:1744: expect (ObjNum 518 snap 135 seq_num 518) 2024-09-16T15:08:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:1741: finishing write tid 1 to smithi16234619-13 2024-09-16T15:08:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:1741: finishing write tid 2 to smithi16234619-13 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: finishing write tid 3 to smithi16234619-13 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: finishing write tid 4 to smithi16234619-13 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: oid 13 updating version 0 to 771 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: oid 13 version 771 is already newer than 770 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: oid 13 updating version 771 to 772 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 772 (ObjNum 629 snap 158 seq_num 629) dirty exists 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1741: left oid 13 (ObjNum 629 snap 158 seq_num 629) 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: finishing write tid 1 to smithi16234619-10 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: finishing write tid 2 to smithi16234619-10 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: finishing write tid 3 to smithi16234619-10 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: finishing write tid 4 to smithi16234619-10 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: finishing write tid 5 to smithi16234619-10 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: oid 10 updating version 0 to 789 2024-09-16T15:08:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:1742: oid 10 version 789 is already newer than 788 2024-09-16T15:08:12.340 INFO:tasks.rados.rados.0.smithi162.stdout:1742: oid 10 version 789 is already newer than 787 2024-09-16T15:08:12.340 INFO:tasks.rados.rados.0.smithi162.stdout:1742: oid 10 version 789 is already newer than 786 2024-09-16T15:08:12.340 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 789 (ObjNum 630 snap 158 seq_num 630) dirty exists 2024-09-16T15:08:12.340 INFO:tasks.rados.rados.0.smithi162.stdout:1742: left oid 10 (ObjNum 630 snap 158 seq_num 630) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1741: done (5 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1742: done (4 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1743: done (3 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1744: done (2 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1746: done (1 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1747: done (0 left) 2024-09-16T15:08:12.842 INFO:tasks.rados.rados.0.smithi162.stdout:1748: delete oid 14 current snap is 159 2024-09-16T15:08:12.845 INFO:tasks.rados.rados.0.smithi162.stdout:1748: done (0 left) 2024-09-16T15:08:12.845 INFO:tasks.rados.rados.0.smithi162.stdout:1749: rollback oid 49 current snap is 159 2024-09-16T15:08:12.845 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 158 2024-09-16T15:08:12.845 INFO:tasks.rados.rados.0.smithi162.stdout:1750: copy_from oid 37 from oid 12 current snap is 159 2024-09-16T15:08:12.845 INFO:tasks.rados.rados.0.smithi162.stdout:1751: delete oid 27 current snap is 159 2024-09-16T15:08:12.850 INFO:tasks.rados.rados.0.smithi162.stdout:1749: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:08:12.851 INFO:tasks.rados.rados.0.smithi162.stdout:1749: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:08:12.851 INFO:tasks.rados.rados.0.smithi162.stdout:1749: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:08:12.851 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 616 (ObjNum 628 snap 157 seq_num 628) dirty exists 2024-09-16T15:08:12.854 INFO:tasks.rados.rados.0.smithi162.stdout:1749: done (2 left) 2024-09-16T15:08:12.854 INFO:tasks.rados.rados.0.smithi162.stdout:1751: done (1 left) 2024-09-16T15:08:12.854 INFO:tasks.rados.rados.0.smithi162.stdout:1752: write oid 46 current snap is 159 2024-09-16T15:08:12.854 INFO:tasks.rados.rados.0.smithi162.stdout:1752: seq_num 634 ranges {655638=565165,1900172=421542} 2024-09-16T15:08:12.861 INFO:tasks.rados.rados.0.smithi162.stdout:1752: writing smithi16234619-46 from 655638 to 1220803 tid 1 2024-09-16T15:08:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:1752: writing smithi16234619-46 from 1900172 to 2321714 tid 2 2024-09-16T15:08:12.868 INFO:tasks.rados.rados.0.smithi162.stdout:1753: snap_create 2024-09-16T15:08:12.886 INFO:tasks.rados.rados.0.smithi162.stdout:1752: finishing write tid 1 to smithi16234619-46 2024-09-16T15:08:12.886 INFO:tasks.rados.rados.0.smithi162.stdout:1752: finishing write tid 2 to smithi16234619-46 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: finishing write tid 3 to smithi16234619-46 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: finishing write tid 4 to smithi16234619-46 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: oid 46 updating version 0 to 259 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: oid 46 version 259 is already newer than 258 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: oid 46 version 259 is already newer than 257 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 259 (ObjNum 634 snap 159 seq_num 634) dirty exists 2024-09-16T15:08:12.890 INFO:tasks.rados.rados.0.smithi162.stdout:1752: left oid 46 (ObjNum 634 snap 159 seq_num 634) 2024-09-16T15:08:12.896 INFO:tasks.rados.rados.0.smithi162.stdout:1750: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:08:12.898 INFO:tasks.rados.rados.0.smithi162.stdout:1750: finishing copy_from to smithi16234619-37 2024-09-16T15:08:12.899 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 293 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:13.852 INFO:tasks.rados.rados.0.smithi162.stdout:1750: done (2 left) 2024-09-16T15:08:13.852 INFO:tasks.rados.rados.0.smithi162.stdout:1752: done (1 left) 2024-09-16T15:08:13.852 INFO:tasks.rados.rados.0.smithi162.stdout:1753: done (0 left) 2024-09-16T15:08:13.852 INFO:tasks.rados.rados.0.smithi162.stdout:1754: read oid 49 snap 156 2024-09-16T15:08:13.852 INFO:tasks.rados.rados.0.smithi162.stdout:1754: expect deleted 2024-09-16T15:08:13.853 INFO:tasks.rados.rados.0.smithi162.stdout:1755: snap_remove snap 157 2024-09-16T15:08:15.472 INFO:tasks.rados.rados.0.smithi162.stdout:1755: done (1 left) 2024-09-16T15:08:15.472 INFO:tasks.rados.rados.0.smithi162.stdout:1756: read oid 38 snap -1 2024-09-16T15:08:15.472 INFO:tasks.rados.rados.0.smithi162.stdout:1756: expect (ObjNum 592 snap 150 seq_num 592) 2024-09-16T15:08:15.472 INFO:tasks.rados.rados.0.smithi162.stdout:1757: rollback oid 30 current snap is 160 2024-09-16T15:08:15.473 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 156 2024-09-16T15:08:15.473 INFO:tasks.rados.rados.0.smithi162.stdout:1754: done (2 left) 2024-09-16T15:08:15.473 INFO:tasks.rados.rados.0.smithi162.stdout:1758: snap_remove snap 159 2024-09-16T15:08:16.479 INFO:tasks.rados.rados.0.smithi162.stdout:1756: expect (ObjNum 592 snap 150 seq_num 592) 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1757: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1757: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1757: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 562 (ObjNum 518 snap 135 seq_num 518) dirty exists 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1756: done (2 left) 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1757: done (1 left) 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1758: done (0 left) 2024-09-16T15:08:16.611 INFO:tasks.rados.rados.0.smithi162.stdout:1759: snap_remove snap 156 2024-09-16T15:08:17.485 INFO:tasks.rados.rados.0.smithi162.stdout:1759: done (0 left) 2024-09-16T15:08:17.485 INFO:tasks.rados.rados.0.smithi162.stdout:1760: copy_from oid 33 from oid 5 current snap is 160 2024-09-16T15:08:17.485 INFO:tasks.rados.rados.0.smithi162.stdout:1761: rollback oid 16 current snap is 160 2024-09-16T15:08:17.485 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 158 2024-09-16T15:08:17.485 INFO:tasks.rados.rados.0.smithi162.stdout:1762: snap_create 2024-09-16T15:08:17.488 INFO:tasks.rados.rados.0.smithi162.stdout:1760: finishing copy_from to smithi16234619-33 2024-09-16T15:08:17.488 INFO:tasks.rados.rados.0.smithi162.stdout:1760: got expected ENOENT (src dne) 2024-09-16T15:08:17.488 INFO:tasks.rados.rados.0.smithi162.stdout:1760: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:08:17.493 INFO:tasks.rados.rados.0.smithi162.stdout:1761: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:08:17.493 INFO:tasks.rados.rados.0.smithi162.stdout:1761: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:08:17.493 INFO:tasks.rados.rados.0.smithi162.stdout:1761: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:08:17.493 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 552 (ObjNum 444 snap 106 seq_num 444) dirty exists 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:1760: done (2 left) 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:1761: done (1 left) 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:1762: done (0 left) 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:1763: rollback oid 24 current snap is 161 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 160 2024-09-16T15:08:17.842 INFO:tasks.rados.rados.0.smithi162.stdout:1764: snap_create 2024-09-16T15:08:17.845 INFO:tasks.rados.rados.0.smithi162.stdout:1763: finishing rollback tid 0 to smithi16234619-24 2024-09-16T15:08:17.845 INFO:tasks.rados.rados.0.smithi162.stdout:1763: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:08:17.847 INFO:tasks.rados.rados.0.smithi162.stdout:1763: finishing rollback tid 2 to smithi16234619-24 2024-09-16T15:08:17.847 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 793 (ObjNum 621 snap 157 seq_num 621) dirty exists 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1763: done (1 left) 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1764: done (0 left) 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1765: read oid 35 snap -1 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1765: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1766: write (excl) oid 38 current snap is 162 2024-09-16T15:08:18.849 INFO:tasks.rados.rados.0.smithi162.stdout:1766: seq_num 635 ranges {672445=640414,1842301=681553} 2024-09-16T15:08:18.856 INFO:tasks.rados.rados.0.smithi162.stdout:1766: writing smithi16234619-38 from 672445 to 1312859 tid 1 2024-09-16T15:08:18.862 INFO:tasks.rados.rados.0.smithi162.stdout:1766: writing smithi16234619-38 from 1842301 to 2523854 tid 2 2024-09-16T15:08:18.862 INFO:tasks.rados.rados.0.smithi162.stdout:1767: snap_remove snap 160 2024-09-16T15:08:20.234 INFO:tasks.rados.rados.0.smithi162.stdout:1767: done (2 left) 2024-09-16T15:08:20.234 INFO:tasks.rados.rados.0.smithi162.stdout:1768: snap_create 2024-09-16T15:08:20.234 INFO:tasks.rados.rados.0.smithi162.stdout:1765: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:20.357 INFO:tasks.rados.rados.0.smithi162.stdout:1766: finishing write tid 1 to smithi16234619-38 2024-09-16T15:08:20.357 INFO:tasks.rados.rados.0.smithi162.stdout:1766: finishing write tid 2 to smithi16234619-38 2024-09-16T15:08:20.357 INFO:tasks.rados.rados.0.smithi162.stdout:1766: finishing write tid 3 to smithi16234619-38 2024-09-16T15:08:20.357 INFO:tasks.rados.rados.0.smithi162.stdout:1766: finishing write tid 4 to smithi16234619-38 2024-09-16T15:08:20.358 INFO:tasks.rados.rados.0.smithi162.stdout:1766: oid 38 updating version 0 to 553 2024-09-16T15:08:20.358 INFO:tasks.rados.rados.0.smithi162.stdout:1766: oid 38 updating version 553 to 555 2024-09-16T15:08:20.358 INFO:tasks.rados.rados.0.smithi162.stdout:1766: oid 38 version 555 is already newer than 554 2024-09-16T15:08:20.358 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 555 (ObjNum 635 snap 162 seq_num 635) dirty exists 2024-09-16T15:08:20.358 INFO:tasks.rados.rados.0.smithi162.stdout:1766: left oid 38 (ObjNum 635 snap 162 seq_num 635) 2024-09-16T15:08:21.252 INFO:tasks.rados.rados.0.smithi162.stdout:1765: done (2 left) 2024-09-16T15:08:21.252 INFO:tasks.rados.rados.0.smithi162.stdout:1766: done (1 left) 2024-09-16T15:08:21.252 INFO:tasks.rados.rados.0.smithi162.stdout:1768: done (0 left) 2024-09-16T15:08:21.253 INFO:tasks.rados.rados.0.smithi162.stdout:1769: snap_remove snap 162 2024-09-16T15:08:22.283 INFO:tasks.rados.rados.0.smithi162.stdout:1769: done (0 left) 2024-09-16T15:08:22.283 INFO:tasks.rados.rados.0.smithi162.stdout:1770: rollback oid 38 current snap is 163 2024-09-16T15:08:22.283 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 158 2024-09-16T15:08:22.283 INFO:tasks.rados.rados.0.smithi162.stdout:1771: snap_create 2024-09-16T15:08:22.287 INFO:tasks.rados.rados.0.smithi162.stdout:1770: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:08:22.291 INFO:tasks.rados.rados.0.smithi162.stdout:1770: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:08:22.291 INFO:tasks.rados.rados.0.smithi162.stdout:1770: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:08:22.291 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 558 (ObjNum 592 snap 150 seq_num 592) dirty exists 2024-09-16T15:08:22.843 INFO:tasks.rados.rados.0.smithi162.stdout:1770: done (1 left) 2024-09-16T15:08:22.843 INFO:tasks.rados.rados.0.smithi162.stdout:1771: done (0 left) 2024-09-16T15:08:22.843 INFO:tasks.rados.rados.0.smithi162.stdout:1772: write oid 10 current snap is 164 2024-09-16T15:08:22.844 INFO:tasks.rados.rados.0.smithi162.stdout:1772: seq_num 636 ranges {689252=715663,1900783=425211} 2024-09-16T15:08:22.852 INFO:tasks.rados.rados.0.smithi162.stdout:1772: writing smithi16234619-10 from 689252 to 1404915 tid 1 2024-09-16T15:08:22.858 INFO:tasks.rados.rados.0.smithi162.stdout:1772: writing smithi16234619-10 from 1900783 to 2325994 tid 2 2024-09-16T15:08:22.858 INFO:tasks.rados.rados.0.smithi162.stdout:1773: copy_from oid 1 from oid 23 current snap is 164 2024-09-16T15:08:22.858 INFO:tasks.rados.rados.0.smithi162.stdout:1774: read oid 27 snap -1 2024-09-16T15:08:22.858 INFO:tasks.rados.rados.0.smithi162.stdout:1774: expect deleted 2024-09-16T15:08:22.859 INFO:tasks.rados.rados.0.smithi162.stdout:1775: write oid 22 current snap is 164 2024-09-16T15:08:22.859 INFO:tasks.rados.rados.0.smithi162.stdout:1775: seq_num 637 ranges {706059=790912,1959265=685222} 2024-09-16T15:08:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:1775: writing smithi16234619-22 from 706059 to 1496971 tid 1 2024-09-16T15:08:22.873 INFO:tasks.rados.rados.0.smithi162.stdout:1775: writing smithi16234619-22 from 1959265 to 2644487 tid 2 2024-09-16T15:08:22.874 INFO:tasks.rados.rados.0.smithi162.stdout:1776: read oid 47 snap -1 2024-09-16T15:08:22.874 INFO:tasks.rados.rados.0.smithi162.stdout:1776: expect deleted 2024-09-16T15:08:22.874 INFO:tasks.rados.rados.0.smithi162.stdout:1777: write oid 43 current snap is 164 2024-09-16T15:08:22.874 INFO:tasks.rados.rados.0.smithi162.stdout:1777: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2024-09-16T15:08:22.880 INFO:tasks.rados.rados.0.smithi162.stdout:1777: writing smithi16234619-43 from 722866 to 1189027 tid 1 2024-09-16T15:08:22.885 INFO:tasks.rados.rados.0.smithi162.stdout:1777: writing smithi16234619-43 from 1617747 to 2046627 tid 2 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1777: writing smithi16234619-43 from 2587634 to 2722866 tid 3 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1778: read oid 28 snap 161 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1778: expect deleted 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1779: write (excl) oid 46 current snap is 164 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1772: finishing write tid 1 to smithi16234619-10 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1772: finishing write tid 2 to smithi16234619-10 2024-09-16T15:08:22.890 INFO:tasks.rados.rados.0.smithi162.stdout:1779: seq_num 639 ranges {739673=657763,2076229=663444} 2024-09-16T15:08:22.898 INFO:tasks.rados.rados.0.smithi162.stdout:1779: writing smithi16234619-46 from 739673 to 1397436 tid 1 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1779: writing smithi16234619-46 from 2076229 to 2739673 tid 2 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1774: done (7 left) 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1776: done (6 left) 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1780: read oid 27 snap 158 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1780: expect (ObjNum 613 snap 154 seq_num 613) 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1781: copy_from oid 36 from oid 48 current snap is 164 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1772: finishing write tid 3 to smithi16234619-10 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1772: finishing write tid 4 to smithi16234619-10 2024-09-16T15:08:22.908 INFO:tasks.rados.rados.0.smithi162.stdout:1772: oid 10 updating version 0 to 798 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1772: oid 10 updating version 798 to 799 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1772: oid 10 version 799 is already newer than 797 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 799 (ObjNum 636 snap 164 seq_num 636) dirty exists 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1772: left oid 10 (ObjNum 636 snap 164 seq_num 636) 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1772: done (7 left) 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1778: done (6 left) 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1782: read oid 26 snap 161 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1782: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1783: copy_from oid 24 from oid 33 current snap is 164 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1784: write (excl) oid 45 current snap is 164 2024-09-16T15:08:22.909 INFO:tasks.rados.rados.0.smithi162.stdout:1784: seq_num 640 ranges {756480=733012,2134711=432549} 2024-09-16T15:08:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:1784: writing smithi16234619-45 from 756480 to 1489492 tid 1 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1784: writing smithi16234619-45 from 2134711 to 2567260 tid 2 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1785: snap_create 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1777: finishing write tid 1 to smithi16234619-43 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1775: finishing write tid 1 to smithi16234619-22 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1775: finishing write tid 2 to smithi16234619-22 2024-09-16T15:08:22.923 INFO:tasks.rados.rados.0.smithi162.stdout:1782: expect (ObjNum 536 snap 139 seq_num 536) 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: finishing write tid 3 to smithi16234619-22 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: finishing write tid 4 to smithi16234619-22 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: oid 22 updating version 0 to 617 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: oid 22 updating version 617 to 618 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: oid 22 updating version 618 to 619 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 619 (ObjNum 637 snap 164 seq_num 637) dirty exists 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1775: left oid 22 (ObjNum 637 snap 164 seq_num 637) 2024-09-16T15:08:23.072 INFO:tasks.rados.rados.0.smithi162.stdout:1777: finishing write tid 2 to smithi16234619-43 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: finishing write tid 3 to smithi16234619-43 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: finishing write tid 4 to smithi16234619-43 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: finishing write tid 5 to smithi16234619-43 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: oid 43 updating version 0 to 519 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: oid 43 updating version 519 to 520 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: oid 43 version 520 is already newer than 518 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: oid 43 version 520 is already newer than 517 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 520 (ObjNum 638 snap 164 seq_num 638) dirty exists 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1777: left oid 43 (ObjNum 638 snap 164 seq_num 638) 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1773: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1781: finishing copy_from to smithi16234619-36 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1781: got expected ENOENT (src dne) 2024-09-16T15:08:23.073 INFO:tasks.rados.rados.0.smithi162.stdout:1781: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:08:23.074 INFO:tasks.rados.rados.0.smithi162.stdout:1780: expect (ObjNum 613 snap 154 seq_num 613) 2024-09-16T15:08:23.176 INFO:tasks.rados.rados.0.smithi162.stdout:1779: finishing write tid 1 to smithi16234619-46 2024-09-16T15:08:23.176 INFO:tasks.rados.rados.0.smithi162.stdout:1779: finishing write tid 2 to smithi16234619-46 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: finishing write tid 3 to smithi16234619-46 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: finishing write tid 4 to smithi16234619-46 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: oid 46 updating version 0 to 261 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: oid 46 updating version 261 to 262 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: oid 46 version 262 is already newer than 260 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 262 (ObjNum 639 snap 164 seq_num 639) dirty exists 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1779: left oid 46 (ObjNum 639 snap 164 seq_num 639) 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1773: finishing copy_from to smithi16234619-1 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 583 (ObjNum 586 snap 147 seq_num 586) dirty exists 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1783: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1783: finishing copy_from to smithi16234619-24 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 552 (ObjNum 557 snap 141 seq_num 557) dirty exists 2024-09-16T15:08:23.177 INFO:tasks.rados.rados.0.smithi162.stdout:1784: finishing write tid 1 to smithi16234619-45 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: finishing write tid 2 to smithi16234619-45 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: finishing write tid 3 to smithi16234619-45 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: finishing write tid 4 to smithi16234619-45 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: oid 45 updating version 0 to 805 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: oid 45 version 805 is already newer than 804 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: oid 45 version 805 is already newer than 803 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 805 (ObjNum 640 snap 164 seq_num 640) dirty exists 2024-09-16T15:08:23.178 INFO:tasks.rados.rados.0.smithi162.stdout:1784: left oid 45 (ObjNum 640 snap 164 seq_num 640) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1773: done (9 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1775: done (8 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1777: done (7 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1779: done (6 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1780: done (5 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1781: done (4 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1782: done (3 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1783: done (2 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1784: done (1 left) 2024-09-16T15:08:23.851 INFO:tasks.rados.rados.0.smithi162.stdout:1785: done (0 left) 2024-09-16T15:08:23.852 INFO:tasks.rados.rados.0.smithi162.stdout:1786: snap_create 2024-09-16T15:08:24.858 INFO:tasks.rados.rados.0.smithi162.stdout:1786: done (0 left) 2024-09-16T15:08:24.859 INFO:tasks.rados.rados.0.smithi162.stdout:1787: snap_remove snap 164 2024-09-16T15:08:25.861 INFO:tasks.rados.rados.0.smithi162.stdout:1787: done (0 left) 2024-09-16T15:08:25.862 INFO:tasks.rados.rados.0.smithi162.stdout:1788: rollback oid 12 current snap is 166 2024-09-16T15:08:25.862 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 158 2024-09-16T15:08:25.862 INFO:tasks.rados.rados.0.smithi162.stdout:1789: write (excl) oid 5 current snap is 166 2024-09-16T15:08:25.862 INFO:tasks.rados.rados.0.smithi162.stdout:1789: seq_num 641 ranges {773287=408261,1793193=576207} 2024-09-16T15:08:25.868 INFO:tasks.rados.rados.0.smithi162.stdout:1789: writing smithi16234619-5 from 773287 to 1181548 tid 1 2024-09-16T15:08:25.876 INFO:tasks.rados.rados.0.smithi162.stdout:1789: writing smithi16234619-5 from 1793193 to 2369400 tid 2 2024-09-16T15:08:25.876 INFO:tasks.rados.rados.0.smithi162.stdout:1790: write (excl) oid 50 current snap is 166 2024-09-16T15:08:25.876 INFO:tasks.rados.rados.0.smithi162.stdout:1790: seq_num 642 ranges {790094=483510,1851675=436218} 2024-09-16T15:08:25.884 INFO:tasks.rados.rados.0.smithi162.stdout:1790: writing smithi16234619-50 from 790094 to 1273604 tid 1 2024-09-16T15:08:25.890 INFO:tasks.rados.rados.0.smithi162.stdout:1790: writing smithi16234619-50 from 1851675 to 2287893 tid 2 2024-09-16T15:08:25.890 INFO:tasks.rados.rados.0.smithi162.stdout:1788: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:08:25.890 INFO:tasks.rados.rados.0.smithi162.stdout:1788: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:08:25.890 INFO:tasks.rados.rados.0.smithi162.stdout:1788: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:08:25.890 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 308 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:25.891 INFO:tasks.rados.rados.0.smithi162.stdout:1791: read oid 49 snap -1 2024-09-16T15:08:25.891 INFO:tasks.rados.rados.0.smithi162.stdout:1791: expect (ObjNum 628 snap 157 seq_num 628) 2024-09-16T15:08:25.891 INFO:tasks.rados.rados.0.smithi162.stdout:1788: done (3 left) 2024-09-16T15:08:25.891 INFO:tasks.rados.rados.0.smithi162.stdout:1792: write oid 24 current snap is 166 2024-09-16T15:08:25.891 INFO:tasks.rados.rados.0.smithi162.stdout:1792: seq_num 643 ranges {406901=558759,1393804=579876,2608396=198505} 2024-09-16T15:08:25.895 INFO:tasks.rados.rados.0.smithi162.stdout:1792: writing smithi16234619-24 from 406901 to 965660 tid 1 2024-09-16T15:08:25.901 INFO:tasks.rados.rados.0.smithi162.stdout:1792: writing smithi16234619-24 from 1393804 to 1973680 tid 2 2024-09-16T15:08:25.906 INFO:tasks.rados.rados.0.smithi162.stdout:1792: writing smithi16234619-24 from 2608396 to 2806901 tid 3 2024-09-16T15:08:25.906 INFO:tasks.rados.rados.0.smithi162.stdout:1793: delete oid 47 current snap is 166 2024-09-16T15:08:25.906 INFO:tasks.rados.rados.0.smithi162.stdout:1791: expect (ObjNum 628 snap 157 seq_num 628) 2024-09-16T15:08:25.987 INFO:tasks.rados.rados.0.smithi162.stdout:1789: finishing write tid 1 to smithi16234619-5 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: finishing write tid 2 to smithi16234619-5 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: finishing write tid 3 to smithi16234619-5 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: finishing write tid 4 to smithi16234619-5 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: oid 5 updating version 0 to 785 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: oid 5 version 785 is already newer than 784 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: oid 5 version 785 is already newer than 783 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 785 (ObjNum 641 snap 166 seq_num 641) dirty exists 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1789: left oid 5 (ObjNum 641 snap 166 seq_num 641) 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1790: finishing write tid 1 to smithi16234619-50 2024-09-16T15:08:25.988 INFO:tasks.rados.rados.0.smithi162.stdout:1790: finishing write tid 2 to smithi16234619-50 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: finishing write tid 3 to smithi16234619-50 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: finishing write tid 4 to smithi16234619-50 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: oid 50 updating version 0 to 656 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: oid 50 version 656 is already newer than 655 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: oid 50 version 656 is already newer than 654 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 656 (ObjNum 642 snap 166 seq_num 642) dirty exists 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: left oid 50 (ObjNum 642 snap 166 seq_num 642) 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1792: finishing write tid 1 to smithi16234619-24 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1792: finishing write tid 2 to smithi16234619-24 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1789: done (4 left) 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1790: done (3 left) 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1791: done (2 left) 2024-09-16T15:08:25.989 INFO:tasks.rados.rados.0.smithi162.stdout:1793: done (1 left) 2024-09-16T15:08:25.990 INFO:tasks.rados.rados.0.smithi162.stdout:1794: write (excl) oid 3 current snap is 166 2024-09-16T15:08:25.990 INFO:tasks.rados.rados.0.smithi162.stdout:1794: seq_num 645 ranges {440515=709257,1910768=583545} 2024-09-16T15:08:25.993 INFO:tasks.rados.rados.0.smithi162.stdout:1794: writing smithi16234619-3 from 440515 to 1149772 tid 1 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1794: writing smithi16234619-3 from 1910768 to 2494313 tid 2 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1795: read oid 18 snap -1 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1795: expect (ObjNum 600 snap 153 seq_num 600) 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1796: snap_create 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: finishing write tid 3 to smithi16234619-24 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: finishing write tid 4 to smithi16234619-24 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: finishing write tid 5 to smithi16234619-24 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: oid 24 updating version 0 to 808 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: oid 24 updating version 808 to 809 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: oid 24 updating version 809 to 810 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: oid 24 version 810 is already newer than 807 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 810 (ObjNum 643 snap 166 seq_num 643) dirty exists 2024-09-16T15:08:26.001 INFO:tasks.rados.rados.0.smithi162.stdout:1792: left oid 24 (ObjNum 643 snap 166 seq_num 643) 2024-09-16T15:08:26.006 INFO:tasks.rados.rados.0.smithi162.stdout:1795: expect (ObjNum 600 snap 153 seq_num 600) 2024-09-16T15:08:26.076 INFO:tasks.rados.rados.0.smithi162.stdout:1794: finishing write tid 1 to smithi16234619-3 2024-09-16T15:08:26.076 INFO:tasks.rados.rados.0.smithi162.stdout:1794: finishing write tid 2 to smithi16234619-3 2024-09-16T15:08:26.076 INFO:tasks.rados.rados.0.smithi162.stdout:1794: finishing write tid 3 to smithi16234619-3 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:1794: finishing write tid 4 to smithi16234619-3 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:1794: oid 3 updating version 0 to 82 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:1794: oid 3 updating version 82 to 83 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:1794: oid 3 updating version 83 to 84 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 84 (ObjNum 645 snap 166 seq_num 645) dirty exists 2024-09-16T15:08:26.077 INFO:tasks.rados.rados.0.smithi162.stdout:1794: left oid 3 (ObjNum 645 snap 166 seq_num 645) 2024-09-16T15:08:26.875 INFO:tasks.rados.rados.0.smithi162.stdout:1792: done (3 left) 2024-09-16T15:08:26.875 INFO:tasks.rados.rados.0.smithi162.stdout:1794: done (2 left) 2024-09-16T15:08:26.876 INFO:tasks.rados.rados.0.smithi162.stdout:1795: done (1 left) 2024-09-16T15:08:26.876 INFO:tasks.rados.rados.0.smithi162.stdout:1796: done (0 left) 2024-09-16T15:08:26.876 INFO:tasks.rados.rados.0.smithi162.stdout:1797: copy_from oid 1 from oid 26 current snap is 167 2024-09-16T15:08:26.876 INFO:tasks.rados.rados.0.smithi162.stdout:1798: write oid 8 current snap is 167 2024-09-16T15:08:26.876 INFO:tasks.rados.rados.0.smithi162.stdout:1798: seq_num 646 ranges {457322=784506,1969250=443556} 2024-09-16T15:08:26.882 INFO:tasks.rados.rados.0.smithi162.stdout:1798: writing smithi16234619-8 from 457322 to 1241828 tid 1 2024-09-16T15:08:26.889 INFO:tasks.rados.rados.0.smithi162.stdout:1798: writing smithi16234619-8 from 1969250 to 2412806 tid 2 2024-09-16T15:08:26.889 INFO:tasks.rados.rados.0.smithi162.stdout:1799: copy_from oid 41 from oid 42 current snap is 167 2024-09-16T15:08:26.889 INFO:tasks.rados.rados.0.smithi162.stdout:1800: delete oid 22 current snap is 167 2024-09-16T15:08:26.891 INFO:tasks.rados.rados.0.smithi162.stdout:1799: finishing copy_from to smithi16234619-41 2024-09-16T15:08:26.892 INFO:tasks.rados.rados.0.smithi162.stdout:1799: got expected ENOENT (src dne) 2024-09-16T15:08:26.892 INFO:tasks.rados.rados.0.smithi162.stdout:1799: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:08:26.892 INFO:tasks.rados.rados.0.smithi162.stdout:1799: done (3 left) 2024-09-16T15:08:26.893 INFO:tasks.rados.rados.0.smithi162.stdout:1800: done (2 left) 2024-09-16T15:08:26.893 INFO:tasks.rados.rados.0.smithi162.stdout:1801: read oid 43 snap 165 2024-09-16T15:08:26.893 INFO:tasks.rados.rados.0.smithi162.stdout:1801: expect (ObjNum 638 snap 164 seq_num 638) 2024-09-16T15:08:26.893 INFO:tasks.rados.rados.0.smithi162.stdout:1802: copy_from oid 48 from oid 6 current snap is 167 2024-09-16T15:08:26.893 INFO:tasks.rados.rados.0.smithi162.stdout:1803: snap_remove snap 161 2024-09-16T15:08:27.852 INFO:tasks.rados.rados.0.smithi162.stdout:1803: done (4 left) 2024-09-16T15:08:27.852 INFO:tasks.rados.rados.0.smithi162.stdout:1804: snap_remove snap 166 2024-09-16T15:08:28.856 INFO:tasks.rados.rados.0.smithi162.stdout:1804: done (4 left) 2024-09-16T15:08:28.856 INFO:tasks.rados.rados.0.smithi162.stdout:1805: snap_create 2024-09-16T15:08:28.856 INFO:tasks.rados.rados.0.smithi162.stdout:1801: expect (ObjNum 638 snap 164 seq_num 638) 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: finishing write tid 1 to smithi16234619-8 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: finishing write tid 2 to smithi16234619-8 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: finishing write tid 3 to smithi16234619-8 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: finishing write tid 4 to smithi16234619-8 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: oid 8 updating version 0 to 574 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: oid 8 version 574 is already newer than 573 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: oid 8 updating version 574 to 575 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 575 (ObjNum 646 snap 167 seq_num 646) dirty exists 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1798: left oid 8 (ObjNum 646 snap 167 seq_num 646) 2024-09-16T15:08:28.979 INFO:tasks.rados.rados.0.smithi162.stdout:1797: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:08:28.980 INFO:tasks.rados.rados.0.smithi162.stdout:1802: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:08:28.980 INFO:tasks.rados.rados.0.smithi162.stdout:1797: finishing copy_from to smithi16234619-1 2024-09-16T15:08:28.980 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 730 (ObjNum 536 snap 139 seq_num 536) dirty exists 2024-09-16T15:08:28.980 INFO:tasks.rados.rados.0.smithi162.stdout:1802: finishing copy_from to smithi16234619-48 2024-09-16T15:08:28.980 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 540 (ObjNum 601 snap 153 seq_num 601) dirty exists 2024-09-16T15:08:29.854 INFO:tasks.rados.rados.0.smithi162.stdout:1797: done (4 left) 2024-09-16T15:08:29.854 INFO:tasks.rados.rados.0.smithi162.stdout:1798: done (3 left) 2024-09-16T15:08:29.855 INFO:tasks.rados.rados.0.smithi162.stdout:1801: done (2 left) 2024-09-16T15:08:29.855 INFO:tasks.rados.rados.0.smithi162.stdout:1802: done (1 left) 2024-09-16T15:08:29.855 INFO:tasks.rados.rados.0.smithi162.stdout:1805: done (0 left) 2024-09-16T15:08:29.855 INFO:tasks.rados.rados.0.smithi162.stdout:1806: snap_remove snap 165 2024-09-16T15:08:31.252 INFO:tasks.rados.rados.0.smithi162.stdout:1806: done (0 left) 2024-09-16T15:08:31.253 INFO:tasks.rados.rados.0.smithi162.stdout:1807: write oid 34 current snap is 168 2024-09-16T15:08:31.253 INFO:tasks.rados.rados.0.smithi162.stdout:1807: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2024-09-16T15:08:31.259 INFO:tasks.rados.rados.0.smithi162.stdout:1807: writing smithi16234619-34 from 490936 to 1142293 tid 1 2024-09-16T15:08:31.264 INFO:tasks.rados.rados.0.smithi162.stdout:1807: writing smithi16234619-34 from 1686214 to 2133439 tid 2 2024-09-16T15:08:31.268 INFO:tasks.rados.rados.0.smithi162.stdout:1807: writing smithi16234619-34 from 2861864 to 2890936 tid 3 2024-09-16T15:08:31.269 INFO:tasks.rados.rados.0.smithi162.stdout:1808: write oid 33 current snap is 168 2024-09-16T15:08:31.269 INFO:tasks.rados.rados.0.smithi162.stdout:1808: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2024-09-16T15:08:31.275 INFO:tasks.rados.rados.0.smithi162.stdout:1808: writing smithi16234619-33 from 507743 to 1234349 tid 1 2024-09-16T15:08:31.282 INFO:tasks.rados.rados.0.smithi162.stdout:1808: writing smithi16234619-33 from 1744696 to 2335579 tid 2 2024-09-16T15:08:31.286 INFO:tasks.rados.rados.0.smithi162.stdout:1808: writing smithi16234619-33 from 2772934 to 2907743 tid 3 2024-09-16T15:08:31.286 INFO:tasks.rados.rados.0.smithi162.stdout:1809: copy_from oid 8 from oid 20 current snap is 168 2024-09-16T15:08:31.286 INFO:tasks.rados.rados.0.smithi162.stdout:1810: copy_from oid 4 from oid 3 current snap is 168 2024-09-16T15:08:31.286 INFO:tasks.rados.rados.0.smithi162.stdout:1811: write oid 18 current snap is 168 2024-09-16T15:08:31.286 INFO:tasks.rados.rados.0.smithi162.stdout:1811: seq_num 650 ranges {524550=401855,1403178=450894,2516710=407840} 2024-09-16T15:08:31.292 INFO:tasks.rados.rados.0.smithi162.stdout:1811: writing smithi16234619-18 from 524550 to 926405 tid 1 2024-09-16T15:08:31.298 INFO:tasks.rados.rados.0.smithi162.stdout:1811: writing smithi16234619-18 from 1403178 to 1854072 tid 2 2024-09-16T15:08:31.304 INFO:tasks.rados.rados.0.smithi162.stdout:1811: writing smithi16234619-18 from 2516710 to 2924550 tid 3 2024-09-16T15:08:31.304 INFO:tasks.rados.rados.0.smithi162.stdout:1812: delete oid 48 current snap is 168 2024-09-16T15:08:31.304 INFO:tasks.rados.rados.0.smithi162.stdout:1807: finishing write tid 1 to smithi16234619-34 2024-09-16T15:08:31.304 INFO:tasks.rados.rados.0.smithi162.stdout:1807: finishing write tid 2 to smithi16234619-34 2024-09-16T15:08:31.304 INFO:tasks.rados.rados.0.smithi162.stdout:1807: finishing write tid 3 to smithi16234619-34 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: finishing write tid 4 to smithi16234619-34 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: finishing write tid 5 to smithi16234619-34 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: oid 34 updating version 0 to 663 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: oid 34 updating version 663 to 665 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: oid 34 version 665 is already newer than 662 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: oid 34 version 665 is already newer than 664 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 665 (ObjNum 648 snap 168 seq_num 648) dirty exists 2024-09-16T15:08:31.307 INFO:tasks.rados.rados.0.smithi162.stdout:1807: left oid 34 (ObjNum 648 snap 168 seq_num 648) 2024-09-16T15:08:31.322 INFO:tasks.rados.rados.0.smithi162.stdout:1808: finishing write tid 1 to smithi16234619-33 2024-09-16T15:08:31.325 INFO:tasks.rados.rados.0.smithi162.stdout:1808: finishing write tid 2 to smithi16234619-33 2024-09-16T15:08:31.325 INFO:tasks.rados.rados.0.smithi162.stdout:1808: finishing write tid 3 to smithi16234619-33 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: finishing write tid 4 to smithi16234619-33 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: finishing write tid 5 to smithi16234619-33 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: oid 33 updating version 0 to 640 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: oid 33 version 640 is already newer than 638 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: oid 33 version 640 is already newer than 637 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: oid 33 version 640 is already newer than 639 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 640 (ObjNum 649 snap 168 seq_num 649) dirty exists 2024-09-16T15:08:31.326 INFO:tasks.rados.rados.0.smithi162.stdout:1808: left oid 33 (ObjNum 649 snap 168 seq_num 649) 2024-09-16T15:08:31.343 INFO:tasks.rados.rados.0.smithi162.stdout:1810: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1810: finishing copy_from to smithi16234619-4 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 84 (ObjNum 645 snap 166 seq_num 645) dirty exists 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1807: done (5 left) 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1808: done (4 left) 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1810: done (3 left) 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1812: done (2 left) 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1813: write (excl) oid 28 current snap is 168 2024-09-16T15:08:31.348 INFO:tasks.rados.rados.0.smithi162.stdout:1813: seq_num 652 ranges {558164=552353,1803789=454563,2855203=102961} 2024-09-16T15:08:31.354 INFO:tasks.rados.rados.0.smithi162.stdout:1813: writing smithi16234619-28 from 558164 to 1110517 tid 1 2024-09-16T15:08:31.360 INFO:tasks.rados.rados.0.smithi162.stdout:1813: writing smithi16234619-28 from 1803789 to 2258352 tid 2 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:1813: writing smithi16234619-28 from 2855203 to 2958164 tid 3 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:1814: rollback oid 25 current snap is 168 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 167 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:1815: rollback oid 39 current snap is 168 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 167 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:1816: rollback oid 47 current snap is 168 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 158 2024-09-16T15:08:31.364 INFO:tasks.rados.rados.0.smithi162.stdout:1817: write oid 22 current snap is 168 2024-09-16T15:08:31.365 INFO:tasks.rados.rados.0.smithi162.stdout:1817: seq_num 653 ranges {574971=627602,1862271=598221} 2024-09-16T15:08:31.370 INFO:tasks.rados.rados.0.smithi162.stdout:1817: writing smithi16234619-22 from 574971 to 1202573 tid 1 2024-09-16T15:08:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:1817: writing smithi16234619-22 from 1862271 to 2460492 tid 2 2024-09-16T15:08:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:1818: delete oid 50 current snap is 168 2024-09-16T15:08:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:1816: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:08:31.377 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 0 (ObjNum 81 snap 0 seq_num 1) dirty dne 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1815: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1815: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1814: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1809: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: finishing write tid 1 to smithi16234619-18 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: finishing write tid 2 to smithi16234619-18 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: finishing write tid 3 to smithi16234619-18 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: finishing write tid 4 to smithi16234619-18 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: finishing write tid 5 to smithi16234619-18 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: oid 18 updating version 0 to 579 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: oid 18 version 579 is already newer than 576 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: oid 18 version 579 is already newer than 577 2024-09-16T15:08:31.378 INFO:tasks.rados.rados.0.smithi162.stdout:1811: oid 18 version 579 is already newer than 578 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 579 (ObjNum 650 snap 168 seq_num 650) dirty exists 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:1811: left oid 18 (ObjNum 650 snap 168 seq_num 650) 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:1814: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:1814: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 669 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:1815: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:08:31.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 789 (ObjNum 625 snap 157 seq_num 625) dirty exists 2024-09-16T15:08:31.380 INFO:tasks.rados.rados.0.smithi162.stdout:1809: finishing copy_from to smithi16234619-8 2024-09-16T15:08:31.380 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 636 (ObjNum 543 snap 140 seq_num 543) dirty exists 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1809: done (7 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1811: done (6 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1814: done (5 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1815: done (4 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1816: done (3 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1818: done (2 left) 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1819: rollback oid 45 current snap is 168 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 167 2024-09-16T15:08:31.382 INFO:tasks.rados.rados.0.smithi162.stdout:1820: snap_remove snap 158 2024-09-16T15:08:32.470 INFO:tasks.rados.rados.0.smithi162.stdout:1820: done (3 left) 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1821: snap_create 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1819: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1819: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: finishing write tid 1 to smithi16234619-28 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1819: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 822 (ObjNum 640 snap 164 seq_num 640) dirty exists 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: finishing write tid 2 to smithi16234619-28 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: finishing write tid 3 to smithi16234619-28 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: finishing write tid 4 to smithi16234619-28 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: finishing write tid 5 to smithi16234619-28 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: oid 28 updating version 0 to 566 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: oid 28 version 566 is already newer than 565 2024-09-16T15:08:32.471 INFO:tasks.rados.rados.0.smithi162.stdout:1813: oid 28 version 566 is already newer than 564 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1813: oid 28 version 566 is already newer than 563 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 566 (ObjNum 652 snap 168 seq_num 652) dirty exists 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1813: left oid 28 (ObjNum 652 snap 168 seq_num 652) 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: finishing write tid 1 to smithi16234619-22 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: finishing write tid 2 to smithi16234619-22 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: finishing write tid 3 to smithi16234619-22 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: finishing write tid 4 to smithi16234619-22 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: oid 22 updating version 0 to 621 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: oid 22 updating version 621 to 622 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: oid 22 updating version 622 to 623 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 623 (ObjNum 653 snap 168 seq_num 653) dirty exists 2024-09-16T15:08:32.472 INFO:tasks.rados.rados.0.smithi162.stdout:1817: left oid 22 (ObjNum 653 snap 168 seq_num 653) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1813: done (3 left) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1817: done (2 left) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1819: done (1 left) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1821: done (0 left) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1822: copy_from oid 30 from oid 37 current snap is 169 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1823: read oid 3 snap -1 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1823: expect (ObjNum 645 snap 166 seq_num 645) 2024-09-16T15:08:32.849 INFO:tasks.rados.rados.0.smithi162.stdout:1824: rollback oid 2 current snap is 169 2024-09-16T15:08:32.850 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 163 2024-09-16T15:08:32.850 INFO:tasks.rados.rados.0.smithi162.stdout:1825: write oid 18 current snap is 169 2024-09-16T15:08:32.850 INFO:tasks.rados.rados.0.smithi162.stdout:1825: seq_num 655 ranges {608585=494453,1579235=601890,2821119=187466} 2024-09-16T15:08:32.855 INFO:tasks.rados.rados.0.smithi162.stdout:1825: writing smithi16234619-18 from 608585 to 1103038 tid 1 2024-09-16T15:08:32.863 INFO:tasks.rados.rados.0.smithi162.stdout:1825: writing smithi16234619-18 from 1579235 to 2181125 tid 2 2024-09-16T15:08:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1825: writing smithi16234619-18 from 2821119 to 3008585 tid 3 2024-09-16T15:08:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1826: write oid 19 current snap is 169 2024-09-16T15:08:32.868 INFO:tasks.rados.rados.0.smithi162.stdout:1826: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2024-09-16T15:08:32.875 INFO:tasks.rados.rados.0.smithi162.stdout:1826: writing smithi16234619-19 from 625392 to 1195094 tid 1 2024-09-16T15:08:32.881 INFO:tasks.rados.rados.0.smithi162.stdout:1826: writing smithi16234619-19 from 1637717 to 2383265 tid 2 2024-09-16T15:08:32.885 INFO:tasks.rados.rados.0.smithi162.stdout:1826: writing smithi16234619-19 from 2848542 to 3025392 tid 3 2024-09-16T15:08:32.885 INFO:tasks.rados.rados.0.smithi162.stdout:1827: delete oid 40 current snap is 169 2024-09-16T15:08:32.885 INFO:tasks.rados.rados.0.smithi162.stdout:1823: expect (ObjNum 645 snap 166 seq_num 645) 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1824: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1824: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1824: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 832 (ObjNum 612 snap 154 seq_num 612) dirty exists 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: finishing write tid 1 to smithi16234619-18 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: finishing write tid 2 to smithi16234619-18 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: finishing write tid 3 to smithi16234619-18 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: finishing write tid 4 to smithi16234619-18 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: finishing write tid 5 to smithi16234619-18 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: oid 18 updating version 0 to 637 2024-09-16T15:08:33.010 INFO:tasks.rados.rados.0.smithi162.stdout:1825: oid 18 updating version 637 to 638 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1825: oid 18 updating version 638 to 640 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1825: oid 18 version 640 is already newer than 639 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 640 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1825: left oid 18 (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1822: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1822: finishing copy_from to smithi16234619-30 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 293 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1826: finishing write tid 1 to smithi16234619-19 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1822: done (5 left) 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1823: done (4 left) 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1824: done (3 left) 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1825: done (2 left) 2024-09-16T15:08:33.011 INFO:tasks.rados.rados.0.smithi162.stdout:1827: done (1 left) 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1828: copy_from oid 46 from oid 27 current snap is 169 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: finishing write tid 2 to smithi16234619-19 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: finishing write tid 3 to smithi16234619-19 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: finishing write tid 4 to smithi16234619-19 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: finishing write tid 5 to smithi16234619-19 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: oid 19 updating version 0 to 681 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: oid 19 updating version 681 to 682 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: oid 19 updating version 682 to 684 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: oid 19 version 684 is already newer than 683 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 684 (ObjNum 656 snap 169 seq_num 656) dirty exists 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: left oid 19 (ObjNum 656 snap 169 seq_num 656) 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1826: done (1 left) 2024-09-16T15:08:33.012 INFO:tasks.rados.rados.0.smithi162.stdout:1829: snap_create 2024-09-16T15:08:33.015 INFO:tasks.rados.rados.0.smithi162.stdout:1828: finishing copy_from to smithi16234619-46 2024-09-16T15:08:33.016 INFO:tasks.rados.rados.0.smithi162.stdout:1828: got expected ENOENT (src dne) 2024-09-16T15:08:33.016 INFO:tasks.rados.rados.0.smithi162.stdout:1828: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:08:33.859 INFO:tasks.rados.rados.0.smithi162.stdout:1828: done (1 left) 2024-09-16T15:08:33.859 INFO:tasks.rados.rados.0.smithi162.stdout:1829: done (0 left) 2024-09-16T15:08:33.859 INFO:tasks.rados.rados.0.smithi162.stdout:1830: copy_from oid 24 from oid 11 current snap is 170 2024-09-16T15:08:33.859 INFO:tasks.rados.rados.0.smithi162.stdout:1831: delete oid 5 current snap is 170 2024-09-16T15:08:33.864 INFO:tasks.rados.rados.0.smithi162.stdout:1831: done (1 left) 2024-09-16T15:08:33.864 INFO:tasks.rados.rados.0.smithi162.stdout:1832: snap_create 2024-09-16T15:08:33.864 INFO:tasks.rados.rados.0.smithi162.stdout:1830: finishing copy_from to smithi16234619-24 2024-09-16T15:08:33.864 INFO:tasks.rados.rados.0.smithi162.stdout:1830: got expected ENOENT (src dne) 2024-09-16T15:08:33.864 INFO:tasks.rados.rados.0.smithi162.stdout:1830: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:08:34.879 INFO:tasks.rados.rados.0.smithi162.stdout:1830: done (1 left) 2024-09-16T15:08:34.879 INFO:tasks.rados.rados.0.smithi162.stdout:1832: done (0 left) 2024-09-16T15:08:34.879 INFO:tasks.rados.rados.0.smithi162.stdout:1833: copy_from oid 21 from oid 18 current snap is 171 2024-09-16T15:08:34.880 INFO:tasks.rados.rados.0.smithi162.stdout:1834: read oid 8 snap 169 2024-09-16T15:08:34.880 INFO:tasks.rados.rados.0.smithi162.stdout:1834: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:08:34.880 INFO:tasks.rados.rados.0.smithi162.stdout:1835: write (excl) oid 20 current snap is 171 2024-09-16T15:08:34.880 INFO:tasks.rados.rados.0.smithi162.stdout:1835: seq_num 659 ranges {675813=795449,2096810=609228} 2024-09-16T15:08:34.887 INFO:tasks.rados.rados.0.smithi162.stdout:1835: writing smithi16234619-20 from 675813 to 1471262 tid 1 2024-09-16T15:08:34.894 INFO:tasks.rados.rados.0.smithi162.stdout:1835: writing smithi16234619-20 from 2096810 to 2706038 tid 2 2024-09-16T15:08:34.894 INFO:tasks.rados.rados.0.smithi162.stdout:1836: copy_from oid 10 from oid 38 current snap is 171 2024-09-16T15:08:34.894 INFO:tasks.rados.rados.0.smithi162.stdout:1837: write (excl) oid 42 current snap is 171 2024-09-16T15:08:34.894 INFO:tasks.rados.rados.0.smithi162.stdout:1837: seq_num 660 ranges {692620=470698,1755292=752886} 2024-09-16T15:08:34.901 INFO:tasks.rados.rados.0.smithi162.stdout:1837: writing smithi16234619-42 from 692620 to 1163318 tid 1 2024-09-16T15:08:34.908 INFO:tasks.rados.rados.0.smithi162.stdout:1837: writing smithi16234619-42 from 1755292 to 2508178 tid 2 2024-09-16T15:08:34.908 INFO:tasks.rados.rados.0.smithi162.stdout:1838: snap_remove snap 167 2024-09-16T15:08:35.879 INFO:tasks.rados.rados.0.smithi162.stdout:1838: done (5 left) 2024-09-16T15:08:35.879 INFO:tasks.rados.rados.0.smithi162.stdout:1839: copy_from oid 15 from oid 2 current snap is 171 2024-09-16T15:08:35.879 INFO:tasks.rados.rados.0.smithi162.stdout:1834: expect (ObjNum 543 snap 140 seq_num 543) 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: finishing write tid 1 to smithi16234619-20 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: finishing write tid 2 to smithi16234619-20 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: finishing write tid 3 to smithi16234619-20 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: finishing write tid 4 to smithi16234619-20 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: oid 20 updating version 0 to 643 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: oid 20 version 643 is already newer than 642 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: oid 20 version 643 is already newer than 641 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 643 (ObjNum 659 snap 171 seq_num 659) dirty exists 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1835: left oid 20 (ObjNum 659 snap 171 seq_num 659) 2024-09-16T15:08:36.028 INFO:tasks.rados.rados.0.smithi162.stdout:1837: finishing write tid 1 to smithi16234619-42 2024-09-16T15:08:36.029 INFO:tasks.rados.rados.0.smithi162.stdout:1834: done (5 left) 2024-09-16T15:08:36.029 INFO:tasks.rados.rados.0.smithi162.stdout:1835: done (4 left) 2024-09-16T15:08:36.029 INFO:tasks.rados.rados.0.smithi162.stdout:1840: snap_remove snap 170 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1840: done (4 left) 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1841: write oid 11 current snap is 171 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: finishing write tid 2 to smithi16234619-42 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: finishing write tid 3 to smithi16234619-42 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: finishing write tid 4 to smithi16234619-42 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: oid 42 updating version 0 to 797 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: oid 42 updating version 797 to 798 2024-09-16T15:08:36.893 INFO:tasks.rados.rados.0.smithi162.stdout:1837: oid 42 version 798 is already newer than 796 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 798 (ObjNum 660 snap 171 seq_num 660) dirty exists 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1837: left oid 42 (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1836: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1833: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1833: finishing copy_from to smithi16234619-21 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 640 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1836: finishing copy_from to smithi16234619-10 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 558 (ObjNum 592 snap 150 seq_num 592) dirty exists 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1839: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1839: finishing copy_from to smithi16234619-15 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 832 (ObjNum 612 snap 154 seq_num 612) dirty exists 2024-09-16T15:08:36.894 INFO:tasks.rados.rados.0.smithi162.stdout:1841: seq_num 661 ranges {709427=545947,1813774=612897,2985657=123770} 2024-09-16T15:08:36.899 INFO:tasks.rados.rados.0.smithi162.stdout:1841: writing smithi16234619-11 from 709427 to 1255374 tid 1 2024-09-16T15:08:36.906 INFO:tasks.rados.rados.0.smithi162.stdout:1841: writing smithi16234619-11 from 1813774 to 2426671 tid 2 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1841: writing smithi16234619-11 from 2985657 to 3109427 tid 3 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1833: done (4 left) 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1836: done (3 left) 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1837: done (2 left) 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1839: done (1 left) 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1842: write oid 34 current snap is 171 2024-09-16T15:08:36.910 INFO:tasks.rados.rados.0.smithi162.stdout:1842: seq_num 662 ranges {726234=737549,1988609=756555} 2024-09-16T15:08:36.918 INFO:tasks.rados.rados.0.smithi162.stdout:1842: writing smithi16234619-34 from 726234 to 1463783 tid 1 2024-09-16T15:08:36.925 INFO:tasks.rados.rados.0.smithi162.stdout:1842: writing smithi16234619-34 from 1988609 to 2745164 tid 2 2024-09-16T15:08:36.925 INFO:tasks.rados.rados.0.smithi162.stdout:1843: copy_from oid 37 from oid 39 current snap is 171 2024-09-16T15:08:36.925 INFO:tasks.rados.rados.0.smithi162.stdout:1844: snap_create 2024-09-16T15:08:36.931 INFO:tasks.rados.rados.0.smithi162.stdout:1841: finishing write tid 1 to smithi16234619-11 2024-09-16T15:08:36.932 INFO:tasks.rados.rados.0.smithi162.stdout:1841: finishing write tid 2 to smithi16234619-11 2024-09-16T15:08:36.932 INFO:tasks.rados.rados.0.smithi162.stdout:1841: finishing write tid 3 to smithi16234619-11 2024-09-16T15:08:36.932 INFO:tasks.rados.rados.0.smithi162.stdout:1841: finishing write tid 4 to smithi16234619-11 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: finishing write tid 5 to smithi16234619-11 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: oid 11 updating version 0 to 134 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: oid 11 updating version 134 to 135 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: oid 11 updating version 135 to 136 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: oid 11 updating version 136 to 137 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 137 (ObjNum 661 snap 171 seq_num 661) dirty exists 2024-09-16T15:08:36.933 INFO:tasks.rados.rados.0.smithi162.stdout:1841: left oid 11 (ObjNum 661 snap 171 seq_num 661) 2024-09-16T15:08:36.958 INFO:tasks.rados.rados.0.smithi162.stdout:1843: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:08:36.965 INFO:tasks.rados.rados.0.smithi162.stdout:1842: finishing write tid 1 to smithi16234619-34 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: finishing write tid 2 to smithi16234619-34 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: finishing write tid 3 to smithi16234619-34 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: finishing write tid 4 to smithi16234619-34 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: oid 34 updating version 0 to 696 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: oid 34 version 696 is already newer than 694 2024-09-16T15:08:36.968 INFO:tasks.rados.rados.0.smithi162.stdout:1842: oid 34 version 696 is already newer than 695 2024-09-16T15:08:36.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 696 (ObjNum 662 snap 171 seq_num 662) dirty exists 2024-09-16T15:08:36.969 INFO:tasks.rados.rados.0.smithi162.stdout:1842: left oid 34 (ObjNum 662 snap 171 seq_num 662) 2024-09-16T15:08:36.978 INFO:tasks.rados.rados.0.smithi162.stdout:1843: finishing copy_from to smithi16234619-37 2024-09-16T15:08:36.978 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 789 (ObjNum 625 snap 157 seq_num 625) dirty exists 2024-09-16T15:08:38.042 INFO:tasks.rados.rados.0.smithi162.stdout:1841: done (3 left) 2024-09-16T15:08:38.042 INFO:tasks.rados.rados.0.smithi162.stdout:1842: done (2 left) 2024-09-16T15:08:38.043 INFO:tasks.rados.rados.0.smithi162.stdout:1843: done (1 left) 2024-09-16T15:08:38.043 INFO:tasks.rados.rados.0.smithi162.stdout:1844: done (0 left) 2024-09-16T15:08:38.043 INFO:tasks.rados.rados.0.smithi162.stdout:1845: delete oid 27 current snap is 172 2024-09-16T15:08:38.053 INFO:tasks.rados.rados.0.smithi162.stdout:1845: done (0 left) 2024-09-16T15:08:38.053 INFO:tasks.rados.rados.0.smithi162.stdout:1846: snap_remove snap 163 2024-09-16T15:08:39.072 INFO:tasks.rados.rados.0.smithi162.stdout:1846: done (0 left) 2024-09-16T15:08:39.072 INFO:tasks.rados.rados.0.smithi162.stdout:1847: delete oid 3 current snap is 172 2024-09-16T15:08:39.077 INFO:tasks.rados.rados.0.smithi162.stdout:1847: done (0 left) 2024-09-16T15:08:39.078 INFO:tasks.rados.rados.0.smithi162.stdout:1848: read oid 36 snap 169 2024-09-16T15:08:39.078 INFO:tasks.rados.rados.0.smithi162.stdout:1848: expect (ObjNum 620 snap 156 seq_num 620) 2024-09-16T15:08:39.078 INFO:tasks.rados.rados.0.smithi162.stdout:1849: delete oid 9 current snap is 172 2024-09-16T15:08:39.086 INFO:tasks.rados.rados.0.smithi162.stdout:1849: done (1 left) 2024-09-16T15:08:39.086 INFO:tasks.rados.rados.0.smithi162.stdout:1850: read oid 33 snap 168 2024-09-16T15:08:39.086 INFO:tasks.rados.rados.0.smithi162.stdout:1850: expect (ObjNum 649 snap 168 seq_num 649) 2024-09-16T15:08:39.087 INFO:tasks.rados.rados.0.smithi162.stdout:1851: snap_remove snap 171 2024-09-16T15:08:40.075 INFO:tasks.rados.rados.0.smithi162.stdout:1851: done (2 left) 2024-09-16T15:08:40.075 INFO:tasks.rados.rados.0.smithi162.stdout:1852: rollback oid 44 current snap is 172 2024-09-16T15:08:40.075 INFO:tasks.rados.rados.0.smithi162.stdout:1850: expect (ObjNum 649 snap 168 seq_num 649) 2024-09-16T15:08:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:1848: expect (ObjNum 620 snap 156 seq_num 620) 2024-09-16T15:08:40.335 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 169 2024-09-16T15:08:40.335 INFO:tasks.rados.rados.0.smithi162.stdout:1848: done (2 left) 2024-09-16T15:08:40.336 INFO:tasks.rados.rados.0.smithi162.stdout:1850: done (1 left) 2024-09-16T15:08:40.336 INFO:tasks.rados.rados.0.smithi162.stdout:1853: snap_remove snap 168 2024-09-16T15:08:41.086 INFO:tasks.rados.rados.0.smithi162.stdout:1852: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1852: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1852: finishing rollback tid 2 to smithi16234619-44 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 561 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1852: done (1 left) 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1853: done (0 left) 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1854: read oid 7 snap -1 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1854: expect deleted 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1855: write oid 20 current snap is 172 2024-09-16T15:08:41.087 INFO:tasks.rados.rados.0.smithi162.stdout:1855: seq_num 666 ranges {793462=638545,2106184=763893} 2024-09-16T15:08:41.095 INFO:tasks.rados.rados.0.smithi162.stdout:1855: writing smithi16234619-20 from 793462 to 1432007 tid 1 2024-09-16T15:08:41.104 INFO:tasks.rados.rados.0.smithi162.stdout:1855: writing smithi16234619-20 from 2106184 to 2870077 tid 2 2024-09-16T15:08:41.104 INFO:tasks.rados.rados.0.smithi162.stdout:1854: done (1 left) 2024-09-16T15:08:41.104 INFO:tasks.rados.rados.0.smithi162.stdout:1856: snap_remove snap 169 2024-09-16T15:08:42.283 INFO:tasks.rados.rados.0.smithi162.stdout:1855: finishing write tid 1 to smithi16234619-20 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: finishing write tid 2 to smithi16234619-20 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: finishing write tid 3 to smithi16234619-20 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: finishing write tid 4 to smithi16234619-20 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: oid 20 updating version 0 to 646 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: oid 20 version 646 is already newer than 644 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: oid 20 version 646 is already newer than 645 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 646 (ObjNum 666 snap 172 seq_num 666) dirty exists 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: left oid 20 (ObjNum 666 snap 172 seq_num 666) 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1855: done (1 left) 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1856: done (0 left) 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1857: copy_from oid 45 from oid 32 current snap is 172 2024-09-16T15:08:42.284 INFO:tasks.rados.rados.0.smithi162.stdout:1858: delete oid 48 current snap is 172 2024-09-16T15:08:42.290 INFO:tasks.rados.rados.0.smithi162.stdout:1858: done (1 left) 2024-09-16T15:08:42.291 INFO:tasks.rados.rados.0.smithi162.stdout:1859: snap_create 2024-09-16T15:08:42.320 INFO:tasks.rados.rados.0.smithi162.stdout:1857: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:08:42.322 INFO:tasks.rados.rados.0.smithi162.stdout:1857: finishing copy_from to smithi16234619-45 2024-09-16T15:08:42.322 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 298 (ObjNum 626 snap 157 seq_num 626) dirty exists 2024-09-16T15:08:42.850 INFO:tasks.rados.rados.0.smithi162.stdout:1857: done (1 left) 2024-09-16T15:08:42.850 INFO:tasks.rados.rados.0.smithi162.stdout:1859: done (0 left) 2024-09-16T15:08:42.851 INFO:tasks.rados.rados.0.smithi162.stdout:1860: write (excl) oid 10 current snap is 173 2024-09-16T15:08:42.851 INFO:tasks.rados.rados.0.smithi162.stdout:1860: seq_num 668 ranges {427076=789043,1706795=767562,2944912=282164} 2024-09-16T15:08:42.857 INFO:tasks.rados.rados.0.smithi162.stdout:1860: writing smithi16234619-10 from 427076 to 1216119 tid 1 2024-09-16T15:08:42.866 INFO:tasks.rados.rados.0.smithi162.stdout:1860: writing smithi16234619-10 from 1706795 to 2474357 tid 2 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1860: writing smithi16234619-10 from 2944912 to 3227076 tid 3 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1861: copy_from oid 42 from oid 48 current snap is 173 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1862: read oid 37 snap -1 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1862: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1863: write oid 17 current snap is 173 2024-09-16T15:08:42.870 INFO:tasks.rados.rados.0.smithi162.stdout:1863: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2024-09-16T15:08:42.875 INFO:tasks.rados.rados.0.smithi162.stdout:1863: writing smithi16234619-17 from 443883 to 908175 tid 1 2024-09-16T15:08:42.881 INFO:tasks.rados.rados.0.smithi162.stdout:1863: writing smithi16234619-17 from 1365277 to 1876497 tid 2 2024-09-16T15:08:42.888 INFO:tasks.rados.rados.0.smithi162.stdout:1863: writing smithi16234619-17 from 2572335 to 3243883 tid 3 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:1864: rollback oid 39 current snap is 173 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 172 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:1861: finishing copy_from to smithi16234619-42 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:1861: got expected ENOENT (src dne) 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:1861: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:08:42.889 INFO:tasks.rados.rados.0.smithi162.stdout:1862: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:42.970 INFO:tasks.rados.rados.0.smithi162.stdout:1864: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:08:42.970 INFO:tasks.rados.rados.0.smithi162.stdout:1864: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:08:42.970 INFO:tasks.rados.rados.0.smithi162.stdout:1864: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 820 (ObjNum 625 snap 157 seq_num 625) dirty exists 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: finishing write tid 1 to smithi16234619-10 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: finishing write tid 2 to smithi16234619-10 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: finishing write tid 3 to smithi16234619-10 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1861: done (4 left) 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1862: done (3 left) 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1864: done (2 left) 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1865: delete oid 2 current snap is 173 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: finishing write tid 4 to smithi16234619-10 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: finishing write tid 5 to smithi16234619-10 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: oid 10 updating version 0 to 861 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: oid 10 updating version 861 to 862 2024-09-16T15:08:42.971 INFO:tasks.rados.rados.0.smithi162.stdout:1860: oid 10 updating version 862 to 863 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1860: oid 10 version 863 is already newer than 860 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 863 (ObjNum 668 snap 173 seq_num 668) dirty exists 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1860: left oid 10 (ObjNum 668 snap 173 seq_num 668) 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: finishing write tid 1 to smithi16234619-17 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: finishing write tid 2 to smithi16234619-17 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: finishing write tid 3 to smithi16234619-17 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: finishing write tid 4 to smithi16234619-17 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: finishing write tid 5 to smithi16234619-17 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: oid 17 updating version 0 to 866 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: oid 17 version 866 is already newer than 864 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: oid 17 version 866 is already newer than 865 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: oid 17 updating version 866 to 867 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 867 (ObjNum 669 snap 173 seq_num 669) dirty exists 2024-09-16T15:08:42.972 INFO:tasks.rados.rados.0.smithi162.stdout:1863: left oid 17 (ObjNum 669 snap 173 seq_num 669) 2024-09-16T15:08:42.973 INFO:tasks.rados.rados.0.smithi162.stdout:1860: done (2 left) 2024-09-16T15:08:42.973 INFO:tasks.rados.rados.0.smithi162.stdout:1863: done (1 left) 2024-09-16T15:08:42.973 INFO:tasks.rados.rados.0.smithi162.stdout:1865: done (0 left) 2024-09-16T15:08:42.973 INFO:tasks.rados.rados.0.smithi162.stdout:1866: snap_remove snap 172 2024-09-16T15:08:44.070 INFO:tasks.rados.rados.0.smithi162.stdout:1866: done (0 left) 2024-09-16T15:08:44.070 INFO:tasks.rados.rados.0.smithi162.stdout:1867: read oid 36 snap -1 2024-09-16T15:08:44.070 INFO:tasks.rados.rados.0.smithi162.stdout:1867: expect (ObjNum 620 snap 156 seq_num 620) 2024-09-16T15:08:44.070 INFO:tasks.rados.rados.0.smithi162.stdout:1868: snap_create 2024-09-16T15:08:44.075 INFO:tasks.rados.rados.0.smithi162.stdout:1867: expect (ObjNum 620 snap 156 seq_num 620) 2024-09-16T15:08:45.083 INFO:tasks.rados.rados.0.smithi162.stdout:1867: done (1 left) 2024-09-16T15:08:45.083 INFO:tasks.rados.rados.0.smithi162.stdout:1868: done (0 left) 2024-09-16T15:08:45.083 INFO:tasks.rados.rados.0.smithi162.stdout:1869: write oid 46 current snap is 174 2024-09-16T15:08:45.083 INFO:tasks.rados.rados.0.smithi162.stdout:1869: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2024-09-16T15:08:45.089 INFO:tasks.rados.rados.0.smithi162.stdout:1869: writing smithi16234619-46 from 477497 to 1208640 tid 1 2024-09-16T15:08:45.096 INFO:tasks.rados.rados.0.smithi162.stdout:1869: writing smithi16234619-46 from 1998594 to 2513483 tid 2 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1869: writing smithi16234619-46 from 3143534 to 3277497 tid 3 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1870: copy_from oid 47 from oid 4 current snap is 174 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1871: copy_from oid 14 from oid 25 current snap is 174 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1872: read oid 43 snap -1 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1872: expect (ObjNum 638 snap 164 seq_num 638) 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1873: read oid 17 snap -1 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1873: expect (ObjNum 669 snap 173 seq_num 669) 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1874: rollback oid 18 current snap is 174 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 173 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1875: read oid 2 snap 173 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1875: expect deleted 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:1876: rollback oid 38 current snap is 174 2024-09-16T15:08:45.102 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 173 2024-09-16T15:08:45.103 INFO:tasks.rados.rados.0.smithi162.stdout:1877: write oid 11 current snap is 174 2024-09-16T15:08:45.103 INFO:tasks.rados.rados.0.smithi162.stdout:1877: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2024-09-16T15:08:45.107 INFO:tasks.rados.rados.0.smithi162.stdout:1877: writing smithi16234619-11 from 494304 to 900696 tid 1 2024-09-16T15:08:45.116 INFO:tasks.rados.rados.0.smithi162.stdout:1877: writing smithi16234619-11 from 1540723 to 2315623 tid 2 2024-09-16T15:08:45.122 INFO:tasks.rados.rados.0.smithi162.stdout:1877: writing smithi16234619-11 from 2770957 to 3294304 tid 3 2024-09-16T15:08:45.122 INFO:tasks.rados.rados.0.smithi162.stdout:1878: copy_from oid 20 from oid 27 current snap is 174 2024-09-16T15:08:45.122 INFO:tasks.rados.rados.0.smithi162.stdout:1872: expect (ObjNum 638 snap 164 seq_num 638) 2024-09-16T15:08:45.257 INFO:tasks.rados.rados.0.smithi162.stdout:1873: expect (ObjNum 669 snap 173 seq_num 669) 2024-09-16T15:08:45.359 INFO:tasks.rados.rados.0.smithi162.stdout:1876: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1876: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1874: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1874: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1874: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 643 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1876: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 564 (ObjNum 592 snap 150 seq_num 592) dirty exists 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1872: done (9 left) 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1873: done (8 left) 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1874: done (7 left) 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1875: done (6 left) 2024-09-16T15:08:45.360 INFO:tasks.rados.rados.0.smithi162.stdout:1876: done (5 left) 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1879: rollback oid 35 current snap is 174 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 173 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1878: finishing copy_from to smithi16234619-20 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1878: got expected ENOENT (src dne) 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1878: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: finishing write tid 1 to smithi16234619-46 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: finishing write tid 2 to smithi16234619-46 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: finishing write tid 3 to smithi16234619-46 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: finishing write tid 4 to smithi16234619-46 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: finishing write tid 5 to smithi16234619-46 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: oid 46 updating version 0 to 266 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: oid 46 version 266 is already newer than 263 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: oid 46 version 266 is already newer than 264 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:1869: oid 46 version 266 is already newer than 265 2024-09-16T15:08:45.361 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 266 (ObjNum 671 snap 174 seq_num 671) dirty exists 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1869: left oid 46 (ObjNum 671 snap 174 seq_num 671) 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: finishing write tid 1 to smithi16234619-11 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1870: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1870: finishing copy_from to smithi16234619-47 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 84 (ObjNum 645 snap 166 seq_num 645) dirty exists 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1871: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: finishing write tid 2 to smithi16234619-11 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: finishing write tid 3 to smithi16234619-11 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: finishing write tid 4 to smithi16234619-11 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: finishing write tid 5 to smithi16234619-11 2024-09-16T15:08:45.362 INFO:tasks.rados.rados.0.smithi162.stdout:1877: oid 11 updating version 0 to 142 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1877: oid 11 version 142 is already newer than 140 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1877: oid 11 version 142 is already newer than 141 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1877: oid 11 version 142 is already newer than 139 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 142 (ObjNum 672 snap 174 seq_num 672) dirty exists 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1877: left oid 11 (ObjNum 672 snap 174 seq_num 672) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1871: finishing copy_from to smithi16234619-14 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 669 (ObjNum 311 snap 66 seq_num 311) dirty exists 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1869: done (5 left) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1870: done (4 left) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1871: done (3 left) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1877: done (2 left) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1878: done (1 left) 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:1880: rollback oid 49 current snap is 174 2024-09-16T15:08:45.363 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 173 2024-09-16T15:08:45.364 INFO:tasks.rados.rados.0.smithi162.stdout:1881: write (excl) oid 1 current snap is 174 2024-09-16T15:08:45.364 INFO:tasks.rados.rados.0.smithi162.stdout:1881: seq_num 673 ranges {511111=481641,1599205=518558,2682027=489947} 2024-09-16T15:08:45.364 INFO:tasks.rados.rados.0.smithi162.stdout:1881: writing smithi16234619-1 from 511111 to 992752 tid 1 2024-09-16T15:08:45.370 INFO:tasks.rados.rados.0.smithi162.stdout:1881: writing smithi16234619-1 from 1599205 to 2117763 tid 2 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1881: writing smithi16234619-1 from 2682027 to 3171974 tid 3 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1882: copy_from oid 8 from oid 15 current snap is 174 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1880: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1880: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1879: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1879: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1879: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 567 (ObjNum 608 snap 154 seq_num 608) dirty exists 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:1880: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:08:45.377 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 626 (ObjNum 628 snap 157 seq_num 628) dirty exists 2024-09-16T15:08:45.378 INFO:tasks.rados.rados.0.smithi162.stdout:1879: done (3 left) 2024-09-16T15:08:45.378 INFO:tasks.rados.rados.0.smithi162.stdout:1880: done (2 left) 2024-09-16T15:08:45.378 INFO:tasks.rados.rados.0.smithi162.stdout:1883: rollback oid 28 current snap is 174 2024-09-16T15:08:45.378 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 173 2024-09-16T15:08:45.378 INFO:tasks.rados.rados.0.smithi162.stdout:1884: snap_create 2024-09-16T15:08:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:1883: finishing rollback tid 0 to smithi16234619-28 2024-09-16T15:08:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:1883: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:08:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:1883: finishing rollback tid 2 to smithi16234619-28 2024-09-16T15:08:45.384 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 835 (ObjNum 652 snap 168 seq_num 652) dirty exists 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: finishing write tid 1 to smithi16234619-1 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: finishing write tid 2 to smithi16234619-1 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: finishing write tid 3 to smithi16234619-1 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: finishing write tid 4 to smithi16234619-1 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: finishing write tid 5 to smithi16234619-1 2024-09-16T15:08:45.402 INFO:tasks.rados.rados.0.smithi162.stdout:1881: oid 1 updating version 0 to 734 2024-09-16T15:08:45.403 INFO:tasks.rados.rados.0.smithi162.stdout:1881: oid 1 updating version 734 to 735 2024-09-16T15:08:45.403 INFO:tasks.rados.rados.0.smithi162.stdout:1881: oid 1 version 735 is already newer than 733 2024-09-16T15:08:45.403 INFO:tasks.rados.rados.0.smithi162.stdout:1881: oid 1 updating version 735 to 736 2024-09-16T15:08:45.403 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 736 (ObjNum 673 snap 174 seq_num 673) dirty exists 2024-09-16T15:08:45.403 INFO:tasks.rados.rados.0.smithi162.stdout:1881: left oid 1 (ObjNum 673 snap 174 seq_num 673) 2024-09-16T15:08:45.413 INFO:tasks.rados.rados.0.smithi162.stdout:1882: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:08:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:1882: finishing copy_from to smithi16234619-8 2024-09-16T15:08:45.416 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 832 (ObjNum 612 snap 154 seq_num 612) dirty exists 2024-09-16T15:08:46.080 INFO:tasks.rados.rados.0.smithi162.stdout:1881: done (3 left) 2024-09-16T15:08:46.080 INFO:tasks.rados.rados.0.smithi162.stdout:1882: done (2 left) 2024-09-16T15:08:46.080 INFO:tasks.rados.rados.0.smithi162.stdout:1883: done (1 left) 2024-09-16T15:08:46.080 INFO:tasks.rados.rados.0.smithi162.stdout:1884: done (0 left) 2024-09-16T15:08:46.080 INFO:tasks.rados.rados.0.smithi162.stdout:1885: read oid 49 snap -1 2024-09-16T15:08:46.081 INFO:tasks.rados.rados.0.smithi162.stdout:1885: expect (ObjNum 628 snap 157 seq_num 628) 2024-09-16T15:08:46.081 INFO:tasks.rados.rados.0.smithi162.stdout:1886: copy_from oid 31 from oid 32 current snap is 175 2024-09-16T15:08:46.081 INFO:tasks.rados.rados.0.smithi162.stdout:1887: snap_create 2024-09-16T15:08:46.089 INFO:tasks.rados.rados.0.smithi162.stdout:1885: expect (ObjNum 628 snap 157 seq_num 628) 2024-09-16T15:08:46.171 INFO:tasks.rados.rados.0.smithi162.stdout:1886: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:08:46.171 INFO:tasks.rados.rados.0.smithi162.stdout:1886: finishing copy_from to smithi16234619-31 2024-09-16T15:08:46.171 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 298 (ObjNum 626 snap 157 seq_num 626) dirty exists 2024-09-16T15:08:47.084 INFO:tasks.rados.rados.0.smithi162.stdout:1885: done (2 left) 2024-09-16T15:08:47.085 INFO:tasks.rados.rados.0.smithi162.stdout:1886: done (1 left) 2024-09-16T15:08:47.085 INFO:tasks.rados.rados.0.smithi162.stdout:1887: done (0 left) 2024-09-16T15:08:47.085 INFO:tasks.rados.rados.0.smithi162.stdout:1888: rollback oid 30 current snap is 176 2024-09-16T15:08:47.085 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 174 2024-09-16T15:08:47.085 INFO:tasks.rados.rados.0.smithi162.stdout:1889: write oid 38 current snap is 176 2024-09-16T15:08:47.086 INFO:tasks.rados.rados.0.smithi162.stdout:1889: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2024-09-16T15:08:47.090 INFO:tasks.rados.rados.0.smithi162.stdout:1889: writing smithi16234619-38 from 527918 to 1084808 tid 1 2024-09-16T15:08:47.097 INFO:tasks.rados.rados.0.smithi162.stdout:1889: writing smithi16234619-38 from 1657687 to 2436256 tid 2 2024-09-16T15:08:47.103 INFO:tasks.rados.rados.0.smithi162.stdout:1889: writing smithi16234619-38 from 3225803 to 3327918 tid 3 2024-09-16T15:08:47.104 INFO:tasks.rados.rados.0.smithi162.stdout:1890: snap_create 2024-09-16T15:08:47.104 INFO:tasks.rados.rados.0.smithi162.stdout:1888: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:08:47.104 INFO:tasks.rados.rados.0.smithi162.stdout:1888: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:08:47.104 INFO:tasks.rados.rados.0.smithi162.stdout:1888: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:08:47.104 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 838 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:47.128 INFO:tasks.rados.rados.0.smithi162.stdout:1889: finishing write tid 1 to smithi16234619-38 2024-09-16T15:08:47.128 INFO:tasks.rados.rados.0.smithi162.stdout:1889: finishing write tid 2 to smithi16234619-38 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: finishing write tid 3 to smithi16234619-38 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: finishing write tid 4 to smithi16234619-38 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: finishing write tid 5 to smithi16234619-38 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: oid 38 updating version 0 to 568 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: oid 38 updating version 568 to 571 2024-09-16T15:08:47.129 INFO:tasks.rados.rados.0.smithi162.stdout:1889: oid 38 version 571 is already newer than 570 2024-09-16T15:08:47.130 INFO:tasks.rados.rados.0.smithi162.stdout:1889: oid 38 version 571 is already newer than 569 2024-09-16T15:08:47.130 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 571 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:08:47.130 INFO:tasks.rados.rados.0.smithi162.stdout:1889: left oid 38 (ObjNum 674 snap 176 seq_num 674) 2024-09-16T15:08:47.849 INFO:tasks.rados.rados.0.smithi162.stdout:1888: done (2 left) 2024-09-16T15:08:47.849 INFO:tasks.rados.rados.0.smithi162.stdout:1889: done (1 left) 2024-09-16T15:08:47.850 INFO:tasks.rados.rados.0.smithi162.stdout:1890: done (0 left) 2024-09-16T15:08:47.850 INFO:tasks.rados.rados.0.smithi162.stdout:1891: write oid 41 current snap is 177 2024-09-16T15:08:47.850 INFO:tasks.rados.rados.0.smithi162.stdout:1891: seq_num 675 ranges {544725=632139,1716169=522227,2736873=512491} 2024-09-16T15:08:47.855 INFO:tasks.rados.rados.0.smithi162.stdout:1891: writing smithi16234619-41 from 544725 to 1176864 tid 1 2024-09-16T15:08:47.861 INFO:tasks.rados.rados.0.smithi162.stdout:1891: writing smithi16234619-41 from 1716169 to 2238396 tid 2 2024-09-16T15:08:47.867 INFO:tasks.rados.rados.0.smithi162.stdout:1891: writing smithi16234619-41 from 2736873 to 3249364 tid 3 2024-09-16T15:08:47.867 INFO:tasks.rados.rados.0.smithi162.stdout:1892: read oid 39 snap -1 2024-09-16T15:08:47.867 INFO:tasks.rados.rados.0.smithi162.stdout:1892: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:47.867 INFO:tasks.rados.rados.0.smithi162.stdout:1893: write oid 18 current snap is 177 2024-09-16T15:08:47.867 INFO:tasks.rados.rados.0.smithi162.stdout:1893: seq_num 676 ranges {561532=707388,2058298=782238} 2024-09-16T15:08:47.874 INFO:tasks.rados.rados.0.smithi162.stdout:1893: writing smithi16234619-18 from 561532 to 1268920 tid 1 2024-09-16T15:08:47.884 INFO:tasks.rados.rados.0.smithi162.stdout:1893: writing smithi16234619-18 from 2058298 to 2840536 tid 2 2024-09-16T15:08:47.884 INFO:tasks.rados.rados.0.smithi162.stdout:1894: snap_remove snap 176 2024-09-16T15:08:49.292 INFO:tasks.rados.rados.0.smithi162.stdout:1894: done (3 left) 2024-09-16T15:08:49.292 INFO:tasks.rados.rados.0.smithi162.stdout:1895: write (excl) oid 24 current snap is 177 2024-09-16T15:08:49.292 INFO:tasks.rados.rados.0.smithi162.stdout:1895: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2024-09-16T15:08:49.298 INFO:tasks.rados.rados.0.smithi162.stdout:1895: writing smithi16234619-24 from 578339 to 1077329 tid 1 2024-09-16T15:08:49.305 INFO:tasks.rados.rados.0.smithi162.stdout:1895: writing smithi16234619-24 from 1833133 to 2359029 tid 2 2024-09-16T15:08:49.310 INFO:tasks.rados.rados.0.smithi162.stdout:1895: writing smithi16234619-24 from 2791719 to 3378339 tid 3 2024-09-16T15:08:49.310 INFO:tasks.rados.rados.0.smithi162.stdout:1896: delete oid 33 current snap is 177 2024-09-16T15:08:49.311 INFO:tasks.rados.rados.0.smithi162.stdout:1892: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: finishing write tid 1 to smithi16234619-41 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: finishing write tid 2 to smithi16234619-41 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: finishing write tid 3 to smithi16234619-41 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: finishing write tid 4 to smithi16234619-41 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: finishing write tid 5 to smithi16234619-41 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: oid 41 updating version 0 to 630 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: oid 41 version 630 is already newer than 627 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: oid 41 version 630 is already newer than 628 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:1891: oid 41 version 630 is already newer than 629 2024-09-16T15:08:49.392 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 630 (ObjNum 675 snap 177 seq_num 675) dirty exists 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1891: left oid 41 (ObjNum 675 snap 177 seq_num 675) 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1891: done (4 left) 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1892: done (3 left) 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1896: done (2 left) 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1897: write oid 4 current snap is 177 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1893: finishing write tid 1 to smithi16234619-18 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1893: finishing write tid 2 to smithi16234619-18 2024-09-16T15:08:49.393 INFO:tasks.rados.rados.0.smithi162.stdout:1897: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2024-09-16T15:08:49.398 INFO:tasks.rados.rados.0.smithi162.stdout:1897: writing smithi16234619-4 from 611953 to 1261441 tid 1 2024-09-16T15:08:49.405 INFO:tasks.rados.rados.0.smithi162.stdout:1897: writing smithi16234619-4 from 1950097 to 2479662 tid 2 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1897: writing smithi16234619-4 from 3246565 to 3411953 tid 3 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1898: copy_from oid 37 from oid 38 current snap is 177 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1893: finishing write tid 3 to smithi16234619-18 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1893: finishing write tid 4 to smithi16234619-18 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1893: oid 18 updating version 0 to 834 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1893: oid 18 updating version 834 to 835 2024-09-16T15:08:49.412 INFO:tasks.rados.rados.0.smithi162.stdout:1893: oid 18 version 835 is already newer than 833 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 835 (ObjNum 676 snap 177 seq_num 676) dirty exists 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1893: left oid 18 (ObjNum 676 snap 177 seq_num 676) 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: finishing write tid 1 to smithi16234619-24 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: finishing write tid 2 to smithi16234619-24 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: finishing write tid 3 to smithi16234619-24 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: finishing write tid 4 to smithi16234619-24 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: finishing write tid 5 to smithi16234619-24 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: oid 24 updating version 0 to 876 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: oid 24 updating version 876 to 878 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: oid 24 version 878 is already newer than 877 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: oid 24 version 878 is already newer than 875 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 878 (ObjNum 677 snap 177 seq_num 677) dirty exists 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: left oid 24 (ObjNum 677 snap 177 seq_num 677) 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1893: done (3 left) 2024-09-16T15:08:49.413 INFO:tasks.rados.rados.0.smithi162.stdout:1895: done (2 left) 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1899: rollback oid 36 current snap is 177 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 175 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1900: copy_from oid 39 from oid 48 current snap is 177 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1901: read oid 2 snap -1 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1901: expect deleted 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1902: write (excl) oid 1 current snap is 177 2024-09-16T15:08:49.414 INFO:tasks.rados.rados.0.smithi162.stdout:1902: seq_num 680 ranges {628760=724737,1892226=673223,3157635=271125} 2024-09-16T15:08:49.420 INFO:tasks.rados.rados.0.smithi162.stdout:1902: writing smithi16234619-1 from 628760 to 1353497 tid 1 2024-09-16T15:08:49.426 INFO:tasks.rados.rados.0.smithi162.stdout:1902: writing smithi16234619-1 from 1892226 to 2565449 tid 2 2024-09-16T15:08:49.432 INFO:tasks.rados.rados.0.smithi162.stdout:1902: writing smithi16234619-1 from 3157635 to 3428760 tid 3 2024-09-16T15:08:49.432 INFO:tasks.rados.rados.0.smithi162.stdout:1903: read oid 35 snap 175 2024-09-16T15:08:49.432 INFO:tasks.rados.rados.0.smithi162.stdout:1903: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:49.432 INFO:tasks.rados.rados.0.smithi162.stdout:1904: write (excl) oid 22 current snap is 177 2024-09-16T15:08:49.432 INFO:tasks.rados.rados.0.smithi162.stdout:1904: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2024-09-16T15:08:49.440 INFO:tasks.rados.rados.0.smithi162.stdout:1904: writing smithi16234619-22 from 645567 to 1445553 tid 1 2024-09-16T15:08:49.445 INFO:tasks.rados.rados.0.smithi162.stdout:1904: writing smithi16234619-22 from 1950708 to 2483942 tid 2 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:1904: writing smithi16234619-22 from 3185058 to 3445567 tid 3 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:1905: rollback oid 16 current snap is 177 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 175 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:1899: finishing rollback tid 0 to smithi16234619-36 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:1899: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:08:49.451 INFO:tasks.rados.rados.0.smithi162.stdout:1903: expect (ObjNum 608 snap 154 seq_num 608) 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1899: finishing rollback tid 2 to smithi16234619-36 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 715 (ObjNum 620 snap 156 seq_num 620) dirty exists 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1897: finishing write tid 1 to smithi16234619-4 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1899: done (8 left) 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1901: done (7 left) 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1903: done (6 left) 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1906: read oid 12 snap -1 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1906: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:08:49.575 INFO:tasks.rados.rados.0.smithi162.stdout:1907: snap_remove snap 173 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1907: done (7 left) 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1908: write (excl) oid 30 current snap is 177 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1897: finishing write tid 2 to smithi16234619-4 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1905: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1905: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:1905: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:08:50.297 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 574 (ObjNum 444 snap 106 seq_num 444) dirty exists 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1900: finishing copy_from to smithi16234619-39 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1900: got expected ENOENT (src dne) 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1900: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1898: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1898: finishing copy_from to smithi16234619-37 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 571 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: finishing write tid 3 to smithi16234619-4 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: finishing write tid 4 to smithi16234619-4 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: finishing write tid 5 to smithi16234619-4 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: oid 4 updating version 0 to 740 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: oid 4 version 740 is already newer than 739 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: oid 4 version 740 is already newer than 737 2024-09-16T15:08:50.298 INFO:tasks.rados.rados.0.smithi162.stdout:1897: oid 4 version 740 is already newer than 738 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 740 (ObjNum 679 snap 177 seq_num 679) dirty exists 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1897: left oid 4 (ObjNum 679 snap 177 seq_num 679) 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: finishing write tid 1 to smithi16234619-1 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: finishing write tid 2 to smithi16234619-1 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: finishing write tid 3 to smithi16234619-1 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: finishing write tid 4 to smithi16234619-1 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: finishing write tid 5 to smithi16234619-1 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: oid 1 updating version 0 to 744 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: oid 1 version 744 is already newer than 741 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: oid 1 version 744 is already newer than 743 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:1902: oid 1 version 744 is already newer than 742 2024-09-16T15:08:50.299 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 744 (ObjNum 680 snap 177 seq_num 680) dirty exists 2024-09-16T15:08:50.300 INFO:tasks.rados.rados.0.smithi162.stdout:1902: left oid 1 (ObjNum 680 snap 177 seq_num 680) 2024-09-16T15:08:50.300 INFO:tasks.rados.rados.0.smithi162.stdout:1904: finishing write tid 1 to smithi16234619-22 2024-09-16T15:08:50.300 INFO:tasks.rados.rados.0.smithi162.stdout:1904: finishing write tid 2 to smithi16234619-22 2024-09-16T15:08:50.300 INFO:tasks.rados.rados.0.smithi162.stdout:1908: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2024-09-16T15:08:50.303 INFO:tasks.rados.rados.0.smithi162.stdout:1908: writing smithi16234619-30 from 662374 to 1137609 tid 1 2024-09-16T15:08:50.309 INFO:tasks.rados.rados.0.smithi162.stdout:1908: writing smithi16234619-30 from 1609190 to 2286082 tid 2 2024-09-16T15:08:50.316 INFO:tasks.rados.rados.0.smithi162.stdout:1908: writing smithi16234619-30 from 2812481 to 3436582 tid 3 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1897: done (7 left) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1898: done (6 left) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1900: done (5 left) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1902: done (4 left) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1905: done (3 left) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1909: read oid 10 snap 175 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1909: expect (ObjNum 668 snap 173 seq_num 668) 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1910: write oid 6 current snap is 177 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1904: finishing write tid 3 to smithi16234619-22 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1904: finishing write tid 4 to smithi16234619-22 2024-09-16T15:08:50.317 INFO:tasks.rados.rados.0.smithi162.stdout:1910: seq_num 683 ranges {679181=550484,1667672=536903,2839904=435373} 2024-09-16T15:08:50.323 INFO:tasks.rados.rados.0.smithi162.stdout:1910: writing smithi16234619-6 from 679181 to 1229665 tid 1 2024-09-16T15:08:50.330 INFO:tasks.rados.rados.0.smithi162.stdout:1910: writing smithi16234619-6 from 1667672 to 2204575 tid 2 2024-09-16T15:08:50.337 INFO:tasks.rados.rados.0.smithi162.stdout:1910: writing smithi16234619-6 from 2839904 to 3275277 tid 3 2024-09-16T15:08:50.337 INFO:tasks.rados.rados.0.smithi162.stdout:1911: write (excl) oid 36 current snap is 177 2024-09-16T15:08:50.337 INFO:tasks.rados.rados.0.smithi162.stdout:1904: finishing write tid 5 to smithi16234619-22 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1904: oid 22 updating version 0 to 634 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1904: oid 22 version 634 is already newer than 632 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1904: oid 22 version 634 is already newer than 631 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1904: oid 22 version 634 is already newer than 633 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 634 (ObjNum 681 snap 177 seq_num 681) dirty exists 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1904: left oid 22 (ObjNum 681 snap 177 seq_num 681) 2024-09-16T15:08:50.338 INFO:tasks.rados.rados.0.smithi162.stdout:1906: expect (ObjNum 585 snap 146 seq_num 585) 2024-09-16T15:08:50.532 INFO:tasks.rados.rados.0.smithi162.stdout:1909: expect (ObjNum 668 snap 173 seq_num 668) 2024-09-16T15:08:50.665 INFO:tasks.rados.rados.0.smithi162.stdout:1908: finishing write tid 1 to smithi16234619-30 2024-09-16T15:08:50.665 INFO:tasks.rados.rados.0.smithi162.stdout:1908: finishing write tid 2 to smithi16234619-30 2024-09-16T15:08:50.665 INFO:tasks.rados.rados.0.smithi162.stdout:1908: finishing write tid 3 to smithi16234619-30 2024-09-16T15:08:50.665 INFO:tasks.rados.rados.0.smithi162.stdout:1908: finishing write tid 4 to smithi16234619-30 2024-09-16T15:08:50.665 INFO:tasks.rados.rados.0.smithi162.stdout:1908: finishing write tid 5 to smithi16234619-30 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1908: oid 30 updating version 0 to 839 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1908: oid 30 updating version 839 to 840 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1908: oid 30 updating version 840 to 842 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1908: oid 30 version 842 is already newer than 841 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 842 (ObjNum 682 snap 177 seq_num 682) dirty exists 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1908: left oid 30 (ObjNum 682 snap 177 seq_num 682) 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1910: finishing write tid 1 to smithi16234619-6 2024-09-16T15:08:50.666 INFO:tasks.rados.rados.0.smithi162.stdout:1911: seq_num 684 ranges {695988=625733,2009801=680561,3150974=345014} 2024-09-16T15:08:50.672 INFO:tasks.rados.rados.0.smithi162.stdout:1911: writing smithi16234619-36 from 695988 to 1321721 tid 1 2024-09-16T15:08:50.679 INFO:tasks.rados.rados.0.smithi162.stdout:1911: writing smithi16234619-36 from 2009801 to 2690362 tid 2 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1911: writing smithi16234619-36 from 3150974 to 3495988 tid 3 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1904: done (5 left) 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1906: done (4 left) 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1908: done (3 left) 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1909: done (2 left) 2024-09-16T15:08:50.684 INFO:tasks.rados.rados.0.smithi162.stdout:1912: snap_remove snap 175 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1912: done (2 left) 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1913: copy_from oid 27 from oid 34 current snap is 177 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: finishing write tid 2 to smithi16234619-6 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: finishing write tid 3 to smithi16234619-6 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: finishing write tid 4 to smithi16234619-6 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: finishing write tid 5 to smithi16234619-6 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: oid 6 updating version 0 to 577 2024-09-16T15:08:51.298 INFO:tasks.rados.rados.0.smithi162.stdout:1910: oid 6 version 577 is already newer than 575 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1910: oid 6 version 577 is already newer than 576 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1910: oid 6 updating version 577 to 578 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 578 (ObjNum 683 snap 177 seq_num 683) dirty exists 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1910: left oid 6 (ObjNum 683 snap 177 seq_num 683) 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: finishing write tid 1 to smithi16234619-36 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: finishing write tid 2 to smithi16234619-36 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: finishing write tid 3 to smithi16234619-36 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: finishing write tid 4 to smithi16234619-36 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: finishing write tid 5 to smithi16234619-36 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: oid 36 updating version 0 to 721 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: oid 36 version 721 is already newer than 720 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: oid 36 version 721 is already newer than 718 2024-09-16T15:08:51.299 INFO:tasks.rados.rados.0.smithi162.stdout:1911: oid 36 version 721 is already newer than 719 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 721 (ObjNum 684 snap 177 seq_num 684) dirty exists 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:1911: left oid 36 (ObjNum 684 snap 177 seq_num 684) 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:1910: done (2 left) 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:1911: done (1 left) 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:1914: write (excl) oid 39 current snap is 177 2024-09-16T15:08:51.300 INFO:tasks.rados.rados.0.smithi162.stdout:1914: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2024-09-16T15:08:51.304 INFO:tasks.rados.rados.0.smithi162.stdout:1914: writing smithi16234619-39 from 712795 to 1130130 tid 1 2024-09-16T15:08:51.311 INFO:tasks.rados.rados.0.smithi162.stdout:1914: writing smithi16234619-39 from 1784636 to 2325208 tid 2 2024-09-16T15:08:51.317 INFO:tasks.rados.rados.0.smithi162.stdout:1914: writing smithi16234619-39 from 2894750 to 3352667 tid 3 2024-09-16T15:08:51.317 INFO:tasks.rados.rados.0.smithi162.stdout:1915: write oid 22 current snap is 177 2024-09-16T15:08:51.317 INFO:tasks.rados.rados.0.smithi162.stdout:1915: seq_num 686 ranges {729602=492584,1843118=684230,3322173=207429} 2024-09-16T15:08:51.325 INFO:tasks.rados.rados.0.smithi162.stdout:1915: writing smithi16234619-22 from 729602 to 1222186 tid 1 2024-09-16T15:08:51.333 INFO:tasks.rados.rados.0.smithi162.stdout:1915: writing smithi16234619-22 from 1843118 to 2527348 tid 2 2024-09-16T15:08:51.339 INFO:tasks.rados.rados.0.smithi162.stdout:1915: writing smithi16234619-22 from 3322173 to 3529602 tid 3 2024-09-16T15:08:51.339 INFO:tasks.rados.rados.0.smithi162.stdout:1916: delete oid 14 current snap is 177 2024-09-16T15:08:51.346 INFO:tasks.rados.rados.0.smithi162.stdout:1914: finishing write tid 1 to smithi16234619-39 2024-09-16T15:08:51.346 INFO:tasks.rados.rados.0.smithi162.stdout:1914: finishing write tid 2 to smithi16234619-39 2024-09-16T15:08:51.353 INFO:tasks.rados.rados.0.smithi162.stdout:1914: finishing write tid 3 to smithi16234619-39 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: finishing write tid 4 to smithi16234619-39 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: finishing write tid 5 to smithi16234619-39 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: oid 39 updating version 0 to 830 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: oid 39 updating version 830 to 833 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: oid 39 version 833 is already newer than 832 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: oid 39 version 833 is already newer than 831 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 833 (ObjNum 685 snap 177 seq_num 685) dirty exists 2024-09-16T15:08:51.354 INFO:tasks.rados.rados.0.smithi162.stdout:1914: left oid 39 (ObjNum 685 snap 177 seq_num 685) 2024-09-16T15:08:51.364 INFO:tasks.rados.rados.0.smithi162.stdout:1915: finishing write tid 1 to smithi16234619-22 2024-09-16T15:08:51.364 INFO:tasks.rados.rados.0.smithi162.stdout:1915: finishing write tid 2 to smithi16234619-22 2024-09-16T15:08:51.365 INFO:tasks.rados.rados.0.smithi162.stdout:1913: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: finishing write tid 3 to smithi16234619-22 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: finishing write tid 4 to smithi16234619-22 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: finishing write tid 5 to smithi16234619-22 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: oid 22 updating version 0 to 638 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: oid 22 version 638 is already newer than 637 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: oid 22 version 638 is already newer than 635 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: oid 22 version 638 is already newer than 636 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 638 (ObjNum 686 snap 177 seq_num 686) dirty exists 2024-09-16T15:08:51.367 INFO:tasks.rados.rados.0.smithi162.stdout:1915: left oid 22 (ObjNum 686 snap 177 seq_num 686) 2024-09-16T15:08:51.387 INFO:tasks.rados.rados.0.smithi162.stdout:1914: done (3 left) 2024-09-16T15:08:51.387 INFO:tasks.rados.rados.0.smithi162.stdout:1915: done (2 left) 2024-09-16T15:08:51.387 INFO:tasks.rados.rados.0.smithi162.stdout:1916: done (1 left) 2024-09-16T15:08:51.388 INFO:tasks.rados.rados.0.smithi162.stdout:1917: snap_create 2024-09-16T15:08:51.388 INFO:tasks.rados.rados.0.smithi162.stdout:1913: finishing copy_from to smithi16234619-27 2024-09-16T15:08:51.388 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 696 (ObjNum 662 snap 171 seq_num 662) dirty exists 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1913: done (1 left) 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1917: done (0 left) 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1918: read oid 13 snap -1 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1918: expect (ObjNum 629 snap 158 seq_num 629) 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1919: copy_from oid 41 from oid 50 current snap is 178 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1920: write (excl) oid 47 current snap is 178 2024-09-16T15:08:52.321 INFO:tasks.rados.rados.0.smithi162.stdout:1920: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2024-09-16T15:08:52.329 INFO:tasks.rados.rados.0.smithi162.stdout:1920: writing smithi16234619-47 from 763216 to 1406298 tid 1 2024-09-16T15:08:52.336 INFO:tasks.rados.rados.0.smithi162.stdout:1920: writing smithi16234619-47 from 1843729 to 2531628 tid 2 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1920: writing smithi16234619-47 from 3260666 to 3563216 tid 3 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1921: write (excl) oid 5 current snap is 178 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1919: finishing copy_from to smithi16234619-41 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1919: got expected ENOENT (src dne) 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1919: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:08:52.343 INFO:tasks.rados.rados.0.smithi162.stdout:1918: expect (ObjNum 629 snap 158 seq_num 629) 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: finishing write tid 1 to smithi16234619-47 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: finishing write tid 2 to smithi16234619-47 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: finishing write tid 3 to smithi16234619-47 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: finishing write tid 4 to smithi16234619-47 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: finishing write tid 5 to smithi16234619-47 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: oid 47 updating version 0 to 523 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: oid 47 version 523 is already newer than 522 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: oid 47 updating version 523 to 524 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: oid 47 version 524 is already newer than 521 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 524 (ObjNum 688 snap 178 seq_num 688) dirty exists 2024-09-16T15:08:52.463 INFO:tasks.rados.rados.0.smithi162.stdout:1920: left oid 47 (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:08:52.464 INFO:tasks.rados.rados.0.smithi162.stdout:1921: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2024-09-16T15:08:52.471 INFO:tasks.rados.rados.0.smithi162.stdout:1921: writing smithi16234619-5 from 780023 to 1498354 tid 1 2024-09-16T15:08:52.476 INFO:tasks.rados.rados.0.smithi162.stdout:1921: writing smithi16234619-5 from 1902211 to 2450121 tid 2 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1921: writing smithi16234619-5 from 3004442 to 3580023 tid 3 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1918: done (3 left) 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1919: done (2 left) 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1920: done (1 left) 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1922: write (excl) oid 49 current snap is 178 2024-09-16T15:08:52.483 INFO:tasks.rados.rados.0.smithi162.stdout:1922: seq_num 690 ranges {796830=793580,2360693=691568} 2024-09-16T15:08:52.491 INFO:tasks.rados.rados.0.smithi162.stdout:1922: writing smithi16234619-49 from 796830 to 1590410 tid 1 2024-09-16T15:08:52.499 INFO:tasks.rados.rados.0.smithi162.stdout:1922: writing smithi16234619-49 from 2360693 to 3052261 tid 2 2024-09-16T15:08:52.499 INFO:tasks.rados.rados.0.smithi162.stdout:1923: delete oid 3 current snap is 178 2024-09-16T15:08:52.502 INFO:tasks.rados.rados.0.smithi162.stdout:1923: done (2 left) 2024-09-16T15:08:52.502 INFO:tasks.rados.rados.0.smithi162.stdout:1924: write oid 22 current snap is 178 2024-09-16T15:08:52.503 INFO:tasks.rados.rados.0.smithi162.stdout:1924: seq_num 692 ranges {430444=660431,1677657=695237,2970358=569527} 2024-09-16T15:08:52.508 INFO:tasks.rados.rados.0.smithi162.stdout:1924: writing smithi16234619-22 from 430444 to 1090875 tid 1 2024-09-16T15:08:52.515 INFO:tasks.rados.rados.0.smithi162.stdout:1924: writing smithi16234619-22 from 1677657 to 2372894 tid 2 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1924: writing smithi16234619-22 from 2970358 to 3539885 tid 3 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1925: read oid 23 snap -1 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1925: expect (ObjNum 586 snap 147 seq_num 586) 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1926: snap_create 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1921: finishing write tid 1 to smithi16234619-5 2024-09-16T15:08:52.522 INFO:tasks.rados.rados.0.smithi162.stdout:1921: finishing write tid 2 to smithi16234619-5 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: finishing write tid 3 to smithi16234619-5 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: finishing write tid 4 to smithi16234619-5 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: finishing write tid 5 to smithi16234619-5 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: oid 5 updating version 0 to 836 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: oid 5 version 836 is already newer than 834 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: oid 5 updating version 836 to 837 2024-09-16T15:08:52.523 INFO:tasks.rados.rados.0.smithi162.stdout:1921: oid 5 version 837 is already newer than 835 2024-09-16T15:08:52.524 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 837 (ObjNum 689 snap 178 seq_num 689) dirty exists 2024-09-16T15:08:52.524 INFO:tasks.rados.rados.0.smithi162.stdout:1921: left oid 5 (ObjNum 689 snap 178 seq_num 689) 2024-09-16T15:08:52.534 INFO:tasks.rados.rados.0.smithi162.stdout:1922: finishing write tid 1 to smithi16234619-49 2024-09-16T15:08:52.535 INFO:tasks.rados.rados.0.smithi162.stdout:1925: expect (ObjNum 586 snap 147 seq_num 586) 2024-09-16T15:08:52.698 INFO:tasks.rados.rados.0.smithi162.stdout:1922: finishing write tid 2 to smithi16234619-49 2024-09-16T15:08:52.698 INFO:tasks.rados.rados.0.smithi162.stdout:1922: finishing write tid 3 to smithi16234619-49 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1922: finishing write tid 4 to smithi16234619-49 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1922: oid 49 updating version 0 to 640 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1922: oid 49 updating version 640 to 641 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1922: oid 49 version 641 is already newer than 639 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 641 (ObjNum 690 snap 178 seq_num 690) dirty exists 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1922: left oid 49 (ObjNum 690 snap 178 seq_num 690) 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: finishing write tid 1 to smithi16234619-22 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: finishing write tid 2 to smithi16234619-22 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: finishing write tid 3 to smithi16234619-22 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: finishing write tid 4 to smithi16234619-22 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: finishing write tid 5 to smithi16234619-22 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: oid 22 updating version 0 to 644 2024-09-16T15:08:52.699 INFO:tasks.rados.rados.0.smithi162.stdout:1924: oid 22 version 644 is already newer than 643 2024-09-16T15:08:52.700 INFO:tasks.rados.rados.0.smithi162.stdout:1924: oid 22 updating version 644 to 645 2024-09-16T15:08:52.700 INFO:tasks.rados.rados.0.smithi162.stdout:1924: oid 22 version 645 is already newer than 642 2024-09-16T15:08:52.700 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 645 (ObjNum 692 snap 178 seq_num 692) dirty exists 2024-09-16T15:08:52.700 INFO:tasks.rados.rados.0.smithi162.stdout:1924: left oid 22 (ObjNum 692 snap 178 seq_num 692) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1921: done (4 left) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1922: done (3 left) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1924: done (2 left) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1925: done (1 left) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1926: done (0 left) 2024-09-16T15:08:52.849 INFO:tasks.rados.rados.0.smithi162.stdout:1927: snap_remove snap 177 2024-09-16T15:08:53.861 INFO:tasks.rados.rados.0.smithi162.stdout:1927: done (0 left) 2024-09-16T15:08:53.862 INFO:tasks.rados.rados.0.smithi162.stdout:1928: copy_from oid 2 from oid 38 current snap is 179 2024-09-16T15:08:53.862 INFO:tasks.rados.rados.0.smithi162.stdout:1929: delete oid 30 current snap is 179 2024-09-16T15:08:53.867 INFO:tasks.rados.rados.0.smithi162.stdout:1929: done (1 left) 2024-09-16T15:08:53.867 INFO:tasks.rados.rados.0.smithi162.stdout:1930: snap_remove snap 178 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1930: done (1 left) 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1931: rollback oid 18 current snap is 179 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 174 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1928: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1928: finishing copy_from to smithi16234619-2 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 571 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1928: done (1 left) 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1932: read oid 50 snap -1 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1932: expect deleted 2024-09-16T15:08:54.868 INFO:tasks.rados.rados.0.smithi162.stdout:1933: read oid 45 snap -1 2024-09-16T15:08:54.869 INFO:tasks.rados.rados.0.smithi162.stdout:1933: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:08:54.869 INFO:tasks.rados.rados.0.smithi162.stdout:1934: copy_from oid 33 from oid 13 current snap is 179 2024-09-16T15:08:54.869 INFO:tasks.rados.rados.0.smithi162.stdout:1935: snap_create 2024-09-16T15:08:54.872 INFO:tasks.rados.rados.0.smithi162.stdout:1931: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:08:54.872 INFO:tasks.rados.rados.0.smithi162.stdout:1931: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:08:54.872 INFO:tasks.rados.rados.0.smithi162.stdout:1931: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:08:54.872 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 838 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:08:54.874 INFO:tasks.rados.rados.0.smithi162.stdout:1933: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:08:54.957 INFO:tasks.rados.rados.0.smithi162.stdout:1934: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:08:54.957 INFO:tasks.rados.rados.0.smithi162.stdout:1934: finishing copy_from to smithi16234619-33 2024-09-16T15:08:54.957 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 772 (ObjNum 629 snap 158 seq_num 629) dirty exists 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1931: done (4 left) 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1932: done (3 left) 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1933: done (2 left) 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1934: done (1 left) 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1935: done (0 left) 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1936: write (excl) oid 29 current snap is 180 2024-09-16T15:08:56.077 INFO:tasks.rados.rados.0.smithi162.stdout:1936: seq_num 694 ranges {464058=410929,1394621=698906,2625204=592071,3656107=7951} 2024-09-16T15:08:56.081 INFO:tasks.rados.rados.0.smithi162.stdout:1936: writing smithi16234619-29 from 464058 to 874987 tid 1 2024-09-16T15:08:56.088 INFO:tasks.rados.rados.0.smithi162.stdout:1936: writing smithi16234619-29 from 1394621 to 2093527 tid 2 2024-09-16T15:08:56.094 INFO:tasks.rados.rados.0.smithi162.stdout:1936: writing smithi16234619-29 from 2625204 to 3217275 tid 3 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:1936: writing smithi16234619-29 from 3656107 to 3664058 tid 4 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:1937: rollback oid 36 current snap is 180 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 174 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:1938: rollback oid 19 current snap is 180 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 179 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:1939: write oid 11 current snap is 180 2024-09-16T15:08:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:1939: seq_num 695 ranges {480865=486178,1453103=442564,2652627=519696} 2024-09-16T15:08:56.102 INFO:tasks.rados.rados.0.smithi162.stdout:1939: writing smithi16234619-11 from 480865 to 967043 tid 1 2024-09-16T15:08:56.107 INFO:tasks.rados.rados.0.smithi162.stdout:1939: writing smithi16234619-11 from 1453103 to 1895667 tid 2 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1939: writing smithi16234619-11 from 2652627 to 3172323 tid 3 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1940: snap_create 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1937: finishing rollback tid 0 to smithi16234619-36 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1937: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1937: finishing rollback tid 2 to smithi16234619-36 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 729 (ObjNum 620 snap 156 seq_num 620) dirty exists 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1938: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1938: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:1938: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:08:56.115 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 733 (ObjNum 656 snap 169 seq_num 656) dirty exists 2024-09-16T15:08:56.122 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 1 to smithi16234619-29 2024-09-16T15:08:56.131 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 2 to smithi16234619-29 2024-09-16T15:08:56.133 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 3 to smithi16234619-29 2024-09-16T15:08:56.133 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 4 to smithi16234619-29 2024-09-16T15:08:56.133 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 5 to smithi16234619-29 2024-09-16T15:08:56.133 INFO:tasks.rados.rados.0.smithi162.stdout:1936: finishing write tid 6 to smithi16234619-29 2024-09-16T15:08:56.133 INFO:tasks.rados.rados.0.smithi162.stdout:1936: oid 29 updating version 0 to 842 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:1936: oid 29 version 842 is already newer than 840 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:1936: oid 29 version 842 is already newer than 841 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:1936: oid 29 version 842 is already newer than 839 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:1936: oid 29 updating version 842 to 843 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 843 (ObjNum 694 snap 180 seq_num 694) dirty exists 2024-09-16T15:08:56.134 INFO:tasks.rados.rados.0.smithi162.stdout:1936: left oid 29 (ObjNum 694 snap 180 seq_num 694) 2024-09-16T15:08:56.144 INFO:tasks.rados.rados.0.smithi162.stdout:1939: finishing write tid 1 to smithi16234619-11 2024-09-16T15:08:56.144 INFO:tasks.rados.rados.0.smithi162.stdout:1939: finishing write tid 2 to smithi16234619-11 2024-09-16T15:08:56.145 INFO:tasks.rados.rados.0.smithi162.stdout:1939: finishing write tid 3 to smithi16234619-11 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: finishing write tid 4 to smithi16234619-11 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: finishing write tid 5 to smithi16234619-11 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: oid 11 updating version 0 to 149 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: oid 11 version 149 is already newer than 147 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: oid 11 version 149 is already newer than 148 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: oid 11 version 149 is already newer than 146 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 149 (ObjNum 695 snap 180 seq_num 695) dirty exists 2024-09-16T15:08:56.146 INFO:tasks.rados.rados.0.smithi162.stdout:1939: left oid 11 (ObjNum 695 snap 180 seq_num 695) 2024-09-16T15:08:57.085 INFO:tasks.rados.rados.0.smithi162.stdout:1936: done (4 left) 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1937: done (3 left) 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1938: done (2 left) 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1939: done (1 left) 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1940: done (0 left) 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1941: write (excl) oid 43 current snap is 181 2024-09-16T15:08:57.086 INFO:tasks.rados.rados.0.smithi162.stdout:1941: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2024-09-16T15:08:57.091 INFO:tasks.rados.rados.0.smithi162.stdout:1941: writing smithi16234619-43 from 497672 to 1059099 tid 1 2024-09-16T15:08:57.100 INFO:tasks.rados.rados.0.smithi162.stdout:1941: writing smithi16234619-43 from 1795232 to 2497807 tid 2 2024-09-16T15:08:57.109 INFO:tasks.rados.rados.0.smithi162.stdout:1941: writing smithi16234619-43 from 2963697 to 3694665 tid 3 2024-09-16T15:08:57.109 INFO:tasks.rados.rados.0.smithi162.stdout:1942: delete oid 4 current snap is 181 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1942: done (1 left) 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1943: read oid 20 snap -1 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1943: expect (ObjNum 666 snap 172 seq_num 666) 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1944: rollback oid 12 current snap is 181 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 179 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1945: read oid 37 snap 174 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1945: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1946: copy_from oid 17 from oid 14 current snap is 181 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1947: write oid 34 current snap is 181 2024-09-16T15:08:57.115 INFO:tasks.rados.rados.0.smithi162.stdout:1947: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2024-09-16T15:08:57.122 INFO:tasks.rados.rados.0.smithi162.stdout:1947: writing smithi16234619-34 from 531286 to 1243211 tid 1 2024-09-16T15:08:57.129 INFO:tasks.rados.rados.0.smithi162.stdout:1947: writing smithi16234619-34 from 1912196 to 2618440 tid 2 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1947: writing smithi16234619-34 from 3018543 to 3731286 tid 3 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1948: read oid 40 snap -1 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1948: expect deleted 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1949: write oid 49 current snap is 181 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1946: finishing copy_from to smithi16234619-17 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1946: got expected ENOENT (src dne) 2024-09-16T15:08:57.136 INFO:tasks.rados.rados.0.smithi162.stdout:1949: seq_num 699 ranges {548093=787174,1970678=449902,3045966=681137} 2024-09-16T15:08:57.143 INFO:tasks.rados.rados.0.smithi162.stdout:1949: writing smithi16234619-49 from 548093 to 1335267 tid 1 2024-09-16T15:08:57.149 INFO:tasks.rados.rados.0.smithi162.stdout:1949: writing smithi16234619-49 from 1970678 to 2420580 tid 2 2024-09-16T15:08:57.156 INFO:tasks.rados.rados.0.smithi162.stdout:1949: writing smithi16234619-49 from 3045966 to 3727103 tid 3 2024-09-16T15:08:57.156 INFO:tasks.rados.rados.0.smithi162.stdout:1950: copy_from oid 28 from oid 42 current snap is 181 2024-09-16T15:08:57.156 INFO:tasks.rados.rados.0.smithi162.stdout:1946: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:08:57.157 INFO:tasks.rados.rados.0.smithi162.stdout:1944: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:08:57.157 INFO:tasks.rados.rados.0.smithi162.stdout:1944: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:08:57.157 INFO:tasks.rados.rados.0.smithi162.stdout:1944: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:08:57.157 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 321 (ObjNum 585 snap 146 seq_num 585) dirty exists 2024-09-16T15:08:57.157 INFO:tasks.rados.rados.0.smithi162.stdout:1945: expect (ObjNum 625 snap 157 seq_num 625) 2024-09-16T15:08:57.236 INFO:tasks.rados.rados.0.smithi162.stdout:1943: expect (ObjNum 666 snap 172 seq_num 666) 2024-09-16T15:08:57.427 INFO:tasks.rados.rados.0.smithi162.stdout:1941: finishing write tid 1 to smithi16234619-43 2024-09-16T15:08:57.427 INFO:tasks.rados.rados.0.smithi162.stdout:1941: finishing write tid 2 to smithi16234619-43 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: finishing write tid 3 to smithi16234619-43 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: finishing write tid 4 to smithi16234619-43 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: finishing write tid 5 to smithi16234619-43 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: oid 43 updating version 0 to 528 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: oid 43 version 528 is already newer than 527 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: oid 43 version 528 is already newer than 526 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: oid 43 version 528 is already newer than 525 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 528 (ObjNum 696 snap 181 seq_num 696) dirty exists 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1941: left oid 43 (ObjNum 696 snap 181 seq_num 696) 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1947: finishing write tid 1 to smithi16234619-34 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1947: finishing write tid 2 to smithi16234619-34 2024-09-16T15:08:57.428 INFO:tasks.rados.rados.0.smithi162.stdout:1947: finishing write tid 3 to smithi16234619-34 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: finishing write tid 4 to smithi16234619-34 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: finishing write tid 5 to smithi16234619-34 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: oid 34 updating version 0 to 738 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: oid 34 version 738 is already newer than 737 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: oid 34 version 738 is already newer than 735 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: oid 34 version 738 is already newer than 736 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 738 (ObjNum 698 snap 181 seq_num 698) dirty exists 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1947: left oid 34 (ObjNum 698 snap 181 seq_num 698) 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1950: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1949: finishing write tid 1 to smithi16234619-49 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1941: done (8 left) 2024-09-16T15:08:57.429 INFO:tasks.rados.rados.0.smithi162.stdout:1943: done (7 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1944: done (6 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1945: done (5 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1946: done (4 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1947: done (3 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1948: done (2 left) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1951: read oid 3 snap -1 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1951: expect deleted 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1952: read oid 20 snap -1 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1952: expect (ObjNum 666 snap 172 seq_num 666) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1953: read oid 16 snap -1 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1953: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1954: read oid 15 snap -1 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1954: expect (ObjNum 612 snap 154 seq_num 612) 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1955: write (excl) oid 33 current snap is 181 2024-09-16T15:08:57.430 INFO:tasks.rados.rados.0.smithi162.stdout:1955: seq_num 700 ranges {564900=578776,1629160=709913,3073389=492409} 2024-09-16T15:08:57.433 INFO:tasks.rados.rados.0.smithi162.stdout:1955: writing smithi16234619-33 from 564900 to 1143676 tid 1 2024-09-16T15:08:57.441 INFO:tasks.rados.rados.0.smithi162.stdout:1955: writing smithi16234619-33 from 1629160 to 2339073 tid 2 2024-09-16T15:08:57.449 INFO:tasks.rados.rados.0.smithi162.stdout:1955: writing smithi16234619-33 from 3073389 to 3565798 tid 3 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1956: snap_create 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: finishing write tid 2 to smithi16234619-49 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1950: finishing copy_from to smithi16234619-28 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 798 (ObjNum 660 snap 171 seq_num 660) dirty exists 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: finishing write tid 3 to smithi16234619-49 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: finishing write tid 4 to smithi16234619-49 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: finishing write tid 5 to smithi16234619-49 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: oid 49 updating version 0 to 649 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: oid 49 version 649 is already newer than 646 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: oid 49 version 649 is already newer than 647 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: oid 49 version 649 is already newer than 648 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 649 (ObjNum 699 snap 181 seq_num 699) dirty exists 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1949: left oid 49 (ObjNum 699 snap 181 seq_num 699) 2024-09-16T15:08:57.450 INFO:tasks.rados.rados.0.smithi162.stdout:1953: expect (ObjNum 444 snap 106 seq_num 444) 2024-09-16T15:08:57.599 INFO:tasks.rados.rados.0.smithi162.stdout:1952: expect (ObjNum 666 snap 172 seq_num 666) 2024-09-16T15:08:57.790 INFO:tasks.rados.rados.0.smithi162.stdout:1954: expect (ObjNum 612 snap 154 seq_num 612) 2024-09-16T15:08:57.919 INFO:tasks.rados.rados.0.smithi162.stdout:1955: finishing write tid 1 to smithi16234619-33 2024-09-16T15:08:57.919 INFO:tasks.rados.rados.0.smithi162.stdout:1955: finishing write tid 2 to smithi16234619-33 2024-09-16T15:08:57.919 INFO:tasks.rados.rados.0.smithi162.stdout:1955: finishing write tid 3 to smithi16234619-33 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: finishing write tid 4 to smithi16234619-33 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: finishing write tid 5 to smithi16234619-33 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: oid 33 updating version 0 to 774 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: oid 33 version 774 is already newer than 773 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: oid 33 updating version 774 to 775 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: oid 33 updating version 775 to 776 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 776 (ObjNum 700 snap 181 seq_num 700) dirty exists 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1955: left oid 33 (ObjNum 700 snap 181 seq_num 700) 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1949: done (7 left) 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1950: done (6 left) 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1951: done (5 left) 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1952: done (4 left) 2024-09-16T15:08:57.920 INFO:tasks.rados.rados.0.smithi162.stdout:1953: done (3 left) 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1954: done (2 left) 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1955: done (1 left) 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1956: done (0 left) 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1957: read oid 33 snap -1 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1957: expect (ObjNum 700 snap 181 seq_num 700) 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1958: copy_from oid 20 from oid 1 current snap is 182 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1959: copy_from oid 49 from oid 13 current snap is 182 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1960: write oid 9 current snap is 182 2024-09-16T15:08:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:1960: seq_num 701 ranges {581707=654025,1687642=453571,2700812=703681} 2024-09-16T15:08:57.926 INFO:tasks.rados.rados.0.smithi162.stdout:1960: writing smithi16234619-9 from 581707 to 1235732 tid 1 2024-09-16T15:08:57.931 INFO:tasks.rados.rados.0.smithi162.stdout:1960: writing smithi16234619-9 from 1687642 to 2141213 tid 2 2024-09-16T15:08:57.939 INFO:tasks.rados.rados.0.smithi162.stdout:1960: writing smithi16234619-9 from 2700812 to 3404493 tid 3 2024-09-16T15:08:57.939 INFO:tasks.rados.rados.0.smithi162.stdout:1961: write (excl) oid 39 current snap is 182 2024-09-16T15:08:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:1961: seq_num 702 ranges {598514=729274,1746124=597229,3128235=514953} 2024-09-16T15:08:57.948 INFO:tasks.rados.rados.0.smithi162.stdout:1961: writing smithi16234619-39 from 598514 to 1327788 tid 1 2024-09-16T15:08:57.954 INFO:tasks.rados.rados.0.smithi162.stdout:1961: writing smithi16234619-39 from 1746124 to 2343353 tid 2 2024-09-16T15:08:57.962 INFO:tasks.rados.rados.0.smithi162.stdout:1961: writing smithi16234619-39 from 3128235 to 3643188 tid 3 2024-09-16T15:08:57.962 INFO:tasks.rados.rados.0.smithi162.stdout:1962: write (excl) oid 32 current snap is 182 2024-09-16T15:08:57.962 INFO:tasks.rados.rados.0.smithi162.stdout:1962: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2024-09-16T15:08:57.967 INFO:tasks.rados.rados.0.smithi162.stdout:1962: writing smithi16234619-32 from 615321 to 1019844 tid 1 2024-09-16T15:08:57.974 INFO:tasks.rados.rados.0.smithi162.stdout:1962: writing smithi16234619-32 from 1804606 to 2261846 tid 2 2024-09-16T15:08:57.981 INFO:tasks.rados.rados.0.smithi162.stdout:1962: writing smithi16234619-32 from 2755658 to 3481883 tid 3 2024-09-16T15:08:57.981 INFO:tasks.rados.rados.0.smithi162.stdout:1963: write oid 42 current snap is 182 2024-09-16T15:08:57.981 INFO:tasks.rados.rados.0.smithi162.stdout:1963: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2024-09-16T15:08:57.987 INFO:tasks.rados.rados.0.smithi162.stdout:1963: writing smithi16234619-42 from 632128 to 1111900 tid 1 2024-09-16T15:08:57.994 INFO:tasks.rados.rados.0.smithi162.stdout:1963: writing smithi16234619-42 from 1746735 to 2347633 tid 2 2024-09-16T15:08:58.001 INFO:tasks.rados.rados.0.smithi162.stdout:1963: writing smithi16234619-42 from 3066728 to 3720578 tid 3 2024-09-16T15:08:58.002 INFO:tasks.rados.rados.0.smithi162.stdout:1964: snap_create 2024-09-16T15:08:58.002 INFO:tasks.rados.rados.0.smithi162.stdout:1957: expect (ObjNum 700 snap 181 seq_num 700) 2024-09-16T15:08:58.225 INFO:tasks.rados.rados.0.smithi162.stdout:1959: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:08:58.225 INFO:tasks.rados.rados.0.smithi162.stdout:1959: finishing copy_from to smithi16234619-49 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 772 (ObjNum 629 snap 158 seq_num 629) dirty exists 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: finishing write tid 1 to smithi16234619-9 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: finishing write tid 2 to smithi16234619-9 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: finishing write tid 3 to smithi16234619-9 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: finishing write tid 4 to smithi16234619-9 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: finishing write tid 5 to smithi16234619-9 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: oid 9 updating version 0 to 749 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: oid 9 version 749 is already newer than 747 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: oid 9 version 749 is already newer than 746 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: oid 9 version 749 is already newer than 748 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 749 (ObjNum 701 snap 182 seq_num 701) dirty exists 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1960: left oid 9 (ObjNum 701 snap 182 seq_num 701) 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1961: finishing write tid 1 to smithi16234619-39 2024-09-16T15:08:58.226 INFO:tasks.rados.rados.0.smithi162.stdout:1962: finishing write tid 1 to smithi16234619-32 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1962: finishing write tid 2 to smithi16234619-32 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1962: finishing write tid 3 to smithi16234619-32 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: finishing write tid 2 to smithi16234619-39 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: finishing write tid 3 to smithi16234619-39 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: finishing write tid 4 to smithi16234619-39 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: finishing write tid 5 to smithi16234619-39 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: oid 39 updating version 0 to 845 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: oid 39 updating version 845 to 846 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: oid 39 updating version 846 to 847 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: oid 39 updating version 847 to 848 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 848 (ObjNum 702 snap 182 seq_num 702) dirty exists 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1961: left oid 39 (ObjNum 702 snap 182 seq_num 702) 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1958: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1962: finishing write tid 4 to smithi16234619-32 2024-09-16T15:08:58.227 INFO:tasks.rados.rados.0.smithi162.stdout:1962: finishing write tid 5 to smithi16234619-32 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1962: oid 32 updating version 0 to 326 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1962: oid 32 version 326 is already newer than 323 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1962: oid 32 version 326 is already newer than 324 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1962: oid 32 version 326 is already newer than 325 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 326 (ObjNum 703 snap 182 seq_num 703) dirty exists 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1962: left oid 32 (ObjNum 703 snap 182 seq_num 703) 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:1958: finishing copy_from to smithi16234619-20 2024-09-16T15:08:58.228 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 744 (ObjNum 680 snap 177 seq_num 680) dirty exists 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: finishing write tid 1 to smithi16234619-42 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: finishing write tid 2 to smithi16234619-42 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: finishing write tid 3 to smithi16234619-42 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: finishing write tid 4 to smithi16234619-42 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: finishing write tid 5 to smithi16234619-42 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: oid 42 updating version 0 to 851 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: oid 42 version 851 is already newer than 850 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: oid 42 updating version 851 to 853 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: oid 42 version 853 is already newer than 852 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 853 (ObjNum 704 snap 182 seq_num 704) dirty exists 2024-09-16T15:08:58.229 INFO:tasks.rados.rados.0.smithi162.stdout:1963: left oid 42 (ObjNum 704 snap 182 seq_num 704) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1957: done (7 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1958: done (6 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1959: done (5 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1960: done (4 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1961: done (3 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1962: done (2 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1963: done (1 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1964: done (0 left) 2024-09-16T15:08:58.856 INFO:tasks.rados.rados.0.smithi162.stdout:1965: read oid 44 snap -1 2024-09-16T15:08:58.857 INFO:tasks.rados.rados.0.smithi162.stdout:1965: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:08:58.857 INFO:tasks.rados.rados.0.smithi162.stdout:1966: snap_create 2024-09-16T15:08:58.863 INFO:tasks.rados.rados.0.smithi162.stdout:1965: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:09:00.279 INFO:tasks.rados.rados.0.smithi162.stdout:1965: done (1 left) 2024-09-16T15:09:00.279 INFO:tasks.rados.rados.0.smithi162.stdout:1966: done (0 left) 2024-09-16T15:09:00.279 INFO:tasks.rados.rados.0.smithi162.stdout:1967: snap_remove snap 179 2024-09-16T15:09:01.475 INFO:tasks.rados.rados.0.smithi162.stdout:1967: done (0 left) 2024-09-16T15:09:01.475 INFO:tasks.rados.rados.0.smithi162.stdout:1968: delete oid 15 current snap is 184 2024-09-16T15:09:01.483 INFO:tasks.rados.rados.0.smithi162.stdout:1968: done (0 left) 2024-09-16T15:09:01.483 INFO:tasks.rados.rados.0.smithi162.stdout:1969: snap_remove snap 181 2024-09-16T15:09:02.494 INFO:tasks.rados.rados.0.smithi162.stdout:1969: done (0 left) 2024-09-16T15:09:02.494 INFO:tasks.rados.rados.0.smithi162.stdout:1970: delete oid 20 current snap is 184 2024-09-16T15:09:02.499 INFO:tasks.rados.rados.0.smithi162.stdout:1970: done (0 left) 2024-09-16T15:09:02.500 INFO:tasks.rados.rados.0.smithi162.stdout:1971: copy_from oid 27 from oid 42 current snap is 184 2024-09-16T15:09:02.500 INFO:tasks.rados.rados.0.smithi162.stdout:1972: write oid 35 current snap is 184 2024-09-16T15:09:02.500 INFO:tasks.rados.rados.0.smithi162.stdout:1972: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2024-09-16T15:09:02.507 INFO:tasks.rados.rados.0.smithi162.stdout:1972: writing smithi16234619-35 from 682549 to 1388068 tid 1 2024-09-16T15:09:02.514 INFO:tasks.rados.rados.0.smithi162.stdout:1972: writing smithi16234619-35 from 1922181 to 2386759 tid 2 2024-09-16T15:09:02.522 INFO:tasks.rados.rados.0.smithi162.stdout:1972: writing smithi16234619-35 from 3148997 to 3636663 tid 3 2024-09-16T15:09:02.522 INFO:tasks.rados.rados.0.smithi162.stdout:1973: write oid 10 current snap is 184 2024-09-16T15:09:02.522 INFO:tasks.rados.rados.0.smithi162.stdout:1973: seq_num 708 ranges {699356=497121,1980663=608236,3176420=415291} 2024-09-16T15:09:02.529 INFO:tasks.rados.rados.0.smithi162.stdout:1973: writing smithi16234619-10 from 699356 to 1196477 tid 1 2024-09-16T15:09:02.538 INFO:tasks.rados.rados.0.smithi162.stdout:1973: writing smithi16234619-10 from 1980663 to 2588899 tid 2 2024-09-16T15:09:02.545 INFO:tasks.rados.rados.0.smithi162.stdout:1973: writing smithi16234619-10 from 3176420 to 3591711 tid 3 2024-09-16T15:09:02.545 INFO:tasks.rados.rados.0.smithi162.stdout:1974: write (excl) oid 15 current snap is 184 2024-09-16T15:09:02.545 INFO:tasks.rados.rados.0.smithi162.stdout:1974: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2024-09-16T15:09:02.553 INFO:tasks.rados.rados.0.smithi162.stdout:1974: writing smithi16234619-15 from 716163 to 1288533 tid 1 2024-09-16T15:09:02.560 INFO:tasks.rados.rados.0.smithi162.stdout:1974: writing smithi16234619-15 from 2039145 to 2507392 tid 2 2024-09-16T15:09:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1974: writing smithi16234619-15 from 3203843 to 3830406 tid 3 2024-09-16T15:09:02.569 INFO:tasks.rados.rados.0.smithi162.stdout:1975: write oid 20 current snap is 184 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: finishing write tid 1 to smithi16234619-35 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: finishing write tid 2 to smithi16234619-35 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: finishing write tid 3 to smithi16234619-35 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1973: finishing write tid 1 to smithi16234619-10 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: finishing write tid 4 to smithi16234619-35 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: finishing write tid 5 to smithi16234619-35 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: oid 35 updating version 0 to 580 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: oid 35 version 580 is already newer than 579 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: oid 35 updating version 580 to 582 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: oid 35 version 582 is already newer than 581 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 582 (ObjNum 707 snap 184 seq_num 707) dirty exists 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1972: left oid 35 (ObjNum 707 snap 184 seq_num 707) 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1973: finishing write tid 2 to smithi16234619-10 2024-09-16T15:09:02.570 INFO:tasks.rados.rados.0.smithi162.stdout:1975: seq_num 710 ranges {732970=647619,2097627=611905,3231266=437835} 2024-09-16T15:09:02.576 INFO:tasks.rados.rados.0.smithi162.stdout:1975: writing smithi16234619-20 from 732970 to 1380589 tid 1 2024-09-16T15:09:02.584 INFO:tasks.rados.rados.0.smithi162.stdout:1975: writing smithi16234619-20 from 2097627 to 2709532 tid 2 2024-09-16T15:09:02.589 INFO:tasks.rados.rados.0.smithi162.stdout:1975: writing smithi16234619-20 from 3231266 to 3669101 tid 3 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1972: done (4 left) 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1976: rollback oid 50 current snap is 184 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 174 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1971: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: finishing write tid 3 to smithi16234619-10 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: finishing write tid 4 to smithi16234619-10 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: finishing write tid 5 to smithi16234619-10 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: oid 10 updating version 0 to 888 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: oid 10 version 888 is already newer than 887 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: oid 10 version 888 is already newer than 885 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: oid 10 version 888 is already newer than 886 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 888 (ObjNum 708 snap 184 seq_num 708) dirty exists 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1973: left oid 10 (ObjNum 708 snap 184 seq_num 708) 2024-09-16T15:09:02.590 INFO:tasks.rados.rados.0.smithi162.stdout:1971: finishing copy_from to smithi16234619-27 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 853 (ObjNum 704 snap 182 seq_num 704) dirty exists 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:1971: done (4 left) 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:1973: done (3 left) 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:1977: delete oid 16 current snap is 184 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:1976: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:09:02.591 INFO:tasks.rados.rados.0.smithi162.stdout:1974: finishing write tid 1 to smithi16234619-15 2024-09-16T15:09:02.592 INFO:tasks.rados.rados.0.smithi162.stdout:1974: finishing write tid 2 to smithi16234619-15 2024-09-16T15:09:02.592 INFO:tasks.rados.rados.0.smithi162.stdout:1976: done (3 left) 2024-09-16T15:09:02.593 INFO:tasks.rados.rados.0.smithi162.stdout:1977: done (2 left) 2024-09-16T15:09:02.593 INFO:tasks.rados.rados.0.smithi162.stdout:1978: snap_remove snap 183 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1978: done (2 left) 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1979: write oid 50 current snap is 184 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1974: finishing write tid 3 to smithi16234619-15 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1974: finishing write tid 4 to smithi16234619-15 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1974: finishing write tid 5 to smithi16234619-15 2024-09-16T15:09:02.852 INFO:tasks.rados.rados.0.smithi162.stdout:1974: oid 15 updating version 0 to 845 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1974: oid 15 updating version 845 to 847 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1974: oid 15 version 847 is already newer than 846 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1974: oid 15 updating version 847 to 848 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 848 (ObjNum 709 snap 184 seq_num 709) dirty exists 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1974: left oid 15 (ObjNum 709 snap 184 seq_num 709) 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: finishing write tid 1 to smithi16234619-20 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: finishing write tid 2 to smithi16234619-20 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: finishing write tid 3 to smithi16234619-20 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: finishing write tid 4 to smithi16234619-20 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: finishing write tid 5 to smithi16234619-20 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: oid 20 updating version 0 to 781 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: oid 20 version 781 is already newer than 779 2024-09-16T15:09:02.853 INFO:tasks.rados.rados.0.smithi162.stdout:1975: oid 20 version 781 is already newer than 778 2024-09-16T15:09:02.854 INFO:tasks.rados.rados.0.smithi162.stdout:1975: oid 20 version 781 is already newer than 780 2024-09-16T15:09:02.854 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 781 (ObjNum 710 snap 184 seq_num 710) dirty exists 2024-09-16T15:09:02.854 INFO:tasks.rados.rados.0.smithi162.stdout:1975: left oid 20 (ObjNum 710 snap 184 seq_num 710) 2024-09-16T15:09:02.854 INFO:tasks.rados.rados.0.smithi162.stdout:1979: seq_num 712 ranges {766584=798117,2098238=615574,3169759=460379} 2024-09-16T15:09:02.860 INFO:tasks.rados.rados.0.smithi162.stdout:1979: writing smithi16234619-50 from 766584 to 1564701 tid 1 2024-09-16T15:09:02.867 INFO:tasks.rados.rados.0.smithi162.stdout:1979: writing smithi16234619-50 from 2098238 to 2713812 tid 2 2024-09-16T15:09:02.872 INFO:tasks.rados.rados.0.smithi162.stdout:1979: writing smithi16234619-50 from 3169759 to 3630138 tid 3 2024-09-16T15:09:02.872 INFO:tasks.rados.rados.0.smithi162.stdout:1974: done (2 left) 2024-09-16T15:09:02.872 INFO:tasks.rados.rados.0.smithi162.stdout:1975: done (1 left) 2024-09-16T15:09:02.872 INFO:tasks.rados.rados.0.smithi162.stdout:1980: copy_from oid 12 from oid 46 current snap is 184 2024-09-16T15:09:02.872 INFO:tasks.rados.rados.0.smithi162.stdout:1981: snap_create 2024-09-16T15:09:02.930 INFO:tasks.rados.rados.0.smithi162.stdout:1979: finishing write tid 1 to smithi16234619-50 2024-09-16T15:09:02.930 INFO:tasks.rados.rados.0.smithi162.stdout:1979: finishing write tid 2 to smithi16234619-50 2024-09-16T15:09:02.930 INFO:tasks.rados.rados.0.smithi162.stdout:1979: finishing write tid 3 to smithi16234619-50 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: finishing write tid 4 to smithi16234619-50 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: finishing write tid 5 to smithi16234619-50 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: oid 50 updating version 0 to 746 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: oid 50 updating version 746 to 747 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: oid 50 version 747 is already newer than 745 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: oid 50 updating version 747 to 748 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 748 (ObjNum 712 snap 184 seq_num 712) dirty exists 2024-09-16T15:09:02.932 INFO:tasks.rados.rados.0.smithi162.stdout:1979: left oid 50 (ObjNum 712 snap 184 seq_num 712) 2024-09-16T15:09:02.935 INFO:tasks.rados.rados.0.smithi162.stdout:1980: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:09:02.939 INFO:tasks.rados.rados.0.smithi162.stdout:1980: finishing copy_from to smithi16234619-12 2024-09-16T15:09:02.939 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 266 (ObjNum 671 snap 174 seq_num 671) dirty exists 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1979: done (2 left) 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1980: done (1 left) 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1981: done (0 left) 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1982: read oid 16 snap -1 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1982: expect deleted 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1983: copy_from oid 34 from oid 14 current snap is 185 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1984: write oid 22 current snap is 185 2024-09-16T15:09:03.856 INFO:tasks.rados.rados.0.smithi162.stdout:1984: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2024-09-16T15:09:03.862 INFO:tasks.rados.rados.0.smithi162.stdout:1984: writing smithi16234619-22 from 783391 to 1256757 tid 1 2024-09-16T15:09:03.867 INFO:tasks.rados.rados.0.smithi162.stdout:1984: writing smithi16234619-22 from 1756720 to 2232305 tid 2 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1984: writing smithi16234619-22 from 2797182 to 3585186 tid 3 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1985: read oid 15 snap -1 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1985: expect (ObjNum 709 snap 184 seq_num 709) 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1986: rollback oid 36 current snap is 185 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 174 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1983: finishing copy_from to smithi16234619-34 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1983: got expected ENOENT (src dne) 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1983: finishing copy_from racing read to smithi16234619-34 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1982: done (4 left) 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1983: done (3 left) 2024-09-16T15:09:03.875 INFO:tasks.rados.rados.0.smithi162.stdout:1987: rollback oid 32 current snap is 185 2024-09-16T15:09:03.876 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 182 2024-09-16T15:09:03.876 INFO:tasks.rados.rados.0.smithi162.stdout:1988: read oid 31 snap 184 2024-09-16T15:09:03.876 INFO:tasks.rados.rados.0.smithi162.stdout:1988: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:09:03.876 INFO:tasks.rados.rados.0.smithi162.stdout:1989: write oid 1 current snap is 185 2024-09-16T15:09:03.876 INFO:tasks.rados.rados.0.smithi162.stdout:1989: seq_num 714 ranges {400198=548615,1415202=584996} 2024-09-16T15:09:03.880 INFO:tasks.rados.rados.0.smithi162.stdout:1989: writing smithi16234619-1 from 400198 to 948813 tid 1 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1989: writing smithi16234619-1 from 1415202 to 2000198 tid 2 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1990: read oid 46 snap -1 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1990: expect (ObjNum 671 snap 174 seq_num 671) 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1991: read oid 11 snap 184 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1991: expect (ObjNum 695 snap 180 seq_num 695) 2024-09-16T15:09:03.886 INFO:tasks.rados.rados.0.smithi162.stdout:1992: snap_remove snap 180 2024-09-16T15:09:04.861 INFO:tasks.rados.rados.0.smithi162.stdout:1986: finishing rollback tid 0 to smithi16234619-36 2024-09-16T15:09:04.861 INFO:tasks.rados.rados.0.smithi162.stdout:1985: expect (ObjNum 709 snap 184 seq_num 709) 2024-09-16T15:09:04.990 INFO:tasks.rados.rados.0.smithi162.stdout:1986: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1986: finishing rollback tid 2 to smithi16234619-36 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 752 (ObjNum 620 snap 156 seq_num 620) dirty exists 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1987: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1987: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1987: finishing rollback tid 2 to smithi16234619-32 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 338 (ObjNum 703 snap 182 seq_num 703) dirty exists 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1985: done (8 left) 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1986: done (7 left) 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1987: done (6 left) 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1992: done (5 left) 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1993: read oid 14 snap -1 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1993: expect deleted 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1994: snap_create 2024-09-16T15:09:04.991 INFO:tasks.rados.rados.0.smithi162.stdout:1988: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:09:05.070 INFO:tasks.rados.rados.0.smithi162.stdout:1990: expect (ObjNum 671 snap 174 seq_num 671) 2024-09-16T15:09:05.267 INFO:tasks.rados.rados.0.smithi162.stdout:1991: expect (ObjNum 695 snap 180 seq_num 695) 2024-09-16T15:09:05.465 INFO:tasks.rados.rados.0.smithi162.stdout:1989: finishing write tid 1 to smithi16234619-1 2024-09-16T15:09:05.465 INFO:tasks.rados.rados.0.smithi162.stdout:1989: finishing write tid 2 to smithi16234619-1 2024-09-16T15:09:05.465 INFO:tasks.rados.rados.0.smithi162.stdout:1984: finishing write tid 1 to smithi16234619-22 2024-09-16T15:09:05.465 INFO:tasks.rados.rados.0.smithi162.stdout:1984: finishing write tid 2 to smithi16234619-22 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: finishing write tid 3 to smithi16234619-1 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: finishing write tid 4 to smithi16234619-1 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: oid 1 updating version 0 to 856 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: oid 1 version 856 is already newer than 854 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: oid 1 version 856 is already newer than 855 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 856 (ObjNum 714 snap 185 seq_num 714) dirty exists 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1989: left oid 1 (ObjNum 714 snap 185 seq_num 714) 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: finishing write tid 3 to smithi16234619-22 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: finishing write tid 4 to smithi16234619-22 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: finishing write tid 5 to smithi16234619-22 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: oid 22 updating version 0 to 774 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: oid 22 updating version 774 to 775 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: oid 22 version 775 is already newer than 773 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:1984: oid 22 updating version 775 to 776 2024-09-16T15:09:05.466 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 776 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:09:05.467 INFO:tasks.rados.rados.0.smithi162.stdout:1984: left oid 22 (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1984: done (6 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1988: done (5 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1989: done (4 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1990: done (3 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1991: done (2 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1993: done (1 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1994: done (0 left) 2024-09-16T15:09:05.865 INFO:tasks.rados.rados.0.smithi162.stdout:1995: read oid 15 snap -1 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1995: expect (ObjNum 709 snap 184 seq_num 709) 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1996: read oid 34 snap 185 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1996: expect (ObjNum 698 snap 181 seq_num 698) 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1997: read oid 39 snap -1 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1997: expect (ObjNum 702 snap 182 seq_num 702) 2024-09-16T15:09:05.866 INFO:tasks.rados.rados.0.smithi162.stdout:1998: snap_create 2024-09-16T15:09:05.874 INFO:tasks.rados.rados.0.smithi162.stdout:1995: expect (ObjNum 709 snap 184 seq_num 709) 2024-09-16T15:09:06.005 INFO:tasks.rados.rados.0.smithi162.stdout:1997: expect (ObjNum 702 snap 182 seq_num 702) 2024-09-16T15:09:06.212 INFO:tasks.rados.rados.0.smithi162.stdout:1996: expect (ObjNum 698 snap 181 seq_num 698) 2024-09-16T15:09:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:1995: done (3 left) 2024-09-16T15:09:06.865 INFO:tasks.rados.rados.0.smithi162.stdout:1996: done (2 left) 2024-09-16T15:09:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:1997: done (1 left) 2024-09-16T15:09:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:1998: done (0 left) 2024-09-16T15:09:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:1999: copy_from oid 48 from oid 37 current snap is 187 2024-09-16T15:09:06.866 INFO:tasks.rados.rados.0.smithi162.stdout:2000: snap_remove snap 174 2024-09-16T15:09:07.855 INFO:tasks.rados.rados.0.smithi162.stdout:2000: done (1 left) 2024-09-16T15:09:07.855 INFO:tasks.rados.rados.0.smithi162.stdout:2001: rollback oid 46 current snap is 187 2024-09-16T15:09:07.855 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 185 2024-09-16T15:09:07.855 INFO:tasks.rados.rados.0.smithi162.stdout:1999: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:1999: finishing copy_from to smithi16234619-48 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 571 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:1999: done (1 left) 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2002: rollback oid 33 current snap is 187 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 186 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2003: read oid 18 snap -1 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2003: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2004: rollback oid 15 current snap is 187 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 182 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2005: read oid 47 snap -1 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2005: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:07.856 INFO:tasks.rados.rados.0.smithi162.stdout:2006: snap_remove snap 184 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2006: done (5 left) 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2007: snap_create 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2002: finishing rollback tid 0 to smithi16234619-33 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2004: finishing rollback tid 0 to smithi16234619-15 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2004: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2004: finishing rollback tid 2 to smithi16234619-15 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 851 (ObjNum 612 snap 154 seq_num 612) dirty exists 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2001: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2001: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2001: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 269 (ObjNum 671 snap 174 seq_num 671) dirty exists 2024-09-16T15:09:08.857 INFO:tasks.rados.rados.0.smithi162.stdout:2002: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:09:08.858 INFO:tasks.rados.rados.0.smithi162.stdout:2002: finishing rollback tid 2 to smithi16234619-33 2024-09-16T15:09:08.858 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 784 (ObjNum 700 snap 181 seq_num 700) dirty exists 2024-09-16T15:09:08.858 INFO:tasks.rados.rados.0.smithi162.stdout:2005: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:09.050 INFO:tasks.rados.rados.0.smithi162.stdout:2003: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:09:09.859 INFO:tasks.rados.rados.0.smithi162.stdout:2001: done (5 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2002: done (4 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2003: done (3 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2004: done (2 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2005: done (1 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2007: done (0 left) 2024-09-16T15:09:09.860 INFO:tasks.rados.rados.0.smithi162.stdout:2008: snap_remove snap 187 2024-09-16T15:09:10.879 INFO:tasks.rados.rados.0.smithi162.stdout:2008: done (0 left) 2024-09-16T15:09:10.879 INFO:tasks.rados.rados.0.smithi162.stdout:2009: snap_create 2024-09-16T15:09:11.882 INFO:tasks.rados.rados.0.smithi162.stdout:2009: done (0 left) 2024-09-16T15:09:11.882 INFO:tasks.rados.rados.0.smithi162.stdout:2010: read oid 22 snap -1 2024-09-16T15:09:11.883 INFO:tasks.rados.rados.0.smithi162.stdout:2010: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:09:11.883 INFO:tasks.rados.rados.0.smithi162.stdout:2011: write oid 8 current snap is 189 2024-09-16T15:09:11.883 INFO:tasks.rados.rados.0.smithi162.stdout:2011: seq_num 715 ranges {417005=740217,1590037=426968} 2024-09-16T15:09:11.888 INFO:tasks.rados.rados.0.smithi162.stdout:2011: writing smithi16234619-8 from 417005 to 1157222 tid 1 2024-09-16T15:09:11.893 INFO:tasks.rados.rados.0.smithi162.stdout:2011: writing smithi16234619-8 from 1590037 to 2017005 tid 2 2024-09-16T15:09:11.893 INFO:tasks.rados.rados.0.smithi162.stdout:2012: snap_create 2024-09-16T15:09:11.907 INFO:tasks.rados.rados.0.smithi162.stdout:2010: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: finishing write tid 1 to smithi16234619-8 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: finishing write tid 2 to smithi16234619-8 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: finishing write tid 3 to smithi16234619-8 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: finishing write tid 4 to smithi16234619-8 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: oid 8 updating version 0 to 839 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: oid 8 updating version 839 to 840 2024-09-16T15:09:12.160 INFO:tasks.rados.rados.0.smithi162.stdout:2011: oid 8 updating version 840 to 841 2024-09-16T15:09:12.161 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 841 (ObjNum 715 snap 189 seq_num 715) dirty exists 2024-09-16T15:09:12.161 INFO:tasks.rados.rados.0.smithi162.stdout:2011: left oid 8 (ObjNum 715 snap 189 seq_num 715) 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2010: done (2 left) 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2011: done (1 left) 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2012: done (0 left) 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2013: copy_from oid 30 from oid 9 current snap is 190 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2014: write oid 12 current snap is 190 2024-09-16T15:09:12.859 INFO:tasks.rados.rados.0.smithi162.stdout:2014: seq_num 716 ranges {433812=415466,1648519=385293} 2024-09-16T15:09:12.863 INFO:tasks.rados.rados.0.smithi162.stdout:2014: writing smithi16234619-12 from 433812 to 849278 tid 1 2024-09-16T15:09:12.870 INFO:tasks.rados.rados.0.smithi162.stdout:2014: writing smithi16234619-12 from 1648519 to 2033812 tid 2 2024-09-16T15:09:12.870 INFO:tasks.rados.rados.0.smithi162.stdout:2015: write oid 4 current snap is 190 2024-09-16T15:09:12.870 INFO:tasks.rados.rados.0.smithi162.stdout:2015: seq_num 717 ranges {450619=490715,1590648=459971} 2024-09-16T15:09:12.875 INFO:tasks.rados.rados.0.smithi162.stdout:2015: writing smithi16234619-4 from 450619 to 941334 tid 1 2024-09-16T15:09:12.881 INFO:tasks.rados.rados.0.smithi162.stdout:2015: writing smithi16234619-4 from 1590648 to 2050619 tid 2 2024-09-16T15:09:12.881 INFO:tasks.rados.rados.0.smithi162.stdout:2016: snap_remove snap 182 2024-09-16T15:09:13.857 INFO:tasks.rados.rados.0.smithi162.stdout:2016: done (3 left) 2024-09-16T15:09:13.857 INFO:tasks.rados.rados.0.smithi162.stdout:2017: rollback oid 2 current snap is 190 2024-09-16T15:09:13.857 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 189 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2018: write oid 45 current snap is 190 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: finishing write tid 1 to smithi16234619-12 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: finishing write tid 2 to smithi16234619-12 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: finishing write tid 3 to smithi16234619-12 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: finishing write tid 4 to smithi16234619-12 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: oid 12 updating version 0 to 352 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: oid 12 version 352 is already newer than 350 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: oid 12 version 352 is already newer than 351 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 352 (ObjNum 716 snap 190 seq_num 716) dirty exists 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2014: left oid 12 (ObjNum 716 snap 190 seq_num 716) 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2015: finishing write tid 1 to smithi16234619-4 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2015: finishing write tid 2 to smithi16234619-4 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2015: finishing write tid 3 to smithi16234619-4 2024-09-16T15:09:13.858 INFO:tasks.rados.rados.0.smithi162.stdout:2015: finishing write tid 4 to smithi16234619-4 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2015: oid 4 updating version 0 to 859 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2015: oid 4 version 859 is already newer than 858 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2015: oid 4 version 859 is already newer than 857 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 859 (ObjNum 717 snap 190 seq_num 717) dirty exists 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2015: left oid 4 (ObjNum 717 snap 190 seq_num 717) 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2013: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2013: finishing copy_from to smithi16234619-30 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 749 (ObjNum 701 snap 182 seq_num 701) dirty exists 2024-09-16T15:09:13.859 INFO:tasks.rados.rados.0.smithi162.stdout:2018: seq_num 718 ranges {467426=565964,1649130=418296} 2024-09-16T15:09:13.862 INFO:tasks.rados.rados.0.smithi162.stdout:2018: writing smithi16234619-45 from 467426 to 1033390 tid 1 2024-09-16T15:09:13.868 INFO:tasks.rados.rados.0.smithi162.stdout:2018: writing smithi16234619-45 from 1649130 to 2067426 tid 2 2024-09-16T15:09:13.869 INFO:tasks.rados.rados.0.smithi162.stdout:2013: done (4 left) 2024-09-16T15:09:13.869 INFO:tasks.rados.rados.0.smithi162.stdout:2014: done (3 left) 2024-09-16T15:09:13.869 INFO:tasks.rados.rados.0.smithi162.stdout:2015: done (2 left) 2024-09-16T15:09:13.869 INFO:tasks.rados.rados.0.smithi162.stdout:2019: delete oid 49 current snap is 190 2024-09-16T15:09:13.871 INFO:tasks.rados.rados.0.smithi162.stdout:2019: done (2 left) 2024-09-16T15:09:13.871 INFO:tasks.rados.rados.0.smithi162.stdout:2020: copy_from oid 26 from oid 22 current snap is 190 2024-09-16T15:09:13.871 INFO:tasks.rados.rados.0.smithi162.stdout:2021: write (excl) oid 6 current snap is 190 2024-09-16T15:09:13.871 INFO:tasks.rados.rados.0.smithi162.stdout:2021: seq_num 720 ranges {501040=716462,1766094=334946} 2024-09-16T15:09:13.877 INFO:tasks.rados.rados.0.smithi162.stdout:2021: writing smithi16234619-6 from 501040 to 1217502 tid 1 2024-09-16T15:09:13.883 INFO:tasks.rados.rados.0.smithi162.stdout:2021: writing smithi16234619-6 from 1766094 to 2101040 tid 2 2024-09-16T15:09:13.883 INFO:tasks.rados.rados.0.smithi162.stdout:2022: write (excl) oid 23 current snap is 190 2024-09-16T15:09:13.883 INFO:tasks.rados.rados.0.smithi162.stdout:2022: seq_num 721 ranges {517847=791711,2108223=9624} 2024-09-16T15:09:13.890 INFO:tasks.rados.rados.0.smithi162.stdout:2022: writing smithi16234619-23 from 517847 to 1309558 tid 1 2024-09-16T15:09:13.895 INFO:tasks.rados.rados.0.smithi162.stdout:2022: writing smithi16234619-23 from 2108223 to 2117847 tid 2 2024-09-16T15:09:13.895 INFO:tasks.rados.rados.0.smithi162.stdout:2023: write (excl) oid 35 current snap is 190 2024-09-16T15:09:13.895 INFO:tasks.rados.rados.0.smithi162.stdout:2017: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:09:13.895 INFO:tasks.rados.rados.0.smithi162.stdout:2017: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:09:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:2017: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:09:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 902 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:09:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:2018: finishing write tid 1 to smithi16234619-45 2024-09-16T15:09:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:2018: finishing write tid 2 to smithi16234619-45 2024-09-16T15:09:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:2023: seq_num 722 ranges {534654=466960,1766705=367949} 2024-09-16T15:09:13.901 INFO:tasks.rados.rados.0.smithi162.stdout:2023: writing smithi16234619-35 from 534654 to 1001614 tid 1 2024-09-16T15:09:13.908 INFO:tasks.rados.rados.0.smithi162.stdout:2023: writing smithi16234619-35 from 1766705 to 2134654 tid 2 2024-09-16T15:09:13.908 INFO:tasks.rados.rados.0.smithi162.stdout:2017: done (5 left) 2024-09-16T15:09:13.908 INFO:tasks.rados.rados.0.smithi162.stdout:2024: write oid 9 current snap is 190 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: finishing write tid 3 to smithi16234619-45 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: finishing write tid 4 to smithi16234619-45 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: oid 45 updating version 0 to 904 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: oid 45 updating version 904 to 905 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: oid 45 updating version 905 to 906 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 906 (ObjNum 718 snap 190 seq_num 718) dirty exists 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2018: left oid 45 (ObjNum 718 snap 190 seq_num 718) 2024-09-16T15:09:13.909 INFO:tasks.rados.rados.0.smithi162.stdout:2024: seq_num 723 ranges {551461=658562,1941540=209921} 2024-09-16T15:09:13.916 INFO:tasks.rados.rados.0.smithi162.stdout:2024: writing smithi16234619-9 from 551461 to 1210023 tid 1 2024-09-16T15:09:13.922 INFO:tasks.rados.rados.0.smithi162.stdout:2024: writing smithi16234619-9 from 1941540 to 2151461 tid 2 2024-09-16T15:09:13.922 INFO:tasks.rados.rados.0.smithi162.stdout:2018: done (5 left) 2024-09-16T15:09:13.922 INFO:tasks.rados.rados.0.smithi162.stdout:2025: snap_remove snap 188 2024-09-16T15:09:14.861 INFO:tasks.rados.rados.0.smithi162.stdout:2025: done (5 left) 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2026: rollback oid 5 current snap is 190 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2021: finishing write tid 1 to smithi16234619-6 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2021: finishing write tid 2 to smithi16234619-6 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: finishing write tid 1 to smithi16234619-23 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: finishing write tid 2 to smithi16234619-23 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: finishing write tid 3 to smithi16234619-23 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: finishing write tid 4 to smithi16234619-23 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: oid 23 updating version 0 to 585 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: oid 23 updating version 585 to 586 2024-09-16T15:09:14.862 INFO:tasks.rados.rados.0.smithi162.stdout:2022: oid 23 version 586 is already newer than 584 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 586 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2022: left oid 23 (ObjNum 721 snap 190 seq_num 721) 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2024: finishing write tid 1 to smithi16234619-9 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2024: finishing write tid 2 to smithi16234619-9 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2024: finishing write tid 3 to smithi16234619-9 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: finishing write tid 3 to smithi16234619-6 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: finishing write tid 4 to smithi16234619-6 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: oid 6 updating version 0 to 585 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: oid 6 version 585 is already newer than 584 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: oid 6 updating version 585 to 586 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 586 (ObjNum 720 snap 190 seq_num 720) dirty exists 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2021: left oid 6 (ObjNum 720 snap 190 seq_num 720) 2024-09-16T15:09:14.863 INFO:tasks.rados.rados.0.smithi162.stdout:2024: finishing write tid 4 to smithi16234619-9 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2024: oid 9 updating version 0 to 862 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2024: oid 9 version 862 is already newer than 860 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2024: oid 9 version 862 is already newer than 861 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 862 (ObjNum 723 snap 190 seq_num 723) dirty exists 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2024: left oid 9 (ObjNum 723 snap 190 seq_num 723) 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: finishing write tid 1 to smithi16234619-35 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: finishing write tid 2 to smithi16234619-35 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: finishing write tid 3 to smithi16234619-35 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: finishing write tid 4 to smithi16234619-35 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: oid 35 updating version 0 to 587 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: oid 35 updating version 587 to 589 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:2023: oid 35 version 589 is already newer than 588 2024-09-16T15:09:14.864 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 589 (ObjNum 722 snap 190 seq_num 722) dirty exists 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2023: left oid 35 (ObjNum 722 snap 190 seq_num 722) 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2020: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2020: finishing copy_from to smithi16234619-26 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 776 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 189 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2020: done (5 left) 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2021: done (4 left) 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2022: done (3 left) 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2023: done (2 left) 2024-09-16T15:09:14.865 INFO:tasks.rados.rados.0.smithi162.stdout:2024: done (1 left) 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2027: read oid 45 snap -1 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2027: expect (ObjNum 718 snap 190 seq_num 718) 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2028: read oid 42 snap -1 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2028: expect (ObjNum 704 snap 182 seq_num 704) 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2029: rollback oid 2 current snap is 190 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 185 2024-09-16T15:09:14.866 INFO:tasks.rados.rados.0.smithi162.stdout:2030: delete oid 38 current snap is 190 2024-09-16T15:09:14.868 INFO:tasks.rados.rados.0.smithi162.stdout:2030: done (4 left) 2024-09-16T15:09:14.869 INFO:tasks.rados.rados.0.smithi162.stdout:2031: copy_from oid 12 from oid 44 current snap is 190 2024-09-16T15:09:14.869 INFO:tasks.rados.rados.0.smithi162.stdout:2032: write (excl) oid 17 current snap is 190 2024-09-16T15:09:14.869 INFO:tasks.rados.rados.0.smithi162.stdout:2032: seq_num 725 ranges {585075=409060,1542151=642924} 2024-09-16T15:09:14.874 INFO:tasks.rados.rados.0.smithi162.stdout:2032: writing smithi16234619-17 from 585075 to 994135 tid 1 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2032: writing smithi16234619-17 from 1542151 to 2185075 tid 2 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2033: read oid 13 snap -1 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2033: expect (ObjNum 629 snap 158 seq_num 629) 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2034: copy_from oid 29 from oid 30 current snap is 190 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2026: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2026: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:09:14.881 INFO:tasks.rados.rados.0.smithi162.stdout:2028: expect (ObjNum 704 snap 182 seq_num 704) 2024-09-16T15:09:15.076 INFO:tasks.rados.rados.0.smithi162.stdout:2026: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:09:15.076 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 879 (ObjNum 689 snap 178 seq_num 689) dirty exists 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2029: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2029: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2026: done (7 left) 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2028: done (6 left) 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2035: snap_create 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2029: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 915 (ObjNum 674 snap 176 seq_num 674) dirty exists 2024-09-16T15:09:15.077 INFO:tasks.rados.rados.0.smithi162.stdout:2027: expect (ObjNum 718 snap 190 seq_num 718) 2024-09-16T15:09:15.158 INFO:tasks.rados.rados.0.smithi162.stdout:2033: expect (ObjNum 629 snap 158 seq_num 629) 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: finishing write tid 1 to smithi16234619-17 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2031: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: finishing write tid 2 to smithi16234619-17 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: finishing write tid 3 to smithi16234619-17 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: finishing write tid 4 to smithi16234619-17 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: oid 17 updating version 0 to 921 2024-09-16T15:09:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:2032: oid 17 version 921 is already newer than 919 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:2032: oid 17 version 921 is already newer than 920 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 921 (ObjNum 725 snap 190 seq_num 725) dirty exists 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:2032: left oid 17 (ObjNum 725 snap 190 seq_num 725) 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:2031: finishing copy_from to smithi16234619-12 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 561 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:2034: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:2034: finishing copy_from to smithi16234619-29 2024-09-16T15:09:15.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 749 (ObjNum 701 snap 182 seq_num 701) dirty exists 2024-09-16T15:09:15.861 INFO:tasks.rados.rados.0.smithi162.stdout:2027: done (6 left) 2024-09-16T15:09:15.861 INFO:tasks.rados.rados.0.smithi162.stdout:2029: done (5 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2031: done (4 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2032: done (3 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2033: done (2 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2034: done (1 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2035: done (0 left) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2036: read oid 28 snap -1 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2036: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2037: read oid 12 snap -1 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2037: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:09:15.862 INFO:tasks.rados.rados.0.smithi162.stdout:2038: delete oid 7 current snap is 191 2024-09-16T15:09:15.869 INFO:tasks.rados.rados.0.smithi162.stdout:2038: done (2 left) 2024-09-16T15:09:15.870 INFO:tasks.rados.rados.0.smithi162.stdout:2039: write oid 50 current snap is 191 2024-09-16T15:09:15.870 INFO:tasks.rados.rados.0.smithi162.stdout:2039: seq_num 727 ranges {618689=559558,1659115=559574} 2024-09-16T15:09:15.877 INFO:tasks.rados.rados.0.smithi162.stdout:2039: writing smithi16234619-50 from 618689 to 1178247 tid 1 2024-09-16T15:09:15.882 INFO:tasks.rados.rados.0.smithi162.stdout:2039: writing smithi16234619-50 from 1659115 to 2218689 tid 2 2024-09-16T15:09:15.883 INFO:tasks.rados.rados.0.smithi162.stdout:2040: copy_from oid 36 from oid 14 current snap is 191 2024-09-16T15:09:15.883 INFO:tasks.rados.rados.0.smithi162.stdout:2037: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:09:16.029 INFO:tasks.rados.rados.0.smithi162.stdout:2036: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:09:16.129 INFO:tasks.rados.rados.0.smithi162.stdout:2039: finishing write tid 1 to smithi16234619-50 2024-09-16T15:09:16.129 INFO:tasks.rados.rados.0.smithi162.stdout:2039: finishing write tid 2 to smithi16234619-50 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2040: finishing copy_from to smithi16234619-36 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2040: got expected ENOENT (src dne) 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2040: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: finishing write tid 3 to smithi16234619-50 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: finishing write tid 4 to smithi16234619-50 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: oid 50 updating version 0 to 770 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: oid 50 updating version 770 to 772 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: oid 50 version 772 is already newer than 771 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 772 (ObjNum 727 snap 191 seq_num 727) dirty exists 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2039: left oid 50 (ObjNum 727 snap 191 seq_num 727) 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2036: done (3 left) 2024-09-16T15:09:16.130 INFO:tasks.rados.rados.0.smithi162.stdout:2037: done (2 left) 2024-09-16T15:09:16.131 INFO:tasks.rados.rados.0.smithi162.stdout:2039: done (1 left) 2024-09-16T15:09:16.131 INFO:tasks.rados.rados.0.smithi162.stdout:2040: done (0 left) 2024-09-16T15:09:16.131 INFO:tasks.rados.rados.0.smithi162.stdout:2041: write oid 13 current snap is 191 2024-09-16T15:09:16.131 INFO:tasks.rados.rados.0.smithi162.stdout:2041: seq_num 728 ranges {635496=634807,1717597=517899} 2024-09-16T15:09:16.136 INFO:tasks.rados.rados.0.smithi162.stdout:2041: writing smithi16234619-13 from 635496 to 1270303 tid 1 2024-09-16T15:09:16.141 INFO:tasks.rados.rados.0.smithi162.stdout:2041: writing smithi16234619-13 from 1717597 to 2235496 tid 2 2024-09-16T15:09:16.141 INFO:tasks.rados.rados.0.smithi162.stdout:2042: snap_create 2024-09-16T15:09:16.163 INFO:tasks.rados.rados.0.smithi162.stdout:2041: finishing write tid 1 to smithi16234619-13 2024-09-16T15:09:16.163 INFO:tasks.rados.rados.0.smithi162.stdout:2041: finishing write tid 2 to smithi16234619-13 2024-09-16T15:09:16.163 INFO:tasks.rados.rados.0.smithi162.stdout:2041: finishing write tid 3 to smithi16234619-13 2024-09-16T15:09:16.163 INFO:tasks.rados.rados.0.smithi162.stdout:2041: finishing write tid 4 to smithi16234619-13 2024-09-16T15:09:16.163 INFO:tasks.rados.rados.0.smithi162.stdout:2041: oid 13 updating version 0 to 885 2024-09-16T15:09:16.164 INFO:tasks.rados.rados.0.smithi162.stdout:2041: oid 13 version 885 is already newer than 884 2024-09-16T15:09:16.164 INFO:tasks.rados.rados.0.smithi162.stdout:2041: oid 13 version 885 is already newer than 883 2024-09-16T15:09:16.164 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 885 (ObjNum 728 snap 191 seq_num 728) dirty exists 2024-09-16T15:09:16.164 INFO:tasks.rados.rados.0.smithi162.stdout:2041: left oid 13 (ObjNum 728 snap 191 seq_num 728) 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:2041: done (1 left) 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:2042: done (0 left) 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:2043: rollback oid 17 current snap is 192 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 191 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:2044: write oid 9 current snap is 192 2024-09-16T15:09:16.878 INFO:tasks.rados.rados.0.smithi162.stdout:2044: seq_num 729 ranges {652303=710056,2059726=192577} 2024-09-16T15:09:16.885 INFO:tasks.rados.rados.0.smithi162.stdout:2044: writing smithi16234619-9 from 652303 to 1362359 tid 1 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2044: writing smithi16234619-9 from 2059726 to 2252303 tid 2 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2045: copy_from oid 45 from oid 26 current snap is 192 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2043: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2043: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2043: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 925 (ObjNum 725 snap 190 seq_num 725) dirty exists 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2043: done (2 left) 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2046: read oid 47 snap -1 2024-09-16T15:09:16.891 INFO:tasks.rados.rados.0.smithi162.stdout:2046: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:16.892 INFO:tasks.rados.rados.0.smithi162.stdout:2047: snap_create 2024-09-16T15:09:16.894 INFO:tasks.rados.rados.0.smithi162.stdout:2046: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: finishing write tid 1 to smithi16234619-9 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: finishing write tid 2 to smithi16234619-9 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: finishing write tid 3 to smithi16234619-9 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: finishing write tid 4 to smithi16234619-9 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: oid 9 updating version 0 to 863 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: oid 9 updating version 863 to 864 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:2044: oid 9 updating version 864 to 865 2024-09-16T15:09:17.092 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 865 (ObjNum 729 snap 192 seq_num 729) dirty exists 2024-09-16T15:09:17.093 INFO:tasks.rados.rados.0.smithi162.stdout:2044: left oid 9 (ObjNum 729 snap 192 seq_num 729) 2024-09-16T15:09:17.093 INFO:tasks.rados.rados.0.smithi162.stdout:2045: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:09:17.093 INFO:tasks.rados.rados.0.smithi162.stdout:2045: finishing copy_from to smithi16234619-45 2024-09-16T15:09:17.093 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 776 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2044: done (3 left) 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2045: done (2 left) 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2046: done (1 left) 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2047: done (0 left) 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2048: write (excl) oid 40 current snap is 193 2024-09-16T15:09:17.857 INFO:tasks.rados.rados.0.smithi162.stdout:2048: seq_num 730 ranges {669110=501658,1834561=434549} 2024-09-16T15:09:17.863 INFO:tasks.rados.rados.0.smithi162.stdout:2048: writing smithi16234619-40 from 669110 to 1170768 tid 1 2024-09-16T15:09:17.870 INFO:tasks.rados.rados.0.smithi162.stdout:2048: writing smithi16234619-40 from 1834561 to 2269110 tid 2 2024-09-16T15:09:17.870 INFO:tasks.rados.rados.0.smithi162.stdout:2049: rollback oid 7 current snap is 193 2024-09-16T15:09:17.870 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 190 2024-09-16T15:09:17.871 INFO:tasks.rados.rados.0.smithi162.stdout:2050: snap_remove snap 191 2024-09-16T15:09:18.868 INFO:tasks.rados.rados.0.smithi162.stdout:2050: done (2 left) 2024-09-16T15:09:18.868 INFO:tasks.rados.rados.0.smithi162.stdout:2051: write (excl) oid 5 current snap is 193 2024-09-16T15:09:18.868 INFO:tasks.rados.rados.0.smithi162.stdout:2051: seq_num 731 ranges {685917=576907,1893043=392874} 2024-09-16T15:09:18.875 INFO:tasks.rados.rados.0.smithi162.stdout:2051: writing smithi16234619-5 from 685917 to 1262824 tid 1 2024-09-16T15:09:18.881 INFO:tasks.rados.rados.0.smithi162.stdout:2051: writing smithi16234619-5 from 1893043 to 2285917 tid 2 2024-09-16T15:09:18.881 INFO:tasks.rados.rados.0.smithi162.stdout:2052: snap_create 2024-09-16T15:09:18.881 INFO:tasks.rados.rados.0.smithi162.stdout:2049: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:09:18.881 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 0 (ObjNum 193 snap 0 seq_num 864486064) dirty dne 2024-09-16T15:09:18.881 INFO:tasks.rados.rados.0.smithi162.stdout:2048: finishing write tid 1 to smithi16234619-40 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: finishing write tid 2 to smithi16234619-40 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: finishing write tid 3 to smithi16234619-40 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: finishing write tid 4 to smithi16234619-40 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: oid 40 updating version 0 to 564 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: oid 40 version 564 is already newer than 563 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: oid 40 version 564 is already newer than 562 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 564 (ObjNum 730 snap 193 seq_num 730) dirty exists 2024-09-16T15:09:18.882 INFO:tasks.rados.rados.0.smithi162.stdout:2048: left oid 40 (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:09:18.901 INFO:tasks.rados.rados.0.smithi162.stdout:2051: finishing write tid 1 to smithi16234619-5 2024-09-16T15:09:18.901 INFO:tasks.rados.rados.0.smithi162.stdout:2051: finishing write tid 2 to smithi16234619-5 2024-09-16T15:09:18.903 INFO:tasks.rados.rados.0.smithi162.stdout:2051: finishing write tid 3 to smithi16234619-5 2024-09-16T15:09:18.903 INFO:tasks.rados.rados.0.smithi162.stdout:2051: finishing write tid 4 to smithi16234619-5 2024-09-16T15:09:18.903 INFO:tasks.rados.rados.0.smithi162.stdout:2051: oid 5 updating version 0 to 889 2024-09-16T15:09:18.904 INFO:tasks.rados.rados.0.smithi162.stdout:2051: oid 5 version 889 is already newer than 888 2024-09-16T15:09:18.904 INFO:tasks.rados.rados.0.smithi162.stdout:2051: oid 5 version 889 is already newer than 887 2024-09-16T15:09:18.904 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 889 (ObjNum 731 snap 193 seq_num 731) dirty exists 2024-09-16T15:09:18.904 INFO:tasks.rados.rados.0.smithi162.stdout:2051: left oid 5 (ObjNum 731 snap 193 seq_num 731) 2024-09-16T15:09:19.885 INFO:tasks.rados.rados.0.smithi162.stdout:2048: done (3 left) 2024-09-16T15:09:19.885 INFO:tasks.rados.rados.0.smithi162.stdout:2049: done (2 left) 2024-09-16T15:09:19.885 INFO:tasks.rados.rados.0.smithi162.stdout:2051: done (1 left) 2024-09-16T15:09:19.885 INFO:tasks.rados.rados.0.smithi162.stdout:2052: done (0 left) 2024-09-16T15:09:19.885 INFO:tasks.rados.rados.0.smithi162.stdout:2053: snap_remove snap 192 2024-09-16T15:09:20.901 INFO:tasks.rados.rados.0.smithi162.stdout:2053: done (0 left) 2024-09-16T15:09:20.901 INFO:tasks.rados.rados.0.smithi162.stdout:2054: read oid 31 snap -1 2024-09-16T15:09:20.901 INFO:tasks.rados.rados.0.smithi162.stdout:2054: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2055: read oid 22 snap 193 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2055: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2056: copy_from oid 43 from oid 13 current snap is 194 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2057: copy_from oid 46 from oid 23 current snap is 194 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2058: write oid 34 current snap is 194 2024-09-16T15:09:20.902 INFO:tasks.rados.rados.0.smithi162.stdout:2058: seq_num 732 ranges {702724=652156,1951525=351199} 2024-09-16T15:09:20.909 INFO:tasks.rados.rados.0.smithi162.stdout:2058: writing smithi16234619-34 from 702724 to 1354880 tid 1 2024-09-16T15:09:20.915 INFO:tasks.rados.rados.0.smithi162.stdout:2058: writing smithi16234619-34 from 1951525 to 2302724 tid 2 2024-09-16T15:09:20.915 INFO:tasks.rados.rados.0.smithi162.stdout:2059: delete oid 6 current snap is 194 2024-09-16T15:09:20.915 INFO:tasks.rados.rados.0.smithi162.stdout:2055: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:09:21.178 INFO:tasks.rados.rados.0.smithi162.stdout:2054: expect (ObjNum 626 snap 157 seq_num 626) 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: finishing write tid 1 to smithi16234619-34 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: finishing write tid 2 to smithi16234619-34 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: finishing write tid 3 to smithi16234619-34 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: finishing write tid 4 to smithi16234619-34 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: oid 34 updating version 0 to 776 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: oid 34 version 776 is already newer than 775 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:2058: oid 34 version 776 is already newer than 774 2024-09-16T15:09:21.258 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 776 (ObjNum 732 snap 194 seq_num 732) dirty exists 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2058: left oid 34 (ObjNum 732 snap 194 seq_num 732) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2057: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2054: done (5 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2055: done (4 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2058: done (3 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2059: done (2 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2060: copy_from oid 30 from oid 33 current snap is 194 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2057: finishing copy_from to smithi16234619-46 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 586 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2056: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2056: finishing copy_from to smithi16234619-43 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 885 (ObjNum 728 snap 191 seq_num 728) dirty exists 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2056: done (2 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2057: done (1 left) 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2061: read oid 13 snap -1 2024-09-16T15:09:21.259 INFO:tasks.rados.rados.0.smithi162.stdout:2061: expect (ObjNum 728 snap 191 seq_num 728) 2024-09-16T15:09:21.260 INFO:tasks.rados.rados.0.smithi162.stdout:2062: delete oid 39 current snap is 194 2024-09-16T15:09:21.261 INFO:tasks.rados.rados.0.smithi162.stdout:2061: expect (ObjNum 728 snap 191 seq_num 728) 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2060: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2060: finishing copy_from to smithi16234619-30 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 784 (ObjNum 700 snap 181 seq_num 700) dirty exists 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2060: done (2 left) 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2061: done (1 left) 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2062: done (0 left) 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2063: rollback oid 14 current snap is 194 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 186 2024-09-16T15:09:21.365 INFO:tasks.rados.rados.0.smithi162.stdout:2064: delete oid 38 current snap is 194 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:2063: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 0 (ObjNum 865782304 snap 21987 seq_num 304) dirty dne 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:2063: done (1 left) 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:2064: done (0 left) 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:2065: write (excl) oid 25 current snap is 194 2024-09-16T15:09:21.366 INFO:tasks.rados.rados.0.smithi162.stdout:2065: seq_num 736 ranges {769952=553152,2069100=300852} 2024-09-16T15:09:21.373 INFO:tasks.rados.rados.0.smithi162.stdout:2065: writing smithi16234619-25 from 769952 to 1323104 tid 1 2024-09-16T15:09:21.379 INFO:tasks.rados.rados.0.smithi162.stdout:2065: writing smithi16234619-25 from 2069100 to 2369952 tid 2 2024-09-16T15:09:21.379 INFO:tasks.rados.rados.0.smithi162.stdout:2066: read oid 18 snap -1 2024-09-16T15:09:21.379 INFO:tasks.rados.rados.0.smithi162.stdout:2066: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:09:21.379 INFO:tasks.rados.rados.0.smithi162.stdout:2067: write (excl) oid 2 current snap is 194 2024-09-16T15:09:21.379 INFO:tasks.rados.rados.0.smithi162.stdout:2067: seq_num 737 ranges {786759=628401,2011229=375530} 2024-09-16T15:09:21.387 INFO:tasks.rados.rados.0.smithi162.stdout:2067: writing smithi16234619-2 from 786759 to 1415160 tid 1 2024-09-16T15:09:21.392 INFO:tasks.rados.rados.0.smithi162.stdout:2067: writing smithi16234619-2 from 2011229 to 2386759 tid 2 2024-09-16T15:09:21.392 INFO:tasks.rados.rados.0.smithi162.stdout:2068: copy_from oid 30 from oid 44 current snap is 194 2024-09-16T15:09:21.392 INFO:tasks.rados.rados.0.smithi162.stdout:2066: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:09:21.561 INFO:tasks.rados.rados.0.smithi162.stdout:2065: finishing write tid 1 to smithi16234619-25 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: finishing write tid 2 to smithi16234619-25 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: finishing write tid 3 to smithi16234619-25 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: finishing write tid 4 to smithi16234619-25 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: oid 25 updating version 0 to 779 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: oid 25 version 779 is already newer than 778 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: oid 25 updating version 779 to 780 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 780 (ObjNum 736 snap 194 seq_num 736) dirty exists 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: left oid 25 (ObjNum 736 snap 194 seq_num 736) 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2067: finishing write tid 1 to smithi16234619-2 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2065: done (3 left) 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2066: done (2 left) 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2069: delete oid 3 current snap is 194 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2067: finishing write tid 2 to smithi16234619-2 2024-09-16T15:09:21.562 INFO:tasks.rados.rados.0.smithi162.stdout:2067: finishing write tid 3 to smithi16234619-2 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2067: finishing write tid 4 to smithi16234619-2 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2067: oid 2 updating version 0 to 934 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2067: oid 2 version 934 is already newer than 933 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2067: oid 2 updating version 934 to 935 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 935 (ObjNum 737 snap 194 seq_num 737) dirty exists 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2067: left oid 2 (ObjNum 737 snap 194 seq_num 737) 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2068: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:2068: finishing copy_from to smithi16234619-30 2024-09-16T15:09:21.563 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 561 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2067: done (2 left) 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2068: done (1 left) 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2069: done (0 left) 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2070: read oid 20 snap -1 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2070: expect (ObjNum 710 snap 184 seq_num 710) 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2071: write oid 31 current snap is 194 2024-09-16T15:09:21.564 INFO:tasks.rados.rados.0.smithi162.stdout:2071: seq_num 739 ranges {420373=495252,1444546=690576} 2024-09-16T15:09:21.567 INFO:tasks.rados.rados.0.smithi162.stdout:2071: writing smithi16234619-31 from 420373 to 915625 tid 1 2024-09-16T15:09:21.574 INFO:tasks.rados.rados.0.smithi162.stdout:2071: writing smithi16234619-31 from 1444546 to 2135122 tid 2 2024-09-16T15:09:21.574 INFO:tasks.rados.rados.0.smithi162.stdout:2072: write oid 48 current snap is 194 2024-09-16T15:09:21.574 INFO:tasks.rados.rados.0.smithi162.stdout:2072: seq_num 740 ranges {437180=570501,1503028=550587} 2024-09-16T15:09:21.580 INFO:tasks.rados.rados.0.smithi162.stdout:2072: writing smithi16234619-48 from 437180 to 1007681 tid 1 2024-09-16T15:09:21.585 INFO:tasks.rados.rados.0.smithi162.stdout:2072: writing smithi16234619-48 from 1503028 to 2053615 tid 2 2024-09-16T15:09:21.585 INFO:tasks.rados.rados.0.smithi162.stdout:2073: snap_remove snap 185 2024-09-16T15:09:21.903 INFO:tasks.rados.rados.0.smithi162.stdout:2070: expect (ObjNum 710 snap 184 seq_num 710) 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: finishing write tid 1 to smithi16234619-31 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: finishing write tid 2 to smithi16234619-31 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: finishing write tid 3 to smithi16234619-31 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: finishing write tid 4 to smithi16234619-31 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: oid 31 updating version 0 to 844 2024-09-16T15:09:22.033 INFO:tasks.rados.rados.0.smithi162.stdout:2071: oid 31 version 844 is already newer than 842 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2071: oid 31 version 844 is already newer than 843 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 844 (ObjNum 739 snap 194 seq_num 739) dirty exists 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2071: left oid 31 (ObjNum 739 snap 194 seq_num 739) 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: finishing write tid 1 to smithi16234619-48 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: finishing write tid 2 to smithi16234619-48 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: finishing write tid 3 to smithi16234619-48 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: finishing write tid 4 to smithi16234619-48 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: oid 48 updating version 0 to 868 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: oid 48 version 868 is already newer than 866 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: oid 48 version 868 is already newer than 867 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 868 (ObjNum 740 snap 194 seq_num 740) dirty exists 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: left oid 48 (ObjNum 740 snap 194 seq_num 740) 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2070: done (3 left) 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2071: done (2 left) 2024-09-16T15:09:22.034 INFO:tasks.rados.rados.0.smithi162.stdout:2072: done (1 left) 2024-09-16T15:09:22.035 INFO:tasks.rados.rados.0.smithi162.stdout:2073: done (0 left) 2024-09-16T15:09:22.035 INFO:tasks.rados.rados.0.smithi162.stdout:2074: write (excl) oid 36 current snap is 194 2024-09-16T15:09:22.035 INFO:tasks.rados.rados.0.smithi162.stdout:2074: seq_num 741 ranges {453987=645750,1845157=608830} 2024-09-16T15:09:22.038 INFO:tasks.rados.rados.0.smithi162.stdout:2074: writing smithi16234619-36 from 453987 to 1099737 tid 1 2024-09-16T15:09:22.045 INFO:tasks.rados.rados.0.smithi162.stdout:2074: writing smithi16234619-36 from 1845157 to 2453987 tid 2 2024-09-16T15:09:22.046 INFO:tasks.rados.rados.0.smithi162.stdout:2075: copy_from oid 17 from oid 23 current snap is 194 2024-09-16T15:09:22.046 INFO:tasks.rados.rados.0.smithi162.stdout:2076: snap_remove snap 186 2024-09-16T15:09:22.861 INFO:tasks.rados.rados.0.smithi162.stdout:2076: done (2 left) 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2077: write oid 43 current snap is 194 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2075: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: finishing write tid 1 to smithi16234619-36 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: finishing write tid 2 to smithi16234619-36 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: finishing write tid 3 to smithi16234619-36 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: finishing write tid 4 to smithi16234619-36 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: oid 36 updating version 0 to 790 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: oid 36 version 790 is already newer than 788 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: oid 36 version 790 is already newer than 789 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 790 (ObjNum 741 snap 194 seq_num 741) dirty exists 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2074: left oid 36 (ObjNum 741 snap 194 seq_num 741) 2024-09-16T15:09:22.862 INFO:tasks.rados.rados.0.smithi162.stdout:2075: finishing copy_from to smithi16234619-17 2024-09-16T15:09:22.863 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 586 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:09:22.863 INFO:tasks.rados.rados.0.smithi162.stdout:2077: seq_num 742 ranges {470794=720999,1903639=554256} 2024-09-16T15:09:22.868 INFO:tasks.rados.rados.0.smithi162.stdout:2077: writing smithi16234619-43 from 470794 to 1191793 tid 1 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2077: writing smithi16234619-43 from 1903639 to 2457895 tid 2 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2074: done (2 left) 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2075: done (1 left) 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2078: copy_from oid 23 from oid 6 current snap is 194 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2079: copy_from oid 37 from oid 17 current snap is 194 2024-09-16T15:09:22.876 INFO:tasks.rados.rados.0.smithi162.stdout:2080: delete oid 3 current snap is 194 2024-09-16T15:09:22.877 INFO:tasks.rados.rados.0.smithi162.stdout:2078: finishing copy_from to smithi16234619-23 2024-09-16T15:09:22.877 INFO:tasks.rados.rados.0.smithi162.stdout:2078: got expected ENOENT (src dne) 2024-09-16T15:09:22.877 INFO:tasks.rados.rados.0.smithi162.stdout:2078: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:09:22.879 INFO:tasks.rados.rados.0.smithi162.stdout:2078: done (3 left) 2024-09-16T15:09:22.879 INFO:tasks.rados.rados.0.smithi162.stdout:2080: done (2 left) 2024-09-16T15:09:22.879 INFO:tasks.rados.rados.0.smithi162.stdout:2081: write oid 9 current snap is 194 2024-09-16T15:09:22.879 INFO:tasks.rados.rados.0.smithi162.stdout:2081: seq_num 744 ranges {504408=471497,1620603=557925} 2024-09-16T15:09:22.885 INFO:tasks.rados.rados.0.smithi162.stdout:2081: writing smithi16234619-9 from 504408 to 975905 tid 1 2024-09-16T15:09:22.891 INFO:tasks.rados.rados.0.smithi162.stdout:2081: writing smithi16234619-9 from 1620603 to 2178528 tid 2 2024-09-16T15:09:22.892 INFO:tasks.rados.rados.0.smithi162.stdout:2082: snap_create 2024-09-16T15:09:22.913 INFO:tasks.rados.rados.0.smithi162.stdout:2079: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:09:22.914 INFO:tasks.rados.rados.0.smithi162.stdout:2081: finishing write tid 1 to smithi16234619-9 2024-09-16T15:09:22.914 INFO:tasks.rados.rados.0.smithi162.stdout:2077: finishing write tid 1 to smithi16234619-43 2024-09-16T15:09:22.914 INFO:tasks.rados.rados.0.smithi162.stdout:2077: finishing write tid 2 to smithi16234619-43 2024-09-16T15:09:22.916 INFO:tasks.rados.rados.0.smithi162.stdout:2079: finishing copy_from to smithi16234619-37 2024-09-16T15:09:22.916 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 586 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2077: finishing write tid 3 to smithi16234619-43 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: finishing write tid 2 to smithi16234619-9 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: finishing write tid 3 to smithi16234619-9 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: finishing write tid 4 to smithi16234619-9 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: oid 9 updating version 0 to 870 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: oid 9 version 870 is already newer than 869 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: oid 9 updating version 870 to 871 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 871 (ObjNum 744 snap 194 seq_num 744) dirty exists 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2081: left oid 9 (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2077: finishing write tid 4 to smithi16234619-43 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2077: oid 43 updating version 0 to 887 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2077: oid 43 version 887 is already newer than 886 2024-09-16T15:09:22.917 INFO:tasks.rados.rados.0.smithi162.stdout:2077: oid 43 updating version 887 to 888 2024-09-16T15:09:22.918 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 888 (ObjNum 742 snap 194 seq_num 742) dirty exists 2024-09-16T15:09:22.918 INFO:tasks.rados.rados.0.smithi162.stdout:2077: left oid 43 (ObjNum 742 snap 194 seq_num 742) 2024-09-16T15:09:23.857 INFO:tasks.rados.rados.0.smithi162.stdout:2077: done (3 left) 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2079: done (2 left) 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2081: done (1 left) 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2082: done (0 left) 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2083: copy_from oid 24 from oid 42 current snap is 195 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2084: write oid 50 current snap is 195 2024-09-16T15:09:23.858 INFO:tasks.rados.rados.0.smithi162.stdout:2084: seq_num 745 ranges {521215=546746,1562732=701583} 2024-09-16T15:09:23.863 INFO:tasks.rados.rados.0.smithi162.stdout:2084: writing smithi16234619-50 from 521215 to 1067961 tid 1 2024-09-16T15:09:23.870 INFO:tasks.rados.rados.0.smithi162.stdout:2084: writing smithi16234619-50 from 1562732 to 2264315 tid 2 2024-09-16T15:09:23.870 INFO:tasks.rados.rados.0.smithi162.stdout:2085: copy_from oid 48 from oid 34 current snap is 195 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2086: read oid 49 snap -1 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2086: expect deleted 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2087: read oid 5 snap -1 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2087: expect (ObjNum 731 snap 193 seq_num 731) 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2088: write (excl) oid 44 current snap is 195 2024-09-16T15:09:23.871 INFO:tasks.rados.rados.0.smithi162.stdout:2088: seq_num 746 ranges {538022=738348,1737567=561594} 2024-09-16T15:09:23.877 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:09:23.877 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-16T15:09:23.877 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:09:23.878 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:09:23.878 INFO:tasks.rados.rados.0.smithi162.stdout:2088: writing smithi16234619-44 from 538022 to 1276370 tid 1 2024-09-16T15:09:23.881 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:09:23.881 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:09:23.881 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-16T15:09:23.881 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-16T15:09:23.881 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-16T15:09:23.883 INFO:tasks.ceph.mon.e:Started 2024-09-16T15:09:23.883 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:09:23.883 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2088: writing smithi16234619-44 from 1737567 to 2299161 tid 2 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2089: read oid 11 snap -1 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2089: expect (ObjNum 695 snap 180 seq_num 695) 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2090: read oid 30 snap -1 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2090: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2091: write oid 20 current snap is 195 2024-09-16T15:09:23.885 INFO:tasks.rados.rados.0.smithi162.stdout:2087: expect (ObjNum 731 snap 193 seq_num 731) 2024-09-16T15:09:23.996 INFO:tasks.rados.rados.0.smithi162.stdout:2090: expect (ObjNum 623 snap 157 seq_num 623) 2024-09-16T15:09:23.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.996+0000 7f6423fed640 10 monclient: get_monmap_and_config 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.996+0000 7f6423fed640 10 monclient: build_initial_monmap 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.996+0000 7f6423fed640 10 monclient: monmap: 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:23.997603+0000 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:23.997603+0000 2024-09-16T15:09:23.998 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:23.999 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 1 Processor -- start 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 1 -- start start 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient: init 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient: _add_conns ranks=[5,2,3,9,4,8,1,6,7,10,0] 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-h con 0x7f641c136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 0x7f641c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-b con 0x7f641c09bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 0x7f641c1370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-g con 0x7f641c136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.997+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c09b600 con 0x7f641c09bfe0 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0a5740 con 0x7f641c136c60 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0a52c0 con 0x7f641c136030 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6423fed640 10 monclient(hunting): authenticate will time out at 1300.587061s 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 0x7f641c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 0x7f641c1370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:38486/0 (socket says 172.21.15.134:38486) 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 0x7f641c09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:41276/0 (socket says 172.21.15.134:41276) 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 1 -- 172.21.15.134:0/3673114987 learned_addr learned my addr 172.21.15.134:0/3673114987 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): get_auth_request con 0x7f641c136c60 auth_method 0 2024-09-16T15:09:24.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): get_auth_request con 0x7f641c136030 auth_method 0 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): get_auth_request con 0x7f641c09bfe0 auth_method 0 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:24.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421588640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient(hunting): handle_auth_done global_id 54264 payload 274 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 -- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 msgr2=0x7f641c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 -- 172.21.15.134:0/3673114987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 msgr2=0x7f641c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 0x7f641c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient: _finish_hunting 0 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 monclient: found mon.noname-g 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:24.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f642258a640 1 -- 172.21.15.134:0/3673114987 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c073250 con 0x7f641c136c60 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:23.998+0000 7f6421d89640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f642258a640 1 -- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 msgr2=0x7f641c1370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f642258a640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 0x7f641c1370c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6414009dc0 tx=0x7f641402a480 comp rx=0 tx=0).ready entity=mon.3 client_cookie=7b0d94ee05249d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 1 -- 172.21.15.134:0/3673114987 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6414007bc0 con 0x7f641c136c60 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: dump: 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.007 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: _finish_auth 0 2024-09-16T15:09:24.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: _check_auth_tickets 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 1 -- 172.21.15.134:0/3673114987 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641402d050 con 0x7f641c136c60 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6423fed640 5 monclient: authenticate success, global_id 54264 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6423fed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 1 -- 172.21.15.134:0/3673114987 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6414007d40 con 0x7f641c136c60 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6420d87640 10 monclient: dump: 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:24.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6423fed640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:24.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.001+0000 7f6423fed640 4 set_mon_vals no callback set 2024-09-16T15:09:24.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 10 monclient: shutdown 2024-09-16T15:09:24.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:24.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 -- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 msgr2=0x7f641c1370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 0x7f641c1370c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6414009dc0 tx=0x7f641402a480 comp rx=0 tx=0).stop 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 -- 172.21.15.134:0/3673114987 shutdown_connections 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f641c136c60 0x7f641c1370c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f641c09bfe0 0x7f641c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 --2- 172.21.15.134:0/3673114987 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 -- 172.21.15.134:0/3673114987 >> 172.21.15.134:0/3673114987 conn(0x7f641c0a9ea0 msgr2=0x7f641c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 -- 172.21.15.134:0/3673114987 shutdown_connections 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 1 -- 172.21.15.134:0/3673114987 wait complete. 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 10 monclient: build_initial_monmap 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.002+0000 7f6423fed640 10 monclient: monmap: 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:24.017 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:24.018 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 Processor -- start 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 -- start start 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient: init 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient: _add_conns ranks=[6,5,7,0,2,8,3,1,4] 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-d con 0x7f641c09bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-h con 0x7f641c136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f641c136c60 0x7f641c135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient(hunting): picked mon.noname-i con 0x7f641c136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:24.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f641c120b30 con 0x7f641c136030 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.003+0000 7f6423fed640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0a7ca0 con 0x7f641c09bfe0 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.004+0000 7f6423fed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.004+0000 7f6423fed640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f641c121e10 con 0x7f641c136c60 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.004+0000 7f6423fed640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.004+0000 7f6423fed640 10 monclient(hunting): authenticate will time out at 1300.593014s 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:54574/0 (socket says 172.21.15.134:54574) 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 -- 172.21.15.134:0/2505464256 learned_addr learned my addr 172.21.15.134:0/2505464256 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): get_auth_request con 0x7f641c09bfe0 auth_method 0 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:24.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f641c136c60 0x7f641c135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): get_auth_request con 0x7f641c136c60 auth_method 0 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f642258a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient(hunting): handle_auth_done global_id 34737 payload 824 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421588640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 -- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f641c136c60 msgr2=0x7f641c135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:24.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f641c136c60 0x7f641c135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 -- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 msgr2=0x7f641c134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c134cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient: _finish_hunting 0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 monclient: found mon.noname-d 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421d89640 1 -- 172.21.15.134:0/2505464256 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64180094d0 con 0x7f641c09bfe0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421588640 10 monclient: get_auth_request con 0x7f641c136030 auth_method 0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.005+0000 7f6421588640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6421d89640 1 -- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 msgr2=0x7f641c11fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6421d89640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f641800e9d0 tx=0x7f641800ee90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f641800cc70 con 0x7f641c09bfe0 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:24.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: dump: 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:24.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: _finish_auth 0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: _check_auth_tickets 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 5 monclient: authenticate success, global_id 34737 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641800cdf0 con 0x7f641c09bfe0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 10 monclient: _renew_subs 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6414009a70 con 0x7f641c09bfe0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 10 monclient: _renew_subs 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6413fff640 4 set_mon_vals no callback set 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c1223d0 con 0x7f641c09bfe0 2024-09-16T15:09:24.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6418013d50 con 0x7f641c09bfe0 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.008+0000 7f64127fc640 10 monclient: dump: 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:24.025 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6421d89640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f641800e9d0 tx=0x7f641800ee90 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f641804a520 osd_map(414..414 src has 1..414) v4 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6418013470 con 0x7f641c09bfe0 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6423fed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6423fed640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6423fed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c09c400 con 0x7f641c09bfe0 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f64127fc640 1 --2- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 0x7f640003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6421588640 1 --2- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 0x7f640003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:24.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.009+0000 7f6421588640 10 monclient: get_auth_request con 0x7f6400038880 auth_method 0 2024-09-16T15:09:24.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.012+0000 7f6421588640 1 --2- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 0x7f640003ad40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f641c0a2720 tx=0x7f640c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:24.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.014+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(414..414 src has 1..414) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f641804a520 con 0x7f641c09bfe0 2024-09-16T15:09:24.141 INFO:tasks.rados.rados.0.smithi162.stdout:2089: expect (ObjNum 695 snap 180 seq_num 695) 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: finishing write tid 1 to smithi16234619-44 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: finishing write tid 2 to smithi16234619-44 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: finishing write tid 3 to smithi16234619-44 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: finishing write tid 4 to smithi16234619-44 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: oid 44 updating version 0 to 610 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: oid 44 updating version 610 to 612 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: oid 44 version 612 is already newer than 611 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 612 (ObjNum 746 snap 195 seq_num 746) dirty exists 2024-09-16T15:09:24.344 INFO:tasks.rados.rados.0.smithi162.stdout:2088: left oid 44 (ObjNum 746 snap 195 seq_num 746) 2024-09-16T15:09:24.345 INFO:tasks.rados.rados.0.smithi162.stdout:2084: finishing write tid 1 to smithi16234619-50 2024-09-16T15:09:24.345 INFO:tasks.rados.rados.0.smithi162.stdout:2091: seq_num 747 ranges {554829=413597,1396049=705252} 2024-09-16T15:09:24.349 INFO:tasks.rados.rados.0.smithi162.stdout:2091: writing smithi16234619-20 from 554829 to 968426 tid 1 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2091: writing smithi16234619-20 from 1396049 to 2101301 tid 2 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2086: done (8 left) 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2087: done (7 left) 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2088: done (6 left) 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2089: done (5 left) 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2090: done (4 left) 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2092: write oid 11 current snap is 195 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2083: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:09:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:2083: finishing copy_from to smithi16234619-24 2024-09-16T15:09:24.357 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 853 (ObjNum 704 snap 182 seq_num 704) dirty exists 2024-09-16T15:09:24.357 INFO:tasks.rados.rados.0.smithi162.stdout:2085: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:09:24.357 INFO:tasks.rados.rados.0.smithi162.stdout:2084: finishing write tid 2 to smithi16234619-50 2024-09-16T15:09:24.357 INFO:tasks.rados.rados.0.smithi162.stdout:2084: finishing write tid 3 to smithi16234619-50 2024-09-16T15:09:24.357 INFO:tasks.rados.rados.0.smithi162.stdout:2092: seq_num 748 ranges {571636=488846,1854531=565263} 2024-09-16T15:09:24.361 INFO:tasks.rados.rados.0.smithi162.stdout:2092: writing smithi16234619-11 from 571636 to 1060482 tid 1 2024-09-16T15:09:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:2092: writing smithi16234619-11 from 1854531 to 2419794 tid 2 2024-09-16T15:09:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:2083: done (4 left) 2024-09-16T15:09:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:2093: write (excl) oid 22 current snap is 195 2024-09-16T15:09:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:2084: finishing write tid 4 to smithi16234619-50 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2084: oid 50 updating version 0 to 802 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2084: oid 50 version 802 is already newer than 800 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2084: oid 50 version 802 is already newer than 801 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 802 (ObjNum 745 snap 195 seq_num 745) dirty exists 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2084: left oid 50 (ObjNum 745 snap 195 seq_num 745) 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2085: finishing copy_from to smithi16234619-48 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 776 (ObjNum 732 snap 194 seq_num 732) dirty exists 2024-09-16T15:09:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:2093: seq_num 749 ranges {588443=564095,1796660=708921} 2024-09-16T15:09:24.375 INFO:tasks.rados.rados.0.smithi162.stdout:2093: writing smithi16234619-22 from 588443 to 1152538 tid 1 2024-09-16T15:09:24.383 INFO:tasks.rados.rados.0.smithi162.stdout:2093: writing smithi16234619-22 from 1796660 to 2505581 tid 2 2024-09-16T15:09:24.383 INFO:tasks.rados.rados.0.smithi162.stdout:2084: done (4 left) 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2085: done (3 left) 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2094: copy_from oid 35 from oid 13 current snap is 195 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: finishing write tid 1 to smithi16234619-20 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: finishing write tid 2 to smithi16234619-20 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: finishing write tid 3 to smithi16234619-20 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: finishing write tid 4 to smithi16234619-20 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: oid 20 updating version 0 to 785 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: oid 20 updating version 785 to 786 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: oid 20 updating version 786 to 787 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 787 (ObjNum 747 snap 195 seq_num 747) dirty exists 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: left oid 20 (ObjNum 747 snap 195 seq_num 747) 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2091: done (3 left) 2024-09-16T15:09:24.384 INFO:tasks.rados.rados.0.smithi162.stdout:2095: snap_remove snap 193 2024-09-16T15:09:24.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.837+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6418017070 con 0x7f641c09bfe0 2024-09-16T15:09:24.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.837+0000 7f64127fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:24.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:24.837+0000 7f64127fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:25.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:25.069+0000 7f6423fed640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:25.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:25.069+0000 7f6423fed640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:25.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:25.069+0000 7f6423fed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:25.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:25.069+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f641c074df0 con 0x7f641c09bfe0 2024-09-16T15:09:27.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:27.004+0000 7f64117fa640 10 monclient: tick 2024-09-16T15:09:27.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:27.004+0000 7f64117fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:27.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:27.004+0000 7f64117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:27.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:27.004+0000 7f64117fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:28.004+0000 7f64117fa640 10 monclient: tick 2024-09-16T15:09:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:28.004+0000 7f64117fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:28.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:28.004+0000 7f64117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:29.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.004+0000 7f64117fa640 10 monclient: tick 2024-09-16T15:09:29.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.004+0000 7f64117fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:29.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.004+0000 7f64117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:29.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.250+0000 7f64127fc640 1 -- 172.21.15.134:0/2505464256 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3189 (secure 0 0 0) 0x7f641803d080 con 0x7f641c09bfe0 2024-09-16T15:09:29.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.251+0000 7f64127fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:29.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.251+0000 7f64127fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:29.253 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":64,"quorum":[1,2,3,4,5,6,7,8],"quorum_names":["f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 msgr2=0x7f640003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 0x7f640003ad40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f641c0a2720 tx=0x7f640c008040 comp rx=0 tx=0).stop 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 10 monclient: shutdown 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 msgr2=0x7f641c11fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f641800e9d0 tx=0x7f641800ee90 comp rx=0 tx=0).stop 2024-09-16T15:09:29.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 shutdown_connections 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> v2:172.21.15.162:6800/980547263 conn(0x7f6400038880 0x7f640003ad40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f641c136c60 0x7f641c135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f641c136030 0x7f641c134cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 --2- 172.21.15.134:0/2505464256 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f641c09bfe0 0x7f641c11fa80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 >> 172.21.15.134:0/2505464256 conn(0x7f641c0a9ea0 msgr2=0x7f641c0a8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 shutdown_connections 2024-09-16T15:09:29.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:29.253+0000 7f6423fed640 1 -- 172.21.15.134:0/2505464256 wait complete. 2024-09-16T15:09:29.783 INFO:tasks.rados.rados.0.smithi162.stdout:2095: done (3 left) 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2096: delete oid 1 current snap is 195 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: finishing write tid 1 to smithi16234619-11 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: finishing write tid 2 to smithi16234619-11 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: finishing write tid 3 to smithi16234619-11 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: finishing write tid 4 to smithi16234619-11 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: oid 11 updating version 0 to 155 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: oid 11 updating version 155 to 157 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: oid 11 version 157 is already newer than 156 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 157 (ObjNum 748 snap 195 seq_num 748) dirty exists 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2092: left oid 11 (ObjNum 748 snap 195 seq_num 748) 2024-09-16T15:09:29.784 INFO:tasks.rados.rados.0.smithi162.stdout:2094: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: finishing write tid 1 to smithi16234619-22 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2094: finishing copy_from to smithi16234619-35 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 885 (ObjNum 728 snap 191 seq_num 728) dirty exists 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: finishing write tid 2 to smithi16234619-22 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: finishing write tid 3 to smithi16234619-22 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: finishing write tid 4 to smithi16234619-22 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: oid 22 updating version 0 to 780 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: oid 22 version 780 is already newer than 778 2024-09-16T15:09:29.785 INFO:tasks.rados.rados.0.smithi162.stdout:2093: oid 22 version 780 is already newer than 779 2024-09-16T15:09:29.786 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 780 (ObjNum 749 snap 195 seq_num 749) dirty exists 2024-09-16T15:09:29.786 INFO:tasks.rados.rados.0.smithi162.stdout:2093: left oid 22 (ObjNum 749 snap 195 seq_num 749) 2024-09-16T15:09:29.797 INFO:tasks.rados.rados.0.smithi162.stdout:2092: done (3 left) 2024-09-16T15:09:29.798 INFO:tasks.rados.rados.0.smithi162.stdout:2093: done (2 left) 2024-09-16T15:09:29.798 INFO:tasks.rados.rados.0.smithi162.stdout:2094: done (1 left) 2024-09-16T15:09:29.798 INFO:tasks.rados.rados.0.smithi162.stdout:2096: done (0 left) 2024-09-16T15:09:29.798 INFO:tasks.rados.rados.0.smithi162.stdout:2097: snap_create 2024-09-16T15:09:32.269 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:32.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 10 monclient: get_monmap_and_config 2024-09-16T15:09:32.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 10 monclient: build_initial_monmap 2024-09-16T15:09:32.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 10 monclient: monmap: 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:32.377061+0000 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:32.377061+0000 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:32.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:32.379 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:32.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 1 Processor -- start 2024-09-16T15:09:32.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 1 -- start start 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.376+0000 7fcce8cb1640 10 monclient: init 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient: _add_conns ranks=[9,1,2,0,4,8,10,6,5,3,7] 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-a con 0x7fcce4136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcce409bfe0 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-b con 0x7fcce409c400 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce4073250 con 0x7fcce409bfe0 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:32.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce409b600 con 0x7fcce4136030 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40a5740 con 0x7fcce409c400 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce8cb1640 10 monclient(hunting): authenticate will time out at 1308.966583s 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:60232/0 (socket says 172.21.15.134:60232) 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 learned_addr learned my addr 172.21.15.134:0/4123815704 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fcce4136030 auth_method 0 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:32.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.377+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:32.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.380+0000 7fcce1d9b640 1 -- 172.21.15.134:0/4123815704 >> v1:172.21.15.162:6789/0 conn(0x7fcce409bfe0 legacy=0x7fcce4136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fcce4136030 auth_method 0 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.578+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:32.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fcce4136030 auth_method 0 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:32.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:32.979+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fcce4136030 auth_method 0 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:33.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:33.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:33.780+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927979464 0 0) 0x7fcce4073250 con 0x7fcce409bfe0 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 10 monclient(hunting): _negotiate 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 10 monclient(hunting): my global_id is 44524 2024-09-16T15:09:35.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.039+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccd0003520 con 0x7fcce409bfe0 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639011840 0 0) 0x7fccd0003520 con 0x7fcce409bfe0 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fcce4137370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fcce4137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 msgr2=0x7fcce4136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 --2- 172.21.15.134:0/4123815704 >> v2:172.21.15.134:3300/0 conn(0x7fcce4136030 0x7fcce4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _finish_hunting 0 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40a52c0 con 0x7fcce409bfe0 2024-09-16T15:09:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _finish_auth 0 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _check_auth_tickets 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fcccc001f60 con 0x7fcce409bfe0 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: dump: 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 >> v1:172.21.15.162:6789/0 conn(0x7fcce409bfe0 legacy=0x7fcce4136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 10 monclient: _add_conns ranks=[4,5,2,3,6,1,8,0,7] 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.042+0000 7fcce159a640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fccd00041c0 0x7fccd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): picked mon.c con 0x7fccd00041c0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fccd0004dd0 0x7fccd0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): picked mon.h con 0x7fccd0004dd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fccd000bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): picked mon.b con 0x7fcce409c400 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce409b600 con 0x7fcce409c400 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40a5740 con 0x7fccd00041c0 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccd000c390 con 0x7fccd0004dd0 2024-09-16T15:09:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce159a640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fccd000bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fccd00041c0 0x7fccd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fccd00041c0 auth_method 0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fccd0004dd0 0x7fccd0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): get_auth_request con 0x7fccd0004dd0 auth_method 0 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient(hunting): handle_auth_done global_id 44524 payload 293 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fccd0004dd0 msgr2=0x7fccd0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:35.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fccd0004dd0 0x7fccd0005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fccd00041c0 msgr2=0x7fccd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fccd00041c0 0x7fccd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 monclient: found mon.b 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0003520 con 0x7fcce409c400 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient: _finish_auth 0 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 10 monclient: _check_auth_tickets 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce2d9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce259c640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fccd00041c0 0x7fccd00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce8cb1640 5 monclient: authenticate success, global_id 44524 2024-09-16T15:09:35.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.043+0000 7fcce8cb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fccd000bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fccd000bc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b90 tx=0x7fccd00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624fbdf239416b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccd402bc50 con 0x7fcce409c400 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: dump: 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:35.058 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd402f070 con 0x7fcce409c400 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce8cb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce8cb1640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:35.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 1 -- 172.21.15.134:0/4123815704 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fccd4035870 con 0x7fcce409c400 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce8cb1640 4 set_mon_vals no callback set 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.046+0000 7fcce159a640 10 monclient: dump: 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:35.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 10 monclient: shutdown 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 -- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 msgr2=0x7fccd000bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/4123815704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcce409c400 0x7fccd000bc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011b90 tx=0x7fccd00057f0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce2d9d640 1 -- 172.21.15.134:0/4123815704 reap_dead start 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 -- 172.21.15.134:0/4123815704 shutdown_connections 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 -- 172.21.15.134:0/4123815704 >> 172.21.15.134:0/4123815704 conn(0x7fcce40a9ea0 msgr2=0x7fcce40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:35.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 -- 172.21.15.134:0/4123815704 shutdown_connections 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.047+0000 7fcce8cb1640 1 -- 172.21.15.134:0/4123815704 wait complete. 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.048+0000 7fcce8cb1640 10 monclient: build_initial_monmap 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.048+0000 7fcce8cb1640 10 monclient: monmap: 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:35.062 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.048+0000 7fcce8cb1640 1 Processor -- start 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 -- start start 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient: init 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient: _add_conns ranks=[5,4,0,7,1,6,2,3,8] 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcce409c400 0x7fcce4123530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-h con 0x7fcce409c400 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-c con 0x7fcce4123a70 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): picked mon.noname-a con 0x7fcce4124390 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:35.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4134e70 con 0x7fcce4124390 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4126e20 con 0x7fcce4123a70 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4127070 con 0x7fcce409c400 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce8cb1640 10 monclient(hunting): authenticate will time out at 1311.638530s 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:46046/0 (socket says 172.21.15.134:46046) 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 -- 172.21.15.134:0/2920100896 learned_addr learned my addr 172.21.15.134:0/2920100896 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): get_auth_request con 0x7fcce4123a70 auth_method 0 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request con 0x7fcce4124390 auth_method 0 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcce409c400 0x7fcce4123530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): get_auth_request con 0x7fcce409c400 auth_method 0 2024-09-16T15:09:35.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce2d9d640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient(hunting): handle_auth_done global_id 5098 payload 824 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcce409c400 msgr2=0x7fcce4123530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcce409c400 0x7fcce4123530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 msgr2=0x7fcce41d1460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 1 monclient: found mon.noname-c 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.049+0000 7fcce1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:35.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.050+0000 7fcce1d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.050+0000 7fcce1d9b640 1 -- 172.21.15.134:0/2920100896 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40a84d0 con 0x7fcce4123a70 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce1d9b640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 msgr2=0x7fcce4123e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce1d9b640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcccc008660 tx=0x7fcccc0101d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcccc010ce0 con 0x7fcce4123a70 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: dump: 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:35.067 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc010e40 con 0x7fcce4123a70 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 5 monclient: authenticate success, global_id 5098 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 10 monclient: _renew_subs 2024-09-16T15:09:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40a48a0 con 0x7fcce4123a70 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcccc013c30 con 0x7fcce4123a70 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fccdaffd640 10 monclient: dump: 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:35.069 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:35.070 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:35.070 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:35.070 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:35.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce0d99640 4 set_mon_vals no callback set 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 10 monclient: _renew_subs 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.052+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce4124ca0 con 0x7fcce4123a70 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcccc013350 con 0x7fcce4123a70 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fccdaffd640 1 --2- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 0x7fccc403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fcce8cb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fcce8cb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fcce8cb1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce41271b0 con 0x7fcce4123a70 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.053+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(415..415 src has 1..415) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fcccc049d70 con 0x7fcce4123a70 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.054+0000 7fcce259c640 1 --2- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 0x7fccc403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.054+0000 7fcce259c640 10 monclient: get_auth_request con 0x7fccc4038880 auth_method 0 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.056+0000 7fcce259c640 1 --2- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 0x7fccc403ad40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009100 tx=0x7fccdc008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:35.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.058+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcce41271b0 con 0x7fcce4123a70 2024-09-16T15:09:35.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.058+0000 7fccdaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:35.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.058+0000 7fccdaffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2097: done (0 left) 2024-09-16T15:09:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2098: write oid 3 current snap is 196 2024-09-16T15:09:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2098: seq_num 751 ranges {622057=714593,1913624=708433} 2024-09-16T15:09:35.115 INFO:tasks.rados.rados.0.smithi162.stdout:2098: writing smithi16234619-3 from 622057 to 1336650 tid 1 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:2098: writing smithi16234619-3 from 1913624 to 2622057 tid 2 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:2099: rollback oid 7 current snap is 196 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 190 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:2100: rollback oid 10 current snap is 196 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 195 2024-09-16T15:09:35.123 INFO:tasks.rados.rados.0.smithi162.stdout:2101: snap_create 2024-09-16T15:09:35.124 INFO:tasks.rados.rados.0.smithi162.stdout:2099: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:09:35.124 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 0 (ObjNum 129 snap 0 seq_num 864452872) dirty dne 2024-09-16T15:09:35.127 INFO:tasks.rados.rados.0.smithi162.stdout:2100: finishing rollback tid 0 to smithi16234619-10 2024-09-16T15:09:35.127 INFO:tasks.rados.rados.0.smithi162.stdout:2100: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:09:35.128 INFO:tasks.rados.rados.0.smithi162.stdout:2100: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:09:35.128 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 965 (ObjNum 708 snap 184 seq_num 708) dirty exists 2024-09-16T15:09:35.147 INFO:tasks.rados.rados.0.smithi162.stdout:2098: finishing write tid 1 to smithi16234619-3 2024-09-16T15:09:35.147 INFO:tasks.rados.rados.0.smithi162.stdout:2098: finishing write tid 2 to smithi16234619-3 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: finishing write tid 3 to smithi16234619-3 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: finishing write tid 4 to smithi16234619-3 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: oid 3 updating version 0 to 105 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: oid 3 version 105 is already newer than 104 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: oid 3 updating version 105 to 106 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 106 (ObjNum 751 snap 196 seq_num 751) dirty exists 2024-09-16T15:09:35.148 INFO:tasks.rados.rados.0.smithi162.stdout:2098: left oid 3 (ObjNum 751 snap 196 seq_num 751) 2024-09-16T15:09:35.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fcce8cb1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fcce8cb1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fcce8cb1640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcce41271b0 con 0x7fcce4123a70 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fccdaffd640 1 -- 172.21.15.134:0/2920100896 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7fcce41271b0 con 0x7fcce4123a70 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fccdaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.291+0000 7fccdaffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:35.293 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":72,"quorum":[1,2,3,4,5,6,7],"quorum_names":["f","b","g","c","h","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:35.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 msgr2=0x7fccc403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:35.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 0x7fccc403ad40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009100 tx=0x7fccdc008040 comp rx=0 tx=0).stop 2024-09-16T15:09:35.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 10 monclient: shutdown 2024-09-16T15:09:35.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:35.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 msgr2=0x7fcce4123e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.293+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcccc008660 tx=0x7fcccc0101d0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 shutdown_connections 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> v2:172.21.15.162:6800/980547263 conn(0x7fccc4038880 0x7fccc403ad40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcce4124390 0x7fcce41d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fcce4123a70 0x7fcce4123e50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 --2- 172.21.15.134:0/2920100896 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcce409c400 0x7fcce4123530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 >> 172.21.15.134:0/2920100896 conn(0x7fcce40a9ea0 msgr2=0x7fcce409fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 shutdown_connections 2024-09-16T15:09:35.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:35.294+0000 7fcce8cb1640 1 -- 172.21.15.134:0/2920100896 wait complete. 2024-09-16T15:09:38.307 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:38.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.414+0000 7fb7949fe640 10 monclient: get_monmap_and_config 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.414+0000 7fb7949fe640 10 monclient: build_initial_monmap 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.414+0000 7fb7949fe640 10 monclient: monmap: 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:38.415381+0000 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:38.415381+0000 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:38.416 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:38.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 Processor -- start 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 -- start start 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient: init 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient: _add_conns ranks=[6,1,7,5,2,3,0,10,4,9,8] 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-d con 0x7fb790136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb790137760 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-i con 0x7fb7900a43d0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb790073250 con 0x7fb790137760 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:38.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900a0f80 con 0x7fb790136e40 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900a1260 con 0x7fb7900a43d0 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb7949fe640 10 monclient(hunting): authenticate will time out at 1315.004632s 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:38734/0 (socket says 172.21.15.134:38734) 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 learned_addr learned my addr 172.21.15.134:0/365930132 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb790136e40 auth_method 0 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:38.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.415+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb7900a43d0 auth_method 0 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.416+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:38.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.418+0000 7fb78e59c640 1 -- 172.21.15.134:0/365930132 >> v1:172.21.15.162:6789/0 conn(0x7fb790137760 legacy=0x7fb7900a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:38.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:38.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb790136e40 auth_method 0 2024-09-16T15:09:38.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:38.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.616+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb7900a43d0 auth_method 0 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:38.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:38.617+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb790136e40 auth_method 0 2024-09-16T15:09:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:39.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:39.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.017+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb7900a43d0 auth_method 0 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:39.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.018+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb790136e40 auth_method 0 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:39.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.818+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb7900a43d0 auth_method 0 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:39.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:39.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:39.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:39.819+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:41.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761500691 0 0) 0x7fb790073250 con 0x7fb790137760 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 10 monclient(hunting): _negotiate 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 10 monclient(hunting): my global_id is 44533 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.048+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb778003520 con 0x7fb790137760 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363880806 0 0) 0x7fb778003520 con 0x7fb790137760 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb79013edc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb79013edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 msgr2=0x7fb790137220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:41.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb790136e40 0x7fb790137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900a09e0 con 0x7fb790137760 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _finish_auth 0 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fb780001be0 con 0x7fb790137760 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb7949fe640 5 monclient: authenticate success, global_id 44533 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb7949fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:41.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: dump: 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:41.058 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 >> v1:172.21.15.162:6789/0 conn(0x7fb790137760 legacy=0x7fb7900a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient: _add_conns ranks=[3,7,0,6,8,4,2,1,5] 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 0x7fb7780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): picked mon.g con 0x7fb778004240 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:41.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb778004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): picked mon.i con 0x7fb7900a43d0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778009970 0x7fb77800bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): picked mon.a con 0x7fb778009970 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900a0f80 con 0x7fb778009970 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900a1260 con 0x7fb778004240 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb77800c460 con 0x7fb7900a43d0 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78dd9b640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb7949fe640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 0x7fb7780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb778004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778009970 0x7fb77800bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb778004240 auth_method 0 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb778009970 auth_method 0 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): get_auth_request con 0x7fb7900a43d0 auth_method 0 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78f59e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.049+0000 7fb78e59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_done global_id 44533 payload 293 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 msgr2=0x7fb778004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7900a43d0 0x7fb778004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778009970 msgr2=0x7fb77800bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb778009970 0x7fb77800bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 monclient: found mon.g 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778003520 con 0x7fb778004240 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 10 monclient: _finish_auth 0 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 10 monclient: _check_auth_tickets 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.050+0000 7fb78ed9d640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78ed9d640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 msgr2=0x7fb7780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 0x7fb7780047c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7840129f0 tx=0x7fb7780057f0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c8eca81bae720909 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb78402bb70 con 0x7fb778004240 2024-09-16T15:09:41.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: dump: 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:41.064 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78402f070 con 0x7fb778004240 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb7949fe640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb7949fe640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 1 -- 172.21.15.134:0/365930132 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb78402bd10 con 0x7fb778004240 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb7949fe640 4 set_mon_vals no callback set 2024-09-16T15:09:41.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.053+0000 7fb78dd9b640 10 monclient: dump: 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:41.066 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 10 monclient: shutdown 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 -- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 msgr2=0x7fb7780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 --2- 172.21.15.134:0/365930132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb778004240 0x7fb7780047c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7840129f0 tx=0x7fb7780057f0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb78f59e640 1 -- 172.21.15.134:0/365930132 reap_dead start 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 -- 172.21.15.134:0/365930132 shutdown_connections 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 -- 172.21.15.134:0/365930132 >> 172.21.15.134:0/365930132 conn(0x7fb790065e30 msgr2=0x7fb790066240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 -- 172.21.15.134:0/365930132 shutdown_connections 2024-09-16T15:09:41.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.054+0000 7fb7949fe640 1 -- 172.21.15.134:0/365930132 wait complete. 2024-09-16T15:09:41.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.055+0000 7fb7949fe640 10 monclient: build_initial_monmap 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.055+0000 7fb7949fe640 10 monclient: monmap: 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:41.073 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 Processor -- start 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 -- start start 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient: init 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient: _add_conns ranks=[1,8,0,7,2,5,6,3,4] 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 0x7fb790135e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-f con 0x7fb778004240 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:41.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-e con 0x7fb79011f910 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): picked mon.noname-a con 0x7fb790120230 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7901cbc70 con 0x7fb790120230 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb790137220 con 0x7fb778004240 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb790122ce0 con 0x7fb79011f910 2024-09-16T15:09:41.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:41.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb7949fe640 10 monclient(hunting): authenticate will time out at 1317.645743s 2024-09-16T15:09:41.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:59320/0 (socket says 172.21.15.134:59320) 2024-09-16T15:09:41.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 1 -- 172.21.15.134:0/3248508032 learned_addr learned my addr 172.21.15.134:0/3248508032 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 10 monclient(hunting): get_auth_request con 0x7fb79011f910 auth_method 0 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78f59e640 10 monclient(hunting): get_auth_request con 0x7fb790120230 auth_method 0 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78f59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78f59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.056+0000 7fb78f59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78e59c640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 msgr2=0x7fb79011fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78e59c640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 msgr2=0x7fb79011fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:09:41.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78e59c640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 0x7fb790135e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78f59e640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 msgr2=0x7fb790120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78f59e640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 msgr2=0x7fb790120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78f59e640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78e59c640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request con 0x7fb778004240 auth_method 0 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78f59e640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:41.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient(hunting): handle_auth_done global_id 44542 payload 824 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 msgr2=0x7fb79011fcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 msgr2=0x7fb790120670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 monclient: found mon.noname-f 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.057+0000 7fb78ed9d640 1 -- 172.21.15.134:0/3248508032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790121d80 con 0x7fb778004240 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.059+0000 7fb78ed9d640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 msgr2=0x7fb790135e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.059+0000 7fb78ed9d640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 0x7fb790135e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb784036000 tx=0x7fb7840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.059+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb784040470 con 0x7fb778004240 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:41.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: dump: 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:41.080 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 5 monclient: authenticate success, global_id 44542 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78400c040 con 0x7fb778004240 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb784040aa0 con 0x7fb778004240 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:41.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7777fe640 10 monclient: dump: 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:41.082 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb78d59a640 4 set_mon_vals no callback set 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 10 monclient: _renew_subs 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7900a8490 con 0x7fb778004240 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 10 monclient: _renew_subs 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.060+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7901232a0 con 0x7fb778004240 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb784040c20 con 0x7fb778004240 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7949fe640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7949fe640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:41.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7949fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:09:41.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb790122f20 con 0x7fb778004240 2024-09-16T15:09:41.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7777fe640 1 --2- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 0x7fb77c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:41.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(416..416 src has 1..416) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fb78406b6c0 con 0x7fb778004240 2024-09-16T15:09:41.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb78e59c640 1 --2- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 0x7fb77c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:41.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.061+0000 7fb78e59c640 10 monclient: get_auth_request con 0x7fb77c038880 auth_method 0 2024-09-16T15:09:41.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.064+0000 7fb78e59c640 1 --2- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 0x7fb77c03ad40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7800014b0 tx=0x7fb78000a040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:41.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.066+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb78400f040 con 0x7fb778004240 2024-09-16T15:09:41.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.066+0000 7fb7777fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:41.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.066+0000 7fb7777fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:41.113 INFO:tasks.rados.rados.0.smithi162.stdout:2098: done (3 left) 2024-09-16T15:09:41.113 INFO:tasks.rados.rados.0.smithi162.stdout:2099: done (2 left) 2024-09-16T15:09:41.113 INFO:tasks.rados.rados.0.smithi162.stdout:2100: done (1 left) 2024-09-16T15:09:41.113 INFO:tasks.rados.rados.0.smithi162.stdout:2101: done (0 left) 2024-09-16T15:09:41.114 INFO:tasks.rados.rados.0.smithi162.stdout:2102: delete oid 22 current snap is 197 2024-09-16T15:09:41.117 INFO:tasks.rados.rados.0.smithi162.stdout:2102: done (0 left) 2024-09-16T15:09:41.117 INFO:tasks.rados.rados.0.smithi162.stdout:2103: write (excl) oid 27 current snap is 197 2024-09-16T15:09:41.117 INFO:tasks.rados.rados.0.smithi162.stdout:2103: seq_num 753 ranges {655671=581444,2030588=625083} 2024-09-16T15:09:41.123 INFO:tasks.rados.rados.0.smithi162.stdout:2103: writing smithi16234619-27 from 655671 to 1237115 tid 1 2024-09-16T15:09:41.131 INFO:tasks.rados.rados.0.smithi162.stdout:2103: writing smithi16234619-27 from 2030588 to 2655671 tid 2 2024-09-16T15:09:41.131 INFO:tasks.rados.rados.0.smithi162.stdout:2104: write oid 5 current snap is 197 2024-09-16T15:09:41.131 INFO:tasks.rados.rados.0.smithi162.stdout:2104: seq_num 754 ranges {672478=656693,2089070=459917} 2024-09-16T15:09:41.138 INFO:tasks.rados.rados.0.smithi162.stdout:2104: writing smithi16234619-5 from 672478 to 1329171 tid 1 2024-09-16T15:09:41.146 INFO:tasks.rados.rados.0.smithi162.stdout:2104: writing smithi16234619-5 from 2089070 to 2548987 tid 2 2024-09-16T15:09:41.146 INFO:tasks.rados.rados.0.smithi162.stdout:2105: snap_remove snap 196 2024-09-16T15:09:41.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.305+0000 7fb7949fe640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:41.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.305+0000 7fb7949fe640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:41.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.305+0000 7fb7949fe640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:09:41.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.305+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb790071c40 con 0x7fb778004240 2024-09-16T15:09:41.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.306+0000 7fb7777fe640 1 -- 172.21.15.134:0/3248508032 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3189 (secure 0 0 0) 0x7fb784044020 con 0x7fb778004240 2024-09-16T15:09:41.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.306+0000 7fb7777fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:41.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.306+0000 7fb7777fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:41.308 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":76,"quorum":[0,1,2,3,4,6,7,8],"quorum_names":["a","f","b","g","c","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:41.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 msgr2=0x7fb77c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 0x7fb77c03ad40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7800014b0 tx=0x7fb78000a040 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 10 monclient: shutdown 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 msgr2=0x7fb790135e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 0x7fb790135e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb784036000 tx=0x7fb7840046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 shutdown_connections 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> v2:172.21.15.162:6800/980547263 conn(0x7fb77c038880 0x7fb77c03ad40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb790120230 0x7fb790120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb79011f910 0x7fb79011fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 --2- 172.21.15.134:0/3248508032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb778004240 0x7fb790135e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 >> 172.21.15.134:0/3248508032 conn(0x7fb790065e30 msgr2=0x7fb7901be910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 shutdown_connections 2024-09-16T15:09:41.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:41.308+0000 7fb7949fe640 1 -- 172.21.15.134:0/3248508032 wait complete. 2024-09-16T15:09:44.321 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.427+0000 7fc702eb1640 10 monclient: get_monmap_and_config 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.427+0000 7fc702eb1640 10 monclient: build_initial_monmap 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.427+0000 7fc702eb1640 10 monclient: monmap: 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:44.428718+0000 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:44.428718+0000 2024-09-16T15:09:44.429 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:44.430 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:44.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.428+0000 7fc702eb1640 1 Processor -- start 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.428+0000 7fc702eb1640 1 -- start start 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.428+0000 7fc702eb1640 10 monclient: init 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient: _add_conns ranks=[6,1,4,3,8,2,9,7,5,0,10] 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-d con 0x7fc6fc136080 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:44.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc6fc0a5700 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-c con 0x7fc6fc0a4690 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc073250 con 0x7fc6fc0a5700 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc09bb00 con 0x7fc6fc0a4690 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc09b4b0 con 0x7fc6fc136080 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc702eb1640 10 monclient(hunting): authenticate will time out at 1321.018186s 2024-09-16T15:09:44.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:38770/0 (socket says 172.21.15.134:38770) 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47100/0 (socket says 172.21.15.134:47100) 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 learned_addr learned my addr 172.21.15.134:0/2722178876 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6fc136080 auth_method 0 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6fc0a4690 auth_method 0 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:44.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> v1:172.21.15.162:6789/0 conn(0x7fc6fc0a5700 legacy=0x7fc6fc0a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> v1:172.21.15.162:6789/0 conn(0x7fc6fc0a5700 legacy=0x7fc6fc0a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:44.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.429+0000 7fc6fbfff640 1 --1- 172.21.15.134:0/2722178876 >> v1:172.21.15.162:6789/0 conn(0x7fc6fc0a5700 0x7fc6fc0a3f80 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-09-16T15:09:44.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:44.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:44.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6fc0a4690 auth_method 0 2024-09-16T15:09:44.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6fc136080 auth_method 0 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:44.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:44.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:44.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.630+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:09:44.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:44.632+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> v1:172.21.15.162:6789/0 conn(0x7fc6fc0a5700 legacy=0x7fc6fc0a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6fc0a4690 auth_method 0 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:45.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6fc136080 auth_method 0 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:45.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.031+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6fc136080 auth_method 0 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6fc0a4690 auth_method 0 2024-09-16T15:09:45.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:45.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:45.832+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:09:47.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146088976 0 0) 0x7fc6fc073250 con 0x7fc6fc0a5700 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient(hunting): _negotiate 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient(hunting): my global_id is 44551 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e4003520 con 0x7fc6fc0a5700 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3904285633 0 0) 0x7fc6e4003520 con 0x7fc6fc0a5700 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6fc136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 msgr2=0x7fc6fc13f0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc6fc0a4690 0x7fc6fc13f0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:47.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc09c6a0 con 0x7fc6fc0a5700 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fc6f00043c0 con 0x7fc6fc0a5700 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.037+0000 7fc6fb7fe640 10 monclient: dump: 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:47.043 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:47.044 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 >> v1:172.21.15.162:6789/0 conn(0x7fc6fc0a5700 legacy=0x7fc6fc0a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient: _add_conns ranks=[6,8,3,7,5,4,1,2,0] 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): picked mon.d con 0x7fc6fc136080 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 0x7fc6e4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): picked mon.e con 0x7fc6e4004360 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:47.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6e4009970 0x7fc6e400bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): picked mon.g con 0x7fc6e4009970 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc09bb00 con 0x7fc6e4009970 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc09b4b0 con 0x7fc6fc136080 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e400c490 con 0x7fc6e4004360 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fb7fe640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 0x7fc6e4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6e4009970 0x7fc6e400bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): get_auth_request con 0x7fc6e4004360 auth_method 0 2024-09-16T15:09:47.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6fc136080 auth_method 0 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6e4009970 auth_method 0 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_done global_id 44551 payload 293 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 msgr2=0x7fc6e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6e4009970 msgr2=0x7fc6e400bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6e4009970 0x7fc6e400bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 monclient: found mon.e 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4003520 con 0x7fc6e4004360 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient: _finish_auth 0 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc6fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc702eb1640 5 monclient: authenticate success, global_id 44551 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc702eb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc700c4d640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc136080 0x7fc6e40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:47.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.038+0000 7fc70144e640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6e4009970 0x7fc6e400bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:47.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.041+0000 7fc6fbfff640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 msgr2=0x7fc6e4004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.041+0000 7fc6fbfff640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 0x7fc6e4004d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00036e0 tx=0x7fc6e40057f0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d8c49e7bc5f2b2af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6f0008540 con 0x7fc6e4004360 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: dump: 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:47.056 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc702eb1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f002d070 con 0x7fc6e4004360 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 1 -- 172.21.15.134:0/2722178876 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6f0005c30 con 0x7fc6e4004360 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc702eb1640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:47.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc702eb1640 4 set_mon_vals no callback set 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.042+0000 7fc6fb7fe640 10 monclient: dump: 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:47.058 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 10 monclient: shutdown 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 -- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 msgr2=0x7fc6e4004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 --2- 172.21.15.134:0/2722178876 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc6e4004360 0x7fc6e4004d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00036e0 tx=0x7fc6e40057f0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc70144e640 1 -- 172.21.15.134:0/2722178876 reap_dead start 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 -- 172.21.15.134:0/2722178876 shutdown_connections 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 -- 172.21.15.134:0/2722178876 >> 172.21.15.134:0/2722178876 conn(0x7fc6fc0a99a0 msgr2=0x7fc6fc0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 -- 172.21.15.134:0/2722178876 shutdown_connections 2024-09-16T15:09:47.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.043+0000 7fc702eb1640 1 -- 172.21.15.134:0/2722178876 wait complete. 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.044+0000 7fc702eb1640 10 monclient: build_initial_monmap 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.044+0000 7fc702eb1640 10 monclient: monmap: 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:47.060 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.044+0000 7fc702eb1640 1 Processor -- start 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 -- start start 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient: init 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient: _add_conns ranks=[2,7,6,5,1,3,0,8,4] 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-b con 0x7fc6e4004360 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc6fc1351e0 0x7fc6fc1355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-i con 0x7fc6fc1351e0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:47.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): picked mon.noname-d con 0x7fc6fc121c20 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc125330 con 0x7fc6e4004360 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc123800 con 0x7fc6fc121c20 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc137500 con 0x7fc6fc1351e0 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.045+0000 7fc702eb1640 10 monclient(hunting): authenticate will time out at 1323.634924s 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47918/0 (socket says 172.21.15.134:47918) 2024-09-16T15:09:47.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57870/0 (socket says 172.21.15.134:57870) 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 1 -- 172.21.15.134:0/387860435 learned_addr learned my addr 172.21.15.134:0/387860435 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): get_auth_request con 0x7fc6e4004360 auth_method 0 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): get_auth_request con 0x7fc6fc121c20 auth_method 0 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc6fc1351e0 0x7fc6fc1355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): get_auth_request con 0x7fc6fc1351e0 auth_method 0 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:47.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:47.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc6fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient(hunting): handle_auth_done global_id 34746 payload 824 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 -- 172.21.15.134:0/387860435 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc6fc1351e0 msgr2=0x7fc6fc1355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc6fc1351e0 0x7fc6fc1355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 -- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 msgr2=0x7fc6fc1cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient: _finish_hunting 0 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 monclient: found mon.noname-d 2024-09-16T15:09:47.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc70144e640 1 -- 172.21.15.134:0/387860435 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc123fc0 con 0x7fc6fc121c20 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.046+0000 7fc700c4d640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc70144e640 1 -- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 msgr2=0x7fc6fc1220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc70144e640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c460 tx=0x7fc6f4016340 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6f4018890 con 0x7fc6fc121c20 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc6f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc6f97fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:47.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.048+0000 7fc6f97fa640 10 monclient: dump: 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:47.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.068 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:47.068 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:47.068 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:47.068 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:47.068 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:47.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 5 monclient: authenticate success, global_id 34746 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f4018a50 con 0x7fc6fc121c20 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6f4018bc0 con 0x7fc6fc121c20 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 10 monclient: _renew_subs 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc125c30 con 0x7fc6fc121c20 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:47.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6f97fa640 10 monclient: dump: 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:47.071 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc6faffd640 4 set_mon_vals no callback set 2024-09-16T15:09:47.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 10 monclient: _renew_subs 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.049+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc137a50 con 0x7fc6fc121c20 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc6f4018d40 con 0x7fc6fc121c20 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc702eb1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc702eb1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:47.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc702eb1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc125470 con 0x7fc6fc121c20 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc6f97fa640 1 --2- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 0x7fc6e803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(417..417 src has 1..417) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc6f404b5c0 con 0x7fc6fc121c20 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc700c4d640 1 --2- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 0x7fc6e803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.050+0000 7fc700c4d640 10 monclient: get_auth_request con 0x7fc6e8038880 auth_method 0 2024-09-16T15:09:47.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.053+0000 7fc700c4d640 1 --2- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 0x7fc6e803ad40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0129c0 tx=0x7fc6ec008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:47.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.055+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc6fc125470 con 0x7fc6fc121c20 2024-09-16T15:09:47.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.055+0000 7fc6f97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:47.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.055+0000 7fc6f97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:47.237 INFO:tasks.rados.rados.0.smithi162.stdout:2105: done (2 left) 2024-09-16T15:09:47.237 INFO:tasks.rados.rados.0.smithi162.stdout:2106: write oid 32 current snap is 197 2024-09-16T15:09:47.237 INFO:tasks.rados.rados.0.smithi162.stdout:2106: seq_num 755 ranges {689285=731942,2147552=541733} 2024-09-16T15:09:47.244 INFO:tasks.rados.rados.0.smithi162.stdout:2106: writing smithi16234619-32 from 689285 to 1421227 tid 1 2024-09-16T15:09:47.252 INFO:tasks.rados.rados.0.smithi162.stdout:2106: writing smithi16234619-32 from 2147552 to 2689285 tid 2 2024-09-16T15:09:47.252 INFO:tasks.rados.rados.0.smithi162.stdout:2107: write (excl) oid 16 current snap is 197 2024-09-16T15:09:47.252 INFO:tasks.rados.rados.0.smithi162.stdout:2107: seq_num 756 ranges {706092=407191,1806034=463586} 2024-09-16T15:09:47.257 INFO:tasks.rados.rados.0.smithi162.stdout:2107: writing smithi16234619-16 from 706092 to 1113283 tid 1 2024-09-16T15:09:47.264 INFO:tasks.rados.rados.0.smithi162.stdout:2107: writing smithi16234619-16 from 1806034 to 2269620 tid 2 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2108: read oid 47 snap -1 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2108: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2109: read oid 38 snap 189 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2109: expect (ObjNum 674 snap 176 seq_num 674) 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2103: finishing write tid 1 to smithi16234619-27 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2103: finishing write tid 2 to smithi16234619-27 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2103: finishing write tid 3 to smithi16234619-27 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2110: read oid 24 snap -1 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2110: expect (ObjNum 704 snap 182 seq_num 704) 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2111: delete oid 41 current snap is 197 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2103: finishing write tid 4 to smithi16234619-27 2024-09-16T15:09:47.265 INFO:tasks.rados.rados.0.smithi162.stdout:2103: oid 27 updating version 0 to 874 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2103: oid 27 version 874 is already newer than 873 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2103: oid 27 updating version 874 to 875 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 875 (ObjNum 753 snap 197 seq_num 753) dirty exists 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2103: left oid 27 (ObjNum 753 snap 197 seq_num 753) 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: finishing write tid 1 to smithi16234619-5 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: finishing write tid 2 to smithi16234619-5 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: finishing write tid 3 to smithi16234619-5 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: finishing write tid 4 to smithi16234619-5 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: oid 5 updating version 0 to 918 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: oid 5 version 918 is already newer than 917 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: oid 5 updating version 918 to 919 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 919 (ObjNum 754 snap 197 seq_num 754) dirty exists 2024-09-16T15:09:47.266 INFO:tasks.rados.rados.0.smithi162.stdout:2104: left oid 5 (ObjNum 754 snap 197 seq_num 754) 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2103: done (7 left) 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2104: done (6 left) 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2111: done (5 left) 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2112: rollback oid 5 current snap is 197 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 190 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2113: write (excl) oid 48 current snap is 197 2024-09-16T15:09:47.269 INFO:tasks.rados.rados.0.smithi162.stdout:2113: seq_num 758 ranges {739706=557689,1806645=467255} 2024-09-16T15:09:47.276 INFO:tasks.rados.rados.0.smithi162.stdout:2113: writing smithi16234619-48 from 739706 to 1297395 tid 1 2024-09-16T15:09:47.282 INFO:tasks.rados.rados.0.smithi162.stdout:2113: writing smithi16234619-48 from 1806645 to 2273900 tid 2 2024-09-16T15:09:47.282 INFO:tasks.rados.rados.0.smithi162.stdout:2114: write oid 1 current snap is 197 2024-09-16T15:09:47.282 INFO:tasks.rados.rados.0.smithi162.stdout:2114: seq_num 759 ranges {756513=632938,1865127=727266} 2024-09-16T15:09:47.289 INFO:tasks.rados.rados.0.smithi162.stdout:2114: writing smithi16234619-1 from 756513 to 1389451 tid 1 2024-09-16T15:09:47.296 INFO:tasks.rados.rados.0.smithi162.stdout:2114: writing smithi16234619-1 from 1865127 to 2592393 tid 2 2024-09-16T15:09:47.297 INFO:tasks.rados.rados.0.smithi162.stdout:2115: write (excl) oid 20 current snap is 197 2024-09-16T15:09:47.297 INFO:tasks.rados.rados.0.smithi162.stdout:2108: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:09:47.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc702eb1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc702eb1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc702eb1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc6fc125470 con 0x7fc6fc121c20 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc6f97fa640 1 -- 172.21.15.134:0/387860435 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3189 (secure 0 0 0) 0x7fc6fc125470 con 0x7fc6fc121c20 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc6f97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.305+0000 7fc6f97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:47.307 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":82,"quorum":[0,1,2,3,4,6,7,8],"quorum_names":["a","f","b","g","c","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 msgr2=0x7fc6e803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 0x7fc6e803ad40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0129c0 tx=0x7fc6ec008040 comp rx=0 tx=0).stop 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 10 monclient: shutdown 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 msgr2=0x7fc6fc1220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c460 tx=0x7fc6f4016340 comp rx=0 tx=0).stop 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 shutdown_connections 2024-09-16T15:09:47.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> v2:172.21.15.162:6800/980547263 conn(0x7fc6e8038880 0x7fc6e803ad40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6fc121c20 0x7fc6fc1220a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc6fc1351e0 0x7fc6fc1355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 --2- 172.21.15.134:0/387860435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6e4004360 0x7fc6fc1cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 >> 172.21.15.134:0/387860435 conn(0x7fc6fc0a99a0 msgr2=0x7fc6fc136460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 shutdown_connections 2024-09-16T15:09:47.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:47.307+0000 7fc702eb1640 1 -- 172.21.15.134:0/387860435 wait complete. 2024-09-16T15:09:47.490 INFO:tasks.rados.rados.0.smithi162.stdout:2110: expect (ObjNum 704 snap 182 seq_num 704) 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:2112: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:2112: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:2112: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 924 (ObjNum 689 snap 178 seq_num 689) dirty exists 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:2107: finishing write tid 1 to smithi16234619-16 2024-09-16T15:09:47.667 INFO:tasks.rados.rados.0.smithi162.stdout:2107: finishing write tid 2 to smithi16234619-16 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: finishing write tid 3 to smithi16234619-16 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: finishing write tid 1 to smithi16234619-32 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: finishing write tid 2 to smithi16234619-32 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: finishing write tid 4 to smithi16234619-16 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: oid 16 updating version 0 to 887 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: oid 16 updating version 887 to 888 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: oid 16 version 888 is already newer than 886 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 888 (ObjNum 756 snap 197 seq_num 756) dirty exists 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2107: left oid 16 (ObjNum 756 snap 197 seq_num 756) 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: finishing write tid 3 to smithi16234619-32 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: finishing write tid 4 to smithi16234619-32 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: oid 32 updating version 0 to 566 2024-09-16T15:09:47.668 INFO:tasks.rados.rados.0.smithi162.stdout:2106: oid 32 updating version 566 to 567 2024-09-16T15:09:47.669 INFO:tasks.rados.rados.0.smithi162.stdout:2106: oid 32 version 567 is already newer than 565 2024-09-16T15:09:47.669 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 567 (ObjNum 755 snap 197 seq_num 755) dirty exists 2024-09-16T15:09:47.669 INFO:tasks.rados.rados.0.smithi162.stdout:2106: left oid 32 (ObjNum 755 snap 197 seq_num 755) 2024-09-16T15:09:47.669 INFO:tasks.rados.rados.0.smithi162.stdout:2109: expect (ObjNum 674 snap 176 seq_num 674) 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: finishing write tid 1 to smithi16234619-48 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: finishing write tid 2 to smithi16234619-48 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: finishing write tid 3 to smithi16234619-48 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: finishing write tid 4 to smithi16234619-48 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: oid 48 updating version 0 to 876 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: oid 48 updating version 876 to 878 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: oid 48 version 878 is already newer than 877 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 878 (ObjNum 758 snap 197 seq_num 758) dirty exists 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2113: left oid 48 (ObjNum 758 snap 197 seq_num 758) 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2114: finishing write tid 1 to smithi16234619-1 2024-09-16T15:09:47.818 INFO:tasks.rados.rados.0.smithi162.stdout:2115: seq_num 760 ranges {773320=708187,1923609=470924} 2024-09-16T15:09:47.825 INFO:tasks.rados.rados.0.smithi162.stdout:2115: writing smithi16234619-20 from 773320 to 1481507 tid 1 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2115: writing smithi16234619-20 from 1923609 to 2394533 tid 2 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2106: done (8 left) 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2107: done (7 left) 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2108: done (6 left) 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2109: done (5 left) 2024-09-16T15:09:47.831 INFO:tasks.rados.rados.0.smithi162.stdout:2110: done (4 left) 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2112: done (3 left) 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2113: done (2 left) 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2116: copy_from oid 19 from oid 25 current snap is 197 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: finishing write tid 2 to smithi16234619-1 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: finishing write tid 3 to smithi16234619-1 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: finishing write tid 4 to smithi16234619-1 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: oid 1 updating version 0 to 880 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: oid 1 updating version 880 to 881 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: oid 1 version 881 is already newer than 879 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 881 (ObjNum 759 snap 197 seq_num 759) dirty exists 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: left oid 1 (ObjNum 759 snap 197 seq_num 759) 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2114: done (2 left) 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:2117: rollback oid 36 current snap is 197 2024-09-16T15:09:47.832 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 194 2024-09-16T15:09:47.833 INFO:tasks.rados.rados.0.smithi162.stdout:2118: snap_create 2024-09-16T15:09:47.836 INFO:tasks.rados.rados.0.smithi162.stdout:2117: finishing rollback tid 0 to smithi16234619-36 2024-09-16T15:09:47.836 INFO:tasks.rados.rados.0.smithi162.stdout:2117: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:09:47.836 INFO:tasks.rados.rados.0.smithi162.stdout:2117: finishing rollback tid 2 to smithi16234619-36 2024-09-16T15:09:47.836 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 814 (ObjNum 741 snap 194 seq_num 741) dirty exists 2024-09-16T15:09:47.855 INFO:tasks.rados.rados.0.smithi162.stdout:2115: finishing write tid 1 to smithi16234619-20 2024-09-16T15:09:47.855 INFO:tasks.rados.rados.0.smithi162.stdout:2115: finishing write tid 2 to smithi16234619-20 2024-09-16T15:09:47.855 INFO:tasks.rados.rados.0.smithi162.stdout:2115: finishing write tid 3 to smithi16234619-20 2024-09-16T15:09:47.855 INFO:tasks.rados.rados.0.smithi162.stdout:2115: finishing write tid 4 to smithi16234619-20 2024-09-16T15:09:47.856 INFO:tasks.rados.rados.0.smithi162.stdout:2115: oid 20 updating version 0 to 788 2024-09-16T15:09:47.856 INFO:tasks.rados.rados.0.smithi162.stdout:2115: oid 20 updating version 788 to 790 2024-09-16T15:09:47.856 INFO:tasks.rados.rados.0.smithi162.stdout:2115: oid 20 version 790 is already newer than 789 2024-09-16T15:09:47.856 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 790 (ObjNum 760 snap 197 seq_num 760) dirty exists 2024-09-16T15:09:47.856 INFO:tasks.rados.rados.0.smithi162.stdout:2115: left oid 20 (ObjNum 760 snap 197 seq_num 760) 2024-09-16T15:09:47.879 INFO:tasks.rados.rados.0.smithi162.stdout:2116: finishing copy_from racing read to smithi16234619-19 2024-09-16T15:09:47.883 INFO:tasks.rados.rados.0.smithi162.stdout:2116: finishing copy_from to smithi16234619-19 2024-09-16T15:09:47.883 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 780 (ObjNum 736 snap 194 seq_num 736) dirty exists 2024-09-16T15:09:50.321 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:50.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.426+0000 7f63f6991640 10 monclient: get_monmap_and_config 2024-09-16T15:09:50.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.426+0000 7f63f6991640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.426+0000 7f63f6991640 10 monclient: monmap: 2024-09-16T15:09:50.427 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.427 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:50.427186+0000 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:50.427186+0000 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:50.429 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:50.429 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 Processor -- start 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 -- start start 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient: init 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient: _add_conns ranks=[2,6,9,10,3,8,4,1,7,0,5] 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 0x7f63f0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-b con 0x7f63f0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f009bfe0 0x7f63f009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-d con 0x7f63f009bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:50.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-a con 0x7f63f0136c60 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f009b600 con 0x7f63f0136c60 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00a5740 con 0x7f63f0136030 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00a52c0 con 0x7f63f009bfe0 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f6991640 10 monclient(hunting): authenticate will time out at 1327.016717s 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 0x7f63f0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63ef7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f009bfe0 0x7f63f009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35582/0 (socket says 172.21.15.134:35582) 2024-09-16T15:09:50.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): get_auth_request con 0x7f63f0136030 auth_method 0 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 1 -- 172.21.15.134:0/3693508531 learned_addr learned my addr 172.21.15.134:0/3693508531 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): get_auth_request con 0x7f63f0136c60 auth_method 0 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63ef7fe640 10 monclient(hunting): get_auth_request con 0x7f63f009bfe0 auth_method 0 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.427+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 10 monclient(hunting): handle_auth_done global_id 44651 payload 274 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 -- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f009bfe0 msgr2=0x7f63f009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 --2- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f009bfe0 0x7f63f009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 -- 172.21.15.134:0/3693508531 >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 msgr2=0x7f63f01370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 --2- 172.21.15.134:0/3693508531 >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 monclient: found mon.noname-b 2024-09-16T15:09:50.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:50.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63effff640 1 -- 172.21.15.134:0/3693508531 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0073250 con 0x7f63f0136030 2024-09-16T15:09:50.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.428+0000 7f63f4f2e640 1 --2- 172.21.15.134:0/3693508531 >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.430+0000 7f63effff640 1 -- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 msgr2=0x7f63f0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.430+0000 7f63effff640 1 --2- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 0x7f63f0136410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63e00097a0 tx=0x7f63e0029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84be981d66f12316 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 1 -- 172.21.15.134:0/3693508531 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63e002ad30 con 0x7f63f0136030 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: dump: 2024-09-16T15:09:50.441 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.442 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 1 -- 172.21.15.134:0/3693508531 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e00307b0 con 0x7f63f0136030 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 5 monclient: authenticate success, global_id 44651 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 1 -- 172.21.15.134:0/3693508531 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63e0030dd0 con 0x7f63f0136030 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 4 set_mon_vals no callback set 2024-09-16T15:09:50.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63eeffd640 10 monclient: dump: 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.444 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 10 monclient: shutdown 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 -- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 msgr2=0x7f63f0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 --2- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 0x7f63f0136410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63e00097a0 tx=0x7f63e0029fd0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 -- 172.21.15.134:0/3693508531 shutdown_connections 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 --2- 172.21.15.134:0/3693508531 >> v2:172.21.15.134:3300/0 conn(0x7f63f0136c60 0x7f63f01370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 --2- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f009bfe0 0x7f63f009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 --2- 172.21.15.134:0/3693508531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136030 0x7f63f0136410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 -- 172.21.15.134:0/3693508531 >> 172.21.15.134:0/3693508531 conn(0x7f63f00a9ea0 msgr2=0x7f63f00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 -- 172.21.15.134:0/3693508531 shutdown_connections 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.431+0000 7f63f6991640 1 -- 172.21.15.134:0/3693508531 wait complete. 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.432+0000 7f63f6991640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.432+0000 7f63f6991640 10 monclient: monmap: 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.446 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.432+0000 7f63f6991640 1 Processor -- start 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.432+0000 7f63f6991640 1 -- start start 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.432+0000 7f63f6991640 10 monclient: init 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient: _add_conns ranks=[4,6,2,0,1,8,7,3,5] 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 0x7f63f01becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-c con 0x7f63f009bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-d con 0x7f63f0136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 0x7f63f01cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): picked mon.noname-b con 0x7f63f0136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00a7c60 con 0x7f63f0136c60 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00a8100 con 0x7f63f009bfe0 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f63f0135870 con 0x7f63f0136030 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f6991640 10 monclient(hunting): authenticate will time out at 1327.022353s 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 0x7f63f01becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57882/0 (socket says 172.21.15.134:57882) 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 0x7f63f01cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 -- 172.21.15.134:0/707688003 learned_addr learned my addr 172.21.15.134:0/707688003 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): get_auth_request con 0x7f63f009bfe0 auth_method 0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): get_auth_request con 0x7f63f0136030 auth_method 0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): get_auth_request con 0x7f63f0136c60 auth_method 0 2024-09-16T15:09:50.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient(hunting): handle_auth_done global_id 34764 payload 824 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 -- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 msgr2=0x7f63f01becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 0x7f63f01becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 -- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 msgr2=0x7f63f01cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 0x7f63f01cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 monclient: found mon.noname-d 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63ef7fe640 1 -- 172.21.15.134:0/707688003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8009b30 con 0x7f63f0136030 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63effff640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 0x7f63f01becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.433+0000 7f63f4f2e640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 0x7f63f01cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ef7fe640 1 -- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 msgr2=0x7f63f0134f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ef7fe640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e980 tx=0x7f63d800ee40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63d8009670 con 0x7f63f0136030 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: dump: 2024-09-16T15:09:50.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 5 monclient: authenticate success, global_id 34764 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d800cc10 con 0x7f63f0136030 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f63d800cd80 con 0x7f63f0136030 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 10 monclient: _renew_subs 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e0009400 con 0x7f63f0136030 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ecff9640 10 monclient: dump: 2024-09-16T15:09:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.454 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63ee7fc640 4 set_mon_vals no callback set 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 10 monclient: _renew_subs 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.436+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f01bf920 con 0x7f63f0136030 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f63d8013470 con 0x7f63f0136030 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63ecff9640 1 --2- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 0x7f63d003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(418..418 src has 1..418) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f63d804a4b0 con 0x7f63f0136030 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63f6991640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63f6991640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63f6991640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f009c400 con 0x7f63f0136030 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63effff640 1 --2- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 0x7f63d003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.437+0000 7f63effff640 10 monclient: get_auth_request con 0x7f63d00388b0 auth_method 0 2024-09-16T15:09:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.440+0000 7f63effff640 1 --2- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 0x7f63d003ad70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63e0034000 tx=0x7f63e002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.441+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f63d8017080 con 0x7f63f0136030 2024-09-16T15:09:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.441+0000 7f63ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.441+0000 7f63ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:50.544 INFO:tasks.rados.rados.0.smithi162.stdout:2115: done (3 left) 2024-09-16T15:09:50.544 INFO:tasks.rados.rados.0.smithi162.stdout:2116: done (2 left) 2024-09-16T15:09:50.544 INFO:tasks.rados.rados.0.smithi162.stdout:2117: done (1 left) 2024-09-16T15:09:50.544 INFO:tasks.rados.rados.0.smithi162.stdout:2118: done (0 left) 2024-09-16T15:09:50.544 INFO:tasks.rados.rados.0.smithi162.stdout:2119: snap_remove snap 197 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63f6991640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63f6991640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63f6991640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f63f0120c90 con 0x7f63f0136030 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63ecff9640 1 -- 172.21.15.134:0/707688003 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f63d80232f0 con 0x7f63f0136030 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.666+0000 7f63ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:50.668 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 msgr2=0x7f63d003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 0x7f63d003ad70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63e0034000 tx=0x7f63e002e000 comp rx=0 tx=0).stop 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 10 monclient: shutdown 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 msgr2=0x7f63f0134f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e980 tx=0x7f63d800ee40 comp rx=0 tx=0).stop 2024-09-16T15:09:50.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 shutdown_connections 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> v2:172.21.15.162:6800/980547263 conn(0x7f63d00388b0 0x7f63d003ad70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63f0136c60 0x7f63f01cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f63f0136030 0x7f63f0134f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 --2- 172.21.15.134:0/707688003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f63f009bfe0 0x7f63f01becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.668+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 >> 172.21.15.134:0/707688003 conn(0x7f63f00a9ea0 msgr2=0x7f63f00a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.669+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 shutdown_connections 2024-09-16T15:09:50.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.669+0000 7f63f6991640 1 -- 172.21.15.134:0/707688003 wait complete. 2024-09-16T15:09:50.680 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:09:50.680 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:09:50.680 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.776+0000 7fa810191640 10 monclient: get_monmap_and_config 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.776+0000 7fa810191640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.776+0000 7fa810191640 10 monclient: monmap: 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:50.777751+0000 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:50.777751+0000 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:50.778 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:50.779 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.777+0000 7fa810191640 1 Processor -- start 2024-09-16T15:09:50.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.777+0000 7fa810191640 1 -- start start 2024-09-16T15:09:50.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.777+0000 7fa810191640 10 monclient: init 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient: _add_conns ranks=[9,5,7,3,2,0,10,4,1,6,8] 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa808136e40 0x7fa80809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): picked mon.noname-a con 0x7fa808136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa808137220 0x7fa808137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): picked mon.noname-h con 0x7fa808137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 0x7fa808136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): picked mon.noname-i con 0x7fa808135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80809abc0 con 0x7fa808136e40 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa80809ba50 con 0x7fa808137220 2024-09-16T15:09:50.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080a53c0 con 0x7fa808135dc0 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa810191640 10 monclient(hunting): authenticate will time out at 1327.367176s 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa808136e40 0x7fa80809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa808136e40 0x7fa80809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35596/0 (socket says 172.21.15.134:35596) 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 -- 172.21.15.134:0/2306369690 learned_addr learned my addr 172.21.15.134:0/2306369690 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): get_auth_request con 0x7fa808136e40 auth_method 0 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 0x7fa808136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 10 monclient(hunting): get_auth_request con 0x7fa808135dc0 auth_method 0 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80d72c640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa808137220 0x7fa808137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80d72c640 10 monclient(hunting): get_auth_request con 0x7fa808137220 auth_method 0 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80d72c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80d72c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80d72c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient(hunting): handle_auth_done global_id 74106 payload 274 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 -- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 msgr2=0x7fa808136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 0x7fa808136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 -- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa808137220 msgr2=0x7fa808137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa808137220 0x7fa808137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 monclient: found mon.noname-a 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80df2d640 1 -- 172.21.15.134:0/2306369690 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808073250 con 0x7fa808136e40 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.778+0000 7fa80e72e640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 0x7fa808136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80df2d640 1 -- 172.21.15.134:0/2306369690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa808136e40 msgr2=0x7fa80809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80df2d640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa808136e40 0x7fa80809c150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20d969bec0220ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 1 -- 172.21.15.134:0/2306369690 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7f8007a70 con 0x7fa808136e40 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:50.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: dump: 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.787 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 1 -- 172.21.15.134:0/2306369690 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f802d050 con 0x7fa808136e40 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa810191640 5 monclient: authenticate success, global_id 74106 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa810191640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa810191640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 1 -- 172.21.15.134:0/2306369690 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7f8007bf0 con 0x7fa808136e40 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa810191640 4 set_mon_vals no callback set 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:50.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa80cf2b640 10 monclient: dump: 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.789 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.790 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.781+0000 7fa810191640 10 monclient: shutdown 2024-09-16T15:09:50.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 -- 172.21.15.134:0/2306369690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa808136e40 msgr2=0x7fa80809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa808136e40 0x7fa80809c150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009e50 tx=0x7fa7f802a560 comp rx=0 tx=0).stop 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 -- 172.21.15.134:0/2306369690 shutdown_connections 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa808135dc0 0x7fa808136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa808137220 0x7fa808137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 --2- 172.21.15.134:0/2306369690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa808136e40 0x7fa80809c150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 -- 172.21.15.134:0/2306369690 >> 172.21.15.134:0/2306369690 conn(0x7fa8080a6110 msgr2=0x7fa8080a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 -- 172.21.15.134:0/2306369690 shutdown_connections 2024-09-16T15:09:50.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 -- 172.21.15.134:0/2306369690 wait complete. 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 10 monclient: monmap: 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.797 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.782+0000 7fa810191640 1 Processor -- start 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 -- start start 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient: init 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient: _add_conns ranks=[6,8,4,0,2,7,3,5,1] 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa808135dc0 0x7fa80809f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): picked mon.noname-d con 0x7fa808135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): picked mon.noname-e con 0x7fa808136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa808137220 0x7fa80811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): picked mon.noname-c con 0x7fa808137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081cc3c0 con 0x7fa808137220 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081cb920 con 0x7fa808135dc0 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa808122930 con 0x7fa808136e40 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa810191640 10 monclient(hunting): authenticate will time out at 1327.372482s 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa808135dc0 0x7fa80809f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33416/0 (socket says 172.21.15.134:33416) 2024-09-16T15:09:50.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 -- 172.21.15.134:0/337704010 learned_addr learned my addr 172.21.15.134:0/337704010 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): get_auth_request con 0x7fa808136e40 auth_method 0 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa808137220 0x7fa80811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): get_auth_request con 0x7fa808135dc0 auth_method 0 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): get_auth_request con 0x7fa808137220 auth_method 0 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80e72e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient(hunting): handle_auth_done global_id 74105 payload 824 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa808135dc0 msgr2=0x7fa80809f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa808135dc0 0x7fa80809f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa808137220 msgr2=0x7fa80811fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa808137220 0x7fa80811fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 monclient: found mon.noname-e 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80d72c640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f8009b30 con 0x7fa808136e40 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.783+0000 7fa80df2d640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa808135dc0 0x7fa80809f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa80d72c640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 msgr2=0x7fa808134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa80d72c640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00d5f0 tx=0x7fa7fc00dab0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7fc012c70 con 0x7fa808136e40 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: dump: 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc012e10 con 0x7fa808136e40 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 5 monclient: authenticate success, global_id 74105 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 10 monclient: _renew_subs 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7fc0148a0 con 0x7fa808136e40 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8081358e0 con 0x7fa808136e40 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:50.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa8067fc640 10 monclient: dump: 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.805 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa807fff640 4 set_mon_vals no callback set 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 10 monclient: _renew_subs 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.786+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa808122370 con 0x7fa808136e40 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa810191640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa810191640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 0x7fa80802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808136450 con 0x7fa8081354b0 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa7fc014430 con 0x7fa808136e40 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 0x7fa80802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient: get_auth_request con 0x7fa8081354b0 auth_method 0 2024-09-16T15:09:50.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa8067fc640 1 --2- 172.21.15.134:0/337704010 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f4038880 0x7fa7f403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(419..419 src has 1..419) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fa7fc04a6a0 con 0x7fa808136e40 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 10 monclient(hunting): handle_auth_done global_id 74115 payload 824 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80df2d640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa808061ae0 con 0x7fa8081354b0 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80e72e640 1 --2- 172.21.15.134:0/337704010 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f4038880 0x7fa7f403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.787+0000 7fa80e72e640 10 monclient: get_auth_request con 0x7fa7f4038880 auth_method 0 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.789+0000 7fa80df2d640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 msgr2=0x7fa80802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.789+0000 7fa80df2d640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 0x7fa80802b260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa808137f00 tx=0x7fa7f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7f8031870 con 0x7fa8081354b0 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 10 monclient: dump: 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.808 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa7f8031400 con 0x7fa8081354b0 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 msgr2=0x7fa80802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 0x7fa80802b260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa808137f00 tx=0x7fa7f8004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:50.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 msgr2=0x7fa80802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:50.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa8067fc640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8081354b0 0x7fa80802b260 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa808137f00 tx=0x7fa7f8004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:50.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.790+0000 7fa80e72e640 1 --2- 172.21.15.134:0/337704010 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f4038880 0x7fa7f403ad40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa800006fd0 tx=0x7fa800004730 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa810191640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa810191640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 0x7fa80802bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa810191640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa808071b20 con 0x7fa80802b7a0 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 0x7fa80802bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient: get_auth_request con 0x7fa80802b7a0 auth_method 0 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 10 monclient(hunting): handle_auth_done global_id 74124 payload 824 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.796+0000 7fa80df2d640 1 -- 172.21.15.134:0/337704010 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa80807a680 con 0x7fa80802b7a0 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa80df2d640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 msgr2=0x7fa80802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa80df2d640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 0x7fa80802bbb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000f80 tx=0x7fa7f802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa7f8049070 con 0x7fa80802b7a0 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 10 monclient: dump: 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.813 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fa7f8007a20 con 0x7fa80802b7a0 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:50.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 msgr2=0x7fa80802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 0x7fa80802bbb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000f80 tx=0x7fa7f802f000 comp rx=0 tx=0).stop 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 msgr2=0x7fa80802bbb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa8067fc640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80802b7a0 0x7fa80802bbb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:50.815 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:50.816 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.817 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:50.818 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:50.819 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:50.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:50.821 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:50.823 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:50.825 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.828 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:50.830 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.833 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.834 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:50.835 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.836 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:50.837 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f4038880 msgr2=0x7fa7f403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 --2- 172.21.15.134:0/337704010 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f4038880 0x7fa7f403ad40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa800006fd0 tx=0x7fa800004730 comp rx=0 tx=0).stop 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 10 monclient: shutdown 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 msgr2=0x7fa808134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00d5f0 tx=0x7fa7fc00dab0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa80e72e640 1 -- 172.21.15.134:0/337704010 reap_dead start 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 shutdown_connections 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 --2- 172.21.15.134:0/337704010 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa808136e40 0x7fa808134f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 >> 172.21.15.134:0/337704010 conn(0x7fa8080a6110 msgr2=0x7fa8080a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 shutdown_connections 2024-09-16T15:09:50.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.799+0000 7fa810191640 1 -- 172.21.15.134:0/337704010 wait complete. 2024-09-16T15:09:50.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:50.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:09:50.839 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.935+0000 7f5d46a80640 10 monclient: get_monmap_and_config 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.935+0000 7f5d46a80640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.935+0000 7f5d46a80640 10 monclient: monmap: 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:50.936631+0000 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:50.936631+0000 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.937 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:50.938 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 Processor -- start 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 -- start start 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient: init 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient: _add_conns ranks=[1,6,10,5,3,9,7,0,2,4,8] 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5d40136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-d con 0x7f5d40137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d4013c370 0x7f5d4013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-f con 0x7f5d4013c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d40073250 con 0x7f5d40136e40 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400a56e0 con 0x7f5d4013c370 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4009abc0 con 0x7f5d40137220 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.936+0000 7f5d46a80640 10 monclient(hunting): authenticate will time out at 1327.525909s 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57910/0 (socket says 172.21.15.134:57910) 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 -- 172.21.15.134:0/3208049454 learned_addr learned my addr 172.21.15.134:0/3208049454 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request con 0x7f5d40137220 auth_method 0 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d4501d640 1 --2- 172.21.15.134:0/3208049454 >> v2:172.21.15.162:3300/0 conn(0x7f5d4013c370 0x7f5d4013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d4501d640 10 monclient(hunting): get_auth_request con 0x7f5d4013c370 auth_method 0 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d4501d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d4501d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d4501d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_done global_id 34782 payload 274 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 -- 172.21.15.134:0/3208049454 >> v2:172.21.15.162:3300/0 conn(0x7f5d4013c370 msgr2=0x7f5d4013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/3208049454 >> v2:172.21.15.162:3300/0 conn(0x7f5d4013c370 0x7f5d4013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 -- 172.21.15.134:0/3208049454 >> v1:172.21.15.162:6789/0 conn(0x7f5d40136e40 legacy=0x7f5d4009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 monclient: found mon.noname-d 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.937+0000 7f5d3ffff640 1 -- 172.21.15.134:0/3208049454 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4009ba50 con 0x7f5d40137220 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3ffff640 1 -- 172.21.15.134:0/3208049454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 msgr2=0x7f5d40137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/3208049454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009870 tx=0x7f5d2c02a100 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3d3d4931902c8092 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 1 -- 172.21.15.134:0/3208049454 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d2c02ae20 con 0x7f5d40137220 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: dump: 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.949 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 1 -- 172.21.15.134:0/3208049454 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c0307b0 con 0x7f5d40137220 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 1 -- 172.21.15.134:0/3208049454 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d2c030d90 con 0x7f5d40137220 2024-09-16T15:09:50.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 5 monclient: authenticate success, global_id 34782 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 4 set_mon_vals no callback set 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d3f7fe640 10 monclient: dump: 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.951 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 10 monclient: shutdown 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 -- 172.21.15.134:0/3208049454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 msgr2=0x7f5d40137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 --2- 172.21.15.134:0/3208049454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009870 tx=0x7f5d2c02a100 comp rx=0 tx=0).stop 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 -- 172.21.15.134:0/3208049454 shutdown_connections 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 --2- 172.21.15.134:0/3208049454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40137220 0x7f5d40137600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 --2- 172.21.15.134:0/3208049454 >> v2:172.21.15.162:3300/0 conn(0x7f5d4013c370 0x7f5d4013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.940+0000 7f5d46a80640 1 -- 172.21.15.134:0/3208049454 >> 172.21.15.134:0/3208049454 conn(0x7f5d400a6110 msgr2=0x7f5d400a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 1 -- 172.21.15.134:0/3208049454 shutdown_connections 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 1 -- 172.21.15.134:0/3208049454 wait complete. 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 10 monclient: build_initial_monmap 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 10 monclient: monmap: 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:50.953 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 1 Processor -- start 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 1 -- start start 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.941+0000 7f5d46a80640 10 monclient: init 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient: _add_conns ranks=[6,8,7,3,1,0,4,2,5] 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 0x7f5d400a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-d con 0x7f5d40136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5d40137220 0x7f5d40135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-e con 0x7f5d40137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5d4013c370 0x7f5d40136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): picked mon.noname-i con 0x7f5d4013c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40122430 con 0x7f5d40136e40 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40122960 con 0x7f5d4013c370 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400a7af0 con 0x7f5d40137220 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d46a80640 10 monclient(hunting): authenticate will time out at 1327.531500s 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 0x7f5d400a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 0x7f5d400a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57912/0 (socket says 172.21.15.134:57912) 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 -- 172.21.15.134:0/1565077316 learned_addr learned my addr 172.21.15.134:0/1565077316 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): get_auth_request con 0x7f5d40136e40 auth_method 0 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5d40137220 0x7f5d40135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request con 0x7f5d40137220 auth_method 0 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4501d640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5d4013c370 0x7f5d40136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4501d640 10 monclient(hunting): get_auth_request con 0x7f5d4013c370 auth_method 0 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4501d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4501d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4501d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient(hunting): handle_auth_done global_id 34791 payload 824 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5d40137220 msgr2=0x7f5d40135f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5d40137220 0x7f5d40135f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5d4013c370 msgr2=0x7f5d40136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5d4013c370 0x7f5d40136480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient: _finish_hunting 0 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 monclient: found mon.noname-d 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.942+0000 7f5d4481c640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0094d0 con 0x7f5d40136e40 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d4481c640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 msgr2=0x7f5d400a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d4481c640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 0x7f5d400a2a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c430 tx=0x7f5d3400c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d34011c10 con 0x7f5d40136e40 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: dump: 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.958 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d34011d90 con 0x7f5d40136e40 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d34012890 con 0x7f5d40136e40 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 5 monclient: authenticate success, global_id 34791 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:50.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3d7fa640 10 monclient: dump: 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.960 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d3effd640 4 set_mon_vals no callback set 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 10 monclient: _renew_subs 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400a75a0 con 0x7f5d40136e40 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 10 monclient: _renew_subs 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.945+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40122050 con 0x7f5d40136e40 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d46a80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d46a80640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 0x7f5d4002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40071b20 con 0x7f5d401cbd30 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5d34012420 con 0x7f5d40136e40 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 0x7f5d4002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient: get_auth_request con 0x7f5d401cbd30 auth_method 0 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3d7fa640 1 --2- 172.21.15.134:0/1565077316 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d24038880 0x7f5d2403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(419..419 src has 1..419) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f5d34049f60 con 0x7f5d40136e40 2024-09-16T15:09:50.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_done global_id 44669 payload 824 2024-09-16T15:09:50.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d3ffff640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d40061ae0 con 0x7f5d401cbd30 2024-09-16T15:09:50.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d4501d640 1 --2- 172.21.15.134:0/1565077316 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d24038880 0x7f5d2403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.946+0000 7f5d4501d640 10 monclient: get_auth_request con 0x7f5d24038880 auth_method 0 2024-09-16T15:09:50.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.948+0000 7f5d3ffff640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 msgr2=0x7f5d4002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.948+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 0x7f5d4002b260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d40137f00 tx=0x7f5d2c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d2c030e20 con 0x7f5d401cbd30 2024-09-16T15:09:50.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 10 monclient: dump: 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.966 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5d2c003610 con 0x7f5d401cbd30 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 msgr2=0x7f5d4002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 0x7f5d4002b260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d40137f00 tx=0x7f5d2c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 msgr2=0x7f5d4002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d3d7fa640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cbd30 0x7f5d4002b260 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d40137f00 tx=0x7f5d2c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.949+0000 7f5d4501d640 1 --2- 172.21.15.134:0/1565077316 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d24038880 0x7f5d2403ad40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d4013ad40 tx=0x7f5d3800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d46a80640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d46a80640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 0x7f5d4002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d46a80640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:50.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40073e50 con 0x7f5d401cc110 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 0x7f5d4002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient: get_auth_request con 0x7f5d401cc110 auth_method 0 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 10 monclient(hunting): handle_auth_done global_id 44678 payload 824 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.955+0000 7f5d3ffff640 1 -- 172.21.15.134:0/1565077316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5d4007a680 con 0x7f5d401cc110 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3ffff640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 msgr2=0x7f5d4002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3ffff640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 0x7f5d4002b7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c02a140 tx=0x7f5d2c009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5d2c003e90 con 0x7f5d401cc110 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:50.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.957+0000 7f5d3d7fa640 10 monclient: dump: 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:50.969 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f5d2c03cb90 con 0x7f5d401cc110 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 msgr2=0x7f5d4002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 0x7f5d4002b7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c02a140 tx=0x7f5d2c009240 comp rx=0 tx=0).stop 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 msgr2=0x7f5d4002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d3d7fa640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d401cc110 0x7f5d4002b7a0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c02a140 tx=0x7f5d2c009240 comp rx=0 tx=0).stop 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:50.970 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:09:50.971 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.972 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:50.974 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:50.975 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.976 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.977 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:50.978 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:50.979 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.980 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.981 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:50.983 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:50.984 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.987 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:50.989 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:50.990 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:50.991 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:50.992 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d24038880 msgr2=0x7f5d2403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 --2- 172.21.15.134:0/1565077316 >> v2:172.21.15.162:6800/980547263 conn(0x7f5d24038880 0x7f5d2403ad40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5d4013ad40 tx=0x7f5d3800b040 comp rx=0 tx=0).stop 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 10 monclient: shutdown 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 msgr2=0x7f5d400a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 --2- 172.21.15.134:0/1565077316 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5d40136e40 0x7f5d400a2a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c430 tx=0x7f5d3400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d4501d640 1 -- 172.21.15.134:0/1565077316 reap_dead start 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d4501d640 1 -- 172.21.15.134:0/1565077316 reap_dead start 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 shutdown_connections 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 >> 172.21.15.134:0/1565077316 conn(0x7f5d400a6110 msgr2=0x7f5d4013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 shutdown_connections 2024-09-16T15:09:50.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:50.958+0000 7f5d46a80640 1 -- 172.21.15.134:0/1565077316 wait complete. 2024-09-16T15:09:50.995 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:50.995 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:09:50.995 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:09:51.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.091+0000 7fa8a390f640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.091+0000 7fa8a390f640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.091+0000 7fa8a390f640 10 monclient: monmap: 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.092255+0000 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.092255+0000 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.094 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.091+0000 7fa8a390f640 1 Processor -- start 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 1 -- start start 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient: init 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient: _add_conns ranks=[0,1,8,4,9,2,10,7,3,5,6] 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa89c136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa89c137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 0x7fa89c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-e con 0x7fa89c13d380 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:51.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c073250 con 0x7fa89c136e40 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c09ba50 con 0x7fa89c137220 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c09c860 con 0x7fa89c13d380 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a390f640 10 monclient(hunting): authenticate will time out at 1327.681590s 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a16ab640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa89c136e40 0x7fa89c09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50816/0 (socket says 172.21.15.134:50816) 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a16ab640 1 -- 172.21.15.134:0/3226734730 learned_addr learned my addr 172.21.15.134:0/3226734730 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 0x7fa89c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): get_auth_request con 0x7fa89c13d380 auth_method 0 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_done global_id 74123 payload 274 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 1 -- 172.21.15.134:0/3226734730 >> v1:172.21.15.162:6789/0 conn(0x7fa89c137220 legacy=0x7fa89c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 1 -- 172.21.15.134:0/3226734730 >> v1:172.21.15.134:6789/0 conn(0x7fa89c136e40 legacy=0x7fa89c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 1 monclient: found mon.noname-e 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:51.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.092+0000 7fa8a1eac640 1 -- 172.21.15.134:0/3226734730 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0a53c0 con 0x7fa89c13d380 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a1eac640 1 -- 172.21.15.134:0/3226734730 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 msgr2=0x7fa89c13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a1eac640 1 --2- 172.21.15.134:0/3226734730 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 0x7fa89c13f770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa894009dc0 tx=0x7fa89402a510 comp rx=0 tx=0).ready entity=mon.8 client_cookie=46945cb6d2557f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 1 -- 172.21.15.134:0/3226734730 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa894007ca0 con 0x7fa89c13d380 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: dump: 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.106 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.107 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 1 -- 172.21.15.134:0/3226734730 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89402d050 con 0x7fa89c13d380 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a390f640 5 monclient: authenticate success, global_id 74123 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a390f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a390f640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 1 -- 172.21.15.134:0/3226734730 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa894007e20 con 0x7fa89c13d380 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa8a390f640 4 set_mon_vals no callback set 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:51.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.095+0000 7fa893fff640 10 monclient: dump: 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.109 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 10 monclient: shutdown 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 -- 172.21.15.134:0/3226734730 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 msgr2=0x7fa89c13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 --2- 172.21.15.134:0/3226734730 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 0x7fa89c13f770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa894009dc0 tx=0x7fa89402a510 comp rx=0 tx=0).stop 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 -- 172.21.15.134:0/3226734730 shutdown_connections 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 --2- 172.21.15.134:0/3226734730 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa89c13d380 0x7fa89c13f770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 -- 172.21.15.134:0/3226734730 >> 172.21.15.134:0/3226734730 conn(0x7fa89c0a6110 msgr2=0x7fa89c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 -- 172.21.15.134:0/3226734730 shutdown_connections 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.096+0000 7fa8a390f640 1 -- 172.21.15.134:0/3226734730 wait complete. 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient: monmap: 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.110 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.111 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 1 Processor -- start 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 1 -- start start 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient: init 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient: _add_conns ranks=[7,1,4,2,0,5,6,8,3] 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa89c136e40 0x7fa89c135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-i con 0x7fa89c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa89c137220 0x7fa89c136350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-f con 0x7fa89c137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 0x7fa89c13a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.097+0000 7fa8a390f640 10 monclient(hunting): picked mon.noname-c con 0x7fa89c13d380 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0a68d0 con 0x7fa89c137220 2024-09-16T15:09:51.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0a64f0 con 0x7fa89c13d380 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c13c5b0 con 0x7fa89c136e40 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a390f640 10 monclient(hunting): authenticate will time out at 1327.687024s 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 0x7fa89c13a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 0x7fa89c13a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52736/0 (socket says 172.21.15.134:52736) 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 learned_addr learned my addr 172.21.15.134:0/2694471305 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): get_auth_request con 0x7fa89c13d380 auth_method 0 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa89c137220 0x7fa89c136350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): get_auth_request con 0x7fa89c137220 auth_method 0 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa89c136e40 0x7fa89c135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): get_auth_request con 0x7fa89c136e40 auth_method 0 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a0eaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient(hunting): handle_auth_done global_id 5125 payload 824 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa89c136e40 msgr2=0x7fa89c135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa89c136e40 0x7fa89c135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa89c137220 msgr2=0x7fa89c136350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa89c137220 0x7fa89c136350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 monclient: found mon.noname-c 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.098+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009a70 con 0x7fa89c13d380 2024-09-16T15:09:51.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 msgr2=0x7fa89c13a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a1eac640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 0x7fa89c13a190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa89402ab00 tx=0x7fa894004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa894007e30 con 0x7fa89c13d380 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: dump: 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.116 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8940046c0 con 0x7fa89c13d380 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a390f640 5 monclient: authenticate success, global_id 5125 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa894004830 con 0x7fa89c13d380 2024-09-16T15:09:51.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a390f640 10 monclient: _renew_subs 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a390f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c1217c0 con 0x7fa89c13d380 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa891ffb640 10 monclient: dump: 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.118 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.100+0000 7fa8937fe640 4 set_mon_vals no callback set 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.101+0000 7fa8a390f640 10 monclient: _renew_subs 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.101+0000 7fa8a390f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.101+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c1214e0 con 0x7fa89c13d380 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a390f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a390f640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 0x7fa89c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c071b20 con 0x7fa89c13ae90 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa894004250 con 0x7fa89c13d380 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 0x7fa89c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient: get_auth_request con 0x7fa89c13ae90 auth_method 0 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa891ffb640 1 --2- 172.21.15.134:0/2694471305 >> v2:172.21.15.162:6800/980547263 conn(0x7fa880038880 0x7fa88003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(419..419 src has 1..419) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fa89406b140 con 0x7fa89c13d380 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_done global_id 5134 payload 824 2024-09-16T15:09:51.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a16ab640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa89c061ae0 con 0x7fa89c13ae90 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a0eaa640 1 --2- 172.21.15.134:0/2694471305 >> v2:172.21.15.162:6800/980547263 conn(0x7fa880038880 0x7fa88003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.102+0000 7fa8a0eaa640 10 monclient: get_auth_request con 0x7fa880038880 auth_method 0 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa8a16ab640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 msgr2=0x7fa89c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa8a16ab640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 0x7fa89c02b260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa89800ea70 tx=0x7fa89800ef30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa89800cc50 con 0x7fa89c13ae90 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa891ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa891ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.104+0000 7fa891ffb640 10 monclient: dump: 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.121 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa89800cdb0 con 0x7fa89c13ae90 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 msgr2=0x7fa89c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 0x7fa89c02b260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa89800ea70 tx=0x7fa89800ef30 comp rx=0 tx=0).stop 2024-09-16T15:09:51.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 msgr2=0x7fa89c02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa891ffb640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13ae90 0x7fa89c02b260 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa89800ea70 tx=0x7fa89800ef30 comp rx=0 tx=0).stop 2024-09-16T15:09:51.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.105+0000 7fa8a0eaa640 1 --2- 172.21.15.134:0/2694471305 >> v2:172.21.15.162:6800/980547263 conn(0x7fa880038880 0x7fa88003ad40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa89c065ee0 tx=0x7fa88c007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a390f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a390f640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 0x7fa89c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a390f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c121670 con 0x7fa89c13b270 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 0x7fa89c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient: get_auth_request con 0x7fa89c13b270 auth_method 0 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 10 monclient(hunting): handle_auth_done global_id 5143 payload 824 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.111+0000 7fa8a16ab640 1 -- 172.21.15.134:0/2694471305 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa89c07a680 con 0x7fa89c13b270 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa8a16ab640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 msgr2=0x7fa89c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa8a16ab640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 0x7fa89c02b7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa898017000 tx=0x7fa898004940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa898015cc0 con 0x7fa89c13b270 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 10 monclient: dump: 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.126 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fa898015850 con 0x7fa89c13b270 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 msgr2=0x7fa89c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.113+0000 7fa891ffb640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 0x7fa89c02b7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa898017000 tx=0x7fa898004940 comp rx=0 tx=0).stop 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa891ffb640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 msgr2=0x7fa89c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa891ffb640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13b270 0x7fa89c02b7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.127 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.128 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.129 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.130 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.131 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.132 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.133 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.134 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.136 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.138 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.140 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.142 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.145 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.147 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.148 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.149 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 >> v2:172.21.15.162:6800/980547263 conn(0x7fa880038880 msgr2=0x7fa88003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 --2- 172.21.15.134:0/2694471305 >> v2:172.21.15.162:6800/980547263 conn(0x7fa880038880 0x7fa88003ad40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa89c065ee0 tx=0x7fa88c007e30 comp rx=0 tx=0).stop 2024-09-16T15:09:51.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 10 monclient: shutdown 2024-09-16T15:09:51.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 msgr2=0x7fa89c13a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 --2- 172.21.15.134:0/2694471305 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa89c13d380 0x7fa89c13a190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa89402ab00 tx=0x7fa894004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 reap_dead start 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a1eac640 1 -- 172.21.15.134:0/2694471305 reap_dead start 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 shutdown_connections 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 >> 172.21.15.134:0/2694471305 conn(0x7fa89c0a6110 msgr2=0x7fa89c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 shutdown_connections 2024-09-16T15:09:51.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.114+0000 7fa8a390f640 1 -- 172.21.15.134:0/2694471305 wait complete. 2024-09-16T15:09:51.151 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.151 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:09:51.151 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:09:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.248+0000 7f78e5af0640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.248+0000 7f78e5af0640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.248+0000 7f78e5af0640 10 monclient: monmap: 2024-09-16T15:09:51.249 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.249195+0000 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.249195+0000 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.250 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.251 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.251 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 Processor -- start 2024-09-16T15:09:51.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 -- start start 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient: init 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient: _add_conns ranks=[0,1,10,3,2,8,6,7,4,5,9] 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f78e0136eb0 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f78e0137290 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f78e013d460 0x7f78e013f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-f con 0x7f78e013d460 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e0073250 con 0x7f78e0136eb0 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00a5560 con 0x7f78e0137290 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e009c610 con 0x7f78e013d460 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78e5af0640 10 monclient(hunting): authenticate will time out at 1327.838724s 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78df7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f78e0136eb0 0x7f78e009bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50830/0 (socket says 172.21.15.134:50830) 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78df7fe640 1 -- 172.21.15.134:0/2555618183 learned_addr learned my addr 172.21.15.134:0/2555618183 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.249+0000 7f78dffff640 1 --2- 172.21.15.134:0/2555618183 >> v2:172.21.15.162:3300/0 conn(0x7f78e013d460 0x7f78e013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): get_auth_request con 0x7f78e013d460 auth_method 0 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient(hunting): handle_auth_done global_id 44578 payload 274 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 1 -- 172.21.15.134:0/2555618183 >> v1:172.21.15.162:6789/0 conn(0x7f78e0137290 legacy=0x7f78e01376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 1 -- 172.21.15.134:0/2555618183 >> v1:172.21.15.134:6789/0 conn(0x7f78e0136eb0 legacy=0x7f78e009bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 1 monclient: found mon.noname-f 2024-09-16T15:09:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:09:51.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.250+0000 7f78dffff640 1 -- 172.21.15.134:0/2555618183 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e009b240 con 0x7f78e013d460 2024-09-16T15:09:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78dffff640 1 -- 172.21.15.134:0/2555618183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78e013d460 msgr2=0x7f78e013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78dffff640 1 --2- 172.21.15.134:0/2555618183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78e013d460 0x7f78e013f850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009d90 tx=0x7f78d802a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2969cd152092b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 1 -- 172.21.15.134:0/2555618183 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78d8007ca0 con 0x7f78e013d460 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: dump: 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.259 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 1 -- 172.21.15.134:0/2555618183 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d802d050 con 0x7f78e013d460 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 1 -- 172.21.15.134:0/2555618183 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78d8007e20 con 0x7f78e013d460 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78deffd640 10 monclient: dump: 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.261 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 5 monclient: authenticate success, global_id 44578 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 4 set_mon_vals no callback set 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 10 monclient: shutdown 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 1 -- 172.21.15.134:0/2555618183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78e013d460 msgr2=0x7f78e013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2555618183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78e013d460 0x7f78e013f850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009d90 tx=0x7f78d802a530 comp rx=0 tx=0).stop 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 1 -- 172.21.15.134:0/2555618183 shutdown_connections 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2555618183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78e013d460 0x7f78e013f850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.253+0000 7f78e5af0640 1 -- 172.21.15.134:0/2555618183 >> 172.21.15.134:0/2555618183 conn(0x7f78e00a99a0 msgr2=0x7f78e00a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 1 -- 172.21.15.134:0/2555618183 shutdown_connections 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 1 -- 172.21.15.134:0/2555618183 wait complete. 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 10 monclient: monmap: 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.263 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 1 Processor -- start 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 1 -- start start 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.254+0000 7f78e5af0640 10 monclient: init 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient: _add_conns ranks=[4,0,5,7,1,6,2,3,8] 2024-09-16T15:09:51.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 0x7f78e0133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-c con 0x7f78e0136eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 0x7f78e0133cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-a con 0x7f78e0137290 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f78e013d460 0x7f78e0138220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): picked mon.noname-h con 0x7f78e013d460 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0134580 con 0x7f78e0137290 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00a7710 con 0x7f78e0136eb0 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78e013a0c0 con 0x7f78e013d460 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78e5af0640 10 monclient(hunting): authenticate will time out at 1327.844332s 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 0x7f78e0133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 0x7f78e0133cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 0x7f78e0133780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52758/0 (socket says 172.21.15.134:52758) 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 1 -- 172.21.15.134:0/2805439188 learned_addr learned my addr 172.21.15.134:0/2805439188 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): get_auth_request con 0x7f78e0136eb0 auth_method 0 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): get_auth_request con 0x7f78e0137290 auth_method 0 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient(hunting): handle_auth_done global_id 74133 payload 824 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f78e013d460 msgr2=0x7f78e0138220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78dffff640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f78e013d460 0x7f78e0138220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f78e013d460 0x7f78e0138220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 msgr2=0x7f78e0133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 0x7f78e0133780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 monclient: found mon.noname-a 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78d7fff640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8009a70 con 0x7f78e0137290 2024-09-16T15:09:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.255+0000 7f78df7fe640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f78e0136eb0 0x7f78e0133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.257+0000 7f78d7fff640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 msgr2=0x7f78e0133cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.257+0000 7f78d7fff640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 0x7f78e0133cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c410 tx=0x7f78c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78c80054a0 con 0x7f78e0137290 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: dump: 2024-09-16T15:09:51.272 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.273 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80117a0 con 0x7f78e0137290 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 5 monclient: authenticate success, global_id 74133 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78c8011d80 con 0x7f78e0137290 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 10 monclient: _renew_subs 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e013c720 con 0x7f78e0137290 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78de7fc640 4 set_mon_vals no callback set 2024-09-16T15:09:51.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78dcff9640 10 monclient: dump: 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.275 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 10 monclient: _renew_subs 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.258+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e013bca0 con 0x7f78e0137290 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78e5af0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 0x7f78e0138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0071b20 con 0x7f78e0138760 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f78c8011910 con 0x7f78e0137290 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 0x7f78e0138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient: get_auth_request con 0x7f78e0138760 auth_method 0 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78dcff9640 1 --2- 172.21.15.134:0/2805439188 >> v2:172.21.15.162:6800/980547263 conn(0x7f78c0038880 0x7f78c003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(419..419 src has 1..419) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f78c8049150 con 0x7f78e0137290 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 10 monclient(hunting): handle_auth_done global_id 34800 payload 824 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78df7fe640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78e0061ae0 con 0x7f78e0138760 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78dffff640 1 --2- 172.21.15.134:0/2805439188 >> v2:172.21.15.162:6800/980547263 conn(0x7f78c0038880 0x7f78c003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.259+0000 7f78dffff640 10 monclient: get_auth_request con 0x7f78c0038880 auth_method 0 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78df7fe640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 msgr2=0x7f78e0138b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78df7fe640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 0x7f78e0138b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d00072e0 tx=0x7f78d00077a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78d000fd40 con 0x7f78e0138760 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78dcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.261+0000 7f78dcff9640 10 monclient: dump: 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.278 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f78d000fea0 con 0x7f78e0138760 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 msgr2=0x7f78e0138b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 0x7f78e0138b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d00072e0 tx=0x7f78d00077a0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 msgr2=0x7f78e0138b40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dcff9640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0138760 0x7f78e0138b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.262+0000 7f78dffff640 1 --2- 172.21.15.134:0/2805439188 >> v2:172.21.15.162:6800/980547263 conn(0x7f78c0038880 0x7f78c003ad40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78e0137f70 tx=0x7f78d802f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78e5af0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 0x7f78e0134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78e5af0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0074b20 con 0x7f78e0139080 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 0x7f78e0134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient: get_auth_request con 0x7f78e0139080 auth_method 0 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 10 monclient(hunting): handle_auth_done global_id 34809 payload 824 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.268+0000 7f78df7fe640 1 -- 172.21.15.134:0/2805439188 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f78e00a5cf0 con 0x7f78e0139080 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78df7fe640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 msgr2=0x7f78e0134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78df7fe640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 0x7f78e0134f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006bf0 tx=0x7f78d0004940 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f78d0023070 con 0x7f78e0139080 2024-09-16T15:09:51.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78dcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.270+0000 7f78dcff9640 10 monclient: dump: 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f78d0006450 con 0x7f78e0139080 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 msgr2=0x7f78e0134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 0x7f78e0134f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006bf0 tx=0x7f78d0004940 comp rx=0 tx=0).stop 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 msgr2=0x7f78e0134f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dcff9640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f78e0139080 0x7f78e0134f30 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006bf0 tx=0x7f78d0004940 comp rx=0 tx=0).stop 2024-09-16T15:09:51.282 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.283 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.284 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.285 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.286 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.287 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.288 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.290 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.292 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.293 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.294 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.296 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.300 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.302 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.303 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.304 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 >> v2:172.21.15.162:6800/980547263 conn(0x7f78c0038880 msgr2=0x7f78c003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2805439188 >> v2:172.21.15.162:6800/980547263 conn(0x7f78c0038880 0x7f78c003ad40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f78e0137f70 tx=0x7f78d802f000 comp rx=0 tx=0).stop 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 10 monclient: shutdown 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 msgr2=0x7f78e0133cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 --2- 172.21.15.134:0/2805439188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e0137290 0x7f78e0133cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c410 tx=0x7f78c800c8d0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dffff640 1 -- 172.21.15.134:0/2805439188 reap_dead start 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78dffff640 1 -- 172.21.15.134:0/2805439188 reap_dead start 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 shutdown_connections 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 >> 172.21.15.134:0/2805439188 conn(0x7f78e00a99a0 msgr2=0x7f78e00a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 shutdown_connections 2024-09-16T15:09:51.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.271+0000 7f78e5af0640 1 -- 172.21.15.134:0/2805439188 wait complete. 2024-09-16T15:09:51.307 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:09:51.307 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:09:51.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.403+0000 7fb5396b1640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.403+0000 7fb5396b1640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.403+0000 7fb5396b1640 10 monclient: monmap: 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.404372+0000 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.404372+0000 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.405 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.406 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 Processor -- start 2024-09-16T15:09:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 -- start start 2024-09-16T15:09:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient: init 2024-09-16T15:09:51.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient: _add_conns ranks=[5,6,4,8,10,0,2,7,9,1,3] 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb534136e40 0x7fb53409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-h con 0x7fb534136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-d con 0x7fb534137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-c con 0x7fb534135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb53409abc0 con 0x7fb534135dc0 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb53409ba50 con 0x7fb534136e40 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340a53c0 con 0x7fb534137220 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5396b1640 10 monclient(hunting): authenticate will time out at 1327.993660s 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52760/0 (socket says 172.21.15.134:52760) 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57934/0 (socket says 172.21.15.134:57934) 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 1 -- 172.21.15.134:0/1255030586 learned_addr learned my addr 172.21.15.134:0/1255030586 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): get_auth_request con 0x7fb534135dc0 auth_method 0 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): get_auth_request con 0x7fb534137220 auth_method 0 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb534136e40 0x7fb53409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): get_auth_request con 0x7fb534136e40 auth_method 0 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb5327fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.404+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 10 monclient(hunting): handle_auth_done global_id 34818 payload 274 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 -- 172.21.15.134:0/1255030586 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb534136e40 msgr2=0x7fb53409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb534136e40 0x7fb53409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 -- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 msgr2=0x7fb534136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 monclient: found mon.noname-d 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5327fc640 1 -- 172.21.15.134:0/1255030586 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534073250 con 0x7fb534137220 2024-09-16T15:09:51.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.405+0000 7fb5337fe640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.407+0000 7fb5327fc640 1 -- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 msgr2=0x7fb534137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.407+0000 7fb5327fc640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb5240097d0 tx=0x7fb524029fb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=a23c59fc4780aa72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.407+0000 7fb531ffb640 1 -- 172.21.15.134:0/1255030586 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb52402ad10 con 0x7fb534137220 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.407+0000 7fb531ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: dump: 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.419 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 5 monclient: authenticate success, global_id 34818 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 1 -- 172.21.15.134:0/1255030586 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52402ae90 con 0x7fb534137220 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 4 set_mon_vals no callback set 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 1 -- 172.21.15.134:0/1255030586 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb524030c20 con 0x7fb534137220 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb531ffb640 10 monclient: dump: 2024-09-16T15:09:51.421 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.422 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 10 monclient: shutdown 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 -- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 msgr2=0x7fb534137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb5240097d0 tx=0x7fb524029fb0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 -- 172.21.15.134:0/1255030586 shutdown_connections 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb534135dc0 0x7fb534136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534137600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 --2- 172.21.15.134:0/1255030586 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb534136e40 0x7fb53409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 -- 172.21.15.134:0/1255030586 >> 172.21.15.134:0/1255030586 conn(0x7fb5340a6110 msgr2=0x7fb5340a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 -- 172.21.15.134:0/1255030586 shutdown_connections 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.408+0000 7fb5396b1640 1 -- 172.21.15.134:0/1255030586 wait complete. 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.409+0000 7fb5396b1640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.409+0000 7fb5396b1640 10 monclient: monmap: 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.423 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.409+0000 7fb5396b1640 1 Processor -- start 2024-09-16T15:09:51.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.409+0000 7fb5396b1640 1 -- start start 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.409+0000 7fb5396b1640 10 monclient: init 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient: _add_conns ranks=[7,1,6,5,2,3,4,8,0] 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb534135dc0 0x7fb5341cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-i con 0x7fb534135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb534136e40 0x7fb534135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-f con 0x7fb534136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): picked mon.noname-d con 0x7fb534137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5341cb7f0 con 0x7fb534136e40 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340a8a50 con 0x7fb534137220 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb5341368e0 con 0x7fb534135dc0 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5396b1640 10 monclient(hunting): authenticate will time out at 1327.999277s 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57950/0 (socket says 172.21.15.134:57950) 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 learned_addr learned my addr 172.21.15.134:0/807648689 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): get_auth_request con 0x7fb534137220 auth_method 0 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb532ffd640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb534135dc0 0x7fb5341cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5327fc640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb534136e40 0x7fb534135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb532ffd640 10 monclient(hunting): get_auth_request con 0x7fb534135dc0 auth_method 0 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb532ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb532ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb532ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5327fc640 10 monclient(hunting): get_auth_request con 0x7fb534136e40 auth_method 0 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5327fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5327fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5327fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient(hunting): handle_auth_done global_id 34827 payload 824 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb534135dc0 msgr2=0x7fb5341cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb534135dc0 0x7fb5341cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb534136e40 msgr2=0x7fb534135020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb534136e40 0x7fb534135020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 monclient: found mon.noname-d 2024-09-16T15:09:51.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.410+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c009b30 con 0x7fb534137220 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.412+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 msgr2=0x7fb534135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.412+0000 7fb5337fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00efc0 tx=0x7fb52c00c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.412+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb52c00cdd0 con 0x7fb534137220 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.412+0000 7fb52b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: dump: 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c00f040 con 0x7fb534137220 2024-09-16T15:09:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 5 monclient: authenticate success, global_id 34827 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 monclient: _renew_subs 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb524009400 con 0x7fb534137220 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb52c014d60 con 0x7fb534137220 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 10 monclient: dump: 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5317fa640 4 set_mon_vals no callback set 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 monclient: _renew_subs 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb534125750 con 0x7fb534137220 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 0x7fb534121eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb52c014480 con 0x7fb534137220 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.413+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb534125250 con 0x7fb534121ad0 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 0x7fb534121eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient: get_auth_request con 0x7fb534121ad0 auth_method 0 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb52b7fe640 1 --2- 172.21.15.134:0/807648689 >> v2:172.21.15.162:6800/980547263 conn(0x7fb514038880 0x7fb51403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(419..419 src has 1..419) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fb52c04a960 con 0x7fb534137220 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 10 monclient(hunting): handle_auth_done global_id 74132 payload 824 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb532ffd640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb534061ae0 con 0x7fb534121ad0 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb5327fc640 1 --2- 172.21.15.134:0/807648689 >> v2:172.21.15.162:6800/980547263 conn(0x7fb514038880 0x7fb51403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.414+0000 7fb5327fc640 10 monclient: get_auth_request con 0x7fb514038880 auth_method 0 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb532ffd640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 msgr2=0x7fb534121eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb532ffd640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 0x7fb534121eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb51c011510 tx=0x7fb51c0119d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb51c0044a0 con 0x7fb534121ad0 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 10 monclient: dump: 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.434 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb5327fc640 1 --2- 172.21.15.134:0/807648689 >> v2:172.21.15.162:6800/980547263 conn(0x7fb514038880 0x7fb51403ad40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb524034000 tx=0x7fb52402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb51c004030 con 0x7fb534121ad0 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 msgr2=0x7fb534121eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 0x7fb534121eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb51c011510 tx=0x7fb51c0119d0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 msgr2=0x7fb534121eb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.416+0000 7fb52b7fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb534121ad0 0x7fb534121eb0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb51c011510 tx=0x7fb51c0119d0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.422+0000 7fb5396b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.422+0000 7fb5396b1640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 0x7fb5340a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb5396b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb534071b20 con 0x7fb5341223f0 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 0x7fb5340a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient: get_auth_request con 0x7fb5341223f0 auth_method 0 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 10 monclient(hunting): handle_auth_done global_id 74141 payload 824 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.423+0000 7fb532ffd640 1 -- 172.21.15.134:0/807648689 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb53407a680 con 0x7fb5341223f0 2024-09-16T15:09:51.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb532ffd640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 msgr2=0x7fb5340a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb532ffd640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 0x7fb5340a1a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51c008010 tx=0x7fb51c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb51c024070 con 0x7fb5341223f0 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb52b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb52b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.425+0000 7fb52b7fe640 10 monclient: dump: 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.438 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fb51c003610 con 0x7fb5341223f0 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 msgr2=0x7fb5340a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 0x7fb5340a1a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51c008010 tx=0x7fb51c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 msgr2=0x7fb5340a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb52b7fe640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb5341223f0 0x7fb5340a1a20 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51c008010 tx=0x7fb51c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.440 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.441 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.442 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.443 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.444 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.446 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.447 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.448 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.451 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.454 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.459 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.460 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 >> v2:172.21.15.162:6800/980547263 conn(0x7fb514038880 msgr2=0x7fb51403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 --2- 172.21.15.134:0/807648689 >> v2:172.21.15.162:6800/980547263 conn(0x7fb514038880 0x7fb51403ad40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb524034000 tx=0x7fb52402e000 comp rx=0 tx=0).stop 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 10 monclient: shutdown 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 msgr2=0x7fb534135560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00efc0 tx=0x7fb52c00c490 comp rx=0 tx=0).stop 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5337fe640 1 -- 172.21.15.134:0/807648689 reap_dead start 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 shutdown_connections 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 --2- 172.21.15.134:0/807648689 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb534137220 0x7fb534135560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 >> 172.21.15.134:0/807648689 conn(0x7fb5340a6110 msgr2=0x7fb53406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 shutdown_connections 2024-09-16T15:09:51.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.426+0000 7fb5396b1640 1 -- 172.21.15.134:0/807648689 wait complete. 2024-09-16T15:09:51.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:09:51.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:09:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2119: done (0 left) 2024-09-16T15:09:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2120: read oid 38 snap -1 2024-09-16T15:09:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2120: expect deleted 2024-09-16T15:09:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2121: rollback oid 29 current snap is 198 2024-09-16T15:09:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 189 2024-09-16T15:09:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2122: copy_from oid 24 from oid 45 current snap is 198 2024-09-16T15:09:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2123: snap_remove snap 190 2024-09-16T15:09:51.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.568+0000 7f30688ec640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.568+0000 7f30688ec640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.568+0000 7f30688ec640 10 monclient: monmap: 2024-09-16T15:09:51.569 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.569402+0000 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.569402+0000 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.570 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.571 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 Processor -- start 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 -- start start 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient: init 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient: _add_conns ranks=[3,0,8,2,9,10,5,6,1,7,4] 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3064136e40 0x7f306409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-g con 0x7f3064136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3064137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 0x7f306413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-e con 0x7f306413c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3064073250 con 0x7f3064137220 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30640a56e0 con 0x7f3064136e40 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f306409abc0 con 0x7f306413c370 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f30688ec640 10 monclient(hunting): authenticate will time out at 1328.158821s 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f305bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3064137220 0x7f3064137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50834/0 (socket says 172.21.15.134:50834) 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f305bfff640 1 -- 172.21.15.134:0/2388777088 learned_addr learned my addr 172.21.15.134:0/2388777088 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f3062d9d640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 0x7f306413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f3062d9d640 10 monclient(hunting): get_auth_request con 0x7f306413c370 auth_method 0 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f3062d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f3062d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.569+0000 7f3062d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3064136e40 0x7f306409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): get_auth_request con 0x7f3064136e40 auth_method 0 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient(hunting): handle_auth_done global_id 74150 payload 274 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 1 -- 172.21.15.134:0/2388777088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3064136e40 msgr2=0x7f306409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3064136e40 0x7f306409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 1 -- 172.21.15.134:0/2388777088 >> v1:172.21.15.134:6789/0 conn(0x7f3064137220 legacy=0x7f3064137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 1 monclient: found mon.noname-e 2024-09-16T15:09:51.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:51.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.570+0000 7f3062d9d640 1 -- 172.21.15.134:0/2388777088 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306409ba50 con 0x7f306413c370 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3062d9d640 1 -- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 msgr2=0x7f306413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3062d9d640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 0x7f306413e760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f305c009dc0 tx=0x7f305c02a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=28e7ea11ffd18c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 1 -- 172.21.15.134:0/2388777088 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f305c007bc0 con 0x7f306413c370 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: dump: 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.582 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 5 monclient: authenticate success, global_id 74150 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 1 -- 172.21.15.134:0/2388777088 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c02d050 con 0x7f306413c370 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 1 -- 172.21.15.134:0/2388777088 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f305c007d40 con 0x7f306413c370 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 4 set_mon_vals no callback set 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f3061d9b640 10 monclient: dump: 2024-09-16T15:09:51.584 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.585 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 10 monclient: shutdown 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 -- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 msgr2=0x7f306413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 0x7f306413e760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f305c009dc0 tx=0x7f305c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 -- 172.21.15.134:0/2388777088 shutdown_connections 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f306413c370 0x7f306413e760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 --2- 172.21.15.134:0/2388777088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3064136e40 0x7f306409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 -- 172.21.15.134:0/2388777088 >> 172.21.15.134:0/2388777088 conn(0x7f30640a6110 msgr2=0x7f30640a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 -- 172.21.15.134:0/2388777088 shutdown_connections 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.573+0000 7f30688ec640 1 -- 172.21.15.134:0/2388777088 wait complete. 2024-09-16T15:09:51.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.574+0000 7f30688ec640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.574+0000 7f30688ec640 10 monclient: monmap: 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.574+0000 7f30688ec640 1 Processor -- start 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 -- start start 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient: init 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient: _add_conns ranks=[6,5,7,2,1,3,8,0,4] 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-d con 0x7f3064136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3064137220 0x7f30641201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-h con 0x7f3064137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f306413c370 0x7f30641206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): picked mon.noname-i con 0x7f306413c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30640a79d0 con 0x7f3064137220 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30640a74a0 con 0x7f3064136e40 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3064121150 con 0x7f306413c370 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f30688ec640 10 monclient(hunting): authenticate will time out at 1328.164496s 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57964/0 (socket says 172.21.15.134:57964) 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 -- 172.21.15.134:0/1496197624 learned_addr learned my addr 172.21.15.134:0/1496197624 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): get_auth_request con 0x7f3064136e40 auth_method 0 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3064137220 0x7f30641201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f306413c370 0x7f30641206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): get_auth_request con 0x7f3064137220 auth_method 0 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 10 monclient(hunting): get_auth_request con 0x7f306413c370 auth_method 0 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f305bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient(hunting): handle_auth_done global_id 34836 payload 824 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f306413c370 msgr2=0x7f30641206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f306413c370 0x7f30641206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3064137220 msgr2=0x7f30641201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3064137220 0x7f30641201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 monclient: found mon.noname-d 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f306259c640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009a70 con 0x7f3064136e40 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.575+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f306413c370 0x7f30641206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f306259c640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 msgr2=0x7f306411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f306259c640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9b0 tx=0x7f305400ee70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3054009670 con 0x7f3064136e40 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: dump: 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.592 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400cbb0 con 0x7f3064136e40 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3054013d50 con 0x7f3064136e40 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f305affd640 10 monclient: dump: 2024-09-16T15:09:51.593 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.594 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 5 monclient: authenticate success, global_id 34836 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f306159a640 4 set_mon_vals no callback set 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 10 monclient: _renew_subs 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3064121ff0 con 0x7f3064136e40 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 10 monclient: _renew_subs 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.578+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30640a3170 con 0x7f3064136e40 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f305400cd20 con 0x7f3064136e40 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f305affd640 1 --2- 172.21.15.134:0/1496197624 >> v2:172.21.15.162:6800/980547263 conn(0x7f3044038880 0x7f304403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(420..420 src has 1..420) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f305404a490 con 0x7f3064136e40 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f30688ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f30688ec640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 0x7f306402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064071b20 con 0x7f3064135810 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f305bfff640 1 --2- 172.21.15.134:0/1496197624 >> v2:172.21.15.162:6800/980547263 conn(0x7f3044038880 0x7f304403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f305bfff640 10 monclient: get_auth_request con 0x7f3044038880 auth_method 0 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 0x7f306402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient: get_auth_request con 0x7f3064135810 auth_method 0 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 10 monclient(hunting): handle_auth_done global_id 44596 payload 824 2024-09-16T15:09:51.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.579+0000 7f3062d9d640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3064061ae0 con 0x7f3064135810 2024-09-16T15:09:51.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305bfff640 1 --2- 172.21.15.134:0/1496197624 >> v2:172.21.15.162:6800/980547263 conn(0x7f3044038880 0x7f304403ad40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f304c007c90 tx=0x7f304c007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f3062d9d640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 msgr2=0x7f306402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 0x7f306402b260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f305c034000 tx=0x7f305c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f305c0044b0 con 0x7f3064135810 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 10 monclient: dump: 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.600 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f305c004610 con 0x7f3064135810 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 msgr2=0x7f306402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 0x7f306402b260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f305c034000 tx=0x7f305c004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 msgr2=0x7f306402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.582+0000 7f305affd640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135810 0x7f306402b260 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f305c034000 tx=0x7f305c004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f30688ec640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f30688ec640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 0x7f306402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f30688ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641217a0 con 0x7f3064135bf0 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 0x7f306402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient: get_auth_request con 0x7f3064135bf0 auth_method 0 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 10 monclient(hunting): handle_auth_done global_id 44605 payload 824 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.594+0000 7f3062d9d640 1 -- 172.21.15.134:0/1496197624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f306407a680 con 0x7f3064135bf0 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f3062d9d640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 msgr2=0x7f306402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f3062d9d640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 0x7f306402b7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9f0 tx=0x7f305c009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f305c004310 con 0x7f3064135bf0 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 10 monclient: dump: 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.603 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f305c04a9a0 con 0x7f3064135bf0 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 msgr2=0x7f306402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 0x7f306402b7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9f0 tx=0x7f305c009750 comp rx=0 tx=0).stop 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 msgr2=0x7f306402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.597+0000 7f305affd640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3064135bf0 0x7f306402b7a0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f305c02a9f0 tx=0x7f305c009750 comp rx=0 tx=0).stop 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.605 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.606 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.607 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.608 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.609 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.610 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.611 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.614 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.615 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.616 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.617 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.618 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.619 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.621 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.625 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.627 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.628 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 >> v2:172.21.15.162:6800/980547263 conn(0x7f3044038880 msgr2=0x7f304403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 --2- 172.21.15.134:0/1496197624 >> v2:172.21.15.162:6800/980547263 conn(0x7f3044038880 0x7f304403ad40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f304c007c90 tx=0x7f304c007be0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 10 monclient: shutdown 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 msgr2=0x7f306411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9b0 tx=0x7f305400ee70 comp rx=0 tx=0).stop 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f3062d9d640 1 -- 172.21.15.134:0/1496197624 reap_dead start 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 shutdown_connections 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 --2- 172.21.15.134:0/1496197624 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3064136e40 0x7f306411fc60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 >> 172.21.15.134:0/1496197624 conn(0x7f30640a6110 msgr2=0x7f306413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 shutdown_connections 2024-09-16T15:09:51.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.598+0000 7f30688ec640 1 -- 172.21.15.134:0/1496197624 wait complete. 2024-09-16T15:09:51.631 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.631 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:09:51.631 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.727+0000 7f85fc9f4640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.727+0000 7f85fc9f4640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.727+0000 7f85fc9f4640 10 monclient: monmap: 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.728724+0000 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.728724+0000 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.730 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 Processor -- start 2024-09-16T15:09:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 -- start start 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient: init 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient: _add_conns ranks=[10,4,7,3,9,1,8,0,2,5,6] 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85f8136e40 0x7f85f809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-f con 0x7f85f8136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-c con 0x7f85f8137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85f8135dc0 0x7f85f8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-i con 0x7f85f8135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f809abc0 con 0x7f85f8136e40 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f809ba50 con 0x7f85f8137220 2024-09-16T15:09:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80a53c0 con 0x7f85f8135dc0 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.728+0000 7f85fc9f4640 10 monclient(hunting): authenticate will time out at 1328.317997s 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52768/0 (socket says 172.21.15.134:52768) 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 -- 172.21.15.134:0/4085335406 learned_addr learned my addr 172.21.15.134:0/4085335406 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): get_auth_request con 0x7f85f8137220 auth_method 0 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient(hunting): handle_auth_done global_id 5170 payload 274 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85f659c640 1 --2- 172.21.15.134:0/4085335406 >> v2:172.21.15.162:3300/0 conn(0x7f85f8136e40 0x7f85f809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 -- 172.21.15.134:0/4085335406 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85f8135dc0 msgr2=0x7f85f8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 --2- 172.21.15.134:0/4085335406 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85f8135dc0 0x7f85f8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 -- 172.21.15.134:0/4085335406 >> v2:172.21.15.162:3300/0 conn(0x7f85f8136e40 msgr2=0x7f85f809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 --2- 172.21.15.134:0/4085335406 >> v2:172.21.15.162:3300/0 conn(0x7f85f8136e40 0x7f85f809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 monclient: found mon.noname-c 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.729+0000 7f85effff640 1 -- 172.21.15.134:0/4085335406 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8073250 con 0x7f85f8137220 2024-09-16T15:09:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85effff640 1 -- 172.21.15.134:0/4085335406 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 msgr2=0x7f85f8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85effff640 1 --2- 172.21.15.134:0/4085335406 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85e00098a0 tx=0x7f85e002a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=aa614749879bf3b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 1 -- 172.21.15.134:0/4085335406 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e00307b0 con 0x7f85f8137220 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: dump: 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.743 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 5 monclient: authenticate success, global_id 5170 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 1 -- 172.21.15.134:0/4085335406 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0030930 con 0x7f85f8137220 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 4 set_mon_vals no callback set 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 1 -- 172.21.15.134:0/4085335406 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e00313a0 con 0x7f85f8137220 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85f5d9b640 10 monclient: dump: 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.745 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 10 monclient: shutdown 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 1 -- 172.21.15.134:0/4085335406 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 msgr2=0x7f85f8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/4085335406 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85e00098a0 tx=0x7f85e002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 1 -- 172.21.15.134:0/4085335406 shutdown_connections 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/4085335406 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f85f8135dc0 0x7f85f8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.732+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/4085335406 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f85f8137220 0x7f85f8137600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/4085335406 >> v2:172.21.15.162:3300/0 conn(0x7f85f8136e40 0x7f85f809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 1 -- 172.21.15.134:0/4085335406 >> 172.21.15.134:0/4085335406 conn(0x7f85f80a6110 msgr2=0x7f85f80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 1 -- 172.21.15.134:0/4085335406 shutdown_connections 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 1 -- 172.21.15.134:0/4085335406 wait complete. 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.733+0000 7f85fc9f4640 10 monclient: monmap: 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.747 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 Processor -- start 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 -- start start 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient: init 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient: _add_conns ranks=[0,2,3,7,6,8,4,1,5] 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 0x7f85f809f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-a con 0x7f85f8135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 0x7f85f8134e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-b con 0x7f85f8136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8137220 0x7f85f8135350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): picked mon.noname-g con 0x7f85f8137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8122210 con 0x7f85f8135dc0 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81364b0 con 0x7f85f8136e40 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8121f30 con 0x7f85f8137220 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85fc9f4640 10 monclient(hunting): authenticate will time out at 1328.323557s 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 0x7f85f809f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 0x7f85f8134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 0x7f85f809f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35638/0 (socket says 172.21.15.134:35638) 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 0x7f85f8134e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47976/0 (socket says 172.21.15.134:47976) 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 -- 172.21.15.134:0/3743945947 learned_addr learned my addr 172.21.15.134:0/3743945947 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): get_auth_request con 0x7f85f8135dc0 auth_method 0 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): get_auth_request con 0x7f85f8136e40 auth_method 0 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8137220 0x7f85f8135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): get_auth_request con 0x7f85f8137220 auth_method 0 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient(hunting): handle_auth_done global_id 74142 payload 824 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8137220 msgr2=0x7f85f8135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8137220 0x7f85f8135350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 msgr2=0x7f85f8134e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 0x7f85f8134e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 1 monclient: found mon.noname-a 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.734+0000 7f85f659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.735+0000 7f85f659c640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00094d0 con 0x7f85f8135dc0 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.735+0000 7f85effff640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f8136e40 0x7f85f8134e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85f659c640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 msgr2=0x7f85f809f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85f659c640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 0x7f85f809f450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c7f0 tx=0x7f85e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e8007ca0 con 0x7f85f8135dc0 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: dump: 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.752 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8007e60 con 0x7f85f8135dc0 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 5 monclient: authenticate success, global_id 74142 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 10 monclient: _renew_subs 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f809fe30 con 0x7f85f8135dc0 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e8011860 con 0x7f85f8135dc0 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85eeffd640 10 monclient: dump: 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.754 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85f559a640 4 set_mon_vals no callback set 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 10 monclient: _renew_subs 2024-09-16T15:09:51.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.737+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f811fc60 con 0x7f85f8135dc0 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85fc9f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 0x7f85f802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8071b20 con 0x7f85f8120670 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f85e80113f0 con 0x7f85f8135dc0 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 0x7f85f802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient: get_auth_request con 0x7f85f8120670 auth_method 0 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85eeffd640 1 --2- 172.21.15.134:0/3743945947 >> v2:172.21.15.162:6800/980547263 conn(0x7f85d8038880 0x7f85d803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(420..420 src has 1..420) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f85e8049b00 con 0x7f85f8135dc0 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85f6d9d640 1 --2- 172.21.15.134:0/3743945947 >> v2:172.21.15.162:6800/980547263 conn(0x7f85d8038880 0x7f85d803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 10 monclient(hunting): handle_auth_done global_id 54291 payload 824 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85effff640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85f8061ae0 con 0x7f85f8120670 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.738+0000 7f85f6d9d640 10 monclient: get_auth_request con 0x7f85d8038880 auth_method 0 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85effff640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 msgr2=0x7f85f802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85effff640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 0x7f85f802b260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85e0034000 tx=0x7f85e00046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e0003c90 con 0x7f85f8120670 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 10 monclient: dump: 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.757 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85f6d9d640 1 --2- 172.21.15.134:0/3743945947 >> v2:172.21.15.162:6800/980547263 conn(0x7f85d8038880 0x7f85d803ad40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85f809fb20 tx=0x7f85f000d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f85e0003df0 con 0x7f85f8120670 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 msgr2=0x7f85f802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 0x7f85f802b260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85e0034000 tx=0x7f85e00046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 msgr2=0x7f85f802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.741+0000 7f85eeffd640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f8120670 0x7f85f802b260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85fc9f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 0x7f85f802bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85fc9f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f807a680 con 0x7f85f802b7a0 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 0x7f85f802bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient: get_auth_request con 0x7f85f802b7a0 auth_method 0 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.747+0000 7f85effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.748+0000 7f85effff640 10 monclient(hunting): handle_auth_done global_id 54300 payload 824 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.748+0000 7f85effff640 1 -- 172.21.15.134:0/3743945947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f85f809b490 con 0x7f85f802b7a0 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85effff640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 msgr2=0x7f85f802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85effff640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 0x7f85f802bbb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0008030 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85e004d070 con 0x7f85f802b7a0 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85eeffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.750+0000 7f85eeffd640 10 monclient: dump: 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.762 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f85e003de50 con 0x7f85f802b7a0 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 msgr2=0x7f85f802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 0x7f85f802bbb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0008030 comp rx=0 tx=0).stop 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 msgr2=0x7f85f802bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85eeffd640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85f802b7a0 0x7f85f802bbb0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0008030 comp rx=0 tx=0).stop 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.764 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.765 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.766 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.767 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.768 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.769 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.772 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.773 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.774 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.775 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.776 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.777 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.778 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.781 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.783 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.784 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.785 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 >> v2:172.21.15.162:6800/980547263 conn(0x7f85d8038880 msgr2=0x7f85d803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/3743945947 >> v2:172.21.15.162:6800/980547263 conn(0x7f85d8038880 0x7f85d803ad40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85f809fb20 tx=0x7f85f000d040 comp rx=0 tx=0).stop 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 10 monclient: shutdown 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 msgr2=0x7f85f809f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 --2- 172.21.15.134:0/3743945947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f8135dc0 0x7f85f809f450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c7f0 tx=0x7f85e800ccb0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85f6d9d640 1 -- 172.21.15.134:0/3743945947 reap_dead start 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85f6d9d640 1 -- 172.21.15.134:0/3743945947 reap_dead start 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 shutdown_connections 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 >> 172.21.15.134:0/3743945947 conn(0x7f85f80a6110 msgr2=0x7f85f80a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 shutdown_connections 2024-09-16T15:09:51.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.751+0000 7f85fc9f4640 1 -- 172.21.15.134:0/3743945947 wait complete. 2024-09-16T15:09:51.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.788 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:09:51.788 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:09:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.884+0000 7fb27d27d640 10 monclient: get_monmap_and_config 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.884+0000 7fb27d27d640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.884+0000 7fb27d27d640 10 monclient: monmap: 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:51.885527+0000 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:51.885527+0000 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:51.887 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 Processor -- start 2024-09-16T15:09:51.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 -- start start 2024-09-16T15:09:51.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient: init 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient: _add_conns ranks=[10,3,4,9,7,2,0,6,8,1,5] 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb278136e40 0x7fb27809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-f con 0x7fb278136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb278137220 0x7fb278137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-g con 0x7fb278137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-c con 0x7fb278135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27809abc0 con 0x7fb278136e40 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27809ba50 con 0x7fb278137220 2024-09-16T15:09:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780a53c0 con 0x7fb278135dc0 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb27d27d640 10 monclient(hunting): authenticate will time out at 1328.474675s 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52782/0 (socket says 172.21.15.134:52782) 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 1 -- 172.21.15.134:0/3059303812 learned_addr learned my addr 172.21.15.134:0/3059303812 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): get_auth_request con 0x7fb278135dc0 auth_method 0 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.885+0000 7fb26e7fc640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb278137220 0x7fb278137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb276ffd640 1 --2- 172.21.15.134:0/3059303812 >> v2:172.21.15.162:3300/0 conn(0x7fb278136e40 0x7fb27809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb26e7fc640 10 monclient(hunting): get_auth_request con 0x7fb278137220 auth_method 0 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb26e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb26e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb26e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb276ffd640 10 monclient(hunting): get_auth_request con 0x7fb278136e40 auth_method 0 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb276ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb276ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb276ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 10 monclient(hunting): handle_auth_done global_id 5179 payload 274 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 -- 172.21.15.134:0/3059303812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb278137220 msgr2=0x7fb278137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb278137220 0x7fb278137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 -- 172.21.15.134:0/3059303812 >> v2:172.21.15.162:3300/0 conn(0x7fb278136e40 msgr2=0x7fb27809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 --2- 172.21.15.134:0/3059303812 >> v2:172.21.15.162:3300/0 conn(0x7fb278136e40 0x7fb27809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 monclient: found mon.noname-c 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.886+0000 7fb2777fe640 1 -- 172.21.15.134:0/3059303812 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278073250 con 0x7fb278135dc0 2024-09-16T15:09:51.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.888+0000 7fb2777fe640 1 -- 172.21.15.134:0/3059303812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 msgr2=0x7fb278136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.888+0000 7fb2777fe640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e80 tx=0x7fb27002a580 comp rx=0 tx=0).ready entity=mon.4 client_cookie=eea09699a77fdbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 1 -- 172.21.15.134:0/3059303812 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb270007a70 con 0x7fb278135dc0 2024-09-16T15:09:51.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.900 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: dump: 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.901 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 1 -- 172.21.15.134:0/3059303812 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27002d050 con 0x7fb278135dc0 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 5 monclient: authenticate success, global_id 5179 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:51.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 4 set_mon_vals no callback set 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 1 -- 172.21.15.134:0/3059303812 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb270007bf0 con 0x7fb278135dc0 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb2767fc640 10 monclient: dump: 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.903 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 10 monclient: shutdown 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 -- 172.21.15.134:0/3059303812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 msgr2=0x7fb278136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e80 tx=0x7fb27002a580 comp rx=0 tx=0).stop 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 -- 172.21.15.134:0/3059303812 shutdown_connections 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb278135dc0 0x7fb278136220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 --2- 172.21.15.134:0/3059303812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb278137220 0x7fb278137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 --2- 172.21.15.134:0/3059303812 >> v2:172.21.15.162:3300/0 conn(0x7fb278136e40 0x7fb27809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 -- 172.21.15.134:0/3059303812 >> 172.21.15.134:0/3059303812 conn(0x7fb2780a6110 msgr2=0x7fb2780a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 -- 172.21.15.134:0/3059303812 shutdown_connections 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.889+0000 7fb27d27d640 1 -- 172.21.15.134:0/3059303812 wait complete. 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.890+0000 7fb27d27d640 10 monclient: build_initial_monmap 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.890+0000 7fb27d27d640 10 monclient: monmap: 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:51.905 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.890+0000 7fb27d27d640 1 Processor -- start 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.890+0000 7fb27d27d640 1 -- start start 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.890+0000 7fb27d27d640 10 monclient: init 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient: _add_conns ranks=[8,7,1,6,4,5,0,2,3] 2024-09-16T15:09:51.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-e con 0x7fb278135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb278136e40 0x7fb278134ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-i con 0x7fb278136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb278137220 0x7fb278135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): picked mon.noname-f con 0x7fb278137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278123600 con 0x7fb278137220 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2781cb6f0 con 0x7fb278136e40 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb278136330 con 0x7fb278135dc0 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb27d27d640 10 monclient(hunting): authenticate will time out at 1328.480201s 2024-09-16T15:09:51.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33454/0 (socket says 172.21.15.134:33454) 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 -- 172.21.15.134:0/2105132808 learned_addr learned my addr 172.21.15.134:0/2105132808 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): get_auth_request con 0x7fb278135dc0 auth_method 0 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb278137220 0x7fb278135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): get_auth_request con 0x7fb278137220 auth_method 0 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb278136e40 0x7fb278134ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): get_auth_request con 0x7fb278136e40 auth_method 0 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb26e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient(hunting): handle_auth_done global_id 74159 payload 824 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb278136e40 msgr2=0x7fb278134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb278136e40 0x7fb278134ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb278137220 msgr2=0x7fb278135410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb278137220 0x7fb278135410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 monclient: found mon.noname-e 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.891+0000 7fb276ffd640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270009b30 con 0x7fb278135dc0 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.893+0000 7fb276ffd640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 msgr2=0x7fb2781cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.893+0000 7fb276ffd640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d800 tx=0x7fb26800dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb26800bd40 con 0x7fb278135dc0 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: dump: 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.911 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: _finish_auth 0 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268014430 con 0x7fb278135dc0 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 5 monclient: authenticate success, global_id 74159 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 10 monclient: _renew_subs 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb268014a10 con 0x7fb278135dc0 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:09:51.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb278125020 con 0x7fb278135dc0 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb275ffb640 4 set_mon_vals no callback set 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 10 monclient: dump: 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.913 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 10 monclient: _renew_subs 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb278125300 con 0x7fb278135dc0 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.894+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb268014bb0 con 0x7fb278135dc0 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb26ffff640 1 --2- 172.21.15.134:0/2105132808 >> v2:172.21.15.162:6800/980547263 conn(0x7fb258038880 0x7fb25803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb27d27d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(420..420 src has 1..420) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fb26804a180 con 0x7fb278135dc0 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb27d27d640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 0x7fb27802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb278071b20 con 0x7fb278122150 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb26e7fc640 1 --2- 172.21.15.134:0/2105132808 >> v2:172.21.15.162:6800/980547263 conn(0x7fb258038880 0x7fb25803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb26e7fc640 10 monclient: get_auth_request con 0x7fb258038880 auth_method 0 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 0x7fb27802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient: get_auth_request con 0x7fb278122150 auth_method 0 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 10 monclient(hunting): handle_auth_done global_id 44555 payload 824 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.895+0000 7fb2777fe640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb278061ae0 con 0x7fb278122150 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.897+0000 7fb26e7fc640 1 --2- 172.21.15.134:0/2105132808 >> v2:172.21.15.162:6800/980547263 conn(0x7fb258038880 0x7fb25803ad40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2781230f0 tx=0x7fb260008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.897+0000 7fb2777fe640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 msgr2=0x7fb27802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.897+0000 7fb2777fe640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 0x7fb27802b260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb270033000 tx=0x7fb270004a40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2700046e0 con 0x7fb278122150 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 10 monclient: dump: 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.916 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb270004840 con 0x7fb278122150 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 msgr2=0x7fb27802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 0x7fb27802b260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb270033000 tx=0x7fb270004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 msgr2=0x7fb27802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.898+0000 7fb26ffff640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122150 0x7fb27802b260 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb270033000 tx=0x7fb270004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb27d27d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb27d27d640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 0x7fb27802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb27d27d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb27807a680 con 0x7fb278122530 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 0x7fb27802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:51.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient: get_auth_request con 0x7fb278122530 auth_method 0 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 10 monclient(hunting): handle_auth_done global_id 44564 payload 824 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.904+0000 7fb2777fe640 1 -- 172.21.15.134:0/2105132808 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb27809b490 con 0x7fb278122530 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb2777fe640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 msgr2=0x7fb27802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb2777fe640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 0x7fb27802b7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb27802c920 tx=0x7fb27002f000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb270049580 con 0x7fb278122530 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 10 monclient: dump: 2024-09-16T15:09:51.920 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:51.921 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fb2700496e0 con 0x7fb278122530 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 msgr2=0x7fb27802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 0x7fb27802b7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb27802c920 tx=0x7fb27002f000 comp rx=0 tx=0).stop 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 msgr2=0x7fb27802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb26ffff640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb278122530 0x7fb27802b7a0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb27802c920 tx=0x7fb27002f000 comp rx=0 tx=0).stop 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:51.922 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:51.923 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.924 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.925 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:51.926 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:51.927 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.928 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.929 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.930 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:51.931 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.932 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:51.933 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.934 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:51.935 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.936 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.937 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.938 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:51.940 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:51.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:51.942 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:51.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:51.944 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:51.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 >> v2:172.21.15.162:6800/980547263 conn(0x7fb258038880 msgr2=0x7fb25803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 --2- 172.21.15.134:0/2105132808 >> v2:172.21.15.162:6800/980547263 conn(0x7fb258038880 0x7fb25803ad40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2781230f0 tx=0x7fb260008040 comp rx=0 tx=0).stop 2024-09-16T15:09:51.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 10 monclient: shutdown 2024-09-16T15:09:51.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 msgr2=0x7fb2781cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb2777fe640 1 -- 172.21.15.134:0/2105132808 reap_dead start 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d800 tx=0x7fb26800dcc0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 shutdown_connections 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 --2- 172.21.15.134:0/2105132808 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb278135dc0 0x7fb2781cb1b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.907+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 >> 172.21.15.134:0/2105132808 conn(0x7fb2780a6110 msgr2=0x7fb27809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.908+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 shutdown_connections 2024-09-16T15:09:51.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:51.908+0000 7fb27d27d640 1 -- 172.21.15.134:0/2105132808 wait complete. 2024-09-16T15:09:51.946 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:51.946 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:09:51.947 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.043+0000 7fb7fb23a640 10 monclient: get_monmap_and_config 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.043+0000 7fb7fb23a640 10 monclient: build_initial_monmap 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.043+0000 7fb7fb23a640 10 monclient: monmap: 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:52.044662+0000 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:52.044662+0000 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:52.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:52.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 1 Processor -- start 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 1 -- start start 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient: init 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient: _add_conns ranks=[0,9,6,2,8,10,7,5,4,1,3] 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb7f4136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 0x7fb7f4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-a con 0x7fb7f4137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 0x7fb7f413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-d con 0x7fb7f413c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f4073250 con 0x7fb7f4136e40 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.044+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7fb23a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40a56e0 con 0x7fb7f4137220 2024-09-16T15:09:52.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7fb23a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f409abc0 con 0x7fb7f413c370 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7fb23a640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7fb23a640 10 monclient(hunting): authenticate will time out at 1328.634016s 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 0x7fb7f4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 0x7fb7f413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f8fd6640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb7f4136e40 0x7fb7f409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50844/0 (socket says 172.21.15.134:50844) 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): get_auth_request con 0x7fb7f4137220 auth_method 0 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f8fd6640 1 -- 172.21.15.134:0/3459355262 learned_addr learned my addr 172.21.15.134:0/3459355262 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): get_auth_request con 0x7fb7f413c370 auth_method 0 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient(hunting): handle_auth_done global_id 34845 payload 274 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 -- 172.21.15.134:0/3459355262 >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 msgr2=0x7fb7f4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 --2- 172.21.15.134:0/3459355262 >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 0x7fb7f4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 -- 172.21.15.134:0/3459355262 >> v1:172.21.15.134:6789/0 conn(0x7fb7f4136e40 legacy=0x7fb7f409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient: _finish_hunting 0 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 monclient: found mon.noname-d 2024-09-16T15:09:52.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:52.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:52.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f97d7640 1 -- 172.21.15.134:0/3459355262 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f409ba50 con 0x7fb7f413c370 2024-09-16T15:09:52.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.045+0000 7fb7f3fff640 1 --2- 172.21.15.134:0/3459355262 >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 0x7fb7f4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:52.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f97d7640 1 -- 172.21.15.134:0/3459355262 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 msgr2=0x7fb7f413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f97d7640 1 --2- 172.21.15.134:0/3459355262 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 0x7fb7f413e760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009d90 tx=0x7fb7ec02a4e0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bbf82351cfcff9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 1 -- 172.21.15.134:0/3459355262 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7ec007bc0 con 0x7fb7f413c370 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: dump: 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.058 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 1 -- 172.21.15.134:0/3459355262 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec02d050 con 0x7fb7f413c370 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 5 monclient: authenticate success, global_id 34845 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:52.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 1 -- 172.21.15.134:0/3459355262 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7ec007d40 con 0x7fb7f413c370 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 4 set_mon_vals no callback set 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7f37fe640 10 monclient: dump: 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 10 monclient: shutdown 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 -- 172.21.15.134:0/3459355262 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 msgr2=0x7fb7f413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/3459355262 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 0x7fb7f413e760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009d90 tx=0x7fb7ec02a4e0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 -- 172.21.15.134:0/3459355262 shutdown_connections 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/3459355262 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb7f413c370 0x7fb7f413e760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/3459355262 >> v2:172.21.15.134:3300/0 conn(0x7fb7f4137220 0x7fb7f4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 -- 172.21.15.134:0/3459355262 >> 172.21.15.134:0/3459355262 conn(0x7fb7f40a6110 msgr2=0x7fb7f40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 -- 172.21.15.134:0/3459355262 shutdown_connections 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 -- 172.21.15.134:0/3459355262 wait complete. 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 10 monclient: build_initial_monmap 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 10 monclient: monmap: 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:52.062 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.048+0000 7fb7fb23a640 1 Processor -- start 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 -- start start 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient: init 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient: _add_conns ranks=[8,0,7,5,6,2,4,3,1] 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 0x7fb7f41cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-e con 0x7fb7f4136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:52.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 0x7fb7f41cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-a con 0x7fb7f4137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413c370 0x7fb7f4139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): picked mon.noname-i con 0x7fb7f413c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4136480 con 0x7fb7f4137220 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4121c80 con 0x7fb7f413c370 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4121ed0 con 0x7fb7f4136e40 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7fb23a640 10 monclient(hunting): authenticate will time out at 1328.639564s 2024-09-16T15:09:52.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 0x7fb7f41cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 0x7fb7f41cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 0x7fb7f41cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35652/0 (socket says 172.21.15.134:35652) 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 -- 172.21.15.134:0/1212660708 learned_addr learned my addr 172.21.15.134:0/1212660708 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): get_auth_request con 0x7fb7f4137220 auth_method 0 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 0x7fb7f41cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33468/0 (socket says 172.21.15.134:33468) 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): get_auth_request con 0x7fb7f4136e40 auth_method 0 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient(hunting): handle_auth_done global_id 74160 payload 824 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f97d7640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413c370 0x7fb7f4139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 msgr2=0x7fb7f41cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 0x7fb7f41cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413c370 msgr2=0x7fb7f4139d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413c370 0x7fb7f4139d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 1 monclient: found mon.noname-a 2024-09-16T15:09:52.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.049+0000 7fb7f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.050+0000 7fb7f3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.050+0000 7fb7f3fff640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009400 con 0x7fb7f4137220 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.050+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb7f4136e40 0x7fb7f41cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f3fff640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 msgr2=0x7fb7f41cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f3fff640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 0x7fb7f41cbd10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ec10 tx=0x7fb7e000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7e0009760 con 0x7fb7f4137220 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: dump: 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:52.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e000ce40 con 0x7fb7f4137220 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 5 monclient: authenticate success, global_id 74160 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 10 monclient: _renew_subs 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec009a70 con 0x7fb7f4137220 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7e0013d50 con 0x7fb7f4137220 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f17fa640 10 monclient: dump: 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.069 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7f2ffd640 4 set_mon_vals no callback set 2024-09-16T15:09:52.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 10 monclient: _renew_subs 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.052+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f40a78d0 con 0x7fb7f4137220 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7fb23a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 0x7fb7f413a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4071b20 con 0x7fb7f4122060 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb7e0013470 con 0x7fb7f4137220 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f17fa640 1 --2- 172.21.15.134:0/1212660708 >> v2:172.21.15.162:6800/980547263 conn(0x7fb7d80388b0 0x7fb7d803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 0x7fb7f413a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(420..420 src has 1..420) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fb7e004a520 con 0x7fb7f4137220 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient: get_auth_request con 0x7fb7f4122060 auth_method 0 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f97d7640 1 --2- 172.21.15.134:0/1212660708 >> v2:172.21.15.162:6800/980547263 conn(0x7fb7d80388b0 0x7fb7d803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.053+0000 7fb7f97d7640 10 monclient: get_auth_request con 0x7fb7d80388b0 auth_method 0 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.054+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_done global_id 54376 payload 824 2024-09-16T15:09:52.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.054+0000 7fb7f8fd6640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7f4061ae0 con 0x7fb7f4122060 2024-09-16T15:09:52.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f8fd6640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 msgr2=0x7fb7f413a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 0x7fb7f413a2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8007730 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f97d7640 1 --2- 172.21.15.134:0/1212660708 >> v2:172.21.15.162:6800/980547263 conn(0x7fb7d80388b0 0x7fb7d803ad70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4137f00 tx=0x7fb7ec02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7e800fd00 con 0x7fb7f4122060 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 10 monclient: dump: 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.075 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb7e800fe60 con 0x7fb7f4122060 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 msgr2=0x7fb7f413a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 0x7fb7f413a2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8007730 comp rx=0 tx=0).stop 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 msgr2=0x7fb7f413a2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.056+0000 7fb7f17fa640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f4122060 0x7fb7f413a2d0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8007730 comp rx=0 tx=0).stop 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.062+0000 7fb7fb23a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.062+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 0x7fb7f413ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.062+0000 7fb7fb23a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.062+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f407a680 con 0x7fb7f413a810 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 0x7fb7f413ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient: get_auth_request con 0x7fb7f413a810 auth_method 0 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 10 monclient(hunting): handle_auth_done global_id 54385 payload 824 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.063+0000 7fb7f8fd6640 1 -- 172.21.15.134:0/1212660708 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb7f40a53c0 con 0x7fb7f413a810 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.065+0000 7fb7f8fd6640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 msgr2=0x7fb7f413ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.065+0000 7fb7f8fd6640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 0x7fb7f413ac20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8004940 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb7e8023070 con 0x7fb7f413a810 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 10 monclient: dump: 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.078 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fb7e801e270 con 0x7fb7f413a810 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:52.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 msgr2=0x7fb7f413ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 0x7fb7f413ac20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8004940 comp rx=0 tx=0).stop 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 msgr2=0x7fb7f413ac20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f17fa640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb7f413a810 0x7fb7f413ac20 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007270 tx=0x7fb7e8004940 comp rx=0 tx=0).stop 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:52.080 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:52.081 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:52.082 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:52.083 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:52.084 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:52.085 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.087 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.088 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:52.090 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.092 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.093 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.094 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:52.095 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:52.096 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:52.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:52.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.101 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:52.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:52.103 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:52.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 >> v2:172.21.15.162:6800/980547263 conn(0x7fb7d80388b0 msgr2=0x7fb7d803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/1212660708 >> v2:172.21.15.162:6800/980547263 conn(0x7fb7d80388b0 0x7fb7d803ad70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4137f00 tx=0x7fb7ec02f000 comp rx=0 tx=0).stop 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 10 monclient: shutdown 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 msgr2=0x7fb7f41cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 --2- 172.21.15.134:0/1212660708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7f4137220 0x7fb7f41cbd10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ec10 tx=0x7fb7e000c520 comp rx=0 tx=0).stop 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f97d7640 1 -- 172.21.15.134:0/1212660708 reap_dead start 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7f97d7640 1 -- 172.21.15.134:0/1212660708 reap_dead start 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 shutdown_connections 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 >> 172.21.15.134:0/1212660708 conn(0x7fb7f40a6110 msgr2=0x7fb7f406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 shutdown_connections 2024-09-16T15:09:52.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.066+0000 7fb7fb23a640 1 -- 172.21.15.134:0/1212660708 wait complete. 2024-09-16T15:09:52.106 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:52.106 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:09:52.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:09:52.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.200+0000 7f60f6576640 10 monclient: get_monmap_and_config 2024-09-16T15:09:52.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.200+0000 7f60f6576640 10 monclient: build_initial_monmap 2024-09-16T15:09:52.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.200+0000 7f60f6576640 10 monclient: monmap: 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:52.202375+0000 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:52.202375+0000 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:52.203 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:52.204 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:52.204 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 Processor -- start 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 -- start start 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient: init 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient: _add_conns ranks=[8,9,1,0,4,2,5,7,3,10,6] 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-e con 0x7f60f0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:52.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 0x7f60f009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-a con 0x7f60f009bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f60f0136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f0073250 con 0x7f60f0136c60 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f009b600 con 0x7f60f009bfe0 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00a5740 con 0x7f60f0136030 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60f6576640 10 monclient(hunting): authenticate will time out at 1328.791747s 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 0x7f60f009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33482/0 (socket says 172.21.15.134:33482) 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 1 -- 172.21.15.134:0/3149681893 learned_addr learned my addr 172.21.15.134:0/3149681893 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:52.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): get_auth_request con 0x7f60f0136030 auth_method 0 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): get_auth_request con 0x7f60f009bfe0 auth_method 0 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.201+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 10 monclient(hunting): handle_auth_done global_id 74177 payload 274 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 1 -- 172.21.15.134:0/3149681893 >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 msgr2=0x7f60f009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 1 --2- 172.21.15.134:0/3149681893 >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 0x7f60f009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 1 -- 172.21.15.134:0/3149681893 >> v1:172.21.15.162:6789/0 conn(0x7f60f0136c60 legacy=0x7f60f0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 1 monclient: found mon.noname-e 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60effff640 1 -- 172.21.15.134:0/3149681893 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00a52c0 con 0x7f60f0136030 2024-09-16T15:09:52.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.202+0000 7f60ef7fe640 1 --2- 172.21.15.134:0/3149681893 >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 0x7f60f009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:52.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.204+0000 7f60effff640 1 -- 172.21.15.134:0/3149681893 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 msgr2=0x7f60f0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.204+0000 7f60effff640 1 --2- 172.21.15.134:0/3149681893 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009870 tx=0x7f60e002a100 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b1b5260c10d7e6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 1 -- 172.21.15.134:0/3149681893 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60e002ae20 con 0x7f60f0136030 2024-09-16T15:09:52.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: dump: 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.215 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 5 monclient: authenticate success, global_id 74177 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 1 -- 172.21.15.134:0/3149681893 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e00307b0 con 0x7f60f0136030 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:52.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 4 set_mon_vals no callback set 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 1 -- 172.21.15.134:0/3149681893 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60e0030d90 con 0x7f60f0136030 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60eeffd640 10 monclient: dump: 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.217 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 10 monclient: shutdown 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 -- 172.21.15.134:0/3149681893 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 msgr2=0x7f60f0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 --2- 172.21.15.134:0/3149681893 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009870 tx=0x7f60e002a100 comp rx=0 tx=0).stop 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 -- 172.21.15.134:0/3149681893 shutdown_connections 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 --2- 172.21.15.134:0/3149681893 >> v2:172.21.15.134:3300/0 conn(0x7f60f009bfe0 0x7f60f009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 --2- 172.21.15.134:0/3149681893 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f60f0136030 0x7f60f0136410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 -- 172.21.15.134:0/3149681893 >> 172.21.15.134:0/3149681893 conn(0x7f60f00a9ea0 msgr2=0x7f60f00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 -- 172.21.15.134:0/3149681893 shutdown_connections 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.205+0000 7f60f6576640 1 -- 172.21.15.134:0/3149681893 wait complete. 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.206+0000 7f60f6576640 10 monclient: build_initial_monmap 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.206+0000 7f60f6576640 10 monclient: monmap: 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:52.219 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.206+0000 7f60f6576640 1 Processor -- start 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.206+0000 7f60f6576640 1 -- start start 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.206+0000 7f60f6576640 10 monclient: init 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient: _add_conns ranks=[0,2,5,8,1,7,4,3,6] 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-a con 0x7f60f009bfe0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:52.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60f0136030 0x7f60f01354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-b con 0x7f60f0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 0x7f60f01cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): picked mon.noname-h con 0x7f60f0136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0120fe0 con 0x7f60f009bfe0 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f01226d0 con 0x7f60f0136030 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0120a20 con 0x7f60f0136c60 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f6576640 10 monclient(hunting): authenticate will time out at 1328.797279s 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60f0136030 0x7f60f01354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35668/0 (socket says 172.21.15.134:35668) 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 -- 172.21.15.134:0/3231669343 learned_addr learned my addr 172.21.15.134:0/3231669343 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): get_auth_request con 0x7f60f009bfe0 auth_method 0 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): get_auth_request con 0x7f60f0136030 auth_method 0 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient(hunting): handle_auth_done global_id 74178 payload 824 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f4b13640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 0x7f60f01cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 -- 172.21.15.134:0/3231669343 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 msgr2=0x7f60f01cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 0x7f60f01cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 -- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60f0136030 msgr2=0x7f60f01354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60f0136030 0x7f60f01354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 monclient: found mon.noname-a 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60effff640 1 -- 172.21.15.134:0/3231669343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4009400 con 0x7f60f009bfe0 2024-09-16T15:09:52.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f4b13640 10 monclient: get_auth_request con 0x7f60f0136c60 auth_method 0 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.207+0000 7f60f4b13640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 0x7f60f01cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.209+0000 7f60effff640 1 -- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 msgr2=0x7f60f0134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.209+0000 7f60effff640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60e00090b0 tx=0x7f60e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60e0030d40 con 0x7f60f009bfe0 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: dump: 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.224 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: _finish_auth 0 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: _check_auth_tickets 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 5 monclient: authenticate success, global_id 74178 2024-09-16T15:09:52.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e0003a80 con 0x7f60f009bfe0 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _renew_subs 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e00094d0 con 0x7f60f009bfe0 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f60e0003bf0 con 0x7f60f009bfe0 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 10 monclient: dump: 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:52.226 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _renew_subs 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f00a7dc0 con 0x7f60f009bfe0 2024-09-16T15:09:52.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ee7fc640 4 set_mon_vals no callback set 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f60e0039070 con 0x7f60f009bfe0 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.210+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f009f920 con 0x7f60f009bfe0 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.211+0000 7f60ecff9640 1 --2- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 0x7f60dc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.211+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(420..420 src has 1..420) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f60e006aa30 con 0x7f60f009bfe0 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.211+0000 7f60ef7fe640 1 --2- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 0x7f60dc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:52.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.211+0000 7f60ef7fe640 10 monclient: get_auth_request con 0x7f60dc038880 auth_method 0 2024-09-16T15:09:52.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.213+0000 7f60ef7fe640 1 --2- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 0x7f60dc03ad40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60f01bf810 tx=0x7f60e4009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:52.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.215+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f60e0003610 con 0x7f60f009bfe0 2024-09-16T15:09:52.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.215+0000 7f60ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:52.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.215+0000 7f60ecff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.441+0000 7f60f6576640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:09:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.441+0000 7f60f6576640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:09:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.441+0000 7f60f6576640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.441+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f60f009f920 con 0x7f60f009bfe0 2024-09-16T15:09:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.442+0000 7f60ecff9640 1 -- 172.21.15.134:0/3231669343 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f60e0037030 con 0x7f60f009bfe0 2024-09-16T15:09:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.442+0000 7f60ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:09:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.442+0000 7f60ecff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=11,kv=11,logm=61} crc {auth=4294515896,config=2586702516,health=3172947059,kv=2676201184,logm=3199137911}) 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=14,config=2,health=12,kv=11,logm=61} crc {auth=2175534627,config=2586702516,health=1417644965,kv=1090497770,logm=1195189129}) 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=14,config=2,health=10,kv=12,logm=62} crc {auth=1920106766,config=2586702516,health=2554269129,kv=1090497770,logm=3018658718}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=15,config=2,health=11,kv=11,logm=61} crc {auth=4294515896,config=2586702516,health=532023007,kv=174400417,logm=1060684309}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=13,config=2,health=11,kv=11,logm=63} crc {auth=2658688966,config=2586702516,health=3874731711,kv=318701324,logm=4030996468}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=14,config=2,health=12,kv=12,logm=60} crc {auth=152625972,config=2586702516,health=1417644965,kv=1090497770,logm=1324489898}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=14,config=2,health=11,kv=11,logm=62} crc {auth=1920106766,config=2586702516,health=674056611,kv=3418666009,logm=2856582675}) 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=13,config=2,health=12,kv=12,logm=61} crc {auth=1478552012,config=2586702516,health=1417644965,kv=1090497770,logm=526236041}) 2024-09-16T15:09:52.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.443+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=14,config=2,health=12,kv=12,logm=60} crc {auth=152625972,config=2586702516,health=1417644965,kv=1090497770,logm=3264449728}) 2024-09-16T15:09:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 msgr2=0x7f60dc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 0x7f60dc03ad40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f60f01bf810 tx=0x7f60e4009190 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 10 monclient: shutdown 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 msgr2=0x7f60f0134f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60e00090b0 tx=0x7f60e00046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 shutdown_connections 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> v2:172.21.15.162:6800/980547263 conn(0x7f60dc038880 0x7f60dc03ad40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f60f0136c60 0x7f60f01cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60f0136030 0x7f60f01354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 --2- 172.21.15.134:0/3231669343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60f009bfe0 0x7f60f0134f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 >> 172.21.15.134:0/3231669343 conn(0x7f60f00a9ea0 msgr2=0x7f60f006bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 shutdown_connections 2024-09-16T15:09:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:52.445+0000 7f60f6576640 1 -- 172.21.15.134:0/3231669343 wait complete. 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1218634226}) 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3053708241}) 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2710298815}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1837355028}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=128315896}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4221892231}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=864041216}) 2024-09-16T15:09:52.461 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1281720581}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.451+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032075834}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2337453287}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=559579672}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1751280957}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.462 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2755207405}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=157095239}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2387710096}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3308101121}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2805470869}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.452+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=165394898}) 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.463 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=738304794}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2386282703}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2188158005}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=297059384}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1972377654}) 2024-09-16T15:09:52.464 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3453680026}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=953793908}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1631575174}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.454+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2425427446}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=880471609}) 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.465 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4127092747}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2780006099}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3093014694}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4208167371}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.466 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1015945676}) 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1704185131}) 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3403843438}) 2024-09-16T15:09:52.467 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3014766226}) 2024-09-16T15:09:52.468 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:09:52.474 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.474 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2665833686}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3340094187}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=926713480}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2379157260}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3838035755}) 2024-09-16T15:09:52.475 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2373325936}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=133673641}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=985735392}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.457+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3236473771}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=7085480}) 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.476 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=262898274}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=554514158}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3548552577}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4002051682}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3080731834}) 2024-09-16T15:09:52.477 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=629021792}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1683464946}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.458+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2524005156}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1883230362}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3768748929}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2331603756}) 2024-09-16T15:09:52.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2798697129}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1866031713}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3427714980}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4037718696}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=49781157}) 2024-09-16T15:09:52.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.460+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1331852443}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3437304909}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=466875025}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3588023328}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2311421135}) 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1761645251}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=596739004}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1247968800}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900863807}) 2024-09-16T15:09:52.481 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.461+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=601312735}) 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622966421}) 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.547 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1240394805}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1967163252}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=257890689}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=560884538}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2167185397}) 2024-09-16T15:09:52.548 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801812579}) 2024-09-16T15:09:52.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705333284}) 2024-09-16T15:09:52.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.544+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=154244080}) 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=926354640}) 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.553 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=579113888}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=64289653}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1403249760}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1988881335}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2093098303}) 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.554 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1878454850}) 2024-09-16T15:09:52.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093984131}) 2024-09-16T15:09:52.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.550+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4194896619}) 2024-09-16T15:09:52.569 INFO:tasks.rados.rados.0.smithi162.stdout:2123: done (3 left) 2024-09-16T15:09:52.569 INFO:tasks.rados.rados.0.smithi162.stdout:2124: write oid 23 current snap is 198 2024-09-16T15:09:52.569 INFO:tasks.rados.rados.0.smithi162.stdout:2124: seq_num 761 ranges {790127=499789,1698444=730935} 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2443750402}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=99334391}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1011058106}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3664651201}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2241939342}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1339518065}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3666339449}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3391961951}) 2024-09-16T15:09:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.570+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2160185541}) 2024-09-16T15:09:52.577 INFO:tasks.rados.rados.0.smithi162.stdout:2124: writing smithi16234619-23 from 790127 to 1289916 tid 1 2024-09-16T15:09:52.583 INFO:tasks.rados.rados.0.smithi162.stdout:2124: writing smithi16234619-23 from 1698444 to 2429379 tid 2 2024-09-16T15:09:52.583 INFO:tasks.rados.rados.0.smithi162.stdout:2125: write oid 16 current snap is 198 2024-09-16T15:09:52.583 INFO:tasks.rados.rados.0.smithi162.stdout:2125: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2024-09-16T15:09:52.589 INFO:tasks.rados.rados.0.smithi162.stdout:2125: writing smithi16234619-16 from 406934 to 981972 tid 1 2024-09-16T15:09:52.595 INFO:tasks.rados.rados.0.smithi162.stdout:2125: writing smithi16234619-16 from 1640573 to 2115166 tid 2 2024-09-16T15:09:52.599 INFO:tasks.rados.rados.0.smithi162.stdout:2125: writing smithi16234619-16 from 2642791 to 2806934 tid 3 2024-09-16T15:09:52.599 INFO:tasks.rados.rados.0.smithi162.stdout:2126: write (excl) oid 34 current snap is 198 2024-09-16T15:09:52.599 INFO:tasks.rados.rados.0.smithi162.stdout:2126: seq_num 763 ranges {423741=650287,1699055=618251} 2024-09-16T15:09:52.605 INFO:tasks.rados.rados.0.smithi162.stdout:2126: writing smithi16234619-34 from 423741 to 1074028 tid 1 2024-09-16T15:09:52.612 INFO:tasks.rados.rados.0.smithi162.stdout:2126: writing smithi16234619-34 from 1699055 to 2317306 tid 2 2024-09-16T15:09:52.612 INFO:tasks.rados.rados.0.smithi162.stdout:2127: read oid 9 snap -1 2024-09-16T15:09:52.612 INFO:tasks.rados.rados.0.smithi162.stdout:2127: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:09:52.612 INFO:tasks.rados.rados.0.smithi162.stdout:2128: write (excl) oid 25 current snap is 198 2024-09-16T15:09:52.612 INFO:tasks.rados.rados.0.smithi162.stdout:2128: seq_num 764 ranges {440548=725536,1757537=478262,2813990=26558} 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3431196672}) 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3055945060}) 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3882749224}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2297665734}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2961574182}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=454391130}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3929372656}) 2024-09-16T15:09:52.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326953090}) 2024-09-16T15:09:52.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3665080235}) 2024-09-16T15:09:52.619 INFO:tasks.rados.rados.0.smithi162.stdout:2128: writing smithi16234619-25 from 440548 to 1166084 tid 1 2024-09-16T15:09:52.625 INFO:tasks.rados.rados.0.smithi162.stdout:2128: writing smithi16234619-25 from 1757537 to 2235799 tid 2 2024-09-16T15:09:52.628 INFO:tasks.rados.rados.0.smithi162.stdout:2128: writing smithi16234619-25 from 2813990 to 2840548 tid 3 2024-09-16T15:09:52.628 INFO:tasks.rados.rados.0.smithi162.stdout:2129: write oid 11 current snap is 198 2024-09-16T15:09:52.629 INFO:tasks.rados.rados.0.smithi162.stdout:2129: seq_num 765 ranges {457355=400785,1416019=621920,2725060=132295} 2024-09-16T15:09:52.633 INFO:tasks.rados.rados.0.smithi162.stdout:2129: writing smithi16234619-11 from 457355 to 858140 tid 1 2024-09-16T15:09:52.640 INFO:tasks.rados.rados.0.smithi162.stdout:2129: writing smithi16234619-11 from 1416019 to 2037939 tid 2 2024-09-16T15:09:52.644 INFO:tasks.rados.rados.0.smithi162.stdout:2129: writing smithi16234619-11 from 2725060 to 2857355 tid 3 2024-09-16T15:09:52.644 INFO:tasks.rados.rados.0.smithi162.stdout:2130: snap_remove snap 194 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933469509}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3456980560}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=564035382}) 2024-09-16T15:09:52.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2301518298}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3572559084}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2621521987}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2156668969}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2668686252}) 2024-09-16T15:09:52.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.725+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1970183255}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2612531164}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=107500618}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3682001708}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2903961857}) 2024-09-16T15:09:52.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1055902538}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2638103705}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2536712258}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2186492285}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.726+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3882181936}) 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.727+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.727+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.727+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2328394278}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.727+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.727+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1087946707}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1680541257}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1583703644}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2710553075}) 2024-09-16T15:09:52.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1215905407}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2213736849}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1603047628}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.728+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3988738931}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3087320581}) 2024-09-16T15:09:52.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3487313166}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=564672698}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3737982502}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=554422462}) 2024-09-16T15:09:52.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255347962}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=233735091}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2126888708}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.729+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=340804721}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3035483771}) 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=917522566}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2869488190}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=990519913}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1652061236}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1678910541}) 2024-09-16T15:09:52.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1293141236}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1625287805}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.730+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=210229163}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1741456590}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4246931330}) 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3335182861}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=315358940}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3030688069}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=495941974}) 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=190887982}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=903679415}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.732+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=59045985}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=788461934}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4251824531}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3631770474}) 2024-09-16T15:09:52.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=55902314}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1995951419}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2014865424}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3440540901}) 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1184316988}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.733+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2936723140}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2200529667}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3003547611}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4258929246}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.734+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=708485162}) 2024-09-16T15:09:52.771 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2703005607}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2569036436}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2049555669}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1537926621}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.735+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1395531993}) 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.772 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3335287405}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2887254115}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3123132451}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1077320592}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.773 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2114469350}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3108239082}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4177997398}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2913401384}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.736+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=576530065}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1568348375}) 2024-09-16T15:09:52.774 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2200596099}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3972385507}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1866494472}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2926523347}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.775 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=708424335}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=843613750}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1758230300}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=998842159}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986945137}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1505778957}) 2024-09-16T15:09:52.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3682633743}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3164263356}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2838452811}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4086460000}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2909013979}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1586671008}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.739+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3858667995}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3981485044}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3574547596}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2458603347}) 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3446091748}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=206832406}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=656670511}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.740+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3438841145}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.741+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.741+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1421164032}) 2024-09-16T15:09:52.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.741+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423619752}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1845796190}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1750083904}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2621786393}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2737430447}) 2024-09-16T15:09:52.780 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1059494513}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=152148980}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3569534098}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=111375609}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.742+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3988526212}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.781 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2581661742}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3133656917}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2694101909}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3798210634}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3079587760}) 2024-09-16T15:09:52.782 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=907364233}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2221033309}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.743+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.744+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2570670559}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.744+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4221883143}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3673284093}) 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.783 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1907919160}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=450163954}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1277082787}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2120539010}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.784 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2346571102}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=463583150}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420043410}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.745+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1743909255}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2306581316}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=202383668}) 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.785 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=189435574}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=382197214}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1530049201}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053030963}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.786 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2475346715}) 2024-09-16T15:09:52.787 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:52.787 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.746+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199322100}) 2024-09-16T15:09:52.787 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:52.747+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3827322184}) 2024-09-16T15:09:53.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3780844152}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2739800813}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4124677808}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3595144238}) 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985743362}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2523676006}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=137301926}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674408691}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.066+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4038894689}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3764902875}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3587882007}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3979756146}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1894195341}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=891829459}) 2024-09-16T15:09:53.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1445390168}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=162169455}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2317742732}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.067+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3885902904}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4165182110}) 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.077 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3284424745}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1128058625}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1380203091}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4018594817}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829769270}) 2024-09-16T15:09:53.078 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2365784160}) 2024-09-16T15:09:53.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1691541974}) 2024-09-16T15:09:53.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.068+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3236608263}) 2024-09-16T15:09:53.468 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:09:53.469 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:53.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=38,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=9} crc {logm=2458352146,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=773795346,mon_config_key=3888147765,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1522844004}) 2024-09-16T15:09:53.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=37,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=16,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=10} crc {logm=2334454170,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3091676096,mon_config_key=779547732,monmap=2838206317,osd_metadata=2856640748,osd_pg_creating=3341260306,osd_snap=4277117603}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=37,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=14,mon_config_key=13,monmap=2,osd_metadata=6,osd_snap=17} crc {logm=4126509773,mdsmap=1531151339,mgr=2385455004,mgr_metadata=3299114077,mgrstat=2876429618,mon_config_key=868770175,monmap=2200900527,osd_metadata=3154723898,osd_snap=3980978118}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=38,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=17,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=8} crc {logm=126936450,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3386236285,mon_config_key=1492070130,monmap=2838206317,osd_metadata=439904794,osd_pg_creating=3341260306,osd_snap=4239014194}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=11} crc {logm=2721692026,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=773795346,mon_config_key=1419983274,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=388207967}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=37,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=16,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=9} crc {logm=3720488122,mdsmap=1531151339,mgr=1248521450,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=716357236,mon_config_key=610347578,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1522844004}) 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.562+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.563+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.563+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=16,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=11} crc {logm=110563680,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1757380723,mon_config_key=779547732,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1097293857}) 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.563+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.563+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=36,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=1716057636,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4069345449,mon_config_key=3356710136,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=1031036907}) 2024-09-16T15:09:53.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.563+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=36,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=2678127731,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2680183338,mon_config_key=351218633,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=3341260306,osd_snap=538480441}) 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.568+0000 7fcffe061640 10 monclient: get_monmap_and_config 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.568+0000 7fcffe061640 10 monclient: build_initial_monmap 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.569+0000 7fcffe061640 10 monclient: monmap: 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:53.570942+0000 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:53.570942+0000 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:53.572 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.569+0000 7fcffe061640 1 Processor -- start 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.569+0000 7fcffe061640 1 -- start start 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.569+0000 7fcffe061640 10 monclient: init 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient: _add_conns ranks=[6,8,0,2,5,9,3,1,4,10,7] 2024-09-16T15:09:53.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 0x7fcff80a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-d con 0x7fcff809fda0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 0x7fcff8069e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-e con 0x7fcff809c630 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcff8135060 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff8073250 con 0x7fcff8135060 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcff809d2e0 con 0x7fcff809fda0 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8135a80 con 0x7fcff809c630 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcffe061640 10 monclient(hunting): authenticate will time out at 1330.160392s 2024-09-16T15:09:53.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff7fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcff8135060 0x7fcff806a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50860/0 (socket says 172.21.15.134:50860) 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff7fff640 1 -- 172.21.15.134:0/4077081022 learned_addr learned my addr 172.21.15.134:0/4077081022 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 0x7fcff8069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): get_auth_request con 0x7fcff809c630 auth_method 0 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 0x7fcff80a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): get_auth_request con 0x7fcff809fda0 auth_method 0 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient(hunting): handle_auth_done global_id 34854 payload 274 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 1 -- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 msgr2=0x7fcff8069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 0x7fcff8069e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 1 -- 172.21.15.134:0/4077081022 >> v1:172.21.15.134:6789/0 conn(0x7fcff8135060 legacy=0x7fcff806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.570+0000 7fcff77fe640 1 monclient: found mon.noname-d 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.571+0000 7fcff77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.571+0000 7fcff77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.571+0000 7fcff77fe640 1 -- 172.21.15.134:0/4077081022 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff809d8c0 con 0x7fcff809fda0 2024-09-16T15:09:53.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.571+0000 7fcff6ffd640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 0x7fcff8069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.574+0000 7fcff77fe640 1 -- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 msgr2=0x7fcff80a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.574+0000 7fcff77fe640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 0x7fcff80a0180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80097d0 tx=0x7fcfe802e410 comp rx=0 tx=0).ready entity=mon.6 client_cookie=61f8330e53e0573 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.578+0000 7fcff67fc640 1 -- 172.21.15.134:0/4077081022 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfe8033b90 con 0x7fcff809fda0 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.578+0000 7fcff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: dump: 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.582 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.583 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: _finish_auth 0 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 1 -- 172.21.15.134:0/4077081022 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8031050 con 0x7fcff809fda0 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 1 -- 172.21.15.134:0/4077081022 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfe8033d10 con 0x7fcff809fda0 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcff67fc640 10 monclient: dump: 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.585 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcffe061640 5 monclient: authenticate success, global_id 34854 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcffe061640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:53.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.579+0000 7fcffe061640 4 set_mon_vals no callback set 2024-09-16T15:09:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.580+0000 7fcffe061640 10 monclient: shutdown 2024-09-16T15:09:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.580+0000 7fcffe061640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.580+0000 7fcffe061640 1 -- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 msgr2=0x7fcff80a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.580+0000 7fcffe061640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 0x7fcff80a0180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80097d0 tx=0x7fcfe802e410 comp rx=0 tx=0).stop 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 -- 172.21.15.134:0/4077081022 shutdown_connections 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fcff809c630 0x7fcff8069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 --2- 172.21.15.134:0/4077081022 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcff809fda0 0x7fcff80a0180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 -- 172.21.15.134:0/4077081022 >> 172.21.15.134:0/4077081022 conn(0x7fcff80a2e20 msgr2=0x7fcff80a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 -- 172.21.15.134:0/4077081022 shutdown_connections 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.581+0000 7fcffe061640 1 -- 172.21.15.134:0/4077081022 wait complete. 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.582+0000 7fcffe061640 10 monclient: build_initial_monmap 2024-09-16T15:09:53.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.582+0000 7fcffe061640 10 monclient: monmap: 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:53.589 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:53.590 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:53.590 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=71,osdmap=29} crc {osd_snap=1229632706,osdmap=2376485251}) 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=74,osdmap=26} crc {osd_snap=2026186081,osdmap=2049437603}) 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.592 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=75,osdmap=25} crc {osd_snap=2994461750,osdmap=114132500}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=3304908900,osdmap=3586152920}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=3679366884,osdmap=209300068}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=70,osdmap=30} crc {osd_snap=2483804703,osdmap=3267995926}) 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.593 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.582+0000 7fcffe061640 1 Processor -- start 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2130: done (9 left) 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2131: copy_from oid 35 from oid 28 current snap is 198 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2121: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2121: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2121: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 928 (ObjNum 694 snap 180 seq_num 694) dirty exists 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2122: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2122: finishing copy_from to smithi16234619-24 2024-09-16T15:09:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 776 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:09:53.595 INFO:tasks.rados.rados.0.smithi162.stdout:2124: finishing write tid 1 to smithi16234619-23 2024-09-16T15:09:53.595 INFO:tasks.rados.rados.0.smithi162.stdout:2124: finishing write tid 2 to smithi16234619-23 2024-09-16T15:09:53.595 INFO:tasks.rados.rados.0.smithi162.stdout:2127: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:09:53.595 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=69,osdmap=31} crc {osd_snap=4214558118,osdmap=2193934329}) 2024-09-16T15:09:53.595 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.595 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=4066115569,osdmap=3390658525}) 2024-09-16T15:09:53.595 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.582+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=71,osdmap=29} crc {osd_snap=2519771369,osdmap=675730386}) 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 1 -- start start 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 10 monclient: init 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 10 monclient: _add_conns ranks=[7,2,0,5,6,1,3,8,4] 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-i con 0x7fcff809c630 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcffe061640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 0x7fcff812b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcff6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 0x7fcff812b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcff77fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcff77fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:41996/0 (socket says 172.21.15.134:41996) 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.583+0000 7fcff6ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 0x7fcff812b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47984/0 (socket says 172.21.15.134:47984) 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-b con 0x7fcff809fda0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 -- 172.21.15.134:0/2001254636 learned_addr learned my addr 172.21.15.134:0/2001254636 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcff8135060 0x7fcff812b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): picked mon.noname-a con 0x7fcff8135060 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80a1a60 con 0x7fcff8135060 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff812e0a0 con 0x7fcff809fda0 2024-09-16T15:09:53.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcff812e380 con 0x7fcff809c630 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcffe061640 10 monclient(hunting): authenticate will time out at 1330.174069s 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): get_auth_request con 0x7fcff809c630 auth_method 0 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): get_auth_request con 0x7fcff809fda0 auth_method 0 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcff8135060 0x7fcff812b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): get_auth_request con 0x7fcff8135060 auth_method 0 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient(hunting): handle_auth_done global_id 54394 payload 824 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 -- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 msgr2=0x7fcff812b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 0x7fcff812b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 -- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcff8135060 msgr2=0x7fcff812b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcff8135060 0x7fcff812b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 monclient: found mon.noname-i 2024-09-16T15:09:53.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:53.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:09:53.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.584+0000 7fcff77fe640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009400 con 0x7fcff809c630 2024-09-16T15:09:53.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.586+0000 7fcff77fe640 1 -- 172.21.15.134:0/2001254636 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 msgr2=0x7fcff8133e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.586+0000 7fcff77fe640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80096f0 tx=0x7fcfe80046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfe8033e10 con 0x7fcff809c630 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: dump: 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.608 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: _finish_auth 0 2024-09-16T15:09:53.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8003610 con 0x7fcff809c630 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcfdffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.588+0000 7fcff5ffb640 4 set_mon_vals no callback set 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.589+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcfe8003bf0 con 0x7fcff809c630 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.589+0000 7fcffe061640 5 monclient: authenticate success, global_id 54394 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.589+0000 7fcffe061640 10 monclient: _renew_subs 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.589+0000 7fcffe061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:09:53.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.589+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe8009400 con 0x7fcff809c630 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcfdffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcfdffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcfdffff640 10 monclient: dump: 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.611 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.612 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.612 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.612 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.612 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.612 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.612 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcffe061640 10 monclient: _renew_subs 2024-09-16T15:09:53.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcffe061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:09:53.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.590+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff812c9e0 con 0x7fcff809c630 2024-09-16T15:09:53.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1620463426}) 2024-09-16T15:09:53.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3183952284}) 2024-09-16T15:09:53.614 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2678256604}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=212222181}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2540325803}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.615 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2953759470}) 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=696134110}) 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3929864954}) 2024-09-16T15:09:53.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2398302220}) 2024-09-16T15:09:53.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.591+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcfe803c070 con 0x7fcff809c630 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.591+0000 7fcfdffff640 1 --2- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 0x7fcfe403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.591+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(422..422 src has 1..422) v4 ==== 4007+0+0 (secure 0 0 0) 0x7fcfe8072f70 con 0x7fcff809c630 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.593+0000 7fcffe061640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.593+0000 7fcffe061640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.593+0000 7fcffe061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.593+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80679b0 con 0x7fcff809c630 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.594+0000 7fcff6ffd640 1 --2- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 0x7fcfe403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.594+0000 7fcff6ffd640 10 monclient: get_auth_request con 0x7fcfe4038810 auth_method 0 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.599+0000 7fcff6ffd640 1 --2- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 0x7fcfe403acd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcff8071560 tx=0x7fcfec009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.602+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcfe803a070 con 0x7fcff809c630 2024-09-16T15:09:53.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.602+0000 7fcfdffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:53.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.602+0000 7fcfdffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:53.708 INFO:tasks.rados.rados.0.smithi162.stdout:2124: finishing write tid 3 to smithi16234619-23 2024-09-16T15:09:53.708 INFO:tasks.rados.rados.0.smithi162.stdout:2124: finishing write tid 4 to smithi16234619-23 2024-09-16T15:09:53.708 INFO:tasks.rados.rados.0.smithi162.stdout:2124: oid 23 updating version 0 to 587 2024-09-16T15:09:53.708 INFO:tasks.rados.rados.0.smithi162.stdout:2124: oid 23 updating version 587 to 589 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2124: oid 23 version 589 is already newer than 588 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 589 (ObjNum 761 snap 198 seq_num 761) dirty exists 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2124: left oid 23 (ObjNum 761 snap 198 seq_num 761) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2125: finishing write tid 1 to smithi16234619-16 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2125: finishing write tid 2 to smithi16234619-16 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2120: done (9 left) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2121: done (8 left) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2122: done (7 left) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2124: done (6 left) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2127: done (5 left) 2024-09-16T15:09:53.709 INFO:tasks.rados.rados.0.smithi162.stdout:2132: rollback oid 4 current snap is 198 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 195 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: finishing write tid 3 to smithi16234619-16 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: finishing write tid 4 to smithi16234619-16 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: finishing write tid 5 to smithi16234619-16 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: oid 16 updating version 0 to 889 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: oid 16 updating version 889 to 892 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: oid 16 version 892 is already newer than 890 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: oid 16 version 892 is already newer than 891 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 892 (ObjNum 762 snap 198 seq_num 762) dirty exists 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2125: left oid 16 (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:09:53.710 INFO:tasks.rados.rados.0.smithi162.stdout:2129: finishing write tid 1 to smithi16234619-11 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: finishing write tid 2 to smithi16234619-11 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: finishing write tid 3 to smithi16234619-11 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: finishing write tid 4 to smithi16234619-11 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: finishing write tid 5 to smithi16234619-11 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: oid 11 updating version 0 to 163 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: oid 11 updating version 163 to 165 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: oid 11 version 165 is already newer than 164 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: oid 11 updating version 165 to 166 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2129: left oid 11 (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2126: finishing write tid 1 to smithi16234619-34 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2126: finishing write tid 2 to smithi16234619-34 2024-09-16T15:09:53.711 INFO:tasks.rados.rados.0.smithi162.stdout:2126: finishing write tid 3 to smithi16234619-34 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2126: finishing write tid 4 to smithi16234619-34 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2126: oid 34 updating version 0 to 823 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2126: oid 34 version 823 is already newer than 822 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2126: oid 34 updating version 823 to 824 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 824 (ObjNum 763 snap 198 seq_num 763) dirty exists 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2126: left oid 34 (ObjNum 763 snap 198 seq_num 763) 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: finishing write tid 1 to smithi16234619-25 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: finishing write tid 2 to smithi16234619-25 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: finishing write tid 3 to smithi16234619-25 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: finishing write tid 4 to smithi16234619-25 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: finishing write tid 5 to smithi16234619-25 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: oid 25 updating version 0 to 829 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: oid 25 version 829 is already newer than 828 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: oid 25 version 829 is already newer than 826 2024-09-16T15:09:53.712 INFO:tasks.rados.rados.0.smithi162.stdout:2128: oid 25 version 829 is already newer than 827 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 829 (ObjNum 764 snap 198 seq_num 764) dirty exists 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2128: left oid 25 (ObjNum 764 snap 198 seq_num 764) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2131: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2131: finishing copy_from to smithi16234619-35 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 798 (ObjNum 660 snap 171 seq_num 660) dirty exists 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2125: done (5 left) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2126: done (4 left) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2128: done (3 left) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2129: done (2 left) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2131: done (1 left) 2024-09-16T15:09:53.713 INFO:tasks.rados.rados.0.smithi162.stdout:2133: snap_create 2024-09-16T15:09:53.714 INFO:tasks.rados.rados.0.smithi162.stdout:2132: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:09:53.714 INFO:tasks.rados.rados.0.smithi162.stdout:2132: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:09:53.714 INFO:tasks.rados.rados.0.smithi162.stdout:2132: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:09:53.714 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 884 (ObjNum 717 snap 190 seq_num 717) dirty exists 2024-09-16T15:09:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.829+0000 7fcffe061640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.829+0000 7fcffe061640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.829+0000 7fcffe061640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:09:53.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.829+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcff80a3c40 con 0x7fcff809c630 2024-09-16T15:09:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.830+0000 7fcfdffff640 1 -- 172.21.15.134:0/2001254636 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7fcfe805dc90 con 0x7fcff809c630 2024-09-16T15:09:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.830+0000 7fcfdffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:53.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.830+0000 7fcfdffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:53.833 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 msgr2=0x7fcfe403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 0x7fcfe403acd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcff8071560 tx=0x7fcfec009190 comp rx=0 tx=0).stop 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 10 monclient: shutdown 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 msgr2=0x7fcff8133e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80096f0 tx=0x7fcfe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 shutdown_connections 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> v2:172.21.15.162:6800/980547263 conn(0x7fcfe4038810 0x7fcfe403acd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcff8135060 0x7fcff812b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcff809fda0 0x7fcff812b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 --2- 172.21.15.134:0/2001254636 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fcff809c630 0x7fcff8133e20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 >> 172.21.15.134:0/2001254636 conn(0x7fcff80a2e20 msgr2=0x7fcff80995a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 shutdown_connections 2024-09-16T15:09:53.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.832+0000 7fcffe061640 1 -- 172.21.15.134:0/2001254636 wait complete. 2024-09-16T15:09:53.845 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:09:53.845 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:09:53.845 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:09:53.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1776532212}) 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.850 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2342700918}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1530404526}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1861445122}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3871738385}) 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.851 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1297523222}) 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=510173782}) 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=367595033}) 2024-09-16T15:09:53.852 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.846+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1769176251}) 2024-09-16T15:09:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.941+0000 7ff54a74c640 10 monclient: get_monmap_and_config 2024-09-16T15:09:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.941+0000 7ff54a74c640 10 monclient: build_initial_monmap 2024-09-16T15:09:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.941+0000 7ff54a74c640 10 monclient: monmap: 2024-09-16T15:09:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:53.943189+0000 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:53.943189+0000 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:53.945 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.941+0000 7ff54a74c640 1 Processor -- start 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 -- start start 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient: init 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient: _add_conns ranks=[5,10,8,4,1,3,2,7,6,9,0] 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544136e40 0x7ff54409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-h con 0x7ff544136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff544137220 0x7ff544137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-f con 0x7ff544137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-e con 0x7ff544135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54409abc0 con 0x7ff544137220 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff54409ba50 con 0x7ff544136e40 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440a53c0 con 0x7ff544135dc0 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff54a74c640 10 monclient(hunting): authenticate will time out at 1330.532563s 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33504/0 (socket says 172.21.15.134:33504) 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 -- 172.21.15.134:0/2848404078 learned_addr learned my addr 172.21.15.134:0/2848404078 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): get_auth_request con 0x7ff544135dc0 auth_method 0 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 1 --2- 172.21.15.134:0/2848404078 >> v2:172.21.15.162:3300/0 conn(0x7ff544137220 0x7ff544137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544136e40 0x7ff54409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): get_auth_request con 0x7ff544137220 auth_method 0 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): get_auth_request con 0x7ff544136e40 auth_method 0 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient(hunting): handle_auth_done global_id 74195 payload 274 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 -- 172.21.15.134:0/2848404078 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544136e40 msgr2=0x7ff54409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544136e40 0x7ff54409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 -- 172.21.15.134:0/2848404078 >> v2:172.21.15.162:3300/0 conn(0x7ff544137220 msgr2=0x7ff544137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 --2- 172.21.15.134:0/2848404078 >> v2:172.21.15.162:3300/0 conn(0x7ff544137220 0x7ff544137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 10 monclient: _finish_hunting 0 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.942+0000 7ff548ce9640 1 monclient: found mon.noname-e 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.943+0000 7ff548ce9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.943+0000 7ff548ce9640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.943+0000 7ff548ce9640 1 -- 172.21.15.134:0/2848404078 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544073250 con 0x7ff544135dc0 2024-09-16T15:09:53.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff548ce9640 1 -- 172.21.15.134:0/2848404078 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 msgr2=0x7ff544136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff548ce9640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009e80 tx=0x7ff53c02a580 comp rx=0 tx=0).ready entity=mon.8 client_cookie=5bac11948cbd770e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 1 -- 172.21.15.134:0/2848404078 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff53c007a70 con 0x7ff544135dc0 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: dump: 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.959 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 1 -- 172.21.15.134:0/2848404078 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c02d050 con 0x7ff544135dc0 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff54a74c640 5 monclient: authenticate success, global_id 74195 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff54a74c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:53.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff54a74c640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 1 -- 172.21.15.134:0/2848404078 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff53c007bf0 con 0x7ff544135dc0 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff54a74c640 4 set_mon_vals no callback set 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.945+0000 7ff542ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff542ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff542ffd640 10 monclient: dump: 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.961 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 10 monclient: shutdown 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 -- 172.21.15.134:0/2848404078 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 msgr2=0x7ff544136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009e80 tx=0x7ff53c02a580 comp rx=0 tx=0).stop 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 -- 172.21.15.134:0/2848404078 shutdown_connections 2024-09-16T15:09:53.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff544135dc0 0x7ff544136220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 --2- 172.21.15.134:0/2848404078 >> v2:172.21.15.162:3300/0 conn(0x7ff544137220 0x7ff544137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 --2- 172.21.15.134:0/2848404078 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544136e40 0x7ff54409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 -- 172.21.15.134:0/2848404078 >> 172.21.15.134:0/2848404078 conn(0x7ff5440a6110 msgr2=0x7ff5440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 -- 172.21.15.134:0/2848404078 shutdown_connections 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.946+0000 7ff54a74c640 1 -- 172.21.15.134:0/2848404078 wait complete. 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.947+0000 7ff54a74c640 10 monclient: build_initial_monmap 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.947+0000 7ff54a74c640 10 monclient: monmap: 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:53.963 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.947+0000 7ff54a74c640 1 Processor -- start 2024-09-16T15:09:53.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.947+0000 7ff54a74c640 1 -- start start 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.947+0000 7ff54a74c640 10 monclient: init 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient: _add_conns ranks=[2,6,5,7,3,4,8,1,0] 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 0x7ff544134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-b con 0x7ff544135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 0x7ff5441352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-d con 0x7ff544136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544137220 0x7ff544121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): picked mon.noname-h con 0x7ff544137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441368a0 con 0x7ff544135dc0 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff544123fd0 con 0x7ff544137220 2024-09-16T15:09:53.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff544123910 con 0x7ff544136e40 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff54a74c640 10 monclient(hunting): authenticate will time out at 1330.538188s 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 0x7ff544134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 0x7ff544134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48000/0 (socket says 172.21.15.134:48000) 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 1 -- 172.21.15.134:0/19652496 learned_addr learned my addr 172.21.15.134:0/19652496 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): get_auth_request con 0x7ff544135dc0 auth_method 0 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 0x7ff5441352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): get_auth_request con 0x7ff544136e40 auth_method 0 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544137220 0x7ff544121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): get_auth_request con 0x7ff544137220 auth_method 0 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff548ce9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient(hunting): handle_auth_done global_id 34863 payload 824 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544137220 msgr2=0x7ff544121730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff544137220 0x7ff544121730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 msgr2=0x7ff544134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 0x7ff544134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient: _finish_hunting 0 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 monclient: found mon.noname-d 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff5437fe640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c009b30 con 0x7ff544136e40 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.948+0000 7ff543fff640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff544135dc0 0x7ff544134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff5437fe640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 msgr2=0x7ff5441352b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff5437fe640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 0x7ff5441352b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00e870 tx=0x7ff52c00ed30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff52c00ee60 con 0x7ff544136e40 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: dump: 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.969 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: _finish_auth 0 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c013470 con 0x7ff544136e40 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff52c013a50 con 0x7ff544136e40 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff54a74c640 5 monclient: authenticate success, global_id 34863 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff54a74c640 10 monclient: _renew_subs 2024-09-16T15:09:53.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff54a74c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5440a8a50 con 0x7ff544136e40 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.950+0000 7ff5427fc640 4 set_mon_vals no callback set 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff540ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff540ff9640 10 monclient: dump: 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.971 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 10 monclient: _renew_subs 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff544125480 con 0x7ff544136e40 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff5437fe640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 0x7ff5441352b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00e870 tx=0x7ff52c00ed30 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7ff52c013bf0 mgrmap(e 5) v1 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff52c013bf0 con 0x7ff544136e40 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 0x7ff5440a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544071b20 con 0x7ff5440a1a20 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff540ff9640 1 --2- 172.21.15.134:0/19652496 >> v2:172.21.15.162:6800/980547263 conn(0x7ff524038880 0x7ff52403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff543fff640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 0x7ff5440a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff543fff640 10 monclient: get_auth_request con 0x7ff5440a1a20 auth_method 0 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.951+0000 7ff543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff548ce9640 1 --2- 172.21.15.134:0/19652496 >> v2:172.21.15.162:6800/980547263 conn(0x7ff524038880 0x7ff52403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff548ce9640 10 monclient: get_auth_request con 0x7ff524038880 auth_method 0 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff543fff640 10 monclient(hunting): handle_auth_done global_id 74196 payload 824 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.952+0000 7ff543fff640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff544061ae0 con 0x7ff5440a1a20 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff543fff640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 msgr2=0x7ff5440a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff543fff640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 0x7ff5440a1e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff534011db0 tx=0x7ff53400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff53400fe70 con 0x7ff5440a1a20 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 10 monclient: dump: 2024-09-16T15:09:53.974 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.975 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff548ce9640 1 --2- 172.21.15.134:0/19652496 >> v2:172.21.15.162:6800/980547263 conn(0x7ff524038880 0x7ff52403ad40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54406cb00 tx=0x7ff53c009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff534004030 con 0x7ff5440a1a20 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 msgr2=0x7ff5440a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 0x7ff5440a1e00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff534011db0 tx=0x7ff53400f610 comp rx=0 tx=0).stop 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 msgr2=0x7ff5440a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:53.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.954+0000 7ff540ff9640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a1a20 0x7ff5440a1e00 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff534011db0 tx=0x7ff53400f610 comp rx=0 tx=0).stop 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.960+0000 7ff54a74c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.960+0000 7ff54a74c640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 0x7ff544121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.960+0000 7ff54a74c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.960+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54407a680 con 0x7ff5440a2340 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 0x7ff544121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient: get_auth_request con 0x7ff5440a2340 auth_method 0 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:53.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 10 monclient(hunting): handle_auth_done global_id 74205 payload 824 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.961+0000 7ff543fff640 1 -- 172.21.15.134:0/19652496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff54409b490 con 0x7ff5440a2340 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff543fff640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 msgr2=0x7ff544121c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff543fff640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 0x7ff544121c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff53401b000 tx=0x7ff5340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff534003ea0 con 0x7ff5440a2340 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 10 monclient: dump: 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:53.979 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7ff534003a30 con 0x7ff5440a2340 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:53.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 msgr2=0x7ff544121c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 0x7ff544121c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff53401b000 tx=0x7ff5340046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 msgr2=0x7ff544121c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.963+0000 7ff540ff9640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5440a2340 0x7ff544121c70 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff53401b000 tx=0x7ff5340046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:53.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:53.982 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:53.983 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:53.984 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:53.985 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:53.986 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.988 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:53.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.992 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:53.994 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:53.996 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:53.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:53.998 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:53.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.001 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:54.002 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.003 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.006 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.006 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.006 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:54.006 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:54.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 >> v2:172.21.15.162:6800/980547263 conn(0x7ff524038880 msgr2=0x7ff52403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 --2- 172.21.15.134:0/19652496 >> v2:172.21.15.162:6800/980547263 conn(0x7ff524038880 0x7ff52403ad40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff54406cb00 tx=0x7ff53c009000 comp rx=0 tx=0).stop 2024-09-16T15:09:54.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 10 monclient: shutdown 2024-09-16T15:09:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 msgr2=0x7ff5441352b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 --2- 172.21.15.134:0/19652496 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff544136e40 0x7ff5441352b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00e870 tx=0x7ff52c00ed30 comp rx=0 tx=0).stop 2024-09-16T15:09:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff548ce9640 1 -- 172.21.15.134:0/19652496 reap_dead start 2024-09-16T15:09:54.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff540ff9640 1 -- 172.21.15.134:0/19652496 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(422..422 src has 1..422) v4 ==== 4007+0+0 (unknown 0 0 0) 0x7ff52c04a2b0 con 0x7ff544136e40 2024-09-16T15:09:54.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff548ce9640 1 -- 172.21.15.134:0/19652496 reap_dead start 2024-09-16T15:09:54.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 shutdown_connections 2024-09-16T15:09:54.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 >> 172.21.15.134:0/19652496 conn(0x7ff5440a6110 msgr2=0x7ff54406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 shutdown_connections 2024-09-16T15:09:54.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:53.964+0000 7ff54a74c640 1 -- 172.21.15.134:0/19652496 wait complete. 2024-09-16T15:09:54.012 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:54.012 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:09:54.013 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:09:54.013 INFO:tasks.rados.rados.0.smithi162.stdout:2132: done (1 left) 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2133: done (0 left) 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2134: delete oid 25 current snap is 199 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2134: done (0 left) 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2135: copy_from oid 44 from oid 8 current snap is 199 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2136: read oid 34 snap -1 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2136: expect (ObjNum 763 snap 198 seq_num 763) 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2137: read oid 22 snap 195 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2137: expect (ObjNum 749 snap 195 seq_num 749) 2024-09-16T15:09:54.014 INFO:tasks.rados.rados.0.smithi162.stdout:2138: snap_create 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=193942128}) 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4218619758}) 2024-09-16T15:09:54.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4261853298}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2703777453}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=602376680}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1606635234}) 2024-09-16T15:09:54.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1552241388}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=966700063}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.983+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3068016436}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3369015493}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1753534140}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=234120788}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1649526289}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=266950933}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4129438679}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.020 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3893097187}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4150948197}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:53.990+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3553501040}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=685517726}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=844310117}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.004+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3128547470}) 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=315978970}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2452660707}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3122499926}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2437780193}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2365858880}) 2024-09-16T15:09:54.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.005+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2091681095}) 2024-09-16T15:09:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:2137: expect (ObjNum 749 snap 195 seq_num 749) 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 10 monclient: get_monmap_and_config 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 10 monclient: monmap: 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:54.148181+0000 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:54.148181+0000 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:54.149 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 1 Processor -- start 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 1 -- start start 2024-09-16T15:09:54.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.146+0000 7ff514908640 10 monclient: init 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient: _add_conns ranks=[1,4,9,5,6,10,2,3,8,0,7] 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff510136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): picked mon.noname-c con 0x7ff510137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): picked mon.noname-a con 0x7ff51013c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff510073250 con 0x7ff510136e40 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5100a56e0 con 0x7ff51013c370 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff51009abc0 con 0x7ff510137220 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff514908640 10 monclient(hunting): authenticate will time out at 1330.737535s 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35728/0 (socket says 172.21.15.134:35728) 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52796/0 (socket says 172.21.15.134:52796) 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 -- 172.21.15.134:0/19353671 learned_addr learned my addr 172.21.15.134:0/19353671 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 10 monclient(hunting): get_auth_request con 0x7ff51013c370 auth_method 0 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request con 0x7ff510137220 auth_method 0 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 -- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 msgr2=0x7ff51013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 -- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 msgr2=0x7ff51013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:54.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 --2- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50ed9d640 1 --2- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_done global_id 5188 payload 274 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 -- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 msgr2=0x7ff51013e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 -- 172.21.15.134:0/19353671 >> v1:172.21.15.162:6789/0 conn(0x7ff510136e40 legacy=0x7ff51009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 monclient: found mon.noname-c 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:54.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.147+0000 7ff50dd9b640 1 -- 172.21.15.134:0/19353671 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51009ba50 con 0x7ff510137220 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50dd9b640 1 -- 172.21.15.134:0/19353671 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 msgr2=0x7ff510137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/19353671 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50402a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f4a1f85718ea1b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50d59a640 1 -- 172.21.15.134:0/19353671 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff504007bc0 con 0x7ff510137220 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.150+0000 7ff50d59a640 10 monclient: dump: 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.158 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 1 -- 172.21.15.134:0/19353671 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50402d050 con 0x7ff510137220 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 5 monclient: authenticate success, global_id 5188 2024-09-16T15:09:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 1 -- 172.21.15.134:0/19353671 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff504007d40 con 0x7ff510137220 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 4 set_mon_vals no callback set 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff50d59a640 10 monclient: dump: 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.160 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 10 monclient: shutdown 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 -- 172.21.15.134:0/19353671 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 msgr2=0x7ff510137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 --2- 172.21.15.134:0/19353671 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff504009d90 tx=0x7ff50402a4e0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 -- 172.21.15.134:0/19353671 shutdown_connections 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 --2- 172.21.15.134:0/19353671 >> v2:172.21.15.134:3300/0 conn(0x7ff51013c370 0x7ff51013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 --2- 172.21.15.134:0/19353671 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff510137220 0x7ff510137600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 -- 172.21.15.134:0/19353671 >> 172.21.15.134:0/19353671 conn(0x7ff5100a6110 msgr2=0x7ff5100a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 -- 172.21.15.134:0/19353671 shutdown_connections 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.151+0000 7ff514908640 1 -- 172.21.15.134:0/19353671 wait complete. 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.152+0000 7ff514908640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.152+0000 7ff514908640 10 monclient: monmap: 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:54.162 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.163 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.152+0000 7ff514908640 1 Processor -- start 2024-09-16T15:09:54.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.152+0000 7ff514908640 1 -- start start 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.152+0000 7ff514908640 10 monclient: init 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient: _add_conns ranks=[6,5,3,4,0,2,7,1,8] 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): picked mon.noname-d con 0x7ff510136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff510137220 0x7ff510135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): picked mon.noname-h con 0x7ff510137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff51013c370 0x7ff510136400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): picked mon.noname-g con 0x7ff51013c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:54.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101c1920 con 0x7ff51013c370 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100a76a0 con 0x7ff510137220 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff510121fd0 con 0x7ff510136e40 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff514908640 10 monclient(hunting): authenticate will time out at 1330.743181s 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57998/0 (socket says 172.21.15.134:57998) 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 -- 172.21.15.134:0/4270791962 learned_addr learned my addr 172.21.15.134:0/4270791962 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): get_auth_request con 0x7ff510136e40 auth_method 0 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff510137220 0x7ff510135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff51013c370 0x7ff510136400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request con 0x7ff510137220 auth_method 0 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): get_auth_request con 0x7ff51013c370 auth_method 0 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient(hunting): handle_auth_done global_id 34872 payload 824 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff510137220 msgr2=0x7ff510135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff510137220 0x7ff510135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff51013c370 msgr2=0x7ff510136400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff51013c370 0x7ff510136400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 monclient: found mon.noname-d 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.153+0000 7ff50e59c640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504009a70 con 0x7ff510136e40 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.155+0000 7ff50e59c640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 msgr2=0x7ff5100a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.155+0000 7ff50e59c640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c430 tx=0x7ff50800c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.155+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff508003860 con 0x7ff510136e40 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: dump: 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.177 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508011c10 con 0x7ff510136e40 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 5 monclient: authenticate success, global_id 34872 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 monclient: _renew_subs 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff510122590 con 0x7ff510136e40 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff508011e00 con 0x7ff510136e40 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 10 monclient: dump: 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.179 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff50cd99640 4 set_mon_vals no callback set 2024-09-16T15:09:54.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 monclient: _renew_subs 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff510121070 con 0x7ff510136e40 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff5080117a0 con 0x7ff510136e40 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 0x7ff510120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff510073e50 con 0x7ff51011fc60 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.156+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 0x7ff510120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff4feffd640 1 --2- 172.21.15.134:0/4270791962 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4f0038880 0x7ff4f003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient: get_auth_request con 0x7ff51011fc60 auth_method 0 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7ff50804a010 con 0x7ff510136e40 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_done global_id 44723 payload 824 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50ed9d640 1 --2- 172.21.15.134:0/4270791962 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4f0038880 0x7ff4f003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50dd9b640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff510061ae0 con 0x7ff51011fc60 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.157+0000 7ff50ed9d640 10 monclient: get_auth_request con 0x7ff4f0038880 auth_method 0 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff50dd9b640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 msgr2=0x7ff510120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 0x7ff510120040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff510137f00 tx=0x7ff504004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5040044b0 con 0x7ff51011fc60 2024-09-16T15:09:54.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 10 monclient: dump: 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.183 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff50ed9d640 1 --2- 172.21.15.134:0/4270791962 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4f0038880 0x7ff4f003ad40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff500006a90 tx=0x7ff500008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff504004610 con 0x7ff51011fc60 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 msgr2=0x7ff510120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 0x7ff510120040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff510137f00 tx=0x7ff504004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 msgr2=0x7ff510120040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.159+0000 7ff4feffd640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff51011fc60 0x7ff510120040 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff510137f00 tx=0x7ff504004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.165+0000 7ff514908640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff514908640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 0x7ff51009e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff514908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff510071b20 con 0x7ff510120580 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 0x7ff51009e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient: get_auth_request con 0x7ff510120580 auth_method 0 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 10 monclient(hunting): handle_auth_done global_id 44732 payload 824 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.166+0000 7ff50dd9b640 1 -- 172.21.15.134:0/4270791962 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff51007a680 con 0x7ff510120580 2024-09-16T15:09:54.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff50dd9b640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 msgr2=0x7ff51009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff50dd9b640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 0x7ff51009e520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff50402a520 tx=0x7ff504030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff50404be10 con 0x7ff510120580 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff4feffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff4feffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.168+0000 7ff4feffd640 10 monclient: dump: 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.186 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7ff50404b9a0 con 0x7ff510120580 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 msgr2=0x7ff51009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 0x7ff51009e520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff50402a520 tx=0x7ff504030000 comp rx=0 tx=0).stop 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 msgr2=0x7ff51009e520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff4feffd640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff510120580 0x7ff51009e520 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff50402a520 tx=0x7ff504030000 comp rx=0 tx=0).stop 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:54.188 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.189 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.190 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:54.191 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:54.192 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.193 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.194 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:54.195 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.197 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.198 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:54.201 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:54.203 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.204 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:54.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:54.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.210 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:09:54.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:54.212 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:54.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4f0038880 msgr2=0x7ff4f003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 --2- 172.21.15.134:0/4270791962 >> v2:172.21.15.162:6800/980547263 conn(0x7ff4f0038880 0x7ff4f003ad40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff500006a90 tx=0x7ff500008040 comp rx=0 tx=0).stop 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 10 monclient: shutdown 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 msgr2=0x7ff5100a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c430 tx=0x7ff50800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff50ed9d640 1 -- 172.21.15.134:0/4270791962 reap_dead start 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 shutdown_connections 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 --2- 172.21.15.134:0/4270791962 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff510136e40 0x7ff5100a28a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 >> 172.21.15.134:0/4270791962 conn(0x7ff5100a6110 msgr2=0x7ff51006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 shutdown_connections 2024-09-16T15:09:54.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.169+0000 7ff514908640 1 -- 172.21.15.134:0/4270791962 wait complete. 2024-09-16T15:09:54.215 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:54.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:09:54.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:09:54.226 INFO:tasks.rados.rados.0.smithi162.stdout:2136: expect (ObjNum 763 snap 198 seq_num 763) 2024-09-16T15:09:54.275 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.276 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.276 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=91} crc {osdmap=83810366}) 2024-09-16T15:09:54.276 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.276 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.276 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=93} crc {osdmap=1784670261}) 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=87} crc {osdmap=3749209198}) 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.277 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=90} crc {osdmap=120580633}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=94} crc {osdmap=2176202049}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=94} crc {osdmap=3783153587}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=92} crc {osdmap=1862631492}) 2024-09-16T15:09:54.278 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:09:54.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=93} crc {osdmap=3277398213}) 2024-09-16T15:09:54.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:09:54.271+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=94} crc {osdmap=1656033676}) 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 10 monclient: get_monmap_and_config 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 10 monclient: monmap: 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:54.313122+0000 2024-09-16T15:09:54.313 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:54.313122+0000 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:54.314 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 1 Processor -- start 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 1 -- start start 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.311+0000 7f379aa68640 10 monclient: init 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient: _add_conns ranks=[1,2,8,7,5,4,10,3,9,0,6] 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3794136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-b con 0x7f3794137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 0x7f379413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-e con 0x7f379413c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3794073250 con 0x7f3794136e40 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940a56e0 con 0x7f3794137220 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f379409abc0 con 0x7f379413c370 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f379aa68640 10 monclient(hunting): authenticate will time out at 1330.902474s 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 0x7f379413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48038/0 (socket says 172.21.15.134:48038) 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 0x7f379413e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33518/0 (socket says 172.21.15.134:33518) 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 -- 172.21.15.134:0/2835069314 learned_addr learned my addr 172.21.15.134:0/2835069314 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): get_auth_request con 0x7f3794137220 auth_method 0 2024-09-16T15:09:54.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): get_auth_request con 0x7f379413c370 auth_method 0 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient(hunting): handle_auth_done global_id 44741 payload 274 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 -- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 msgr2=0x7f379413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 --2- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 0x7f379413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 -- 172.21.15.134:0/2835069314 >> v1:172.21.15.162:6789/0 conn(0x7f3794136e40 legacy=0x7f379409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 monclient: found mon.noname-b 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:54.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.312+0000 7f3793fff640 1 -- 172.21.15.134:0/2835069314 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379409ba50 con 0x7f3794137220 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f3793fff640 1 -- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 msgr2=0x7f3794137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f3793fff640 1 --2- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3784009e80 tx=0x7f378402a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be7e4089aba9f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 1 -- 172.21.15.134:0/2835069314 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3784007a70 con 0x7f3794137220 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:54.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: dump: 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.328 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 1 -- 172.21.15.134:0/2835069314 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378402d050 con 0x7f3794137220 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f379aa68640 5 monclient: authenticate success, global_id 44741 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f379aa68640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 1 -- 172.21.15.134:0/2835069314 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3784007bf0 con 0x7f3794137220 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f379aa68640 4 set_mon_vals no callback set 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:54.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.315+0000 7f37937fe640 10 monclient: dump: 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.330 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 10 monclient: shutdown 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 -- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 msgr2=0x7f3794137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 --2- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3784009e80 tx=0x7f378402a580 comp rx=0 tx=0).stop 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 -- 172.21.15.134:0/2835069314 shutdown_connections 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 --2- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f379413c370 0x7f379413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 --2- 172.21.15.134:0/2835069314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794137220 0x7f3794137600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 -- 172.21.15.134:0/2835069314 >> 172.21.15.134:0/2835069314 conn(0x7f37940a6110 msgr2=0x7f37940a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 -- 172.21.15.134:0/2835069314 shutdown_connections 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 1 -- 172.21.15.134:0/2835069314 wait complete. 2024-09-16T15:09:54.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.316+0000 7f379aa68640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient: monmap: 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.332 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 Processor -- start 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 -- start start 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient: init 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient: _add_conns ranks=[2,0,1,6,7,4,5,8,3] 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-b con 0x7f3794136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3794137220 0x7f3794135d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-a con 0x7f3794137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:54.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f379413c370 0x7f3794136260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): picked mon.noname-f con 0x7f379413c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941223f0 con 0x7f3794137220 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379413af90 con 0x7f379413c370 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379413b510 con 0x7f3794136e40 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.317+0000 7f379aa68640 10 monclient(hunting): authenticate will time out at 1330.907955s 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3794137220 0x7f3794135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48048/0 (socket says 172.21.15.134:48048) 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 -- 172.21.15.134:0/2016511167 learned_addr learned my addr 172.21.15.134:0/2016511167 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): get_auth_request con 0x7f3794136e40 auth_method 0 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): get_auth_request con 0x7f3794137220 auth_method 0 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f379413c370 0x7f3794136260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): get_auth_request con 0x7f379413c370 auth_method 0 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3799005640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient(hunting): handle_auth_done global_id 44750 payload 824 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f379413c370 msgr2=0x7f3794136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f379413c370 0x7f3794136260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3794137220 msgr2=0x7f3794135d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3794137220 0x7f3794135d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 monclient: found mon.noname-b 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.318+0000 7f3798804640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009b30 con 0x7f3794136e40 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f3798804640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 msgr2=0x7f3794139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f3798804640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c430 tx=0x7f378c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f378c011c10 con 0x7f3794136e40 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: dump: 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.337 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c011d90 con 0x7f3794136e40 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 5 monclient: authenticate success, global_id 44750 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 10 monclient: _renew_subs 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940a7af0 con 0x7f3794136e40 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f378c012890 con 0x7f3794136e40 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f37917fa640 10 monclient: dump: 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.339 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f3792ffd640 4 set_mon_vals no callback set 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 10 monclient: _renew_subs 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.320+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3794122110 con 0x7f3794136e40 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f379aa68640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f379aa68640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 0x7f379402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3794071b20 con 0x7f3794120420 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f378c012420 con 0x7f3794136e40 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 0x7f379402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient: get_auth_request con 0x7f3794120420 auth_method 0 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f37917fa640 1 --2- 172.21.15.134:0/2016511167 >> v2:172.21.15.162:6800/980547263 conn(0x7f3774038880 0x7f377403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f378c049f80 con 0x7f3794136e40 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 10 monclient(hunting): handle_auth_done global_id 5197 payload 824 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3793fff640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f379413aaf0 con 0x7f3794120420 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3799005640 1 --2- 172.21.15.134:0/2016511167 >> v2:172.21.15.162:6800/980547263 conn(0x7f3774038880 0x7f377403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.321+0000 7f3799005640 10 monclient: get_auth_request con 0x7f3774038880 auth_method 0 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f3793fff640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 msgr2=0x7f379402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f3793fff640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 0x7f379402b260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3794137f00 tx=0x7f3784004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3784007ca0 con 0x7f3794120420 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 10 monclient: dump: 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.343 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f3799005640 1 --2- 172.21.15.134:0/2016511167 >> v2:172.21.15.162:6800/980547263 conn(0x7f3774038880 0x7f377403ad40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f379406b7d0 tx=0x7f377c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3784007e00 con 0x7f3794120420 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 msgr2=0x7f379402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 0x7f379402b260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3794137f00 tx=0x7f3784004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:54.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 msgr2=0x7f379402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.324+0000 7f37917fa640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120420 0x7f379402b260 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3794137f00 tx=0x7f3784004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:54.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f379aa68640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:54.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f379aa68640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 0x7f379402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f379aa68640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3794061ae0 con 0x7f3794120800 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 0x7f379402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient: get_auth_request con 0x7f3794120800 auth_method 0 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.330+0000 7f3793fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.331+0000 7f3793fff640 10 monclient(hunting): handle_auth_done global_id 5206 payload 824 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.331+0000 7f3793fff640 1 -- 172.21.15.134:0/2016511167 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f379407a680 con 0x7f3794120800 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f3793fff640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 msgr2=0x7f379402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f3793fff640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 0x7f379402b7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3784009e20 tx=0x7f378402f000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3784049580 con 0x7f3794120800 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 10 monclient: dump: 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.348 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.349 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f3784004640 con 0x7f3794120800 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 msgr2=0x7f379402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 0x7f379402b7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3784009e20 tx=0x7f378402f000 comp rx=0 tx=0).stop 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 msgr2=0x7f379402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.333+0000 7f37917fa640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3794120800 0x7f379402b7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.350 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:54.351 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.352 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.353 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:54.354 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:54.356 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.357 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.359 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:54.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:54.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.376 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:54.376 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:54.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 >> v2:172.21.15.162:6800/980547263 conn(0x7f3774038880 msgr2=0x7f377403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 --2- 172.21.15.134:0/2016511167 >> v2:172.21.15.162:6800/980547263 conn(0x7f3774038880 0x7f377403ad40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f379406b7d0 tx=0x7f377c009190 comp rx=0 tx=0).stop 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 10 monclient: shutdown 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 msgr2=0x7f3794139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c430 tx=0x7f378c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f3799005640 1 -- 172.21.15.134:0/2016511167 reap_dead start 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 shutdown_connections 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 --2- 172.21.15.134:0/2016511167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3794136e40 0x7f3794139f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 >> 172.21.15.134:0/2016511167 conn(0x7f37940a6110 msgr2=0x7f37940a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 shutdown_connections 2024-09-16T15:09:54.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.334+0000 7f379aa68640 1 -- 172.21.15.134:0/2016511167 wait complete. 2024-09-16T15:09:54.379 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:54.379 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:09:54.379 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:09:54.423 INFO:tasks.rados.rados.0.smithi162.stdout:2135: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:09:54.423 INFO:tasks.rados.rados.0.smithi162.stdout:2135: finishing copy_from to smithi16234619-44 2024-09-16T15:09:54.423 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 841 (ObjNum 715 snap 189 seq_num 715) dirty exists 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.474+0000 7f5e3dd11640 10 monclient: get_monmap_and_config 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.474+0000 7f5e3dd11640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.474+0000 7f5e3dd11640 10 monclient: monmap: 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:54.476467+0000 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:54.476467+0000 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.477 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:54.478 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 Processor -- start 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 -- start start 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient: init 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient: _add_conns ranks=[0,7,9,2,1,10,4,3,8,6,5] 2024-09-16T15:09:54.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5e38136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e38137220 0x7f5e38137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-i con 0x7f5e38137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-a con 0x7f5e3813c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e38073250 con 0x7f5e38136e40 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380a56e0 con 0x7f5e3813c370 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3809abc0 con 0x7f5e38137220 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e3dd11640 10 monclient(hunting): authenticate will time out at 1331.065835s 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e37fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e377fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5e38136e40 0x7f5e3809c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50874/0 (socket says 172.21.15.134:50874) 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e377fe640 1 -- 172.21.15.134:0/2506198192 learned_addr learned my addr 172.21.15.134:0/2506198192 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e37fff640 10 monclient(hunting): get_auth_request con 0x7f5e3813c370 auth_method 0 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e37fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.475+0000 7f5e37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e38137220 0x7f5e38137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request con 0x7f5e38137220 auth_method 0 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient(hunting): handle_auth_done global_id 74223 payload 274 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e38137220 msgr2=0x7f5e38137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e38137220 0x7f5e38137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 >> v1:172.21.15.134:6789/0 conn(0x7f5e38136e40 legacy=0x7f5e3809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 monclient: found mon.noname-a 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3809ba50 con 0x7f5e3813c370 2024-09-16T15:09:54.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 msgr2=0x7f5e3813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:09:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 msgr2=0x7f5e3813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:09:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f5e30009dc0 tx=0x7f5e3002a510 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:09:54.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.476+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:54.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 10 monclient: get_auth_request con 0x7f5e3813c370 auth_method 0 2024-09-16T15:09:54.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 0 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T15:09:54.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient: ms_handle_reset current mon v2:172.21.15.134:3300/0 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 msgr2=0x7f5e3813e760 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 --2- 172.21.15.134:0/2506198192 >> v2:172.21.15.134:3300/0 conn(0x7f5e3813c370 0x7f5e3813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient: _add_conns ranks=[6,2,3,10,4,1,5,8] 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e20003630 0x7f5e20005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): picked mon.noname-d con 0x7f5e20003630 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 0x7f5e20008820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): picked mon.noname-b con 0x7f5e20006390 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20009190 0x7f5e2000b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): picked mon.noname-g con 0x7f5e20009190 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2000bfa0 con 0x7f5e20006390 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2000c0e0 con 0x7f5e20009190 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2000c2a0 con 0x7f5e20003630 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e367fc640 10 monclient(hunting): ms_handle_reset stray mon v2:172.21.15.134:3300/0 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e20003630 0x7f5e20005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 0x7f5e20008820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): get_auth_request con 0x7f5e20003630 auth_method 0 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:54.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request con 0x7f5e20006390 auth_method 0 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_done global_id 74223 payload 293 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 -- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e20003630 msgr2=0x7f5e20005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e20003630 0x7f5e20005a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 -- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20009190 msgr2=0x7f5e2000b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20009190 0x7f5e2000b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20009190 0x7f5e2000b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 monclient: found mon.noname-b 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e36ffd640 1 -- 172.21.15.134:0/2506198192 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2000c520 con 0x7f5e20006390 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 10 monclient: get_auth_request con 0x7f5e20009190 auth_method 0 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e20003630 0x7f5e20005a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20009190 0x7f5e2000b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.678+0000 7f5e37fff640 1 -- 172.21.15.134:0/2506198192 reap_dead start 2024-09-16T15:09:54.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e36ffd640 1 -- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 msgr2=0x7f5e20008820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 0x7f5e20008820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009ac0 tx=0x7f5e30009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a11190e48e55c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e2c013470 con 0x7f5e20006390 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: dump: 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e3dd11640 5 monclient: authenticate success, global_id 74223 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c013610 con 0x7f5e20006390 2024-09-16T15:09:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e3dd11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e3dd11640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e3dd11640 4 set_mon_vals no callback set 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 1 -- 172.21.15.134:0/2506198192 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e2c013bf0 con 0x7f5e20006390 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.680+0000 7f5e367fc640 10 monclient: dump: 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 10 monclient: shutdown 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 msgr2=0x7f5e20008820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 0x7f5e20008820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009ac0 tx=0x7f5e30009750 comp rx=0 tx=0).stop 2024-09-16T15:09:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2506198192 shutdown_connections 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2506198192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e20006390 0x7f5e20008820 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2506198192 >> 172.21.15.134:0/2506198192 conn(0x7f5e380a6110 msgr2=0x7f5e380a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2506198192 shutdown_connections 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2506198192 wait complete. 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.681+0000 7f5e3dd11640 10 monclient: monmap: 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.682+0000 7f5e3dd11640 1 Processor -- start 2024-09-16T15:09:54.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.682+0000 7f5e3dd11640 1 -- start start 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.682+0000 7f5e3dd11640 10 monclient: init 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient: _add_conns ranks=[3,7,0,1,2,5,6,4,8] 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20006390 0x7f5e38070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-g con 0x7f5e20006390 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e3806d170 0x7f5e3806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-i con 0x7f5e3806d170 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 0x7f5e381d6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): picked mon.noname-a con 0x7f5e3806dad0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380a7a80 con 0x7f5e3806dad0 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380a7530 con 0x7f5e20006390 2024-09-16T15:09:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e3806d170 0x7f5e3806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20006390 0x7f5e38070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e3806d170 0x7f5e3806d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:42008/0 (socket says 172.21.15.134:42008) 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 1 -- 172.21.15.134:0/2279623674 learned_addr learned my addr 172.21.15.134:0/2279623674 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3813b510 con 0x7f5e3806d170 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e3dd11640 10 monclient(hunting): authenticate will time out at 1331.273811s 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request con 0x7f5e3806d170 auth_method 0 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e377fe640 10 monclient(hunting): get_auth_request con 0x7f5e20006390 auth_method 0 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 0x7f5e381d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e377fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.683+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e36ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): get_auth_request con 0x7f5e3806dad0 auth_method 0 2024-09-16T15:09:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient(hunting): handle_auth_done global_id 74232 payload 824 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e3806d170 msgr2=0x7f5e3806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5e3806d170 0x7f5e3806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20006390 msgr2=0x7f5e38070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e20006390 0x7f5e38070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 monclient: found mon.noname-a 2024-09-16T15:09:54.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:54.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.684+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3806df50 con 0x7f5e3806dad0 2024-09-16T15:09:54.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.687+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 msgr2=0x7f5e381d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.687+0000 7f5e37fff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 0x7f5e381d6580 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009dc0 tx=0x7f5e30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e30030c30 con 0x7f5e3806dad0 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 10 monclient: dump: 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 5 monclient: authenticate success, global_id 74232 2024-09-16T15:09:54.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 10 monclient: _renew_subs 2024-09-16T15:09:54.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3806f7c0 con 0x7f5e3806dad0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 10 monclient: _renew_subs 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.689+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3806f200 con 0x7f5e3806dad0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.690+0000 7f5e3dd11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.690+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 0x7f5e3809d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.690+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.690+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38071b20 con 0x7f5e3809d510 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 0x7f5e3809d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient: get_auth_request con 0x7f5e3809d510 auth_method 0 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3002d050 con 0x7f5e3806dad0 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e30030d90 con 0x7f5e3806dad0 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 10 monclient: dump: 2024-09-16T15:09:54.712 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.713 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e35ffb640 4 set_mon_vals no callback set 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 10 monclient(hunting): handle_auth_done global_id 34881 payload 824 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e377fe640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e38061ae0 con 0x7f5e3809d510 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.691+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5e300307c0 con 0x7f5e3806dad0 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.692+0000 7f5e1ffff640 1 --2- 172.21.15.134:0/2279623674 >> v2:172.21.15.162:6800/980547263 conn(0x7f5e24038880 0x7f5e2403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.692+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f5e3006a590 con 0x7f5e3806dad0 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.692+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2279623674 >> v2:172.21.15.162:6800/980547263 conn(0x7f5e24038880 0x7f5e2403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.692+0000 7f5e36ffd640 10 monclient: get_auth_request con 0x7f5e24038880 auth_method 0 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e377fe640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 msgr2=0x7f5e3809d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 0x7f5e3809d8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003470 tx=0x7f5e28010db0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e28004ea0 con 0x7f5e3809d510 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 10 monclient: dump: 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.715 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5e2800eb30 con 0x7f5e3809d510 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:54.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 msgr2=0x7f5e3809d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.693+0000 7f5e1ffff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 0x7f5e3809d8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003470 tx=0x7f5e28010db0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.694+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 msgr2=0x7f5e3809d8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.694+0000 7f5e1ffff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809d510 0x7f5e3809d8f0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003470 tx=0x7f5e28010db0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.694+0000 7f5e36ffd640 1 --2- 172.21.15.134:0/2279623674 >> v2:172.21.15.162:6800/980547263 conn(0x7f5e24038880 0x7f5e2403ad40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c008030 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e3dd11640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 0x7f5e381daae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e3dd11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3804e210 con 0x7f5e3809de30 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 0x7f5e381daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient: get_auth_request con 0x7f5e3809de30 auth_method 0 2024-09-16T15:09:54.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 10 monclient(hunting): handle_auth_done global_id 34890 payload 824 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.700+0000 7f5e377fe640 1 -- 172.21.15.134:0/2279623674 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5e3807a680 con 0x7f5e3809de30 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e377fe640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 msgr2=0x7f5e381daae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e377fe640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 0x7f5e381daae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e2801f460 tx=0x7f5e28004980 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5e28024070 con 0x7f5e3809de30 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e1ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e1ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.702+0000 7f5e1ffff640 10 monclient: dump: 2024-09-16T15:09:54.718 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.719 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f5e2801f270 con 0x7f5e3809de30 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 msgr2=0x7f5e381daae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 0x7f5e381daae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e2801f460 tx=0x7f5e28004980 comp rx=0 tx=0).stop 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 msgr2=0x7f5e381daae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e1ffff640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5e3809de30 0x7f5e381daae0 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e2801f460 tx=0x7f5e28004980 comp rx=0 tx=0).stop 2024-09-16T15:09:54.720 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:54.721 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.722 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:54.724 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:54.725 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.726 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:54.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:54.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:54.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:54.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:54.743 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.744 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.745 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:54.746 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 >> v2:172.21.15.162:6800/980547263 conn(0x7f5e24038880 msgr2=0x7f5e2403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2279623674 >> v2:172.21.15.162:6800/980547263 conn(0x7f5e24038880 0x7f5e2403ad40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c008030 comp rx=0 tx=0).stop 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 10 monclient: shutdown 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 msgr2=0x7f5e381d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 --2- 172.21.15.134:0/2279623674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e3806dad0 0x7f5e381d6580 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009dc0 tx=0x7f5e30004a40 comp rx=0 tx=0).stop 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 reap_dead start 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e37fff640 1 -- 172.21.15.134:0/2279623674 reap_dead start 2024-09-16T15:09:54.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 shutdown_connections 2024-09-16T15:09:54.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 >> 172.21.15.134:0/2279623674 conn(0x7f5e380a6110 msgr2=0x7f5e3813b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 shutdown_connections 2024-09-16T15:09:54.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.703+0000 7f5e3dd11640 1 -- 172.21.15.134:0/2279623674 wait complete. 2024-09-16T15:09:54.748 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:54.748 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:09:54.748 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:09:54.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.843+0000 7f9546dc7640 10 monclient: get_monmap_and_config 2024-09-16T15:09:54.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.843+0000 7f9546dc7640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.843+0000 7f9546dc7640 10 monclient: monmap: 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:54.845206+0000 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:54.845206+0000 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.846 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.847 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.847 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.847 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:54.847 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:54.847 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 Processor -- start 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 -- start start 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient: init 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient: _add_conns ranks=[8,10,7,9,6,4,1,5,2,0,3] 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-e con 0x7f9540136eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9540137290 0x7f9540137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-f con 0x7f9540137290 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9540133d10 0x7f9540134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-i con 0x7f9540133d10 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400a5560 con 0x7f9540137290 2024-09-16T15:09:54.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f954009b240 con 0x7f9540133d10 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f954009b520 con 0x7f9540136eb0 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9546dc7640 10 monclient(hunting): authenticate will time out at 1331.434545s 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33530/0 (socket says 172.21.15.134:33530) 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 -- 172.21.15.134:0/2054390242 learned_addr learned my addr 172.21.15.134:0/2054390242 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): get_auth_request con 0x7f9540136eb0 auth_method 0 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9540133d10 0x7f9540134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): get_auth_request con 0x7f9540133d10 auth_method 0 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 1 --2- 172.21.15.134:0/2054390242 >> v2:172.21.15.162:3300/0 conn(0x7f9540137290 0x7f9540137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): get_auth_request con 0x7f9540137290 auth_method 0 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient(hunting): handle_auth_done global_id 74213 payload 274 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 -- 172.21.15.134:0/2054390242 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9540133d10 msgr2=0x7f9540134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9540133d10 0x7f9540134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 -- 172.21.15.134:0/2054390242 >> v2:172.21.15.162:3300/0 conn(0x7f9540137290 msgr2=0x7f9540137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 --2- 172.21.15.134:0/2054390242 >> v2:172.21.15.162:3300/0 conn(0x7f9540137290 0x7f9540137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 monclient: found mon.noname-e 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.844+0000 7f9544b63640 1 -- 172.21.15.134:0/2054390242 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540073250 con 0x7f9540136eb0 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9544b63640 1 -- 172.21.15.134:0/2054390242 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 msgr2=0x7f954009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9544b63640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9534009870 tx=0x7f953402a0a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a70edf16e677fa33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 1 -- 172.21.15.134:0/2054390242 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f953402ae00 con 0x7f9540136eb0 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: dump: 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9546dc7640 5 monclient: authenticate success, global_id 74213 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 1 -- 172.21.15.134:0/2054390242 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95340307b0 con 0x7f9540136eb0 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9546dc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9546dc7640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f9546dc7640 4 set_mon_vals no callback set 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 1 -- 172.21.15.134:0/2054390242 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9534030dd0 con 0x7f9540136eb0 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.847+0000 7f953f7fe640 10 monclient: dump: 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.861 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 10 monclient: shutdown 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 -- 172.21.15.134:0/2054390242 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 msgr2=0x7f954009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9534009870 tx=0x7f953402a0a0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 -- 172.21.15.134:0/2054390242 shutdown_connections 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9540133d10 0x7f9540134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 --2- 172.21.15.134:0/2054390242 >> v2:172.21.15.162:3300/0 conn(0x7f9540137290 0x7f9540137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 --2- 172.21.15.134:0/2054390242 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9540136eb0 0x7f954009bf00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 -- 172.21.15.134:0/2054390242 >> 172.21.15.134:0/2054390242 conn(0x7f95400a99a0 msgr2=0x7f95400a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 -- 172.21.15.134:0/2054390242 shutdown_connections 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.848+0000 7f9546dc7640 1 -- 172.21.15.134:0/2054390242 wait complete. 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient: build_initial_monmap 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient: monmap: 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:54.863 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 1 Processor -- start 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 1 -- start start 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient: init 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient: _add_conns ranks=[5,6,0,2,3,8,7,1,4] 2024-09-16T15:09:54.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9540133d10 0x7f95401cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-h con 0x7f9540133d10 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 0x7f9540132ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-d con 0x7f9540136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.849+0000 7f9546dc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9540137290 0x7f95401333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): picked mon.noname-a con 0x7f9540137290 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401cb840 con 0x7f9540137290 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95400a7510 con 0x7f9540133d10 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9540121c40 con 0x7f9540136eb0 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:54.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9546dc7640 10 monclient(hunting): authenticate will time out at 1331.440028s 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 0x7f9540132ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9540137290 0x7f95401333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 0x7f9540132ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58042/0 (socket says 172.21.15.134:58042) 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 -- 172.21.15.134:0/3717988487 learned_addr learned my addr 172.21.15.134:0/3717988487 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): get_auth_request con 0x7f9540136eb0 auth_method 0 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): get_auth_request con 0x7f9540137290 auth_method 0 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9540133d10 0x7f95401cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): get_auth_request con 0x7f9540133d10 auth_method 0 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient(hunting): handle_auth_done global_id 34899 payload 824 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9540133d10 msgr2=0x7f95401cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9540133d10 0x7f95401cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9540137290 msgr2=0x7f95401333e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9540137290 0x7f95401333e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:54.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 monclient: found mon.noname-d 2024-09-16T15:09:54.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:54.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f953ffff640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340094d0 con 0x7f9540136eb0 2024-09-16T15:09:54.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.850+0000 7f9545364640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9540137290 0x7f95401333e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:54.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953ffff640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 msgr2=0x7f9540132ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953ffff640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 0x7f9540132ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e840 tx=0x7f952c00ed00 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f952c0094f0 con 0x7f9540136eb0 2024-09-16T15:09:54.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: dump: 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.875 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 5 monclient: authenticate success, global_id 34899 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c013470 con 0x7f9540136eb0 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 10 monclient: _renew_subs 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95401362c0 con 0x7f9540136eb0 2024-09-16T15:09:54.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f952c013a50 con 0x7f9540136eb0 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953d7fa640 10 monclient: dump: 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.877 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f953effd640 4 set_mon_vals no callback set 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 10 monclient: _renew_subs 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.852+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95401235b0 con 0x7f9540136eb0 2024-09-16T15:09:54.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9546dc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9546dc7640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 0x7f954011ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9540073880 con 0x7f954011fbd0 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f952c013bb0 con 0x7f9540136eb0 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 0x7f954011ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient: get_auth_request con 0x7f954011fbd0 auth_method 0 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.853+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f953d7fa640 1 --2- 172.21.15.134:0/3717988487 >> v2:172.21.15.162:6800/980547263 conn(0x7f9524038880 0x7f952403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f9544b63640 10 monclient(hunting): handle_auth_done global_id 74222 payload 824 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f9544b63640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9540061ae0 con 0x7f954011fbd0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f952c04a3a0 con 0x7f9540136eb0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f9545364640 1 --2- 172.21.15.134:0/3717988487 >> v2:172.21.15.162:6800/980547263 conn(0x7f9524038880 0x7f952403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.854+0000 7f9545364640 10 monclient: get_auth_request con 0x7f9524038880 auth_method 0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f9544b63640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 msgr2=0x7f954011ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f9544b63640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 0x7f954011ffb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95400a67d0 tx=0x7f95340046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9534003610 con 0x7f954011fbd0 2024-09-16T15:09:54.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 10 monclient: dump: 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.881 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f9545364640 1 --2- 172.21.15.134:0/3717988487 >> v2:172.21.15.162:6800/980547263 conn(0x7f9524038880 0x7f952403ad40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95400a3870 tx=0x7f953800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9534030b30 con 0x7f954011fbd0 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 msgr2=0x7f954011ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 0x7f954011ffb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95400a67d0 tx=0x7f95340046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 msgr2=0x7f954011ffb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.856+0000 7f953d7fa640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f954011fbd0 0x7f954011ffb0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95400a67d0 tx=0x7f95340046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9546dc7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:54.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9546dc7640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 0x7f9540134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9546dc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9540071b20 con 0x7f95401204f0 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 0x7f9540134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient: get_auth_request con 0x7f95401204f0 auth_method 0 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.862+0000 7f9544b63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.863+0000 7f9544b63640 10 monclient(hunting): handle_auth_done global_id 74231 payload 824 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.863+0000 7f9544b63640 1 -- 172.21.15.134:0/3717988487 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95400a5cf0 con 0x7f95401204f0 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f9544b63640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 msgr2=0x7f9540134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:54.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f9544b63640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 0x7f9540134180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95340097c0 tx=0x7f9534008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f953403c470 con 0x7f95401204f0 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 10 monclient: dump: 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:54.884 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f953404b070 con 0x7f95401204f0 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 msgr2=0x7f9540134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 0x7f9540134180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95340097c0 tx=0x7f9534008030 comp rx=0 tx=0).stop 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 msgr2=0x7f9540134180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:54.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.865+0000 7f953d7fa640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f95401204f0 0x7f9540134180 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95340097c0 tx=0x7f9534008030 comp rx=0 tx=0).stop 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:54.886 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.887 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:54.889 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:54.890 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:54.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:54.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:54.893 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.896 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.898 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.905 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:54.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.908 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:54.909 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 >> v2:172.21.15.162:6800/980547263 conn(0x7f9524038880 msgr2=0x7f952403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 --2- 172.21.15.134:0/3717988487 >> v2:172.21.15.162:6800/980547263 conn(0x7f9524038880 0x7f952403ad40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95400a3870 tx=0x7f953800b040 comp rx=0 tx=0).stop 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 10 monclient: shutdown 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 msgr2=0x7f9540132ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 --2- 172.21.15.134:0/3717988487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9540136eb0 0x7f9540132ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e840 tx=0x7f952c00ed00 comp rx=0 tx=0).stop 2024-09-16T15:09:54.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9545364640 1 -- 172.21.15.134:0/3717988487 reap_dead start 2024-09-16T15:09:54.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9545364640 1 -- 172.21.15.134:0/3717988487 reap_dead start 2024-09-16T15:09:54.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 shutdown_connections 2024-09-16T15:09:54.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 >> 172.21.15.134:0/3717988487 conn(0x7f95400a99a0 msgr2=0x7f954006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:54.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 shutdown_connections 2024-09-16T15:09:54.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:54.866+0000 7f9546dc7640 1 -- 172.21.15.134:0/3717988487 wait complete. 2024-09-16T15:09:54.912 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:54.912 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:09:54.912 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 10 monclient: get_monmap_and_config 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 10 monclient: monmap: 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:55.010003+0000 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:55.010003+0000 2024-09-16T15:09:55.010 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:55.011 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 1 Processor -- start 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 1 -- start start 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.008+0000 7f50f87d4640 10 monclient: init 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient: _add_conns ranks=[10,2,9,5,0,8,3,7,1,4,6] 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 0x7f50f01342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-f con 0x7f50f0133f10 addr v2:172.21.15.162:3300/0 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-b con 0x7f50f0134830 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f50f0071bd0 0x7f50f01351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-a con 0x7f50f0071bd0 addr v2:172.21.15.134:3300/0 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f00a3d50 con 0x7f50f0071bd0 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f006f630 con 0x7f50f0133f10 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00a1e10 con 0x7f50f0134830 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f87d4640 10 monclient(hunting): authenticate will time out at 1331.599416s 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f50f0071bd0 0x7f50f01351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f50f0071bd0 0x7f50f01351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35780/0 (socket says 172.21.15.134:35780) 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 -- 172.21.15.134:0/411114964 learned_addr learned my addr 172.21.15.134:0/411114964 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): get_auth_request con 0x7f50f0071bd0 auth_method 0 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48072/0 (socket says 172.21.15.134:48072) 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request con 0x7f50f0134830 auth_method 0 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 1 --2- 172.21.15.134:0/411114964 >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 0x7f50f01342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 10 monclient(hunting): get_auth_request con 0x7f50f0133f10 auth_method 0 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient(hunting): handle_auth_done global_id 74250 payload 274 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 -- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 msgr2=0x7f50f0134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 -- 172.21.15.134:0/411114964 >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 msgr2=0x7f50f01342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 --2- 172.21.15.134:0/411114964 >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 0x7f50f01342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 monclient: found mon.noname-a 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6d71640 1 -- 172.21.15.134:0/411114964 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0073250 con 0x7f50f0071bd0 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.009+0000 7f50f6570640 1 --2- 172.21.15.134:0/411114964 >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 0x7f50f01342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:09:55.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f6d71640 1 -- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f0071bd0 msgr2=0x7f50f01351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f6d71640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f0071bd0 0x7f50f01351f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50f0135d70 tx=0x7f50e802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ec301a4b17b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 1 -- 172.21.15.134:0/411114964 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50e8007a70 con 0x7f50f0071bd0 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: dump: 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.025 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 1 -- 172.21.15.134:0/411114964 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e802d050 con 0x7f50f0071bd0 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f87d4640 5 monclient: authenticate success, global_id 74250 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f87d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f87d4640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 1 -- 172.21.15.134:0/411114964 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50e8007bf0 con 0x7f50f0071bd0 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f87d4640 4 set_mon_vals no callback set 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.012+0000 7f50f556e640 10 monclient: dump: 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.027 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 10 monclient: shutdown 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 -- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f0071bd0 msgr2=0x7f50f01351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f0071bd0 0x7f50f01351f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50f0135d70 tx=0x7f50e802a570 comp rx=0 tx=0).stop 2024-09-16T15:09:55.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 -- 172.21.15.134:0/411114964 shutdown_connections 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50f0071bd0 0x7f50f01351f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 --2- 172.21.15.134:0/411114964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0134830 0x7f50f0134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 --2- 172.21.15.134:0/411114964 >> v2:172.21.15.162:3300/0 conn(0x7f50f0133f10 0x7f50f01342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 -- 172.21.15.134:0/411114964 >> 172.21.15.134:0/411114964 conn(0x7f50f00a7410 msgr2=0x7f50f00a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 -- 172.21.15.134:0/411114964 shutdown_connections 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 1 -- 172.21.15.134:0/411114964 wait complete. 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.013+0000 7f50f87d4640 10 monclient: monmap: 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.029 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 Processor -- start 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 -- start start 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient: init 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient: _add_conns ranks=[4,2,5,6,8,7,1,0,3] 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-c con 0x7f50f0071bd0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 0x7f50f0132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-b con 0x7f50f0133f10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f50f0134830 0x7f50f01332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): picked mon.noname-h con 0x7f50f0134830 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f011ea80 con 0x7f50f0133f10 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50f011df70 con 0x7f50f0071bd0 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50f006c920 con 0x7f50f0134830 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f87d4640 10 monclient(hunting): authenticate will time out at 1331.604749s 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 0x7f50f0132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52832/0 (socket says 172.21.15.134:52832) 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 0x7f50f0132d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48084/0 (socket says 172.21.15.134:48084) 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 1 -- 172.21.15.134:0/2740975682 learned_addr learned my addr 172.21.15.134:0/2740975682 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): get_auth_request con 0x7f50f0071bd0 auth_method 0 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request con 0x7f50f0133f10 auth_method 0 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f6570640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.014+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6d71640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f50f0134830 0x7f50f01332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 10 monclient(hunting): handle_auth_done global_id 5215 payload 824 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f50f0134830 msgr2=0x7f50f01332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f50f0134830 0x7f50f01332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 msgr2=0x7f50f0132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 0x7f50f0132d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 monclient: found mon.noname-c 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6570640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e000d8a0 con 0x7f50f0071bd0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6d71640 10 monclient: get_auth_request con 0x7f50f0134830 auth_method 0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50f0133f10 0x7f50f0132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.015+0000 7f50f6d71640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f50f0134830 0x7f50f01332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f6570640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 msgr2=0x7f50f009d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f6570640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e810 tx=0x7f50ec00ecd0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50ec0044a0 con 0x7f50f0071bd0 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: dump: 2024-09-16T15:09:55.034 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.035 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 5 monclient: authenticate success, global_id 5215 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec00c6d0 con 0x7f50f0071bd0 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 10 monclient: _renew_subs 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50ec00ccb0 con 0x7f50f0071bd0 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e8009b30 con 0x7f50f0071bd0 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:55.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50e6ffd640 10 monclient: dump: 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.037 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f4d6d640 4 set_mon_vals no callback set 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 10 monclient: _renew_subs 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.017+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f011fe90 con 0x7f50f0071bd0 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f87d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f50ec00c840 con 0x7f50f0071bd0 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f87d4640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 0x7f50f01c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f0073880 con 0x7f50f0120510 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50e6ffd640 1 --2- 172.21.15.134:0/2740975682 >> v2:172.21.15.162:6800/980547263 conn(0x7f50d4038880 0x7f50d403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f50ec049de0 con 0x7f50f0071bd0 2024-09-16T15:09:55.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 0x7f50f01c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f6d71640 1 --2- 172.21.15.134:0/2740975682 >> v2:172.21.15.162:6800/980547263 conn(0x7f50d4038880 0x7f50d403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient: get_auth_request con 0x7f50f0120510 auth_method 0 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f6d71640 10 monclient: get_auth_request con 0x7f50d4038880 auth_method 0 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.018+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.019+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_done global_id 44668 payload 824 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.019+0000 7f50f5d6f640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50f0061ae0 con 0x7f50f0120510 2024-09-16T15:09:55.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50f6d71640 1 --2- 172.21.15.134:0/2740975682 >> v2:172.21.15.162:6800/980547263 conn(0x7f50d4038880 0x7f50d403ad40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f50f00a3370 tx=0x7f50e8009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50f5d6f640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 msgr2=0x7f50f01c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 0x7f50f01c6dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50e0015460 tx=0x7f50e0015920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50e0012040 con 0x7f50f0120510 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 10 monclient: dump: 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.040 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f50e0013040 con 0x7f50f0120510 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 msgr2=0x7f50f01c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 0x7f50f01c6dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50e0015460 tx=0x7f50e0015920 comp rx=0 tx=0).stop 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 msgr2=0x7f50f01c6dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.021+0000 7f50e6ffd640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f0120510 0x7f50f01c6dc0 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50e0015460 tx=0x7f50e0015920 comp rx=0 tx=0).stop 2024-09-16T15:09:55.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.028+0000 7f50f87d4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:55.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.028+0000 7f50f87d4640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 0x7f50f01c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.028+0000 7f50f87d4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.028+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f006a3f0 con 0x7f50f01c7300 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 0x7f50f01c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient: get_auth_request con 0x7f50f01c7300 auth_method 0 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 10 monclient(hunting): handle_auth_done global_id 44677 payload 824 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.029+0000 7f50f5d6f640 1 -- 172.21.15.134:0/2740975682 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f50f007a680 con 0x7f50f01c7300 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.031+0000 7f50f5d6f640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 msgr2=0x7f50f01c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.031+0000 7f50f5d6f640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 0x7f50f01c7710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50e0015460 tx=0x7f50e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.031+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50e0027070 con 0x7f50f01c7300 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.031+0000 7f50e6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 10 monclient: dump: 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.045 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f50e0022af0 con 0x7f50f01c7300 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 msgr2=0x7f50f01c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 0x7f50f01c7710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50e0015460 tx=0x7f50e00046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 msgr2=0x7f50f01c7710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50e6ffd640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50f01c7300 0x7f50f01c7710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:55.069 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 >> v2:172.21.15.162:6800/980547263 conn(0x7f50d4038880 msgr2=0x7f50d403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 --2- 172.21.15.134:0/2740975682 >> v2:172.21.15.162:6800/980547263 conn(0x7f50d4038880 0x7f50d403ad40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f50f00a3370 tx=0x7f50e8009000 comp rx=0 tx=0).stop 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 10 monclient: shutdown 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 msgr2=0x7f50f009d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e810 tx=0x7f50ec00ecd0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f6d71640 1 -- 172.21.15.134:0/2740975682 reap_dead start 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 shutdown_connections 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 --2- 172.21.15.134:0/2740975682 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f50f0071bd0 0x7f50f009d1d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 >> 172.21.15.134:0/2740975682 conn(0x7f50f00a7410 msgr2=0x7f50f006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 shutdown_connections 2024-09-16T15:09:55.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.032+0000 7f50f87d4640 1 -- 172.21.15.134:0/2740975682 wait complete. 2024-09-16T15:09:55.072 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:55.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:09:55.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:09:55.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.166+0000 7f8d684c9640 10 monclient: get_monmap_and_config 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.166+0000 7f8d684c9640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.167+0000 7f8d684c9640 10 monclient: monmap: 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:55.168953+0000 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:55.168953+0000 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.170 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.167+0000 7f8d684c9640 1 Processor -- start 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.167+0000 7f8d684c9640 1 -- start start 2024-09-16T15:09:55.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.167+0000 7f8d684c9640 10 monclient: init 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient: _add_conns ranks=[3,5,2,4,7,0,10,1,6,8,9] 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60136e40 0x7f8d6009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-g con 0x7f8d60136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8d60137220 0x7f8d60137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-h con 0x7f8d60137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-b con 0x7f8d60135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6009abc0 con 0x7f8d60135dc0 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6009ba50 con 0x7f8d60136e40 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d600a53c0 con 0x7f8d60137220 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d684c9640 10 monclient(hunting): authenticate will time out at 1331.758306s 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48098/0 (socket says 172.21.15.134:48098) 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 -- 172.21.15.134:0/3847215316 learned_addr learned my addr 172.21.15.134:0/3847215316 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): get_auth_request con 0x7f8d60135dc0 auth_method 0 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66265640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60136e40 0x7f8d6009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d65a64640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8d60137220 0x7f8d60137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d65a64640 10 monclient(hunting): get_auth_request con 0x7f8d60137220 auth_method 0 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d65a64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d65a64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d65a64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66265640 10 monclient(hunting): get_auth_request con 0x7f8d60136e40 auth_method 0 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66265640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66265640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66265640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient(hunting): handle_auth_done global_id 44777 payload 274 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 -- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8d60137220 msgr2=0x7f8d60137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8d60137220 0x7f8d60137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 -- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60136e40 msgr2=0x7f8d6009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60136e40 0x7f8d6009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 monclient: found mon.noname-b 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.168+0000 7f8d66a66640 1 -- 172.21.15.134:0/3847215316 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60073250 con 0x7f8d60135dc0 2024-09-16T15:09:55.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d66a66640 1 -- 172.21.15.134:0/3847215316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 msgr2=0x7f8d60136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d66a66640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009e80 tx=0x7f8d5802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd461df4b79c2939 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 1 -- 172.21.15.134:0/3847215316 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d58007a70 con 0x7f8d60135dc0 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: dump: 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.184 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 1 -- 172.21.15.134:0/3847215316 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5802d050 con 0x7f8d60135dc0 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d684c9640 5 monclient: authenticate success, global_id 44777 2024-09-16T15:09:55.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d684c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d684c9640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 1 -- 172.21.15.134:0/3847215316 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d58007bf0 con 0x7f8d60135dc0 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d684c9640 4 set_mon_vals no callback set 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.171+0000 7f8d65263640 10 monclient: dump: 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 10 monclient: shutdown 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 -- 172.21.15.134:0/3847215316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 msgr2=0x7f8d60136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009e80 tx=0x7f8d5802a530 comp rx=0 tx=0).stop 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 -- 172.21.15.134:0/3847215316 shutdown_connections 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d60135dc0 0x7f8d60136220 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8d60137220 0x7f8d60137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 --2- 172.21.15.134:0/3847215316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60136e40 0x7f8d6009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 -- 172.21.15.134:0/3847215316 >> 172.21.15.134:0/3847215316 conn(0x7f8d600a6110 msgr2=0x7f8d600a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 -- 172.21.15.134:0/3847215316 shutdown_connections 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 1 -- 172.21.15.134:0/3847215316 wait complete. 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.172+0000 7f8d684c9640 10 monclient: monmap: 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:55.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 Processor -- start 2024-09-16T15:09:55.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 -- start start 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient: init 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient: _add_conns ranks=[6,4,8,5,1,7,0,2,3] 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 0x7f8d600a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-d con 0x7f8d60135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 0x7f8d60134d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-c con 0x7f8d60136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8d60137220 0x7f8d601352c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): picked mon.noname-e con 0x7f8d60137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60136430 con 0x7f8d60136e40 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60122740 con 0x7f8d60135dc0 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8d600a8b10 con 0x7f8d60137220 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d684c9640 10 monclient(hunting): authenticate will time out at 1331.763670s 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 0x7f8d600a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 0x7f8d60134d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 0x7f8d600a27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58046/0 (socket says 172.21.15.134:58046) 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 1 -- 172.21.15.134:0/141233003 learned_addr learned my addr 172.21.15.134:0/141233003 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 0x7f8d60134d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52840/0 (socket says 172.21.15.134:52840) 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): get_auth_request con 0x7f8d60135dc0 auth_method 0 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): get_auth_request con 0x7f8d60136e40 auth_method 0 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8d60137220 0x7f8d601352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): get_auth_request con 0x7f8d60137220 auth_method 0 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66265640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.173+0000 7f8d66a66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 10 monclient(hunting): handle_auth_done global_id 34908 payload 824 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8d60137220 msgr2=0x7f8d601352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8d60137220 0x7f8d601352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 msgr2=0x7f8d60134d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 0x7f8d60134d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 monclient: found mon.noname-d 2024-09-16T15:09:55.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66265640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d58009b30 con 0x7f8d60135dc0 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d65a64640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8d60136e40 0x7f8d60134d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.174+0000 7f8d66a66640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8d60137220 0x7f8d601352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d66265640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 msgr2=0x7f8d600a27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d66265640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 0x7f8d600a27d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00d800 tx=0x7f8d5c00dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d5c00bd40 con 0x7f8d60135dc0 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: dump: 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.194 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c014430 con 0x7f8d60135dc0 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 5 monclient: authenticate success, global_id 34908 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 10 monclient: _renew_subs 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d5c014a10 con 0x7f8d60135dc0 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d600a8730 con 0x7f8d60135dc0 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d64a62640 4 set_mon_vals no callback set 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d56ffd640 10 monclient: dump: 2024-09-16T15:09:55.196 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.197 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 10 monclient: _renew_subs 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.176+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d600a3780 con 0x7f8d60135dc0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8d5c014b90 con 0x7f8d60135dc0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d684c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d684c9640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 0x7f8d6002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60071b20 con 0x7f8d60120420 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d56ffd640 1 --2- 172.21.15.134:0/141233003 >> v2:172.21.15.162:6800/980547263 conn(0x7f8d44038880 0x7f8d4403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(423..423 src has 1..423) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f8d5c04a210 con 0x7f8d60135dc0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d66a66640 1 --2- 172.21.15.134:0/141233003 >> v2:172.21.15.162:6800/980547263 conn(0x7f8d44038880 0x7f8d4403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d66a66640 10 monclient: get_auth_request con 0x7f8d44038880 auth_method 0 2024-09-16T15:09:55.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 0x7f8d6002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient: get_auth_request con 0x7f8d60120420 auth_method 0 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.177+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.178+0000 7f8d65a64640 10 monclient(hunting): handle_auth_done global_id 54345 payload 824 2024-09-16T15:09:55.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.178+0000 7f8d65a64640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d60121ea0 con 0x7f8d60120420 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d66a66640 1 --2- 172.21.15.134:0/141233003 >> v2:172.21.15.162:6800/980547263 conn(0x7f8d44038880 0x7f8d4403ad40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009e50 tx=0x7f8d58009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d65a64640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 msgr2=0x7f8d6002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d65a64640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 0x7f8d6002b260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d50010b60 tx=0x7f8d5000e5a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d50010e60 con 0x7f8d60120420 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 10 monclient: dump: 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.202 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8d5001d450 con 0x7f8d60120420 2024-09-16T15:09:55.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 msgr2=0x7f8d6002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 0x7f8d6002b260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d50010b60 tx=0x7f8d5000e5a0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 msgr2=0x7f8d6002b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.180+0000 7f8d56ffd640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120420 0x7f8d6002b260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.186+0000 7f8d684c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.186+0000 7f8d684c9640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 0x7f8d6002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.186+0000 7f8d684c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.186+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60061ae0 con 0x7f8d60120800 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 0x7f8d6002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient: get_auth_request con 0x7f8d60120800 auth_method 0 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 10 monclient(hunting): handle_auth_done global_id 54354 payload 824 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.187+0000 7f8d65a64640 1 -- 172.21.15.134:0/141233003 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8d6007a680 con 0x7f8d60120800 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.189+0000 7f8d65a64640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 msgr2=0x7f8d6002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.189+0000 7f8d65a64640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 0x7f8d6002b7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d5001a000 tx=0x7f8d500046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8d5002b070 con 0x7f8d60120800 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 10 monclient: dump: 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.205 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.206 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f8d50023040 con 0x7f8d60120800 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 msgr2=0x7f8d6002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 0x7f8d6002b7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d5001a000 tx=0x7f8d500046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 msgr2=0x7f8d6002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d56ffd640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d60120800 0x7f8d6002b7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:55.207 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:55.208 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.209 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.210 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:55.211 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.212 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.216 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:55.218 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.219 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.221 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:55.223 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.225 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.226 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.227 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:55.228 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.229 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.230 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:55.230 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 >> v2:172.21.15.162:6800/980547263 conn(0x7f8d44038880 msgr2=0x7f8d4403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 --2- 172.21.15.134:0/141233003 >> v2:172.21.15.162:6800/980547263 conn(0x7f8d44038880 0x7f8d4403ad40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009e50 tx=0x7f8d58009000 comp rx=0 tx=0).stop 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 10 monclient: shutdown 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 msgr2=0x7f8d600a27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 --2- 172.21.15.134:0/141233003 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8d60135dc0 0x7f8d600a27d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00d800 tx=0x7f8d5c00dcc0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d66a66640 1 -- 172.21.15.134:0/141233003 reap_dead start 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d66a66640 1 -- 172.21.15.134:0/141233003 reap_dead start 2024-09-16T15:09:55.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 shutdown_connections 2024-09-16T15:09:55.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 >> 172.21.15.134:0/141233003 conn(0x7f8d600a6110 msgr2=0x7f8d6009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 shutdown_connections 2024-09-16T15:09:55.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.190+0000 7f8d684c9640 1 -- 172.21.15.134:0/141233003 wait complete. 2024-09-16T15:09:55.232 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:55.232 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:09:55.232 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:09:55.239 INFO:tasks.rados.rados.0.smithi162.stdout:2135: done (3 left) 2024-09-16T15:09:55.239 INFO:tasks.rados.rados.0.smithi162.stdout:2136: done (2 left) 2024-09-16T15:09:55.239 INFO:tasks.rados.rados.0.smithi162.stdout:2137: done (1 left) 2024-09-16T15:09:55.239 INFO:tasks.rados.rados.0.smithi162.stdout:2138: done (0 left) 2024-09-16T15:09:55.239 INFO:tasks.rados.rados.0.smithi162.stdout:2139: snap_remove snap 198 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.328+0000 7f868a4e5640 10 monclient: get_monmap_and_config 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.328+0000 7f868a4e5640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.328+0000 7f868a4e5640 10 monclient: monmap: 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:55.330731+0000 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:55.330731+0000 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:55.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:55.332 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.329+0000 7f868a4e5640 1 Processor -- start 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.329+0000 7f868a4e5640 1 -- start start 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.329+0000 7f868a4e5640 10 monclient: init 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient: _add_conns ranks=[0,6,8,1,9,4,3,7,10,2,5] 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8684136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 0x7f8684137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-d con 0x7f8684137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 0x7f868413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-e con 0x7f868413c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:55.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8684073250 con 0x7f8684136e40 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f86840a56e0 con 0x7f8684137220 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f868409abc0 con 0x7f868413c370 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f868a4e5640 10 monclient(hunting): authenticate will time out at 1331.920293s 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 0x7f8684137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 0x7f868413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8683fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8684136e40 0x7f868409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50876/0 (socket says 172.21.15.134:50876) 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8683fff640 1 -- 172.21.15.134:0/574170223 learned_addr learned my addr 172.21.15.134:0/574170223 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): get_auth_request con 0x7f8684137220 auth_method 0 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): get_auth_request con 0x7f868413c370 auth_method 0 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient(hunting): handle_auth_done global_id 74249 payload 274 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 -- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 msgr2=0x7f8684137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 0x7f8684137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 -- 172.21.15.134:0/574170223 >> v1:172.21.15.134:6789/0 conn(0x7f8684136e40 legacy=0x7f868409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 monclient: found mon.noname-e 2024-09-16T15:09:55.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:09:55.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f8688a82640 1 -- 172.21.15.134:0/574170223 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868409ba50 con 0x7f868413c370 2024-09-16T15:09:55.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.330+0000 7f867bfff640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 0x7f8684137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f8688a82640 1 -- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 msgr2=0x7f868413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f8688a82640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 0x7f868413e760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f867c009dc0 tx=0x7f867c02a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7baa67ea7270d061 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 1 -- 172.21.15.134:0/574170223 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867c007bc0 con 0x7f868413c370 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: dump: 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f868a4e5640 5 monclient: authenticate success, global_id 74249 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 1 -- 172.21.15.134:0/574170223 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c02d050 con 0x7f868413c370 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f868a4e5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f868a4e5640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f868a4e5640 4 set_mon_vals no callback set 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 1 -- 172.21.15.134:0/574170223 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867c007d40 con 0x7f868413c370 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.333+0000 7f86837fe640 10 monclient: dump: 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.347 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 10 monclient: shutdown 2024-09-16T15:09:55.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 -- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 msgr2=0x7f868413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 0x7f868413e760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f867c009dc0 tx=0x7f867c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 -- 172.21.15.134:0/574170223 shutdown_connections 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f868413c370 0x7f868413e760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 --2- 172.21.15.134:0/574170223 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8684137220 0x7f8684137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 -- 172.21.15.134:0/574170223 >> 172.21.15.134:0/574170223 conn(0x7f86840a6110 msgr2=0x7f86840a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 -- 172.21.15.134:0/574170223 shutdown_connections 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 1 -- 172.21.15.134:0/574170223 wait complete. 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.334+0000 7f868a4e5640 10 monclient: monmap: 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:55.349 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 Processor -- start 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 -- start start 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient: init 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient: _add_conns ranks=[2,4,1,5,6,0,8,7,3] 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 0x7f8684135720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-b con 0x7f8684136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 0x7f8684135c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-c con 0x7f8684137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868413c370 0x7f86841361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): picked mon.noname-f con 0x7f868413c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86841226d0 con 0x7f868413c370 2024-09-16T15:09:55.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840a7af0 con 0x7f8684136e40 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86840a75a0 con 0x7f8684137220 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f868a4e5640 10 monclient(hunting): authenticate will time out at 1331.925691s 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 0x7f8684135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 0x7f8684135c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 0x7f8684135720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48108/0 (socket says 172.21.15.134:48108) 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 0x7f8684135c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52844/0 (socket says 172.21.15.134:52844) 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 1 -- 172.21.15.134:0/2543408273 learned_addr learned my addr 172.21.15.134:0/2543408273 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): get_auth_request con 0x7f8684136e40 auth_method 0 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): get_auth_request con 0x7f8684137220 auth_method 0 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f867bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8688a82640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868413c370 0x7f86841361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8688a82640 10 monclient(hunting): get_auth_request con 0x7f868413c370 auth_method 0 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8688a82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8688a82640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.335+0000 7f8688a82640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 10 monclient(hunting): handle_auth_done global_id 5233 payload 824 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 msgr2=0x7f8684135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 0x7f8684135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868413c370 msgr2=0x7f86841361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868413c370 0x7f86841361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 monclient: found mon.noname-c 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f867bfff640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c009400 con 0x7f8684137220 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f8683fff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8684136e40 0x7f8684135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.336+0000 7f8688a82640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868413c370 0x7f86841361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:09:55.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f867bfff640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 msgr2=0x7f8684135c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f867bfff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 0x7f8684135c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f866c00e930 tx=0x7f866c00edf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f866c00cc10 con 0x7f8684137220 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: dump: 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.355 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c00cd90 con 0x7f8684137220 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 5 monclient: authenticate success, global_id 5233 2024-09-16T15:09:55.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 10 monclient: _renew_subs 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c009a70 con 0x7f8684137220 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f866c013d50 con 0x7f8684137220 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f8682ffd640 4 set_mon_vals no callback set 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f86817fa640 10 monclient: dump: 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.357 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 10 monclient: _renew_subs 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.338+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840a3a60 con 0x7f8684137220 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f868a4e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f868a4e5640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 0x7f868402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8684071b20 con 0x7f8684120420 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f866c013470 con 0x7f8684137220 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f86817fa640 1 --2- 172.21.15.134:0/2543408273 >> v2:172.21.15.162:6800/980547263 conn(0x7f8664038880 0x7f866403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 0x7f868402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient: get_auth_request con 0x7f8684120420 auth_method 0 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f866c04a410 con 0x7f8684137220 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8688a82640 1 --2- 172.21.15.134:0/2543408273 >> v2:172.21.15.162:6800/980547263 conn(0x7f8664038880 0x7f866403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.339+0000 7f8688a82640 10 monclient: get_auth_request con 0x7f8664038880 auth_method 0 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.340+0000 7f8683fff640 10 monclient(hunting): handle_auth_done global_id 44618 payload 824 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.340+0000 7f8683fff640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8684122840 con 0x7f8684120420 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f8683fff640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 msgr2=0x7f868402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f8683fff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 0x7f868402b260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86740072e0 tx=0x7f86740077a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f867400fd40 con 0x7f8684120420 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 10 monclient: dump: 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.360 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f8688a82640 1 --2- 172.21.15.134:0/2543408273 >> v2:172.21.15.162:6800/980547263 conn(0x7f8664038880 0x7f866403ad40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f867c009dc0 tx=0x7f867c02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f867400fea0 con 0x7f8684120420 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 msgr2=0x7f868402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 0x7f868402b260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86740072e0 tx=0x7f86740077a0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 msgr2=0x7f868402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.342+0000 7f86817fa640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120420 0x7f868402b260 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86740072e0 tx=0x7f86740077a0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.348+0000 7f868a4e5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:55.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.348+0000 7f868a4e5640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 0x7f868402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.348+0000 7f868a4e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.348+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86840677b0 con 0x7f8684120800 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 0x7f868402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient: get_auth_request con 0x7f8684120800 auth_method 0 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 10 monclient(hunting): handle_auth_done global_id 44627 payload 824 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.349+0000 7f8683fff640 1 -- 172.21.15.134:0/2543408273 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8684061ae0 con 0x7f8684120800 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.351+0000 7f8683fff640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 msgr2=0x7f868402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.351+0000 7f8683fff640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 0x7f868402b7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8674006bf0 tx=0x7f8674004940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8674023070 con 0x7f8684120800 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 10 monclient: dump: 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.365 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f8674006240 con 0x7f8684120800 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 msgr2=0x7f868402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 0x7f868402b7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8674006bf0 tx=0x7f8674004940 comp rx=0 tx=0).stop 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 msgr2=0x7f868402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.352+0000 7f86817fa640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8684120800 0x7f868402b7a0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8674006bf0 tx=0x7f8674004940 comp rx=0 tx=0).stop 2024-09-16T15:09:55.366 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:55.367 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.368 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.369 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.370 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.373 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:55.375 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.377 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.380 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.381 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.382 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.383 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.388 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:55.389 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:55.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 >> v2:172.21.15.162:6800/980547263 conn(0x7f8664038880 msgr2=0x7f866403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 --2- 172.21.15.134:0/2543408273 >> v2:172.21.15.162:6800/980547263 conn(0x7f8664038880 0x7f866403ad40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f867c009dc0 tx=0x7f867c02f000 comp rx=0 tx=0).stop 2024-09-16T15:09:55.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 10 monclient: shutdown 2024-09-16T15:09:55.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 msgr2=0x7f8684135c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 --2- 172.21.15.134:0/2543408273 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8684137220 0x7f8684135c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f866c00e930 tx=0x7f866c00edf0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f8688a82640 1 -- 172.21.15.134:0/2543408273 reap_dead start 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f8688a82640 1 -- 172.21.15.134:0/2543408273 reap_dead start 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 shutdown_connections 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 >> 172.21.15.134:0/2543408273 conn(0x7f86840a6110 msgr2=0x7f868413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 shutdown_connections 2024-09-16T15:09:55.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.353+0000 7f868a4e5640 1 -- 172.21.15.134:0/2543408273 wait complete. 2024-09-16T15:09:55.391 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:55.391 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:09:55.392 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 10 monclient: get_monmap_and_config 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 10 monclient: monmap: 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:55.489756+0000 2024-09-16T15:09:55.490 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:55.489756+0000 2024-09-16T15:09:55.491 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.491 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:55.492 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 1 Processor -- start 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 1 -- start start 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.488+0000 7fc150fae640 10 monclient: init 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient: _add_conns ranks=[8,4,2,10,7,6,5,3,9,0,1] 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-e con 0x7fc14c133f10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:55.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 0x7fc14c134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-c con 0x7fc14c134830 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 0x7fc14c1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-b con 0x7fc14c071bd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0a3d50 con 0x7fc14c071bd0 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c06f630 con 0x7fc14c134830 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0a1e10 con 0x7fc14c133f10 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc150fae640 10 monclient(hunting): authenticate will time out at 1332.079103s 2024-09-16T15:09:55.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 0x7fc14c134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:33546/0 (socket says 172.21.15.134:33546) 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 1 -- 172.21.15.134:0/1039769538 learned_addr learned my addr 172.21.15.134:0/1039769538 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): get_auth_request con 0x7fc14c133f10 auth_method 0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): get_auth_request con 0x7fc14c134830 auth_method 0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 0x7fc14c1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): get_auth_request con 0x7fc14c071bd0 auth_method 0 2024-09-16T15:09:55.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient(hunting): handle_auth_done global_id 5242 payload 274 2024-09-16T15:09:55.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 -- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 msgr2=0x7fc14c1342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 -- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 msgr2=0x7fc14c1351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 0x7fc14c1351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 monclient: found mon.noname-c 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc149d9b640 1 -- 172.21.15.134:0/1039769538 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c073250 con 0x7fc14c134830 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14a59c640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.489+0000 7fc14ad9d640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 0x7fc14c1351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc149d9b640 1 -- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 msgr2=0x7fc14c134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc149d9b640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 0x7fc14c134cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc14000dbf0 tx=0x7fc14002e2a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=dd8c57bfdb7c113e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 1 -- 172.21.15.134:0/1039769538 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc14000d2e0 con 0x7fc14c134830 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: dump: 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.507 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 1 -- 172.21.15.134:0/1039769538 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14002edf0 con 0x7fc14c134830 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 5 monclient: authenticate success, global_id 5242 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 4 set_mon_vals no callback set 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 1 -- 172.21.15.134:0/1039769538 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc14000bd30 con 0x7fc14c134830 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc14959a640 10 monclient: dump: 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.509 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 10 monclient: shutdown 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.492+0000 7fc150fae640 1 -- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 msgr2=0x7fc14c134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 0x7fc14c134cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc14000dbf0 tx=0x7fc14002e2a0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 -- 172.21.15.134:0/1039769538 shutdown_connections 2024-09-16T15:09:55.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc14c071bd0 0x7fc14c1351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c134830 0x7fc14c134cb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 --2- 172.21.15.134:0/1039769538 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc14c133f10 0x7fc14c1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 -- 172.21.15.134:0/1039769538 >> 172.21.15.134:0/1039769538 conn(0x7fc14c0a7410 msgr2=0x7fc14c0a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 -- 172.21.15.134:0/1039769538 shutdown_connections 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 1 -- 172.21.15.134:0/1039769538 wait complete. 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.493+0000 7fc150fae640 10 monclient: monmap: 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.511 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 Processor -- start 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 -- start start 2024-09-16T15:09:55.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient: init 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient: _add_conns ranks=[5,4,0,8,3,7,2,6,1] 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc14c071bd0 0x7fc14c1c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-h con 0x7fc14c071bd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 0x7fc14c132000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-c con 0x7fc14c133f10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c134830 0x7fc14c132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): picked mon.noname-a con 0x7fc14c134830 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1211d0 con 0x7fc14c134830 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c06cb00 con 0x7fc14c133f10 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c121700 con 0x7fc14c071bd0 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc150fae640 10 monclient(hunting): authenticate will time out at 1332.084687s 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 0x7fc14c132000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 0x7fc14c132000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52864/0 (socket says 172.21.15.134:52864) 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 1 -- 172.21.15.134:0/3525634126 learned_addr learned my addr 172.21.15.134:0/3525634126 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): get_auth_request con 0x7fc14c133f10 auth_method 0 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c134830 0x7fc14c132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): get_auth_request con 0x7fc14c134830 auth_method 0 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc149d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc14c071bd0 0x7fc14c1c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14a59c640 10 monclient(hunting): get_auth_request con 0x7fc14c071bd0 auth_method 0 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.494+0000 7fc14a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 10 monclient(hunting): handle_auth_done global_id 5251 payload 824 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc14c071bd0 msgr2=0x7fc14c1c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc14c071bd0 0x7fc14c1c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c134830 msgr2=0x7fc14c132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c134830 0x7fc14c132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 monclient: found mon.noname-c 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc149d9b640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009a70 con 0x7fc14c133f10 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc14c071bd0 0x7fc14c1c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.495+0000 7fc14ad9d640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc14c134830 0x7fc14c132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc149d9b640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 msgr2=0x7fc14c132000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc149d9b640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 0x7fc14c132000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc14002e7c0 tx=0x7fc1400046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc14000d2e0 con 0x7fc14c133f10 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: dump: 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.517 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 5 monclient: authenticate success, global_id 5251 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc140003de0 con 0x7fc14c133f10 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 10 monclient: _renew_subs 2024-09-16T15:09:55.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1440094d0 con 0x7fc14c133f10 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc14000bd30 con 0x7fc14c133f10 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc13affd640 10 monclient: dump: 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.519 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 10 monclient: _renew_subs 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14000d8a0 con 0x7fc14c133f10 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.497+0000 7fc148d99640 4 set_mon_vals no callback set 2024-09-16T15:09:55.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc150fae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc150fae640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 0x7fc14c11e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc14000b450 con 0x7fc14c133f10 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c06a3f0 con 0x7fc14c11dc70 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc13affd640 1 --2- 172.21.15.134:0/3525634126 >> v2:172.21.15.162:6800/980547263 conn(0x7fc12c038880 0x7fc12c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 0x7fc14c11e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient: get_auth_request con 0x7fc14c11dc70 auth_method 0 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc14006f0a0 con 0x7fc14c133f10 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14ad9d640 1 --2- 172.21.15.134:0/3525634126 >> v2:172.21.15.162:6800/980547263 conn(0x7fc12c038880 0x7fc12c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14ad9d640 10 monclient: get_auth_request con 0x7fc12c038880 auth_method 0 2024-09-16T15:09:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 10 monclient(hunting): handle_auth_done global_id 54430 payload 824 2024-09-16T15:09:55.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.498+0000 7fc14a59c640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc14c061ae0 con 0x7fc14c11dc70 2024-09-16T15:09:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc14a59c640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 msgr2=0x7fc14c11e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 0x7fc14c11e050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc144011cf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc14ad9d640 1 --2- 172.21.15.134:0/3525634126 >> v2:172.21.15.162:6800/980547263 conn(0x7fc12c038880 0x7fc12c03ad40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009fd0 tx=0x7fc13c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc1440044a0 con 0x7fc14c11dc70 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 10 monclient: dump: 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.525 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc14400fc50 con 0x7fc14c11dc70 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 msgr2=0x7fc14c11e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 0x7fc14c11e050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc144011cf0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 msgr2=0x7fc14c11e050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.501+0000 7fc13affd640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11dc70 0x7fc14c11e050 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc144011cf0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.507+0000 7fc150fae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.507+0000 7fc150fae640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 0x7fc14c09e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.507+0000 7fc150fae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.507+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c04e3b0 con 0x7fc14c11e590 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 0x7fc14c09e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient: get_auth_request con 0x7fc14c11e590 auth_method 0 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 10 monclient(hunting): handle_auth_done global_id 54439 payload 824 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.508+0000 7fc14a59c640 1 -- 172.21.15.134:0/3525634126 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc14c07a680 con 0x7fc14c11e590 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc14a59c640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 msgr2=0x7fc14c09e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc14a59c640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 0x7fc14c09e810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc1440046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc144003e10 con 0x7fc14c11e590 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc13affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc13affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.510+0000 7fc13affd640 10 monclient: dump: 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fc144015460 con 0x7fc14c11e590 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:09:55.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 msgr2=0x7fc14c09e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 0x7fc14c09e810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc1440046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 msgr2=0x7fc14c09e810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc13affd640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc14c11e590 0x7fc14c09e810 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc144011830 tx=0x7fc1440046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 86, 2024-09-16T15:09:55.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.531 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.532 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.533 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:09:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:09:55.535 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:09:55.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:09:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:09:55.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:09:55.546 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:09:55.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:09:55.552 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:09:55.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:09:55.553 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:09:55.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:09:55.553 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:09:55.553 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:09:55.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 >> v2:172.21.15.162:6800/980547263 conn(0x7fc12c038880 msgr2=0x7fc12c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 --2- 172.21.15.134:0/3525634126 >> v2:172.21.15.162:6800/980547263 conn(0x7fc12c038880 0x7fc12c03ad40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009fd0 tx=0x7fc13c00b040 comp rx=0 tx=0).stop 2024-09-16T15:09:55.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 10 monclient: shutdown 2024-09-16T15:09:55.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 msgr2=0x7fc14c132000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 --2- 172.21.15.134:0/3525634126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc14c133f10 0x7fc14c132000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc14002e7c0 tx=0x7fc1400046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc14ad9d640 1 -- 172.21.15.134:0/3525634126 reap_dead start 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc14ad9d640 1 -- 172.21.15.134:0/3525634126 reap_dead start 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 shutdown_connections 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 >> 172.21.15.134:0/3525634126 conn(0x7fc14c0a7410 msgr2=0x7fc14c133070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 shutdown_connections 2024-09-16T15:09:55.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.511+0000 7fc150fae640 1 -- 172.21.15.134:0/3525634126 wait complete. 2024-09-16T15:09:55.555 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:09:55.555 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g', 'e'] 2024-09-16T15:09:55.556 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:09:55.556 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-16T15:09:55.556 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:09:55.556 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-16T15:09:55.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:09:55.568 INFO:tasks.ceph.mon.g:Stopped 2024-09-16T15:09:55.568 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-16T15:09:55.568 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:09:55.569 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-16T15:09:55.569 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:09:55.579 INFO:tasks.ceph.mon.e:Stopped 2024-09-16T15:09:55.579 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-16T15:09:55.579 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:55.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: get_monmap_and_config 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: monmap: 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:55.715367+0000 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:55.715367+0000 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:55.716 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.717 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:55.718 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:55.718 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 Processor -- start 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 -- start start 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: init 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient: _add_conns ranks=[6,2,3,7,9,1,0,4,8,10,5] 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): picked mon.noname-d con 0x7f705c136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): picked mon.noname-b con 0x7f705c09bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 0x7f705c1370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): picked mon.noname-g con 0x7f705c136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:55.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c09b600 con 0x7f705c09bfe0 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0a5740 con 0x7f705c136c60 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0a52c0 con 0x7f705c136030 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f70646be640 10 monclient(hunting): authenticate will time out at 1332.304762s 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:48128/0 (socket says 172.21.15.134:48128) 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:58056/0 (socket says 172.21.15.134:58056) 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 1 -- 172.21.15.134:0/129804487 learned_addr learned my addr 172.21.15.134:0/129804487 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 10 monclient(hunting): get_auth_request con 0x7f705c09bfe0 auth_method 0 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7061c59640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 10 monclient(hunting): get_auth_request con 0x7f705c136030 auth_method 0 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f706245a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7062c5b640 1 -- 172.21.15.134:0/129804487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 msgr2=0x7f705c1370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.714+0000 7f7062c5b640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 0x7f705c1370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f706245a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f706245a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f706245a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient(hunting): handle_auth_done global_id 44804 payload 274 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 -- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 msgr2=0x7f705c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 -- 172.21.15.134:0/129804487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 msgr2=0x7f705c1370c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 0x7f705c1370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 monclient: found mon.noname-b 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f7061c59640 1 -- 172.21.15.134:0/129804487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c073250 con 0x7f705c09bfe0 2024-09-16T15:09:55.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.715+0000 7f706245a640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.717+0000 7f7061c59640 1 -- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 msgr2=0x7f705c09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.717+0000 7f7061c59640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f704c0098a0 tx=0x7f704c02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4497175aa4a15c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 1 -- 172.21.15.134:0/129804487 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f704c02ae00 con 0x7f705c09bfe0 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: dump: 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.732 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 1 -- 172.21.15.134:0/129804487 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0307b0 con 0x7f705c09bfe0 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 5 monclient: authenticate success, global_id 44804 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 1 -- 172.21.15.134:0/129804487 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f704c030d90 con 0x7f705c09bfe0 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 4 set_mon_vals no callback set 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f7061458640 10 monclient: dump: 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.734 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 10 monclient: shutdown 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 -- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 msgr2=0x7f705c09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f704c0098a0 tx=0x7f704c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 -- 172.21.15.134:0/129804487 shutdown_connections 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f705c136c60 0x7f705c1370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f705c09bfe0 0x7f705c09c400 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 --2- 172.21.15.134:0/129804487 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f705c136030 0x7f705c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 -- 172.21.15.134:0/129804487 >> 172.21.15.134:0/129804487 conn(0x7f705c0a9ea0 msgr2=0x7f705c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 -- 172.21.15.134:0/129804487 shutdown_connections 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.718+0000 7f70646be640 1 -- 172.21.15.134:0/129804487 wait complete. 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.719+0000 7f70646be640 10 monclient: build_initial_monmap 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.719+0000 7f70646be640 10 monclient: monmap: 2024-09-16T15:09:55.736 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:55.737 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.719+0000 7f70646be640 1 Processor -- start 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.719+0000 7f70646be640 1 -- start start 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.719+0000 7f70646be640 10 monclient: init 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient: _add_conns ranks=[5,0,4,7,3,2,8,6,1] 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f705c09bfe0 0x7f705c0a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): picked mon.noname-h con 0x7f705c09bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): picked mon.noname-a con 0x7f705c136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): picked mon.noname-c con 0x7f705c136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c11ff90 con 0x7f705c136030 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f705c122340 con 0x7f705c136c60 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f705c121d80 con 0x7f705c09bfe0 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f70646be640 10 monclient(hunting): authenticate will time out at 1332.310204s 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35788/0 (socket says 172.21.15.134:35788) 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:52870/0 (socket says 172.21.15.134:52870) 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 1 -- 172.21.15.134:0/905267658 learned_addr learned my addr 172.21.15.134:0/905267658 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): get_auth_request con 0x7f705c136c60 auth_method 0 2024-09-16T15:09:55.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): get_auth_request con 0x7f705c136030 auth_method 0 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:55.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f706245a640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f705c09bfe0 0x7f705c0a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient(hunting): handle_auth_done global_id 5260 payload 824 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 -- 172.21.15.134:0/905267658 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f705c09bfe0 msgr2=0x7f705c0a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f705c09bfe0 0x7f705c0a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 -- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 msgr2=0x7f705c134c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient: _finish_hunting 0 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 monclient: found mon.noname-c 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7062c5b640 1 -- 172.21.15.134:0/905267658 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058009400 con 0x7f705c136c60 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.720+0000 7f7061c59640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7062c5b640 1 -- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 msgr2=0x7f705c1351a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:55.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7062c5b640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f705400bdc0 tx=0x7f7054009eb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7054006960 con 0x7f705c136c60 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 10 monclient: dump: 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.742 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 10 monclient: _finish_auth 0 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.722+0000 7f7052ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400f070 con 0x7f705c136c60 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 5 monclient: authenticate success, global_id 5260 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _renew_subs 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c0094d0 con 0x7f705c136c60 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7054006b20 con 0x7f705c136c60 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 10 monclient: dump: 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:55.744 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7060c57640 4 set_mon_vals no callback set 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _renew_subs 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c0a3020 con 0x7f705c136c60 2024-09-16T15:09:55.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f70540064f0 con 0x7f705c136c60 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.723+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c1224b0 con 0x7f705c136c60 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.724+0000 7f7052ffd640 1 --2- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 0x7f704003ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.724+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f705404acb0 con 0x7f705c136c60 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.724+0000 7f706245a640 1 --2- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 0x7f704003ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.724+0000 7f706245a640 10 monclient: get_auth_request con 0x7f70400388e0 auth_method 0 2024-09-16T15:09:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.726+0000 7f706245a640 1 --2- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 0x7f704003ada0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7058009b80 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.729+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f70540067a0 con 0x7f705c136c60 2024-09-16T15:09:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.729+0000 7f7052ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.729+0000 7f7052ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:55.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.954+0000 7f70646be640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.954+0000 7f70646be640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.954+0000 7f70646be640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.954+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f705c120ba0 con 0x7f705c136c60 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.955+0000 7f7052ffd640 1 -- 172.21.15.134:0/905267658 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f7054012b20 con 0x7f705c136c60 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.955+0000 7f7052ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:55.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.955+0000 7f7052ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:55.958 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:55.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 msgr2=0x7f704003ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 0x7f704003ada0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7058009b80 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-09-16T15:09:55.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 10 monclient: shutdown 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 msgr2=0x7f705c1351a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f705400bdc0 tx=0x7f7054009eb0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 shutdown_connections 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> v2:172.21.15.162:6800/980547263 conn(0x7f70400388e0 0x7f704003ada0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f705c136c60 0x7f705c1351a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f705c136030 0x7f705c134c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 --2- 172.21.15.134:0/905267658 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f705c09bfe0 0x7f705c0a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 >> 172.21.15.134:0/905267658 conn(0x7f705c0a9ea0 msgr2=0x7f705c1bee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 shutdown_connections 2024-09-16T15:09:55.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:55.957+0000 7f70646be640 1 -- 172.21.15.134:0/905267658 wait complete. 2024-09-16T15:09:58.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:09:59.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.078+0000 7faa9f6c9640 10 monclient: get_monmap_and_config 2024-09-16T15:09:59.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.078+0000 7faa9f6c9640 10 monclient: build_initial_monmap 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.078+0000 7faa9f6c9640 10 monclient: monmap: 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:09:59.079424+0000 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:09:59.079424+0000 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:59.080 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:09:59.081 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 Processor -- start 2024-09-16T15:09:59.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 -- start start 2024-09-16T15:09:59.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient: init 2024-09-16T15:09:59.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient: _add_conns ranks=[3,2,8,9,4,7,0,1,5,6,10] 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 0x7faa98136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-g con 0x7faa98136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-b con 0x7faa9809bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 0x7faa981370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-e con 0x7faa98136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9809b600 con 0x7faa9809bfe0 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980a5740 con 0x7faa98136030 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faa980a52c0 con 0x7faa98136c60 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9f6c9640 10 monclient(hunting): authenticate will time out at 1335.668810s 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9dc66640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 msgr2=0x7faa981370c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9dc66640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 0x7faa981370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9d465640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 msgr2=0x7faa98136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9d465640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 0x7faa98136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59324/0 (socket says 172.21.15.134:59324) 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 1 -- 172.21.15.134:0/1555574022 learned_addr learned my addr 172.21.15.134:0/1555574022 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 10 monclient(hunting): get_auth_request con 0x7faa9809bfe0 auth_method 0 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.079+0000 7faa9cc64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:59.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient(hunting): handle_auth_done global_id 44813 payload 274 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 -- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 msgr2=0x7faa981370c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 0x7faa981370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 -- 172.21.15.134:0/1555574022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 msgr2=0x7faa98136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 0x7faa98136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient: _finish_hunting 0 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 monclient: found mon.noname-b 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:09:59.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.080+0000 7faa9cc64640 1 -- 172.21.15.134:0/1555574022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98073250 con 0x7faa9809bfe0 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9cc64640 1 -- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 msgr2=0x7faa9809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9cc64640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa880098a0 tx=0x7faa8802a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324fa9ccceebb32a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 1 -- 172.21.15.134:0/1555574022 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa880307b0 con 0x7faa9809bfe0 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: dump: 2024-09-16T15:09:59.092 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:59.093 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: _finish_auth 0 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 1 -- 172.21.15.134:0/1555574022 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88030930 con 0x7faa9809bfe0 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 5 monclient: authenticate success, global_id 44813 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 1 -- 172.21.15.134:0/1555574022 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa880313a0 con 0x7faa9809bfe0 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 10 monclient: get_monmap_and_config success 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 4 set_mon_vals no callback set 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa8ffff640 10 monclient: dump: 2024-09-16T15:09:59.094 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 10 monclient: shutdown 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 -- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 msgr2=0x7faa9809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa880098a0 tx=0x7faa8802a0b0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 -- 172.21.15.134:0/1555574022 shutdown_connections 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faa98136c60 0x7faa981370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa9809bfe0 0x7faa9809c400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/1555574022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa98136030 0x7faa98136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 -- 172.21.15.134:0/1555574022 >> 172.21.15.134:0/1555574022 conn(0x7faa980a9ea0 msgr2=0x7faa980a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 -- 172.21.15.134:0/1555574022 shutdown_connections 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.083+0000 7faa9f6c9640 1 -- 172.21.15.134:0/1555574022 wait complete. 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.084+0000 7faa9f6c9640 10 monclient: build_initial_monmap 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.084+0000 7faa9f6c9640 10 monclient: monmap: 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:09:59.096 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.084+0000 7faa9f6c9640 1 Processor -- start 2024-09-16T15:09:59.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.084+0000 7faa9f6c9640 1 -- start start 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.084+0000 7faa9f6c9640 10 monclient: init 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient: _add_conns ranks=[1,4,2,0,8,7,6,5,3] 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa9809bfe0 0x7faa98134b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-f con 0x7faa9809bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-c con 0x7faa98136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98136c60 0x7faa9811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): picked mon.noname-b con 0x7faa98136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9809ff20 con 0x7faa9809bfe0 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980a7ef0 con 0x7faa98136c60 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:09:59.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa980a7b30 con 0x7faa98136030 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9f6c9640 10 monclient(hunting): authenticate will time out at 1335.674362s 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98136c60 0x7faa9811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:32944/0 (socket says 172.21.15.134:32944) 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 -- 172.21.15.134:0/673251361 learned_addr learned my addr 172.21.15.134:0/673251361 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): get_auth_request con 0x7faa98136030 auth_method 0 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): get_auth_request con 0x7faa98136c60 auth_method 0 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:59.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9dc66640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9d465640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa9809bfe0 0x7faa98134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient(hunting): handle_auth_done global_id 5269 payload 824 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 -- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98136c60 msgr2=0x7faa9811fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98136c60 0x7faa9811fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 -- 172.21.15.134:0/673251361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa9809bfe0 msgr2=0x7faa98134b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa9809bfe0 0x7faa98134b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient: _finish_hunting 0 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 monclient: found mon.noname-c 2024-09-16T15:09:59.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.085+0000 7faa9cc64640 1 -- 172.21.15.134:0/673251361 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa880094d0 con 0x7faa98136030 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.087+0000 7faa9cc64640 1 -- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 msgr2=0x7faa98135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.087+0000 7faa9cc64640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa88000c00 tx=0x7faa880046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa88030770 con 0x7faa98136030 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: dump: 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:59.101 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: _finish_auth 0 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88004460 con 0x7faa98136030 2024-09-16T15:09:59.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 5 monclient: authenticate success, global_id 5269 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 10 monclient: _renew_subs 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98120d10 con 0x7faa98136030 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faa88032440 con 0x7faa98136030 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8dffb640 10 monclient: dump: 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:09:59.103 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 10 monclient: _renew_subs 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa8f7fe640 4 set_mon_vals no callback set 2024-09-16T15:09:59.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.088+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa98122510 con 0x7faa98136030 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7faa88039070 con 0x7faa98136030 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9f6c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9f6c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9f6c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9809c400 con 0x7faa98136030 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa8dffb640 1 --2- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 0x7faa7c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7faa8806a410 con 0x7faa98136030 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9d465640 1 --2- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 0x7faa7c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:09:59.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.089+0000 7faa9d465640 10 monclient: get_auth_request con 0x7faa7c0388b0 auth_method 0 2024-09-16T15:09:59.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.092+0000 7faa9d465640 1 --2- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 0x7faa7c03ad70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faa94003930 tx=0x7faa94008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:09:59.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.094+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7faa8803a970 con 0x7faa98136030 2024-09-16T15:09:59.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.094+0000 7faa8dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:09:59.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.094+0000 7faa8dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.317+0000 7faa9f6c9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.317+0000 7faa9f6c9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.317+0000 7faa9f6c9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.317+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faa980008d0 con 0x7faa98136030 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.318+0000 7faa8dffb640 1 -- 172.21.15.134:0/673251361 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7faa88037080 con 0x7faa98136030 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.318+0000 7faa8dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:09:59.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.318+0000 7faa8dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:09:59.320 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:09:59.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 msgr2=0x7faa7c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:59.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 0x7faa7c03ad70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faa94003930 tx=0x7faa94008040 comp rx=0 tx=0).stop 2024-09-16T15:09:59.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 10 monclient: shutdown 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 msgr2=0x7faa98135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.319+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faa88000c00 tx=0x7faa880046c0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 shutdown_connections 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> v2:172.21.15.162:6800/980547263 conn(0x7faa7c0388b0 0x7faa7c03ad70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98136c60 0x7faa9811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faa98136030 0x7faa98135090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 --2- 172.21.15.134:0/673251361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa9809bfe0 0x7faa98134b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 >> 172.21.15.134:0/673251361 conn(0x7faa980a9ea0 msgr2=0x7faa981bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 shutdown_connections 2024-09-16T15:09:59.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:09:59.320+0000 7faa9f6c9640 1 -- 172.21.15.134:0/673251361 wait complete. 2024-09-16T15:10:02.333 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:10:02.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.440+0000 7fd5bc5e2640 10 monclient: get_monmap_and_config 2024-09-16T15:10:02.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.440+0000 7fd5bc5e2640 10 monclient: build_initial_monmap 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.440+0000 7fd5bc5e2640 10 monclient: monmap: 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:02.441371+0000 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:02.441371+0000 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:02.442 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:02.443 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 Processor -- start 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 -- start start 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient: init 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient: _add_conns ranks=[8,5,4,6,0,9,7,3,1,2,10] 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 0x7fd5b4136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-e con 0x7fd5b4136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:02.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 0x7fd5b409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-h con 0x7fd5b409bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-c con 0x7fd5b4136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b409b600 con 0x7fd5b4136c60 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40a5740 con 0x7fd5b409bfe0 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40a52c0 con 0x7fd5b4136030 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bc5e2640 10 monclient(hunting): authenticate will time out at 1339.030796s 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5ba37e640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 msgr2=0x7fd5b4136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5ba37e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 0x7fd5b4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:02.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:32952/0 (socket says 172.21.15.134:32952) 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 1 -- 172.21.15.134:0/4266166203 learned_addr learned my addr 172.21.15.134:0/4266166203 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 10 monclient(hunting): get_auth_request con 0x7fd5b4136c60 auth_method 0 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.441+0000 7fd5bab7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5b9b7d640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 0x7fd5b409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_done global_id 5278 payload 274 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 -- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 msgr2=0x7fd5b4136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 0x7fd5b4136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 -- 172.21.15.134:0/4266166203 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 msgr2=0x7fd5b409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 0x7fd5b409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient: _finish_hunting 0 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 monclient: found mon.noname-c 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5bab7f640 1 -- 172.21.15.134:0/4266166203 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4073250 con 0x7fd5b4136c60 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5b9b7d640 10 monclient: get_auth_request con 0x7fd5b409bfe0 auth_method 0 2024-09-16T15:10:02.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.442+0000 7fd5b9b7d640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 0x7fd5b409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bab7f640 1 -- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 msgr2=0x7fd5b41370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bab7f640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009e80 tx=0x7fd5ac02a560 comp rx=0 tx=0).ready entity=mon.4 client_cookie=c1eeb4eb3597e458 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 1 -- 172.21.15.134:0/4266166203 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5ac007a70 con 0x7fd5b4136c60 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: dump: 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.454 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: _finish_auth 0 2024-09-16T15:10:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: _check_auth_tickets 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 1 -- 172.21.15.134:0/4266166203 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac02d050 con 0x7fd5b4136c60 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 5 monclient: authenticate success, global_id 5278 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 1 -- 172.21.15.134:0/4266166203 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5ac007bf0 con 0x7fd5b4136c60 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 4 set_mon_vals no callback set 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5b937c640 10 monclient: dump: 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 10 monclient: shutdown 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 msgr2=0x7fd5b41370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009e80 tx=0x7fd5ac02a560 comp rx=0 tx=0).stop 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/4266166203 shutdown_connections 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd5b4136c60 0x7fd5b41370c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b409bfe0 0x7fd5b409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/4266166203 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd5b4136030 0x7fd5b4136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/4266166203 >> 172.21.15.134:0/4266166203 conn(0x7fd5b40a9ea0 msgr2=0x7fd5b40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/4266166203 shutdown_connections 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.445+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/4266166203 wait complete. 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.446+0000 7fd5bc5e2640 10 monclient: build_initial_monmap 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.446+0000 7fd5bc5e2640 10 monclient: monmap: 2024-09-16T15:10:02.458 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.446+0000 7fd5bc5e2640 1 Processor -- start 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.446+0000 7fd5bc5e2640 1 -- start start 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.446+0000 7fd5bc5e2640 10 monclient: init 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient: _add_conns ranks=[2,5,6,7,3,0,8,4,1] 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-b con 0x7fd5b409bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b4136030 0x7fd5b4134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-h con 0x7fd5b4136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 0x7fd5b41353b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): picked mon.noname-d con 0x7fd5b4136c60 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40a7d90 con 0x7fd5b409bfe0 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b409fc70 con 0x7fd5b4136030 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40a8230 con 0x7fd5b4136c60 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bc5e2640 10 monclient(hunting): authenticate will time out at 1339.036343s 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 0x7fd5b41353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59356/0 (socket says 172.21.15.134:59356) 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 -- 172.21.15.134:0/2539194459 learned_addr learned my addr 172.21.15.134:0/2539194459 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): get_auth_request con 0x7fd5b409bfe0 auth_method 0 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): get_auth_request con 0x7fd5b4136c60 auth_method 0 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5b9b7d640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b4136030 0x7fd5b4134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5b9b7d640 10 monclient(hunting): get_auth_request con 0x7fd5b4136030 auth_method 0 2024-09-16T15:10:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5b9b7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5b9b7d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5b9b7d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient(hunting): handle_auth_done global_id 44831 payload 824 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 -- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 msgr2=0x7fd5b41353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 0x7fd5b41353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 -- 172.21.15.134:0/2539194459 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b4136030 msgr2=0x7fd5b4134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b4136030 0x7fd5b4134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient: _finish_hunting 0 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 monclient: found mon.noname-b 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5ba37e640 1 -- 172.21.15.134:0/2539194459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac009b30 con 0x7fd5b409bfe0 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.447+0000 7fd5bab7f640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 0x7fd5b41353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.449+0000 7fd5ba37e640 1 -- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 msgr2=0x7fd5b41beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:02.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5ba37e640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b550 tx=0x7fd5a400ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5a4003dd0 con 0x7fd5b409bfe0 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: dump: 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:02.464 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: _finish_auth 0 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 5 monclient: authenticate success, global_id 44831 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a40054e0 con 0x7fd5b409bfe0 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd5a4005690 con 0x7fd5b409bfe0 2024-09-16T15:10:02.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5b8b7b640 4 set_mon_vals no callback set 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5aaffd640 10 monclient: dump: 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:02.466 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 10 monclient: _renew_subs 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b4122550 con 0x7fd5b409bfe0 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 10 monclient: _renew_subs 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.450+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4120db0 con 0x7fd5b409bfe0 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd5a4005070 con 0x7fd5b409bfe0 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5bc5e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5bc5e2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5bc5e2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b4122120 con 0x7fd5b409bfe0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5aaffd640 1 --2- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 0x7fd59803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fd5a4049ed0 con 0x7fd5b409bfe0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5b9b7d640 1 --2- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 0x7fd59803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.451+0000 7fd5b9b7d640 10 monclient: get_auth_request con 0x7fd598038880 auth_method 0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.454+0000 7fd5b9b7d640 1 --2- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 0x7fd59803ad40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4074400 tx=0x7fd5b0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.456+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd5a4005320 con 0x7fd5b409bfe0 2024-09-16T15:10:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.456+0000 7fd5aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.456+0000 7fd5aaffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:02.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5bc5e2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5bc5e2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5bc5e2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd5b409c400 con 0x7fd5b409bfe0 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5aaffd640 1 -- 172.21.15.134:0/2539194459 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fd5a400db10 con 0x7fd5b409bfe0 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.681+0000 7fd5aaffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:02.683 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:10:02.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 msgr2=0x7fd59803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:02.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 0x7fd59803ad40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4074400 tx=0x7fd5b0008040 comp rx=0 tx=0).stop 2024-09-16T15:10:02.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 10 monclient: shutdown 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 msgr2=0x7fd5b41beb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400b550 tx=0x7fd5a400ba10 comp rx=0 tx=0).stop 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 shutdown_connections 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> v2:172.21.15.162:6800/980547263 conn(0x7fd598038880 0x7fd59803ad40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd5b4136c60 0x7fd5b41353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd5b4136030 0x7fd5b4134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 --2- 172.21.15.134:0/2539194459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5b409bfe0 0x7fd5b41beb30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.683+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 >> 172.21.15.134:0/2539194459 conn(0x7fd5b40a9ea0 msgr2=0x7fd5b406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.684+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 shutdown_connections 2024-09-16T15:10:02.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:02.684+0000 7fd5bc5e2640 1 -- 172.21.15.134:0/2539194459 wait complete. 2024-09-16T15:10:05.699 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.805+0000 7f2bf3ed7640 10 monclient: get_monmap_and_config 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.805+0000 7f2bf3ed7640 10 monclient: build_initial_monmap 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.805+0000 7f2bf3ed7640 10 monclient: monmap: 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:05.806790+0000 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:05.806790+0000 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:05.807 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:05.808 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.806+0000 7f2bf3ed7640 1 Processor -- start 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.806+0000 7f2bf3ed7640 1 -- start start 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.806+0000 7f2bf3ed7640 10 monclient: init 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient: _add_conns ranks=[8,9,1,3,7,4,5,2,10,6,0] 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-e con 0x7f2bec136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-a con 0x7f2bec09bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f2bec136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec073250 con 0x7f2bec136c60 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:05.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec09b600 con 0x7f2bec09bfe0 2024-09-16T15:10:05.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:05.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0a5740 con 0x7f2bec136030 2024-09-16T15:10:05.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:05.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf3ed7640 10 monclient(hunting): authenticate will time out at 1342.396306s 2024-09-16T15:10:05.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1c73640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 msgr2=0x7f2bec136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1c73640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46148/0 (socket says 172.21.15.134:46148) 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 learned_addr learned my addr 172.21.15.134:0/3305217003 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 10 monclient(hunting): get_auth_request con 0x7f2bec09bfe0 auth_method 0 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:05.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:05.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:10:05.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:05.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:05.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.807+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:05.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:05.809+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> v1:172.21.15.162:6789/0 conn(0x7f2bec136c60 legacy=0x7f2bec137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:06.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 msgr2=0x7f2bec136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:06.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 10 monclient(hunting): get_auth_request con 0x7f2bec09bfe0 auth_method 0 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:06.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.008+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:10:06.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 msgr2=0x7f2bec136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 10 monclient(hunting): get_auth_request con 0x7f2bec09bfe0 auth_method 0 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:06.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:06.409+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 msgr2=0x7f2bec136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 10 monclient(hunting): get_auth_request con 0x7f2bec09bfe0 auth_method 0 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:07.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:07.210+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:10:08.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): tick 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 monclient(hunting): continuing hunt 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 msgr2=0x7f2bec136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2bec136030 0x7f2bec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 msgr2=0x7f2bec09c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 --2- 172.21.15.134:0/3305217003 >> v2:172.21.15.134:3300/0 conn(0x7f2bec09bfe0 0x7f2bec09c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 >> v1:172.21.15.162:6789/0 conn(0x7f2bec136c60 legacy=0x7f2bec137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient: _add_conns ranks=[4,0,2,5,6,7,3,10] 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:08.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): picked mon.noname-c con 0x7f2bd0003890 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2bd0006310 addr v1:172.21.15.134:6789/0 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): picked mon.noname-b con 0x7f2bd000a010 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0a52c0 con 0x7f2bd0006310 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec09b600 con 0x7f2bd000a010 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0a5740 con 0x7f2bd0003890 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2be37fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd000a010 auth_method 0 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:08.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd0003890 auth_method 0 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:08.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.807+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:08.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:08.809+0000 7f2bf1472640 1 -- 172.21.15.134:0/3305217003 >> v1:172.21.15.134:6789/0 conn(0x7f2bd0006310 legacy=0x7f2bd00087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:09.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:09.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd0003890 auth_method 0 2024-09-16T15:10:09.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:09.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd000a010 auth_method 0 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:09.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:09.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:09.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.008+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd0003890 auth_method 0 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd000a010 auth_method 0 2024-09-16T15:10:09.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:10:09.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:09.409+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd000a010 auth_method 0 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:10.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd0003890 auth_method 0 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:10:10.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.210+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:10:10.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167825007 0 0) 0x7f2bec0a52c0 con 0x7f2bd0006310 2024-09-16T15:10:10.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 10 monclient(hunting): _negotiate 2024-09-16T15:10:10.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:10.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 10 monclient(hunting): my global_id is 74277 2024-09-16T15:10:10.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.249+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd8003520 con 0x7f2bd0006310 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274270834 0 0) 0x7f2bd8003520 con 0x7f2bd0006310 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 msgr2=0x7f2bd0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2bd0003890 0x7f2bd0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 msgr2=0x7f2bd000c4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd000a010 0x7f2bd000c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _finish_hunting 0 2024-09-16T15:10:10.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 reap_dead start 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd000dd50 con 0x7f2bd0006310 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _finish_auth 0 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _check_auth_tickets 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f2be800caf0 con 0x7f2bd0006310 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf3ed7640 5 monclient: authenticate success, global_id 74277 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf3ed7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:10:10.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: dump: 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:10.256 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 >> v1:172.21.15.134:6789/0 conn(0x7f2bd0006310 legacy=0x7f2bd00087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient: _add_conns ranks=[1,7,2,3,0,6,8,5,4] 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd80041c0 0x7f2bd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): picked mon.f con 0x7f2bd80041c0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2bd8004dd0 0x7f2bd8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): picked mon.i con 0x7f2bd8004dd0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 0x7f2bd800c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): picked mon.b con 0x7f2bd8009bb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:10.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec09b600 con 0x7f2bd80041c0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0a5740 con 0x7f2bd8009bb0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd800c950 con 0x7f2bd8004dd0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf0c71640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf3ed7640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 0x7f2bd800c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd8009bb0 auth_method 0 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2bd8004dd0 0x7f2bd8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd80041c0 0x7f2bd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): get_auth_request con 0x7f2bd8004dd0 auth_method 0 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd80041c0 auth_method 0 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:10.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1472640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.250+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 10 monclient(hunting): handle_auth_done global_id 74277 payload 293 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2bd8004dd0 msgr2=0x7f2bd8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2bd8004dd0 0x7f2bd8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd80041c0 msgr2=0x7f2bd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd80041c0 0x7f2bd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 10 monclient: _finish_hunting 0 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 monclient: found mon.b 2024-09-16T15:10:10.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:10.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:10.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8003520 con 0x7f2bd8009bb0 2024-09-16T15:10:10.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 10 monclient: _finish_auth 0 2024-09-16T15:10:10.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 10 monclient: _check_auth_tickets 2024-09-16T15:10:10.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.251+0000 7f2bf2474640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.254+0000 7f2bf2474640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 msgr2=0x7f2bd800c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.254+0000 7f2bf2474640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 0x7f2bd800c040 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0a3af0 tx=0x7f2bd80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeee223aec5ddd46 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2be402b9b0 con 0x7f2bd8009bb0 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: dump: 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be402f070 con 0x7f2bd8009bb0 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf3ed7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 1 -- 172.21.15.134:0/3305217003 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2be4035400 con 0x7f2bd8009bb0 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf3ed7640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf3ed7640 4 set_mon_vals no callback set 2024-09-16T15:10:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.255+0000 7f2bf0c71640 10 monclient: dump: 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:10.270 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 10 monclient: shutdown 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 msgr2=0x7f2bd800c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 0x7f2bd800c040 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0a3af0 tx=0x7f2bd80057f0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/3305217003 shutdown_connections 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8009bb0 0x7f2bd800c040 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f2bd8004dd0 0x7f2bd8005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/3305217003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bd80041c0 0x7f2bd80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/3305217003 >> 172.21.15.134:0/3305217003 conn(0x7f2bec0a9ea0 msgr2=0x7f2bec0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/3305217003 shutdown_connections 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.256+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/3305217003 wait complete. 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.257+0000 7f2bf3ed7640 10 monclient: build_initial_monmap 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.257+0000 7f2bf3ed7640 10 monclient: monmap: 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:10.272 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.257+0000 7f2bf3ed7640 1 Processor -- start 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 -- start start 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient: init 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient: _add_conns ranks=[6,3,2,8,0,7,5,1,4] 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-d con 0x7f2bd0006310 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:10.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 0x7f2bec1cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-g con 0x7f2bd80041c0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 0x7f2bec121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): picked mon.noname-b con 0x7f2bd8004dd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec124c60 con 0x7f2bd8004dd0 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1230a0 con 0x7f2bd80041c0 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec134fb0 con 0x7f2bd0006310 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf3ed7640 10 monclient(hunting): authenticate will time out at 1346.847648s 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1472640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 msgr2=0x7f2bec1cb670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1472640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 0x7f2bec1cb670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 0x7f2bec121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40476/0 (socket says 172.21.15.134:40476) 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 1 -- 172.21.15.134:0/295789491 learned_addr learned my addr 172.21.15.134:0/295789491 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request con 0x7f2bd0006310 auth_method 0 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): get_auth_request con 0x7f2bd8004dd0 auth_method 0 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.258+0000 7f2bf2474640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 10 monclient(hunting): handle_auth_done global_id 34944 payload 824 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 -- 172.21.15.134:0/295789491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 msgr2=0x7f2bec1cb670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 0x7f2bec1cb670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 -- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 msgr2=0x7f2bec121600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 0x7f2bec121600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 10 monclient: _finish_hunting 0 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 monclient: found mon.noname-d 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf1c73640 1 -- 172.21.15.134:0/295789491 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0a78e0 con 0x7f2bd0006310 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.259+0000 7f2bf2474640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 0x7f2bec121600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf1c73640 1 -- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 msgr2=0x7f2bec1cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf1c73640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc003890 tx=0x7f2bdc00a7b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2bdc009e80 con 0x7f2bd0006310 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: dump: 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: _finish_auth 0 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011050 con 0x7f2bd0006310 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 5 monclient: authenticate success, global_id 34944 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 10 monclient: _renew_subs 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2bdc0173c0 con 0x7f2bd0006310 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec123c70 con 0x7f2bd0006310 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be3fff640 4 set_mon_vals no callback set 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2be27fc640 10 monclient: dump: 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 10 monclient: _renew_subs 2024-09-16T15:10:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.261+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec125470 con 0x7f2bd0006310 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2bdc018630 con 0x7f2bd0006310 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2bf3ed7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2bf3ed7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2bf3ed7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec09c400 con 0x7f2bd0006310 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2be27fc640 1 --2- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 0x7f2bd003bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.262+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f2bdc04b760 con 0x7f2bd0006310 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.263+0000 7f2bf1472640 1 --2- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 0x7f2bd003bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.263+0000 7f2bf1472640 10 monclient: get_auth_request con 0x7f2bd0039b30 auth_method 0 2024-09-16T15:10:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.265+0000 7f2bf1472640 1 --2- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 0x7f2bd003bff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2be8000c00 tx=0x7f2be8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.267+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f2bdc0209a0 con 0x7f2bd0006310 2024-09-16T15:10:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.267+0000 7f2be27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.267+0000 7f2be27fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:10.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2bf3ed7640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2bf3ed7640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2bf3ed7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2bec072070 con 0x7f2bd0006310 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2be27fc640 1 -- 172.21.15.134:0/295789491 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3177 (secure 0 0 0) 0x7f2bdc0203b0 con 0x7f2bd0006310 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2be27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.495+0000 7f2be27fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:10.497 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":90,"quorum":[0,1,2,4,6,7],"quorum_names":["a","f","b","c","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 msgr2=0x7f2bd003bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 0x7f2bd003bff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2be8000c00 tx=0x7f2be8008040 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 10 monclient: shutdown 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 msgr2=0x7f2bec1cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.497+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc003890 tx=0x7f2bdc00a7b0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 shutdown_connections 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bd8004dd0 0x7f2bec121600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> v2:172.21.15.162:6800/980547263 conn(0x7f2bd0039b30 0x7f2bd003bff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bd80041c0 0x7f2bec1cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 --2- 172.21.15.134:0/295789491 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2bd0006310 0x7f2bec1cb130 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:10.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 >> 172.21.15.134:0/295789491 conn(0x7f2bec0a9ea0 msgr2=0x7f2bec1351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:10.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 shutdown_connections 2024-09-16T15:10:10.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:10.498+0000 7f2bf3ed7640 1 -- 172.21.15.134:0/295789491 wait complete. 2024-09-16T15:10:13.511 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:10:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.617+0000 7f8be9f3e640 10 monclient: get_monmap_and_config 2024-09-16T15:10:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.617+0000 7f8be9f3e640 10 monclient: build_initial_monmap 2024-09-16T15:10:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.617+0000 7f8be9f3e640 10 monclient: monmap: 2024-09-16T15:10:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:13.618 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:13.618168+0000 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:13.618168+0000 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:13.619 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:13.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.617+0000 7f8be9f3e640 1 Processor -- start 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 1 -- start start 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient: init 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient: _add_conns ranks=[1,0,9,2,3,7,6,8,5,4,10] 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8be4136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8be409bfe0 addr v1:172.21.15.134:6789/0 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-a con 0x7f8be409c400 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be4073250 con 0x7f8be409bfe0 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be409b600 con 0x7f8be4136030 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be40a5740 con 0x7f8be409c400 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be9f3e640 10 monclient(hunting): authenticate will time out at 1350.207592s 2024-09-16T15:10:13.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be2ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8be409bfe0 0x7f8be413bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:51870/0 (socket says 172.21.15.134:51870) 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be2ffd640 1 -- 172.21.15.134:0/380935187 learned_addr learned my addr 172.21.15.134:0/380935187 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be409c400 auth_method 0 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.618+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.620+0000 7f8be2ffd640 1 -- 172.21.15.134:0/380935187 >> v1:172.21.15.134:6789/0 conn(0x7f8be409bfe0 legacy=0x7f8be413bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:13.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.620+0000 7f8be37fe640 1 -- 172.21.15.134:0/380935187 >> v1:172.21.15.162:6789/0 conn(0x7f8be4136030 legacy=0x7f8be4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:13.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be409c400 auth_method 0 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:13.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:13.819+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:10:14.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:14.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be409c400 auth_method 0 2024-09-16T15:10:14.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:14.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:14.220+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be409c400 auth_method 0 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:15.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:15.021+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966794612 0 0) 0x7f8be409b600 con 0x7f8be4136030 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): _negotiate 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): my global_id is 44695 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd0003520 con 0x7f8be4136030 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746124596 0 0) 0x7f8bd0003520 con 0x7f8be4136030 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 msgr2=0x7f8be413f6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 --2- 172.21.15.134:0/380935187 >> v2:172.21.15.134:3300/0 conn(0x7f8be409c400 0x7f8be413f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 >> v1:172.21.15.134:6789/0 conn(0x7f8be409bfe0 legacy=0x7f8be413bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40a52c0 con 0x7f8be4136030 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7f8bd4001be0 con 0x7f8be4136030 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: dump: 2024-09-16T15:10:16.256 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.257 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 >> v1:172.21.15.162:6789/0 conn(0x7f8be4136030 legacy=0x7f8be4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient: _add_conns ranks=[3,8,2,6,1,0,7,5,4] 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd00041c0 0x7f8bd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): picked mon.g con 0x7f8bd00041c0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8bd0004dd0 0x7f8bd0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): picked mon.e con 0x7f8bd0004dd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 0x7f8bd000c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): picked mon.b con 0x7f8bd0009c90 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:16.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40a5740 con 0x7f8bd0009c90 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd000ca00 con 0x7f8bd00041c0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd000cb70 con 0x7f8bd0004dd0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.251+0000 7f8be27fc640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be2ffd640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8bd0004dd0 msgr2=0x7f8bd0005250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be37fe640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd00041c0 msgr2=0x7f8bd00047c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be2ffd640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8bd0004dd0 0x7f8bd0005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be37fe640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd00041c0 0x7f8bd00047c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 0x7f8bd000c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8bd0009c90 auth_method 0 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient(hunting): handle_auth_done global_id 44695 payload 293 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8bd0004dd0 msgr2=0x7f8bd0005250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8bd0004dd0 0x7f8bd0005250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd00041c0 msgr2=0x7f8bd00047c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd00041c0 0x7f8bd00047c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 monclient: found mon.b 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0003520 con 0x7f8bd0009c90 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be9f3e640 5 monclient: authenticate success, global_id 44695 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.252+0000 7f8be9f3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:16.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.253+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 reap_dead start 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.255+0000 7f8be3fff640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 msgr2=0x7f8bd000c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.255+0000 7f8be3fff640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 0x7f8bd000c120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc004360 tx=0x7f8bd00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49139dce296d591a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bdc02bc90 con 0x7f8bd0009c90 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: dump: 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.267 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc02f070 con 0x7f8bd0009c90 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 1 -- 172.21.15.134:0/380935187 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bdc02be70 con 0x7f8bd0009c90 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 4 set_mon_vals no callback set 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be27fc640 10 monclient: dump: 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.269 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 10 monclient: shutdown 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 1 -- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 msgr2=0x7f8bd000c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.256+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 0x7f8bd000c120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc004360 tx=0x7f8bd00057f0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 1 -- 172.21.15.134:0/380935187 shutdown_connections 2024-09-16T15:10:16.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/380935187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bd0009c90 0x7f8bd000c120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 1 -- 172.21.15.134:0/380935187 >> 172.21.15.134:0/380935187 conn(0x7f8be40a9ea0 msgr2=0x7f8be40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 1 -- 172.21.15.134:0/380935187 shutdown_connections 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 1 -- 172.21.15.134:0/380935187 wait complete. 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.257+0000 7f8be9f3e640 10 monclient: monmap: 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.271 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.258+0000 7f8be9f3e640 1 Processor -- start 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.258+0000 7f8be9f3e640 1 -- start start 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.258+0000 7f8be9f3e640 10 monclient: init 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient: _add_conns ranks=[3,2,4,8,1,0,6,7,5] 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 0x7f8be4134d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-g con 0x7f8bd0009c90 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 0x7f8be4135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-b con 0x7f8be4120ed0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 0x7f8be41cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be37fe640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 msgr2=0x7f8be4134d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 0x7f8be4134d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): picked mon.noname-c con 0x7f8be41cb0d0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40a31f0 con 0x7f8be4120ed0 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be409fb00 con 0x7f8bd0009c90 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40a4780 con 0x7f8be41cb0d0 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be9f3e640 10 monclient(hunting): authenticate will time out at 1352.848826s 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 0x7f8be4135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.259+0000 7f8be3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 0x7f8be41cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 0x7f8be41cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40756/0 (socket says 172.21.15.134:40756) 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 1 -- 172.21.15.134:0/423805019 learned_addr learned my addr 172.21.15.134:0/423805019 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be41cb0d0 auth_method 0 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): get_auth_request con 0x7f8be4120ed0 auth_method 0 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient(hunting): handle_auth_done global_id 44849 payload 824 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 1 -- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 msgr2=0x7f8be41cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 0x7f8be41cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 1 -- 172.21.15.134:0/423805019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 msgr2=0x7f8be4134d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 0x7f8be4134d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 1 monclient: found mon.noname-b 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.260+0000 7f8be2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.261+0000 7f8be2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.261+0000 7f8be2ffd640 1 -- 172.21.15.134:0/423805019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4121e30 con 0x7f8be4120ed0 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be2ffd640 1 -- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 msgr2=0x7f8be4135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be2ffd640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 0x7f8be4135250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc001460 tx=0x7f8bcc00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bcc007e50 con 0x7f8be4120ed0 2024-09-16T15:10:16.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: dump: 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.276 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc005cf0 con 0x7f8be4120ed0 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 5 monclient: authenticate success, global_id 44849 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 10 monclient: _renew_subs 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be4122110 con 0x7f8be4120ed0 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bcc005e60 con 0x7f8be4120ed0 2024-09-16T15:10:16.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8bdbfff640 10 monclient: dump: 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be1ffb640 4 set_mon_vals no callback set 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 10 monclient: _renew_subs 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.263+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be40a2900 con 0x7f8be4120ed0 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8bcc007070 con 0x7f8be4120ed0 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8bdbfff640 1 --2- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 0x7f8bc403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8be9f3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8be9f3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8be9f3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.265+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(424..424 src has 1..424) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f8bcc049110 con 0x7f8be4120ed0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.264+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be4120120 con 0x7f8be4120ed0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.265+0000 7f8be37fe640 1 --2- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 0x7f8bc403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.265+0000 7f8be37fe640 10 monclient: get_auth_request con 0x7f8bc4038880 auth_method 0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.267+0000 7f8be37fe640 1 --2- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 0x7f8bc403ad40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4001eb0 tx=0x7f8bd40017b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.270+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8be4120120 con 0x7f8be4120ed0 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.270+0000 7f8bdbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.270+0000 7f8bdbfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:16.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.495+0000 7f8be9f3e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:10:16.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.495+0000 7f8be9f3e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:16.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.495+0000 7f8be9f3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.495+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8be4120120 con 0x7f8be4120ed0 2024-09-16T15:10:16.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.496+0000 7f8bdbfff640 1 -- 172.21.15.134:0/423805019 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3183 (secure 0 0 0) 0x7f8be4120120 con 0x7f8be4120ed0 2024-09-16T15:10:16.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.496+0000 7f8bdbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:10:16.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.496+0000 7f8bdbfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:16.498 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":92,"quorum":[0,1,2,4,5,6,7],"quorum_names":["a","f","b","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:10:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 msgr2=0x7f8bc403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 0x7f8bc403ad40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4001eb0 tx=0x7f8bd40017b0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 10 monclient: shutdown 2024-09-16T15:10:16.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 msgr2=0x7f8be4135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 0x7f8be4135250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc001460 tx=0x7f8bcc00c820 comp rx=0 tx=0).stop 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 shutdown_connections 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8be41cb0d0 0x7f8be41cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8be4120ed0 0x7f8be4135250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bc4038880 0x7f8bc403ad40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 --2- 172.21.15.134:0/423805019 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bd0009c90 0x7f8be4134d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 >> 172.21.15.134:0/423805019 conn(0x7f8be40a9ea0 msgr2=0x7f8be409d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 shutdown_connections 2024-09-16T15:10:16.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.498+0000 7f8be9f3e640 1 -- 172.21.15.134:0/423805019 wait complete. 2024-09-16T15:10:16.510 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-16T15:10:16.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-16T15:10:16.510 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 10 monclient: get_monmap_and_config 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 10 monclient: monmap: 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.608 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:16.608135+0000 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:16.608135+0000 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:16.609 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:16.610 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 1 Processor -- start 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 1 -- start start 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.607+0000 7fbbaecb7640 10 monclient: init 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient: _add_conns ranks=[8,3,10,7,2,9,4,5,0,6,1] 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 0x7fbba809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-e con 0x7fbba8136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 0x7fbba8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-g con 0x7fbba8137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbba8135dc0 0x7fbba8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-f con 0x7fbba8135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba809abc0 con 0x7fbba8135dc0 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba809ba50 con 0x7fbba8137220 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80a53c0 con 0x7fbba8136e40 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaecb7640 10 monclient(hunting): authenticate will time out at 1353.197452s 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaca53640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 msgr2=0x7fbba809c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbaca53640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 0x7fbba809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbba7fff640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 msgr2=0x7fbba8137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 0x7fbba8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbba8135dc0 0x7fbba8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbba8135dc0 0x7fbba8136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:57634/0 (socket says 172.21.15.134:57634) 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 1 -- 172.21.15.134:0/4137811796 learned_addr learned my addr 172.21.15.134:0/4137811796 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): get_auth_request con 0x7fbba8135dc0 auth_method 0 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.608+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 10 monclient(hunting): handle_auth_done global_id 44704 payload 274 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 -- 172.21.15.134:0/4137811796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 msgr2=0x7fbba809c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 0x7fbba809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 -- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 msgr2=0x7fbba8137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 0x7fbba8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 monclient: found mon.noname-f 2024-09-16T15:10:16.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:10:16.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.609+0000 7fbbad254640 1 -- 172.21.15.134:0/4137811796 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8073250 con 0x7fbba8135dc0 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbad254640 1 -- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8135dc0 msgr2=0x7fbba8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbad254640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8135dc0 0x7fbba8136220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba002a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8141f8521f5892a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 1 -- 172.21.15.134:0/4137811796 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba0007a70 con 0x7fbba8135dc0 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: dump: 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.619 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 1 -- 172.21.15.134:0/4137811796 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba002d050 con 0x7fbba8135dc0 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 1 -- 172.21.15.134:0/4137811796 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba0007bf0 con 0x7fbba8135dc0 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 5 monclient: authenticate success, global_id 44704 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 4 set_mon_vals no callback set 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbba77fe640 10 monclient: dump: 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.621 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.622 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 10 monclient: shutdown 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 -- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8135dc0 msgr2=0x7fbba8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8135dc0 0x7fbba8136220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba002a590 comp rx=0 tx=0).stop 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 -- 172.21.15.134:0/4137811796 shutdown_connections 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8135dc0 0x7fbba8136220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba8137220 0x7fbba8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/4137811796 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbba8136e40 0x7fbba809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 -- 172.21.15.134:0/4137811796 >> 172.21.15.134:0/4137811796 conn(0x7fbba80a6110 msgr2=0x7fbba80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 -- 172.21.15.134:0/4137811796 shutdown_connections 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.612+0000 7fbbaecb7640 1 -- 172.21.15.134:0/4137811796 wait complete. 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.613+0000 7fbbaecb7640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.613+0000 7fbbaecb7640 10 monclient: monmap: 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:16.623 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.613+0000 7fbbaecb7640 1 Processor -- start 2024-09-16T15:10:16.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.613+0000 7fbbaecb7640 1 -- start start 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.613+0000 7fbbaecb7640 10 monclient: init 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient: _add_conns ranks=[5,1,6,7,2,4,8,3,0] 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbba8135dc0 0x7fbba8134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-h con 0x7fbba8135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8136e40 0x7fbba81352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-f con 0x7fbba8136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): picked mon.noname-d con 0x7fbba8137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8123c20 con 0x7fbba8136e40 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81367d0 con 0x7fbba8135dc0 2024-09-16T15:10:16.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81362a0 con 0x7fbba8137220 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaecb7640 10 monclient(hunting): authenticate will time out at 1353.203557s 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:37240/0 (socket says 172.21.15.134:37240) 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 learned_addr learned my addr 172.21.15.134:0/2693259159 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): get_auth_request con 0x7fbba8137220 auth_method 0 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbba8135dc0 0x7fbba8134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8136e40 0x7fbba81352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): get_auth_request con 0x7fbba8136e40 auth_method 0 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): get_auth_request con 0x7fbba8135dc0 auth_method 0 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbba7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 10 monclient(hunting): handle_auth_done global_id 34953 payload 824 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.614+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbba8135dc0 msgr2=0x7fbba8134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbba8135dc0 0x7fbba8134db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8136e40 msgr2=0x7fbba81352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8136e40 0x7fbba81352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 1 monclient: found mon.noname-d 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.615+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009b30 con 0x7fbba8137220 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.617+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 msgr2=0x7fbba8121810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.617+0000 7fbbad254640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009f70 tx=0x7fbba0004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.629 INFO:tasks.rados.rados.0.smithi162.stdout:2139: done (0 left) 2024-09-16T15:10:16.629 INFO:tasks.rados.rados.0.smithi162.stdout:2140: snap_remove snap 189 2024-09-16T15:10:16.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba0004710 con 0x7fbba8137220 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: dump: 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.633 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0004890 con 0x7fbba8137220 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbba003b400 con 0x7fbba8137220 2024-09-16T15:10:16.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba57fa640 10 monclient: dump: 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.635 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 5 monclient: authenticate success, global_id 34953 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 10 monclient: _renew_subs 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba8124fb0 con 0x7fbba8137220 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbba6ffd640 4 set_mon_vals no callback set 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 10 monclient: _renew_subs 2024-09-16T15:10:16.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.620+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba80a8930 con 0x7fbba8137220 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbba0007600 con 0x7fbba8137220 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaecb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 0x7fbba80a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8071b20 con 0x7fbba80a1a20 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbba57fa640 1 --2- 172.21.15.134:0/2693259159 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb8c038880 0x7fbb8c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fbba006a980 con 0x7fbba8137220 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaca53640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 0x7fbba80a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaca53640 10 monclient: get_auth_request con 0x7fbba80a1a20 auth_method 0 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaca53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaca53640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.621+0000 7fbbaca53640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbba7fff640 1 --2- 172.21.15.134:0/2693259159 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb8c038880 0x7fbb8c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbba7fff640 10 monclient: get_auth_request con 0x7fbb8c038880 auth_method 0 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbbaca53640 10 monclient(hunting): handle_auth_done global_id 74286 payload 824 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.622+0000 7fbbaca53640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbba8061ae0 con 0x7fbba80a1a20 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbbaca53640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 msgr2=0x7fbba80a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbbaca53640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 0x7fbba80a1e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009610 tx=0x7fbb9c009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c012d60 con 0x7fbba80a1a20 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 10 monclient: dump: 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.638 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.639 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba7fff640 1 --2- 172.21.15.134:0/2693259159 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb8c038880 0x7fbb8c03ad40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbb940093d0 tx=0x7fbb94009360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbb9c0128f0 con 0x7fbba80a1a20 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 msgr2=0x7fbba80a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.624+0000 7fbba57fa640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 0x7fbba80a1e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009610 tx=0x7fbb9c009ad0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.625+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 msgr2=0x7fbba80a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.625+0000 7fbba57fa640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a1a20 0x7fbba80a1e00 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009610 tx=0x7fbb9c009ad0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaecb7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 0x7fbba8121d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaecb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba807a680 con 0x7fbba80a2340 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 0x7fbba8121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient: get_auth_request con 0x7fbba80a2340 auth_method 0 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 10 monclient(hunting): handle_auth_done global_id 74295 payload 824 2024-09-16T15:10:16.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.631+0000 7fbbaca53640 1 -- 172.21.15.134:0/2693259159 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbba809b490 con 0x7fbba80a2340 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbbaca53640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 msgr2=0x7fbba8121d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbbaca53640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 0x7fbba8121d50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009610 tx=0x7fbb9c0059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb9c005c50 con 0x7fbba80a2340 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbba57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbba57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.633+0000 7fbba57fa640 10 monclient: dump: 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.643 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9175 (secure 0 0 0) 0x7fbb9c005db0 con 0x7fbba80a2340 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 msgr2=0x7fbba8121d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 0x7fbba8121d50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009610 tx=0x7fbb9c0059d0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 msgr2=0x7fbba8121d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:16.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbba57fa640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbba80a2340 0x7fbba8121d50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:16.645 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.647 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:16.648 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.649 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.651 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:16.655 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:16.656 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:16.658 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.659 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:16.666 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.667 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:16.668 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:16.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb8c038880 msgr2=0x7fbb8c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/2693259159 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb8c038880 0x7fbb8c03ad40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbb940093d0 tx=0x7fbb94009360 comp rx=0 tx=0).stop 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 10 monclient: shutdown 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 msgr2=0x7fbba8121810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbad254640 1 -- 172.21.15.134:0/2693259159 reap_dead start 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009f70 tx=0x7fbba0004a40 comp rx=0 tx=0).stop 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 shutdown_connections 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 --2- 172.21.15.134:0/2693259159 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbba8137220 0x7fbba8121810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 >> 172.21.15.134:0/2693259159 conn(0x7fbba80a6110 msgr2=0x7fbba806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 shutdown_connections 2024-09-16T15:10:16.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.634+0000 7fbbaecb7640 1 -- 172.21.15.134:0/2693259159 wait complete. 2024-09-16T15:10:16.671 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:16.671 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-09-16T15:10:16.671 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:10:16.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.767+0000 7f8e73345640 10 monclient: get_monmap_and_config 2024-09-16T15:10:16.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.767+0000 7f8e73345640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.767+0000 7f8e73345640 10 monclient: monmap: 2024-09-16T15:10:16.768 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.768 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:16.768133+0000 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:16.768133+0000 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.769 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.770 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:16.770 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:16.770 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.767+0000 7f8e73345640 1 Processor -- start 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 -- start start 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient: init 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient: _add_conns ranks=[9,1,2,10,7,3,0,6,5,8,4] 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-a con 0x7f8e6c136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8e6c137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:10:16.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-b con 0x7f8e6c13c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c073250 con 0x7f8e6c137220 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0a56e0 con 0x7f8e6c136e40 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c09abc0 con 0x7f8e6c13c370 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e73345640 10 monclient(hunting): authenticate will time out at 1353.357636s 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38986/0 (socket says 172.21.15.134:38986) 2024-09-16T15:10:16.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58808/0 (socket says 172.21.15.134:58808) 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 1 -- 172.21.15.134:0/2510523878 learned_addr learned my addr 172.21.15.134:0/2510523878 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): get_auth_request con 0x7f8e6c136e40 auth_method 0 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): get_auth_request con 0x7f8e6c13c370 auth_method 0 2024-09-16T15:10:16.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.768+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 10 monclient(hunting): handle_auth_done global_id 44858 payload 274 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 1 -- 172.21.15.134:0/2510523878 >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 msgr2=0x7f8e6c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 1 --2- 172.21.15.134:0/2510523878 >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 1 -- 172.21.15.134:0/2510523878 >> v1:172.21.15.162:6789/0 conn(0x7f8e6c137220 legacy=0x7f8e6c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 1 monclient: found mon.noname-b 2024-09-16T15:10:16.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e718e2640 1 -- 172.21.15.134:0/2510523878 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c09ba50 con 0x7f8e6c13c370 2024-09-16T15:10:16.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.769+0000 7f8e710e1640 1 --2- 172.21.15.134:0/2510523878 >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:16.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e718e2640 1 -- 172.21.15.134:0/2510523878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 msgr2=0x7f8e6c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e718e2640 1 --2- 172.21.15.134:0/2510523878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009d90 tx=0x7f8e6402a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a8d770d10e7c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 1 -- 172.21.15.134:0/2510523878 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e64007bc0 con 0x7f8e6c13c370 2024-09-16T15:10:16.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:10:16.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: dump: 2024-09-16T15:10:16.784 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.784 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.784 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.784 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 1 -- 172.21.15.134:0/2510523878 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6402d050 con 0x7f8e6c13c370 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e73345640 5 monclient: authenticate success, global_id 44858 2024-09-16T15:10:16.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e73345640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 1 -- 172.21.15.134:0/2510523878 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e64007d40 con 0x7f8e6c13c370 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e73345640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e73345640 4 set_mon_vals no callback set 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.771+0000 7f8e63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e63fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e63fff640 10 monclient: dump: 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.788 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.789 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 10 monclient: shutdown 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 -- 172.21.15.134:0/2510523878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 msgr2=0x7f8e6c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 --2- 172.21.15.134:0/2510523878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009d90 tx=0x7f8e6402a4e0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 -- 172.21.15.134:0/2510523878 shutdown_connections 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 --2- 172.21.15.134:0/2510523878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c13e760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 --2- 172.21.15.134:0/2510523878 >> v2:172.21.15.134:3300/0 conn(0x7f8e6c136e40 0x7f8e6c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 -- 172.21.15.134:0/2510523878 >> 172.21.15.134:0/2510523878 conn(0x7f8e6c0a6110 msgr2=0x7f8e6c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 -- 172.21.15.134:0/2510523878 shutdown_connections 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.772+0000 7f8e73345640 1 -- 172.21.15.134:0/2510523878 wait complete. 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient: monmap: 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.791 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 Processor -- start 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 -- start start 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient: init 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient: _add_conns ranks=[1,7,2,4,6,5,0,3,8] 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6c136e40 0x7f8e6c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-f con 0x7f8e6c136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:10:16.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8e6c137220 0x7f8e6c1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-i con 0x7f8e6c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): picked mon.noname-b con 0x7f8e6c13c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c121e70 con 0x7f8e6c136e40 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c136b00 con 0x7f8e6c13c370 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c1223a0 con 0x7f8e6c137220 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.773+0000 7f8e73345640 10 monclient(hunting): authenticate will time out at 1353.362970s 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c1206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58812/0 (socket says 172.21.15.134:58812) 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 learned_addr learned my addr 172.21.15.134:0/3060759139 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): get_auth_request con 0x7f8e6c13c370 auth_method 0 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6c136e40 0x7f8e6c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8e6c137220 0x7f8e6c1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 10 monclient(hunting): get_auth_request con 0x7f8e6c136e40 auth_method 0 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 10 monclient(hunting): get_auth_request con 0x7f8e6c137220 auth_method 0 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient(hunting): handle_auth_done global_id 44867 payload 824 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8e6c137220 msgr2=0x7f8e6c1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8e6c137220 0x7f8e6c1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6c136e40 msgr2=0x7f8e6c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6c136e40 0x7f8e6c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 monclient: found mon.noname-b 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680094d0 con 0x7f8e6c13c370 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e708e0640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8e6c137220 0x7f8e6c1201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.774+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e6c136e40 0x7f8e6c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 msgr2=0x7f8e6c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e718e2640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c1206e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402aa00 tx=0x7f8e64004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e64007ea0 con 0x7f8e6c13c370 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: dump: 2024-09-16T15:10:16.796 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.797 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: _finish_auth 0 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e73345640 5 monclient: authenticate success, global_id 44867 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e640044b0 con 0x7f8e6c13c370 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e64004620 con 0x7f8e6c13c370 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e637fe640 4 set_mon_vals no callback set 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e61ffb640 10 monclient: dump: 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.798 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.799 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e73345640 10 monclient: _renew_subs 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e73345640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.776+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e64009a70 con 0x7f8e6c13c370 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 10 monclient: _renew_subs 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c0a3a30 con 0x7f8e6c13c370 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 0x7f8e6c135b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8e64004040 con 0x7f8e6c13c370 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.777+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c071b20 con 0x7f8e6c135760 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 0x7f8e6c135b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient: get_auth_request con 0x7f8e6c135760 auth_method 0 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e61ffb640 1 --2- 172.21.15.134:0/3060759139 >> v2:172.21.15.162:6800/980547263 conn(0x7f8e50038880 0x7f8e5003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f8e6406b080 con 0x7f8e6c13c370 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 10 monclient(hunting): handle_auth_done global_id 44876 payload 824 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e710e1640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e6c061ae0 con 0x7f8e6c135760 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e708e0640 1 --2- 172.21.15.134:0/3060759139 >> v2:172.21.15.162:6800/980547263 conn(0x7f8e50038880 0x7f8e5003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.778+0000 7f8e708e0640 10 monclient: get_auth_request con 0x7f8e50038880 auth_method 0 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e710e1640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 msgr2=0x7f8e6c135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 0x7f8e6c135b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e6800fc00 con 0x7f8e6c135760 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e61ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.780+0000 7f8e61ffb640 10 monclient: dump: 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.802 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e708e0640 1 --2- 172.21.15.134:0/3060759139 >> v2:172.21.15.162:6800/980547263 conn(0x7f8e50038880 0x7f8e5003ad40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0a2f50 tx=0x7f8e5c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8e6800fd60 con 0x7f8e6c135760 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 msgr2=0x7f8e6c135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 0x7f8e6c135b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68011cf0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 msgr2=0x7f8e6c135b40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:16.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.781+0000 7f8e61ffb640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c135760 0x7f8e6c135b40 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68011cf0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e73345640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e73345640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 0x7f8e6c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e73345640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0a74d0 con 0x7f8e6c136080 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 0x7f8e6c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient: get_auth_request con 0x7f8e6c136080 auth_method 0 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 10 monclient(hunting): handle_auth_done global_id 44885 payload 824 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.787+0000 7f8e710e1640 1 -- 172.21.15.134:0/3060759139 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8e6c07a680 con 0x7f8e6c136080 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e710e1640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 msgr2=0x7f8e6c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e710e1640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 0x7f8e6c09d4f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8e68022040 con 0x7f8e6c136080 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e61ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e61ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.789+0000 7f8e61ffb640 10 monclient: dump: 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.806 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7f8e68015e30 con 0x7f8e6c136080 2024-09-16T15:10:16.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 msgr2=0x7f8e6c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 0x7f8e6c09d4f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68004730 comp rx=0 tx=0).stop 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 msgr2=0x7f8e6c09d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e61ffb640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c136080 0x7f8e6c09d4f0 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e68011830 tx=0x7f8e68004730 comp rx=0 tx=0).stop 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:16.808 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:16.809 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.810 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:16.811 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:16.812 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.813 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.814 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.817 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.818 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:16.820 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:16.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:16.824 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:16.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:16.829 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:10:16.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:16.831 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:16.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 >> v2:172.21.15.162:6800/980547263 conn(0x7f8e50038880 msgr2=0x7f8e5003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 --2- 172.21.15.134:0/3060759139 >> v2:172.21.15.162:6800/980547263 conn(0x7f8e50038880 0x7f8e5003ad40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0a2f50 tx=0x7f8e5c009040 comp rx=0 tx=0).stop 2024-09-16T15:10:16.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 10 monclient: shutdown 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 msgr2=0x7f8e6c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 --2- 172.21.15.134:0/3060759139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e6c13c370 0x7f8e6c1206e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402aa00 tx=0x7f8e64004a40 comp rx=0 tx=0).stop 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 reap_dead start 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e718e2640 1 -- 172.21.15.134:0/3060759139 reap_dead start 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 shutdown_connections 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 >> 172.21.15.134:0/3060759139 conn(0x7f8e6c0a6110 msgr2=0x7f8e6c13a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 shutdown_connections 2024-09-16T15:10:16.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.790+0000 7f8e73345640 1 -- 172.21.15.134:0/3060759139 wait complete. 2024-09-16T15:10:16.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:16.833 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-16T15:10:16.834 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:10:16.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.930+0000 7f699b1bc640 10 monclient: get_monmap_and_config 2024-09-16T15:10:16.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.930+0000 7f699b1bc640 10 monclient: build_initial_monmap 2024-09-16T15:10:16.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.930+0000 7f699b1bc640 10 monclient: monmap: 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:16.931201+0000 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:16.931201+0000 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:16.932 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:16.933 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:16.933 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:16.933 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 Processor -- start 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 -- start start 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient: init 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient: _add_conns ranks=[10,7,3,4,0,5,6,1,8,9,2] 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-f con 0x7f69940a4010 addr v2:172.21.15.162:3300/0 2024-09-16T15:10:16.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f69940a43f0 0x7f69940a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-i con 0x7f69940a43f0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6994133da0 0x7f6994134200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-g con 0x7f6994133da0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699409bdc0 con 0x7f69940a4010 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699409b520 con 0x7f6994133da0 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f699409c720 con 0x7f69940a43f0 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f699b1bc640 10 monclient(hunting): authenticate will time out at 1353.520581s 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6999759640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6994133da0 msgr2=0x7f6994134200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:16.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6999759640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6994133da0 0x7f6994134200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:57654/0 (socket says 172.21.15.134:57654) 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6993fff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f69940a43f0 0x7f69940a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 learned_addr learned my addr 172.21.15.134:0/2289905606 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): get_auth_request con 0x7f69940a4010 auth_method 0 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6993fff640 10 monclient(hunting): get_auth_request con 0x7f69940a43f0 auth_method 0 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6993fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6993fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:16.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.931+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 10 monclient(hunting): handle_auth_done global_id 44731 payload 274 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f69940a43f0 msgr2=0x7f69940a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f69940a43f0 0x7f69940a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6994133da0 msgr2=0x7f6994134200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6994133da0 0x7f6994134200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 10 monclient: _finish_hunting 0 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 monclient: found mon.noname-f 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:10:16.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6994073250 con 0x7f69940a4010 2024-09-16T15:10:16.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 msgr2=0x7f699409c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:10:16.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 -- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 msgr2=0x7f699409c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:10:16.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c02a150 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:10:16.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:16.932+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient: get_auth_request con 0x7f69940a4010 auth_method 0 2024-09-16T15:10:17.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 0 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient: ms_handle_reset current mon v2:172.21.15.162:3300/0 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 msgr2=0x7f699409c1e0 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.162:3300/0 conn(0x7f69940a4010 0x7f699409c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient: _add_conns ranks=[2,6,9,5,0,8,1,4] 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6980003630 0x7f6980005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): picked mon.noname-b con 0x7f6980003630 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 0x7f69800087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): picked mon.noname-d con 0x7f6980006340 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.134:3300/0 conn(0x7f69800090e0 0x7f698000b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): picked mon.noname-a con 0x7f69800090e0 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698000beb0 con 0x7f69800090e0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698000bff0 con 0x7f6980003630 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f698000c1b0 con 0x7f6980006340 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f69937fe640 10 monclient(hunting): ms_handle_reset stray mon v2:172.21.15.162:3300/0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6980003630 0x7f6980005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 0x7f69800087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.134:3300/0 conn(0x7f69800090e0 0x7f698000b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): get_auth_request con 0x7f6980003630 auth_method 0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): get_auth_request con 0x7f6980006340 auth_method 0 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): get_auth_request con 0x7f69800090e0 auth_method 0 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6999759640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:10:17.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 10 monclient(hunting): handle_auth_done global_id 44731 payload 293 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 1 -- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6980003630 msgr2=0x7f6980005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.133+0000 7f6993fff640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6980003630 0x7f6980005a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 1 -- 172.21.15.134:0/2289905606 >> v2:172.21.15.134:3300/0 conn(0x7f69800090e0 msgr2=0x7f698000b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.134:3300/0 conn(0x7f69800090e0 0x7f698000b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 1 monclient: found mon.noname-d 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6993fff640 1 -- 172.21.15.134:0/2289905606 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698000c430 con 0x7f6980006340 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6998f58640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6980003630 0x7f6980005a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6999759640 1 --2- 172.21.15.134:0/2289905606 >> v2:172.21.15.134:3300/0 conn(0x7f69800090e0 0x7f698000b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.134+0000 7f6999759640 1 -- 172.21.15.134:0/2289905606 reap_dead start 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.136+0000 7f6993fff640 1 -- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 msgr2=0x7f69800087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.136+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.136+0000 7f6993fff640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 0x7f69800087d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69940a07d0 tx=0x7f698c009240 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8e884db3732bae22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f69840148f0 con 0x7f6980006340 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: dump: 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.146 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698400f040 con 0x7f6980006340 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 1 -- 172.21.15.134:0/2289905606 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f69840133e0 con 0x7f6980006340 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f69937fe640 10 monclient: dump: 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.147 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.148 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f699b1bc640 5 monclient: authenticate success, global_id 44731 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f699b1bc640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.138+0000 7f699b1bc640 4 set_mon_vals no callback set 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 10 monclient: shutdown 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 -- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 msgr2=0x7f69800087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 0x7f69800087d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69940a07d0 tx=0x7f698c009240 comp rx=0 tx=0).stop 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 -- 172.21.15.134:0/2289905606 shutdown_connections 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 --2- 172.21.15.134:0/2289905606 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6980006340 0x7f69800087d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 -- 172.21.15.134:0/2289905606 >> 172.21.15.134:0/2289905606 conn(0x7f69940a62f0 msgr2=0x7f69940a6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 -- 172.21.15.134:0/2289905606 shutdown_connections 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 1 -- 172.21.15.134:0/2289905606 wait complete. 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.139+0000 7f699b1bc640 10 monclient: monmap: 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.150 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 Processor -- start 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 -- start start 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient: init 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient: _add_conns ranks=[4,6,1,2,7,5,0,3,8] 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6980006340 0x7f69941cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-c con 0x7f6980006340 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 0x7f69941bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-d con 0x7f6994133070 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994133470 0x7f69941bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient(hunting): picked mon.noname-f con 0x7f6994133470 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.140+0000 7f699b1bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69941cc520 con 0x7f6994133470 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6994133b60 con 0x7f6980006340 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f69940a3450 con 0x7f6994133070 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f699b1bc640 10 monclient(hunting): authenticate will time out at 1353.730020s 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 0x7f69941bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6980006340 0x7f69941cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 0x7f69941bbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:37250/0 (socket says 172.21.15.134:37250) 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 -- 172.21.15.134:0/1994165263 learned_addr learned my addr 172.21.15.134:0/1994165263 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): get_auth_request con 0x7f6994133070 auth_method 0 2024-09-16T15:10:17.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): get_auth_request con 0x7f6980006340 auth_method 0 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6999759640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994133470 0x7f69941bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient(hunting): handle_auth_done global_id 34962 payload 824 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6980006340 msgr2=0x7f69941cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6980006340 0x7f69941cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994133470 msgr2=0x7f69941bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994133470 0x7f69941bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 monclient: found mon.noname-d 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6993fff640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69941be340 con 0x7f6994133070 2024-09-16T15:10:17.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6999759640 10 monclient: get_auth_request con 0x7f6994133470 auth_method 0 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6998f58640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6980006340 0x7f69941cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.141+0000 7f6999759640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6994133470 0x7f69941bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f6993fff640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 msgr2=0x7f69941bbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f6993fff640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 0x7f69941bbdb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6994133920 tx=0x7f6984009750 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6984003e00 con 0x7f6994133070 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: dump: 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.155 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 5 monclient: authenticate success, global_id 34962 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6984023070 con 0x7f6994133070 2024-09-16T15:10:17.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 10 monclient: _renew_subs 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940a39a0 con 0x7f6994133070 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f69840044b0 con 0x7f6994133070 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f69917fa640 10 monclient: dump: 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 10 monclient: _renew_subs 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f6992ffd640 4 set_mon_vals no callback set 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.143+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69940a8da0 con 0x7f6994133070 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6984025030 con 0x7f6994133070 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f699b1bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f699b1bc640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 0x7f69941203d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6994071b20 con 0x7f69941bc830 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 0x7f69941203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient: get_auth_request con 0x7f69941bc830 auth_method 0 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f69917fa640 1 --2- 172.21.15.134:0/1994165263 >> v2:172.21.15.162:6800/980547263 conn(0x7f6974038880 0x7f697403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f698404c020 con 0x7f6994133070 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.144+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.145+0000 7f6998f58640 10 monclient(hunting): handle_auth_done global_id 5305 payload 824 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.145+0000 7f6998f58640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6994061ae0 con 0x7f69941bc830 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.145+0000 7f6999759640 1 --2- 172.21.15.134:0/1994165263 >> v2:172.21.15.162:6800/980547263 conn(0x7f6974038880 0x7f697403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.145+0000 7f6999759640 10 monclient: get_auth_request con 0x7f6974038880 auth_method 0 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f6998f58640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 msgr2=0x7f69941203d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f6998f58640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 0x7f69941203d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f698c02afd0 tx=0x7f698c0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f698c02ae50 con 0x7f69941bc830 2024-09-16T15:10:17.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 10 monclient: dump: 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.164 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f6999759640 1 --2- 172.21.15.134:0/1994165263 >> v2:172.21.15.162:6800/980547263 conn(0x7f6974038880 0x7f697403ad40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f699406caf0 tx=0x7f697c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f698c03b070 con 0x7f69941bc830 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 msgr2=0x7f69941203d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 0x7f69941203d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f698c02afd0 tx=0x7f698c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 msgr2=0x7f69941203d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.147+0000 7f69917fa640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69941bc830 0x7f69941203d0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f698c02afd0 tx=0x7f698c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.153+0000 7f699b1bc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.153+0000 7f699b1bc640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 0x7f69940a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.153+0000 7f699b1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.153+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f699407a680 con 0x7f69940a1700 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 0x7f69940a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient: get_auth_request con 0x7f69940a1700 auth_method 0 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 10 monclient(hunting): handle_auth_done global_id 5314 payload 824 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.154+0000 7f6998f58640 1 -- 172.21.15.134:0/1994165263 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f699409bae0 con 0x7f69940a1700 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f6998f58640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 msgr2=0x7f69940a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f6998f58640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 0x7f69940a1b10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097c0 tx=0x7f699409c890 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f698c037d90 con 0x7f69940a1700 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 10 monclient: dump: 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.167 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7f698c050450 con 0x7f69940a1700 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 msgr2=0x7f69940a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 0x7f69940a1b10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097c0 tx=0x7f699409c890 comp rx=0 tx=0).stop 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 msgr2=0x7f69940a1b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.156+0000 7f69917fa640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f69940a1700 0x7f69940a1b10 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097c0 tx=0x7f699409c890 comp rx=0 tx=0).stop 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:17.169 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.170 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.171 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:17.172 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.173 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.176 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.179 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.184 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.186 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.188 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.189 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.190 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.191 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:17.191 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:17.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 >> v2:172.21.15.162:6800/980547263 conn(0x7f6974038880 msgr2=0x7f697403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 --2- 172.21.15.134:0/1994165263 >> v2:172.21.15.162:6800/980547263 conn(0x7f6974038880 0x7f697403ad40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f699406caf0 tx=0x7f697c008040 comp rx=0 tx=0).stop 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 10 monclient: shutdown 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 msgr2=0x7f69941bbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 --2- 172.21.15.134:0/1994165263 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6994133070 0x7f69941bbdb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6994133920 tx=0x7f6984009750 comp rx=0 tx=0).stop 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f6999759640 1 -- 172.21.15.134:0/1994165263 reap_dead start 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f6999759640 1 -- 172.21.15.134:0/1994165263 reap_dead start 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 shutdown_connections 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 >> 172.21.15.134:0/1994165263 conn(0x7f69940a62f0 msgr2=0x7f69940a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 shutdown_connections 2024-09-16T15:10:17.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.157+0000 7f699b1bc640 1 -- 172.21.15.134:0/1994165263 wait complete. 2024-09-16T15:10:17.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:17.194 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-09-16T15:10:17.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:10:17.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.290+0000 7ff04b29c640 10 monclient: get_monmap_and_config 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.290+0000 7ff04b29c640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.290+0000 7ff04b29c640 10 monclient: monmap: 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:17.291450+0000 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:17.291450+0000 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.292 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:17.293 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 Processor -- start 2024-09-16T15:10:17.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 -- start start 2024-09-16T15:10:17.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient: init 2024-09-16T15:10:17.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient: _add_conns ranks=[0,2,9,10,8,4,5,6,1,3,7] 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff044136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 0x7ff044137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-b con 0x7ff044137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff04413c370 0x7ff04413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-a con 0x7ff04413c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff044073250 con 0x7ff044136e40 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0440a56e0 con 0x7ff04413c370 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409abc0 con 0x7ff044137220 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff04b29c640 10 monclient(hunting): authenticate will time out at 1353.880783s 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff048837640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 0x7ff044137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049839640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff04413c370 0x7ff04413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049038640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff044136e40 0x7ff04409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:55846/0 (socket says 172.21.15.134:55846) 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049038640 1 -- 172.21.15.134:0/263869754 learned_addr learned my addr 172.21.15.134:0/263869754 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff048837640 10 monclient(hunting): get_auth_request con 0x7ff044137220 auth_method 0 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff048837640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff048837640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff048837640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049839640 10 monclient(hunting): get_auth_request con 0x7ff04413c370 auth_method 0 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049839640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.291+0000 7ff049839640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient(hunting): handle_auth_done global_id 74313 payload 274 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 1 -- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 msgr2=0x7ff044137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 0x7ff044137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 1 -- 172.21.15.134:0/263869754 >> v1:172.21.15.134:6789/0 conn(0x7ff044136e40 legacy=0x7ff04409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 1 monclient: found mon.noname-a 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff049839640 1 -- 172.21.15.134:0/263869754 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04409ba50 con 0x7ff04413c370 2024-09-16T15:10:17.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.292+0000 7ff048837640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 0x7ff044137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.294+0000 7ff049839640 1 -- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff04413c370 msgr2=0x7ff04413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff049839640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff04413c370 0x7ff04413e760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009dc0 tx=0x7ff03c02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b018151f3c0cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 1 -- 172.21.15.134:0/263869754 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c007bc0 con 0x7ff04413c370 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: dump: 2024-09-16T15:10:17.304 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.305 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 1 -- 172.21.15.134:0/263869754 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c02d050 con 0x7ff04413c370 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 5 monclient: authenticate success, global_id 74313 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 1 -- 172.21.15.134:0/263869754 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff03c007d40 con 0x7ff04413c370 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 4 set_mon_vals no callback set 2024-09-16T15:10:17.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff037fff640 10 monclient: dump: 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.307 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 10 monclient: shutdown 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 -- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff04413c370 msgr2=0x7ff04413e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff04413c370 0x7ff04413e760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009dc0 tx=0x7ff03c02a510 comp rx=0 tx=0).stop 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 -- 172.21.15.134:0/263869754 shutdown_connections 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff04413c370 0x7ff04413e760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 --2- 172.21.15.134:0/263869754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044137220 0x7ff044137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 -- 172.21.15.134:0/263869754 >> 172.21.15.134:0/263869754 conn(0x7ff0440a6110 msgr2=0x7ff0440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.295+0000 7ff04b29c640 1 -- 172.21.15.134:0/263869754 shutdown_connections 2024-09-16T15:10:17.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 1 -- 172.21.15.134:0/263869754 wait complete. 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 10 monclient: monmap: 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.309 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 1 Processor -- start 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 1 -- start start 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.296+0000 7ff04b29c640 10 monclient: init 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient: _add_conns ranks=[2,8,1,4,7,5,6,3,0] 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 0x7ff044135b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-b con 0x7ff044136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff044137220 0x7ff044136040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-e con 0x7ff044137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04413c370 0x7ff04411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): picked mon.noname-f con 0x7ff04413c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441224a0 con 0x7ff04413c370 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441226f0 con 0x7ff044136e40 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff0440a7530 con 0x7ff044137220 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff04b29c640 10 monclient(hunting): authenticate will time out at 1353.886447s 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff048837640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff044137220 msgr2=0x7ff044136040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff048837640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff044137220 0x7ff044136040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 0x7ff044135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 0x7ff044135b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:58866/0 (socket says 172.21.15.134:58866) 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 -- 172.21.15.134:0/1429848247 learned_addr learned my addr 172.21.15.134:0/1429848247 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): get_auth_request con 0x7ff044136e40 auth_method 0 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049839640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04413c370 0x7ff04411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049839640 10 monclient(hunting): get_auth_request con 0x7ff04413c370 auth_method 0 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049839640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049839640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient(hunting): handle_auth_done global_id 44903 payload 824 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff044137220 msgr2=0x7ff044136040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff044137220 0x7ff044136040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04413c370 msgr2=0x7ff04411fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04413c370 0x7ff04411fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 monclient: found mon.noname-b 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.297+0000 7ff049038640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009400 con 0x7ff044136e40 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.299+0000 7ff049038640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 msgr2=0x7ff044135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.299+0000 7ff049038640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 0x7ff044135b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c490 tx=0x7ff04000c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff040005cf0 con 0x7ff044136e40 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: dump: 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.314 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 5 monclient: authenticate success, global_id 44903 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff040005e90 con 0x7ff044136e40 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 10 monclient: _renew_subs 2024-09-16T15:10:17.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c009a70 con 0x7ff044136e40 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0400074e0 con 0x7ff044136e40 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff035ffb640 10 monclient: dump: 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 10 monclient: _renew_subs 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff0377fe640 4 set_mon_vals no callback set 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:10:17.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.300+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0440a7a80 con 0x7ff044136e40 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff040007640 con 0x7ff044136e40 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff04b29c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff04b29c640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 0x7ff044120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff044122880 con 0x7ff0441201a0 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff035ffb640 1 --2- 172.21.15.134:0/1429848247 >> v2:172.21.15.162:6800/980547263 conn(0x7ff028038880 0x7ff02803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 0x7ff044120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7ff040049c00 con 0x7ff044136e40 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient: get_auth_request con 0x7ff0441201a0 auth_method 0 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff049839640 1 --2- 172.21.15.134:0/1429848247 >> v2:172.21.15.162:6800/980547263 conn(0x7ff028038880 0x7ff02803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff049839640 10 monclient: get_auth_request con 0x7ff028038880 auth_method 0 2024-09-16T15:10:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 10 monclient(hunting): handle_auth_done global_id 34971 payload 824 2024-09-16T15:10:17.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.301+0000 7ff048837640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff044061ae0 con 0x7ff0441201a0 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.303+0000 7ff048837640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 msgr2=0x7ff044120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.303+0000 7ff048837640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 0x7ff044120580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff03800ec90 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0380044a0 con 0x7ff0441201a0 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 10 monclient: dump: 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.322 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff049839640 1 --2- 172.21.15.134:0/1429848247 >> v2:172.21.15.162:6800/980547263 conn(0x7ff028038880 0x7ff02803ad40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff044137f00 tx=0x7ff03c02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff03800cbc0 con 0x7ff0441201a0 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 msgr2=0x7ff044120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 0x7ff044120580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff03800ec90 comp rx=0 tx=0).stop 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 msgr2=0x7ff044120580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.304+0000 7ff035ffb640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff0441201a0 0x7ff044120580 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff03800ec90 comp rx=0 tx=0).stop 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff04b29c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff04b29c640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 0x7ff04409d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff04b29c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff044071b20 con 0x7ff04409d4f0 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 0x7ff04409d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient: get_auth_request con 0x7ff04409d4f0 auth_method 0 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 10 monclient(hunting): handle_auth_done global_id 34980 payload 824 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.311+0000 7ff048837640 1 -- 172.21.15.134:0/1429848247 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff04407a680 con 0x7ff04409d4f0 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff048837640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 msgr2=0x7ff04409d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff048837640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 0x7ff04409d900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff038004730 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff038005e90 con 0x7ff04409d4f0 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 10 monclient: dump: 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.325 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7ff038003950 con 0x7ff04409d4f0 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 msgr2=0x7ff04409d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 0x7ff04409d900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff038004730 comp rx=0 tx=0).stop 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 msgr2=0x7ff04409d900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff035ffb640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff04409d4f0 0x7ff04409d900 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff04413ac40 tx=0x7ff038004730 comp rx=0 tx=0).stop 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:17.327 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.328 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.329 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:17.330 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.331 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:17.332 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:17.334 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.335 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.336 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.337 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.338 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.339 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.340 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.341 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.342 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.343 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:17.345 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.346 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.347 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:17.348 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 >> v2:172.21.15.162:6800/980547263 conn(0x7ff028038880 msgr2=0x7ff02803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.314+0000 7ff04b29c640 1 --2- 172.21.15.134:0/1429848247 >> v2:172.21.15.162:6800/980547263 conn(0x7ff028038880 0x7ff02803ad40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff044137f00 tx=0x7ff03c02f000 comp rx=0 tx=0).stop 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 10 monclient: shutdown 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 msgr2=0x7ff044135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 --2- 172.21.15.134:0/1429848247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff044136e40 0x7ff044135b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c490 tx=0x7ff04000c950 comp rx=0 tx=0).stop 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff049839640 1 -- 172.21.15.134:0/1429848247 reap_dead start 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff049839640 1 -- 172.21.15.134:0/1429848247 reap_dead start 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 shutdown_connections 2024-09-16T15:10:17.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 >> 172.21.15.134:0/1429848247 conn(0x7ff0440a6110 msgr2=0x7ff04413b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 shutdown_connections 2024-09-16T15:10:17.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.315+0000 7ff04b29c640 1 -- 172.21.15.134:0/1429848247 wait complete. 2024-09-16T15:10:17.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:17.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-16T15:10:17.354 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:10:17.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.450+0000 7f1b3cf1f640 10 monclient: get_monmap_and_config 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.450+0000 7f1b3cf1f640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.450+0000 7f1b3cf1f640 10 monclient: monmap: 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:17.451477+0000 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:17.451477+0000 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:17.453 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 Processor -- start 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 -- start start 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient: init 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient: _add_conns ranks=[3,1,9,8,2,5,10,7,0,4,6] 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 0x7f1b3809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-g con 0x7f1b38136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1b38137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b3813c370 0x7f1b3813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-a con 0x7f1b3813c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b38073250 con 0x7f1b38137220 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380a56e0 con 0x7f1b3813c370 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3809abc0 con 0x7f1b38136e40 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3cf1f640 10 monclient(hunting): authenticate will time out at 1354.040802s 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b3813c370 0x7f1b3813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3659c640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 msgr2=0x7f1b3809c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b3813c370 0x7f1b3813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39024/0 (socket says 172.21.15.134:39024) 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 1 -- 172.21.15.134:0/452767894 learned_addr learned my addr 172.21.15.134:0/452767894 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 10 monclient(hunting): get_auth_request con 0x7f1b3813c370 auth_method 0 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b36d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.451+0000 7f1b3659c640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 0x7f1b3809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient(hunting): handle_auth_done global_id 74322 payload 274 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 1 -- 172.21.15.134:0/452767894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 msgr2=0x7f1b3809c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 0x7f1b3809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 1 -- 172.21.15.134:0/452767894 >> v1:172.21.15.162:6789/0 conn(0x7f1b38137220 legacy=0x7f1b38137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 1 monclient: found mon.noname-a 2024-09-16T15:10:17.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.452+0000 7f1b36d9d640 1 -- 172.21.15.134:0/452767894 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3809ba50 con 0x7f1b3813c370 2024-09-16T15:10:17.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.454+0000 7f1b36d9d640 1 -- 172.21.15.134:0/452767894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3813c370 msgr2=0x7f1b3813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.454+0000 7f1b36d9d640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3813c370 0x7f1b3813e760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009e80 tx=0x7f1b3002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5143566272acb9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 1 -- 172.21.15.134:0/452767894 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b30007a70 con 0x7f1b3813c370 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: dump: 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.464 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 5 monclient: authenticate success, global_id 74322 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 1 -- 172.21.15.134:0/452767894 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3002d050 con 0x7f1b3813c370 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 1 -- 172.21.15.134:0/452767894 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b30007bf0 con 0x7f1b3813c370 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 4 set_mon_vals no callback set 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3559a640 10 monclient: dump: 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.466 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 10 monclient: shutdown 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/452767894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3813c370 msgr2=0x7f1b3813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3813c370 0x7f1b3813e760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009e80 tx=0x7f1b3002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/452767894 shutdown_connections 2024-09-16T15:10:17.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3813c370 0x7f1b3813e760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/452767894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b38136e40 0x7f1b3809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/452767894 >> 172.21.15.134:0/452767894 conn(0x7f1b380a6110 msgr2=0x7f1b380a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/452767894 shutdown_connections 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.455+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/452767894 wait complete. 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.456+0000 7f1b3cf1f640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.456+0000 7f1b3cf1f640 10 monclient: monmap: 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:17.468 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.456+0000 7f1b3cf1f640 1 Processor -- start 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.456+0000 7f1b3cf1f640 1 -- start start 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.456+0000 7f1b3cf1f640 10 monclient: init 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient: _add_conns ranks=[6,4,8,7,1,3,2,5,0] 2024-09-16T15:10:17.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 0x7f1b3811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-d con 0x7f1b38136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 0x7f1b381201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-c con 0x7f1b38137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1b3813c370 0x7f1b381206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): picked mon.noname-e con 0x7f1b3813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b38121dd0 con 0x7f1b38137220 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b38120e70 con 0x7f1b38136e40 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380a7530 con 0x7f1b3813c370 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3cf1f640 10 monclient(hunting): authenticate will time out at 1354.046301s 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b36d9d640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1b3813c370 msgr2=0x7f1b381206e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b36d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1b3813c370 0x7f1b381206e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 0x7f1b381201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 0x7f1b3811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 0x7f1b381201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40792/0 (socket says 172.21.15.134:40792) 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 1 -- 172.21.15.134:0/3753014361 learned_addr learned my addr 172.21.15.134:0/3753014361 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): get_auth_request con 0x7f1b38137220 auth_method 0 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 0x7f1b3811fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:37266/0 (socket says 172.21.15.134:37266) 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): get_auth_request con 0x7f1b38136e40 auth_method 0 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient(hunting): handle_auth_done global_id 34989 payload 824 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1b3813c370 msgr2=0x7f1b381206e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1b3813c370 0x7f1b381206e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 msgr2=0x7f1b381201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 0x7f1b381201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 monclient: found mon.noname-d 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b3659c640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b30009b30 con 0x7f1b38136e40 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.457+0000 7f1b35d9b640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1b38137220 0x7f1b381201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.459+0000 7f1b3659c640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 msgr2=0x7f1b3811fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.459+0000 7f1b3659c640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 0x7f1b3811fc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b28003930 tx=0x7f1b2800b9b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b280092e0 con 0x7f1b38136e40 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: dump: 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.473 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28003dd0 con 0x7f1b38136e40 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 5 monclient: authenticate success, global_id 34989 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 monclient: _renew_subs 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b380a3770 con 0x7f1b38136e40 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b28010400 con 0x7f1b38136e40 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 10 monclient: dump: 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.475 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.476 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b34d99640 4 set_mon_vals no callback set 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 monclient: _renew_subs 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b380a7a80 con 0x7f1b38136e40 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1b28005070 con 0x7f1b38136e40 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 0x7f1b381356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.460+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380a3ac0 con 0x7f1b380a2a60 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b26ffd640 1 --2- 172.21.15.134:0/3753014361 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b18038880 0x7f1b1803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f1b28049c10 con 0x7f1b38136e40 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 0x7f1b381356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient: get_auth_request con 0x7f1b380a2a60 auth_method 0 2024-09-16T15:10:17.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b36d9d640 1 --2- 172.21.15.134:0/3753014361 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b18038880 0x7f1b1803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b36d9d640 10 monclient: get_auth_request con 0x7f1b18038880 auth_method 0 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_done global_id 44749 payload 824 2024-09-16T15:10:17.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.461+0000 7f1b35d9b640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b38061ae0 con 0x7f1b380a2a60 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.463+0000 7f1b36d9d640 1 --2- 172.21.15.134:0/3753014361 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b18038880 0x7f1b1803ad40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b30033000 tx=0x7f1b30009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.463+0000 7f1b35d9b640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 msgr2=0x7f1b381356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.463+0000 7f1b35d9b640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 0x7f1b381356c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1b380a27c0 tx=0x7f1b2c00ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b2c0054a0 con 0x7f1b380a2a60 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 10 monclient: dump: 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.481 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1b2c00de20 con 0x7f1b380a2a60 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:17.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 msgr2=0x7f1b381356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 0x7f1b381356c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1b380a27c0 tx=0x7f1b2c00ff70 comp rx=0 tx=0).stop 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 msgr2=0x7f1b381356c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.464+0000 7f1b26ffd640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b380a2a60 0x7f1b381356c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b3cf1f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 0x7f1b38136010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b3cf1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b38071b20 con 0x7f1b38135c00 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b35d9b640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 0x7f1b38136010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b35d9b640 10 monclient: get_auth_request con 0x7f1b38135c00 auth_method 0 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b35d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b35d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.470+0000 7f1b35d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.471+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.471+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.471+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.471+0000 7f1b35d9b640 10 monclient(hunting): handle_auth_done global_id 44758 payload 824 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.471+0000 7f1b35d9b640 1 -- 172.21.15.134:0/3753014361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1b3807a680 con 0x7f1b38135c00 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b35d9b640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 msgr2=0x7f1b38136010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b35d9b640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 0x7f1b38136010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c018000 tx=0x7f1b2c003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b2c023070 con 0x7f1b38135c00 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b26ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.473+0000 7f1b26ffd640 10 monclient: dump: 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.484 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9173 (secure 0 0 0) 0x7f1b2c004ae0 con 0x7f1b38135c00 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 msgr2=0x7f1b38136010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 0x7f1b38136010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c018000 tx=0x7f1b2c003760 comp rx=0 tx=0).stop 2024-09-16T15:10:17.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 msgr2=0x7f1b38136010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b26ffd640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b38135c00 0x7f1b38136010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:17.486 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.487 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.488 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.489 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:17.490 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.491 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:17.492 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.494 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.498 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.499 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.502 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.503 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.504 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.505 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.506 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:17.507 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.508 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.508 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.508 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:17.508 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b18038880 msgr2=0x7f1b1803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/3753014361 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b18038880 0x7f1b1803ad40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b30033000 tx=0x7f1b30009000 comp rx=0 tx=0).stop 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 10 monclient: shutdown 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 msgr2=0x7f1b3811fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 --2- 172.21.15.134:0/3753014361 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b38136e40 0x7f1b3811fc60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b28003930 tx=0x7f1b2800b9b0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b36d9d640 1 -- 172.21.15.134:0/3753014361 reap_dead start 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b36d9d640 1 -- 172.21.15.134:0/3753014361 reap_dead start 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 shutdown_connections 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 >> 172.21.15.134:0/3753014361 conn(0x7f1b380a6110 msgr2=0x7f1b3813b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 shutdown_connections 2024-09-16T15:10:17.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.474+0000 7f1b3cf1f640 1 -- 172.21.15.134:0/3753014361 wait complete. 2024-09-16T15:10:17.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:17.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-16T15:10:17.510 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.606+0000 7f7a87f57640 10 monclient: get_monmap_and_config 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.606+0000 7f7a87f57640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.606+0000 7f7a87f57640 10 monclient: monmap: 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:17.607695+0000 2024-09-16T15:10:17.608 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:17.607695+0000 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:17.609 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:17.610 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.607+0000 7f7a87f57640 1 Processor -- start 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.607+0000 7f7a87f57640 1 -- start start 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.607+0000 7f7a87f57640 10 monclient: init 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient: _add_conns ranks=[4,6,9,10,7,2,8,0,3,1,5] 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-c con 0x7f7a80136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 0x7f7a80137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-d con 0x7f7a80137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7a80135dc0 0x7f7a80136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-a con 0x7f7a80135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8009abc0 con 0x7f7a80135dc0 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8009ba50 con 0x7f7a80136e40 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800a53c0 con 0x7f7a80137220 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a87f57640 10 monclient(hunting): authenticate will time out at 1354.197166s 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7a80135dc0 0x7f7a80136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40798/0 (socket says 172.21.15.134:40798) 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 1 -- 172.21.15.134:0/1956990698 learned_addr learned my addr 172.21.15.134:0/1956990698 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): get_auth_request con 0x7f7a80135dc0 auth_method 0 2024-09-16T15:10:17.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 0x7f7a80137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): get_auth_request con 0x7f7a80136e40 auth_method 0 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): get_auth_request con 0x7f7a80137220 auth_method 0 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient(hunting): handle_auth_done global_id 74331 payload 274 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 -- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 msgr2=0x7f7a80137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 0x7f7a80137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 -- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 msgr2=0x7f7a8009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 monclient: found mon.noname-a 2024-09-16T15:10:17.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a864f4640 1 -- 172.21.15.134:0/1956990698 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80073250 con 0x7f7a80135dc0 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.608+0000 7f7a854f2640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 0x7f7a80137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a864f4640 1 -- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80135dc0 msgr2=0x7f7a80136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a864f4640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80135dc0 0x7f7a80136220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009d90 tx=0x7f7a7802a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8397d45a3b8fc28a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 1 -- 172.21.15.134:0/1956990698 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a78007bc0 con 0x7f7a80135dc0 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: dump: 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.618 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 1 -- 172.21.15.134:0/1956990698 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7802d050 con 0x7f7a80135dc0 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a87f57640 5 monclient: authenticate success, global_id 74331 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a87f57640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a87f57640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 1 -- 172.21.15.134:0/1956990698 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a78007d40 con 0x7f7a80135dc0 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a87f57640 4 set_mon_vals no callback set 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.611+0000 7f7a84cf1640 10 monclient: dump: 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.620 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 10 monclient: shutdown 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 -- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80135dc0 msgr2=0x7f7a80136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80135dc0 0x7f7a80136220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009d90 tx=0x7f7a7802a490 comp rx=0 tx=0).stop 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 -- 172.21.15.134:0/1956990698 shutdown_connections 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80135dc0 0x7f7a80136220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7a80137220 0x7f7a80137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 --2- 172.21.15.134:0/1956990698 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7a80136e40 0x7f7a8009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 -- 172.21.15.134:0/1956990698 >> 172.21.15.134:0/1956990698 conn(0x7f7a800a6110 msgr2=0x7f7a800a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 -- 172.21.15.134:0/1956990698 shutdown_connections 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 1 -- 172.21.15.134:0/1956990698 wait complete. 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.612+0000 7f7a87f57640 10 monclient: monmap: 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:17.622 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 Processor -- start 2024-09-16T15:10:17.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 -- start start 2024-09-16T15:10:17.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient: init 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient: _add_conns ranks=[3,5,0,1,7,4,8,2,6] 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a80135dc0 0x7f7a801cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-g con 0x7f7a80135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a80136e40 0x7f7a80134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-h con 0x7f7a80136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 0x7f7a80135480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): picked mon.noname-a con 0x7f7a80137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801cb6f0 con 0x7f7a80137220 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800a89c0 con 0x7f7a80135dc0 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80136840 con 0x7f7a80136e40 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a87f57640 10 monclient(hunting): authenticate will time out at 1354.202638s 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 0x7f7a80135480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a85cf3640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a80135dc0 msgr2=0x7f7a801cb1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 0x7f7a80135480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39048/0 (socket says 172.21.15.134:39048) 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 learned_addr learned my addr 172.21.15.134:0/299289741 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): get_auth_request con 0x7f7a80137220 auth_method 0 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a80135dc0 0x7f7a801cb1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a854f2640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a80136e40 0x7f7a80134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a854f2640 10 monclient(hunting): get_auth_request con 0x7f7a80136e40 auth_method 0 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a854f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a854f2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a854f2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.613+0000 7f7a864f4640 10 monclient(hunting): handle_auth_done global_id 74340 payload 824 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a80136e40 msgr2=0x7f7a80134f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a80136e40 0x7f7a80134f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a80135dc0 msgr2=0x7f7a801cb1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a80135dc0 0x7f7a801cb1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 monclient: found mon.noname-a 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.614+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c0094d0 con 0x7f7a80137220 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 msgr2=0x7f7a80135480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a864f4640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 0x7f7a80135480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a7802a9b0 tx=0x7f7a78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a78007ea0 con 0x7f7a80137220 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 10 monclient: dump: 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.633 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 5 monclient: authenticate success, global_id 74340 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 10 monclient: _renew_subs 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a70009400 con 0x7f7a80137220 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 10 monclient: _renew_subs 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.616+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78009a70 con 0x7f7a80137220 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78004040 con 0x7f7a80137220 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a78004620 con 0x7f7a80137220 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a77fff640 4 set_mon_vals no callback set 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 10 monclient: dump: 2024-09-16T15:10:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.636 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a87f57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a87f57640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 0x7f7a68004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68007c70 con 0x7f7a68002840 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 0x7f7a68004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient: get_auth_request con 0x7f7a68002840 auth_method 0 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7a780041b0 con 0x7f7a80137220 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.617+0000 7f7a767fc640 1 --2- 172.21.15.134:0/299289741 >> v2:172.21.15.162:6800/980547263 conn(0x7f7a64038880 0x7f7a6403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.618+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_done global_id 44672 payload 824 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.618+0000 7f7a85cf3640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a68007e30 con 0x7f7a68002840 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.618+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f7a7806b0d0 con 0x7f7a80137220 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.618+0000 7f7a854f2640 1 --2- 172.21.15.134:0/299289741 >> v2:172.21.15.162:6800/980547263 conn(0x7f7a64038880 0x7f7a6403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.618+0000 7f7a854f2640 10 monclient: get_auth_request con 0x7f7a64038880 auth_method 0 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a85cf3640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 msgr2=0x7f7a68004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 0x7f7a68004ce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006c80 tx=0x7f7a7c00ef70 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a7c00cd30 con 0x7f7a68002840 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 10 monclient: dump: 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a854f2640 1 --2- 172.21.15.134:0/299289741 >> v2:172.21.15.162:6800/980547263 conn(0x7f7a64038880 0x7f7a6403ad40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a800a3ca0 tx=0x7f7a70008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7a7c00ce90 con 0x7f7a68002840 2024-09-16T15:10:17.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 msgr2=0x7f7a68004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 0x7f7a68004ce0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006c80 tx=0x7f7a7c00ef70 comp rx=0 tx=0).stop 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 msgr2=0x7f7a68004ce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.620+0000 7f7a767fc640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68002840 0x7f7a68004ce0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a87f57640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a87f57640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 0x7f7a680070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a87f57640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6800a130 con 0x7f7a68006c60 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 0x7f7a680070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient: get_auth_request con 0x7f7a68006c60 auth_method 0 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 10 monclient(hunting): handle_auth_done global_id 44681 payload 824 2024-09-16T15:10:17.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.627+0000 7f7a85cf3640 1 -- 172.21.15.134:0/299289741 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7a6800a2f0 con 0x7f7a68006c60 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a85cf3640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 msgr2=0x7f7a680070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a85cf3640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 0x7f7a680070d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c019000 tx=0x7f7a7c004730 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7a7c023070 con 0x7f7a68006c60 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 10 monclient: dump: 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.642 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7f7a7c006890 con 0x7f7a68006c60 2024-09-16T15:10:17.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 msgr2=0x7f7a680070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 0x7f7a680070d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c019000 tx=0x7f7a7c004730 comp rx=0 tx=0).stop 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 msgr2=0x7f7a680070d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a767fc640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7a68006c60 0x7f7a680070d0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c019000 tx=0x7f7a7c004730 comp rx=0 tx=0).stop 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.644 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.645 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:17.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:17.647 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:17.648 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.649 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:17.653 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.654 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.656 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.657 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.659 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.662 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.664 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.665 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:17.666 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 >> v2:172.21.15.162:6800/980547263 conn(0x7f7a64038880 msgr2=0x7f7a6403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 1 --2- 172.21.15.134:0/299289741 >> v2:172.21.15.162:6800/980547263 conn(0x7f7a64038880 0x7f7a6403ad40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a800a3ca0 tx=0x7f7a70008040 comp rx=0 tx=0).stop 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 10 monclient: shutdown 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 msgr2=0x7f7a80135480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a87f57640 1 --2- 172.21.15.134:0/299289741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a80137220 0x7f7a80135480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a7802a9b0 tx=0x7f7a78004a40 comp rx=0 tx=0).stop 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.630+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 reap_dead start 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.631+0000 7f7a864f4640 1 -- 172.21.15.134:0/299289741 reap_dead start 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.631+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 shutdown_connections 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.631+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 >> 172.21.15.134:0/299289741 conn(0x7f7a800a6110 msgr2=0x7f7a8006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.631+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 shutdown_connections 2024-09-16T15:10:17.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.631+0000 7f7a87f57640 1 -- 172.21.15.134:0/299289741 wait complete. 2024-09-16T15:10:17.669 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:17.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-16T15:10:17.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:10:17.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.765+0000 7fa233f5a640 10 monclient: get_monmap_and_config 2024-09-16T15:10:17.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.765+0000 7fa233f5a640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.765+0000 7fa233f5a640 10 monclient: monmap: 2024-09-16T15:10:17.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:10:17.766340+0000 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:10:17.766340+0000 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.767 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:10:17.768 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:10:17.768 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.765+0000 7fa233f5a640 1 Processor -- start 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 -- start start 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient: init 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient: _add_conns ranks=[4,9,8,0,6,10,3,2,7,5,1] 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-c con 0x7fa22c136eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 0x7fa22c137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fa22c137290 addr v2:172.21.15.134:3300/0 2024-09-16T15:10:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 0x7fa22c134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-e con 0x7fa22c133d10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c0a5560 con 0x7fa22c137290 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c09b240 con 0x7fa22c136eb0 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c09b520 con 0x7fa22c133d10 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa233f5a640 10 monclient(hunting): authenticate will time out at 1354.355683s 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2324f7640 1 -- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 msgr2=0x7fa22c134180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3304/0 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2324f7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 0x7fa22c134180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 0x7fa22c137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40814/0 (socket says 172.21.15.134:40814) 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 1 -- 172.21.15.134:0/322226674 learned_addr learned my addr 172.21.15.134:0/322226674 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): get_auth_request con 0x7fa22c136eb0 auth_method 0 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): get_auth_request con 0x7fa22c137290 auth_method 0 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.766+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 10 monclient(hunting): handle_auth_done global_id 5341 payload 274 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 -- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 msgr2=0x7fa22c134180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 --2- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 0x7fa22c134180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 -- 172.21.15.134:0/322226674 >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 msgr2=0x7fa22c137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 --2- 172.21.15.134:0/322226674 >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 0x7fa22c137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 monclient: found mon.noname-c 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa231cf6640 1 -- 172.21.15.134:0/322226674 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c073250 con 0x7fa22c136eb0 2024-09-16T15:10:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.767+0000 7fa2314f5640 1 --2- 172.21.15.134:0/322226674 >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 0x7fa22c137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.769+0000 7fa231cf6640 1 -- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 msgr2=0x7fa22c09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.769+0000 7fa231cf6640 1 --2- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22402a0b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8c3edd5a3573656a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 1 -- 172.21.15.134:0/322226674 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa22402ae10 con 0x7fa22c136eb0 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: dump: 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 1 -- 172.21.15.134:0/322226674 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240307b0 con 0x7fa22c136eb0 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 5 monclient: authenticate success, global_id 5341 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 10 monclient: get_monmap_and_config success 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 1 -- 172.21.15.134:0/322226674 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa224030dd0 con 0x7fa22c136eb0 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 4 set_mon_vals no callback set 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa230cf4640 10 monclient: dump: 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.782 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 10 monclient: shutdown 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 -- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 msgr2=0x7fa22c09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 --2- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22402a0b0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 -- 172.21.15.134:0/322226674 shutdown_connections 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 --2- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa22c133d10 0x7fa22c134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 --2- 172.21.15.134:0/322226674 >> v2:172.21.15.134:3300/0 conn(0x7fa22c137290 0x7fa22c137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 --2- 172.21.15.134:0/322226674 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c09bf00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 -- 172.21.15.134:0/322226674 >> 172.21.15.134:0/322226674 conn(0x7fa22c0a99a0 msgr2=0x7fa22c0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 -- 172.21.15.134:0/322226674 shutdown_connections 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.770+0000 7fa233f5a640 1 -- 172.21.15.134:0/322226674 wait complete. 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.771+0000 7fa233f5a640 10 monclient: build_initial_monmap 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.771+0000 7fa233f5a640 10 monclient: monmap: 2024-09-16T15:10:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.771+0000 7fa233f5a640 1 Processor -- start 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.771+0000 7fa233f5a640 1 -- start start 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.771+0000 7fa233f5a640 10 monclient: init 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient: _add_conns ranks=[0,4,3,6,5,2,1,7,8] 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 0x7fa22c132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-a con 0x7fa22c133d10 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c1332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-c con 0x7fa22c136eb0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa22c137290 0x7fa22c09d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): picked mon.noname-g con 0x7fa22c137290 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c09f970 con 0x7fa22c133d10 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c120450 con 0x7fa22c137290 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0a7710 con 0x7fa22c136eb0 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa233f5a640 10 monclient(hunting): authenticate will time out at 1354.361399s 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 0x7fa22c132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2324f7640 1 -- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa22c137290 msgr2=0x7fa22c09d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3301/0 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c1332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 0x7fa22c132d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39060/0 (socket says 172.21.15.134:39060) 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2324f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa22c137290 0x7fa22c09d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 -- 172.21.15.134:0/154615851 learned_addr learned my addr 172.21.15.134:0/154615851 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): get_auth_request con 0x7fa22c133d10 auth_method 0 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): get_auth_request con 0x7fa22c136eb0 auth_method 0 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient(hunting): handle_auth_done global_id 74358 payload 824 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 msgr2=0x7fa22c1332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c1332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa22c137290 msgr2=0x7fa22c09d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa22c137290 0x7fa22c09d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient: _finish_hunting 0 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 monclient: found mon.noname-a 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa231cf6640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220009a70 con 0x7fa22c133d10 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.772+0000 7fa2314f5640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa22c136eb0 0x7fa22c1332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.774+0000 7fa231cf6640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 msgr2=0x7fa22c132d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.774+0000 7fa231cf6640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 0x7fa22c132d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2240090b0 tx=0x7fa2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa224030d40 con 0x7fa22c133d10 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: dump: 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.788 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224003a80 con 0x7fa22c133d10 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 5 monclient: authenticate success, global_id 74358 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 10 monclient: _renew_subs 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2240094d0 con 0x7fa22c133d10 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa224003bf0 con 0x7fa22c133d10 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 10 monclient: dump: 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 10 monclient: _renew_subs 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22bfff640 4 set_mon_vals no callback set 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c0a7c60 con 0x7fa22c133d10 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.775+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa224039070 con 0x7fa22c133d10 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa233f5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa233f5a640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 0x7fa22c09dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c073880 con 0x7fa22c09d9c0 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa22a7fc640 1 --2- 172.21.15.134:0/154615851 >> v2:172.21.15.162:6800/980547263 conn(0x7fa218038880 0x7fa21803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(425..425 src has 1..425) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fa22406aaa0 con 0x7fa22c133d10 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2324f7640 1 --2- 172.21.15.134:0/154615851 >> v2:172.21.15.162:6800/980547263 conn(0x7fa218038880 0x7fa21803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 0x7fa22c09dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2324f7640 10 monclient: get_auth_request con 0x7fa218038880 auth_method 0 2024-09-16T15:10:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient: get_auth_request con 0x7fa22c09d9c0 auth_method 0 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 10 monclient(hunting): handle_auth_done global_id 54466 payload 824 2024-09-16T15:10:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.776+0000 7fa2314f5640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa22c11fbd0 con 0x7fa22c09d9c0 2024-09-16T15:10:17.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.778+0000 7fa2324f7640 1 --2- 172.21.15.134:0/154615851 >> v2:172.21.15.162:6800/980547263 conn(0x7fa218038880 0x7fa21803ad40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa21c003930 tx=0x7fa21c006c40 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa2314f5640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 msgr2=0x7fa22c09dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa2314f5640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 0x7fa22c09dda0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0a3a40 tx=0x7fa220007b30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa220017440 con 0x7fa22c09d9c0 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 10 monclient: dump: 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa22000f040 con 0x7fa22c09d9c0 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 msgr2=0x7fa22c09dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 0x7fa22c09dda0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0a3a40 tx=0x7fa220007b30 comp rx=0 tx=0).stop 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 msgr2=0x7fa22c09dda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.779+0000 7fa22a7fc640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09d9c0 0x7fa22c09dda0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0a3a40 tx=0x7fa220007b30 comp rx=0 tx=0).stop 2024-09-16T15:10:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.785+0000 7fa233f5a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.785+0000 7fa233f5a640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 0x7fa22c1341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.785+0000 7fa233f5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.785+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c071b20 con 0x7fa22c09e2e0 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 0x7fa22c1341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient: get_auth_request con 0x7fa22c09e2e0 auth_method 0 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 10 monclient(hunting): handle_auth_done global_id 54475 payload 824 2024-09-16T15:10:17.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.786+0000 7fa2314f5640 1 -- 172.21.15.134:0/154615851 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa22c061ae0 con 0x7fa22c09e2e0 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.788+0000 7fa2314f5640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 msgr2=0x7fa22c1341d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.788+0000 7fa2314f5640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 0x7fa22c1341d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa220000f80 tx=0x7fa220004ab0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa22002e070 con 0x7fa22c09e2e0 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 10 monclient: dump: 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:10:17.799 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9339 (secure 0 0 0) 0x7fa220015410 con 0x7fa22c09e2e0 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 msgr2=0x7fa22c1341d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 0x7fa22c1341d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa220000f80 tx=0x7fa220004ab0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 msgr2=0x7fa22c1341d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa22a7fc640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa22c09e2e0 0x7fa22c1341d0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa220000f80 tx=0x7fa220004ab0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 92, 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:10:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: 7 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:10:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:10:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.809 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.811 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.817 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:10:17.821 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:10:17.822 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:10:17.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 >> v2:172.21.15.162:6800/980547263 conn(0x7fa218038880 msgr2=0x7fa21803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 --2- 172.21.15.134:0/154615851 >> v2:172.21.15.162:6800/980547263 conn(0x7fa218038880 0x7fa21803ad40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa21c003930 tx=0x7fa21c006c40 comp rx=0 tx=0).stop 2024-09-16T15:10:17.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 10 monclient: shutdown 2024-09-16T15:10:17.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 msgr2=0x7fa22c132d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 --2- 172.21.15.134:0/154615851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c133d10 0x7fa22c132d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2240090b0 tx=0x7fa2240046c0 comp rx=0 tx=0).stop 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa2324f7640 1 -- 172.21.15.134:0/154615851 reap_dead start 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa2324f7640 1 -- 172.21.15.134:0/154615851 reap_dead start 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 shutdown_connections 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 >> 172.21.15.134:0/154615851 conn(0x7fa22c0a99a0 msgr2=0x7fa22c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 shutdown_connections 2024-09-16T15:10:17.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:10:17.789+0000 7fa233f5a640 1 -- 172.21.15.134:0/154615851 wait complete. 2024-09-16T15:10:17.824 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:10:17.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:10:17.991 INFO:tasks.rados.rados.0.smithi162.stdout:2140: done (0 left) 2024-09-16T15:10:17.991 INFO:tasks.rados.rados.0.smithi162.stdout:2141: snap_create 2024-09-16T15:10:18.994 INFO:tasks.rados.rados.0.smithi162.stdout:2141: done (0 left) 2024-09-16T15:10:18.994 INFO:tasks.rados.rados.0.smithi162.stdout:2142: read oid 27 snap 200 2024-09-16T15:10:18.994 INFO:tasks.rados.rados.0.smithi162.stdout:2142: expect (ObjNum 753 snap 197 seq_num 753) 2024-09-16T15:10:18.994 INFO:tasks.rados.rados.0.smithi162.stdout:2143: write (excl) oid 32 current snap is 201 2024-09-16T15:10:18.994 INFO:tasks.rados.rados.0.smithi162.stdout:2143: seq_num 767 ranges {490969=551283,1816630=625589} 2024-09-16T15:10:18.999 INFO:tasks.rados.rados.0.smithi162.stdout:2143: writing smithi16234619-32 from 490969 to 1042252 tid 1 2024-09-16T15:10:19.007 INFO:tasks.rados.rados.0.smithi162.stdout:2143: writing smithi16234619-32 from 1816630 to 2442219 tid 2 2024-09-16T15:10:19.007 INFO:tasks.rados.rados.0.smithi162.stdout:2144: snap_create 2024-09-16T15:10:19.007 INFO:tasks.rados.rados.0.smithi162.stdout:2142: expect (ObjNum 753 snap 197 seq_num 753) 2024-09-16T15:10:19.119 INFO:tasks.rados.rados.0.smithi162.stdout:2143: finishing write tid 1 to smithi16234619-32 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: finishing write tid 2 to smithi16234619-32 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: finishing write tid 3 to smithi16234619-32 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: finishing write tid 4 to smithi16234619-32 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: oid 32 updating version 0 to 569 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: oid 32 updating version 569 to 570 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: oid 32 version 570 is already newer than 568 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 570 (ObjNum 767 snap 201 seq_num 767) dirty exists 2024-09-16T15:10:19.120 INFO:tasks.rados.rados.0.smithi162.stdout:2143: left oid 32 (ObjNum 767 snap 201 seq_num 767) 2024-09-16T15:10:20.000 INFO:tasks.rados.rados.0.smithi162.stdout:2142: done (2 left) 2024-09-16T15:10:20.000 INFO:tasks.rados.rados.0.smithi162.stdout:2143: done (1 left) 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:2144: done (0 left) 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:2145: copy_from oid 43 from oid 39 current snap is 202 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:2146: rollback oid 3 current snap is 202 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 195 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:2147: write (excl) oid 29 current snap is 202 2024-09-16T15:10:20.001 INFO:tasks.rados.rados.0.smithi162.stdout:2147: seq_num 768 ranges {507776=742885,1991465=485600} 2024-09-16T15:10:20.004 INFO:tasks.rados.rados.0.smithi162.stdout:2147: writing smithi16234619-29 from 507776 to 1250661 tid 1 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2147: writing smithi16234619-29 from 1991465 to 2477065 tid 2 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2148: copy_from oid 17 from oid 6 current snap is 202 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2145: finishing copy_from to smithi16234619-43 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2145: got expected ENOENT (src dne) 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2145: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2146: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:2146: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:10:20.012 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 109 (ObjNum 401 snap 0 seq_num 2027920992) dirty dne 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2145: done (3 left) 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2146: done (2 left) 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2149: copy_from oid 39 from oid 30 current snap is 202 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2150: copy_from oid 24 from oid 36 current snap is 202 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2151: copy_from oid 38 from oid 47 current snap is 202 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2152: delete oid 49 current snap is 202 2024-09-16T15:10:20.013 INFO:tasks.rados.rados.0.smithi162.stdout:2148: finishing copy_from to smithi16234619-17 2024-09-16T15:10:20.014 INFO:tasks.rados.rados.0.smithi162.stdout:2148: got expected ENOENT (src dne) 2024-09-16T15:10:20.014 INFO:tasks.rados.rados.0.smithi162.stdout:2148: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:10:20.014 INFO:tasks.rados.rados.0.smithi162.stdout:2148: done (5 left) 2024-09-16T15:10:20.014 INFO:tasks.rados.rados.0.smithi162.stdout:2152: done (4 left) 2024-09-16T15:10:20.014 INFO:tasks.rados.rados.0.smithi162.stdout:2153: snap_remove snap 201 2024-09-16T15:10:21.014 INFO:tasks.rados.rados.0.smithi162.stdout:2153: done (4 left) 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2154: read oid 9 snap -1 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2154: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2155: rollback oid 25 current snap is 202 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 195 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2156: delete oid 10 current snap is 202 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: finishing write tid 1 to smithi16234619-29 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: finishing write tid 2 to smithi16234619-29 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: finishing write tid 3 to smithi16234619-29 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: finishing write tid 4 to smithi16234619-29 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: oid 29 updating version 0 to 956 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: oid 29 version 956 is already newer than 955 2024-09-16T15:10:21.015 INFO:tasks.rados.rados.0.smithi162.stdout:2147: oid 29 version 956 is already newer than 954 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 956 (ObjNum 768 snap 202 seq_num 768) dirty exists 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2147: left oid 29 (ObjNum 768 snap 202 seq_num 768) 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2151: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2151: finishing copy_from to smithi16234619-38 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 524 (ObjNum 688 snap 178 seq_num 688) dirty exists 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2150: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2150: finishing copy_from to smithi16234619-24 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 814 (ObjNum 741 snap 194 seq_num 741) dirty exists 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2149: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:2149: finishing copy_from to smithi16234619-39 2024-09-16T15:10:21.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 561 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:10:21.017 INFO:tasks.rados.rados.0.smithi162.stdout:2154: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2155: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2155: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 861 (ObjNum 736 snap 194 seq_num 736) dirty exists 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2147: done (6 left) 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2149: done (5 left) 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2150: done (4 left) 2024-09-16T15:10:21.129 INFO:tasks.rados.rados.0.smithi162.stdout:2151: done (3 left) 2024-09-16T15:10:21.130 INFO:tasks.rados.rados.0.smithi162.stdout:2154: done (2 left) 2024-09-16T15:10:21.130 INFO:tasks.rados.rados.0.smithi162.stdout:2155: done (1 left) 2024-09-16T15:10:21.130 INFO:tasks.rados.rados.0.smithi162.stdout:2156: done (0 left) 2024-09-16T15:10:21.130 INFO:tasks.rados.rados.0.smithi162.stdout:2157: snap_create 2024-09-16T15:10:22.623 INFO:tasks.rados.rados.0.smithi162.stdout:2157: done (0 left) 2024-09-16T15:10:22.623 INFO:tasks.rados.rados.0.smithi162.stdout:2158: rollback oid 2 current snap is 203 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 200 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:2159: rollback oid 42 current snap is 203 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 200 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:2160: read oid 41 snap -1 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:2160: expect deleted 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:2161: write oid 44 current snap is 203 2024-09-16T15:10:22.624 INFO:tasks.rados.rados.0.smithi162.stdout:2161: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2024-09-16T15:10:22.629 INFO:tasks.rados.rados.0.smithi162.stdout:2161: writing smithi16234619-44 from 558197 to 1126829 tid 1 2024-09-16T15:10:22.636 INFO:tasks.rados.rados.0.smithi162.stdout:2161: writing smithi16234619-44 from 1650558 to 2283485 tid 2 2024-09-16T15:10:22.640 INFO:tasks.rados.rados.0.smithi162.stdout:2161: writing smithi16234619-44 from 2773245 to 2958197 tid 3 2024-09-16T15:10:22.640 INFO:tasks.rados.rados.0.smithi162.stdout:2159: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2158: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2158: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2162: read oid 35 snap -1 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2162: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2160: done (4 left) 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2163: read oid 47 snap -1 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2163: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2164: write (excl) oid 38 current snap is 203 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2159: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2158: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1003 (ObjNum 737 snap 194 seq_num 737) dirty exists 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:2159: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:10:22.641 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 963 (ObjNum 704 snap 182 seq_num 704) dirty exists 2024-09-16T15:10:22.642 INFO:tasks.rados.rados.0.smithi162.stdout:2164: seq_num 772 ranges {575004=643881,1709040=492938,2917021=57983} 2024-09-16T15:10:22.647 INFO:tasks.rados.rados.0.smithi162.stdout:2164: writing smithi16234619-38 from 575004 to 1218885 tid 1 2024-09-16T15:10:22.653 INFO:tasks.rados.rados.0.smithi162.stdout:2164: writing smithi16234619-38 from 1709040 to 2201978 tid 2 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:2164: writing smithi16234619-38 from 2917021 to 2975004 tid 3 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:2158: done (5 left) 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:2159: done (4 left) 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:2165: rollback oid 26 current snap is 203 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 199 2024-09-16T15:10:22.657 INFO:tasks.rados.rados.0.smithi162.stdout:2163: expect (ObjNum 688 snap 178 seq_num 688) 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:2165: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:2165: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:2165: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1007 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:2161: finishing write tid 1 to smithi16234619-44 2024-09-16T15:10:22.866 INFO:tasks.rados.rados.0.smithi162.stdout:2161: finishing write tid 2 to smithi16234619-44 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: finishing write tid 3 to smithi16234619-44 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: finishing write tid 4 to smithi16234619-44 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: finishing write tid 5 to smithi16234619-44 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: oid 44 updating version 0 to 893 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: oid 44 updating version 893 to 894 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: oid 44 updating version 894 to 895 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: oid 44 updating version 895 to 896 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 896 (ObjNum 771 snap 203 seq_num 771) dirty exists 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2161: left oid 44 (ObjNum 771 snap 203 seq_num 771) 2024-09-16T15:10:22.867 INFO:tasks.rados.rados.0.smithi162.stdout:2162: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:10:22.968 INFO:tasks.rados.rados.0.smithi162.stdout:2164: finishing write tid 1 to smithi16234619-38 2024-09-16T15:10:22.968 INFO:tasks.rados.rados.0.smithi162.stdout:2164: finishing write tid 2 to smithi16234619-38 2024-09-16T15:10:22.968 INFO:tasks.rados.rados.0.smithi162.stdout:2161: done (4 left) 2024-09-16T15:10:22.968 INFO:tasks.rados.rados.0.smithi162.stdout:2162: done (3 left) 2024-09-16T15:10:22.968 INFO:tasks.rados.rados.0.smithi162.stdout:2163: done (2 left) 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2165: done (1 left) 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2166: delete oid 33 current snap is 203 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: finishing write tid 3 to smithi16234619-38 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: finishing write tid 4 to smithi16234619-38 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: finishing write tid 5 to smithi16234619-38 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: oid 38 updating version 0 to 897 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: oid 38 updating version 897 to 900 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: oid 38 version 900 is already newer than 898 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: oid 38 version 900 is already newer than 899 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 900 (ObjNum 772 snap 203 seq_num 772) dirty exists 2024-09-16T15:10:22.969 INFO:tasks.rados.rados.0.smithi162.stdout:2164: left oid 38 (ObjNum 772 snap 203 seq_num 772) 2024-09-16T15:10:22.970 INFO:tasks.rados.rados.0.smithi162.stdout:2164: done (1 left) 2024-09-16T15:10:22.970 INFO:tasks.rados.rados.0.smithi162.stdout:2166: done (0 left) 2024-09-16T15:10:22.970 INFO:tasks.rados.rados.0.smithi162.stdout:2167: snap_create 2024-09-16T15:10:23.725 INFO:tasks.rados.rados.0.smithi162.stdout:2167: done (0 left) 2024-09-16T15:10:23.725 INFO:tasks.rados.rados.0.smithi162.stdout:2168: snap_remove snap 203 2024-09-16T15:10:23.993 INFO:tasks.rados.rados.0.smithi162.stdout:2168: done (0 left) 2024-09-16T15:10:23.993 INFO:tasks.rados.rados.0.smithi162.stdout:2169: read oid 41 snap -1 2024-09-16T15:10:23.993 INFO:tasks.rados.rados.0.smithi162.stdout:2169: expect deleted 2024-09-16T15:10:23.993 INFO:tasks.rados.rados.0.smithi162.stdout:2170: delete oid 3 current snap is 204 2024-09-16T15:10:24.004 INFO:tasks.rados.rados.0.smithi162.stdout:2169: done (1 left) 2024-09-16T15:10:24.004 INFO:tasks.rados.rados.0.smithi162.stdout:2170: done (0 left) 2024-09-16T15:10:24.004 INFO:tasks.rados.rados.0.smithi162.stdout:2171: snap_remove snap 199 2024-09-16T15:10:25.007 INFO:tasks.rados.rados.0.smithi162.stdout:2171: done (0 left) 2024-09-16T15:10:25.007 INFO:tasks.rados.rados.0.smithi162.stdout:2172: delete oid 47 current snap is 204 2024-09-16T15:10:25.011 INFO:tasks.rados.rados.0.smithi162.stdout:2172: done (0 left) 2024-09-16T15:10:25.011 INFO:tasks.rados.rados.0.smithi162.stdout:2173: write (excl) oid 22 current snap is 204 2024-09-16T15:10:25.011 INFO:tasks.rados.rados.0.smithi162.stdout:2173: seq_num 776 ranges {642232=661230,1942968=783923} 2024-09-16T15:10:25.018 INFO:tasks.rados.rados.0.smithi162.stdout:2173: writing smithi16234619-22 from 642232 to 1303462 tid 1 2024-09-16T15:10:25.026 INFO:tasks.rados.rados.0.smithi162.stdout:2173: writing smithi16234619-22 from 1942968 to 2726891 tid 2 2024-09-16T15:10:25.026 INFO:tasks.rados.rados.0.smithi162.stdout:2174: write oid 15 current snap is 204 2024-09-16T15:10:25.026 INFO:tasks.rados.rados.0.smithi162.stdout:2174: seq_num 777 ranges {659039=736479,2001450=643934} 2024-09-16T15:10:25.034 INFO:tasks.rados.rados.0.smithi162.stdout:2174: writing smithi16234619-15 from 659039 to 1395518 tid 1 2024-09-16T15:10:25.041 INFO:tasks.rados.rados.0.smithi162.stdout:2174: writing smithi16234619-15 from 2001450 to 2645384 tid 2 2024-09-16T15:10:25.041 INFO:tasks.rados.rados.0.smithi162.stdout:2175: write oid 20 current snap is 204 2024-09-16T15:10:25.041 INFO:tasks.rados.rados.0.smithi162.stdout:2175: seq_num 778 ranges {675846=411728,1543579=787592,2848853=226993} 2024-09-16T15:10:25.046 INFO:tasks.rados.rados.0.smithi162.stdout:2175: writing smithi16234619-20 from 675846 to 1087574 tid 1 2024-09-16T15:10:25.053 INFO:tasks.rados.rados.0.smithi162.stdout:2175: writing smithi16234619-20 from 1543579 to 2331171 tid 2 2024-09-16T15:10:25.057 INFO:tasks.rados.rados.0.smithi162.stdout:2175: writing smithi16234619-20 from 2848853 to 3075846 tid 3 2024-09-16T15:10:25.057 INFO:tasks.rados.rados.0.smithi162.stdout:2176: write (excl) oid 33 current snap is 204 2024-09-16T15:10:25.058 INFO:tasks.rados.rados.0.smithi162.stdout:2176: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2024-09-16T15:10:25.064 INFO:tasks.rados.rados.0.smithi162.stdout:2176: writing smithi16234619-33 from 692653 to 1179630 tid 1 2024-09-16T15:10:25.071 INFO:tasks.rados.rados.0.smithi162.stdout:2176: writing smithi16234619-33 from 1602061 to 2249664 tid 2 2024-09-16T15:10:25.076 INFO:tasks.rados.rados.0.smithi162.stdout:2176: writing smithi16234619-33 from 2992629 to 3092653 tid 3 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2177: copy_from oid 12 from oid 43 current snap is 204 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: finishing write tid 1 to smithi16234619-22 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: finishing write tid 2 to smithi16234619-22 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: finishing write tid 3 to smithi16234619-22 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: finishing write tid 4 to smithi16234619-22 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: oid 22 updating version 0 to 783 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: oid 22 updating version 783 to 784 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: oid 22 updating version 784 to 785 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 785 (ObjNum 776 snap 204 seq_num 776) dirty exists 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: left oid 22 (ObjNum 776 snap 204 seq_num 776) 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2173: done (4 left) 2024-09-16T15:10:25.077 INFO:tasks.rados.rados.0.smithi162.stdout:2178: delete oid 4 current snap is 204 2024-09-16T15:10:25.078 INFO:tasks.rados.rados.0.smithi162.stdout:2174: finishing write tid 1 to smithi16234619-15 2024-09-16T15:10:25.079 INFO:tasks.rados.rados.0.smithi162.stdout:2178: done (4 left) 2024-09-16T15:10:25.079 INFO:tasks.rados.rados.0.smithi162.stdout:2179: delete oid 22 current snap is 204 2024-09-16T15:10:25.080 INFO:tasks.rados.rados.0.smithi162.stdout:2174: finishing write tid 2 to smithi16234619-15 2024-09-16T15:10:25.080 INFO:tasks.rados.rados.0.smithi162.stdout:2174: finishing write tid 3 to smithi16234619-15 2024-09-16T15:10:25.081 INFO:tasks.rados.rados.0.smithi162.stdout:2179: done (4 left) 2024-09-16T15:10:25.081 INFO:tasks.rados.rados.0.smithi162.stdout:2180: read oid 21 snap 200 2024-09-16T15:10:25.081 INFO:tasks.rados.rados.0.smithi162.stdout:2180: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:10:25.081 INFO:tasks.rados.rados.0.smithi162.stdout:2181: snap_remove snap 202 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2181: done (5 left) 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2182: delete oid 47 current snap is 204 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2174: finishing write tid 4 to smithi16234619-15 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2174: oid 15 updating version 0 to 854 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2174: oid 15 version 854 is already newer than 852 2024-09-16T15:10:26.008 INFO:tasks.rados.rados.0.smithi162.stdout:2174: oid 15 version 854 is already newer than 853 2024-09-16T15:10:26.009 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 854 (ObjNum 777 snap 204 seq_num 777) dirty exists 2024-09-16T15:10:26.009 INFO:tasks.rados.rados.0.smithi162.stdout:2174: left oid 15 (ObjNum 777 snap 204 seq_num 777) 2024-09-16T15:10:26.009 INFO:tasks.rados.rados.0.smithi162.stdout:2180: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:10:26.181 INFO:tasks.rados.rados.0.smithi162.stdout:2175: finishing write tid 1 to smithi16234619-20 2024-09-16T15:10:26.181 INFO:tasks.rados.rados.0.smithi162.stdout:2175: finishing write tid 2 to smithi16234619-20 2024-09-16T15:10:26.181 INFO:tasks.rados.rados.0.smithi162.stdout:2177: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:10:26.181 INFO:tasks.rados.rados.0.smithi162.stdout:2175: finishing write tid 3 to smithi16234619-20 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: finishing write tid 4 to smithi16234619-20 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: finishing write tid 5 to smithi16234619-20 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: oid 20 updating version 0 to 792 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: oid 20 updating version 792 to 794 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: oid 20 version 794 is already newer than 793 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: oid 20 updating version 794 to 795 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 795 (ObjNum 778 snap 204 seq_num 778) dirty exists 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: left oid 20 (ObjNum 778 snap 204 seq_num 778) 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2176: finishing write tid 1 to smithi16234619-33 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2174: done (5 left) 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2175: done (4 left) 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2180: done (3 left) 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2182: done (2 left) 2024-09-16T15:10:26.182 INFO:tasks.rados.rados.0.smithi162.stdout:2183: read oid 16 snap -1 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2183: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2184: read oid 14 snap -1 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2184: expect deleted 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2185: delete oid 38 current snap is 204 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2177: finishing copy_from to smithi16234619-12 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 888 (ObjNum 742 snap 194 seq_num 742) dirty exists 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: finishing write tid 2 to smithi16234619-33 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: finishing write tid 3 to smithi16234619-33 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: finishing write tid 4 to smithi16234619-33 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: finishing write tid 5 to smithi16234619-33 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: oid 33 updating version 0 to 797 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: oid 33 updating version 797 to 799 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: oid 33 version 799 is already newer than 798 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:2176: oid 33 version 799 is already newer than 796 2024-09-16T15:10:26.183 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 799 (ObjNum 779 snap 204 seq_num 779) dirty exists 2024-09-16T15:10:26.184 INFO:tasks.rados.rados.0.smithi162.stdout:2176: left oid 33 (ObjNum 779 snap 204 seq_num 779) 2024-09-16T15:10:26.185 INFO:tasks.rados.rados.0.smithi162.stdout:2183: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:10:26.308 INFO:tasks.rados.rados.0.smithi162.stdout:2176: done (4 left) 2024-09-16T15:10:26.309 INFO:tasks.rados.rados.0.smithi162.stdout:2177: done (3 left) 2024-09-16T15:10:26.309 INFO:tasks.rados.rados.0.smithi162.stdout:2183: done (2 left) 2024-09-16T15:10:26.309 INFO:tasks.rados.rados.0.smithi162.stdout:2184: done (1 left) 2024-09-16T15:10:26.309 INFO:tasks.rados.rados.0.smithi162.stdout:2185: done (0 left) 2024-09-16T15:10:26.309 INFO:tasks.rados.rados.0.smithi162.stdout:2186: snap_create 2024-09-16T15:10:27.033 INFO:tasks.rados.rados.0.smithi162.stdout:2186: done (0 left) 2024-09-16T15:10:27.033 INFO:tasks.rados.rados.0.smithi162.stdout:2187: delete oid 20 current snap is 205 2024-09-16T15:10:27.036 INFO:tasks.rados.rados.0.smithi162.stdout:2187: done (0 left) 2024-09-16T15:10:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:2188: rollback oid 19 current snap is 205 2024-09-16T15:10:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 204 2024-09-16T15:10:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:2189: write oid 22 current snap is 205 2024-09-16T15:10:27.037 INFO:tasks.rados.rados.0.smithi162.stdout:2189: seq_num 785 ranges {793495=654824,1952953=658610,3157167=36328} 2024-09-16T15:10:27.044 INFO:tasks.rados.rados.0.smithi162.stdout:2189: writing smithi16234619-22 from 793495 to 1448319 tid 1 2024-09-16T15:10:27.050 INFO:tasks.rados.rados.0.smithi162.stdout:2189: writing smithi16234619-22 from 1952953 to 2611563 tid 2 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:2189: writing smithi16234619-22 from 3157167 to 3193495 tid 3 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:2190: snap_create 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:2188: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:2188: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:2188: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:10:27.054 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 865 (ObjNum 736 snap 194 seq_num 736) dirty exists 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: finishing write tid 1 to smithi16234619-22 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: finishing write tid 2 to smithi16234619-22 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: finishing write tid 3 to smithi16234619-22 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: finishing write tid 4 to smithi16234619-22 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: finishing write tid 5 to smithi16234619-22 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: oid 22 updating version 0 to 788 2024-09-16T15:10:27.077 INFO:tasks.rados.rados.0.smithi162.stdout:2189: oid 22 updating version 788 to 790 2024-09-16T15:10:27.078 INFO:tasks.rados.rados.0.smithi162.stdout:2189: oid 22 version 790 is already newer than 789 2024-09-16T15:10:27.078 INFO:tasks.rados.rados.0.smithi162.stdout:2189: oid 22 version 790 is already newer than 787 2024-09-16T15:10:27.078 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 790 (ObjNum 785 snap 205 seq_num 785) dirty exists 2024-09-16T15:10:27.078 INFO:tasks.rados.rados.0.smithi162.stdout:2189: left oid 22 (ObjNum 785 snap 205 seq_num 785) 2024-09-16T15:10:28.041 INFO:tasks.rados.rados.0.smithi162.stdout:2188: done (2 left) 2024-09-16T15:10:28.041 INFO:tasks.rados.rados.0.smithi162.stdout:2189: done (1 left) 2024-09-16T15:10:28.041 INFO:tasks.rados.rados.0.smithi162.stdout:2190: done (0 left) 2024-09-16T15:10:28.041 INFO:tasks.rados.rados.0.smithi162.stdout:2191: delete oid 5 current snap is 206 2024-09-16T15:10:28.043 INFO:tasks.rados.rados.0.smithi162.stdout:2191: done (0 left) 2024-09-16T15:10:28.043 INFO:tasks.rados.rados.0.smithi162.stdout:2192: read oid 38 snap -1 2024-09-16T15:10:28.043 INFO:tasks.rados.rados.0.smithi162.stdout:2192: expect deleted 2024-09-16T15:10:28.044 INFO:tasks.rados.rados.0.smithi162.stdout:2193: write oid 33 current snap is 206 2024-09-16T15:10:28.044 INFO:tasks.rados.rados.0.smithi162.stdout:2193: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2024-09-16T15:10:28.048 INFO:tasks.rados.rados.0.smithi162.stdout:2193: writing smithi16234619-33 from 427109 to 832431 tid 1 2024-09-16T15:10:28.055 INFO:tasks.rados.rados.0.smithi162.stdout:2193: writing smithi16234619-33 from 1553564 to 2099490 tid 2 2024-09-16T15:10:28.064 INFO:tasks.rados.rados.0.smithi162.stdout:2193: writing smithi16234619-33 from 2579307 to 3227109 tid 3 2024-09-16T15:10:28.064 INFO:tasks.rados.rados.0.smithi162.stdout:2194: snap_create 2024-09-16T15:10:28.086 INFO:tasks.rados.rados.0.smithi162.stdout:2193: finishing write tid 1 to smithi16234619-33 2024-09-16T15:10:28.087 INFO:tasks.rados.rados.0.smithi162.stdout:2193: finishing write tid 2 to smithi16234619-33 2024-09-16T15:10:28.087 INFO:tasks.rados.rados.0.smithi162.stdout:2193: finishing write tid 3 to smithi16234619-33 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: finishing write tid 4 to smithi16234619-33 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: finishing write tid 5 to smithi16234619-33 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: oid 33 updating version 0 to 803 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: oid 33 updating version 803 to 804 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: oid 33 version 804 is already newer than 802 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:2193: oid 33 version 804 is already newer than 801 2024-09-16T15:10:28.090 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 804 (ObjNum 787 snap 206 seq_num 787) dirty exists 2024-09-16T15:10:28.091 INFO:tasks.rados.rados.0.smithi162.stdout:2193: left oid 33 (ObjNum 787 snap 206 seq_num 787) 2024-09-16T15:10:28.995 INFO:tasks.rados.rados.0.smithi162.stdout:2192: done (2 left) 2024-09-16T15:10:28.995 INFO:tasks.rados.rados.0.smithi162.stdout:2193: done (1 left) 2024-09-16T15:10:28.995 INFO:tasks.rados.rados.0.smithi162.stdout:2194: done (0 left) 2024-09-16T15:10:28.995 INFO:tasks.rados.rados.0.smithi162.stdout:2195: write oid 44 current snap is 207 2024-09-16T15:10:28.995 INFO:tasks.rados.rados.0.smithi162.stdout:2195: seq_num 788 ranges {443916=480571,1612046=405937,2606730=495052} 2024-09-16T15:10:28.999 INFO:tasks.rados.rados.0.smithi162.stdout:2195: writing smithi16234619-44 from 443916 to 924487 tid 1 2024-09-16T15:10:29.006 INFO:tasks.rados.rados.0.smithi162.stdout:2195: writing smithi16234619-44 from 1612046 to 2017983 tid 2 2024-09-16T15:10:29.013 INFO:tasks.rados.rados.0.smithi162.stdout:2195: writing smithi16234619-44 from 2606730 to 3101782 tid 3 2024-09-16T15:10:29.013 INFO:tasks.rados.rados.0.smithi162.stdout:2196: rollback oid 31 current snap is 207 2024-09-16T15:10:29.014 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 206 2024-09-16T15:10:29.014 INFO:tasks.rados.rados.0.smithi162.stdout:2197: delete oid 36 current snap is 207 2024-09-16T15:10:29.025 INFO:tasks.rados.rados.0.smithi162.stdout:2197: done (2 left) 2024-09-16T15:10:29.025 INFO:tasks.rados.rados.0.smithi162.stdout:2198: snap_remove snap 200 2024-09-16T15:10:29.997 INFO:tasks.rados.rados.0.smithi162.stdout:2196: finishing rollback tid 0 to smithi16234619-31 2024-09-16T15:10:29.997 INFO:tasks.rados.rados.0.smithi162.stdout:2198: done (2 left) 2024-09-16T15:10:29.997 INFO:tasks.rados.rados.0.smithi162.stdout:2199: read oid 43 snap -1 2024-09-16T15:10:29.997 INFO:tasks.rados.rados.0.smithi162.stdout:2199: expect (ObjNum 742 snap 194 seq_num 742) 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2196: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2196: finishing rollback tid 2 to smithi16234619-31 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 847 (ObjNum 739 snap 194 seq_num 739) dirty exists 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: finishing write tid 1 to smithi16234619-44 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: finishing write tid 2 to smithi16234619-44 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: finishing write tid 3 to smithi16234619-44 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: finishing write tid 4 to smithi16234619-44 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: finishing write tid 5 to smithi16234619-44 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: oid 44 updating version 0 to 905 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: oid 44 version 905 is already newer than 904 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: oid 44 version 905 is already newer than 903 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:2195: oid 44 version 905 is already newer than 902 2024-09-16T15:10:29.998 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 905 (ObjNum 788 snap 207 seq_num 788) dirty exists 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2195: left oid 44 (ObjNum 788 snap 207 seq_num 788) 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2195: done (2 left) 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2196: done (1 left) 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2200: rollback oid 16 current snap is 207 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 206 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2201: write (excl) oid 30 current snap is 207 2024-09-16T15:10:29.999 INFO:tasks.rados.rados.0.smithi162.stdout:2201: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2024-09-16T15:10:30.004 INFO:tasks.rados.rados.0.smithi162.stdout:2201: writing smithi16234619-30 from 477530 to 1108599 tid 1 2024-09-16T15:10:30.012 INFO:tasks.rados.rados.0.smithi162.stdout:2201: writing smithi16234619-30 from 1612657 to 2022263 tid 2 2024-09-16T15:10:30.017 INFO:tasks.rados.rados.0.smithi162.stdout:2201: writing smithi16234619-30 from 2545223 to 3062819 tid 3 2024-09-16T15:10:30.017 INFO:tasks.rados.rados.0.smithi162.stdout:2202: copy_from oid 28 from oid 37 current snap is 207 2024-09-16T15:10:30.017 INFO:tasks.rados.rados.0.smithi162.stdout:2199: expect (ObjNum 742 snap 194 seq_num 742) 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2200: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2200: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2200: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 908 (ObjNum 762 snap 198 seq_num 762) dirty exists 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2201: finishing write tid 1 to smithi16234619-30 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2201: finishing write tid 2 to smithi16234619-30 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2201: finishing write tid 3 to smithi16234619-30 2024-09-16T15:10:30.156 INFO:tasks.rados.rados.0.smithi162.stdout:2201: finishing write tid 4 to smithi16234619-30 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: finishing write tid 5 to smithi16234619-30 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: oid 30 updating version 0 to 856 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: oid 30 version 856 is already newer than 855 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: oid 30 updating version 856 to 857 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: oid 30 updating version 857 to 858 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 858 (ObjNum 790 snap 207 seq_num 790) dirty exists 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2201: left oid 30 (ObjNum 790 snap 207 seq_num 790) 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2202: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2202: finishing copy_from to smithi16234619-28 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 586 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:10:30.157 INFO:tasks.rados.rados.0.smithi162.stdout:2199: done (3 left) 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2200: done (2 left) 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2201: done (1 left) 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2202: done (0 left) 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2203: copy_from oid 36 from oid 18 current snap is 207 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2204: delete oid 21 current snap is 207 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2204: done (1 left) 2024-09-16T15:10:30.158 INFO:tasks.rados.rados.0.smithi162.stdout:2205: snap_remove snap 205 2024-09-16T15:10:31.004 INFO:tasks.rados.rados.0.smithi162.stdout:2203: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:10:31.004 INFO:tasks.rados.rados.0.smithi162.stdout:2203: finishing copy_from to smithi16234619-36 2024-09-16T15:10:31.004 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 838 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:10:31.005 INFO:tasks.rados.rados.0.smithi162.stdout:2203: done (1 left) 2024-09-16T15:10:31.005 INFO:tasks.rados.rados.0.smithi162.stdout:2205: done (0 left) 2024-09-16T15:10:31.005 INFO:tasks.rados.rados.0.smithi162.stdout:2206: snap_create 2024-09-16T15:10:32.005 INFO:tasks.rados.rados.0.smithi162.stdout:2206: done (0 left) 2024-09-16T15:10:32.005 INFO:tasks.rados.rados.0.smithi162.stdout:2207: delete oid 42 current snap is 208 2024-09-16T15:10:32.009 INFO:tasks.rados.rados.0.smithi162.stdout:2207: done (0 left) 2024-09-16T15:10:32.009 INFO:tasks.rados.rados.0.smithi162.stdout:2208: snap_remove snap 204 2024-09-16T15:10:33.027 INFO:tasks.rados.rados.0.smithi162.stdout:2208: done (0 left) 2024-09-16T15:10:33.027 INFO:tasks.rados.rados.0.smithi162.stdout:2209: read oid 18 snap 195 2024-09-16T15:10:33.027 INFO:tasks.rados.rados.0.smithi162.stdout:2209: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:10:33.027 INFO:tasks.rados.rados.0.smithi162.stdout:2210: copy_from oid 6 from oid 7 current snap is 208 2024-09-16T15:10:33.027 INFO:tasks.rados.rados.0.smithi162.stdout:2211: delete oid 13 current snap is 208 2024-09-16T15:10:33.030 INFO:tasks.rados.rados.0.smithi162.stdout:2211: done (2 left) 2024-09-16T15:10:33.030 INFO:tasks.rados.rados.0.smithi162.stdout:2212: snap_create 2024-09-16T15:10:33.032 INFO:tasks.rados.rados.0.smithi162.stdout:2210: finishing copy_from to smithi16234619-6 2024-09-16T15:10:33.032 INFO:tasks.rados.rados.0.smithi162.stdout:2210: got expected ENOENT (src dne) 2024-09-16T15:10:33.032 INFO:tasks.rados.rados.0.smithi162.stdout:2210: finishing copy_from racing read to smithi16234619-6 2024-09-16T15:10:33.037 INFO:tasks.rados.rados.0.smithi162.stdout:2209: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:10:33.995 INFO:tasks.rados.rados.0.smithi162.stdout:2209: done (2 left) 2024-09-16T15:10:33.995 INFO:tasks.rados.rados.0.smithi162.stdout:2210: done (1 left) 2024-09-16T15:10:33.995 INFO:tasks.rados.rados.0.smithi162.stdout:2212: done (0 left) 2024-09-16T15:10:33.995 INFO:tasks.rados.rados.0.smithi162.stdout:2213: copy_from oid 29 from oid 11 current snap is 209 2024-09-16T15:10:33.995 INFO:tasks.rados.rados.0.smithi162.stdout:2214: delete oid 17 current snap is 209 2024-09-16T15:10:34.007 INFO:tasks.rados.rados.0.smithi162.stdout:2214: done (1 left) 2024-09-16T15:10:34.007 INFO:tasks.rados.rados.0.smithi162.stdout:2215: read oid 24 snap -1 2024-09-16T15:10:34.007 INFO:tasks.rados.rados.0.smithi162.stdout:2215: expect (ObjNum 741 snap 194 seq_num 741) 2024-09-16T15:10:34.008 INFO:tasks.rados.rados.0.smithi162.stdout:2216: snap_create 2024-09-16T15:10:34.013 INFO:tasks.rados.rados.0.smithi162.stdout:2215: expect (ObjNum 741 snap 194 seq_num 741) 2024-09-16T15:10:34.165 INFO:tasks.rados.rados.0.smithi162.stdout:2213: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:10:34.166 INFO:tasks.rados.rados.0.smithi162.stdout:2213: finishing copy_from to smithi16234619-29 2024-09-16T15:10:34.166 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:10:35.013 INFO:tasks.rados.rados.0.smithi162.stdout:2213: done (2 left) 2024-09-16T15:10:35.013 INFO:tasks.rados.rados.0.smithi162.stdout:2215: done (1 left) 2024-09-16T15:10:35.013 INFO:tasks.rados.rados.0.smithi162.stdout:2216: done (0 left) 2024-09-16T15:10:35.014 INFO:tasks.rados.rados.0.smithi162.stdout:2217: write oid 6 current snap is 210 2024-09-16T15:10:35.014 INFO:tasks.rados.rados.0.smithi162.stdout:2217: seq_num 795 ranges {561565=723667,1905067=560602,3082338=279227} 2024-09-16T15:10:35.020 INFO:tasks.rados.rados.0.smithi162.stdout:2217: writing smithi16234619-6 from 561565 to 1285232 tid 1 2024-09-16T15:10:35.027 INFO:tasks.rados.rados.0.smithi162.stdout:2217: writing smithi16234619-6 from 1905067 to 2465669 tid 2 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2217: writing smithi16234619-6 from 3082338 to 3361565 tid 3 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2218: read oid 41 snap 208 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2218: expect deleted 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2219: rollback oid 1 current snap is 210 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 207 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2220: copy_from oid 33 from oid 4 current snap is 210 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:2221: rollback oid 18 current snap is 210 2024-09-16T15:10:35.033 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 209 2024-09-16T15:10:35.034 INFO:tasks.rados.rados.0.smithi162.stdout:2222: write (excl) oid 50 current snap is 210 2024-09-16T15:10:35.034 INFO:tasks.rados.rados.0.smithi162.stdout:2222: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2024-09-16T15:10:35.041 INFO:tasks.rados.rados.0.smithi162.stdout:2222: writing smithi16234619-50 from 578372 to 1377288 tid 1 2024-09-16T15:10:35.047 INFO:tasks.rados.rados.0.smithi162.stdout:2222: writing smithi16234619-50 from 1963549 to 2384162 tid 2 2024-09-16T15:10:35.053 INFO:tasks.rados.rados.0.smithi162.stdout:2222: writing smithi16234619-50 from 2826114 to 3244048 tid 3 2024-09-16T15:10:35.053 INFO:tasks.rados.rados.0.smithi162.stdout:2223: snap_create 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2220: finishing copy_from to smithi16234619-33 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2220: got expected ENOENT (src dne) 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2220: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2219: finishing rollback tid 0 to smithi16234619-1 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2221: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2221: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2219: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:2219: finishing rollback tid 2 to smithi16234619-1 2024-09-16T15:10:35.054 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 888 (ObjNum 759 snap 197 seq_num 759) dirty exists 2024-09-16T15:10:35.055 INFO:tasks.rados.rados.0.smithi162.stdout:2221: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:10:35.055 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 850 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:10:35.063 INFO:tasks.rados.rados.0.smithi162.stdout:2217: finishing write tid 1 to smithi16234619-6 2024-09-16T15:10:35.064 INFO:tasks.rados.rados.0.smithi162.stdout:2217: finishing write tid 2 to smithi16234619-6 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: finishing write tid 3 to smithi16234619-6 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: finishing write tid 4 to smithi16234619-6 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: finishing write tid 5 to smithi16234619-6 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: oid 6 updating version 0 to 909 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: oid 6 updating version 909 to 911 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: oid 6 version 911 is already newer than 910 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:2217: oid 6 updating version 911 to 912 2024-09-16T15:10:35.067 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 912 (ObjNum 795 snap 210 seq_num 795) dirty exists 2024-09-16T15:10:35.068 INFO:tasks.rados.rados.0.smithi162.stdout:2217: left oid 6 (ObjNum 795 snap 210 seq_num 795) 2024-09-16T15:10:35.095 INFO:tasks.rados.rados.0.smithi162.stdout:2222: finishing write tid 1 to smithi16234619-50 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: finishing write tid 2 to smithi16234619-50 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: finishing write tid 3 to smithi16234619-50 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: finishing write tid 4 to smithi16234619-50 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: finishing write tid 5 to smithi16234619-50 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: oid 50 updating version 0 to 881 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: oid 50 updating version 881 to 882 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: oid 50 version 882 is already newer than 880 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: oid 50 updating version 882 to 883 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 883 (ObjNum 796 snap 210 seq_num 796) dirty exists 2024-09-16T15:10:35.096 INFO:tasks.rados.rados.0.smithi162.stdout:2222: left oid 50 (ObjNum 796 snap 210 seq_num 796) 2024-09-16T15:10:36.049 INFO:tasks.rados.rados.0.smithi162.stdout:2217: done (6 left) 2024-09-16T15:10:36.049 INFO:tasks.rados.rados.0.smithi162.stdout:2218: done (5 left) 2024-09-16T15:10:36.049 INFO:tasks.rados.rados.0.smithi162.stdout:2219: done (4 left) 2024-09-16T15:10:36.049 INFO:tasks.rados.rados.0.smithi162.stdout:2220: done (3 left) 2024-09-16T15:10:36.050 INFO:tasks.rados.rados.0.smithi162.stdout:2221: done (2 left) 2024-09-16T15:10:36.050 INFO:tasks.rados.rados.0.smithi162.stdout:2222: done (1 left) 2024-09-16T15:10:36.050 INFO:tasks.rados.rados.0.smithi162.stdout:2223: done (0 left) 2024-09-16T15:10:36.050 INFO:tasks.rados.rados.0.smithi162.stdout:2224: write oid 30 current snap is 211 2024-09-16T15:10:36.050 INFO:tasks.rados.rados.0.smithi162.stdout:2224: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2024-09-16T15:10:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:2224: writing smithi16234619-30 from 595179 to 1069344 tid 1 2024-09-16T15:10:36.061 INFO:tasks.rados.rados.0.smithi162.stdout:2224: writing smithi16234619-30 from 1622031 to 2186302 tid 2 2024-09-16T15:10:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2224: writing smithi16234619-30 from 2737184 to 3366390 tid 3 2024-09-16T15:10:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2225: read oid 2 snap -1 2024-09-16T15:10:36.068 INFO:tasks.rados.rados.0.smithi162.stdout:2225: expect (ObjNum 737 snap 194 seq_num 737) 2024-09-16T15:10:36.068 INFO:tasks.rados.rados.0.smithi162.stdout:2226: snap_create 2024-09-16T15:10:36.074 INFO:tasks.rados.rados.0.smithi162.stdout:2225: expect (ObjNum 737 snap 194 seq_num 737) 2024-09-16T15:10:36.197 INFO:tasks.rados.rados.0.smithi162.stdout:2224: finishing write tid 1 to smithi16234619-30 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: finishing write tid 2 to smithi16234619-30 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: finishing write tid 3 to smithi16234619-30 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: finishing write tid 4 to smithi16234619-30 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: finishing write tid 5 to smithi16234619-30 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: oid 30 updating version 0 to 859 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: oid 30 updating version 859 to 862 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: oid 30 version 862 is already newer than 861 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: oid 30 version 862 is already newer than 860 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 862 (ObjNum 797 snap 211 seq_num 797) dirty exists 2024-09-16T15:10:36.198 INFO:tasks.rados.rados.0.smithi162.stdout:2224: left oid 30 (ObjNum 797 snap 211 seq_num 797) 2024-09-16T15:10:37.619 INFO:tasks.rados.rados.0.smithi162.stdout:2224: done (2 left) 2024-09-16T15:10:37.620 INFO:tasks.rados.rados.0.smithi162.stdout:2225: done (1 left) 2024-09-16T15:10:37.620 INFO:tasks.rados.rados.0.smithi162.stdout:2226: done (0 left) 2024-09-16T15:10:37.620 INFO:tasks.rados.rados.0.smithi162.stdout:2227: delete oid 4 current snap is 212 2024-09-16T15:10:37.622 INFO:tasks.rados.rados.0.smithi162.stdout:2227: done (0 left) 2024-09-16T15:10:37.622 INFO:tasks.rados.rados.0.smithi162.stdout:2228: read oid 49 snap -1 2024-09-16T15:10:37.622 INFO:tasks.rados.rados.0.smithi162.stdout:2228: expect deleted 2024-09-16T15:10:37.622 INFO:tasks.rados.rados.0.smithi162.stdout:2229: write (excl) oid 34 current snap is 212 2024-09-16T15:10:37.622 INFO:tasks.rados.rados.0.smithi162.stdout:2229: seq_num 799 ranges {628793=741016,2138995=567940,3192030=236763} 2024-09-16T15:10:37.629 INFO:tasks.rados.rados.0.smithi162.stdout:2229: writing smithi16234619-34 from 628793 to 1369809 tid 1 2024-09-16T15:10:37.636 INFO:tasks.rados.rados.0.smithi162.stdout:2229: writing smithi16234619-34 from 2138995 to 2706935 tid 2 2024-09-16T15:10:37.640 INFO:tasks.rados.rados.0.smithi162.stdout:2229: writing smithi16234619-34 from 3192030 to 3428793 tid 3 2024-09-16T15:10:37.640 INFO:tasks.rados.rados.0.smithi162.stdout:2230: snap_remove snap 211 2024-09-16T15:10:38.624 INFO:tasks.rados.rados.0.smithi162.stdout:2230: done (2 left) 2024-09-16T15:10:38.624 INFO:tasks.rados.rados.0.smithi162.stdout:2231: write (excl) oid 28 current snap is 212 2024-09-16T15:10:38.624 INFO:tasks.rados.rados.0.smithi162.stdout:2231: seq_num 800 ranges {645600=416265,1797477=711598,3219453=226147} 2024-09-16T15:10:38.629 INFO:tasks.rados.rados.0.smithi162.stdout:2231: writing smithi16234619-28 from 645600 to 1061865 tid 1 2024-09-16T15:10:38.637 INFO:tasks.rados.rados.0.smithi162.stdout:2231: writing smithi16234619-28 from 1797477 to 2509075 tid 2 2024-09-16T15:10:38.642 INFO:tasks.rados.rados.0.smithi162.stdout:2231: writing smithi16234619-28 from 3219453 to 3445600 tid 3 2024-09-16T15:10:38.642 INFO:tasks.rados.rados.0.smithi162.stdout:2232: write oid 24 current snap is 212 2024-09-16T15:10:38.643 INFO:tasks.rados.rados.0.smithi162.stdout:2232: seq_num 801 ranges {662407=491514,1855959=571609,2846876=615531} 2024-09-16T15:10:38.648 INFO:tasks.rados.rados.0.smithi162.stdout:2232: writing smithi16234619-24 from 662407 to 1153921 tid 1 2024-09-16T15:10:38.655 INFO:tasks.rados.rados.0.smithi162.stdout:2232: writing smithi16234619-24 from 1855959 to 2427568 tid 2 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2232: writing smithi16234619-24 from 2846876 to 3462407 tid 3 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2233: rollback oid 29 current snap is 212 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 208 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: finishing write tid 1 to smithi16234619-34 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: finishing write tid 2 to smithi16234619-34 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: finishing write tid 3 to smithi16234619-34 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: finishing write tid 4 to smithi16234619-34 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: finishing write tid 5 to smithi16234619-34 2024-09-16T15:10:38.662 INFO:tasks.rados.rados.0.smithi162.stdout:2229: oid 34 updating version 0 to 886 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2229: oid 34 updating version 886 to 888 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2229: oid 34 version 888 is already newer than 887 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2229: oid 34 version 888 is already newer than 885 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 888 (ObjNum 799 snap 212 seq_num 799) dirty exists 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2229: left oid 34 (ObjNum 799 snap 212 seq_num 799) 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2228: done (4 left) 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2229: done (3 left) 2024-09-16T15:10:38.663 INFO:tasks.rados.rados.0.smithi162.stdout:2234: snap_create 2024-09-16T15:10:38.666 INFO:tasks.rados.rados.0.smithi162.stdout:2233: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:10:38.666 INFO:tasks.rados.rados.0.smithi162.stdout:2233: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:10:38.667 INFO:tasks.rados.rados.0.smithi162.stdout:2231: finishing write tid 1 to smithi16234619-28 2024-09-16T15:10:38.667 INFO:tasks.rados.rados.0.smithi162.stdout:2231: finishing write tid 2 to smithi16234619-28 2024-09-16T15:10:38.667 INFO:tasks.rados.rados.0.smithi162.stdout:2231: finishing write tid 3 to smithi16234619-28 2024-09-16T15:10:38.667 INFO:tasks.rados.rados.0.smithi162.stdout:2231: finishing write tid 4 to smithi16234619-28 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: finishing write tid 5 to smithi16234619-28 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: oid 28 updating version 0 to 866 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: oid 28 version 866 is already newer than 865 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: oid 28 version 866 is already newer than 863 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: oid 28 version 866 is already newer than 864 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 866 (ObjNum 800 snap 212 seq_num 800) dirty exists 2024-09-16T15:10:38.668 INFO:tasks.rados.rados.0.smithi162.stdout:2231: left oid 28 (ObjNum 800 snap 212 seq_num 800) 2024-09-16T15:10:38.669 INFO:tasks.rados.rados.0.smithi162.stdout:2233: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:10:38.669 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 993 (ObjNum 768 snap 202 seq_num 768) dirty exists 2024-09-16T15:10:38.690 INFO:tasks.rados.rados.0.smithi162.stdout:2232: finishing write tid 1 to smithi16234619-24 2024-09-16T15:10:38.690 INFO:tasks.rados.rados.0.smithi162.stdout:2232: finishing write tid 2 to smithi16234619-24 2024-09-16T15:10:38.691 INFO:tasks.rados.rados.0.smithi162.stdout:2232: finishing write tid 3 to smithi16234619-24 2024-09-16T15:10:38.692 INFO:tasks.rados.rados.0.smithi162.stdout:2232: finishing write tid 4 to smithi16234619-24 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: finishing write tid 5 to smithi16234619-24 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: oid 24 updating version 0 to 1038 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: oid 24 updating version 1038 to 1040 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: oid 24 version 1040 is already newer than 1039 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: oid 24 version 1040 is already newer than 1037 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1040 (ObjNum 801 snap 212 seq_num 801) dirty exists 2024-09-16T15:10:38.693 INFO:tasks.rados.rados.0.smithi162.stdout:2232: left oid 24 (ObjNum 801 snap 212 seq_num 801) 2024-09-16T15:10:39.001 INFO:tasks.rados.rados.0.smithi162.stdout:2231: done (3 left) 2024-09-16T15:10:39.001 INFO:tasks.rados.rados.0.smithi162.stdout:2232: done (2 left) 2024-09-16T15:10:39.002 INFO:tasks.rados.rados.0.smithi162.stdout:2233: done (1 left) 2024-09-16T15:10:39.002 INFO:tasks.rados.rados.0.smithi162.stdout:2234: done (0 left) 2024-09-16T15:10:39.002 INFO:tasks.rados.rados.0.smithi162.stdout:2235: rollback oid 1 current snap is 213 2024-09-16T15:10:39.002 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 195 2024-09-16T15:10:39.002 INFO:tasks.rados.rados.0.smithi162.stdout:2236: snap_remove snap 209 2024-09-16T15:10:39.999 INFO:tasks.rados.rados.0.smithi162.stdout:2236: done (1 left) 2024-09-16T15:10:39.999 INFO:tasks.rados.rados.0.smithi162.stdout:2237: snap_remove snap 210 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:2237: done (1 left) 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:2238: rollback oid 38 current snap is 213 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 208 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:2235: finishing rollback tid 0 to smithi16234619-1 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:2235: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 890 (ObjNum 129 snap 0 seq_num 866865160) dirty dne 2024-09-16T15:10:41.199 INFO:tasks.rados.rados.0.smithi162.stdout:2235: done (1 left) 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:2239: read oid 40 snap -1 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:2239: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:2240: rollback oid 26 current snap is 213 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 206 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:2241: snap_create 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:2238: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:10:41.200 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:10:41.207 INFO:tasks.rados.rados.0.smithi162.stdout:2240: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:10:41.207 INFO:tasks.rados.rados.0.smithi162.stdout:2240: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:10:41.210 INFO:tasks.rados.rados.0.smithi162.stdout:2240: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:10:41.210 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1048 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:10:41.210 INFO:tasks.rados.rados.0.smithi162.stdout:2239: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:2238: done (3 left) 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:2239: done (2 left) 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:2240: done (1 left) 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:2241: done (0 left) 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:2242: rollback oid 16 current snap is 214 2024-09-16T15:10:42.190 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 208 2024-09-16T15:10:42.191 INFO:tasks.rados.rados.0.smithi162.stdout:2243: snap_remove snap 207 2024-09-16T15:10:43.204 INFO:tasks.rados.rados.0.smithi162.stdout:2243: done (1 left) 2024-09-16T15:10:43.204 INFO:tasks.rados.rados.0.smithi162.stdout:2244: snap_create 2024-09-16T15:10:43.204 INFO:tasks.rados.rados.0.smithi162.stdout:2242: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:10:43.205 INFO:tasks.rados.rados.0.smithi162.stdout:2242: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:10:43.205 INFO:tasks.rados.rados.0.smithi162.stdout:2242: finishing rollback tid 2 to smithi16234619-16 2024-09-16T15:10:43.205 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 915 (ObjNum 762 snap 198 seq_num 762) dirty exists 2024-09-16T15:10:43.999 INFO:tasks.rados.rados.0.smithi162.stdout:2242: done (1 left) 2024-09-16T15:10:43.999 INFO:tasks.rados.rados.0.smithi162.stdout:2244: done (0 left) 2024-09-16T15:10:43.999 INFO:tasks.rados.rados.0.smithi162.stdout:2245: snap_create 2024-09-16T15:10:45.010 INFO:tasks.rados.rados.0.smithi162.stdout:2245: done (0 left) 2024-09-16T15:10:45.011 INFO:tasks.rados.rados.0.smithi162.stdout:2246: snap_remove snap 208 2024-09-16T15:10:46.009 INFO:tasks.rados.rados.0.smithi162.stdout:2246: done (0 left) 2024-09-16T15:10:46.009 INFO:tasks.rados.rados.0.smithi162.stdout:2247: read oid 26 snap 195 2024-09-16T15:10:46.009 INFO:tasks.rados.rados.0.smithi162.stdout:2247: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:10:46.009 INFO:tasks.rados.rados.0.smithi162.stdout:2248: rollback oid 50 current snap is 216 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 214 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:2249: read oid 15 snap -1 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:2249: expect (ObjNum 777 snap 204 seq_num 777) 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:2250: rollback oid 35 current snap is 216 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 214 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:2251: write oid 12 current snap is 216 2024-09-16T15:10:46.010 INFO:tasks.rados.rados.0.smithi162.stdout:2251: seq_num 802 ranges {679214=566763,1798088=715267,3157946=321268} 2024-09-16T15:10:46.016 INFO:tasks.rados.rados.0.smithi162.stdout:2251: writing smithi16234619-12 from 679214 to 1245977 tid 1 2024-09-16T15:10:46.024 INFO:tasks.rados.rados.0.smithi162.stdout:2251: writing smithi16234619-12 from 1798088 to 2513355 tid 2 2024-09-16T15:10:46.029 INFO:tasks.rados.rados.0.smithi162.stdout:2251: writing smithi16234619-12 from 3157946 to 3479214 tid 3 2024-09-16T15:10:46.029 INFO:tasks.rados.rados.0.smithi162.stdout:2252: rollback oid 34 current snap is 216 2024-09-16T15:10:46.029 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 215 2024-09-16T15:10:46.030 INFO:tasks.rados.rados.0.smithi162.stdout:2250: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:10:46.030 INFO:tasks.rados.rados.0.smithi162.stdout:2248: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:10:46.030 INFO:tasks.rados.rados.0.smithi162.stdout:2247: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:10:46.283 INFO:tasks.rados.rados.0.smithi162.stdout:2250: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:10:46.283 INFO:tasks.rados.rados.0.smithi162.stdout:2250: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:10:46.283 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 918 (ObjNum 660 snap 171 seq_num 660) dirty exists 2024-09-16T15:10:46.283 INFO:tasks.rados.rados.0.smithi162.stdout:2248: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:10:46.284 INFO:tasks.rados.rados.0.smithi162.stdout:2248: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:10:46.284 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 908 (ObjNum 796 snap 210 seq_num 796) dirty exists 2024-09-16T15:10:46.284 INFO:tasks.rados.rados.0.smithi162.stdout:2249: expect (ObjNum 777 snap 204 seq_num 777) 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2252: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2252: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2252: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 912 (ObjNum 799 snap 212 seq_num 799) dirty exists 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2251: finishing write tid 1 to smithi16234619-12 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2251: finishing write tid 2 to smithi16234619-12 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2251: finishing write tid 3 to smithi16234619-12 2024-09-16T15:10:46.461 INFO:tasks.rados.rados.0.smithi162.stdout:2251: finishing write tid 4 to smithi16234619-12 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: finishing write tid 5 to smithi16234619-12 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: oid 12 updating version 0 to 889 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: oid 12 updating version 889 to 892 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: oid 12 version 892 is already newer than 891 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: oid 12 version 892 is already newer than 890 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 892 (ObjNum 802 snap 216 seq_num 802) dirty exists 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: left oid 12 (ObjNum 802 snap 216 seq_num 802) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2247: done (5 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2248: done (4 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2249: done (3 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2250: done (2 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2251: done (1 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2252: done (0 left) 2024-09-16T15:10:46.462 INFO:tasks.rados.rados.0.smithi162.stdout:2253: delete oid 46 current snap is 216 2024-09-16T15:10:46.463 INFO:tasks.rados.rados.0.smithi162.stdout:2253: done (0 left) 2024-09-16T15:10:46.464 INFO:tasks.rados.rados.0.smithi162.stdout:2254: rollback oid 50 current snap is 216 2024-09-16T15:10:46.464 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 214 2024-09-16T15:10:46.464 INFO:tasks.rados.rados.0.smithi162.stdout:2255: write (excl) oid 28 current snap is 216 2024-09-16T15:10:46.464 INFO:tasks.rados.rados.0.smithi162.stdout:2255: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2024-09-16T15:10:46.471 INFO:tasks.rados.rados.0.smithi162.stdout:2255: writing smithi16234619-28 from 712828 to 1430089 tid 1 2024-09-16T15:10:46.477 INFO:tasks.rados.rados.0.smithi162.stdout:2255: writing smithi16234619-28 from 1915052 to 2633988 tid 2 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2255: writing smithi16234619-28 from 3212792 to 3512828 tid 3 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2256: read oid 16 snap -1 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2256: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2257: snap_create 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2254: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2254: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:10:46.482 INFO:tasks.rados.rados.0.smithi162.stdout:2254: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:10:46.483 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 915 (ObjNum 796 snap 210 seq_num 796) dirty exists 2024-09-16T15:10:46.484 INFO:tasks.rados.rados.0.smithi162.stdout:2256: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: finishing write tid 1 to smithi16234619-28 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: finishing write tid 2 to smithi16234619-28 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: finishing write tid 3 to smithi16234619-28 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: finishing write tid 4 to smithi16234619-28 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: finishing write tid 5 to smithi16234619-28 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: oid 28 updating version 0 to 867 2024-09-16T15:10:46.605 INFO:tasks.rados.rados.0.smithi162.stdout:2255: oid 28 updating version 867 to 868 2024-09-16T15:10:46.606 INFO:tasks.rados.rados.0.smithi162.stdout:2255: oid 28 updating version 868 to 869 2024-09-16T15:10:46.606 INFO:tasks.rados.rados.0.smithi162.stdout:2255: oid 28 updating version 869 to 870 2024-09-16T15:10:46.606 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 870 (ObjNum 804 snap 216 seq_num 804) dirty exists 2024-09-16T15:10:46.606 INFO:tasks.rados.rados.0.smithi162.stdout:2255: left oid 28 (ObjNum 804 snap 216 seq_num 804) 2024-09-16T15:10:47.386 INFO:tasks.rados.rados.0.smithi162.stdout:2254: done (3 left) 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2255: done (2 left) 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2256: done (1 left) 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2257: done (0 left) 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2258: read oid 41 snap -1 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2258: expect deleted 2024-09-16T15:10:47.387 INFO:tasks.rados.rados.0.smithi162.stdout:2259: snap_remove snap 214 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2259: done (1 left) 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2260: read oid 32 snap -1 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2260: expect (ObjNum 767 snap 201 seq_num 767) 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2261: read oid 40 snap -1 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2261: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:10:48.400 INFO:tasks.rados.rados.0.smithi162.stdout:2262: snap_create 2024-09-16T15:10:48.411 INFO:tasks.rados.rados.0.smithi162.stdout:2260: expect (ObjNum 767 snap 201 seq_num 767) 2024-09-16T15:10:48.568 INFO:tasks.rados.rados.0.smithi162.stdout:2261: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:10:48.999 INFO:tasks.rados.rados.0.smithi162.stdout:2258: done (3 left) 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2260: done (2 left) 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2261: done (1 left) 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2262: done (0 left) 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2263: read oid 46 snap 206 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2263: expect (ObjNum 721 snap 190 seq_num 721) 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2264: rollback oid 42 current snap is 218 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 215 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2265: write oid 37 current snap is 218 2024-09-16T15:10:49.000 INFO:tasks.rados.rados.0.smithi162.stdout:2265: seq_num 805 ranges {729635=792510,1973534=578947,3240215=289420} 2024-09-16T15:10:49.007 INFO:tasks.rados.rados.0.smithi162.stdout:2265: writing smithi16234619-37 from 729635 to 1522145 tid 1 2024-09-16T15:10:49.013 INFO:tasks.rados.rados.0.smithi162.stdout:2265: writing smithi16234619-37 from 1973534 to 2552481 tid 2 2024-09-16T15:10:49.019 INFO:tasks.rados.rados.0.smithi162.stdout:2265: writing smithi16234619-37 from 3240215 to 3529635 tid 3 2024-09-16T15:10:49.019 INFO:tasks.rados.rados.0.smithi162.stdout:2266: write (excl) oid 8 current snap is 218 2024-09-16T15:10:49.019 INFO:tasks.rados.rados.0.smithi162.stdout:2264: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:10:49.019 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 981 (ObjNum 33 snap 0 seq_num 1840586953) dirty dne 2024-09-16T15:10:49.019 INFO:tasks.rados.rados.0.smithi162.stdout:2266: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2024-09-16T15:10:49.026 INFO:tasks.rados.rados.0.smithi162.stdout:2266: writing smithi16234619-8 from 746442 to 1330554 tid 1 2024-09-16T15:10:49.032 INFO:tasks.rados.rados.0.smithi162.stdout:2266: writing smithi16234619-8 from 1748369 to 2470974 tid 2 2024-09-16T15:10:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:2266: writing smithi16234619-8 from 2983991 to 3546442 tid 3 2024-09-16T15:10:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:2264: done (3 left) 2024-09-16T15:10:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:2267: snap_remove snap 215 2024-09-16T15:10:50.001 INFO:tasks.rados.rados.0.smithi162.stdout:2267: done (3 left) 2024-09-16T15:10:50.001 INFO:tasks.rados.rados.0.smithi162.stdout:2268: write (excl) oid 14 current snap is 218 2024-09-16T15:10:50.002 INFO:tasks.rados.rados.0.smithi162.stdout:2268: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2024-09-16T15:10:50.009 INFO:tasks.rados.rados.0.smithi162.stdout:2268: writing smithi16234619-14 from 763249 to 1422610 tid 1 2024-09-16T15:10:50.016 INFO:tasks.rados.rados.0.smithi162.stdout:2268: writing smithi16234619-14 from 2090498 to 2673114 tid 2 2024-09-16T15:10:50.021 INFO:tasks.rados.rados.0.smithi162.stdout:2268: writing smithi16234619-14 from 3295061 to 3563249 tid 3 2024-09-16T15:10:50.021 INFO:tasks.rados.rados.0.smithi162.stdout:2269: write oid 38 current snap is 218 2024-09-16T15:10:50.021 INFO:tasks.rados.rados.0.smithi162.stdout:2263: expect (ObjNum 721 snap 190 seq_num 721) 2024-09-16T15:10:50.152 INFO:tasks.rados.rados.0.smithi162.stdout:2265: finishing write tid 1 to smithi16234619-37 2024-09-16T15:10:50.152 INFO:tasks.rados.rados.0.smithi162.stdout:2265: finishing write tid 2 to smithi16234619-37 2024-09-16T15:10:50.152 INFO:tasks.rados.rados.0.smithi162.stdout:2265: finishing write tid 3 to smithi16234619-37 2024-09-16T15:10:50.152 INFO:tasks.rados.rados.0.smithi162.stdout:2265: finishing write tid 4 to smithi16234619-37 2024-09-16T15:10:50.152 INFO:tasks.rados.rados.0.smithi162.stdout:2269: seq_num 808 ranges {780056=734610,2148980=726274,3322484=257572} 2024-09-16T15:10:50.159 INFO:tasks.rados.rados.0.smithi162.stdout:2269: writing smithi16234619-38 from 780056 to 1514666 tid 1 2024-09-16T15:10:50.168 INFO:tasks.rados.rados.0.smithi162.stdout:2269: writing smithi16234619-38 from 2148980 to 2875254 tid 2 2024-09-16T15:10:50.173 INFO:tasks.rados.rados.0.smithi162.stdout:2269: writing smithi16234619-38 from 3322484 to 3580056 tid 3 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2263: done (4 left) 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2270: read oid 46 snap 216 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2270: expect deleted 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2271: snap_remove snap 212 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: finishing write tid 5 to smithi16234619-37 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: oid 37 updating version 0 to 1009 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: oid 37 updating version 1009 to 1011 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: oid 37 version 1011 is already newer than 1010 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: oid 37 version 1011 is already newer than 1008 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1011 (ObjNum 805 snap 218 seq_num 805) dirty exists 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2265: left oid 37 (ObjNum 805 snap 218 seq_num 805) 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2266: finishing write tid 1 to smithi16234619-8 2024-09-16T15:10:50.174 INFO:tasks.rados.rados.0.smithi162.stdout:2266: finishing write tid 2 to smithi16234619-8 2024-09-16T15:10:51.471 INFO:tasks.rados.rados.0.smithi162.stdout:2265: done (5 left) 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2271: done (4 left) 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2272: copy_from oid 47 from oid 4 current snap is 218 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: finishing write tid 3 to smithi16234619-8 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: finishing write tid 4 to smithi16234619-8 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: finishing write tid 5 to smithi16234619-8 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: oid 8 updating version 0 to 851 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: oid 8 updating version 851 to 852 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: oid 8 updating version 852 to 854 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: oid 8 version 854 is already newer than 853 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 854 (ObjNum 806 snap 218 seq_num 806) dirty exists 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2266: left oid 8 (ObjNum 806 snap 218 seq_num 806) 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2268: finishing write tid 1 to smithi16234619-14 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2268: finishing write tid 2 to smithi16234619-14 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2268: finishing write tid 3 to smithi16234619-14 2024-09-16T15:10:51.472 INFO:tasks.rados.rados.0.smithi162.stdout:2268: finishing write tid 4 to smithi16234619-14 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: finishing write tid 5 to smithi16234619-14 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: oid 14 updating version 0 to 925 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: oid 14 version 925 is already newer than 924 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: oid 14 updating version 925 to 927 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: oid 14 version 927 is already newer than 926 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 927 (ObjNum 807 snap 218 seq_num 807) dirty exists 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2268: left oid 14 (ObjNum 807 snap 218 seq_num 807) 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: finishing write tid 1 to smithi16234619-38 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: finishing write tid 2 to smithi16234619-38 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: finishing write tid 3 to smithi16234619-38 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: finishing write tid 4 to smithi16234619-38 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: finishing write tid 5 to smithi16234619-38 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: oid 38 updating version 0 to 922 2024-09-16T15:10:51.473 INFO:tasks.rados.rados.0.smithi162.stdout:2269: oid 38 version 922 is already newer than 919 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2269: oid 38 version 922 is already newer than 920 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2269: oid 38 version 922 is already newer than 921 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 922 (ObjNum 808 snap 218 seq_num 808) dirty exists 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2269: left oid 38 (ObjNum 808 snap 218 seq_num 808) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2266: done (4 left) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2268: done (3 left) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2269: done (2 left) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2270: done (1 left) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2273: read oid 28 snap -1 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2273: expect (ObjNum 804 snap 216 seq_num 804) 2024-09-16T15:10:51.474 INFO:tasks.rados.rados.0.smithi162.stdout:2274: snap_remove snap 195 2024-09-16T15:10:52.494 INFO:tasks.rados.rados.0.smithi162.stdout:2274: done (2 left) 2024-09-16T15:10:52.494 INFO:tasks.rados.rados.0.smithi162.stdout:2275: snap_remove snap 217 2024-09-16T15:10:53.500 INFO:tasks.rados.rados.0.smithi162.stdout:2275: done (2 left) 2024-09-16T15:10:53.500 INFO:tasks.rados.rados.0.smithi162.stdout:2276: write (excl) oid 43 current snap is 218 2024-09-16T15:10:53.500 INFO:tasks.rados.rados.0.smithi162.stdout:2272: finishing copy_from to smithi16234619-47 2024-09-16T15:10:53.500 INFO:tasks.rados.rados.0.smithi162.stdout:2272: got expected ENOENT (src dne) 2024-09-16T15:10:53.500 INFO:tasks.rados.rados.0.smithi162.stdout:2276: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2024-09-16T15:10:53.506 INFO:tasks.rados.rados.0.smithi162.stdout:2276: writing smithi16234619-43 from 796863 to 1206722 tid 1 2024-09-16T15:10:53.513 INFO:tasks.rados.rados.0.smithi162.stdout:2276: writing smithi16234619-43 from 1807462 to 2393747 tid 2 2024-09-16T15:10:53.520 INFO:tasks.rados.rados.0.smithi162.stdout:2276: writing smithi16234619-43 from 3066260 to 3596863 tid 3 2024-09-16T15:10:53.520 INFO:tasks.rados.rados.0.smithi162.stdout:2277: write (excl) oid 29 current snap is 218 2024-09-16T15:10:53.520 INFO:tasks.rados.rados.0.smithi162.stdout:2272: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:10:53.520 INFO:tasks.rados.rados.0.smithi162.stdout:2273: expect (ObjNum 804 snap 216 seq_num 804) 2024-09-16T15:10:53.727 INFO:tasks.rados.rados.0.smithi162.stdout:2276: finishing write tid 1 to smithi16234619-43 2024-09-16T15:10:53.727 INFO:tasks.rados.rados.0.smithi162.stdout:2276: finishing write tid 2 to smithi16234619-43 2024-09-16T15:10:53.727 INFO:tasks.rados.rados.0.smithi162.stdout:2277: seq_num 810 ranges {413670=485108,1465944=729943,2977330=524801} 2024-09-16T15:10:53.731 INFO:tasks.rados.rados.0.smithi162.stdout:2277: writing smithi16234619-29 from 413670 to 898778 tid 1 2024-09-16T15:10:53.739 INFO:tasks.rados.rados.0.smithi162.stdout:2277: writing smithi16234619-29 from 1465944 to 2195887 tid 2 2024-09-16T15:10:53.746 INFO:tasks.rados.rados.0.smithi162.stdout:2277: writing smithi16234619-29 from 2977330 to 3502131 tid 3 2024-09-16T15:10:53.746 INFO:tasks.rados.rados.0.smithi162.stdout:2272: done (3 left) 2024-09-16T15:10:53.746 INFO:tasks.rados.rados.0.smithi162.stdout:2273: done (2 left) 2024-09-16T15:10:53.746 INFO:tasks.rados.rados.0.smithi162.stdout:2278: read oid 6 snap -1 2024-09-16T15:10:53.747 INFO:tasks.rados.rados.0.smithi162.stdout:2278: expect (ObjNum 795 snap 210 seq_num 795) 2024-09-16T15:10:53.747 INFO:tasks.rados.rados.0.smithi162.stdout:2279: write (excl) oid 50 current snap is 218 2024-09-16T15:10:53.747 INFO:tasks.rados.rados.0.smithi162.stdout:2276: finishing write tid 3 to smithi16234619-43 2024-09-16T15:10:53.747 INFO:tasks.rados.rados.0.smithi162.stdout:2276: finishing write tid 4 to smithi16234619-43 2024-09-16T15:10:53.747 INFO:tasks.rados.rados.0.smithi162.stdout:2279: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2024-09-16T15:10:53.752 INFO:tasks.rados.rados.0.smithi162.stdout:2279: writing smithi16234619-50 from 430477 to 990834 tid 1 2024-09-16T15:10:53.757 INFO:tasks.rados.rados.0.smithi162.stdout:2279: writing smithi16234619-50 from 1408073 to 1881674 tid 2 2024-09-16T15:10:53.764 INFO:tasks.rados.rados.0.smithi162.stdout:2279: writing smithi16234619-50 from 2488400 to 3224473 tid 3 2024-09-16T15:10:53.764 INFO:tasks.rados.rados.0.smithi162.stdout:2280: copy_from oid 11 from oid 42 current snap is 218 2024-09-16T15:10:53.764 INFO:tasks.rados.rados.0.smithi162.stdout:2276: finishing write tid 5 to smithi16234619-43 2024-09-16T15:10:53.764 INFO:tasks.rados.rados.0.smithi162.stdout:2276: oid 43 updating version 0 to 890 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2276: oid 43 updating version 890 to 892 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2276: oid 43 updating version 892 to 893 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2276: oid 43 version 893 is already newer than 891 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 893 (ObjNum 809 snap 218 seq_num 809) dirty exists 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2276: left oid 43 (ObjNum 809 snap 218 seq_num 809) 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2276: done (4 left) 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2281: snap_create 2024-09-16T15:10:53.765 INFO:tasks.rados.rados.0.smithi162.stdout:2278: expect (ObjNum 795 snap 210 seq_num 795) 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2280: finishing copy_from to smithi16234619-11 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2280: got expected ENOENT (src dne) 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2280: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: finishing write tid 1 to smithi16234619-29 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: finishing write tid 2 to smithi16234619-29 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: finishing write tid 3 to smithi16234619-29 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: finishing write tid 4 to smithi16234619-29 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: finishing write tid 5 to smithi16234619-29 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: oid 29 updating version 0 to 1030 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: oid 29 updating version 1030 to 1031 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: oid 29 updating version 1031 to 1032 2024-09-16T15:10:53.872 INFO:tasks.rados.rados.0.smithi162.stdout:2277: oid 29 version 1032 is already newer than 1029 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1032 (ObjNum 810 snap 218 seq_num 810) dirty exists 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2277: left oid 29 (ObjNum 810 snap 218 seq_num 810) 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: finishing write tid 1 to smithi16234619-50 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: finishing write tid 2 to smithi16234619-50 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: finishing write tid 3 to smithi16234619-50 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: finishing write tid 4 to smithi16234619-50 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: finishing write tid 5 to smithi16234619-50 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: oid 50 updating version 0 to 946 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: oid 50 version 946 is already newer than 943 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: oid 50 version 946 is already newer than 944 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: oid 50 version 946 is already newer than 945 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 946 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:10:53.873 INFO:tasks.rados.rados.0.smithi162.stdout:2279: left oid 50 (ObjNum 811 snap 218 seq_num 811) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2277: done (4 left) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2278: done (3 left) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2279: done (2 left) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2280: done (1 left) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2281: done (0 left) 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2282: read oid 20 snap -1 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2282: expect deleted 2024-09-16T15:10:54.003 INFO:tasks.rados.rados.0.smithi162.stdout:2283: snap_remove snap 218 2024-09-16T15:10:55.015 INFO:tasks.rados.rados.0.smithi162.stdout:2283: done (1 left) 2024-09-16T15:10:55.015 INFO:tasks.rados.rados.0.smithi162.stdout:2284: copy_from oid 2 from oid 16 current snap is 219 2024-09-16T15:10:55.015 INFO:tasks.rados.rados.0.smithi162.stdout:2282: done (1 left) 2024-09-16T15:10:55.016 INFO:tasks.rados.rados.0.smithi162.stdout:2285: write oid 46 current snap is 219 2024-09-16T15:10:55.016 INFO:tasks.rados.rados.0.smithi162.stdout:2285: seq_num 812 ranges {447284=635606,1866555=733612,3315823=331461} 2024-09-16T15:10:55.021 INFO:tasks.rados.rados.0.smithi162.stdout:2285: writing smithi16234619-46 from 447284 to 1082890 tid 1 2024-09-16T15:10:55.030 INFO:tasks.rados.rados.0.smithi162.stdout:2285: writing smithi16234619-46 from 1866555 to 2600167 tid 2 2024-09-16T15:10:55.036 INFO:tasks.rados.rados.0.smithi162.stdout:2285: writing smithi16234619-46 from 3315823 to 3647284 tid 3 2024-09-16T15:10:55.036 INFO:tasks.rados.rados.0.smithi162.stdout:2286: read oid 42 snap -1 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2286: expect deleted 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2287: rollback oid 37 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 213 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2288: rollback oid 39 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 216 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2289: copy_from oid 50 from oid 4 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2290: copy_from oid 33 from oid 11 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2291: rollback oid 7 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 206 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2292: copy_from oid 12 from oid 21 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2293: write (excl) oid 14 current snap is 219 2024-09-16T15:10:55.037 INFO:tasks.rados.rados.0.smithi162.stdout:2293: seq_num 813 ranges {464091=710855,1925037=477270,2943246=474970} 2024-09-16T15:10:55.043 INFO:tasks.rados.rados.0.smithi162.stdout:2293: writing smithi16234619-14 from 464091 to 1174946 tid 1 2024-09-16T15:10:55.050 INFO:tasks.rados.rados.0.smithi162.stdout:2293: writing smithi16234619-14 from 1925037 to 2402307 tid 2 2024-09-16T15:10:55.057 INFO:tasks.rados.rados.0.smithi162.stdout:2293: writing smithi16234619-14 from 2943246 to 3418216 tid 3 2024-09-16T15:10:55.057 INFO:tasks.rados.rados.0.smithi162.stdout:2294: write (excl) oid 22 current snap is 219 2024-09-16T15:10:55.057 INFO:tasks.rados.rados.0.smithi162.stdout:2294: seq_num 814 ranges {480898=502457,1699872=737281,3087022=593876} 2024-09-16T15:10:55.063 INFO:tasks.rados.rados.0.smithi162.stdout:2294: writing smithi16234619-22 from 480898 to 983355 tid 1 2024-09-16T15:10:55.073 INFO:tasks.rados.rados.0.smithi162.stdout:2294: writing smithi16234619-22 from 1699872 to 2437153 tid 2 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2294: writing smithi16234619-22 from 3087022 to 3680898 tid 3 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2295: rollback oid 45 current snap is 219 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 206 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2289: finishing copy_from to smithi16234619-50 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2289: got expected ENOENT (src dne) 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2289: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2291: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2287: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2287: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:2287: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:10:55.082 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1035 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2288: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2288: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2288: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1039 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2284: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2292: finishing copy_from to smithi16234619-12 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2292: got expected ENOENT (src dne) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2292: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2284: finishing copy_from to smithi16234619-2 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 915 (ObjNum 762 snap 198 seq_num 762) dirty exists 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2284: done (11 left) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2286: done (10 left) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2287: done (9 left) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2288: done (8 left) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2289: done (7 left) 2024-09-16T15:10:55.083 INFO:tasks.rados.rados.0.smithi162.stdout:2291: done (6 left) 2024-09-16T15:10:55.084 INFO:tasks.rados.rados.0.smithi162.stdout:2292: done (5 left) 2024-09-16T15:10:55.084 INFO:tasks.rados.rados.0.smithi162.stdout:2296: snap_create 2024-09-16T15:10:55.084 INFO:tasks.rados.rados.0.smithi162.stdout:2285: finishing write tid 1 to smithi16234619-46 2024-09-16T15:10:55.086 INFO:tasks.rados.rados.0.smithi162.stdout:2285: finishing write tid 2 to smithi16234619-46 2024-09-16T15:10:55.088 INFO:tasks.rados.rados.0.smithi162.stdout:2295: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:10:55.088 INFO:tasks.rados.rados.0.smithi162.stdout:2295: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:10:55.088 INFO:tasks.rados.rados.0.smithi162.stdout:2285: finishing write tid 3 to smithi16234619-46 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: finishing write tid 4 to smithi16234619-46 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2295: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1080 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: finishing write tid 5 to smithi16234619-46 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: oid 46 updating version 0 to 591 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: oid 46 version 591 is already newer than 588 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: oid 46 version 591 is already newer than 589 2024-09-16T15:10:55.089 INFO:tasks.rados.rados.0.smithi162.stdout:2285: oid 46 version 591 is already newer than 590 2024-09-16T15:10:55.090 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 591 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:10:55.090 INFO:tasks.rados.rados.0.smithi162.stdout:2285: left oid 46 (ObjNum 812 snap 219 seq_num 812) 2024-09-16T15:10:55.091 INFO:tasks.rados.rados.0.smithi162.stdout:2290: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:10:55.094 INFO:tasks.rados.rados.0.smithi162.stdout:2293: finishing write tid 1 to smithi16234619-14 2024-09-16T15:10:55.102 INFO:tasks.rados.rados.0.smithi162.stdout:2293: finishing write tid 2 to smithi16234619-14 2024-09-16T15:10:55.102 INFO:tasks.rados.rados.0.smithi162.stdout:2293: finishing write tid 3 to smithi16234619-14 2024-09-16T15:10:55.102 INFO:tasks.rados.rados.0.smithi162.stdout:2293: finishing write tid 4 to smithi16234619-14 2024-09-16T15:10:55.102 INFO:tasks.rados.rados.0.smithi162.stdout:2293: finishing write tid 5 to smithi16234619-14 2024-09-16T15:10:55.102 INFO:tasks.rados.rados.0.smithi162.stdout:2293: oid 14 updating version 0 to 948 2024-09-16T15:10:55.103 INFO:tasks.rados.rados.0.smithi162.stdout:2293: oid 14 updating version 948 to 950 2024-09-16T15:10:55.103 INFO:tasks.rados.rados.0.smithi162.stdout:2293: oid 14 version 950 is already newer than 947 2024-09-16T15:10:55.103 INFO:tasks.rados.rados.0.smithi162.stdout:2293: oid 14 version 950 is already newer than 949 2024-09-16T15:10:55.103 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 950 (ObjNum 813 snap 219 seq_num 813) dirty exists 2024-09-16T15:10:55.103 INFO:tasks.rados.rados.0.smithi162.stdout:2293: left oid 14 (ObjNum 813 snap 219 seq_num 813) 2024-09-16T15:10:55.105 INFO:tasks.rados.rados.0.smithi162.stdout:2290: finishing copy_from to smithi16234619-33 2024-09-16T15:10:55.105 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:10:55.115 INFO:tasks.rados.rados.0.smithi162.stdout:2294: finishing write tid 1 to smithi16234619-22 2024-09-16T15:10:55.116 INFO:tasks.rados.rados.0.smithi162.stdout:2294: finishing write tid 2 to smithi16234619-22 2024-09-16T15:10:55.116 INFO:tasks.rados.rados.0.smithi162.stdout:2294: finishing write tid 3 to smithi16234619-22 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: finishing write tid 4 to smithi16234619-22 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: finishing write tid 5 to smithi16234619-22 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: oid 22 updating version 0 to 791 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: oid 22 updating version 791 to 794 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: oid 22 version 794 is already newer than 792 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: oid 22 version 794 is already newer than 793 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 794 (ObjNum 814 snap 219 seq_num 814) dirty exists 2024-09-16T15:10:55.117 INFO:tasks.rados.rados.0.smithi162.stdout:2294: left oid 22 (ObjNum 814 snap 219 seq_num 814) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2285: done (5 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2290: done (4 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2293: done (3 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2294: done (2 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2295: done (1 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2296: done (0 left) 2024-09-16T15:10:56.026 INFO:tasks.rados.rados.0.smithi162.stdout:2297: snap_remove snap 213 2024-09-16T15:10:57.031 INFO:tasks.rados.rados.0.smithi162.stdout:2297: done (0 left) 2024-09-16T15:10:57.031 INFO:tasks.rados.rados.0.smithi162.stdout:2298: snap_create 2024-09-16T15:10:58.033 INFO:tasks.rados.rados.0.smithi162.stdout:2298: done (0 left) 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:2299: read oid 6 snap -1 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:2299: expect (ObjNum 795 snap 210 seq_num 795) 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:2300: rollback oid 20 current snap is 221 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 219 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:2301: delete oid 10 current snap is 221 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:2300: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:10:58.034 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 0 (ObjNum 33 snap 0 seq_num 5) dirty dne 2024-09-16T15:10:58.036 INFO:tasks.rados.rados.0.smithi162.stdout:2300: done (2 left) 2024-09-16T15:10:58.036 INFO:tasks.rados.rados.0.smithi162.stdout:2301: done (1 left) 2024-09-16T15:10:58.036 INFO:tasks.rados.rados.0.smithi162.stdout:2302: read oid 24 snap 216 2024-09-16T15:10:58.036 INFO:tasks.rados.rados.0.smithi162.stdout:2302: expect (ObjNum 801 snap 212 seq_num 801) 2024-09-16T15:10:58.036 INFO:tasks.rados.rados.0.smithi162.stdout:2303: delete oid 28 current snap is 221 2024-09-16T15:10:58.038 INFO:tasks.rados.rados.0.smithi162.stdout:2299: expect (ObjNum 795 snap 210 seq_num 795) 2024-09-16T15:10:58.146 INFO:tasks.rados.rados.0.smithi162.stdout:2302: expect (ObjNum 801 snap 212 seq_num 801) 2024-09-16T15:10:58.353 INFO:tasks.rados.rados.0.smithi162.stdout:2299: done (2 left) 2024-09-16T15:10:58.353 INFO:tasks.rados.rados.0.smithi162.stdout:2302: done (1 left) 2024-09-16T15:10:58.353 INFO:tasks.rados.rados.0.smithi162.stdout:2303: done (0 left) 2024-09-16T15:10:58.353 INFO:tasks.rados.rados.0.smithi162.stdout:2304: write oid 5 current snap is 221 2024-09-16T15:10:58.353 INFO:tasks.rados.rados.0.smithi162.stdout:2304: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2024-09-16T15:10:58.359 INFO:tasks.rados.rados.0.smithi162.stdout:2304: writing smithi16234619-5 from 531319 to 1259523 tid 1 2024-09-16T15:10:58.365 INFO:tasks.rados.rados.0.smithi162.stdout:2304: writing smithi16234619-5 from 1758965 to 2243573 tid 2 2024-09-16T15:10:58.370 INFO:tasks.rados.rados.0.smithi162.stdout:2304: writing smithi16234619-5 from 2652938 to 3172996 tid 3 2024-09-16T15:10:58.370 INFO:tasks.rados.rados.0.smithi162.stdout:2305: write oid 22 current snap is 221 2024-09-16T15:10:58.370 INFO:tasks.rados.rados.0.smithi162.stdout:2305: seq_num 818 ranges {548126=403453,1417447=744619,2680361=447683,3680450=67676} 2024-09-16T15:10:58.375 INFO:tasks.rados.rados.0.smithi162.stdout:2305: writing smithi16234619-22 from 548126 to 951579 tid 1 2024-09-16T15:10:58.383 INFO:tasks.rados.rados.0.smithi162.stdout:2305: writing smithi16234619-22 from 1417447 to 2162066 tid 2 2024-09-16T15:10:58.389 INFO:tasks.rados.rados.0.smithi162.stdout:2305: writing smithi16234619-22 from 2680361 to 3128044 tid 3 2024-09-16T15:10:58.393 INFO:tasks.rados.rados.0.smithi162.stdout:2305: writing smithi16234619-22 from 3680450 to 3748126 tid 4 2024-09-16T15:10:58.393 INFO:tasks.rados.rados.0.smithi162.stdout:2306: delete oid 30 current snap is 221 2024-09-16T15:10:58.399 INFO:tasks.rados.rados.0.smithi162.stdout:2306: done (2 left) 2024-09-16T15:10:58.399 INFO:tasks.rados.rados.0.smithi162.stdout:2307: read oid 13 snap -1 2024-09-16T15:10:58.399 INFO:tasks.rados.rados.0.smithi162.stdout:2307: expect deleted 2024-09-16T15:10:58.399 INFO:tasks.rados.rados.0.smithi162.stdout:2308: delete oid 49 current snap is 221 2024-09-16T15:10:58.402 INFO:tasks.rados.rados.0.smithi162.stdout:2304: finishing write tid 1 to smithi16234619-5 2024-09-16T15:10:58.408 INFO:tasks.rados.rados.0.smithi162.stdout:2304: finishing write tid 2 to smithi16234619-5 2024-09-16T15:10:58.408 INFO:tasks.rados.rados.0.smithi162.stdout:2304: finishing write tid 3 to smithi16234619-5 2024-09-16T15:10:58.408 INFO:tasks.rados.rados.0.smithi162.stdout:2304: finishing write tid 4 to smithi16234619-5 2024-09-16T15:10:58.408 INFO:tasks.rados.rados.0.smithi162.stdout:2304: finishing write tid 5 to smithi16234619-5 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:2304: oid 5 updating version 0 to 1049 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:2304: oid 5 version 1049 is already newer than 1046 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:2304: oid 5 version 1049 is already newer than 1047 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:2304: oid 5 version 1049 is already newer than 1048 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1049 (ObjNum 817 snap 221 seq_num 817) dirty exists 2024-09-16T15:10:58.409 INFO:tasks.rados.rados.0.smithi162.stdout:2304: left oid 5 (ObjNum 817 snap 221 seq_num 817) 2024-09-16T15:10:58.424 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 1 to smithi16234619-22 2024-09-16T15:10:58.425 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 2 to smithi16234619-22 2024-09-16T15:10:58.425 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 3 to smithi16234619-22 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 4 to smithi16234619-22 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 5 to smithi16234619-22 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: finishing write tid 6 to smithi16234619-22 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: oid 22 updating version 0 to 796 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: oid 22 updating version 796 to 798 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: oid 22 updating version 798 to 799 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: oid 22 version 799 is already newer than 795 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: oid 22 version 799 is already newer than 797 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 799 (ObjNum 818 snap 221 seq_num 818) dirty exists 2024-09-16T15:10:58.426 INFO:tasks.rados.rados.0.smithi162.stdout:2305: left oid 22 (ObjNum 818 snap 221 seq_num 818) 2024-09-16T15:10:58.427 INFO:tasks.rados.rados.0.smithi162.stdout:2304: done (3 left) 2024-09-16T15:10:58.427 INFO:tasks.rados.rados.0.smithi162.stdout:2305: done (2 left) 2024-09-16T15:10:58.427 INFO:tasks.rados.rados.0.smithi162.stdout:2307: done (1 left) 2024-09-16T15:10:58.427 INFO:tasks.rados.rados.0.smithi162.stdout:2308: done (0 left) 2024-09-16T15:10:58.427 INFO:tasks.rados.rados.0.smithi162.stdout:2309: snap_create 2024-09-16T15:10:59.037 INFO:tasks.rados.rados.0.smithi162.stdout:2309: done (0 left) 2024-09-16T15:10:59.037 INFO:tasks.rados.rados.0.smithi162.stdout:2310: write oid 44 current snap is 222 2024-09-16T15:10:59.037 INFO:tasks.rados.rados.0.smithi162.stdout:2310: seq_num 821 ranges {598547=629200,1876540=491946,3046277=681499} 2024-09-16T15:10:59.043 INFO:tasks.rados.rados.0.smithi162.stdout:2310: writing smithi16234619-44 from 598547 to 1227747 tid 1 2024-09-16T15:10:59.050 INFO:tasks.rados.rados.0.smithi162.stdout:2310: writing smithi16234619-44 from 1876540 to 2368486 tid 2 2024-09-16T15:10:59.059 INFO:tasks.rados.rados.0.smithi162.stdout:2310: writing smithi16234619-44 from 3046277 to 3727776 tid 3 2024-09-16T15:10:59.059 INFO:tasks.rados.rados.0.smithi162.stdout:2311: copy_from oid 7 from oid 43 current snap is 222 2024-09-16T15:10:59.060 INFO:tasks.rados.rados.0.smithi162.stdout:2312: write (excl) oid 15 current snap is 222 2024-09-16T15:10:59.060 INFO:tasks.rados.rados.0.smithi162.stdout:2312: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2024-09-16T15:10:59.066 INFO:tasks.rados.rados.0.smithi162.stdout:2312: writing smithi16234619-15 from 615354 to 1036156 tid 1 2024-09-16T15:10:59.073 INFO:tasks.rados.rados.0.smithi162.stdout:2312: writing smithi16234619-15 from 1651375 to 2403332 tid 2 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:2312: writing smithi16234619-15 from 3190053 to 3682824 tid 3 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:2313: copy_from oid 10 from oid 33 current snap is 222 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:2314: rollback oid 45 current snap is 222 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 219 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:2315: write oid 27 current snap is 222 2024-09-16T15:10:59.081 INFO:tasks.rados.rados.0.smithi162.stdout:2315: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2024-09-16T15:10:59.087 INFO:tasks.rados.rados.0.smithi162.stdout:2315: writing smithi16234619-27 from 632161 to 1128212 tid 1 2024-09-16T15:10:59.093 INFO:tasks.rados.rados.0.smithi162.stdout:2315: writing smithi16234619-27 from 1593504 to 2089119 tid 2 2024-09-16T15:10:59.098 INFO:tasks.rados.rados.0.smithi162.stdout:2315: writing smithi16234619-27 from 2701123 to 3121519 tid 3 2024-09-16T15:10:59.102 INFO:tasks.rados.rados.0.smithi162.stdout:2315: writing smithi16234619-27 from 3611645 to 3832161 tid 4 2024-09-16T15:10:59.102 INFO:tasks.rados.rados.0.smithi162.stdout:2316: read oid 14 snap 220 2024-09-16T15:10:59.103 INFO:tasks.rados.rados.0.smithi162.stdout:2316: expect (ObjNum 813 snap 219 seq_num 813) 2024-09-16T15:10:59.103 INFO:tasks.rados.rados.0.smithi162.stdout:2317: delete oid 48 current snap is 222 2024-09-16T15:10:59.106 INFO:tasks.rados.rados.0.smithi162.stdout:2310: finishing write tid 1 to smithi16234619-44 2024-09-16T15:10:59.107 INFO:tasks.rados.rados.0.smithi162.stdout:2310: finishing write tid 2 to smithi16234619-44 2024-09-16T15:10:59.107 INFO:tasks.rados.rados.0.smithi162.stdout:2310: finishing write tid 3 to smithi16234619-44 2024-09-16T15:10:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:2310: finishing write tid 4 to smithi16234619-44 2024-09-16T15:10:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:2310: finishing write tid 5 to smithi16234619-44 2024-09-16T15:10:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:2310: oid 44 updating version 0 to 924 2024-09-16T15:10:59.109 INFO:tasks.rados.rados.0.smithi162.stdout:2310: oid 44 version 924 is already newer than 923 2024-09-16T15:10:59.110 INFO:tasks.rados.rados.0.smithi162.stdout:2310: oid 44 updating version 924 to 925 2024-09-16T15:10:59.110 INFO:tasks.rados.rados.0.smithi162.stdout:2310: oid 44 updating version 925 to 926 2024-09-16T15:10:59.110 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 926 (ObjNum 821 snap 222 seq_num 821) dirty exists 2024-09-16T15:10:59.110 INFO:tasks.rados.rados.0.smithi162.stdout:2310: left oid 44 (ObjNum 821 snap 222 seq_num 821) 2024-09-16T15:10:59.115 INFO:tasks.rados.rados.0.smithi162.stdout:2316: expect (ObjNum 813 snap 219 seq_num 813) 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2311: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: finishing write tid 1 to smithi16234619-15 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: finishing write tid 2 to smithi16234619-15 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: finishing write tid 3 to smithi16234619-15 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: finishing write tid 4 to smithi16234619-15 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: finishing write tid 5 to smithi16234619-15 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: oid 15 updating version 0 to 876 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: oid 15 version 876 is already newer than 873 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: oid 15 version 876 is already newer than 874 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:2312: oid 15 version 876 is already newer than 875 2024-09-16T15:10:59.283 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 876 (ObjNum 822 snap 222 seq_num 822) dirty exists 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2312: left oid 15 (ObjNum 822 snap 222 seq_num 822) 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2313: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2310: done (7 left) 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2312: done (6 left) 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2316: done (5 left) 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2317: done (4 left) 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2318: snap_create 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2314: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2314: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2314: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1089 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 1 to smithi16234619-27 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 2 to smithi16234619-27 2024-09-16T15:10:59.284 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 3 to smithi16234619-27 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 4 to smithi16234619-27 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 5 to smithi16234619-27 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: finishing write tid 6 to smithi16234619-27 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: oid 27 updating version 0 to 895 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: oid 27 version 895 is already newer than 891 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: oid 27 version 895 is already newer than 893 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: oid 27 version 895 is already newer than 892 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: oid 27 version 895 is already newer than 894 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 895 (ObjNum 823 snap 222 seq_num 823) dirty exists 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2315: left oid 27 (ObjNum 823 snap 222 seq_num 823) 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2311: finishing copy_from to smithi16234619-7 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 893 (ObjNum 809 snap 218 seq_num 809) dirty exists 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:2313: finishing copy_from to smithi16234619-10 2024-09-16T15:10:59.285 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2311: done (4 left) 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2313: done (3 left) 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2314: done (2 left) 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2315: done (1 left) 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2318: done (0 left) 2024-09-16T15:11:00.014 INFO:tasks.rados.rados.0.smithi162.stdout:2319: snap_remove snap 220 2024-09-16T15:11:01.028 INFO:tasks.rados.rados.0.smithi162.stdout:2319: done (0 left) 2024-09-16T15:11:01.028 INFO:tasks.rados.rados.0.smithi162.stdout:2320: write (excl) oid 14 current snap is 223 2024-09-16T15:11:01.028 INFO:tasks.rados.rados.0.smithi162.stdout:2320: seq_num 825 ranges {665775=646549,2110468=499284,3155969=442940} 2024-09-16T15:11:01.036 INFO:tasks.rados.rados.0.smithi162.stdout:2320: writing smithi16234619-14 from 665775 to 1312324 tid 1 2024-09-16T15:11:01.043 INFO:tasks.rados.rados.0.smithi162.stdout:2320: writing smithi16234619-14 from 2110468 to 2609752 tid 2 2024-09-16T15:11:01.048 INFO:tasks.rados.rados.0.smithi162.stdout:2320: writing smithi16234619-14 from 3155969 to 3598909 tid 3 2024-09-16T15:11:01.048 INFO:tasks.rados.rados.0.smithi162.stdout:2321: snap_create 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: finishing write tid 1 to smithi16234619-14 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: finishing write tid 2 to smithi16234619-14 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: finishing write tid 3 to smithi16234619-14 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: finishing write tid 4 to smithi16234619-14 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: finishing write tid 5 to smithi16234619-14 2024-09-16T15:11:01.075 INFO:tasks.rados.rados.0.smithi162.stdout:2320: oid 14 updating version 0 to 958 2024-09-16T15:11:01.076 INFO:tasks.rados.rados.0.smithi162.stdout:2320: oid 14 updating version 958 to 959 2024-09-16T15:11:01.076 INFO:tasks.rados.rados.0.smithi162.stdout:2320: oid 14 version 959 is already newer than 957 2024-09-16T15:11:01.076 INFO:tasks.rados.rados.0.smithi162.stdout:2320: oid 14 version 959 is already newer than 956 2024-09-16T15:11:01.076 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 959 (ObjNum 825 snap 223 seq_num 825) dirty exists 2024-09-16T15:11:01.076 INFO:tasks.rados.rados.0.smithi162.stdout:2320: left oid 14 (ObjNum 825 snap 223 seq_num 825) 2024-09-16T15:11:02.034 INFO:tasks.rados.rados.0.smithi162.stdout:2320: done (1 left) 2024-09-16T15:11:02.034 INFO:tasks.rados.rados.0.smithi162.stdout:2321: done (0 left) 2024-09-16T15:11:02.034 INFO:tasks.rados.rados.0.smithi162.stdout:2322: snap_remove snap 223 2024-09-16T15:11:03.039 INFO:tasks.rados.rados.0.smithi162.stdout:2322: done (0 left) 2024-09-16T15:11:03.039 INFO:tasks.rados.rados.0.smithi162.stdout:2323: read oid 33 snap 219 2024-09-16T15:11:03.039 INFO:tasks.rados.rados.0.smithi162.stdout:2323: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:03.039 INFO:tasks.rados.rados.0.smithi162.stdout:2324: read oid 28 snap -1 2024-09-16T15:11:03.040 INFO:tasks.rados.rados.0.smithi162.stdout:2324: expect deleted 2024-09-16T15:11:03.040 INFO:tasks.rados.rados.0.smithi162.stdout:2325: copy_from oid 29 from oid 18 current snap is 224 2024-09-16T15:11:03.040 INFO:tasks.rados.rados.0.smithi162.stdout:2326: read oid 21 snap -1 2024-09-16T15:11:03.040 INFO:tasks.rados.rados.0.smithi162.stdout:2326: expect deleted 2024-09-16T15:11:03.040 INFO:tasks.rados.rados.0.smithi162.stdout:2327: delete oid 27 current snap is 224 2024-09-16T15:11:03.044 INFO:tasks.rados.rados.0.smithi162.stdout:2323: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2325: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2325: finishing copy_from to smithi16234619-29 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 850 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2323: done (4 left) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2324: done (3 left) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2325: done (2 left) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2326: done (1 left) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2327: done (0 left) 2024-09-16T15:11:03.210 INFO:tasks.rados.rados.0.smithi162.stdout:2328: write oid 49 current snap is 224 2024-09-16T15:11:03.211 INFO:tasks.rados.rados.0.smithi162.stdout:2328: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2024-09-16T15:11:03.217 INFO:tasks.rados.rados.0.smithi162.stdout:2328: writing smithi16234619-49 from 699389 to 1496436 tid 1 2024-09-16T15:11:03.223 INFO:tasks.rados.rados.0.smithi162.stdout:2328: writing smithi16234619-49 from 2111079 to 2614032 tid 2 2024-09-16T15:11:03.229 INFO:tasks.rados.rados.0.smithi162.stdout:2328: writing smithi16234619-49 from 3094462 to 3676299 tid 3 2024-09-16T15:11:03.229 INFO:tasks.rados.rados.0.smithi162.stdout:2329: snap_remove snap 221 2024-09-16T15:11:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:2329: done (1 left) 2024-09-16T15:11:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:2330: copy_from oid 43 from oid 46 current snap is 224 2024-09-16T15:11:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:2328: finishing write tid 1 to smithi16234619-49 2024-09-16T15:11:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:2328: finishing write tid 2 to smithi16234619-49 2024-09-16T15:11:04.418 INFO:tasks.rados.rados.0.smithi162.stdout:2331: read oid 42 snap -1 2024-09-16T15:11:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:2331: expect deleted 2024-09-16T15:11:04.419 INFO:tasks.rados.rados.0.smithi162.stdout:2332: snap_remove snap 222 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2332: done (3 left) 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2333: write oid 4 current snap is 224 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: finishing write tid 3 to smithi16234619-49 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: finishing write tid 4 to smithi16234619-49 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: finishing write tid 5 to smithi16234619-49 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: oid 49 updating version 0 to 803 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: oid 49 version 803 is already newer than 802 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: oid 49 version 803 is already newer than 800 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: oid 49 version 803 is already newer than 801 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2328: left oid 49 (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:11:06.133 INFO:tasks.rados.rados.0.smithi162.stdout:2333: seq_num 828 ranges {716196=472296,1769561=646611,3121885=793109} 2024-09-16T15:11:06.138 INFO:tasks.rados.rados.0.smithi162.stdout:2333: writing smithi16234619-4 from 716196 to 1188492 tid 1 2024-09-16T15:11:06.145 INFO:tasks.rados.rados.0.smithi162.stdout:2333: writing smithi16234619-4 from 1769561 to 2416172 tid 2 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2333: writing smithi16234619-4 from 3121885 to 3914994 tid 3 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2328: done (3 left) 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2331: done (2 left) 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2334: read oid 40 snap -1 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2334: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2335: write (excl) oid 23 current snap is 224 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2330: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:11:06.154 INFO:tasks.rados.rados.0.smithi162.stdout:2330: finishing copy_from to smithi16234619-43 2024-09-16T15:11:06.155 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 591 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:11:06.155 INFO:tasks.rados.rados.0.smithi162.stdout:2335: seq_num 829 ranges {733003=663898,1944396=506622,2865661=604381} 2024-09-16T15:11:06.161 INFO:tasks.rados.rados.0.smithi162.stdout:2335: writing smithi16234619-23 from 733003 to 1396901 tid 1 2024-09-16T15:11:06.167 INFO:tasks.rados.rados.0.smithi162.stdout:2335: writing smithi16234619-23 from 1944396 to 2451018 tid 2 2024-09-16T15:11:06.173 INFO:tasks.rados.rados.0.smithi162.stdout:2335: writing smithi16234619-23 from 2865661 to 3470042 tid 3 2024-09-16T15:11:06.173 INFO:tasks.rados.rados.0.smithi162.stdout:2330: done (3 left) 2024-09-16T15:11:06.173 INFO:tasks.rados.rados.0.smithi162.stdout:2336: write oid 20 current snap is 224 2024-09-16T15:11:06.173 INFO:tasks.rados.rados.0.smithi162.stdout:2336: seq_num 830 ranges {749810=739147,2002878=650280,3293084=415653} 2024-09-16T15:11:06.181 INFO:tasks.rados.rados.0.smithi162.stdout:2336: writing smithi16234619-20 from 749810 to 1488957 tid 1 2024-09-16T15:11:06.188 INFO:tasks.rados.rados.0.smithi162.stdout:2336: writing smithi16234619-20 from 2002878 to 2653158 tid 2 2024-09-16T15:11:06.194 INFO:tasks.rados.rados.0.smithi162.stdout:2336: writing smithi16234619-20 from 3293084 to 3708737 tid 3 2024-09-16T15:11:06.194 INFO:tasks.rados.rados.0.smithi162.stdout:2337: copy_from oid 38 from oid 13 current snap is 224 2024-09-16T15:11:06.194 INFO:tasks.rados.rados.0.smithi162.stdout:2334: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: finishing write tid 1 to smithi16234619-4 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: finishing write tid 2 to smithi16234619-4 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: finishing write tid 3 to smithi16234619-4 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: finishing write tid 4 to smithi16234619-4 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: finishing write tid 5 to smithi16234619-4 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: oid 4 updating version 0 to 901 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: oid 4 version 901 is already newer than 900 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: oid 4 version 901 is already newer than 898 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: oid 4 version 901 is already newer than 899 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 901 (ObjNum 828 snap 224 seq_num 828) dirty exists 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2333: left oid 4 (ObjNum 828 snap 224 seq_num 828) 2024-09-16T15:11:06.274 INFO:tasks.rados.rados.0.smithi162.stdout:2337: finishing copy_from to smithi16234619-38 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2337: got expected ENOENT (src dne) 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2337: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2335: finishing write tid 1 to smithi16234619-23 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2333: done (4 left) 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2334: done (3 left) 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2337: done (2 left) 2024-09-16T15:11:06.275 INFO:tasks.rados.rados.0.smithi162.stdout:2338: snap_remove snap 206 2024-09-16T15:11:07.036 INFO:tasks.rados.rados.0.smithi162.stdout:2338: done (2 left) 2024-09-16T15:11:07.036 INFO:tasks.rados.rados.0.smithi162.stdout:2339: delete oid 22 current snap is 224 2024-09-16T15:11:07.036 INFO:tasks.rados.rados.0.smithi162.stdout:2335: finishing write tid 2 to smithi16234619-23 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: finishing write tid 3 to smithi16234619-23 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: finishing write tid 4 to smithi16234619-23 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: finishing write tid 5 to smithi16234619-23 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: oid 23 updating version 0 to 591 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: oid 23 updating version 591 to 593 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: oid 23 version 593 is already newer than 590 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: oid 23 version 593 is already newer than 592 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 593 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2335: left oid 23 (ObjNum 829 snap 224 seq_num 829) 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2336: finishing write tid 1 to smithi16234619-20 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2336: finishing write tid 2 to smithi16234619-20 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2336: finishing write tid 3 to smithi16234619-20 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2336: finishing write tid 4 to smithi16234619-20 2024-09-16T15:11:07.037 INFO:tasks.rados.rados.0.smithi162.stdout:2336: finishing write tid 5 to smithi16234619-20 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:2336: oid 20 updating version 0 to 805 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:2336: oid 20 updating version 805 to 807 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:2336: oid 20 updating version 807 to 808 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:2336: oid 20 version 808 is already newer than 806 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 808 (ObjNum 830 snap 224 seq_num 830) dirty exists 2024-09-16T15:11:07.038 INFO:tasks.rados.rados.0.smithi162.stdout:2336: left oid 20 (ObjNum 830 snap 224 seq_num 830) 2024-09-16T15:11:07.042 INFO:tasks.rados.rados.0.smithi162.stdout:2335: done (2 left) 2024-09-16T15:11:07.042 INFO:tasks.rados.rados.0.smithi162.stdout:2336: done (1 left) 2024-09-16T15:11:07.042 INFO:tasks.rados.rados.0.smithi162.stdout:2339: done (0 left) 2024-09-16T15:11:07.042 INFO:tasks.rados.rados.0.smithi162.stdout:2340: write (excl) oid 21 current snap is 224 2024-09-16T15:11:07.042 INFO:tasks.rados.rados.0.smithi162.stdout:2340: seq_num 832 ranges {783424=489645,2003489=653949,3231577=554550} 2024-09-16T15:11:07.049 INFO:tasks.rados.rados.0.smithi162.stdout:2340: writing smithi16234619-21 from 783424 to 1273069 tid 1 2024-09-16T15:11:07.059 INFO:tasks.rados.rados.0.smithi162.stdout:2340: writing smithi16234619-21 from 2003489 to 2657438 tid 2 2024-09-16T15:11:07.066 INFO:tasks.rados.rados.0.smithi162.stdout:2340: writing smithi16234619-21 from 3231577 to 3786127 tid 3 2024-09-16T15:11:07.066 INFO:tasks.rados.rados.0.smithi162.stdout:2341: delete oid 47 current snap is 224 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2341: done (1 left) 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2342: copy_from oid 36 from oid 28 current snap is 224 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2343: read oid 9 snap -1 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2343: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2344: read oid 18 snap -1 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2344: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2345: write (excl) oid 45 current snap is 224 2024-09-16T15:11:07.068 INFO:tasks.rados.rados.0.smithi162.stdout:2345: seq_num 834 ranges {417038=640143,1720453=296585} 2024-09-16T15:11:07.074 INFO:tasks.rados.rados.0.smithi162.stdout:2345: writing smithi16234619-45 from 417038 to 1057181 tid 1 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2345: writing smithi16234619-45 from 1720453 to 2017038 tid 2 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2346: read oid 41 snap -1 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2346: expect deleted 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2347: read oid 47 snap -1 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2347: expect deleted 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2348: copy_from oid 42 from oid 19 current snap is 224 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2342: finishing copy_from to smithi16234619-36 2024-09-16T15:11:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:2342: got expected ENOENT (src dne) 2024-09-16T15:11:07.081 INFO:tasks.rados.rados.0.smithi162.stdout:2342: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:11:07.081 INFO:tasks.rados.rados.0.smithi162.stdout:2343: expect (ObjNum 744 snap 194 seq_num 744) 2024-09-16T15:11:07.209 INFO:tasks.rados.rados.0.smithi162.stdout:2344: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:07.377 INFO:tasks.rados.rados.0.smithi162.stdout:2340: finishing write tid 1 to smithi16234619-21 2024-09-16T15:11:07.377 INFO:tasks.rados.rados.0.smithi162.stdout:2340: finishing write tid 2 to smithi16234619-21 2024-09-16T15:11:07.377 INFO:tasks.rados.rados.0.smithi162.stdout:2340: finishing write tid 3 to smithi16234619-21 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: finishing write tid 4 to smithi16234619-21 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: finishing write tid 5 to smithi16234619-21 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: oid 21 updating version 0 to 1107 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: oid 21 updating version 1107 to 1109 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: oid 21 version 1109 is already newer than 1108 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: oid 21 updating version 1109 to 1110 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1110 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: left oid 21 (ObjNum 832 snap 224 seq_num 832) 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2345: finishing write tid 1 to smithi16234619-45 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2345: finishing write tid 2 to smithi16234619-45 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2340: done (7 left) 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2342: done (6 left) 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2343: done (5 left) 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2344: done (4 left) 2024-09-16T15:11:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2346: done (3 left) 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2347: done (2 left) 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2349: delete oid 18 current snap is 224 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: finishing write tid 3 to smithi16234619-45 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: finishing write tid 4 to smithi16234619-45 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: oid 45 updating version 0 to 1112 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: oid 45 updating version 1112 to 1113 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: oid 45 version 1113 is already newer than 1111 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1113 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2345: left oid 45 (ObjNum 834 snap 224 seq_num 834) 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2348: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:2348: finishing copy_from to smithi16234619-42 2024-09-16T15:11:07.379 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 865 (ObjNum 736 snap 194 seq_num 736) dirty exists 2024-09-16T15:11:07.380 INFO:tasks.rados.rados.0.smithi162.stdout:2345: done (2 left) 2024-09-16T15:11:07.380 INFO:tasks.rados.rados.0.smithi162.stdout:2348: done (1 left) 2024-09-16T15:11:07.380 INFO:tasks.rados.rados.0.smithi162.stdout:2349: done (0 left) 2024-09-16T15:11:07.380 INFO:tasks.rados.rados.0.smithi162.stdout:2350: delete oid 9 current snap is 224 2024-09-16T15:11:07.382 INFO:tasks.rados.rados.0.smithi162.stdout:2350: done (0 left) 2024-09-16T15:11:07.382 INFO:tasks.rados.rados.0.smithi162.stdout:2351: write oid 30 current snap is 224 2024-09-16T15:11:07.382 INFO:tasks.rados.rados.0.smithi162.stdout:2351: seq_num 837 ranges {467459=582243,1495899=404945} 2024-09-16T15:11:07.388 INFO:tasks.rados.rados.0.smithi162.stdout:2351: writing smithi16234619-30 from 467459 to 1049702 tid 1 2024-09-16T15:11:07.392 INFO:tasks.rados.rados.0.smithi162.stdout:2351: writing smithi16234619-30 from 1495899 to 1900844 tid 2 2024-09-16T15:11:07.392 INFO:tasks.rados.rados.0.smithi162.stdout:2352: read oid 27 snap -1 2024-09-16T15:11:07.392 INFO:tasks.rados.rados.0.smithi162.stdout:2352: expect deleted 2024-09-16T15:11:07.393 INFO:tasks.rados.rados.0.smithi162.stdout:2353: snap_create 2024-09-16T15:11:07.411 INFO:tasks.rados.rados.0.smithi162.stdout:2351: finishing write tid 1 to smithi16234619-30 2024-09-16T15:11:07.411 INFO:tasks.rados.rados.0.smithi162.stdout:2351: finishing write tid 2 to smithi16234619-30 2024-09-16T15:11:07.411 INFO:tasks.rados.rados.0.smithi162.stdout:2351: finishing write tid 3 to smithi16234619-30 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:2351: finishing write tid 4 to smithi16234619-30 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:2351: oid 30 updating version 0 to 878 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:2351: oid 30 version 878 is already newer than 877 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:2351: oid 30 updating version 878 to 879 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 879 (ObjNum 837 snap 224 seq_num 837) dirty exists 2024-09-16T15:11:07.412 INFO:tasks.rados.rados.0.smithi162.stdout:2351: left oid 30 (ObjNum 837 snap 224 seq_num 837) 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2351: done (2 left) 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2352: done (1 left) 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2353: done (0 left) 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2354: read oid 7 snap 219 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2354: expect deleted 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2355: read oid 37 snap -1 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2355: expect (ObjNum 721 snap 190 seq_num 721) 2024-09-16T15:11:08.037 INFO:tasks.rados.rados.0.smithi162.stdout:2356: delete oid 19 current snap is 225 2024-09-16T15:11:08.039 INFO:tasks.rados.rados.0.smithi162.stdout:2355: expect (ObjNum 721 snap 190 seq_num 721) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2354: done (2 left) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2355: done (1 left) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2356: done (0 left) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2357: read oid 26 snap -1 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2357: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2358: read oid 2 snap -1 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2358: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2359: copy_from oid 42 from oid 5 current snap is 225 2024-09-16T15:11:08.170 INFO:tasks.rados.rados.0.smithi162.stdout:2360: delete oid 8 current snap is 225 2024-09-16T15:11:08.173 INFO:tasks.rados.rados.0.smithi162.stdout:2360: done (3 left) 2024-09-16T15:11:08.173 INFO:tasks.rados.rados.0.smithi162.stdout:2361: copy_from oid 33 from oid 10 current snap is 225 2024-09-16T15:11:08.173 INFO:tasks.rados.rados.0.smithi162.stdout:2362: write oid 11 current snap is 225 2024-09-16T15:11:08.173 INFO:tasks.rados.rados.0.smithi162.stdout:2362: seq_num 840 ranges {517880=407990,1554992=562888} 2024-09-16T15:11:08.179 INFO:tasks.rados.rados.0.smithi162.stdout:2362: writing smithi16234619-11 from 517880 to 925870 tid 1 2024-09-16T15:11:08.185 INFO:tasks.rados.rados.0.smithi162.stdout:2362: writing smithi16234619-11 from 1554992 to 2117880 tid 2 2024-09-16T15:11:08.186 INFO:tasks.rados.rados.0.smithi162.stdout:2363: copy_from oid 49 from oid 18 current snap is 225 2024-09-16T15:11:08.186 INFO:tasks.rados.rados.0.smithi162.stdout:2357: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:11:08.445 INFO:tasks.rados.rados.0.smithi162.stdout:2358: expect (ObjNum 762 snap 198 seq_num 762) 2024-09-16T15:11:08.567 INFO:tasks.rados.rados.0.smithi162.stdout:2363: finishing copy_from to smithi16234619-49 2024-09-16T15:11:08.567 INFO:tasks.rados.rados.0.smithi162.stdout:2363: got expected ENOENT (src dne) 2024-09-16T15:11:08.567 INFO:tasks.rados.rados.0.smithi162.stdout:2363: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:11:08.567 INFO:tasks.rados.rados.0.smithi162.stdout:2362: finishing write tid 1 to smithi16234619-11 2024-09-16T15:11:08.567 INFO:tasks.rados.rados.0.smithi162.stdout:2362: finishing write tid 2 to smithi16234619-11 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: finishing write tid 3 to smithi16234619-11 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: finishing write tid 4 to smithi16234619-11 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: oid 11 updating version 0 to 176 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: oid 11 version 176 is already newer than 175 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: oid 11 version 176 is already newer than 174 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 176 (ObjNum 840 snap 225 seq_num 840) dirty exists 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: left oid 11 (ObjNum 840 snap 225 seq_num 840) 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2359: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2357: done (5 left) 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2358: done (4 left) 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2362: done (3 left) 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2363: done (2 left) 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2364: write (excl) oid 29 current snap is 225 2024-09-16T15:11:08.568 INFO:tasks.rados.rados.0.smithi162.stdout:2359: finishing copy_from to smithi16234619-42 2024-09-16T15:11:08.569 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1049 (ObjNum 817 snap 221 seq_num 817) dirty exists 2024-09-16T15:11:08.569 INFO:tasks.rados.rados.0.smithi162.stdout:2361: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:11:08.569 INFO:tasks.rados.rados.0.smithi162.stdout:2361: finishing copy_from to smithi16234619-33 2024-09-16T15:11:08.569 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:11:08.569 INFO:tasks.rados.rados.0.smithi162.stdout:2364: seq_num 841 ranges {534687=483239,1613474=412283} 2024-09-16T15:11:08.572 INFO:tasks.rados.rados.0.smithi162.stdout:2364: writing smithi16234619-29 from 534687 to 1017926 tid 1 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2364: writing smithi16234619-29 from 1613474 to 2025757 tid 2 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2359: done (2 left) 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2361: done (1 left) 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2365: read oid 46 snap -1 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2365: expect (ObjNum 812 snap 219 seq_num 812) 2024-09-16T15:11:08.578 INFO:tasks.rados.rados.0.smithi162.stdout:2366: snap_remove snap 216 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2366: done (2 left) 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2367: read oid 40 snap -1 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2367: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2368: copy_from oid 25 from oid 23 current snap is 225 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2369: delete oid 4 current snap is 225 2024-09-16T15:11:09.011 INFO:tasks.rados.rados.0.smithi162.stdout:2365: expect (ObjNum 812 snap 219 seq_num 812) 2024-09-16T15:11:09.131 INFO:tasks.rados.rados.0.smithi162.stdout:2364: finishing write tid 1 to smithi16234619-29 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: finishing write tid 2 to smithi16234619-29 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: finishing write tid 3 to smithi16234619-29 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: finishing write tid 4 to smithi16234619-29 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: oid 29 updating version 0 to 1070 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: oid 29 updating version 1070 to 1071 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: oid 29 updating version 1071 to 1072 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1072 (ObjNum 841 snap 225 seq_num 841) dirty exists 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2364: left oid 29 (ObjNum 841 snap 225 seq_num 841) 2024-09-16T15:11:09.132 INFO:tasks.rados.rados.0.smithi162.stdout:2367: expect (ObjNum 730 snap 193 seq_num 730) 2024-09-16T15:11:09.205 INFO:tasks.rados.rados.0.smithi162.stdout:2368: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:11:09.205 INFO:tasks.rados.rados.0.smithi162.stdout:2368: finishing copy_from to smithi16234619-25 2024-09-16T15:11:09.205 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 593 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:09.205 INFO:tasks.rados.rados.0.smithi162.stdout:2364: done (4 left) 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2365: done (3 left) 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2367: done (2 left) 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2368: done (1 left) 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2369: done (0 left) 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2370: write (excl) oid 28 current snap is 225 2024-09-16T15:11:09.206 INFO:tasks.rados.rados.0.smithi162.stdout:2370: seq_num 843 ranges {568301=633737,1614085=415952} 2024-09-16T15:11:09.211 INFO:tasks.rados.rados.0.smithi162.stdout:2370: writing smithi16234619-28 from 568301 to 1202038 tid 1 2024-09-16T15:11:09.216 INFO:tasks.rados.rados.0.smithi162.stdout:2370: writing smithi16234619-28 from 1614085 to 2030037 tid 2 2024-09-16T15:11:09.216 INFO:tasks.rados.rados.0.smithi162.stdout:2371: read oid 3 snap -1 2024-09-16T15:11:09.216 INFO:tasks.rados.rados.0.smithi162.stdout:2371: expect deleted 2024-09-16T15:11:09.216 INFO:tasks.rados.rados.0.smithi162.stdout:2372: write (excl) oid 46 current snap is 225 2024-09-16T15:11:09.216 INFO:tasks.rados.rados.0.smithi162.stdout:2372: seq_num 844 ranges {585108=425339,1788920=396188} 2024-09-16T15:11:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:2372: writing smithi16234619-46 from 585108 to 1010447 tid 1 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2372: writing smithi16234619-46 from 1788920 to 2185108 tid 2 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2373: read oid 47 snap 219 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2373: expect deleted 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2374: read oid 48 snap -1 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2374: expect deleted 2024-09-16T15:11:09.228 INFO:tasks.rados.rados.0.smithi162.stdout:2371: done (4 left) 2024-09-16T15:11:09.229 INFO:tasks.rados.rados.0.smithi162.stdout:2375: write oid 16 current snap is 225 2024-09-16T15:11:09.229 INFO:tasks.rados.rados.0.smithi162.stdout:2375: seq_num 845 ranges {601915=500588,1847402=354513} 2024-09-16T15:11:09.234 INFO:tasks.rados.rados.0.smithi162.stdout:2375: writing smithi16234619-16 from 601915 to 1102503 tid 1 2024-09-16T15:11:09.240 INFO:tasks.rados.rados.0.smithi162.stdout:2375: writing smithi16234619-16 from 1847402 to 2201915 tid 2 2024-09-16T15:11:09.240 INFO:tasks.rados.rados.0.smithi162.stdout:2376: write (excl) oid 24 current snap is 225 2024-09-16T15:11:09.240 INFO:tasks.rados.rados.0.smithi162.stdout:2376: seq_num 846 ranges {618722=575837,1905884=312838} 2024-09-16T15:11:09.246 INFO:tasks.rados.rados.0.smithi162.stdout:2376: writing smithi16234619-24 from 618722 to 1194559 tid 1 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2376: writing smithi16234619-24 from 1905884 to 2218722 tid 2 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2377: copy_from oid 42 from oid 43 current snap is 225 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: finishing write tid 1 to smithi16234619-28 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: finishing write tid 2 to smithi16234619-28 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: finishing write tid 3 to smithi16234619-28 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: finishing write tid 4 to smithi16234619-28 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: oid 28 updating version 0 to 881 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: oid 28 updating version 881 to 882 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: oid 28 version 882 is already newer than 880 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 882 (ObjNum 843 snap 225 seq_num 843) dirty exists 2024-09-16T15:11:09.253 INFO:tasks.rados.rados.0.smithi162.stdout:2370: left oid 28 (ObjNum 843 snap 225 seq_num 843) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: finishing write tid 1 to smithi16234619-46 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: finishing write tid 2 to smithi16234619-46 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: finishing write tid 3 to smithi16234619-46 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: finishing write tid 4 to smithi16234619-46 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: oid 46 updating version 0 to 593 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: oid 46 updating version 593 to 594 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: oid 46 version 594 is already newer than 592 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 594 (ObjNum 844 snap 225 seq_num 844) dirty exists 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: left oid 46 (ObjNum 844 snap 225 seq_num 844) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2370: done (6 left) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2372: done (5 left) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2373: done (4 left) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2374: done (3 left) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2378: read oid 36 snap 224 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2378: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:09.254 INFO:tasks.rados.rados.0.smithi162.stdout:2379: snap_remove snap 219 2024-09-16T15:11:10.009 INFO:tasks.rados.rados.0.smithi162.stdout:2379: done (4 left) 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2380: copy_from oid 25 from oid 41 current snap is 225 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: finishing write tid 1 to smithi16234619-16 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: finishing write tid 2 to smithi16234619-16 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: finishing write tid 3 to smithi16234619-16 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: finishing write tid 4 to smithi16234619-16 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: oid 16 updating version 0 to 927 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: oid 16 updating version 927 to 929 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: oid 16 version 929 is already newer than 928 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 929 (ObjNum 845 snap 225 seq_num 845) dirty exists 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2375: left oid 16 (ObjNum 845 snap 225 seq_num 845) 2024-09-16T15:11:10.010 INFO:tasks.rados.rados.0.smithi162.stdout:2378: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: finishing write tid 1 to smithi16234619-24 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: finishing write tid 2 to smithi16234619-24 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: finishing write tid 3 to smithi16234619-24 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: finishing write tid 4 to smithi16234619-24 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: oid 24 updating version 0 to 1119 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: oid 24 updating version 1119 to 1121 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: oid 24 version 1121 is already newer than 1120 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1121 (ObjNum 846 snap 225 seq_num 846) dirty exists 2024-09-16T15:11:10.181 INFO:tasks.rados.rados.0.smithi162.stdout:2376: left oid 24 (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2377: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2377: finishing copy_from to smithi16234619-42 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 591 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2375: done (4 left) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2376: done (3 left) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2377: done (2 left) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2378: done (1 left) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2381: copy_from oid 38 from oid 50 current snap is 225 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2380: finishing copy_from to smithi16234619-25 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2380: got expected ENOENT (src dne) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2380: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2380: done (1 left) 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2382: copy_from oid 28 from oid 21 current snap is 225 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2383: copy_from oid 16 from oid 8 current snap is 225 2024-09-16T15:11:10.182 INFO:tasks.rados.rados.0.smithi162.stdout:2384: write (excl) oid 34 current snap is 225 2024-09-16T15:11:10.183 INFO:tasks.rados.rados.0.smithi162.stdout:2384: seq_num 847 ranges {635529=651086,1848013=387516} 2024-09-16T15:11:10.188 INFO:tasks.rados.rados.0.smithi162.stdout:2384: writing smithi16234619-34 from 635529 to 1286615 tid 1 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2384: writing smithi16234619-34 from 1848013 to 2235529 tid 2 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2385: read oid 15 snap -1 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2385: expect (ObjNum 822 snap 222 seq_num 822) 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2383: finishing copy_from to smithi16234619-16 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2383: got expected ENOENT (src dne) 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2383: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2383: done (4 left) 2024-09-16T15:11:10.194 INFO:tasks.rados.rados.0.smithi162.stdout:2386: delete oid 14 current snap is 225 2024-09-16T15:11:10.208 INFO:tasks.rados.rados.0.smithi162.stdout:2385: expect (ObjNum 822 snap 222 seq_num 822) 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2384: finishing write tid 1 to smithi16234619-34 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2384: finishing write tid 2 to smithi16234619-34 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2385: done (4 left) 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2386: done (3 left) 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2387: read oid 30 snap -1 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2387: expect (ObjNum 837 snap 224 seq_num 837) 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2388: snap_create 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2384: finishing write tid 3 to smithi16234619-34 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2384: finishing write tid 4 to smithi16234619-34 2024-09-16T15:11:10.434 INFO:tasks.rados.rados.0.smithi162.stdout:2384: oid 34 updating version 0 to 986 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2384: oid 34 version 986 is already newer than 985 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2384: oid 34 updating version 986 to 987 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 987 (ObjNum 847 snap 225 seq_num 847) dirty exists 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2384: left oid 34 (ObjNum 847 snap 225 seq_num 847) 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2382: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2382: finishing copy_from to smithi16234619-28 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1110 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2381: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:2381: finishing copy_from to smithi16234619-38 2024-09-16T15:11:10.435 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 946 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:11:10.440 INFO:tasks.rados.rados.0.smithi162.stdout:2387: expect (ObjNum 837 snap 224 seq_num 837) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2381: done (4 left) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2382: done (3 left) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2384: done (2 left) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2387: done (1 left) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2388: done (0 left) 2024-09-16T15:11:11.016 INFO:tasks.rados.rados.0.smithi162.stdout:2389: delete oid 34 current snap is 226 2024-09-16T15:11:11.021 INFO:tasks.rados.rados.0.smithi162.stdout:2389: done (0 left) 2024-09-16T15:11:11.021 INFO:tasks.rados.rados.0.smithi162.stdout:2390: copy_from oid 17 from oid 10 current snap is 226 2024-09-16T15:11:11.021 INFO:tasks.rados.rados.0.smithi162.stdout:2391: write (excl) oid 19 current snap is 226 2024-09-16T15:11:11.021 INFO:tasks.rados.rados.0.smithi162.stdout:2391: seq_num 850 ranges {685950=476833,1623459=570617} 2024-09-16T15:11:11.027 INFO:tasks.rados.rados.0.smithi162.stdout:2391: writing smithi16234619-19 from 685950 to 1162783 tid 1 2024-09-16T15:11:11.033 INFO:tasks.rados.rados.0.smithi162.stdout:2391: writing smithi16234619-19 from 1623459 to 2194076 tid 2 2024-09-16T15:11:11.033 INFO:tasks.rados.rados.0.smithi162.stdout:2392: read oid 36 snap -1 2024-09-16T15:11:11.033 INFO:tasks.rados.rados.0.smithi162.stdout:2392: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:11.033 INFO:tasks.rados.rados.0.smithi162.stdout:2393: read oid 18 snap 225 2024-09-16T15:11:11.033 INFO:tasks.rados.rados.0.smithi162.stdout:2393: expect deleted 2024-09-16T15:11:11.034 INFO:tasks.rados.rados.0.smithi162.stdout:2394: snap_create 2024-09-16T15:11:11.059 INFO:tasks.rados.rados.0.smithi162.stdout:2392: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:11.227 INFO:tasks.rados.rados.0.smithi162.stdout:2391: finishing write tid 1 to smithi16234619-19 2024-09-16T15:11:11.227 INFO:tasks.rados.rados.0.smithi162.stdout:2391: finishing write tid 2 to smithi16234619-19 2024-09-16T15:11:11.227 INFO:tasks.rados.rados.0.smithi162.stdout:2391: finishing write tid 3 to smithi16234619-19 2024-09-16T15:11:11.227 INFO:tasks.rados.rados.0.smithi162.stdout:2391: finishing write tid 4 to smithi16234619-19 2024-09-16T15:11:11.227 INFO:tasks.rados.rados.0.smithi162.stdout:2391: oid 19 updating version 0 to 994 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:2391: oid 19 version 994 is already newer than 993 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:2391: oid 19 version 994 is already newer than 992 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 994 (ObjNum 850 snap 226 seq_num 850) dirty exists 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:2391: left oid 19 (ObjNum 850 snap 226 seq_num 850) 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:2390: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:2390: finishing copy_from to smithi16234619-17 2024-09-16T15:11:11.228 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:11:12.018 INFO:tasks.rados.rados.0.smithi162.stdout:2390: done (4 left) 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2391: done (3 left) 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2392: done (2 left) 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2393: done (1 left) 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2394: done (0 left) 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2395: rollback oid 24 current snap is 227 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 224 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2396: copy_from oid 16 from oid 44 current snap is 227 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2397: write oid 7 current snap is 227 2024-09-16T15:11:12.019 INFO:tasks.rados.rados.0.smithi162.stdout:2397: seq_num 851 ranges {702757=552082,1681941=430628} 2024-09-16T15:11:12.025 INFO:tasks.rados.rados.0.smithi162.stdout:2397: writing smithi16234619-7 from 702757 to 1254839 tid 1 2024-09-16T15:11:12.030 INFO:tasks.rados.rados.0.smithi162.stdout:2397: writing smithi16234619-7 from 1681941 to 2112569 tid 2 2024-09-16T15:11:12.030 INFO:tasks.rados.rados.0.smithi162.stdout:2398: delete oid 8 current snap is 227 2024-09-16T15:11:12.031 INFO:tasks.rados.rados.0.smithi162.stdout:2395: finishing rollback tid 0 to smithi16234619-24 2024-09-16T15:11:12.031 INFO:tasks.rados.rados.0.smithi162.stdout:2395: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:11:12.031 INFO:tasks.rados.rados.0.smithi162.stdout:2395: finishing rollback tid 2 to smithi16234619-24 2024-09-16T15:11:12.031 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1130 (ObjNum 801 snap 212 seq_num 801) dirty exists 2024-09-16T15:11:12.052 INFO:tasks.rados.rados.0.smithi162.stdout:2397: finishing write tid 1 to smithi16234619-7 2024-09-16T15:11:12.052 INFO:tasks.rados.rados.0.smithi162.stdout:2397: finishing write tid 2 to smithi16234619-7 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: finishing write tid 3 to smithi16234619-7 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: finishing write tid 4 to smithi16234619-7 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: oid 7 updating version 0 to 898 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: oid 7 version 898 is already newer than 896 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: oid 7 version 898 is already newer than 897 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 898 (ObjNum 851 snap 227 seq_num 851) dirty exists 2024-09-16T15:11:12.054 INFO:tasks.rados.rados.0.smithi162.stdout:2397: left oid 7 (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2395: done (3 left) 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2397: done (2 left) 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2398: done (1 left) 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2399: rollback oid 35 current snap is 227 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 224 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2400: copy_from oid 30 from oid 49 current snap is 227 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2401: rollback oid 19 current snap is 227 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 226 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2402: write (excl) oid 28 current snap is 227 2024-09-16T15:11:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2402: seq_num 853 ranges {736371=418933,1798905=434297} 2024-09-16T15:11:12.063 INFO:tasks.rados.rados.0.smithi162.stdout:2402: writing smithi16234619-28 from 736371 to 1155304 tid 1 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2402: writing smithi16234619-28 from 1798905 to 2233202 tid 2 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2403: copy_from oid 34 from oid 41 current snap is 227 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2401: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2401: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2401: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 998 (ObjNum 850 snap 226 seq_num 850) dirty exists 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2401: done (5 left) 2024-09-16T15:11:12.069 INFO:tasks.rados.rados.0.smithi162.stdout:2404: snap_remove snap 225 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2404: done (5 left) 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2405: copy_from oid 4 from oid 25 current snap is 227 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2396: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2396: finishing copy_from to smithi16234619-16 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 926 (ObjNum 821 snap 222 seq_num 821) dirty exists 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2399: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2399: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:2399: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:11:13.033 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 949 (ObjNum 660 snap 171 seq_num 660) dirty exists 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2403: finishing copy_from to smithi16234619-34 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2403: got expected ENOENT (src dne) 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2400: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2403: finishing copy_from racing read to smithi16234619-34 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2400: finishing copy_from to smithi16234619-30 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: finishing write tid 1 to smithi16234619-28 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: finishing write tid 2 to smithi16234619-28 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: finishing write tid 3 to smithi16234619-28 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: finishing write tid 4 to smithi16234619-28 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: oid 28 updating version 0 to 1113 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: oid 28 version 1113 is already newer than 1111 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: oid 28 version 1113 is already newer than 1112 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1113 (ObjNum 853 snap 227 seq_num 853) dirty exists 2024-09-16T15:11:13.034 INFO:tasks.rados.rados.0.smithi162.stdout:2402: left oid 28 (ObjNum 853 snap 227 seq_num 853) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2396: done (5 left) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2399: done (4 left) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2400: done (3 left) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2402: done (2 left) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2403: done (1 left) 2024-09-16T15:11:13.035 INFO:tasks.rados.rados.0.smithi162.stdout:2406: delete oid 48 current snap is 227 2024-09-16T15:11:13.038 INFO:tasks.rados.rados.0.smithi162.stdout:2406: done (1 left) 2024-09-16T15:11:13.038 INFO:tasks.rados.rados.0.smithi162.stdout:2407: delete oid 16 current snap is 227 2024-09-16T15:11:13.041 INFO:tasks.rados.rados.0.smithi162.stdout:2407: done (1 left) 2024-09-16T15:11:13.041 INFO:tasks.rados.rados.0.smithi162.stdout:2408: copy_from oid 20 from oid 42 current snap is 227 2024-09-16T15:11:13.041 INFO:tasks.rados.rados.0.smithi162.stdout:2409: write oid 46 current snap is 227 2024-09-16T15:11:13.041 INFO:tasks.rados.rados.0.smithi162.stdout:2409: seq_num 856 ranges {786792=644680,1857998=528794} 2024-09-16T15:11:13.049 INFO:tasks.rados.rados.0.smithi162.stdout:2409: writing smithi16234619-46 from 786792 to 1431472 tid 1 2024-09-16T15:11:13.055 INFO:tasks.rados.rados.0.smithi162.stdout:2409: writing smithi16234619-46 from 1857998 to 2386792 tid 2 2024-09-16T15:11:13.055 INFO:tasks.rados.rados.0.smithi162.stdout:2410: copy_from oid 47 from oid 21 current snap is 227 2024-09-16T15:11:13.055 INFO:tasks.rados.rados.0.smithi162.stdout:2411: snap_remove snap 224 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2411: done (4 left) 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2412: delete oid 41 current snap is 227 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: finishing write tid 1 to smithi16234619-46 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: finishing write tid 2 to smithi16234619-46 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: finishing write tid 3 to smithi16234619-46 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: finishing write tid 4 to smithi16234619-46 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: oid 46 updating version 0 to 597 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: oid 46 version 597 is already newer than 595 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: oid 46 version 597 is already newer than 596 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 597 (ObjNum 856 snap 227 seq_num 856) dirty exists 2024-09-16T15:11:14.007 INFO:tasks.rados.rados.0.smithi162.stdout:2409: left oid 46 (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2405: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2405: finishing copy_from to smithi16234619-4 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 593 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2408: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2408: finishing copy_from to smithi16234619-20 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 591 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2410: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:2410: finishing copy_from to smithi16234619-47 2024-09-16T15:11:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1110 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2405: done (4 left) 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2408: done (3 left) 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2409: done (2 left) 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2410: done (1 left) 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2412: done (0 left) 2024-09-16T15:11:14.015 INFO:tasks.rados.rados.0.smithi162.stdout:2413: write (excl) oid 32 current snap is 227 2024-09-16T15:11:14.016 INFO:tasks.rados.rados.0.smithi162.stdout:2413: seq_num 858 ranges {420406=795178,1974962=445444} 2024-09-16T15:11:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:2413: writing smithi16234619-32 from 420406 to 1215584 tid 1 2024-09-16T15:11:14.030 INFO:tasks.rados.rados.0.smithi162.stdout:2413: writing smithi16234619-32 from 1974962 to 2420406 tid 2 2024-09-16T15:11:14.030 INFO:tasks.rados.rados.0.smithi162.stdout:2414: read oid 35 snap -1 2024-09-16T15:11:14.030 INFO:tasks.rados.rados.0.smithi162.stdout:2414: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:11:14.030 INFO:tasks.rados.rados.0.smithi162.stdout:2415: write (excl) oid 1 current snap is 227 2024-09-16T15:11:14.031 INFO:tasks.rados.rados.0.smithi162.stdout:2415: seq_num 859 ranges {437213=470427,1633444=728951} 2024-09-16T15:11:14.035 INFO:tasks.rados.rados.0.smithi162.stdout:2415: writing smithi16234619-1 from 437213 to 907640 tid 1 2024-09-16T15:11:14.044 INFO:tasks.rados.rados.0.smithi162.stdout:2415: writing smithi16234619-1 from 1633444 to 2362395 tid 2 2024-09-16T15:11:14.044 INFO:tasks.rados.rados.0.smithi162.stdout:2416: rollback oid 24 current snap is 227 2024-09-16T15:11:14.044 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 226 2024-09-16T15:11:14.044 INFO:tasks.rados.rados.0.smithi162.stdout:2414: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2416: finishing rollback tid 0 to smithi16234619-24 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2416: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2416: finishing rollback tid 2 to smithi16234619-24 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1137 (ObjNum 846 snap 225 seq_num 846) dirty exists 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2415: finishing write tid 1 to smithi16234619-1 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2413: finishing write tid 1 to smithi16234619-32 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2415: finishing write tid 2 to smithi16234619-1 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2413: finishing write tid 2 to smithi16234619-32 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2415: finishing write tid 3 to smithi16234619-1 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2413: finishing write tid 3 to smithi16234619-32 2024-09-16T15:11:14.148 INFO:tasks.rados.rados.0.smithi162.stdout:2414: done (3 left) 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2416: done (2 left) 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2417: delete oid 15 current snap is 227 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2415: finishing write tid 4 to smithi16234619-1 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2415: oid 1 updating version 0 to 904 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2415: oid 1 updating version 904 to 906 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2415: oid 1 version 906 is already newer than 905 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 906 (ObjNum 859 snap 227 seq_num 859) dirty exists 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2415: left oid 1 (ObjNum 859 snap 227 seq_num 859) 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2413: finishing write tid 4 to smithi16234619-32 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2413: oid 32 updating version 0 to 895 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2413: oid 32 version 895 is already newer than 894 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2413: oid 32 version 895 is already newer than 893 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 895 (ObjNum 858 snap 227 seq_num 858) dirty exists 2024-09-16T15:11:14.149 INFO:tasks.rados.rados.0.smithi162.stdout:2413: left oid 32 (ObjNum 858 snap 227 seq_num 858) 2024-09-16T15:11:14.150 INFO:tasks.rados.rados.0.smithi162.stdout:2413: done (2 left) 2024-09-16T15:11:14.150 INFO:tasks.rados.rados.0.smithi162.stdout:2415: done (1 left) 2024-09-16T15:11:14.150 INFO:tasks.rados.rados.0.smithi162.stdout:2417: done (0 left) 2024-09-16T15:11:14.151 INFO:tasks.rados.rados.0.smithi162.stdout:2418: read oid 44 snap -1 2024-09-16T15:11:14.151 INFO:tasks.rados.rados.0.smithi162.stdout:2418: expect (ObjNum 821 snap 222 seq_num 821) 2024-09-16T15:11:14.151 INFO:tasks.rados.rados.0.smithi162.stdout:2419: read oid 18 snap 226 2024-09-16T15:11:14.151 INFO:tasks.rados.rados.0.smithi162.stdout:2419: expect deleted 2024-09-16T15:11:14.151 INFO:tasks.rados.rados.0.smithi162.stdout:2420: snap_create 2024-09-16T15:11:14.154 INFO:tasks.rados.rados.0.smithi162.stdout:2418: expect (ObjNum 821 snap 222 seq_num 821) 2024-09-16T15:11:15.638 INFO:tasks.rados.rados.0.smithi162.stdout:2418: done (2 left) 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2419: done (1 left) 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2420: done (0 left) 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2421: read oid 44 snap -1 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2421: expect (ObjNum 821 snap 222 seq_num 821) 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2422: rollback oid 3 current snap is 228 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 227 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2423: read oid 36 snap -1 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2423: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2424: snap_create 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:2422: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:11:15.639 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 0 (ObjNum 545 snap 0 seq_num 2027921136) dirty dne 2024-09-16T15:11:15.642 INFO:tasks.rados.rados.0.smithi162.stdout:2421: expect (ObjNum 821 snap 222 seq_num 821) 2024-09-16T15:11:15.885 INFO:tasks.rados.rados.0.smithi162.stdout:2423: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:16.649 INFO:tasks.rados.rados.0.smithi162.stdout:2421: done (3 left) 2024-09-16T15:11:16.649 INFO:tasks.rados.rados.0.smithi162.stdout:2422: done (2 left) 2024-09-16T15:11:16.649 INFO:tasks.rados.rados.0.smithi162.stdout:2423: done (1 left) 2024-09-16T15:11:16.650 INFO:tasks.rados.rados.0.smithi162.stdout:2424: done (0 left) 2024-09-16T15:11:16.650 INFO:tasks.rados.rados.0.smithi162.stdout:2425: delete oid 34 current snap is 229 2024-09-16T15:11:16.654 INFO:tasks.rados.rados.0.smithi162.stdout:2425: done (0 left) 2024-09-16T15:11:16.654 INFO:tasks.rados.rados.0.smithi162.stdout:2426: snap_create 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2426: done (0 left) 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2427: read oid 36 snap -1 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2427: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2428: read oid 35 snap -1 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2428: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2429: copy_from oid 42 from oid 23 current snap is 230 2024-09-16T15:11:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:2430: snap_create 2024-09-16T15:11:17.653 INFO:tasks.rados.rados.0.smithi162.stdout:2428: expect (ObjNum 660 snap 171 seq_num 660) 2024-09-16T15:11:17.759 INFO:tasks.rados.rados.0.smithi162.stdout:2427: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:17.926 INFO:tasks.rados.rados.0.smithi162.stdout:2429: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:11:17.927 INFO:tasks.rados.rados.0.smithi162.stdout:2429: finishing copy_from to smithi16234619-42 2024-09-16T15:11:17.927 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 593 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:19.234 INFO:tasks.rados.rados.0.smithi162.stdout:2427: done (3 left) 2024-09-16T15:11:19.234 INFO:tasks.rados.rados.0.smithi162.stdout:2428: done (2 left) 2024-09-16T15:11:19.234 INFO:tasks.rados.rados.0.smithi162.stdout:2429: done (1 left) 2024-09-16T15:11:19.234 INFO:tasks.rados.rados.0.smithi162.stdout:2430: done (0 left) 2024-09-16T15:11:19.235 INFO:tasks.rados.rados.0.smithi162.stdout:2431: read oid 30 snap -1 2024-09-16T15:11:19.235 INFO:tasks.rados.rados.0.smithi162.stdout:2431: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:11:19.235 INFO:tasks.rados.rados.0.smithi162.stdout:2432: snap_create 2024-09-16T15:11:19.247 INFO:tasks.rados.rados.0.smithi162.stdout:2431: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:11:20.235 INFO:tasks.rados.rados.0.smithi162.stdout:2431: done (1 left) 2024-09-16T15:11:20.235 INFO:tasks.rados.rados.0.smithi162.stdout:2432: done (0 left) 2024-09-16T15:11:20.235 INFO:tasks.rados.rados.0.smithi162.stdout:2433: read oid 36 snap -1 2024-09-16T15:11:20.235 INFO:tasks.rados.rados.0.smithi162.stdout:2433: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:20.235 INFO:tasks.rados.rados.0.smithi162.stdout:2434: delete oid 17 current snap is 232 2024-09-16T15:11:20.243 INFO:tasks.rados.rados.0.smithi162.stdout:2434: done (1 left) 2024-09-16T15:11:20.243 INFO:tasks.rados.rados.0.smithi162.stdout:2435: read oid 49 snap -1 2024-09-16T15:11:20.243 INFO:tasks.rados.rados.0.smithi162.stdout:2435: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:11:20.243 INFO:tasks.rados.rados.0.smithi162.stdout:2436: write (excl) oid 38 current snap is 232 2024-09-16T15:11:20.244 INFO:tasks.rados.rados.0.smithi162.stdout:2436: seq_num 863 ranges {504441=487776,1467372=736289} 2024-09-16T15:11:20.249 INFO:tasks.rados.rados.0.smithi162.stdout:2436: writing smithi16234619-38 from 504441 to 992217 tid 1 2024-09-16T15:11:20.256 INFO:tasks.rados.rados.0.smithi162.stdout:2436: writing smithi16234619-38 from 1467372 to 2203661 tid 2 2024-09-16T15:11:20.256 INFO:tasks.rados.rados.0.smithi162.stdout:2437: read oid 16 snap -1 2024-09-16T15:11:20.257 INFO:tasks.rados.rados.0.smithi162.stdout:2437: expect deleted 2024-09-16T15:11:20.257 INFO:tasks.rados.rados.0.smithi162.stdout:2438: copy_from oid 43 from oid 50 current snap is 232 2024-09-16T15:11:20.257 INFO:tasks.rados.rados.0.smithi162.stdout:2433: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:20.432 INFO:tasks.rados.rados.0.smithi162.stdout:2435: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:11:20.554 INFO:tasks.rados.rados.0.smithi162.stdout:2436: finishing write tid 1 to smithi16234619-38 2024-09-16T15:11:20.554 INFO:tasks.rados.rados.0.smithi162.stdout:2436: finishing write tid 2 to smithi16234619-38 2024-09-16T15:11:20.554 INFO:tasks.rados.rados.0.smithi162.stdout:2436: finishing write tid 3 to smithi16234619-38 2024-09-16T15:11:20.554 INFO:tasks.rados.rados.0.smithi162.stdout:2436: finishing write tid 4 to smithi16234619-38 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2436: oid 38 updating version 0 to 953 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2436: oid 38 version 953 is already newer than 951 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2436: oid 38 version 953 is already newer than 952 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 953 (ObjNum 863 snap 232 seq_num 863) dirty exists 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2436: left oid 38 (ObjNum 863 snap 232 seq_num 863) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2438: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2438: finishing copy_from to smithi16234619-43 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 946 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2433: done (4 left) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2435: done (3 left) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2436: done (2 left) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2437: done (1 left) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2438: done (0 left) 2024-09-16T15:11:20.555 INFO:tasks.rados.rados.0.smithi162.stdout:2439: snap_create 2024-09-16T15:11:21.636 INFO:tasks.rados.rados.0.smithi162.stdout:2439: done (0 left) 2024-09-16T15:11:21.637 INFO:tasks.rados.rados.0.smithi162.stdout:2440: snap_remove snap 229 2024-09-16T15:11:23.641 INFO:tasks.rados.rados.0.smithi162.stdout:2440: done (0 left) 2024-09-16T15:11:23.641 INFO:tasks.rados.rados.0.smithi162.stdout:2441: write oid 30 current snap is 233 2024-09-16T15:11:23.641 INFO:tasks.rados.rados.0.smithi162.stdout:2441: seq_num 864 ranges {521248=563025,1809501=596300} 2024-09-16T15:11:23.647 INFO:tasks.rados.rados.0.smithi162.stdout:2441: writing smithi16234619-30 from 521248 to 1084273 tid 1 2024-09-16T15:11:23.655 INFO:tasks.rados.rados.0.smithi162.stdout:2441: writing smithi16234619-30 from 1809501 to 2405801 tid 2 2024-09-16T15:11:23.655 INFO:tasks.rados.rados.0.smithi162.stdout:2442: read oid 24 snap -1 2024-09-16T15:11:23.655 INFO:tasks.rados.rados.0.smithi162.stdout:2442: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:11:23.655 INFO:tasks.rados.rados.0.smithi162.stdout:2443: snap_create 2024-09-16T15:11:23.663 INFO:tasks.rados.rados.0.smithi162.stdout:2442: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: finishing write tid 1 to smithi16234619-30 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: finishing write tid 2 to smithi16234619-30 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: finishing write tid 3 to smithi16234619-30 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: finishing write tid 4 to smithi16234619-30 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: oid 30 updating version 0 to 1116 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: oid 30 updating version 1116 to 1117 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: oid 30 version 1117 is already newer than 1115 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1117 (ObjNum 864 snap 233 seq_num 864) dirty exists 2024-09-16T15:11:23.836 INFO:tasks.rados.rados.0.smithi162.stdout:2441: left oid 30 (ObjNum 864 snap 233 seq_num 864) 2024-09-16T15:11:24.009 INFO:tasks.rados.rados.0.smithi162.stdout:2441: done (2 left) 2024-09-16T15:11:24.009 INFO:tasks.rados.rados.0.smithi162.stdout:2442: done (1 left) 2024-09-16T15:11:24.009 INFO:tasks.rados.rados.0.smithi162.stdout:2443: done (0 left) 2024-09-16T15:11:24.009 INFO:tasks.rados.rados.0.smithi162.stdout:2444: delete oid 42 current snap is 234 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:2444: done (0 left) 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:2445: rollback oid 45 current snap is 234 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 227 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:2446: rollback oid 37 current snap is 234 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 227 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:2447: rollback oid 5 current snap is 234 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 226 2024-09-16T15:11:24.014 INFO:tasks.rados.rados.0.smithi162.stdout:2448: snap_remove snap 233 2024-09-16T15:11:25.653 INFO:tasks.rados.rados.0.smithi162.stdout:2448: done (3 left) 2024-09-16T15:11:25.654 INFO:tasks.rados.rados.0.smithi162.stdout:2449: write oid 14 current snap is 234 2024-09-16T15:11:25.654 INFO:tasks.rados.rados.0.smithi162.stdout:2446: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:11:25.654 INFO:tasks.rados.rados.0.smithi162.stdout:2449: seq_num 866 ranges {554862=713523,1926465=599969} 2024-09-16T15:11:25.660 INFO:tasks.rados.rados.0.smithi162.stdout:2449: writing smithi16234619-14 from 554862 to 1268385 tid 1 2024-09-16T15:11:25.667 INFO:tasks.rados.rados.0.smithi162.stdout:2449: writing smithi16234619-14 from 1926465 to 2526434 tid 2 2024-09-16T15:11:25.667 INFO:tasks.rados.rados.0.smithi162.stdout:2450: snap_create 2024-09-16T15:11:25.667 INFO:tasks.rados.rados.0.smithi162.stdout:2445: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:11:25.667 INFO:tasks.rados.rados.0.smithi162.stdout:2445: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:11:25.667 INFO:tasks.rados.rados.0.smithi162.stdout:2445: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1145 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:2446: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:2446: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1095 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:2447: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:2447: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:2447: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:11:25.668 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1099 (ObjNum 817 snap 221 seq_num 817) dirty exists 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: finishing write tid 1 to smithi16234619-14 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: finishing write tid 2 to smithi16234619-14 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: finishing write tid 3 to smithi16234619-14 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: finishing write tid 4 to smithi16234619-14 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: oid 14 updating version 0 to 1012 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: oid 14 version 1012 is already newer than 1010 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: oid 14 version 1012 is already newer than 1011 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1012 (ObjNum 866 snap 234 seq_num 866) dirty exists 2024-09-16T15:11:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:2449: left oid 14 (ObjNum 866 snap 234 seq_num 866) 2024-09-16T15:11:27.159 INFO:tasks.rados.rados.0.smithi162.stdout:2445: done (4 left) 2024-09-16T15:11:27.159 INFO:tasks.rados.rados.0.smithi162.stdout:2446: done (3 left) 2024-09-16T15:11:27.159 INFO:tasks.rados.rados.0.smithi162.stdout:2447: done (2 left) 2024-09-16T15:11:27.160 INFO:tasks.rados.rados.0.smithi162.stdout:2449: done (1 left) 2024-09-16T15:11:27.160 INFO:tasks.rados.rados.0.smithi162.stdout:2450: done (0 left) 2024-09-16T15:11:27.160 INFO:tasks.rados.rados.0.smithi162.stdout:2451: write (excl) oid 8 current snap is 235 2024-09-16T15:11:27.160 INFO:tasks.rados.rados.0.smithi162.stdout:2451: seq_num 867 ranges {571669=505125,1701300=743627} 2024-09-16T15:11:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:2451: writing smithi16234619-8 from 571669 to 1076794 tid 1 2024-09-16T15:11:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:2451: writing smithi16234619-8 from 1701300 to 2444927 tid 2 2024-09-16T15:11:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:2452: delete oid 38 current snap is 235 2024-09-16T15:11:27.176 INFO:tasks.rados.rados.0.smithi162.stdout:2452: done (1 left) 2024-09-16T15:11:27.176 INFO:tasks.rados.rados.0.smithi162.stdout:2453: read oid 15 snap -1 2024-09-16T15:11:27.176 INFO:tasks.rados.rados.0.smithi162.stdout:2453: expect deleted 2024-09-16T15:11:27.176 INFO:tasks.rados.rados.0.smithi162.stdout:2454: snap_remove snap 226 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2454: done (2 left) 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2455: read oid 33 snap 234 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2455: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2456: read oid 10 snap -1 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2456: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2457: copy_from oid 17 from oid 16 current snap is 235 2024-09-16T15:11:28.170 INFO:tasks.rados.rados.0.smithi162.stdout:2451: finishing write tid 1 to smithi16234619-8 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: finishing write tid 2 to smithi16234619-8 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: finishing write tid 3 to smithi16234619-8 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: finishing write tid 4 to smithi16234619-8 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: oid 8 updating version 0 to 899 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: oid 8 updating version 899 to 901 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: oid 8 version 901 is already newer than 900 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 901 (ObjNum 867 snap 235 seq_num 867) dirty exists 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: left oid 8 (ObjNum 867 snap 235 seq_num 867) 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2451: done (4 left) 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2453: done (3 left) 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2458: write (excl) oid 39 current snap is 235 2024-09-16T15:11:28.171 INFO:tasks.rados.rados.0.smithi162.stdout:2458: seq_num 869 ranges {605283=655623,1701911=747296} 2024-09-16T15:11:28.177 INFO:tasks.rados.rados.0.smithi162.stdout:2458: writing smithi16234619-39 from 605283 to 1260906 tid 1 2024-09-16T15:11:28.185 INFO:tasks.rados.rados.0.smithi162.stdout:2458: writing smithi16234619-39 from 1701911 to 2449207 tid 2 2024-09-16T15:11:28.185 INFO:tasks.rados.rados.0.smithi162.stdout:2459: snap_create 2024-09-16T15:11:28.186 INFO:tasks.rados.rados.0.smithi162.stdout:2456: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:28.348 INFO:tasks.rados.rados.0.smithi162.stdout:2455: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2457: finishing copy_from to smithi16234619-17 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2457: got expected ENOENT (src dne) 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2457: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: finishing write tid 1 to smithi16234619-39 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: finishing write tid 2 to smithi16234619-39 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: finishing write tid 3 to smithi16234619-39 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: finishing write tid 4 to smithi16234619-39 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: oid 39 updating version 0 to 1112 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: oid 39 updating version 1112 to 1113 2024-09-16T15:11:28.509 INFO:tasks.rados.rados.0.smithi162.stdout:2458: oid 39 version 1113 is already newer than 1111 2024-09-16T15:11:28.510 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1113 (ObjNum 869 snap 235 seq_num 869) dirty exists 2024-09-16T15:11:28.510 INFO:tasks.rados.rados.0.smithi162.stdout:2458: left oid 39 (ObjNum 869 snap 235 seq_num 869) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2455: done (4 left) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2456: done (3 left) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2457: done (2 left) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2458: done (1 left) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2459: done (0 left) 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2460: copy_from oid 27 from oid 16 current snap is 236 2024-09-16T15:11:29.008 INFO:tasks.rados.rados.0.smithi162.stdout:2461: snap_create 2024-09-16T15:11:29.010 INFO:tasks.rados.rados.0.smithi162.stdout:2460: finishing copy_from to smithi16234619-27 2024-09-16T15:11:29.010 INFO:tasks.rados.rados.0.smithi162.stdout:2460: got expected ENOENT (src dne) 2024-09-16T15:11:29.010 INFO:tasks.rados.rados.0.smithi162.stdout:2460: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:11:30.636 INFO:tasks.rados.rados.0.smithi162.stdout:2460: done (1 left) 2024-09-16T15:11:30.636 INFO:tasks.rados.rados.0.smithi162.stdout:2461: done (0 left) 2024-09-16T15:11:30.636 INFO:tasks.rados.rados.0.smithi162.stdout:2462: rollback oid 50 current snap is 237 2024-09-16T15:11:30.636 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 235 2024-09-16T15:11:30.636 INFO:tasks.rados.rados.0.smithi162.stdout:2463: snap_remove snap 236 2024-09-16T15:11:31.641 INFO:tasks.rados.rados.0.smithi162.stdout:2463: done (1 left) 2024-09-16T15:11:31.641 INFO:tasks.rados.rados.0.smithi162.stdout:2464: read oid 24 snap 227 2024-09-16T15:11:31.641 INFO:tasks.rados.rados.0.smithi162.stdout:2464: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:11:31.641 INFO:tasks.rados.rados.0.smithi162.stdout:2465: delete oid 44 current snap is 237 2024-09-16T15:11:31.642 INFO:tasks.rados.rados.0.smithi162.stdout:2462: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:11:31.642 INFO:tasks.rados.rados.0.smithi162.stdout:2462: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:11:31.642 INFO:tasks.rados.rados.0.smithi162.stdout:2462: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:11:31.642 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1018 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:11:31.645 INFO:tasks.rados.rados.0.smithi162.stdout:2464: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:11:31.819 INFO:tasks.rados.rados.0.smithi162.stdout:2462: done (2 left) 2024-09-16T15:11:31.819 INFO:tasks.rados.rados.0.smithi162.stdout:2464: done (1 left) 2024-09-16T15:11:31.819 INFO:tasks.rados.rados.0.smithi162.stdout:2465: done (0 left) 2024-09-16T15:11:31.819 INFO:tasks.rados.rados.0.smithi162.stdout:2466: snap_remove snap 231 2024-09-16T15:11:33.415 INFO:tasks.rados.rados.0.smithi162.stdout:2466: done (0 left) 2024-09-16T15:11:33.416 INFO:tasks.rados.rados.0.smithi162.stdout:2467: delete oid 28 current snap is 237 2024-09-16T15:11:33.419 INFO:tasks.rados.rados.0.smithi162.stdout:2467: done (0 left) 2024-09-16T15:11:33.419 INFO:tasks.rados.rados.0.smithi162.stdout:2468: copy_from oid 16 from oid 20 current snap is 237 2024-09-16T15:11:33.419 INFO:tasks.rados.rados.0.smithi162.stdout:2469: write oid 32 current snap is 237 2024-09-16T15:11:33.419 INFO:tasks.rados.rados.0.smithi162.stdout:2469: seq_num 872 ranges {655704=481370,1761004=494623} 2024-09-16T15:11:33.425 INFO:tasks.rados.rados.0.smithi162.stdout:2469: writing smithi16234619-32 from 655704 to 1137074 tid 1 2024-09-16T15:11:33.432 INFO:tasks.rados.rados.0.smithi162.stdout:2469: writing smithi16234619-32 from 1761004 to 2255627 tid 2 2024-09-16T15:11:33.432 INFO:tasks.rados.rados.0.smithi162.stdout:2470: snap_create 2024-09-16T15:11:33.450 INFO:tasks.rados.rados.0.smithi162.stdout:2469: finishing write tid 1 to smithi16234619-32 2024-09-16T15:11:33.452 INFO:tasks.rados.rados.0.smithi162.stdout:2469: finishing write tid 2 to smithi16234619-32 2024-09-16T15:11:33.452 INFO:tasks.rados.rados.0.smithi162.stdout:2469: finishing write tid 3 to smithi16234619-32 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:2469: finishing write tid 4 to smithi16234619-32 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:2469: oid 32 updating version 0 to 897 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:2469: oid 32 updating version 897 to 898 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:2469: oid 32 version 898 is already newer than 896 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 898 (ObjNum 872 snap 237 seq_num 872) dirty exists 2024-09-16T15:11:33.453 INFO:tasks.rados.rados.0.smithi162.stdout:2469: left oid 32 (ObjNum 872 snap 237 seq_num 872) 2024-09-16T15:11:33.469 INFO:tasks.rados.rados.0.smithi162.stdout:2468: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:11:33.471 INFO:tasks.rados.rados.0.smithi162.stdout:2468: finishing copy_from to smithi16234619-16 2024-09-16T15:11:33.471 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 591 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:11:34.011 INFO:tasks.rados.rados.0.smithi162.stdout:2468: done (2 left) 2024-09-16T15:11:34.011 INFO:tasks.rados.rados.0.smithi162.stdout:2469: done (1 left) 2024-09-16T15:11:34.011 INFO:tasks.rados.rados.0.smithi162.stdout:2470: done (0 left) 2024-09-16T15:11:34.011 INFO:tasks.rados.rados.0.smithi162.stdout:2471: write oid 27 current snap is 238 2024-09-16T15:11:34.011 INFO:tasks.rados.rados.0.smithi162.stdout:2471: seq_num 873 ranges {672511=556619,1819486=754634} 2024-09-16T15:11:34.018 INFO:tasks.rados.rados.0.smithi162.stdout:2471: writing smithi16234619-27 from 672511 to 1229130 tid 1 2024-09-16T15:11:34.026 INFO:tasks.rados.rados.0.smithi162.stdout:2471: writing smithi16234619-27 from 1819486 to 2574120 tid 2 2024-09-16T15:11:34.026 INFO:tasks.rados.rados.0.smithi162.stdout:2472: snap_create 2024-09-16T15:11:34.054 INFO:tasks.rados.rados.0.smithi162.stdout:2471: finishing write tid 1 to smithi16234619-27 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: finishing write tid 2 to smithi16234619-27 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: finishing write tid 3 to smithi16234619-27 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: finishing write tid 4 to smithi16234619-27 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: oid 27 updating version 0 to 909 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: oid 27 version 909 is already newer than 908 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: oid 27 version 909 is already newer than 907 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 909 (ObjNum 873 snap 238 seq_num 873) dirty exists 2024-09-16T15:11:34.057 INFO:tasks.rados.rados.0.smithi162.stdout:2471: left oid 27 (ObjNum 873 snap 238 seq_num 873) 2024-09-16T15:11:35.025 INFO:tasks.rados.rados.0.smithi162.stdout:2471: done (1 left) 2024-09-16T15:11:35.025 INFO:tasks.rados.rados.0.smithi162.stdout:2472: done (0 left) 2024-09-16T15:11:35.025 INFO:tasks.rados.rados.0.smithi162.stdout:2473: read oid 13 snap -1 2024-09-16T15:11:35.025 INFO:tasks.rados.rados.0.smithi162.stdout:2473: expect deleted 2024-09-16T15:11:35.025 INFO:tasks.rados.rados.0.smithi162.stdout:2474: copy_from oid 23 from oid 18 current snap is 239 2024-09-16T15:11:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:2475: delete oid 40 current snap is 239 2024-09-16T15:11:35.027 INFO:tasks.rados.rados.0.smithi162.stdout:2474: finishing copy_from to smithi16234619-23 2024-09-16T15:11:35.028 INFO:tasks.rados.rados.0.smithi162.stdout:2474: got expected ENOENT (src dne) 2024-09-16T15:11:35.028 INFO:tasks.rados.rados.0.smithi162.stdout:2474: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:11:35.029 INFO:tasks.rados.rados.0.smithi162.stdout:2473: done (2 left) 2024-09-16T15:11:35.029 INFO:tasks.rados.rados.0.smithi162.stdout:2474: done (1 left) 2024-09-16T15:11:35.029 INFO:tasks.rados.rados.0.smithi162.stdout:2475: done (0 left) 2024-09-16T15:11:35.030 INFO:tasks.rados.rados.0.smithi162.stdout:2476: snap_create 2024-09-16T15:11:36.026 INFO:tasks.rados.rados.0.smithi162.stdout:2476: done (0 left) 2024-09-16T15:11:36.026 INFO:tasks.rados.rados.0.smithi162.stdout:2477: write oid 30 current snap is 240 2024-09-16T15:11:36.026 INFO:tasks.rados.rados.0.smithi162.stdout:2477: seq_num 875 ranges {706125=423470,1652803=758303} 2024-09-16T15:11:36.032 INFO:tasks.rados.rados.0.smithi162.stdout:2477: writing smithi16234619-30 from 706125 to 1129595 tid 1 2024-09-16T15:11:36.039 INFO:tasks.rados.rados.0.smithi162.stdout:2477: writing smithi16234619-30 from 1652803 to 2411106 tid 2 2024-09-16T15:11:36.039 INFO:tasks.rados.rados.0.smithi162.stdout:2478: copy_from oid 38 from oid 27 current snap is 240 2024-09-16T15:11:36.039 INFO:tasks.rados.rados.0.smithi162.stdout:2479: snap_create 2024-09-16T15:11:36.063 INFO:tasks.rados.rados.0.smithi162.stdout:2477: finishing write tid 1 to smithi16234619-30 2024-09-16T15:11:36.066 INFO:tasks.rados.rados.0.smithi162.stdout:2477: finishing write tid 2 to smithi16234619-30 2024-09-16T15:11:36.066 INFO:tasks.rados.rados.0.smithi162.stdout:2477: finishing write tid 3 to smithi16234619-30 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2477: finishing write tid 4 to smithi16234619-30 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2477: oid 30 updating version 0 to 1120 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2477: oid 30 version 1120 is already newer than 1119 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2477: oid 30 updating version 1120 to 1121 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1121 (ObjNum 875 snap 240 seq_num 875) dirty exists 2024-09-16T15:11:36.067 INFO:tasks.rados.rados.0.smithi162.stdout:2477: left oid 30 (ObjNum 875 snap 240 seq_num 875) 2024-09-16T15:11:36.076 INFO:tasks.rados.rados.0.smithi162.stdout:2478: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:11:36.079 INFO:tasks.rados.rados.0.smithi162.stdout:2478: finishing copy_from to smithi16234619-38 2024-09-16T15:11:36.079 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 909 (ObjNum 873 snap 238 seq_num 873) dirty exists 2024-09-16T15:11:37.036 INFO:tasks.rados.rados.0.smithi162.stdout:2477: done (2 left) 2024-09-16T15:11:37.036 INFO:tasks.rados.rados.0.smithi162.stdout:2478: done (1 left) 2024-09-16T15:11:37.036 INFO:tasks.rados.rados.0.smithi162.stdout:2479: done (0 left) 2024-09-16T15:11:37.036 INFO:tasks.rados.rados.0.smithi162.stdout:2480: delete oid 30 current snap is 241 2024-09-16T15:11:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:2480: done (0 left) 2024-09-16T15:11:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:2481: copy_from oid 29 from oid 45 current snap is 241 2024-09-16T15:11:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:2482: snap_create 2024-09-16T15:11:37.075 INFO:tasks.rados.rados.0.smithi162.stdout:2481: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:11:37.079 INFO:tasks.rados.rados.0.smithi162.stdout:2481: finishing copy_from to smithi16234619-29 2024-09-16T15:11:37.079 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1145 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:11:38.654 INFO:tasks.rados.rados.0.smithi162.stdout:2481: done (1 left) 2024-09-16T15:11:38.654 INFO:tasks.rados.rados.0.smithi162.stdout:2482: done (0 left) 2024-09-16T15:11:38.654 INFO:tasks.rados.rados.0.smithi162.stdout:2483: write (excl) oid 28 current snap is 242 2024-09-16T15:11:38.654 INFO:tasks.rados.rados.0.smithi162.stdout:2483: seq_num 877 ranges {739739=573968,2053414=686325} 2024-09-16T15:11:38.661 INFO:tasks.rados.rados.0.smithi162.stdout:2483: writing smithi16234619-28 from 739739 to 1313707 tid 1 2024-09-16T15:11:38.669 INFO:tasks.rados.rados.0.smithi162.stdout:2483: writing smithi16234619-28 from 2053414 to 2739739 tid 2 2024-09-16T15:11:38.669 INFO:tasks.rados.rados.0.smithi162.stdout:2484: write (excl) oid 18 current snap is 242 2024-09-16T15:11:38.669 INFO:tasks.rados.rados.0.smithi162.stdout:2484: seq_num 878 ranges {756546=649217,2111896=505630} 2024-09-16T15:11:38.676 INFO:tasks.rados.rados.0.smithi162.stdout:2484: writing smithi16234619-18 from 756546 to 1405763 tid 1 2024-09-16T15:11:38.683 INFO:tasks.rados.rados.0.smithi162.stdout:2484: writing smithi16234619-18 from 2111896 to 2617526 tid 2 2024-09-16T15:11:38.683 INFO:tasks.rados.rados.0.smithi162.stdout:2485: copy_from oid 43 from oid 49 current snap is 242 2024-09-16T15:11:38.683 INFO:tasks.rados.rados.0.smithi162.stdout:2486: snap_remove snap 239 2024-09-16T15:11:39.012 INFO:tasks.rados.rados.0.smithi162.stdout:2486: done (3 left) 2024-09-16T15:11:39.012 INFO:tasks.rados.rados.0.smithi162.stdout:2487: write (excl) oid 6 current snap is 242 2024-09-16T15:11:39.013 INFO:tasks.rados.rados.0.smithi162.stdout:2487: seq_num 879 ranges {773353=724466,2170378=602975} 2024-09-16T15:11:39.020 INFO:tasks.rados.rados.0.smithi162.stdout:2487: writing smithi16234619-6 from 773353 to 1497819 tid 1 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2487: writing smithi16234619-6 from 2170378 to 2773353 tid 2 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2488: read oid 4 snap 227 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2488: expect (ObjNum 829 snap 224 seq_num 829) 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2489: snap_create 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: finishing write tid 1 to smithi16234619-28 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: finishing write tid 2 to smithi16234619-28 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: finishing write tid 3 to smithi16234619-28 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: finishing write tid 4 to smithi16234619-28 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: oid 28 updating version 0 to 1123 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: oid 28 updating version 1123 to 1125 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:2483: oid 28 version 1125 is already newer than 1124 2024-09-16T15:11:39.028 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1125 (ObjNum 877 snap 242 seq_num 877) dirty exists 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2483: left oid 28 (ObjNum 877 snap 242 seq_num 877) 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: finishing write tid 1 to smithi16234619-18 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: finishing write tid 2 to smithi16234619-18 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: finishing write tid 3 to smithi16234619-18 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: finishing write tid 4 to smithi16234619-18 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: oid 18 updating version 0 to 902 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: oid 18 updating version 902 to 904 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: oid 18 version 904 is already newer than 903 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 904 (ObjNum 878 snap 242 seq_num 878) dirty exists 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2484: left oid 18 (ObjNum 878 snap 242 seq_num 878) 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2485: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:2485: finishing copy_from to smithi16234619-43 2024-09-16T15:11:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:11:39.036 INFO:tasks.rados.rados.0.smithi162.stdout:2488: expect (ObjNum 829 snap 224 seq_num 829) 2024-09-16T15:11:39.256 INFO:tasks.rados.rados.0.smithi162.stdout:2487: finishing write tid 1 to smithi16234619-6 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: finishing write tid 2 to smithi16234619-6 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: finishing write tid 3 to smithi16234619-6 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: finishing write tid 4 to smithi16234619-6 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: oid 6 updating version 0 to 958 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: oid 6 version 958 is already newer than 956 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: oid 6 version 958 is already newer than 957 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 958 (ObjNum 879 snap 242 seq_num 879) dirty exists 2024-09-16T15:11:39.257 INFO:tasks.rados.rados.0.smithi162.stdout:2487: left oid 6 (ObjNum 879 snap 242 seq_num 879) 2024-09-16T15:11:40.297 INFO:tasks.rados.rados.0.smithi162.stdout:2483: done (5 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2484: done (4 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2485: done (3 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2487: done (2 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2488: done (1 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2489: done (0 left) 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2490: rollback oid 37 current snap is 243 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 241 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2491: rollback oid 34 current snap is 243 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 238 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2492: snap_create 2024-09-16T15:11:40.298 INFO:tasks.rados.rados.0.smithi162.stdout:2491: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:11:40.299 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:11:40.300 INFO:tasks.rados.rados.0.smithi162.stdout:2490: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:11:40.300 INFO:tasks.rados.rados.0.smithi162.stdout:2490: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:11:40.301 INFO:tasks.rados.rados.0.smithi162.stdout:2490: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:11:40.301 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1148 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2490: done (2 left) 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2491: done (1 left) 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2492: done (0 left) 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2493: read oid 45 snap -1 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2493: expect (ObjNum 834 snap 224 seq_num 834) 2024-09-16T15:11:41.652 INFO:tasks.rados.rados.0.smithi162.stdout:2494: snap_create 2024-09-16T15:11:41.656 INFO:tasks.rados.rados.0.smithi162.stdout:2493: expect (ObjNum 834 snap 224 seq_num 834) 2024-09-16T15:11:42.657 INFO:tasks.rados.rados.0.smithi162.stdout:2493: done (1 left) 2024-09-16T15:11:42.657 INFO:tasks.rados.rados.0.smithi162.stdout:2494: done (0 left) 2024-09-16T15:11:42.657 INFO:tasks.rados.rados.0.smithi162.stdout:2495: write (excl) oid 26 current snap is 245 2024-09-16T15:11:42.657 INFO:tasks.rados.rados.0.smithi162.stdout:2495: seq_num 880 ranges {790160=799715,2112507=509299} 2024-09-16T15:11:42.665 INFO:tasks.rados.rados.0.smithi162.stdout:2495: writing smithi16234619-26 from 790160 to 1589875 tid 1 2024-09-16T15:11:42.671 INFO:tasks.rados.rados.0.smithi162.stdout:2495: writing smithi16234619-26 from 2112507 to 2621806 tid 2 2024-09-16T15:11:42.671 INFO:tasks.rados.rados.0.smithi162.stdout:2496: rollback oid 13 current snap is 245 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 241 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2497: read oid 47 snap -1 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2497: expect (ObjNum 832 snap 224 seq_num 832) 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2498: read oid 19 snap -1 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2498: expect (ObjNum 850 snap 226 seq_num 850) 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2499: write (excl) oid 45 current snap is 245 2024-09-16T15:11:42.672 INFO:tasks.rados.rados.0.smithi162.stdout:2499: seq_num 881 ranges {406967=474964,1370989=769310,2677186=129781} 2024-09-16T15:11:42.676 INFO:tasks.rados.rados.0.smithi162.stdout:2499: writing smithi16234619-45 from 406967 to 881931 tid 1 2024-09-16T15:11:42.684 INFO:tasks.rados.rados.0.smithi162.stdout:2499: writing smithi16234619-45 from 1370989 to 2140299 tid 2 2024-09-16T15:11:42.688 INFO:tasks.rados.rados.0.smithi162.stdout:2499: writing smithi16234619-45 from 2677186 to 2806967 tid 3 2024-09-16T15:11:42.689 INFO:tasks.rados.rados.0.smithi162.stdout:2500: snap_create 2024-09-16T15:11:42.689 INFO:tasks.rados.rados.0.smithi162.stdout:2496: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:11:42.689 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 0 (ObjNum 193 snap 0 seq_num 864627152) dirty dne 2024-09-16T15:11:42.689 INFO:tasks.rados.rados.0.smithi162.stdout:2497: expect (ObjNum 832 snap 224 seq_num 832) 2024-09-16T15:11:42.820 INFO:tasks.rados.rados.0.smithi162.stdout:2498: expect (ObjNum 850 snap 226 seq_num 850) 2024-09-16T15:11:42.893 INFO:tasks.rados.rados.0.smithi162.stdout:2495: finishing write tid 1 to smithi16234619-26 2024-09-16T15:11:42.893 INFO:tasks.rados.rados.0.smithi162.stdout:2495: finishing write tid 2 to smithi16234619-26 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: finishing write tid 3 to smithi16234619-26 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: finishing write tid 1 to smithi16234619-45 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: finishing write tid 4 to smithi16234619-26 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: oid 26 updating version 0 to 1159 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: oid 26 updating version 1159 to 1161 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: oid 26 version 1161 is already newer than 1160 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1161 (ObjNum 880 snap 245 seq_num 880) dirty exists 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2495: left oid 26 (ObjNum 880 snap 245 seq_num 880) 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: finishing write tid 2 to smithi16234619-45 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: finishing write tid 3 to smithi16234619-45 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: finishing write tid 4 to smithi16234619-45 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: finishing write tid 5 to smithi16234619-45 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: oid 45 updating version 0 to 1165 2024-09-16T15:11:42.894 INFO:tasks.rados.rados.0.smithi162.stdout:2499: oid 45 version 1165 is already newer than 1163 2024-09-16T15:11:42.895 INFO:tasks.rados.rados.0.smithi162.stdout:2499: oid 45 updating version 1165 to 1166 2024-09-16T15:11:42.895 INFO:tasks.rados.rados.0.smithi162.stdout:2499: oid 45 version 1166 is already newer than 1164 2024-09-16T15:11:42.895 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1166 (ObjNum 881 snap 245 seq_num 881) dirty exists 2024-09-16T15:11:42.895 INFO:tasks.rados.rados.0.smithi162.stdout:2499: left oid 45 (ObjNum 881 snap 245 seq_num 881) 2024-09-16T15:11:43.661 INFO:tasks.rados.rados.0.smithi162.stdout:2495: done (5 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2496: done (4 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2497: done (3 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2498: done (2 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2499: done (1 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2500: done (0 left) 2024-09-16T15:11:43.662 INFO:tasks.rados.rados.0.smithi162.stdout:2501: snap_remove snap 245 2024-09-16T15:11:44.021 INFO:tasks.rados.rados.0.smithi162.stdout:2501: done (0 left) 2024-09-16T15:11:44.021 INFO:tasks.rados.rados.0.smithi162.stdout:2502: rollback oid 26 current snap is 246 2024-09-16T15:11:44.021 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 232 2024-09-16T15:11:44.021 INFO:tasks.rados.rados.0.smithi162.stdout:2503: write (excl) oid 28 current snap is 246 2024-09-16T15:11:44.021 INFO:tasks.rados.rados.0.smithi162.stdout:2503: seq_num 882 ranges {423774=666566,1545824=512968,2704609=119165} 2024-09-16T15:11:44.027 INFO:tasks.rados.rados.0.smithi162.stdout:2503: writing smithi16234619-28 from 423774 to 1090340 tid 1 2024-09-16T15:11:44.033 INFO:tasks.rados.rados.0.smithi162.stdout:2503: writing smithi16234619-28 from 1545824 to 2058792 tid 2 2024-09-16T15:11:44.037 INFO:tasks.rados.rados.0.smithi162.stdout:2503: writing smithi16234619-28 from 2704609 to 2823774 tid 3 2024-09-16T15:11:44.037 INFO:tasks.rados.rados.0.smithi162.stdout:2504: write oid 29 current snap is 246 2024-09-16T15:11:44.037 INFO:tasks.rados.rados.0.smithi162.stdout:2502: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:11:44.037 INFO:tasks.rados.rados.0.smithi162.stdout:2504: seq_num 883 ranges {440581=741815,1604306=656626,2732032=108549} 2024-09-16T15:11:44.044 INFO:tasks.rados.rados.0.smithi162.stdout:2504: writing smithi16234619-29 from 440581 to 1182396 tid 1 2024-09-16T15:11:44.050 INFO:tasks.rados.rados.0.smithi162.stdout:2504: writing smithi16234619-29 from 1604306 to 2260932 tid 2 2024-09-16T15:11:44.053 INFO:tasks.rados.rados.0.smithi162.stdout:2504: writing smithi16234619-29 from 2732032 to 2840581 tid 3 2024-09-16T15:11:44.053 INFO:tasks.rados.rados.0.smithi162.stdout:2505: rollback oid 5 current snap is 246 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 235 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:2502: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:2502: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1169 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:2502: done (3 left) 2024-09-16T15:11:44.054 INFO:tasks.rados.rados.0.smithi162.stdout:2506: snap_remove snap 244 2024-09-16T15:11:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:2506: done (3 left) 2024-09-16T15:11:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:2507: read oid 36 snap 242 2024-09-16T15:11:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:2507: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:45.571 INFO:tasks.rados.rados.0.smithi162.stdout:2508: snap_remove snap 243 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2508: done (4 left) 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2509: delete oid 34 current snap is 246 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: finishing write tid 1 to smithi16234619-28 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: finishing write tid 2 to smithi16234619-28 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: finishing write tid 3 to smithi16234619-28 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: finishing write tid 4 to smithi16234619-28 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: finishing write tid 5 to smithi16234619-28 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: oid 28 updating version 0 to 1128 2024-09-16T15:11:46.570 INFO:tasks.rados.rados.0.smithi162.stdout:2503: oid 28 version 1128 is already newer than 1126 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2503: oid 28 updating version 1128 to 1129 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2503: oid 28 version 1129 is already newer than 1127 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1129 (ObjNum 882 snap 246 seq_num 882) dirty exists 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2503: left oid 28 (ObjNum 882 snap 246 seq_num 882) 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: finishing write tid 1 to smithi16234619-29 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: finishing write tid 2 to smithi16234619-29 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: finishing write tid 3 to smithi16234619-29 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: finishing write tid 4 to smithi16234619-29 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: finishing write tid 5 to smithi16234619-29 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: oid 29 updating version 0 to 1151 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: oid 29 updating version 1151 to 1152 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: oid 29 version 1152 is already newer than 1150 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: oid 29 version 1152 is already newer than 1149 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1152 (ObjNum 883 snap 246 seq_num 883) dirty exists 2024-09-16T15:11:46.571 INFO:tasks.rados.rados.0.smithi162.stdout:2504: left oid 29 (ObjNum 883 snap 246 seq_num 883) 2024-09-16T15:11:46.572 INFO:tasks.rados.rados.0.smithi162.stdout:2505: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:11:46.572 INFO:tasks.rados.rados.0.smithi162.stdout:2505: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:11:46.572 INFO:tasks.rados.rados.0.smithi162.stdout:2505: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:11:46.572 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1155 (ObjNum 817 snap 221 seq_num 817) dirty exists 2024-09-16T15:11:46.572 INFO:tasks.rados.rados.0.smithi162.stdout:2507: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2503: done (4 left) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2504: done (3 left) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2505: done (2 left) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2507: done (1 left) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2509: done (0 left) 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2510: write oid 35 current snap is 246 2024-09-16T15:11:46.738 INFO:tasks.rados.rados.0.smithi162.stdout:2510: seq_num 885 ranges {474195=492313,1604917=660295,2670525=203670} 2024-09-16T15:11:46.743 INFO:tasks.rados.rados.0.smithi162.stdout:2510: writing smithi16234619-35 from 474195 to 966508 tid 1 2024-09-16T15:11:46.750 INFO:tasks.rados.rados.0.smithi162.stdout:2510: writing smithi16234619-35 from 1604917 to 2265212 tid 2 2024-09-16T15:11:46.753 INFO:tasks.rados.rados.0.smithi162.stdout:2510: writing smithi16234619-35 from 2670525 to 2874195 tid 3 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:2511: rollback oid 23 current snap is 246 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 241 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:2512: rollback oid 26 current snap is 246 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 230 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:2513: rollback oid 39 current snap is 246 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 234 2024-09-16T15:11:46.754 INFO:tasks.rados.rados.0.smithi162.stdout:2514: delete oid 18 current snap is 246 2024-09-16T15:11:46.758 INFO:tasks.rados.rados.0.smithi162.stdout:2514: done (4 left) 2024-09-16T15:11:46.758 INFO:tasks.rados.rados.0.smithi162.stdout:2515: rollback oid 28 current snap is 246 2024-09-16T15:11:46.758 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 241 2024-09-16T15:11:46.758 INFO:tasks.rados.rados.0.smithi162.stdout:2516: snap_remove snap 228 2024-09-16T15:11:47.679 INFO:tasks.rados.rados.0.smithi162.stdout:2516: done (5 left) 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2517: rollback oid 34 current snap is 246 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 230 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2512: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2513: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2512: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2513: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2512: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1180 (ObjNum 713 snap 185 seq_num 713) dirty exists 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2513: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1158 (ObjNum 623 snap 157 seq_num 623) dirty exists 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2511: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:11:47.680 INFO:tasks.rados.rados.0.smithi162.stdout:2511: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2515: finishing rollback tid 0 to smithi16234619-28 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2511: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 596 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2515: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1131 (ObjNum 193 snap 0 seq_num 864627152) dirty dne 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: finishing write tid 1 to smithi16234619-35 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: finishing write tid 2 to smithi16234619-35 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: finishing write tid 3 to smithi16234619-35 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: finishing write tid 4 to smithi16234619-35 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: finishing write tid 5 to smithi16234619-35 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: oid 35 updating version 0 to 961 2024-09-16T15:11:47.681 INFO:tasks.rados.rados.0.smithi162.stdout:2510: oid 35 updating version 961 to 962 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2510: oid 35 version 962 is already newer than 959 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2510: oid 35 version 962 is already newer than 960 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 962 (ObjNum 885 snap 246 seq_num 885) dirty exists 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2510: left oid 35 (ObjNum 885 snap 246 seq_num 885) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2510: done (5 left) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2511: done (4 left) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2512: done (3 left) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2513: done (2 left) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2515: done (1 left) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2518: rollback oid 15 current snap is 246 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 15 to 232 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2519: read oid 35 snap -1 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2519: expect (ObjNum 885 snap 246 seq_num 885) 2024-09-16T15:11:47.682 INFO:tasks.rados.rados.0.smithi162.stdout:2520: copy_from oid 44 from oid 1 current snap is 246 2024-09-16T15:11:47.683 INFO:tasks.rados.rados.0.smithi162.stdout:2521: snap_remove snap 237 2024-09-16T15:11:47.826 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-16T15:11:47.826 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-16T15:11:47.826 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-16T15:11:47.826 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-16T15:11:47.829 INFO:tasks.ceph.mon.g:Started 2024-09-16T15:11:47.829 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-16T15:11:47.830 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-16T15:11:47.830 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-16T15:11:47.830 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-16T15:11:47.833 INFO:tasks.ceph.mon.e:Started 2024-09-16T15:11:47.833 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:11:47.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.941+0000 7fcdafa2b640 10 monclient: get_monmap_and_config 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.941+0000 7fcdafa2b640 10 monclient: build_initial_monmap 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.941+0000 7fcdafa2b640 10 monclient: monmap: 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:47.942610+0000 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:47.942610+0000 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:47.943 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:47.944 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 Processor -- start 2024-09-16T15:11:47.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 -- start start 2024-09-16T15:11:47.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient: init 2024-09-16T15:11:47.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient: _add_conns ranks=[3,5,1,10,2,0,8,4,9,6,7] 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-g con 0x7fcda8136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 0x7fcda809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-h con 0x7fcda809bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fcda8136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda8073250 con 0x7fcda8136c60 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda809b600 con 0x7fcda8136030 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:47.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80a5740 con 0x7fcda809bfe0 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.942+0000 7fcdafa2b640 10 monclient(hunting): authenticate will time out at 1444.531977s 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 0x7fcda809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdadfc8640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fcda8136c60 0x7fcda81370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:35368/0 (socket says 172.21.15.134:35368) 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:40992/0 (socket says 172.21.15.134:40992) 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdadfc8640 1 -- 172.21.15.134:0/1558626232 learned_addr learned my addr 172.21.15.134:0/1558626232 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 10 monclient(hunting): get_auth_request con 0x7fcda8136030 auth_method 0 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:47.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): get_auth_request con 0x7fcda809bfe0 auth_method 0 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 -- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 msgr2=0x7fcda8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 -- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 msgr2=0x7fcda8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_done global_id 44690 payload 274 2024-09-16T15:11:47.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 -- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 msgr2=0x7fcda8136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 -- 172.21.15.134:0/1558626232 >> v1:172.21.15.162:6789/0 conn(0x7fcda8136c60 legacy=0x7fcda81370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient: _finish_hunting 0 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 monclient: found mon.noname-h 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.943+0000 7fcdacfc6640 1 -- 172.21.15.134:0/1558626232 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda80a52c0 con 0x7fcda809bfe0 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.946+0000 7fcdacfc6640 1 -- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 msgr2=0x7fcda809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:47.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.946+0000 7fcdacfc6640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 0x7fcda809c400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097d0 tx=0x7fcd9802a000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=b3cf1c40f556336b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:47.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 1 -- 172.21.15.134:0/1558626232 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9802ad20 con 0x7fcda809bfe0 2024-09-16T15:11:47.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: dump: 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:47.958 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: _finish_auth 0 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 1 -- 172.21.15.134:0/1558626232 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd980307b0 con 0x7fcda809bfe0 2024-09-16T15:11:47.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 1 -- 172.21.15.134:0/1558626232 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd9802aea0 con 0x7fcda809bfe0 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcd9ffff640 10 monclient: dump: 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.960 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcdafa2b640 5 monclient: authenticate success, global_id 44690 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcdafa2b640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.948+0000 7fcdafa2b640 4 set_mon_vals no callback set 2024-09-16T15:11:47.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 10 monclient: shutdown 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 -- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 msgr2=0x7fcda809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 0x7fcda809c400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097d0 tx=0x7fcd9802a000 comp rx=0 tx=0).stop 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 -- 172.21.15.134:0/1558626232 shutdown_connections 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcda809bfe0 0x7fcda809c400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/1558626232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda8136030 0x7fcda8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 -- 172.21.15.134:0/1558626232 >> 172.21.15.134:0/1558626232 conn(0x7fcda80a9ea0 msgr2=0x7fcda80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 -- 172.21.15.134:0/1558626232 shutdown_connections 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.949+0000 7fcdafa2b640 1 -- 172.21.15.134:0/1558626232 wait complete. 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient: build_initial_monmap 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient: monmap: 2024-09-16T15:11:47.962 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:47.963 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 Processor -- start 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 -- start start 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient: init 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient: _add_conns ranks=[1,2,0,4,6,5,3,7,8] 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 0x7fcda8134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-f con 0x7fcda809bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 0x7fcda81352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-b con 0x7fcda8136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): picked mon.noname-a con 0x7fcda8136c60 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80a8390 con 0x7fcda8136c60 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda8121eb0 con 0x7fcda809bfe0 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80a7e60 con 0x7fcda8136030 2024-09-16T15:11:47.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdafa2b640 10 monclient(hunting): authenticate will time out at 1444.539871s 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdadfc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.950+0000 7fcdacfc6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 0x7fcda81352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37406/0 (socket says 172.21.15.134:37406) 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 1 -- 172.21.15.134:0/3652397166 learned_addr learned my addr 172.21.15.134:0/3652397166 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): get_auth_request con 0x7fcda8136c60 auth_method 0 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): get_auth_request con 0x7fcda8136030 auth_method 0 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:47.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 0x7fcda8134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 10 monclient(hunting): get_auth_request con 0x7fcda809bfe0 auth_method 0 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:47.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient(hunting): handle_auth_done global_id 44912 payload 824 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 -- 172.21.15.134:0/3652397166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 msgr2=0x7fcda8134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 0x7fcda8134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 -- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 msgr2=0x7fcda811fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient: _finish_hunting 0 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 monclient: found mon.noname-b 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:11:47.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdacfc6640 1 -- 172.21.15.134:0/3652397166 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98009400 con 0x7fcda8136030 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdadfc8640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.951+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 0x7fcda8134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdacfc6640 1 -- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 msgr2=0x7fcda81352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdacfc6640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 0x7fcda81352d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcda80a5880 tx=0x7fcd980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd98030cf0 con 0x7fcda8136030 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: dump: 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:47.970 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: _finish_auth 0 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdafa2b640 5 monclient: authenticate success, global_id 44912 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98004250 con 0x7fcda8136030 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdafa2b640 10 monclient: _renew_subs 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdafa2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda8120f50 con 0x7fcda8136030 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd980043c0 con 0x7fcda8136030 2024-09-16T15:11:47.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9dffb640 10 monclient: dump: 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:47.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.953+0000 7fcd9f7fe640 4 set_mon_vals no callback set 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 10 monclient: _renew_subs 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda81cb7c0 con 0x7fcda8136030 2024-09-16T15:11:47.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcd98003610 con 0x7fcda8136030 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.954+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda8122020 con 0x7fcda8136030 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.955+0000 7fcd9dffb640 1 --2- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 0x7fcd8c03ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.955+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(510..510 src has 1..510) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fcd9806abb0 con 0x7fcda8136030 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.955+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 0x7fcd8c03ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:47.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.955+0000 7fcdad7c7640 10 monclient: get_auth_request con 0x7fcd8c0388a0 auth_method 0 2024-09-16T15:11:47.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.958+0000 7fcdad7c7640 1 --2- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 0x7fcd8c03ad60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006fd0 tx=0x7fcda4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:47.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.959+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fcd980467c0 con 0x7fcda8136030 2024-09-16T15:11:47.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.959+0000 7fcd9dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:47.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:47.959+0000 7fcd9dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:48.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:48.195+0000 7fcdafa2b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:11:48.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:48.195+0000 7fcdafa2b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:11:48.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:48.195+0000 7fcdafa2b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:48.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:48.195+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcda809c400 con 0x7fcda8136030 2024-09-16T15:11:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:50.950+0000 7fcd9cff9640 10 monclient: tick 2024-09-16T15:11:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:50.950+0000 7fcd9cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:11:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:50.950+0000 7fcd9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:50.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:50.950+0000 7fcd9cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:51.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:51.951+0000 7fcd9cff9640 10 monclient: tick 2024-09-16T15:11:51.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:51.951+0000 7fcd9cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:11:51.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:51.951+0000 7fcd9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:52.951+0000 7fcd9cff9640 10 monclient: tick 2024-09-16T15:11:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:52.951+0000 7fcd9cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:11:52.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:52.951+0000 7fcd9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.183+0000 7fcd9dffb640 1 -- 172.21.15.134:0/3652397166 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3189 (secure 0 0 0) 0x7fcd98037070 con 0x7fcda8136030 2024-09-16T15:11:53.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.183+0000 7fcd9dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:11:53.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.183+0000 7fcd9dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:53.185 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":94,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:11:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 msgr2=0x7fcd8c03ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 0x7fcd8c03ad60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006fd0 tx=0x7fcda4008040 comp rx=0 tx=0).stop 2024-09-16T15:11:53.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 10 monclient: shutdown 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 msgr2=0x7fcda81352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.186+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 0x7fcda81352d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcda80a5880 tx=0x7fcd980046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 shutdown_connections 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> v2:172.21.15.162:6800/980547263 conn(0x7fcd8c0388a0 0x7fcd8c03ad60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda8136c60 0x7fcda811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda8136030 0x7fcda81352d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 --2- 172.21.15.134:0/3652397166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda809bfe0 0x7fcda8134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 >> 172.21.15.134:0/3652397166 conn(0x7fcda80a9ea0 msgr2=0x7fcda806bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 shutdown_connections 2024-09-16T15:11:53.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:53.187+0000 7fcdafa2b640 1 -- 172.21.15.134:0/3652397166 wait complete. 2024-09-16T15:11:53.459 INFO:tasks.rados.rados.0.smithi162.stdout:2521: done (4 left) 2024-09-16T15:11:53.459 INFO:tasks.rados.rados.0.smithi162.stdout:2522: read oid 9 snap 238 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:2522: expect deleted 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:2523: delete oid 22 current snap is 246 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:2518: finishing rollback tid 1 to smithi16234619-15 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:2517: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:11:53.460 INFO:tasks.rados.rados.0.smithi162.stdout:2519: expect (ObjNum 885 snap 246 seq_num 885) 2024-09-16T15:11:53.586 INFO:tasks.rados.rados.0.smithi162.stdout:2520: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2520: finishing copy_from to smithi16234619-44 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 906 (ObjNum 859 snap 227 seq_num 859) dirty exists 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2517: done (5 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2518: done (4 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2519: done (3 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2520: done (2 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2522: done (1 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2523: done (0 left) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2524: read oid 25 snap -1 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2524: expect (ObjNum 829 snap 224 seq_num 829) 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2525: write oid 39 current snap is 246 2024-09-16T15:11:53.587 INFO:tasks.rados.rados.0.smithi162.stdout:2525: seq_num 888 ranges {524616=718060,1664010=523975,2752794=171822} 2024-09-16T15:11:53.594 INFO:tasks.rados.rados.0.smithi162.stdout:2525: writing smithi16234619-39 from 524616 to 1242676 tid 1 2024-09-16T15:11:53.600 INFO:tasks.rados.rados.0.smithi162.stdout:2525: writing smithi16234619-39 from 1664010 to 2187985 tid 2 2024-09-16T15:11:53.604 INFO:tasks.rados.rados.0.smithi162.stdout:2525: writing smithi16234619-39 from 2752794 to 2924616 tid 3 2024-09-16T15:11:53.604 INFO:tasks.rados.rados.0.smithi162.stdout:2526: rollback oid 21 current snap is 246 2024-09-16T15:11:53.604 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 238 2024-09-16T15:11:53.604 INFO:tasks.rados.rados.0.smithi162.stdout:2524: expect (ObjNum 829 snap 224 seq_num 829) 2024-09-16T15:11:53.818 INFO:tasks.rados.rados.0.smithi162.stdout:2526: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:11:53.819 INFO:tasks.rados.rados.0.smithi162.stdout:2526: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:11:53.819 INFO:tasks.rados.rados.0.smithi162.stdout:2524: done (2 left) 2024-09-16T15:11:53.819 INFO:tasks.rados.rados.0.smithi162.stdout:2527: snap_remove snap 240 2024-09-16T15:11:54.299 INFO:tasks.rados.rados.0.smithi162.stdout:2527: done (2 left) 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2528: delete oid 38 current snap is 246 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2526: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1194 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: finishing write tid 1 to smithi16234619-39 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: finishing write tid 2 to smithi16234619-39 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: finishing write tid 3 to smithi16234619-39 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: finishing write tid 4 to smithi16234619-39 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: finishing write tid 5 to smithi16234619-39 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: oid 39 updating version 0 to 1171 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: oid 39 updating version 1171 to 1174 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: oid 39 version 1174 is already newer than 1172 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:2525: oid 39 version 1174 is already newer than 1173 2024-09-16T15:11:54.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1174 (ObjNum 888 snap 246 seq_num 888) dirty exists 2024-09-16T15:11:54.301 INFO:tasks.rados.rados.0.smithi162.stdout:2525: left oid 39 (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:11:54.306 INFO:tasks.rados.rados.0.smithi162.stdout:2525: done (2 left) 2024-09-16T15:11:54.306 INFO:tasks.rados.rados.0.smithi162.stdout:2526: done (1 left) 2024-09-16T15:11:54.306 INFO:tasks.rados.rados.0.smithi162.stdout:2528: done (0 left) 2024-09-16T15:11:54.306 INFO:tasks.rados.rados.0.smithi162.stdout:2529: delete oid 1 current snap is 246 2024-09-16T15:11:54.310 INFO:tasks.rados.rados.0.smithi162.stdout:2529: done (0 left) 2024-09-16T15:11:54.310 INFO:tasks.rados.rados.0.smithi162.stdout:2530: copy_from oid 40 from oid 33 current snap is 246 2024-09-16T15:11:54.310 INFO:tasks.rados.rados.0.smithi162.stdout:2531: write (excl) oid 41 current snap is 246 2024-09-16T15:11:54.310 INFO:tasks.rados.rados.0.smithi162.stdout:2531: seq_num 891 ranges {575037=660160,1955809=671302} 2024-09-16T15:11:54.317 INFO:tasks.rados.rados.0.smithi162.stdout:2531: writing smithi16234619-41 from 575037 to 1235197 tid 1 2024-09-16T15:11:54.325 INFO:tasks.rados.rados.0.smithi162.stdout:2531: writing smithi16234619-41 from 1955809 to 2627111 tid 2 2024-09-16T15:11:54.325 INFO:tasks.rados.rados.0.smithi162.stdout:2532: write (excl) oid 27 current snap is 246 2024-09-16T15:11:54.325 INFO:tasks.rados.rados.0.smithi162.stdout:2532: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2024-09-16T15:11:54.332 INFO:tasks.rados.rados.0.smithi162.stdout:2532: writing smithi16234619-27 from 591844 to 1327253 tid 1 2024-09-16T15:11:54.338 INFO:tasks.rados.rados.0.smithi162.stdout:2532: writing smithi16234619-27 from 1897938 to 2429251 tid 2 2024-09-16T15:11:54.341 INFO:tasks.rados.rados.0.smithi162.stdout:2532: writing smithi16234619-27 from 2862486 to 2991844 tid 3 2024-09-16T15:11:54.341 INFO:tasks.rados.rados.0.smithi162.stdout:2533: snap_remove snap 232 2024-09-16T15:11:55.679 INFO:tasks.rados.rados.0.smithi162.stdout:2533: done (3 left) 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2534: snap_create 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: finishing write tid 1 to smithi16234619-41 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: finishing write tid 2 to smithi16234619-41 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: finishing write tid 3 to smithi16234619-41 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: finishing write tid 4 to smithi16234619-41 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: oid 41 updating version 0 to 805 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: oid 41 updating version 805 to 807 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: oid 41 version 807 is already newer than 806 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 807 (ObjNum 891 snap 246 seq_num 891) dirty exists 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2531: left oid 41 (ObjNum 891 snap 246 seq_num 891) 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2530: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2530: finishing copy_from to smithi16234619-40 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 166 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:11:55.680 INFO:tasks.rados.rados.0.smithi162.stdout:2532: finishing write tid 1 to smithi16234619-27 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: finishing write tid 2 to smithi16234619-27 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: finishing write tid 3 to smithi16234619-27 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: finishing write tid 4 to smithi16234619-27 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: finishing write tid 5 to smithi16234619-27 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: oid 27 updating version 0 to 913 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: oid 27 version 913 is already newer than 912 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: oid 27 updating version 913 to 914 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: oid 27 version 914 is already newer than 911 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 914 (ObjNum 892 snap 246 seq_num 892) dirty exists 2024-09-16T15:11:55.681 INFO:tasks.rados.rados.0.smithi162.stdout:2532: left oid 27 (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:11:56.202 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:11:56.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 10 monclient: get_monmap_and_config 2024-09-16T15:11:56.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 10 monclient: monmap: 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:56.304099+0000 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:56.304099+0000 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.305 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.306 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.306 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.306 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:56.306 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:56.306 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 1 Processor -- start 2024-09-16T15:11:56.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 1 -- start start 2024-09-16T15:11:56.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.303+0000 7fd7883db640 10 monclient: init 2024-09-16T15:11:56.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient: _add_conns ranks=[6,5,9,10,7,1,4,3,0,2,8] 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-d con 0x7fd780136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd78009bfe0 0x7fd78009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-h con 0x7fd78009bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 0x7fd7801370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-a con 0x7fd780136c60 addr v2:172.21.15.134:3300/0 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78009b600 con 0x7fd780136c60 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800a5740 con 0x7fd78009bfe0 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800a52c0 con 0x7fd780136030 2024-09-16T15:11:56.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd7883db640 10 monclient(hunting): authenticate will time out at 1452.893459s 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51950/0 (socket says 172.21.15.134:51950) 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 -- 172.21.15.134:0/727051046 learned_addr learned my addr 172.21.15.134:0/727051046 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): get_auth_request con 0x7fd780136030 auth_method 0 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 1 --2- 172.21.15.134:0/727051046 >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 0x7fd7801370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): get_auth_request con 0x7fd780136c60 auth_method 0 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd785976640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd78009bfe0 0x7fd78009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd785976640 10 monclient(hunting): get_auth_request con 0x7fd78009bfe0 auth_method 0 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd785976640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd785976640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd785976640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient(hunting): handle_auth_done global_id 35007 payload 274 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 -- 172.21.15.134:0/727051046 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd78009bfe0 msgr2=0x7fd78009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd78009bfe0 0x7fd78009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 -- 172.21.15.134:0/727051046 >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 msgr2=0x7fd7801370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 --2- 172.21.15.134:0/727051046 >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 0x7fd7801370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 monclient: found mon.noname-d 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786177640 1 -- 172.21.15.134:0/727051046 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780073250 con 0x7fd780136030 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.304+0000 7fd786978640 1 --2- 172.21.15.134:0/727051046 >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 0x7fd7801370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd786177640 1 -- 172.21.15.134:0/727051046 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 msgr2=0x7fd780136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd786177640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd778009870 tx=0x7fd77802a0a0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=5ae89c3b4f30ff8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 1 -- 172.21.15.134:0/727051046 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd77802ae00 con 0x7fd780136030 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: dump: 2024-09-16T15:11:56.314 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.315 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd7883db640 5 monclient: authenticate success, global_id 35007 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 1 -- 172.21.15.134:0/727051046 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7780307b0 con 0x7fd780136030 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd7883db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd7883db640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd7883db640 4 set_mon_vals no callback set 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 1 -- 172.21.15.134:0/727051046 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd778030d90 con 0x7fd780136030 2024-09-16T15:11:56.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.307+0000 7fd785175640 10 monclient: dump: 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 10 monclient: shutdown 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 -- 172.21.15.134:0/727051046 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 msgr2=0x7fd780136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd778009870 tx=0x7fd77802a0a0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 -- 172.21.15.134:0/727051046 shutdown_connections 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd78009bfe0 0x7fd78009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 --2- 172.21.15.134:0/727051046 >> v2:172.21.15.134:3300/0 conn(0x7fd780136c60 0x7fd7801370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 --2- 172.21.15.134:0/727051046 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd780136030 0x7fd780136410 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 -- 172.21.15.134:0/727051046 >> 172.21.15.134:0/727051046 conn(0x7fd7800a9ea0 msgr2=0x7fd7800a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 -- 172.21.15.134:0/727051046 shutdown_connections 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.308+0000 7fd7883db640 1 -- 172.21.15.134:0/727051046 wait complete. 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient: monmap: 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:56.319 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 Processor -- start 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 -- start start 2024-09-16T15:11:56.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient: init 2024-09-16T15:11:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient: _add_conns ranks=[6,2,0,4,8,5,7,3,1] 2024-09-16T15:11:56.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-d con 0x7fd78009bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd780136030 0x7fd78011fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-b con 0x7fd780136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 0x7fd780134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): picked mon.noname-a con 0x7fd780136c60 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780120ca0 con 0x7fd780136c60 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801220d0 con 0x7fd780136030 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd780121df0 con 0x7fd78009bfe0 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd7883db640 10 monclient(hunting): authenticate will time out at 1452.898886s 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.309+0000 7fd786177640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 0x7fd780134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51962/0 (socket says 172.21.15.134:51962) 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 -- 172.21.15.134:0/2055065702 learned_addr learned my addr 172.21.15.134:0/2055065702 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd780136030 0x7fd78011fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): get_auth_request con 0x7fd78009bfe0 auth_method 0 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): get_auth_request con 0x7fd780136c60 auth_method 0 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): get_auth_request con 0x7fd780136030 auth_method 0 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd785976640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient(hunting): handle_auth_done global_id 35016 payload 824 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 -- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd780136030 msgr2=0x7fd78011fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd780136030 0x7fd78011fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 -- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 msgr2=0x7fd780134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 0x7fd780134cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 monclient: found mon.noname-d 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786177640 1 -- 172.21.15.134:0/2055065702 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009a70 con 0x7fd78009bfe0 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.310+0000 7fd786978640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 0x7fd780134cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd786177640 1 -- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 msgr2=0x7fd7801beb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd786177640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd7780090b0 tx=0x7fd7780046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd778003610 con 0x7fd78009bfe0 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:56.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: dump: 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.329 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7780037b0 con 0x7fd78009bfe0 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 5 monclient: authenticate success, global_id 35016 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 10 monclient: _renew_subs 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7780094d0 con 0x7fd78009bfe0 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd778003d90 con 0x7fd78009bfe0 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd776ffd640 10 monclient: dump: 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.330 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd784974640 4 set_mon_vals no callback set 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 10 monclient: _renew_subs 2024-09-16T15:11:56.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.312+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd780122690 con 0x7fd78009bfe0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.313+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd778039070 con 0x7fd78009bfe0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.313+0000 7fd7883db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.313+0000 7fd7883db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.313+0000 7fd7883db640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.313+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78009c400 con 0x7fd78009bfe0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.314+0000 7fd776ffd640 1 --2- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 0x7fd76403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.314+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fd77806aae0 con 0x7fd78009bfe0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.314+0000 7fd785976640 1 --2- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 0x7fd76403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.314+0000 7fd785976640 10 monclient: get_auth_request con 0x7fd764038880 auth_method 0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.316+0000 7fd785976640 1 --2- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 0x7fd76403ad40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7801bf2a0 tx=0x7fd770008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.318+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd77806ce60 con 0x7fd78009bfe0 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.318+0000 7fd776ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.318+0000 7fd776ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.543+0000 7fd7883db640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.543+0000 7fd7883db640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.543+0000 7fd7883db640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.543+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd780072070 con 0x7fd78009bfe0 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.544+0000 7fd776ffd640 1 -- 172.21.15.134:0/2055065702 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7fd778041070 con 0x7fd78009bfe0 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.544+0000 7fd776ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:11:56.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.544+0000 7fd776ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:56.546 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 msgr2=0x7fd76403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 0x7fd76403ad40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd7801bf2a0 tx=0x7fd770008040 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 10 monclient: shutdown 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 msgr2=0x7fd7801beb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd7780090b0 tx=0x7fd7780046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 shutdown_connections 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> v2:172.21.15.162:6800/980547263 conn(0x7fd764038880 0x7fd76403ad40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd780136c60 0x7fd780134cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd780136030 0x7fd78011fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 --2- 172.21.15.134:0/2055065702 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd78009bfe0 0x7fd7801beb50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 >> 172.21.15.134:0/2055065702 conn(0x7fd7800a9ea0 msgr2=0x7fd78006bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 shutdown_connections 2024-09-16T15:11:56.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.546+0000 7fd7883db640 1 -- 172.21.15.134:0/2055065702 wait complete. 2024-09-16T15:11:56.558 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:11:56.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:11:56.558 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.654+0000 7fd2293b5640 10 monclient: get_monmap_and_config 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.654+0000 7fd2293b5640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.654+0000 7fd2293b5640 10 monclient: monmap: 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:56.655710+0000 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:56.655710+0000 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:56.657 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 Processor -- start 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 -- start start 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient: init 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient: _add_conns ranks=[6,0,3,8,10,7,5,2,1,4,9] 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 0x7fd22409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-d con 0x7fd224136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd224137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd22413c370 0x7fd22413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-g con 0x7fd22413c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd224073250 con 0x7fd224137220 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240a56e0 con 0x7fd22413c370 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd22409abc0 con 0x7fd224136e40 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2293b5640 10 monclient(hunting): authenticate will time out at 1453.244851s 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2227fc640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd224137220 0x7fd224137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50784/0 (socket says 172.21.15.134:50784) 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd222ffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 0x7fd22409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd2227fc640 1 -- 172.21.15.134:0/2788855532 learned_addr learned my addr 172.21.15.134:0/2788855532 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd222ffd640 10 monclient(hunting): get_auth_request con 0x7fd224136e40 auth_method 0 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd222ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd222ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.655+0000 7fd222ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd2237fe640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd22413c370 0x7fd22413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd2237fe640 10 monclient(hunting): get_auth_request con 0x7fd22413c370 auth_method 0 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd2237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd2237fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd2237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient(hunting): handle_auth_done global_id 35025 payload 274 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 1 -- 172.21.15.134:0/2788855532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd22413c370 msgr2=0x7fd22413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd22413c370 0x7fd22413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 1 -- 172.21.15.134:0/2788855532 >> v1:172.21.15.134:6789/0 conn(0x7fd224137220 legacy=0x7fd224137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 1 monclient: found mon.noname-d 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:56.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.656+0000 7fd222ffd640 1 -- 172.21.15.134:0/2788855532 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22409ba50 con 0x7fd224136e40 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd222ffd640 1 -- 172.21.15.134:0/2788855532 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 msgr2=0x7fd22409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd222ffd640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 0x7fd22409c150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd2140097d0 tx=0x7fd21402a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=bcfe84ff5fa3ff68 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 1 -- 172.21.15.134:0/2788855532 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd21402ad20 con 0x7fd224136e40 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: dump: 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.670 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 5 monclient: authenticate success, global_id 35025 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 1 -- 172.21.15.134:0/2788855532 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2140307b0 con 0x7fd224136e40 2024-09-16T15:11:56.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 1 -- 172.21.15.134:0/2788855532 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd21402aea0 con 0x7fd224136e40 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 4 set_mon_vals no callback set 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd221ffb640 10 monclient: dump: 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.672 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 10 monclient: shutdown 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 -- 172.21.15.134:0/2788855532 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 msgr2=0x7fd22409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 0x7fd22409c150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd2140097d0 tx=0x7fd21402a000 comp rx=0 tx=0).stop 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 -- 172.21.15.134:0/2788855532 shutdown_connections 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd22413c370 0x7fd22413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 --2- 172.21.15.134:0/2788855532 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd224136e40 0x7fd22409c150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.659+0000 7fd2293b5640 1 -- 172.21.15.134:0/2788855532 >> 172.21.15.134:0/2788855532 conn(0x7fd2240a6110 msgr2=0x7fd2240a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.660+0000 7fd2293b5640 1 -- 172.21.15.134:0/2788855532 shutdown_connections 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.660+0000 7fd2293b5640 1 -- 172.21.15.134:0/2788855532 wait complete. 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.660+0000 7fd2293b5640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.660+0000 7fd2293b5640 10 monclient: monmap: 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 Processor -- start 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 -- start start 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient: init 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient: _add_conns ranks=[1,5,8,0,3,6,2,4,7] 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 0x7fd2240a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-f con 0x7fd224136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd224137220 0x7fd224135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-h con 0x7fd224137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): picked mon.noname-e con 0x7fd22413c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22413b610 con 0x7fd224136e40 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240a7be0 con 0x7fd224137220 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd2241225c0 con 0x7fd22413c370 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2293b5640 10 monclient(hunting): authenticate will time out at 1453.250509s 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56812/0 (socket says 172.21.15.134:56812) 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 -- 172.21.15.134:0/646500286 learned_addr learned my addr 172.21.15.134:0/646500286 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 10 monclient(hunting): get_auth_request con 0x7fd22413c370 auth_method 0 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 msgr2=0x7fd224136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 msgr2=0x7fd224136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2237fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 0x7fd2240a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): get_auth_request con 0x7fd224136e40 auth_method 0 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd224137220 0x7fd224135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): get_auth_request con 0x7fd224137220 auth_method 0 2024-09-16T15:11:56.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd222ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.661+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 10 monclient(hunting): handle_auth_done global_id 44776 payload 824 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 msgr2=0x7fd224136150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd22413c370 0x7fd224136150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd224137220 msgr2=0x7fd224135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd224137220 0x7fd224135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 monclient: found mon.noname-f 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd222ffd640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009400 con 0x7fd224136e40 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.662+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd224137220 0x7fd224135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd222ffd640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 msgr2=0x7fd2240a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd222ffd640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 0x7fd2240a2720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd2140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd214030bf0 con 0x7fd224136e40 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: dump: 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.679 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.680 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214003610 con 0x7fd224136e40 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd214003bf0 con 0x7fd224136e40 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd20b7fe640 10 monclient: dump: 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 5 monclient: authenticate success, global_id 44776 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 10 monclient: _renew_subs 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2240a76d0 con 0x7fd224136e40 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2217fa640 4 set_mon_vals no callback set 2024-09-16T15:11:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 10 monclient: _renew_subs 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 10 monclient: _send_mon_message to mon.f at v2:172.21.15.162:3300/0 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.664+0000 7fd2293b5640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2240a32c0 con 0x7fd224136e40 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd214003d50 con 0x7fd224136e40 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2293b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2293b5640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 0x7fd224120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2293b5640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224065c50 con 0x7fd22411fc60 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 0x7fd224120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient: get_auth_request con 0x7fd22411fc60 auth_method 0 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd20b7fe640 1 --2- 172.21.15.134:0/646500286 >> v2:172.21.15.162:6800/980547263 conn(0x7fd210038870 0x7fd21003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.665+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.666+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fd21406aaa0 con 0x7fd224136e40 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.666+0000 7fd2227fc640 10 monclient(hunting): handle_auth_done global_id 74394 payload 824 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.666+0000 7fd2227fc640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd224061ae0 con 0x7fd22411fc60 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.666+0000 7fd2237fe640 1 --2- 172.21.15.134:0/646500286 >> v2:172.21.15.162:6800/980547263 conn(0x7fd210038870 0x7fd21003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.666+0000 7fd2237fe640 10 monclient: get_auth_request con 0x7fd210038870 auth_method 0 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd2227fc640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 msgr2=0x7fd224120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 0x7fd224120040 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd21800fc40 con 0x7fd22411fc60 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 10 monclient: dump: 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.684 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd2237fe640 1 --2- 172.21.15.134:0/646500286 >> v2:172.21.15.162:6800/980547263 conn(0x7fd210038870 0x7fd21003ad30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0094c0 tx=0x7fd21c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd21800fda0 con 0x7fd22411fc60 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 msgr2=0x7fd224120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 0x7fd224120040 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218007700 comp rx=0 tx=0).stop 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 msgr2=0x7fd224120040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:56.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.668+0000 7fd20b7fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd22411fc60 0x7fd224120040 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218007700 comp rx=0 tx=0).stop 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2293b5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2293b5640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 0x7fd22409e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2293b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2293b5640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224071b20 con 0x7fd224120580 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 0x7fd22409e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient: get_auth_request con 0x7fd224120580 auth_method 0 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.674+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.675+0000 7fd2227fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.675+0000 7fd2227fc640 10 monclient(hunting): handle_auth_done global_id 74403 payload 824 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.675+0000 7fd2227fc640 1 -- 172.21.15.134:0/646500286 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd22407a680 con 0x7fd224120580 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd2227fc640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 msgr2=0x7fd22409e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd2227fc640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 0x7fd22409e520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd218023070 con 0x7fd224120580 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 10 monclient: dump: 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.689 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fd2180058b0 con 0x7fd224120580 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.677+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 msgr2=0x7fd22409e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd20b7fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 0x7fd22409e520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218004940 comp rx=0 tx=0).stop 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd20b7fe640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 msgr2=0x7fd22409e520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:11:56.691 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.692 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.693 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:56.695 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.696 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:56.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.700 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.701 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.702 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:56.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:56.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:56.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:56.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:56.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.713 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:56.713 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:56.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd20b7fe640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd224120580 0x7fd22409e520 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218004940 comp rx=0 tx=0).stop 2024-09-16T15:11:56.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 >> v2:172.21.15.162:6800/980547263 conn(0x7fd210038870 msgr2=0x7fd21003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 --2- 172.21.15.134:0/646500286 >> v2:172.21.15.162:6800/980547263 conn(0x7fd210038870 0x7fd21003ad30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0094c0 tx=0x7fd21c00b040 comp rx=0 tx=0).stop 2024-09-16T15:11:56.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 10 monclient: shutdown 2024-09-16T15:11:56.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 msgr2=0x7fd2240a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 --2- 172.21.15.134:0/646500286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd224136e40 0x7fd2240a2720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd2140046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2237fe640 1 -- 172.21.15.134:0/646500286 reap_dead start 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2237fe640 1 -- 172.21.15.134:0/646500286 reap_dead start 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 shutdown_connections 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 >> 172.21.15.134:0/646500286 conn(0x7fd2240a6110 msgr2=0x7fd22406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.678+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 shutdown_connections 2024-09-16T15:11:56.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.679+0000 7fd2097fa640 1 -- 172.21.15.134:0/646500286 wait complete. 2024-09-16T15:11:56.715 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:56.715 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:11:56.716 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:11:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.811+0000 7f9652cc9640 10 monclient: get_monmap_and_config 2024-09-16T15:11:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.811+0000 7f9652cc9640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.811+0000 7f9652cc9640 10 monclient: monmap: 2024-09-16T15:11:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:56.812214+0000 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:56.812214+0000 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:56.814 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.811+0000 7f9652cc9640 1 Processor -- start 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 -- start start 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient: init 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient: _add_conns ranks=[10,9,5,3,4,2,0,8,1,7,6] 2024-09-16T15:11:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f964c136e40 0x7f964c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-f con 0x7f964c136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f964c137220 0x7f964c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-a con 0x7f964c137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c135dc0 0x7f964c136250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-h con 0x7f964c135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c09abc0 con 0x7f964c137220 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c09ba50 con 0x7f964c136e40 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0a53c0 con 0x7f964c135dc0 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9652cc9640 10 monclient(hunting): authenticate will time out at 1453.401520s 2024-09-16T15:11:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f964c137220 0x7f964c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f964c137220 0x7f964c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58142/0 (socket says 172.21.15.134:58142) 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 -- 172.21.15.134:0/4063507096 learned_addr learned my addr 172.21.15.134:0/4063507096 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): get_auth_request con 0x7f964c137220 auth_method 0 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient(hunting): handle_auth_done global_id 74412 payload 274 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9651266640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c135dc0 0x7f964c136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 -- 172.21.15.134:0/4063507096 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c135dc0 msgr2=0x7f964c136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c135dc0 0x7f964c136250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 -- 172.21.15.134:0/4063507096 >> v2:172.21.15.162:3300/0 conn(0x7f964c136e40 msgr2=0x7f964c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f9650a65640 1 --2- 172.21.15.134:0/4063507096 >> v2:172.21.15.162:3300/0 conn(0x7f964c136e40 0x7f964c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 --2- 172.21.15.134:0/4063507096 >> v2:172.21.15.162:3300/0 conn(0x7f964c136e40 0x7f964c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 monclient: found mon.noname-a 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:11:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.812+0000 7f964bfff640 1 -- 172.21.15.134:0/4063507096 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c073250 con 0x7f964c137220 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.815+0000 7f964bfff640 1 -- 172.21.15.134:0/4063507096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f964c137220 msgr2=0x7f964c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.815+0000 7f964bfff640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f964c137220 0x7f964c137600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f964c137f00 tx=0x7f963802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e787577932424f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 1 -- 172.21.15.134:0/4063507096 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96380307b0 con 0x7f964c137220 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: dump: 2024-09-16T15:11:56.825 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.826 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 1 -- 172.21.15.134:0/4063507096 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638030930 con 0x7f964c137220 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 5 monclient: authenticate success, global_id 74412 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 1 -- 172.21.15.134:0/4063507096 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96380313a0 con 0x7f964c137220 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 4 set_mon_vals no callback set 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:11:56.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f964b7fe640 10 monclient: dump: 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 10 monclient: shutdown 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 -- 172.21.15.134:0/4063507096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f964c137220 msgr2=0x7f964c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f964c137220 0x7f964c137600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f964c137f00 tx=0x7f963802a0b0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 -- 172.21.15.134:0/4063507096 shutdown_connections 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c135dc0 0x7f964c136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 --2- 172.21.15.134:0/4063507096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f964c137220 0x7f964c137600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 --2- 172.21.15.134:0/4063507096 >> v2:172.21.15.162:3300/0 conn(0x7f964c136e40 0x7f964c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 -- 172.21.15.134:0/4063507096 >> 172.21.15.134:0/4063507096 conn(0x7f964c0a6110 msgr2=0x7f964c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 -- 172.21.15.134:0/4063507096 shutdown_connections 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.816+0000 7f9652cc9640 1 -- 172.21.15.134:0/4063507096 wait complete. 2024-09-16T15:11:56.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.817+0000 7f9652cc9640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.817+0000 7f9652cc9640 10 monclient: monmap: 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.817+0000 7f9652cc9640 1 Processor -- start 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.817+0000 7f9652cc9640 1 -- start start 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.817+0000 7f9652cc9640 10 monclient: init 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient: _add_conns ranks=[8,2,5,6,0,7,4,1,3] 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-e con 0x7f964c135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 0x7f964c1350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-b con 0x7f964c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c137220 0x7f964c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): picked mon.noname-h con 0x7f964c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c136500 con 0x7f964c136e40 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964c122480 con 0x7f964c137220 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0a8bf0 con 0x7f964c135dc0 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9652cc9640 10 monclient(hunting): authenticate will time out at 1453.407413s 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 0x7f964c1350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56820/0 (socket says 172.21.15.134:56820) 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 learned_addr learned my addr 172.21.15.134:0/388273609 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 10 monclient(hunting): get_auth_request con 0x7f964c135dc0 auth_method 0 2024-09-16T15:11:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): get_auth_request con 0x7f964c136e40 auth_method 0 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 msgr2=0x7f964c134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 msgr2=0x7f964c134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9651266640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c137220 0x7f964c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient(hunting): handle_auth_done global_id 44930 payload 824 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 msgr2=0x7f964c134b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f964c135dc0 0x7f964c134b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c137220 msgr2=0x7f964c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c137220 0x7f964c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 monclient: found mon.noname-b 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f964bfff640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380094d0 con 0x7f964c136e40 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9651266640 10 monclient: get_auth_request con 0x7f964c137220 auth_method 0 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.818+0000 7f9651266640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f964c137220 0x7f964c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.820+0000 7f964bfff640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 msgr2=0x7f964c1350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f964bfff640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 0x7f964c1350c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f9638004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f96380307b0 con 0x7f964c136e40 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: dump: 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 5 monclient: authenticate success, global_id 44930 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638004460 con 0x7f964c136e40 2024-09-16T15:11:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 10 monclient: _renew_subs 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c0a8830 con 0x7f964c136e40 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f963803d450 con 0x7f964c136e40 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 10 monclient: dump: 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.837 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f964affd640 4 set_mon_vals no callback set 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 10 monclient: _renew_subs 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1221a0 con 0x7f964c136e40 2024-09-16T15:11:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.821+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9638039070 con 0x7f964c136e40 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9652cc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9652cc9640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 0x7f964c02b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c067a20 con 0x7f964c02b260 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 0x7f964c02b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient: get_auth_request con 0x7f964c02b260 auth_method 0 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f96497fa640 1 --2- 172.21.15.134:0/388273609 >> v2:172.21.15.162:6800/980547263 conn(0x7f9630038880 0x7f963003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f963806aa90 con 0x7f964c136e40 2024-09-16T15:11:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 10 monclient(hunting): handle_auth_done global_id 44939 payload 824 2024-09-16T15:11:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f964c061ae0 con 0x7f964c02b260 2024-09-16T15:11:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9651266640 1 --2- 172.21.15.134:0/388273609 >> v2:172.21.15.162:6800/980547263 conn(0x7f9630038880 0x7f963003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.822+0000 7f9651266640 10 monclient: get_auth_request con 0x7f9630038880 auth_method 0 2024-09-16T15:11:56.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 msgr2=0x7f964c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 0x7f964c02b640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f964000f410 tx=0x7f964000f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9640014df0 con 0x7f964c02b260 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f96497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f96497fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.824+0000 7f96497fa640 10 monclient: dump: 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.843 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f964000c030 con 0x7f964c02b260 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 msgr2=0x7f964c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 0x7f964c02b640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f964000f410 tx=0x7f964000f8d0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 msgr2=0x7f964c02b640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f96497fa640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02b260 0x7f964c02b640 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.825+0000 7f9651266640 1 --2- 172.21.15.134:0/388273609 >> v2:172.21.15.162:6800/980547263 conn(0x7f9630038880 0x7f963003ad40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f964c0a3020 tx=0x7f9644007360 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9652cc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9652cc9640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 0x7f964c02bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9652cc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c071b20 con 0x7f964c02bb80 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 0x7f964c02bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient: get_auth_request con 0x7f964c02bb80 auth_method 0 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 10 monclient(hunting): handle_auth_done global_id 44948 payload 824 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.831+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f964c07a680 con 0x7f964c02bb80 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f9650a65640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 msgr2=0x7f964c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f9650a65640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 0x7f964c02bf90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9640019000 tx=0x7f96400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9640023070 con 0x7f964c02bb80 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 10 monclient: dump: 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.846 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f9640003cb0 con 0x7f964c02bb80 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 msgr2=0x7f964c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.833+0000 7f96497fa640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 0x7f964c02bf90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9640019000 tx=0x7f96400046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f96497fa640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 msgr2=0x7f964c02bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f96497fa640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c02bb80 0x7f964c02bf90 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9640019000 tx=0x7f96400046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.850 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:56.851 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:56.852 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.854 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:56.856 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.859 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.862 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:56.864 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.865 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:56.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.867 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:56.868 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:56.869 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 >> v2:172.21.15.162:6800/980547263 conn(0x7f9630038880 msgr2=0x7f963003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 --2- 172.21.15.134:0/388273609 >> v2:172.21.15.162:6800/980547263 conn(0x7f9630038880 0x7f963003ad40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f964c0a3020 tx=0x7f9644007360 comp rx=0 tx=0).stop 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 10 monclient: shutdown 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 msgr2=0x7f964c1350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 --2- 172.21.15.134:0/388273609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f964c136e40 0x7f964c1350c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9638034000 tx=0x7f9638004730 comp rx=0 tx=0).stop 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9651266640 1 -- 172.21.15.134:0/388273609 reap_dead start 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9651266640 1 -- 172.21.15.134:0/388273609 reap_dead start 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 shutdown_connections 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 >> 172.21.15.134:0/388273609 conn(0x7f964c0a6110 msgr2=0x7f964c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 shutdown_connections 2024-09-16T15:11:56.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.834+0000 7f9652cc9640 1 -- 172.21.15.134:0/388273609 wait complete. 2024-09-16T15:11:56.872 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:56.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:11:56.872 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:11:56.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.968+0000 7f7c0bd01640 10 monclient: get_monmap_and_config 2024-09-16T15:11:56.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.968+0000 7f7c0bd01640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.968+0000 7f7c0bd01640 10 monclient: monmap: 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:56.969487+0000 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:56.969487+0000 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:56.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 Processor -- start 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 -- start start 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient: init 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient: _add_conns ranks=[3,0,4,7,9,5,2,1,8,10,6] 2024-09-16T15:11:56.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 0x7f7c0409bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-g con 0x7f7c04136eb0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7c04137290 addr v1:172.21.15.134:6789/0 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 0x7f7c0413e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-c con 0x7f7c0413c450 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c04073250 con 0x7f7c04137290 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040a5560 con 0x7f7c04136eb0 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0409b240 con 0x7f7c0413c450 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0bd01640 10 monclient(hunting): authenticate will time out at 1453.558817s 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0929c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7c04137290 0x7f7c04137690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50798/0 (socket says 172.21.15.134:50798) 2024-09-16T15:11:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0929c640 1 -- 172.21.15.134:0/3709959205 learned_addr learned my addr 172.21.15.134:0/3709959205 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 0x7f7c0413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0a29e640 10 monclient(hunting): get_auth_request con 0x7f7c0413c450 auth_method 0 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0a29e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0a29e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.969+0000 7f7c0a29e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c09a9d640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 0x7f7c0409bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient(hunting): handle_auth_done global_id 5359 payload 274 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3709959205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 msgr2=0x7f7c0409bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 0x7f7c0409bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3709959205 >> v1:172.21.15.134:6789/0 conn(0x7f7c04137290 legacy=0x7f7c04137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 1 monclient: found mon.noname-c 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3709959205 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0409b520 con 0x7f7c0413c450 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c09a9d640 10 monclient: get_auth_request con 0x7f7c04136eb0 auth_method 0 2024-09-16T15:11:56.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.970+0000 7f7c09a9d640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 0x7f7c0409bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 msgr2=0x7f7c0413e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 0x7f7c0413e840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009dc0 tx=0x7f7bfc02a4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=6ed54b97af581a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 1 -- 172.21.15.134:0/3709959205 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc007ca0 con 0x7f7c0413c450 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: dump: 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.982 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 1 -- 172.21.15.134:0/3709959205 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc02d050 con 0x7f7c0413c450 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 5 monclient: authenticate success, global_id 5359 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 1 -- 172.21.15.134:0/3709959205 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bfc007e60 con 0x7f7c0413c450 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 4 set_mon_vals no callback set 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c08a9b640 10 monclient: dump: 2024-09-16T15:11:56.984 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.985 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 10 monclient: shutdown 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 msgr2=0x7f7c0413e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 0x7f7c0413e840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009dc0 tx=0x7f7bfc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3709959205 shutdown_connections 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c0413c450 0x7f7c0413e840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3709959205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c04136eb0 0x7f7c0409bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.973+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3709959205 >> 172.21.15.134:0/3709959205 conn(0x7f7c040a99a0 msgr2=0x7f7c040a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3709959205 shutdown_connections 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3709959205 wait complete. 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 10 monclient: build_initial_monmap 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 10 monclient: monmap: 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:56.986 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 1 Processor -- start 2024-09-16T15:11:56.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 1 -- start start 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.974+0000 7f7c0bd01640 10 monclient: init 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient: _add_conns ranks=[0,7,1,2,8,5,6,4,3] 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 0x7f7c041cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-a con 0x7f7c04136eb0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7c04137290 0x7f7c041cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-i con 0x7f7c04137290 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c0413c450 0x7f7c041cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): picked mon.noname-f con 0x7f7c0413c450 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04122360 con 0x7f7c04136eb0 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040a77a0 con 0x7f7c0413c450 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04121da0 con 0x7f7c04137290 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0bd01640 10 monclient(hunting): authenticate will time out at 1453.564484s 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 0x7f7c041cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 0x7f7c041cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58154/0 (socket says 172.21.15.134:58154) 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 -- 172.21.15.134:0/3010843029 learned_addr learned my addr 172.21.15.134:0/3010843029 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): get_auth_request con 0x7f7c04136eb0 auth_method 0 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c0413c450 0x7f7c041cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 10 monclient(hunting): get_auth_request con 0x7f7c0413c450 auth_method 0 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7c04137290 0x7f7c041cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient(hunting): handle_auth_done global_id 74421 payload 824 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7c04137290 msgr2=0x7f7c041cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7c04137290 0x7f7c041cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c0413c450 msgr2=0x7f7c041cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c0413c450 0x7f7c041cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient: _finish_hunting 0 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 monclient: found mon.noname-a 2024-09-16T15:11:56.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c09a9d640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc009a70 con 0x7f7c04136eb0 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0929c640 10 monclient: get_auth_request con 0x7f7c04137290 auth_method 0 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7c04137290 0x7f7c041cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.975+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c0413c450 0x7f7c041cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c09a9d640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 msgr2=0x7f7c041cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c09a9d640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 0x7f7c041cb8d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000c7f0 tx=0x7f7c0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c0000ce40 con 0x7f7c04136eb0 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: dump: 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.991 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: _finish_auth 0 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 5 monclient: authenticate success, global_id 74421 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c000113f0 con 0x7f7c04136eb0 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 10 monclient: _renew_subs 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c04122080 con 0x7f7c04136eb0 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c000119d0 con 0x7f7c04136eb0 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfbfff640 4 set_mon_vals no callback set 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7bfa7fc640 10 monclient: dump: 2024-09-16T15:11:56.993 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:56.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 10 monclient: _renew_subs 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.978+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040a3890 con 0x7f7c04136eb0 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0bd01640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 0x7f7c0411fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04073880 con 0x7f7c04123010 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 0x7f7c0411fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient: get_auth_request con 0x7f7c04123010 auth_method 0 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7c00011bb0 con 0x7f7c04136eb0 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:56.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/3010843029 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:56.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 10 monclient(hunting): handle_auth_done global_id 5368 payload 824 2024-09-16T15:11:56.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0929c640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c04061ae0 con 0x7f7c04123010 2024-09-16T15:11:56.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f7c0004a1f0 con 0x7f7c04136eb0 2024-09-16T15:11:56.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3010843029 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:56.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.979+0000 7f7c0a29e640 10 monclient: get_auth_request con 0x7f7be8038880 auth_method 0 2024-09-16T15:11:56.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7c0929c640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 msgr2=0x7f7c0411fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:56.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 0x7f7c0411fbd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a6df0 tx=0x7f7bf400d950 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:56.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf4005cf0 con 0x7f7c04123010 2024-09-16T15:11:56.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7bfa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.981+0000 7f7bfa7fc640 10 monclient: dump: 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:56.999 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7bf4005e50 con 0x7f7c04123010 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 msgr2=0x7f7c0411fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 0x7f7c0411fbd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a6df0 tx=0x7f7bf400d950 comp rx=0 tx=0).stop 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 msgr2=0x7f7c0411fbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04123010 0x7f7c0411fbd0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7c040a6df0 tx=0x7f7bf400d950 comp rx=0 tx=0).stop 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.982+0000 7f7c0a29e640 1 --2- 172.21.15.134:0/3010843029 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0bd01640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 0x7f7c04120520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0bd01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c04071b20 con 0x7f7c04120110 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 0x7f7c04120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient: get_auth_request con 0x7f7c04120110 auth_method 0 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 10 monclient(hunting): handle_auth_done global_id 5377 payload 824 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.988+0000 7f7c0929c640 1 -- 172.21.15.134:0/3010843029 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7c0407a680 con 0x7f7c04120110 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7c0929c640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 msgr2=0x7f7c04120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7c0929c640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 0x7f7c04120520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4016000 tx=0x7f7bf4004940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bf4021070 con 0x7f7c04120110 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7bfa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.990+0000 7f7bfa7fc640 10 monclient: dump: 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.002 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f7bf4012410 con 0x7f7c04120110 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 msgr2=0x7f7c04120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 0x7f7c04120520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4016000 tx=0x7f7bf4004940 comp rx=0 tx=0).stop 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 msgr2=0x7f7c04120520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7bfa7fc640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c04120110 0x7f7c04120520 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4016000 tx=0x7f7bf4004940 comp rx=0 tx=0).stop 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:57.003 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:57.004 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.005 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.006 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:57.007 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.008 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.010 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:57.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.013 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:57.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.020 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:57.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:57.023 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:57.024 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:57.025 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:57.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 msgr2=0x7f7be803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3010843029 >> v2:172.21.15.162:6800/980547263 conn(0x7f7be8038880 0x7f7be803ad40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc034000 tx=0x7f7bfc02f000 comp rx=0 tx=0).stop 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 10 monclient: shutdown 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 msgr2=0x7f7c041cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 --2- 172.21.15.134:0/3010843029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c04136eb0 0x7f7c041cb8d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000c7f0 tx=0x7f7c0000cbf0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3010843029 reap_dead start 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0a29e640 1 -- 172.21.15.134:0/3010843029 reap_dead start 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 shutdown_connections 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 >> 172.21.15.134:0/3010843029 conn(0x7f7c040a99a0 msgr2=0x7f7c0413b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 shutdown_connections 2024-09-16T15:11:57.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:56.991+0000 7f7c0bd01640 1 -- 172.21.15.134:0/3010843029 wait complete. 2024-09-16T15:11:57.028 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:57.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:11:57.028 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.124+0000 7f15719f4640 10 monclient: get_monmap_and_config 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.124+0000 7f15719f4640 10 monclient: build_initial_monmap 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.124+0000 7f15719f4640 10 monclient: monmap: 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:57.125861+0000 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:57.125861+0000 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:57.127 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 Processor -- start 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 -- start start 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient: init 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient: _add_conns ranks=[3,1,5,2,6,0,7,10,4,9,8] 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-g con 0x7f156c069e50 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f156c09d890 addr v1:172.21.15.162:6789/0 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 0x7f156c13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-h con 0x7f156c13c750 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c073250 con 0x7f156c09d890 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c06cf00 con 0x7f156c069e50 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a6200 con 0x7f156c13c750 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.125+0000 7f15719f4640 10 monclient(hunting): authenticate will time out at 1453.714990s 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 0x7f156c13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156a7fc640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f156c09d890 0x7f156c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:60688/0 (socket says 172.21.15.134:60688) 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156a7fc640 1 -- 172.21.15.134:0/2440006954 learned_addr learned my addr 172.21.15.134:0/2440006954 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 10 monclient(hunting): get_auth_request con 0x7f156c069e50 auth_method 0 2024-09-16T15:11:57.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): get_auth_request con 0x7f156c13c750 auth_method 0 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 1 -- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 msgr2=0x7f156c06a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 1 -- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 msgr2=0x7f156c06a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156affd640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:57.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient(hunting): handle_auth_done global_id 44717 payload 274 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 -- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 msgr2=0x7f156c06a230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 -- 172.21.15.134:0/2440006954 >> v1:172.21.15.162:6789/0 conn(0x7f156c09d890 legacy=0x7f156c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 monclient: found mon.noname-h 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:11:57.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.126+0000 7f156b7fe640 1 -- 172.21.15.134:0/2440006954 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0a3d10 con 0x7f156c13c750 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f156b7fe640 1 -- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 msgr2=0x7f156c13eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f156b7fe640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 0x7f156c13eb40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1564009dc0 tx=0x7f156402a4d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=ab1ac1e026f46773 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 1 -- 172.21.15.134:0/2440006954 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1564007bc0 con 0x7f156c13c750 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: dump: 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.141 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: _finish_auth 0 2024-09-16T15:11:57.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 1 -- 172.21.15.134:0/2440006954 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156402d050 con 0x7f156c13c750 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 1 -- 172.21.15.134:0/2440006954 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1564007d40 con 0x7f156c13c750 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f1569ffb640 10 monclient: dump: 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.143 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f15719f4640 5 monclient: authenticate success, global_id 44717 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f15719f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.129+0000 7f15719f4640 4 set_mon_vals no callback set 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 10 monclient: shutdown 2024-09-16T15:11:57.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 -- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 msgr2=0x7f156c13eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 0x7f156c13eb40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1564009dc0 tx=0x7f156402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 -- 172.21.15.134:0/2440006954 shutdown_connections 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f156c13c750 0x7f156c13eb40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 --2- 172.21.15.134:0/2440006954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c069e50 0x7f156c06a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 -- 172.21.15.134:0/2440006954 >> 172.21.15.134:0/2440006954 conn(0x7f156c0a2e90 msgr2=0x7f156c0a32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 -- 172.21.15.134:0/2440006954 shutdown_connections 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.130+0000 7f15719f4640 1 -- 172.21.15.134:0/2440006954 wait complete. 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient: build_initial_monmap 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient: monmap: 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:57.145 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 Processor -- start 2024-09-16T15:11:57.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 -- start start 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient: init 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient: _add_conns ranks=[7,3,4,5,0,2,1,6,8] 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f156c069e50 0x7f156c1cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-i con 0x7f156c069e50 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 0x7f156c098910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-g con 0x7f156c09d890 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): picked mon.noname-c con 0x7f156c13c750 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1cbf90 con 0x7f156c09d890 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1cc9c0 con 0x7f156c13c750 2024-09-16T15:11:57.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f156c13b7f0 con 0x7f156c069e50 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.131+0000 7f15719f4640 10 monclient(hunting): authenticate will time out at 1453.720989s 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55168/0 (socket says 172.21.15.134:55168) 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 learned_addr learned my addr 172.21.15.134:0/170163166 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): get_auth_request con 0x7f156c13c750 auth_method 0 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f156c069e50 0x7f156c1cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 0x7f156c098910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 10 monclient(hunting): get_auth_request con 0x7f156c09d890 auth_method 0 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): get_auth_request con 0x7f156c069e50 auth_method 0 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 msgr2=0x7f156c098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 msgr2=0x7f156c098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 0x7f156c098910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156a7fc640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 0x7f156c098910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:57.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient(hunting): handle_auth_done global_id 5386 payload 824 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f156c069e50 msgr2=0x7f156c1cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f156c069e50 0x7f156c1cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 msgr2=0x7f156c098910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c09d890 0x7f156c098910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 monclient: found mon.noname-c 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.132+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009a70 con 0x7f156c13c750 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 msgr2=0x7f156c098e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f156b7fe640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1564009d90 tx=0x7f1564004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15640044b0 con 0x7f156c13c750 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 10 monclient: dump: 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.151 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 10 monclient: _finish_auth 0 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.134+0000 7f1553fff640 10 monclient: _check_auth_tickets 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564004650 con 0x7f156c13c750 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 5 monclient: authenticate success, global_id 5386 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 monclient: _renew_subs 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0a0e20 con 0x7f156c13c750 2024-09-16T15:11:57.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15640047c0 con 0x7f156c13c750 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 10 monclient: dump: 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.153 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 monclient: _renew_subs 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c091dd0 con 0x7f156c13c750 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15697fa640 4 set_mon_vals no callback set 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 0x7f156c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f156c06b5d0 con 0x7f156c070c20 2024-09-16T15:11:57.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1564004040 con 0x7f156c13c750 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f156affd640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 0x7f156c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f156affd640 10 monclient: get_auth_request con 0x7f156c070c20 auth_method 0 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f156affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f156affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.135+0000 7f156affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f1553fff640 1 --2- 172.21.15.134:0/170163166 >> v2:172.21.15.162:6800/980547263 conn(0x7f1558038880 0x7f155803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f156406b120 con 0x7f156c13c750 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156affd640 10 monclient(hunting): handle_auth_done global_id 35034 payload 824 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156affd640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f156c061ae0 con 0x7f156c070c20 2024-09-16T15:11:57.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156a7fc640 1 --2- 172.21.15.134:0/170163166 >> v2:172.21.15.162:6800/980547263 conn(0x7f1558038880 0x7f155803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.136+0000 7f156a7fc640 10 monclient: get_auth_request con 0x7f1558038880 auth_method 0 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f156affd640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 msgr2=0x7f156c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f156affd640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 0x7f156c02b260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f155c0107f0 tx=0x7f155c010cb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c00ec40 con 0x7f156c070c20 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 10 monclient: dump: 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.156 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f155c00eda0 con 0x7f156c070c20 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:57.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 msgr2=0x7f156c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 0x7f156c02b260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f155c0107f0 tx=0x7f155c010cb0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 msgr2=0x7f156c02b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:57.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.138+0000 7f1553fff640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c070c20 0x7f156c02b260 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.139+0000 7f156a7fc640 1 --2- 172.21.15.134:0/170163166 >> v2:172.21.15.162:6800/980547263 conn(0x7f1558038880 0x7f155803ad40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f156c06d820 tx=0x7f1560009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f15719f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f15719f4640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 0x7f156c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f15719f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f156c066d60 con 0x7f156c071000 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 0x7f156c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient: get_auth_request con 0x7f156c071000 auth_method 0 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 10 monclient(hunting): handle_auth_done global_id 35043 payload 824 2024-09-16T15:11:57.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.145+0000 7f156affd640 1 -- 172.21.15.134:0/170163166 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f156c07a680 con 0x7f156c071000 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f156affd640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 msgr2=0x7f156c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f156affd640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 0x7f156c02b7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f155c01a000 tx=0x7f155c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f155c014cc0 con 0x7f156c071000 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 10 monclient: dump: 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.161 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f155c023040 con 0x7f156c071000 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 msgr2=0x7f156c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 0x7f156c02b7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f155c01a000 tx=0x7f155c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 msgr2=0x7f156c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.147+0000 7f1553fff640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f156c071000 0x7f156c02b7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:57.163 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.164 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.165 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:57.166 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:57.167 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:57.168 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.171 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.174 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:57.177 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:57.178 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:57.180 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:57.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:11:57.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:57.183 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:57.184 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:57.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 >> v2:172.21.15.162:6800/980547263 conn(0x7f1558038880 msgr2=0x7f155803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 --2- 172.21.15.134:0/170163166 >> v2:172.21.15.162:6800/980547263 conn(0x7f1558038880 0x7f155803ad40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f156c06d820 tx=0x7f1560009040 comp rx=0 tx=0).stop 2024-09-16T15:11:57.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 10 monclient: shutdown 2024-09-16T15:11:57.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 msgr2=0x7f156c098e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1564009d90 tx=0x7f1564004a40 comp rx=0 tx=0).stop 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f156b7fe640 1 -- 172.21.15.134:0/170163166 reap_dead start 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 shutdown_connections 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 --2- 172.21.15.134:0/170163166 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f156c13c750 0x7f156c098e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 >> 172.21.15.134:0/170163166 conn(0x7f156c0a2e90 msgr2=0x7f156c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 shutdown_connections 2024-09-16T15:11:57.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.148+0000 7f15719f4640 1 -- 172.21.15.134:0/170163166 wait complete. 2024-09-16T15:11:57.187 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:57.187 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:11:57.188 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:11:57.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.285+0000 7f4c6a696640 10 monclient: get_monmap_and_config 2024-09-16T15:11:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.285+0000 7f4c6a696640 10 monclient: build_initial_monmap 2024-09-16T15:11:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.285+0000 7f4c6a696640 10 monclient: monmap: 2024-09-16T15:11:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.287 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:57.286029+0000 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:57.286029+0000 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:57.288 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:57.289 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:57.290 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:57.290 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:57.290 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.285+0000 7f4c6a696640 1 Processor -- start 2024-09-16T15:11:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 -- start start 2024-09-16T15:11:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient: init 2024-09-16T15:11:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient: _add_conns ranks=[8,2,7,10,3,4,9,6,0,1,5] 2024-09-16T15:11:57.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-e con 0x7f4c64136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-b con 0x7f4c64137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c64135dc0 0x7f4c64136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-i con 0x7f4c64135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:57.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6409abc0 con 0x7f4c64137220 2024-09-16T15:11:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6409ba50 con 0x7f4c64135dc0 2024-09-16T15:11:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640a53c0 con 0x7f4c64136e40 2024-09-16T15:11:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c6a696640 10 monclient(hunting): authenticate will time out at 1453.875528s 2024-09-16T15:11:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56824/0 (socket says 172.21.15.134:56824) 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54292/0 (socket says 172.21.15.134:54292) 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 -- 172.21.15.134:0/622198756 learned_addr learned my addr 172.21.15.134:0/622198756 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 10 monclient(hunting): get_auth_request con 0x7f4c64136e40 auth_method 0 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): get_auth_request con 0x7f4c64137220 auth_method 0 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 msgr2=0x7f4c6409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 msgr2=0x7f4c6409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c64135dc0 0x7f4c64136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): get_auth_request con 0x7f4c64135dc0 auth_method 0 2024-09-16T15:11:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c63fff640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:57.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient(hunting): handle_auth_done global_id 44957 payload 274 2024-09-16T15:11:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 msgr2=0x7f4c6409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:11:57.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c64135dc0 msgr2=0x7f4c64136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c64135dc0 0x7f4c64136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 monclient: found mon.noname-b 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:57.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.286+0000 7f4c5bfff640 1 -- 172.21.15.134:0/622198756 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64073250 con 0x7f4c64137220 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c5bfff640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 msgr2=0x7f4c64137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097d0 tx=0x7f4c4c029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953e8dcca6a7073e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 1 -- 172.21.15.134:0/622198756 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c4c02ad10 con 0x7f4c64137220 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:11:57.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: dump: 2024-09-16T15:11:57.321 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.321 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.321 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.321 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.322 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: _finish_auth 0 2024-09-16T15:11:57.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: _check_auth_tickets 2024-09-16T15:11:57.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:57.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c6a696640 5 monclient: authenticate success, global_id 44957 2024-09-16T15:11:57.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 1 -- 172.21.15.134:0/622198756 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c02ae90 con 0x7f4c64137220 2024-09-16T15:11:57.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c6a696640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 1 -- 172.21.15.134:0/622198756 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c4c030c20 con 0x7f4c64137220 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c637fe640 10 monclient: dump: 2024-09-16T15:11:57.325 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.326 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.326 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.326 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.326 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.326 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.327 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.328 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.328 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.328 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.328 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c6a696640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.289+0000 7f4c6a696640 4 set_mon_vals no callback set 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 10 monclient: shutdown 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 -- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 msgr2=0x7f4c64137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097d0 tx=0x7f4c4c029fb0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 -- 172.21.15.134:0/622198756 shutdown_connections 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4c64135dc0 0x7f4c64136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c64137220 0x7f4c64137600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 --2- 172.21.15.134:0/622198756 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c64136e40 0x7f4c6409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 -- 172.21.15.134:0/622198756 >> 172.21.15.134:0/622198756 conn(0x7f4c640a6110 msgr2=0x7f4c640a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 -- 172.21.15.134:0/622198756 shutdown_connections 2024-09-16T15:11:57.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.290+0000 7f4c6a696640 1 -- 172.21.15.134:0/622198756 wait complete. 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 10 monclient: build_initial_monmap 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 10 monclient: monmap: 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:57.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:57.332 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:57.333 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 1 Processor -- start 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 1 -- start start 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 10 monclient: init 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 10 monclient: _add_conns ranks=[1,3,6,5,4,7,2,8,0] 2024-09-16T15:11:57.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.291+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c64135dc0 0x7f4c64134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-f con 0x7f4c64135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 0x7f4c64135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-g con 0x7f4c64136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): picked mon.noname-d con 0x7f4c64137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:57.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64125350 con 0x7f4c64135dc0 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640a8980 con 0x7f4c64136e40 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64136320 con 0x7f4c64137220 2024-09-16T15:11:57.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c6a696640 10 monclient(hunting): authenticate will time out at 1453.881059s 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51998/0 (socket says 172.21.15.134:51998) 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 learned_addr learned my addr 172.21.15.134:0/2068234918 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): get_auth_request con 0x7f4c64137220 auth_method 0 2024-09-16T15:11:57.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c64135dc0 0x7f4c64134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 0x7f4c64135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): get_auth_request con 0x7f4c64135dc0 auth_method 0 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 10 monclient(hunting): get_auth_request con 0x7f4c64136e40 auth_method 0 2024-09-16T15:11:57.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient(hunting): handle_auth_done global_id 35052 payload 824 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 msgr2=0x7f4c64135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 msgr2=0x7f4c64135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:57.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 0x7f4c64135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 msgr2=0x7f4c64135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 0x7f4c64135380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c64136e40 0x7f4c64135380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c64135dc0 msgr2=0x7f4c64134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c64135dc0 0x7f4c64134e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:57.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient: _finish_hunting 0 2024-09-16T15:11:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 monclient: found mon.noname-d 2024-09-16T15:11:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.292+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009400 con 0x7f4c64137220 2024-09-16T15:11:57.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 msgr2=0x7f4c641218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c68c33640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00efc0 tx=0x7f4c5c00c490 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c5c009ca0 con 0x7f4c64137220 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: dump: 2024-09-16T15:11:57.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.347 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.347 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.347 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: _finish_auth 0 2024-09-16T15:11:57.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: _check_auth_tickets 2024-09-16T15:11:57.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:57.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00f040 con 0x7f4c64137220 2024-09-16T15:11:57.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c6a696640 5 monclient: authenticate success, global_id 35052 2024-09-16T15:11:57.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c6a696640 10 monclient: _renew_subs 2024-09-16T15:11:57.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c6a696640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c64125830 con 0x7f4c64137220 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c5c014d60 con 0x7f4c64137220 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c617fa640 10 monclient: dump: 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:57.349 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 10 monclient: _renew_subs 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c64125550 con 0x7f4c64137220 2024-09-16T15:11:57.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.294+0000 7f4c62ffd640 4 set_mon_vals no callback set 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64071b20 con 0x7f4c640a1a20 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4c5c014480 con 0x7f4c64137220 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c640a1a20 auth_method 0 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:57.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.295+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.296+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 0x7f4c4403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.296+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(513..513 src has 1..513) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f4c5c04af00 con 0x7f4c64137220 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.296+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 0x7f4c4403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.296+0000 7f4c5bfff640 10 monclient: get_auth_request con 0x7f4c440388b0 auth_method 0 2024-09-16T15:11:57.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.298+0000 7f4c5bfff640 1 --2- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 0x7f4c4403ad70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097a0 tx=0x7f4c4c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:57.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c640a1a20 auth_method 0 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.496+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2530: done (3 left) 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2531: done (2 left) 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2532: done (1 left) 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2534: done (0 left) 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2535: rollback oid 20 current snap is 247 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 230 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2536: write (excl) oid 30 current snap is 247 2024-09-16T15:11:57.635 INFO:tasks.rados.rados.0.smithi162.stdout:2536: seq_num 893 ranges {608651=410658,1556420=674971,2773556=235095} 2024-09-16T15:11:57.640 INFO:tasks.rados.rados.0.smithi162.stdout:2536: writing smithi16234619-30 from 608651 to 1019309 tid 1 2024-09-16T15:11:57.647 INFO:tasks.rados.rados.0.smithi162.stdout:2536: writing smithi16234619-30 from 1556420 to 2231391 tid 2 2024-09-16T15:11:57.651 INFO:tasks.rados.rados.0.smithi162.stdout:2536: writing smithi16234619-30 from 2773556 to 3008651 tid 3 2024-09-16T15:11:57.651 INFO:tasks.rados.rados.0.smithi162.stdout:2537: write oid 19 current snap is 247 2024-09-16T15:11:57.652 INFO:tasks.rados.rados.0.smithi162.stdout:2535: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:11:57.652 INFO:tasks.rados.rados.0.smithi162.stdout:2535: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:11:57.652 INFO:tasks.rados.rados.0.smithi162.stdout:2535: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:11:57.652 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 811 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:11:57.652 INFO:tasks.rados.rados.0.smithi162.stdout:2537: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2024-09-16T15:11:57.657 INFO:tasks.rados.rados.0.smithi162.stdout:2537: writing smithi16234619-19 from 625458 to 1111365 tid 1 2024-09-16T15:11:57.663 INFO:tasks.rados.rados.0.smithi162.stdout:2537: writing smithi16234619-19 from 1614902 to 2149884 tid 2 2024-09-16T15:11:57.667 INFO:tasks.rados.rados.0.smithi162.stdout:2537: writing smithi16234619-19 from 2917332 to 3025458 tid 3 2024-09-16T15:11:57.668 INFO:tasks.rados.rados.0.smithi162.stdout:2535: done (2 left) 2024-09-16T15:11:57.668 INFO:tasks.rados.rados.0.smithi162.stdout:2538: delete oid 35 current snap is 247 2024-09-16T15:11:57.670 INFO:tasks.rados.rados.0.smithi162.stdout:2538: done (2 left) 2024-09-16T15:11:57.670 INFO:tasks.rados.rados.0.smithi162.stdout:2539: delete oid 47 current snap is 247 2024-09-16T15:11:57.672 INFO:tasks.rados.rados.0.smithi162.stdout:2539: done (2 left) 2024-09-16T15:11:57.672 INFO:tasks.rados.rados.0.smithi162.stdout:2540: read oid 33 snap -1 2024-09-16T15:11:57.672 INFO:tasks.rados.rados.0.smithi162.stdout:2540: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:57.673 INFO:tasks.rados.rados.0.smithi162.stdout:2541: delete oid 6 current snap is 247 2024-09-16T15:11:57.675 INFO:tasks.rados.rados.0.smithi162.stdout:2540: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:11:57.836 INFO:tasks.rados.rados.0.smithi162.stdout:2536: finishing write tid 1 to smithi16234619-30 2024-09-16T15:11:57.836 INFO:tasks.rados.rados.0.smithi162.stdout:2536: finishing write tid 2 to smithi16234619-30 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: finishing write tid 3 to smithi16234619-30 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: finishing write tid 4 to smithi16234619-30 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2540: done (3 left) 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2541: done (2 left) 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2542: write oid 44 current snap is 247 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: finishing write tid 5 to smithi16234619-30 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: oid 30 updating version 0 to 1132 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: oid 30 updating version 1132 to 1135 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: oid 30 version 1135 is already newer than 1134 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: oid 30 version 1135 is already newer than 1133 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1135 (ObjNum 893 snap 247 seq_num 893) dirty exists 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2536: left oid 30 (ObjNum 893 snap 247 seq_num 893) 2024-09-16T15:11:57.837 INFO:tasks.rados.rados.0.smithi162.stdout:2537: finishing write tid 1 to smithi16234619-19 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: finishing write tid 2 to smithi16234619-19 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: finishing write tid 3 to smithi16234619-19 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: finishing write tid 4 to smithi16234619-19 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: finishing write tid 5 to smithi16234619-19 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: oid 19 updating version 0 to 1030 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: oid 19 updating version 1030 to 1032 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: oid 19 updating version 1032 to 1033 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: oid 19 version 1033 is already newer than 1031 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1033 (ObjNum 894 snap 247 seq_num 894) dirty exists 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2537: left oid 19 (ObjNum 894 snap 247 seq_num 894) 2024-09-16T15:11:57.838 INFO:tasks.rados.rados.0.smithi162.stdout:2542: seq_num 898 ranges {692686=503256,1848830=425967,2910671=182015} 2024-09-16T15:11:57.842 INFO:tasks.rados.rados.0.smithi162.stdout:2542: writing smithi16234619-44 from 692686 to 1195942 tid 1 2024-09-16T15:11:57.848 INFO:tasks.rados.rados.0.smithi162.stdout:2542: writing smithi16234619-44 from 1848830 to 2274797 tid 2 2024-09-16T15:11:57.853 INFO:tasks.rados.rados.0.smithi162.stdout:2542: writing smithi16234619-44 from 2910671 to 3092686 tid 3 2024-09-16T15:11:57.853 INFO:tasks.rados.rados.0.smithi162.stdout:2536: done (2 left) 2024-09-16T15:11:57.853 INFO:tasks.rados.rados.0.smithi162.stdout:2537: done (1 left) 2024-09-16T15:11:57.853 INFO:tasks.rados.rados.0.smithi162.stdout:2543: delete oid 34 current snap is 247 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2543: done (1 left) 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2544: rollback oid 21 current snap is 247 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 235 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2545: read oid 28 snap -1 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2545: expect deleted 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2546: copy_from oid 11 from oid 25 current snap is 247 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2547: write (excl) oid 3 current snap is 247 2024-09-16T15:11:57.855 INFO:tasks.rados.rados.0.smithi162.stdout:2547: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2024-09-16T15:11:57.863 INFO:tasks.rados.rados.0.smithi162.stdout:2547: writing smithi16234619-3 from 726300 to 1380054 tid 1 2024-09-16T15:11:57.869 INFO:tasks.rados.rados.0.smithi162.stdout:2547: writing smithi16234619-3 from 1965794 to 2395430 tid 2 2024-09-16T15:11:57.874 INFO:tasks.rados.rados.0.smithi162.stdout:2547: writing smithi16234619-3 from 2965517 to 3126300 tid 3 2024-09-16T15:11:57.874 INFO:tasks.rados.rados.0.smithi162.stdout:2548: snap_create 2024-09-16T15:11:57.874 INFO:tasks.rados.rados.0.smithi162.stdout:2544: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:11:57.874 INFO:tasks.rados.rados.0.smithi162.stdout:2544: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2544: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1210 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: finishing write tid 1 to smithi16234619-44 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: finishing write tid 2 to smithi16234619-44 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: finishing write tid 3 to smithi16234619-44 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: finishing write tid 4 to smithi16234619-44 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: finishing write tid 5 to smithi16234619-44 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: oid 44 updating version 0 to 968 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: oid 44 version 968 is already newer than 966 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: oid 44 updating version 968 to 969 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: oid 44 version 969 is already newer than 967 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 969 (ObjNum 898 snap 247 seq_num 898) dirty exists 2024-09-16T15:11:57.875 INFO:tasks.rados.rados.0.smithi162.stdout:2542: left oid 44 (ObjNum 898 snap 247 seq_num 898) 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: finishing write tid 1 to smithi16234619-3 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: finishing write tid 2 to smithi16234619-3 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: finishing write tid 3 to smithi16234619-3 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: finishing write tid 4 to smithi16234619-3 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: finishing write tid 5 to smithi16234619-3 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: oid 3 updating version 0 to 118 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: oid 3 updating version 118 to 120 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: oid 3 version 120 is already newer than 119 2024-09-16T15:11:57.894 INFO:tasks.rados.rados.0.smithi162.stdout:2547: oid 3 version 120 is already newer than 117 2024-09-16T15:11:57.895 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 120 (ObjNum 900 snap 247 seq_num 900) dirty exists 2024-09-16T15:11:57.895 INFO:tasks.rados.rados.0.smithi162.stdout:2547: left oid 3 (ObjNum 900 snap 247 seq_num 900) 2024-09-16T15:11:57.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:57.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c640a1a20 auth_method 0 2024-09-16T15:11:57.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:57.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:57.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:57.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:11:57.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:57.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:57.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:57.897+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:11:57.909 INFO:tasks.rados.rados.0.smithi162.stdout:2546: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:11:57.914 INFO:tasks.rados.rados.0.smithi162.stdout:2546: finishing copy_from to smithi16234619-11 2024-09-16T15:11:57.914 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 593 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:11:58.642 INFO:tasks.rados.rados.0.smithi162.stdout:2542: done (5 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2544: done (4 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2545: done (3 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2546: done (2 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2547: done (1 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2548: done (0 left) 2024-09-16T15:11:58.643 INFO:tasks.rados.rados.0.smithi162.stdout:2549: delete oid 23 current snap is 248 2024-09-16T15:11:58.646 INFO:tasks.rados.rados.0.smithi162.stdout:2549: done (0 left) 2024-09-16T15:11:58.646 INFO:tasks.rados.rados.0.smithi162.stdout:2550: snap_create 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c640a1a20 auth_method 0 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 10 monclient(hunting): handle_auth_done global_id 84113 payload 824 2024-09-16T15:11:58.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.698+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64061ae0 con 0x7f4c640a1a20 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.700+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.700+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c54017750 tx=0x7f4c54017c10 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c54015dd0 con 0x7f4c640a1a20 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 10 monclient: dump: 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.703 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4c5401c070 con 0x7f4c640a1a20 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c54017750 tx=0x7f4c54017c10 comp rx=0 tx=0).stop 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 msgr2=0x7f4c640a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.701+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a1a20 0x7f4c640a1e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c6a696640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c6a696640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c64137600 con 0x7f4c640a2340 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c640a2340 auth_method 0 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.707+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.708+0000 7f4c63fff640 10 monclient(hunting): handle_auth_done global_id 84122 payload 824 2024-09-16T15:11:58.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.708+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4c6407a680 con 0x7f4c640a2340 2024-09-16T15:11:58.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 msgr2=0x7f4c64121e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c54017750 tx=0x7f4c54004730 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 msgr2=0x7f4c64121e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 msgr2=0x7f4c64121e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c54017750 tx=0x7f4c54004730 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c54017750 tx=0x7f4c54004730 comp rx=0 tx=0).stop 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 msgr2=0x7f4c64121e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 0x7f4c440412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 msgr2=0x7f4c64121e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c640a2340 0x7f4c64121e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4403e930 con 0x7f4c4403eec0 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 0x7f4c440412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient: get_auth_request con 0x7f4c4403eec0 auth_method 0 2024-09-16T15:11:58.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 10 monclient(hunting): handle_auth_done global_id 84131 payload 824 2024-09-16T15:11:58.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.710+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4c5401c070 con 0x7f4c4403eec0 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.712+0000 7f4c63fff640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 msgr2=0x7f4c440412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c63fff640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 0x7f4c440412b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000c00 tx=0x7f4c5401c000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c540223e0 con 0x7f4c4403eec0 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 10 monclient: dump: 2024-09-16T15:11:58.715 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.716 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f4c54036040 con 0x7f4c4403eec0 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 msgr2=0x7f4c440412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 0x7f4c440412b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000c00 tx=0x7f4c5401c000 comp rx=0 tx=0).stop 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 msgr2=0x7f4c440412b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c617fa640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4c4403eec0 0x7f4c440412b0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4c54000c00 tx=0x7f4c5401c000 comp rx=0 tx=0).stop 2024-09-16T15:11:58.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c68c33640 1 -- 172.21.15.134:0/2068234918 reap_dead start 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:58.718 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.719 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.720 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:58.721 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.722 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:58.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.725 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:58.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.730 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.733 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.734 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:58.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:58.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.737 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.739 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:58.740 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.741 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.741 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:58.741 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 msgr2=0x7f4c4403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 0x7f4c4403ad70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0097a0 tx=0x7f4c4c02e000 comp rx=0 tx=0).stop 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 10 monclient: shutdown 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 msgr2=0x7f4c641218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00efc0 tx=0x7f4c5c00c490 comp rx=0 tx=0).stop 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 shutdown_connections 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4c64137220 0x7f4c641218d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 --2- 172.21.15.134:0/2068234918 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c440388b0 0x7f4c4403ad70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 >> 172.21.15.134:0/2068234918 conn(0x7f4c640a6110 msgr2=0x7f4c6406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 shutdown_connections 2024-09-16T15:11:58.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.713+0000 7f4c6a696640 1 -- 172.21.15.134:0/2068234918 wait complete. 2024-09-16T15:11:58.743 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:58.743 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:11:58.744 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:11:58.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.839+0000 7f4254d45640 10 monclient: get_monmap_and_config 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.839+0000 7f4254d45640 10 monclient: build_initial_monmap 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.839+0000 7f4254d45640 10 monclient: monmap: 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:58.840158+0000 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:58.840158+0000 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:58.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:58.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.839+0000 7f4254d45640 1 Processor -- start 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 -- start start 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient: init 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient: _add_conns ranks=[2,0,9,6,1,4,8,5,3,7,10] 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f425009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-b con 0x7f4250136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4250137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f425013c370 0x7f425013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-a con 0x7f425013c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4250073250 con 0x7f4250137220 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42500a56e0 con 0x7f425013c370 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425009abc0 con 0x7f4250136e40 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4254d45640 10 monclient(hunting): authenticate will time out at 1455.429557s 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f425013c370 0x7f425013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4247fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f4250137220 0x7f4250137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50800/0 (socket says 172.21.15.134:50800) 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f4247fff640 1 -- 172.21.15.134:0/3363026877 learned_addr learned my addr 172.21.15.134:0/3363026877 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): get_auth_request con 0x7f425013c370 auth_method 0 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f425009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): get_auth_request con 0x7f4250136e40 auth_method 0 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient(hunting): handle_auth_done global_id 74430 payload 274 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 -- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 msgr2=0x7f425009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f425009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 -- 172.21.15.134:0/3363026877 >> v1:172.21.15.134:6789/0 conn(0x7f4250137220 legacy=0x7f4250137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 monclient: found mon.noname-a 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:58.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424ed9d640 1 -- 172.21.15.134:0/3363026877 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425009ba50 con 0x7f425013c370 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.840+0000 7f424e59c640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f425009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424ed9d640 1 -- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f425013c370 msgr2=0x7f425013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424ed9d640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f425013c370 0x7f425013e760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4248009dc0 tx=0x7f424802a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100851cde44461a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 1 -- 172.21.15.134:0/3363026877 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4248007bc0 con 0x7f425013c370 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: dump: 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.872 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: _finish_auth 0 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 1 -- 172.21.15.134:0/3363026877 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424802d050 con 0x7f425013c370 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f4254d45640 5 monclient: authenticate success, global_id 74430 2024-09-16T15:11:58.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 1 -- 172.21.15.134:0/3363026877 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4248007d40 con 0x7f425013c370 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f424dd9b640 10 monclient: dump: 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.874 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f4254d45640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.843+0000 7f4254d45640 4 set_mon_vals no callback set 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 10 monclient: shutdown 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 -- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f425013c370 msgr2=0x7f425013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f425013c370 0x7f425013e760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4248009dc0 tx=0x7f424802a510 comp rx=0 tx=0).stop 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 -- 172.21.15.134:0/3363026877 shutdown_connections 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f425013c370 0x7f425013e760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 --2- 172.21.15.134:0/3363026877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f425009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 -- 172.21.15.134:0/3363026877 >> 172.21.15.134:0/3363026877 conn(0x7f42500a6110 msgr2=0x7f42500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 -- 172.21.15.134:0/3363026877 shutdown_connections 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.844+0000 7f4254d45640 1 -- 172.21.15.134:0/3363026877 wait complete. 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient: build_initial_monmap 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient: monmap: 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 1 Processor -- start 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 1 -- start start 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient: init 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient: _add_conns ranks=[2,5,7,1,0,4,8,3,6] 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-b con 0x7f4250136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.845+0000 7f4254d45640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4250137220 0x7f4250135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-h con 0x7f4250137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:58.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f425013c370 0x7f425011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): picked mon.noname-i con 0x7f425013c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4250122360 con 0x7f4250136e40 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4250120eb0 con 0x7f4250137220 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f425013b4a0 con 0x7f425013c370 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4254d45640 10 monclient(hunting): authenticate will time out at 1455.435044s 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54326/0 (socket says 172.21.15.134:54326) 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 -- 172.21.15.134:0/3606323875 learned_addr learned my addr 172.21.15.134:0/3606323875 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): get_auth_request con 0x7f4250136e40 auth_method 0 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient(hunting): handle_auth_done global_id 44975 payload 824 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424ed9d640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f425013c370 0x7f425011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4250137220 0x7f4250135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f425013c370 msgr2=0x7f425011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f425013c370 0x7f425011fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4250137220 msgr2=0x7f4250135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4250137220 0x7f4250135fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 monclient: found mon.noname-b 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:11:58.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f424e59c640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4240009400 con 0x7f4250136e40 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4247fff640 10 monclient: get_auth_request con 0x7f4250137220 auth_method 0 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.846+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4250137220 0x7f4250135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f424e59c640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 msgr2=0x7f4250135a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f424e59c640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424000ebe0 tx=0x7f424000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f424000eee0 con 0x7f4250136e40 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: dump: 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.880 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: _finish_auth 0 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424000ce40 con 0x7f4250136e40 2024-09-16T15:11:58.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 5 monclient: authenticate success, global_id 44975 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 10 monclient: _renew_subs 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248009a70 con 0x7f4250136e40 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4240013d50 con 0x7f4250136e40 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4246ffd640 10 monclient: dump: 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.882 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f424d59a640 4 set_mon_vals no callback set 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 10 monclient: _renew_subs 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.848+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42500a3300 con 0x7f4250136e40 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4254d45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4240013470 con 0x7f4250136e40 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4254d45640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 0x7f4250120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250071b20 con 0x7f42501201a0 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4246ffd640 1 --2- 172.21.15.134:0/3606323875 >> v2:172.21.15.162:6800/980547263 conn(0x7f4230038880 0x7f423003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(515..515 src has 1..515) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f424004a710 con 0x7f4250136e40 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 0x7f4250120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient: get_auth_request con 0x7f42501201a0 auth_method 0 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f424ed9d640 1 --2- 172.21.15.134:0/3606323875 >> v2:172.21.15.162:6800/980547263 conn(0x7f4230038880 0x7f423003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.849+0000 7f424ed9d640 10 monclient: get_auth_request con 0x7f4230038880 auth_method 0 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.850+0000 7f4247fff640 10 monclient(hunting): handle_auth_done global_id 44803 payload 824 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.850+0000 7f4247fff640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4250061ae0 con 0x7f42501201a0 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4247fff640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 msgr2=0x7f4250120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 0x7f4250120580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f423800d4c0 tx=0x7f423800d980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f424ed9d640 1 --2- 172.21.15.134:0/3606323875 >> v2:172.21.15.162:6800/980547263 conn(0x7f4230038880 0x7f423003ad40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f425006cb00 tx=0x7f424802f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4238005cf0 con 0x7f42501201a0 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 10 monclient: dump: 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.885 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4238005e50 con 0x7f42501201a0 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:58.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 msgr2=0x7f4250120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 0x7f4250120580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f423800d4c0 tx=0x7f423800d980 comp rx=0 tx=0).stop 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 msgr2=0x7f4250120580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.852+0000 7f4246ffd640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42501201a0 0x7f4250120580 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f423800d4c0 tx=0x7f423800d980 comp rx=0 tx=0).stop 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.857+0000 7f4254d45640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.857+0000 7f4254d45640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 0x7f425009d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.857+0000 7f4254d45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.857+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425007a680 con 0x7f425009d510 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 0x7f425009d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient: get_auth_request con 0x7f425009d510 auth_method 0 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:58.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 10 monclient(hunting): handle_auth_done global_id 44812 payload 824 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.858+0000 7f4247fff640 1 -- 172.21.15.134:0/3606323875 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42500a53c0 con 0x7f425009d510 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4247fff640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 msgr2=0x7f425009d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4247fff640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 0x7f425009d920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4238016000 tx=0x7f4238004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4238021070 con 0x7f425009d510 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4246ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4246ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.860+0000 7f4246ffd640 10 monclient: dump: 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:58.888 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f4238012410 con 0x7f425009d510 2024-09-16T15:11:58.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 msgr2=0x7f425009d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 0x7f425009d920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4238016000 tx=0x7f4238004940 comp rx=0 tx=0).stop 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 msgr2=0x7f425009d920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4246ffd640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f425009d510 0x7f425009d920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:58.890 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:58.891 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.892 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.893 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:58.894 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:58.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.896 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:58.897 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.901 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:58.903 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:58.905 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:58.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:58.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:58.911 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:58.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 >> v2:172.21.15.162:6800/980547263 conn(0x7f4230038880 msgr2=0x7f423003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 --2- 172.21.15.134:0/3606323875 >> v2:172.21.15.162:6800/980547263 conn(0x7f4230038880 0x7f423003ad40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f425006cb00 tx=0x7f424802f000 comp rx=0 tx=0).stop 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 10 monclient: shutdown 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 msgr2=0x7f4250135a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424000ebe0 tx=0x7f424000c520 comp rx=0 tx=0).stop 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f424ed9d640 1 -- 172.21.15.134:0/3606323875 reap_dead start 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 shutdown_connections 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 --2- 172.21.15.134:0/3606323875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4250136e40 0x7f4250135a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 >> 172.21.15.134:0/3606323875 conn(0x7f42500a6110 msgr2=0x7f425013b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 shutdown_connections 2024-09-16T15:11:58.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:58.861+0000 7f4254d45640 1 -- 172.21.15.134:0/3606323875 wait complete. 2024-09-16T15:11:58.914 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:58.914 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:11:58.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.010+0000 7f9dd3a5a640 10 monclient: get_monmap_and_config 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.010+0000 7f9dd3a5a640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.010+0000 7f9dd3a5a640 10 monclient: monmap: 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:59.012441+0000 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:59.012441+0000 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:59.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:59.014 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.011+0000 7f9dd3a5a640 1 Processor -- start 2024-09-16T15:11:59.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.011+0000 7f9dd3a5a640 1 -- start start 2024-09-16T15:11:59.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.011+0000 7f9dd3a5a640 10 monclient: init 2024-09-16T15:11:59.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient: _add_conns ranks=[3,6,0,7,2,1,10,5,9,4,8] 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-g con 0x7f9dcc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 0x7f9dcc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-d con 0x7f9dcc137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9dcc135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dcc073250 con 0x7f9dcc135dc0 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc09abc0 con 0x7f9dcc136e40 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc09ba50 con 0x7f9dcc137220 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd3a5a640 10 monclient(hunting): authenticate will time out at 1455.602385s 2024-09-16T15:11:59.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd1ff7640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9dcc135dc0 0x7f9dcc136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50812/0 (socket says 172.21.15.134:50812) 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/1023372457 learned_addr learned my addr 172.21.15.134:0/1023372457 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.012+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd17f6640 1 -- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 msgr2=0x7f9dcc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd17f6640 1 -- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 msgr2=0x7f9dcc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 0x7f9dcc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): get_auth_request con 0x7f9dcc137220 auth_method 0 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.013+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_done global_id 35061 payload 274 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 1 -- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 msgr2=0x7f9dcc09c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 1 -- 172.21.15.134:0/1023372457 >> v1:172.21.15.134:6789/0 conn(0x7f9dcc135dc0 legacy=0x7f9dcc136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 1 monclient: found mon.noname-d 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:59.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.014+0000 7f9dd0ff5640 1 -- 172.21.15.134:0/1023372457 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0a53c0 con 0x7f9dcc137220 2024-09-16T15:11:59.021 INFO:tasks.rados.rados.0.smithi162.stdout:2550: done (0 left) 2024-09-16T15:11:59.021 INFO:tasks.rados.rados.0.smithi162.stdout:2551: copy_from oid 15 from oid 31 current snap is 249 2024-09-16T15:11:59.021 INFO:tasks.rados.rados.0.smithi162.stdout:2552: write oid 47 current snap is 249 2024-09-16T15:11:59.021 INFO:tasks.rados.rados.0.smithi162.stdout:2552: seq_num 902 ranges {759914=404252,1566405=433305,2504010=655904} 2024-09-16T15:11:59.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.017+0000 7f9dd0ff5640 1 -- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 msgr2=0x7f9dcc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.017+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 0x7f9dcc137600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009800 tx=0x7f9dbc02a030 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f5fec9e27cf91170 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 1 -- 172.21.15.134:0/1023372457 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dbc02ae30 con 0x7f9dcc137220 2024-09-16T15:11:59.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:59.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: dump: 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.024 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 1 -- 172.21.15.134:0/1023372457 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc0307b0 con 0x7f9dcc137220 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dd3a5a640 5 monclient: authenticate success, global_id 35061 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dd3a5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dd3a5a640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 1 -- 172.21.15.134:0/1023372457 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dbc030d90 con 0x7f9dcc137220 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dd3a5a640 4 set_mon_vals no callback set 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:59.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.021+0000 7f9dc3fff640 10 monclient: dump: 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.026 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.027 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.027 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.027 INFO:tasks.rados.rados.0.smithi162.stdout:2552: writing smithi16234619-47 from 759914 to 1164166 tid 1 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.022+0000 7f9dd3a5a640 10 monclient: shutdown 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.022+0000 7f9dd3a5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.022+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 msgr2=0x7f9dcc137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.022+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 0x7f9dcc137600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009800 tx=0x7f9dbc02a030 comp rx=0 tx=0).stop 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.023+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/1023372457 shutdown_connections 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.023+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9dcc137220 0x7f9dcc137600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.023+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/1023372457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc136e40 0x7f9dcc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.023+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/1023372457 >> 172.21.15.134:0/1023372457 conn(0x7f9dcc0a6110 msgr2=0x7f9dcc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.024+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/1023372457 shutdown_connections 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.024+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/1023372457 wait complete. 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.025+0000 7f9dd3a5a640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.025+0000 7f9dd3a5a640 10 monclient: monmap: 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:59.034 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.025+0000 7f9dd3a5a640 1 Processor -- start 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.025+0000 7f9dd3a5a640 1 -- start start 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.025+0000 7f9dd3a5a640 10 monclient: init 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 10 monclient: _add_conns ranks=[1,0,5,2,3,6,4,8,7] 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dcc135dc0 0x7f9dcc0a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-f con 0x7f9dcc135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 0x7f9dcc134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-a con 0x7f9dcc136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd0ff5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 0x7f9dcc134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.026+0000 7f9dd0ff5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 0x7f9dcc134e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58166/0 (socket says 172.21.15.134:58166) 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd17f6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dcc135dc0 0x7f9dcc0a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd17f6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dcc135dc0 0x7f9dcc0a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:54428/0 (socket says 172.21.15.134:54428) 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 0x7f9dcc1cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd0ff5640 1 -- 172.21.15.134:0/733156211 learned_addr learned my addr 172.21.15.134:0/733156211 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): picked mon.noname-h con 0x7f9dcc137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd1ff7640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 0x7f9dcc1cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc136500 con 0x7f9dcc136e40 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc121fa0 con 0x7f9dcc135dc0 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc0a8830 con 0x7f9dcc137220 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.027+0000 7f9dd3a5a640 10 monclient(hunting): authenticate will time out at 1455.617969s 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): get_auth_request con 0x7f9dcc136e40 auth_method 0 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd17f6640 10 monclient(hunting): get_auth_request con 0x7f9dcc135dc0 auth_method 0 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd17f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd17f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd17f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): get_auth_request con 0x7f9dcc137220 auth_method 0 2024-09-16T15:11:59.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd0ff5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 10 monclient(hunting): handle_auth_done global_id 44726 payload 824 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dcc135dc0 msgr2=0x7f9dcc0a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dcc135dc0 0x7f9dcc0a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.028+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 msgr2=0x7f9dcc134e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 0x7f9dcc134e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 1 monclient: found mon.noname-h 2024-09-16T15:11:59.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:11:59.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc009400 con 0x7f9dcc137220 2024-09-16T15:11:59.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.029+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dcc136e40 0x7f9dcc134e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.040 INFO:tasks.rados.rados.0.smithi162.stdout:2552: writing smithi16234619-47 from 1566405 to 1999710 tid 2 2024-09-16T15:11:59.040 INFO:tasks.rados.rados.0.smithi162.stdout:2552: writing smithi16234619-47 from 2504010 to 3159914 tid 3 2024-09-16T15:11:59.040 INFO:tasks.rados.rados.0.smithi162.stdout:2553: snap_remove snap 246 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 msgr2=0x7f9dcc1cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd1ff7640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 0x7f9dcc1cb430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400caf0 tx=0x7f9dc400cef0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dc400abe0 con 0x7f9dcc137220 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: dump: 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.043 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400ad80 con 0x7f9dcc137220 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd3a5a640 5 monclient: authenticate success, global_id 44726 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd3a5a640 10 monclient: _renew_subs 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd3a5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc1357f0 con 0x7f9dcc137220 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dc4012ce0 con 0x7f9dcc137220 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc1ffb640 10 monclient: dump: 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.032+0000 7f9dc37fe640 4 set_mon_vals no callback set 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.033+0000 7f9dd3a5a640 10 monclient: _renew_subs 2024-09-16T15:11:59.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.033+0000 7f9dd3a5a640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.033+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc0a3460 con 0x7f9dcc137220 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.033+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9dc4012400 con 0x7f9dcc137220 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.033+0000 7f9dd3a5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 0x7f9dcc02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc071b20 con 0x7f9dcc1353c0 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 0x7f9dcc02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient: get_auth_request con 0x7f9dcc1353c0 auth_method 0 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dc1ffb640 1 --2- 172.21.15.134:0/733156211 >> v2:172.21.15.162:6800/980547263 conn(0x7f9db0038880 0x7f9db003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(516..516 src has 1..516) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f9dc404a580 con 0x7f9dcc137220 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_done global_id 84108 payload 824 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd17f6640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dcc061ae0 con 0x7f9dcc1353c0 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/733156211 >> v2:172.21.15.162:6800/980547263 conn(0x7f9db0038880 0x7f9db003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.034+0000 7f9dd0ff5640 10 monclient: get_auth_request con 0x7f9db0038880 auth_method 0 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dd17f6640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 msgr2=0x7f9dcc02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 0x7f9dcc02b260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800e770 tx=0x7f9dc800ec30 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dc800aba0 con 0x7f9dcc1353c0 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 10 monclient: dump: 2024-09-16T15:11:59.048 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.049 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dd0ff5640 1 --2- 172.21.15.134:0/733156211 >> v2:172.21.15.162:6800/980547263 conn(0x7f9db0038880 0x7f9db003ad40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc073640 tx=0x7f9dbc02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9dc800ad00 con 0x7f9dcc1353c0 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 msgr2=0x7f9dcc02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 0x7f9dcc02b260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800e770 tx=0x7f9dc800ec30 comp rx=0 tx=0).stop 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 msgr2=0x7f9dcc02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.037+0000 7f9dc1ffb640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1353c0 0x7f9dcc02b260 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800e770 tx=0x7f9dc800ec30 comp rx=0 tx=0).stop 2024-09-16T15:11:59.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.044+0000 7f9dd3a5a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:59.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 0x7f9dcc02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd3a5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc07a680 con 0x7f9dcc1210b0 2024-09-16T15:11:59.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 0x7f9dcc02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient: get_auth_request con 0x7f9dcc1210b0 auth_method 0 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 10 monclient(hunting): handle_auth_done global_id 84117 payload 824 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.045+0000 7f9dd17f6640 1 -- 172.21.15.134:0/733156211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9dcc09b490 con 0x7f9dcc1210b0 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.047+0000 7f9dd17f6640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 msgr2=0x7f9dcc02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.047+0000 7f9dd17f6640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 0x7f9dcc02b7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8019000 tx=0x7f9dc80046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9dc8003d10 con 0x7f9dcc1210b0 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 10 monclient: dump: 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.054 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f9dc8023070 con 0x7f9dcc1210b0 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 msgr2=0x7f9dcc02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 0x7f9dcc02b7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8019000 tx=0x7f9dc80046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 msgr2=0x7f9dcc02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dc1ffb640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dcc1210b0 0x7f9dcc02b7a0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8019000 tx=0x7f9dc80046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.055 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:59.056 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.057 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.058 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:59.059 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.060 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.062 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:59.064 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.070 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:59.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.073 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.075 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.076 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.077 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:59.077 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 >> v2:172.21.15.162:6800/980547263 conn(0x7f9db0038880 msgr2=0x7f9db003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/733156211 >> v2:172.21.15.162:6800/980547263 conn(0x7f9db0038880 0x7f9db003ad40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc073640 tx=0x7f9dbc02e000 comp rx=0 tx=0).stop 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 10 monclient: shutdown 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 msgr2=0x7f9dcc1cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 --2- 172.21.15.134:0/733156211 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9dcc137220 0x7f9dcc1cb430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400caf0 tx=0x7f9dc400cef0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 reap_dead start 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd1ff7640 1 -- 172.21.15.134:0/733156211 reap_dead start 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 shutdown_connections 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.048+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 >> 172.21.15.134:0/733156211 conn(0x7f9dcc0a6110 msgr2=0x7f9dcc09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.049+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 shutdown_connections 2024-09-16T15:11:59.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.049+0000 7f9dd3a5a640 1 -- 172.21.15.134:0/733156211 wait complete. 2024-09-16T15:11:59.079 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:59.079 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:11:59.079 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 10 monclient: get_monmap_and_config 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 10 monclient: monmap: 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.177 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:59.177155+0000 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:59.177155+0000 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:59.178 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 1 Processor -- start 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 1 -- start start 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.175+0000 7f5b5ccc9640 10 monclient: init 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient: _add_conns ranks=[10,8,5,6,2,0,4,7,9,3,1] 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b58136e40 0x7f5b5809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-f con 0x7f5b58136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-e con 0x7f5b58137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b58135dc0 0x7f5b58136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-h con 0x7f5b58135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5809abc0 con 0x7f5b58136e40 2024-09-16T15:11:59.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5809ba50 con 0x7f5b58135dc0 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a53c0 con 0x7f5b58137220 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5ccc9640 10 monclient(hunting): authenticate will time out at 1455.766492s 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56886/0 (socket says 172.21.15.134:56886) 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 -- 172.21.15.134:0/3332934473 learned_addr learned my addr 172.21.15.134:0/3332934473 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): get_auth_request con 0x7f5b58137220 auth_method 0 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b58135dc0 0x7f5b58136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3332934473 >> v2:172.21.15.162:3300/0 conn(0x7f5b58136e40 0x7f5b5809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): get_auth_request con 0x7f5b58135dc0 auth_method 0 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): get_auth_request con 0x7f5b58136e40 auth_method 0 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_done global_id 84140 payload 274 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 -- 172.21.15.134:0/3332934473 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b58135dc0 msgr2=0x7f5b58136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b58135dc0 0x7f5b58136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 -- 172.21.15.134:0/3332934473 >> v2:172.21.15.162:3300/0 conn(0x7f5b58136e40 msgr2=0x7f5b5809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3332934473 >> v2:172.21.15.162:3300/0 conn(0x7f5b58136e40 0x7f5b5809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 monclient: found mon.noname-e 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:11:59.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.176+0000 7f5b55d9b640 1 -- 172.21.15.134:0/3332934473 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b58073250 con 0x7f5b58137220 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b55d9b640 1 -- 172.21.15.134:0/3332934473 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 msgr2=0x7f5b58137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b400098a0 tx=0x7f5b4002a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=61c220e718f5644b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 1 -- 172.21.15.134:0/3332934473 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b4002adf0 con 0x7f5b58137220 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: dump: 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.190 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 1 -- 172.21.15.134:0/3332934473 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b400307b0 con 0x7f5b58137220 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5ccc9640 5 monclient: authenticate success, global_id 84140 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5ccc9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 1 -- 172.21.15.134:0/3332934473 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b40030d90 con 0x7f5b58137220 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5559a640 10 monclient: dump: 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.192 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5ccc9640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.179+0000 7f5b5ccc9640 4 set_mon_vals no callback set 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 10 monclient: shutdown 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3332934473 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 msgr2=0x7f5b58137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b400098a0 tx=0x7f5b4002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3332934473 shutdown_connections 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b58135dc0 0x7f5b58136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3332934473 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58137220 0x7f5b58137600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3332934473 >> v2:172.21.15.162:3300/0 conn(0x7f5b58136e40 0x7f5b5809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3332934473 >> 172.21.15.134:0/3332934473 conn(0x7f5b580a6110 msgr2=0x7f5b580a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3332934473 shutdown_connections 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.180+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3332934473 wait complete. 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.181+0000 7f5b5ccc9640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.181+0000 7f5b5ccc9640 10 monclient: monmap: 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.194 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.181+0000 7f5b5ccc9640 1 Processor -- start 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.181+0000 7f5b5ccc9640 1 -- start start 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.181+0000 7f5b5ccc9640 10 monclient: init 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient: _add_conns ranks=[0,8,4,7,6,3,1,5,2] 2024-09-16T15:11:59.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 0x7f5b581cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-a con 0x7f5b58135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58136e40 0x7f5b58134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-e con 0x7f5b58136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 0x7f5b58135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): picked mon.noname-c con 0x7f5b58137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581cb760 con 0x7f5b58135dc0 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58123cc0 con 0x7f5b58137220 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58123570 con 0x7f5b58136e40 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5ccc9640 10 monclient(hunting): authenticate will time out at 1455.772294s 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58136e40 0x7f5b58134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 0x7f5b581cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 0x7f5b581cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58172/0 (socket says 172.21.15.134:58172) 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 1 -- 172.21.15.134:0/3801142811 learned_addr learned my addr 172.21.15.134:0/3801142811 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): get_auth_request con 0x7f5b58135dc0 auth_method 0 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): get_auth_request con 0x7f5b58136e40 auth_method 0 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 0x7f5b58135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): get_auth_request con 0x7f5b58137220 auth_method 0 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient(hunting): handle_auth_done global_id 5395 payload 824 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58136e40 msgr2=0x7f5b58134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58136e40 0x7f5b58134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 msgr2=0x7f5b581cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 0x7f5b581cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 monclient: found mon.noname-c 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b56d9d640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400094d0 con 0x7f5b58137220 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b58135dc0 0x7f5b581cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.182+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5b58136e40 0x7f5b58134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.184+0000 7f5b56d9d640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 msgr2=0x7f5b58135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.184+0000 7f5b56d9d640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 0x7f5b58135490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009d90 tx=0x7f5b4800c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b480148f0 con 0x7f5b58137220 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: dump: 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.199 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 5 monclient: authenticate success, global_id 5395 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800f040 con 0x7f5b58137220 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 10 monclient: _renew_subs 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b580a8a30 con 0x7f5b58137220 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b480133e0 con 0x7f5b58137220 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 10 monclient: dump: 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.201 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 10 monclient: _renew_subs 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b54d99640 4 set_mon_vals no callback set 2024-09-16T15:11:59.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b581363b0 con 0x7f5b58137220 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5b48014480 con 0x7f5b58137220 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.185+0000 7f5b5ccc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 0x7f5b581221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58071b20 con 0x7f5b581255c0 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b4effd640 1 --2- 172.21.15.134:0/3801142811 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b3c0388b0 0x7f5b3c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 0x7f5b581221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient: get_auth_request con 0x7f5b581255c0 auth_method 0 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(516..516 src has 1..516) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f5b4804ae40 con 0x7f5b58137220 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3801142811 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b3c0388b0 0x7f5b3c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b55d9b640 10 monclient: get_auth_request con 0x7f5b3c0388b0 auth_method 0 2024-09-16T15:11:59.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 10 monclient(hunting): handle_auth_done global_id 44744 payload 824 2024-09-16T15:11:59.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.186+0000 7f5b5659c640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b58061ae0 con 0x7f5b581255c0 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b5659c640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 msgr2=0x7f5b581221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 0x7f5b581221f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b5000f610 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b55d9b640 1 --2- 172.21.15.134:0/3801142811 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b3c0388b0 0x7f5b3c03ad70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b58137f00 tx=0x7f5b4002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b5000fe70 con 0x7f5b581255c0 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b4effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.188+0000 7f5b4effd640 10 monclient: dump: 2024-09-16T15:11:59.206 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.207 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5b50004030 con 0x7f5b581255c0 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 msgr2=0x7f5b581221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 0x7f5b581221f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b5000f610 comp rx=0 tx=0).stop 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 msgr2=0x7f5b581221f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.189+0000 7f5b4effd640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b581255c0 0x7f5b581221f0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b5000f610 comp rx=0 tx=0).stop 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5ccc9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 0x7f5b5802b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5ccc9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5807a680 con 0x7f5b5802b260 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 0x7f5b5802b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient: get_auth_request con 0x7f5b5802b260 auth_method 0 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 10 monclient(hunting): handle_auth_done global_id 44753 payload 824 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.195+0000 7f5b5659c640 1 -- 172.21.15.134:0/3801142811 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5b5809b490 con 0x7f5b5802b260 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.197+0000 7f5b5659c640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 msgr2=0x7f5b5802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.197+0000 7f5b5659c640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 0x7f5b5802b670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b500046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5b50003ea0 con 0x7f5b5802b260 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 10 monclient: dump: 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.210 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9527 (secure 0 0 0) 0x7f5b50003a30 con 0x7f5b5802b260 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 msgr2=0x7f5b5802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 0x7f5b5802b670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b500046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 msgr2=0x7f5b5802b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b4effd640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5b5802b260 0x7f5b5802b670 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b50011d10 tx=0x7f5b500046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.211 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:59.212 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.213 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.214 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:59.215 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.216 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.218 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:59.220 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:59.222 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:59.224 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.225 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.231 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:11:59.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:11:59.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:59.234 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:59.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b3c0388b0 msgr2=0x7f5b3c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3801142811 >> v2:172.21.15.162:6800/980547263 conn(0x7f5b3c0388b0 0x7f5b3c03ad70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b58137f00 tx=0x7f5b4002e000 comp rx=0 tx=0).stop 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 10 monclient: shutdown 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 msgr2=0x7f5b58135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b56d9d640 1 -- 172.21.15.134:0/3801142811 reap_dead start 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 0x7f5b58135490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009d90 tx=0x7f5b4800c490 comp rx=0 tx=0).stop 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 shutdown_connections 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 --2- 172.21.15.134:0/3801142811 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5b58137220 0x7f5b58135490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 >> 172.21.15.134:0/3801142811 conn(0x7f5b580a6110 msgr2=0x7f5b5809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 shutdown_connections 2024-09-16T15:11:59.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.198+0000 7f5b5ccc9640 1 -- 172.21.15.134:0/3801142811 wait complete. 2024-09-16T15:11:59.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:59.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:11:59.237 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 10 monclient: get_monmap_and_config 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 10 monclient: monmap: 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:59.335087+0000 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:59.335087+0000 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:59.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 1 Processor -- start 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 1 -- start start 2024-09-16T15:11:59.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.333+0000 7f153035e640 10 monclient: init 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient: _add_conns ranks=[8,5,9,1,2,7,3,6,0,4,10] 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): picked mon.noname-e con 0x7f1528136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 0x7f1528137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): picked mon.noname-h con 0x7f1528137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): picked mon.noname-a con 0x7f1528135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152809abc0 con 0x7f1528135dc0 2024-09-16T15:11:59.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f152809ba50 con 0x7f1528137220 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f15280a53c0 con 0x7f1528136e40 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f153035e640 10 monclient(hunting): authenticate will time out at 1455.924400s 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56914/0 (socket says 172.21.15.134:56914) 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58178/0 (socket says 172.21.15.134:58178) 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 -- 172.21.15.134:0/2082095547 learned_addr learned my addr 172.21.15.134:0/2082095547 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): get_auth_request con 0x7f1528136e40 auth_method 0 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): get_auth_request con 0x7f1528135dc0 auth_method 0 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 0x7f1528137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): get_auth_request con 0x7f1528137220 auth_method 0 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient(hunting): handle_auth_done global_id 84158 payload 274 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 -- 172.21.15.134:0/2082095547 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 msgr2=0x7f1528137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 0x7f1528137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 -- 172.21.15.134:0/2082095547 >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 msgr2=0x7f1528136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2082095547 >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 monclient: found mon.noname-e 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e0fa640 1 -- 172.21.15.134:0/2082095547 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1528073250 con 0x7f1528136e40 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152d8f9640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 0x7f1528137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.334+0000 7f152e8fb640 1 --2- 172.21.15.134:0/2082095547 >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152e0fa640 1 -- 172.21.15.134:0/2082095547 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 msgr2=0x7f152809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15200098a0 tx=0x7f152002a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=243fd646a51aac5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 1 -- 172.21.15.134:0/2082095547 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f152002adf0 con 0x7f1528136e40 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: dump: 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 1 -- 172.21.15.134:0/2082095547 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15200307b0 con 0x7f1528136e40 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f153035e640 5 monclient: authenticate success, global_id 84158 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f153035e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f153035e640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 1 -- 172.21.15.134:0/2082095547 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1520030d90 con 0x7f1528136e40 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f153035e640 4 set_mon_vals no callback set 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.337+0000 7f152d0f8640 10 monclient: dump: 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.353 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 10 monclient: shutdown 2024-09-16T15:11:59.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 -- 172.21.15.134:0/2082095547 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 msgr2=0x7f152809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15200098a0 tx=0x7f152002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 -- 172.21.15.134:0/2082095547 shutdown_connections 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 --2- 172.21.15.134:0/2082095547 >> v2:172.21.15.134:3300/0 conn(0x7f1528135dc0 0x7f1528136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1528137220 0x7f1528137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 --2- 172.21.15.134:0/2082095547 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1528136e40 0x7f152809c150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 -- 172.21.15.134:0/2082095547 >> 172.21.15.134:0/2082095547 conn(0x7f15280a6110 msgr2=0x7f15280a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 -- 172.21.15.134:0/2082095547 shutdown_connections 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.338+0000 7f153035e640 1 -- 172.21.15.134:0/2082095547 wait complete. 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient: monmap: 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:59.355 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 Processor -- start 2024-09-16T15:11:59.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 -- start start 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient: init 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient: _add_conns ranks=[3,7,4,5,0,2,1,6,8] 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1528135dc0 0x7f1528120120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): picked mon.noname-g con 0x7f1528135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1528136e40 0x7f1528135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): picked mon.noname-i con 0x7f1528136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): picked mon.noname-c con 0x7f1528137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1528120eb0 con 0x7f1528135dc0 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1528122ad0 con 0x7f1528137220 2024-09-16T15:11:59.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1528136930 con 0x7f1528136e40 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.339+0000 7f153035e640 10 monclient(hunting): authenticate will time out at 1455.929935s 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55190/0 (socket says 172.21.15.134:55190) 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 learned_addr learned my addr 172.21.15.134:0/2310103416 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1528135dc0 0x7f1528120120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): get_auth_request con 0x7f1528137220 auth_method 0 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1528136e40 0x7f1528135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): get_auth_request con 0x7f1528135dc0 auth_method 0 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): get_auth_request con 0x7f1528136e40 auth_method 0 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152d8f9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient(hunting): handle_auth_done global_id 5404 payload 824 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1528136e40 msgr2=0x7f1528135080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1528136e40 0x7f1528135080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1528135dc0 msgr2=0x7f1528120120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1528135dc0 0x7f1528120120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 monclient: found mon.noname-c 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.340+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518009400 con 0x7f1528137220 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 msgr2=0x7f15281355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f152e8fb640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f151800e930 tx=0x7f151800edf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1518009700 con 0x7f1528137220 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: dump: 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.361 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 5 monclient: authenticate success, global_id 5404 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800cc10 con 0x7f1528137220 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 10 monclient: _renew_subs 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c009a70 con 0x7f1528137220 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f151800cd80 con 0x7f1528137220 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f152c8f7640 4 set_mon_vals no callback set 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f1526ffd640 10 monclient: dump: 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.363 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 10 monclient: _renew_subs 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.342+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200094d0 con 0x7f1528137220 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f153035e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1518013470 con 0x7f1528137220 2024-09-16T15:11:59.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f153035e640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 0x7f15281cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1528075c90 con 0x7f15281221a0 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f1526ffd640 1 --2- 172.21.15.134:0/2310103416 >> v2:172.21.15.162:6800/980547263 conn(0x7f15140388b0 0x7f151403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 0x7f15281cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(516..516 src has 1..516) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f151804a1d0 con 0x7f1528137220 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient: get_auth_request con 0x7f15281221a0 auth_method 0 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152d8f9640 1 --2- 172.21.15.134:0/2310103416 >> v2:172.21.15.162:6800/980547263 conn(0x7f15140388b0 0x7f151403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.343+0000 7f152d8f9640 10 monclient: get_auth_request con 0x7f15140388b0 auth_method 0 2024-09-16T15:11:59.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.344+0000 7f152e0fa640 10 monclient(hunting): handle_auth_done global_id 54511 payload 824 2024-09-16T15:11:59.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.344+0000 7f152e0fa640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1528061ae0 con 0x7f15281221a0 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f152e0fa640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 msgr2=0x7f15281cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 0x7f15281cbcf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1520034000 tx=0x7f15200046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f152d8f9640 1 --2- 172.21.15.134:0/2310103416 >> v2:172.21.15.162:6800/980547263 conn(0x7f15140388b0 0x7f151403ad70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f151c009fd0 tx=0x7f151c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1520030d90 con 0x7f15281221a0 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 10 monclient: dump: 2024-09-16T15:11:59.368 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.369 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1520030920 con 0x7f15281221a0 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 msgr2=0x7f15281cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 0x7f15281cbcf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1520034000 tx=0x7f15200046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 msgr2=0x7f15281cbcf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.346+0000 7f1526ffd640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f15281221a0 0x7f15281cbcf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.352+0000 7f153035e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.352+0000 7f153035e640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 0x7f152802b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.352+0000 7f153035e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.352+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1528071b20 con 0x7f152802b260 2024-09-16T15:11:59.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 0x7f152802b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient: get_auth_request con 0x7f152802b260 auth_method 0 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 10 monclient(hunting): handle_auth_done global_id 54520 payload 824 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.353+0000 7f152e0fa640 1 -- 172.21.15.134:0/2310103416 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f152807a680 con 0x7f152802b260 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.355+0000 7f152e0fa640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 msgr2=0x7f152802b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.355+0000 7f152e0fa640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 0x7f152802b640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f152002a110 tx=0x7f1520008030 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1520049c30 con 0x7f152802b260 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 10 monclient: dump: 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.372 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f152004d070 con 0x7f152802b260 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 msgr2=0x7f152802b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 0x7f152802b640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f152002a110 tx=0x7f1520008030 comp rx=0 tx=0).stop 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 msgr2=0x7f152802b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f1526ffd640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f152802b260 0x7f152802b640 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f152002a110 tx=0x7f1520008030 comp rx=0 tx=0).stop 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:11:59.373 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:11:59.374 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.375 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.376 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:11:59.377 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:11:59.378 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.381 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:11:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.383 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.384 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.385 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:11:59.387 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:11:59.391 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:11:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:11:59.393 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.394 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:11:59.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:11:59.396 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:11:59.396 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 >> v2:172.21.15.162:6800/980547263 conn(0x7f15140388b0 msgr2=0x7f151403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 --2- 172.21.15.134:0/2310103416 >> v2:172.21.15.162:6800/980547263 conn(0x7f15140388b0 0x7f151403ad70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f151c009fd0 tx=0x7f151c00b040 comp rx=0 tx=0).stop 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 10 monclient: shutdown 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 msgr2=0x7f15281355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f151800e930 tx=0x7f151800edf0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f152e8fb640 1 -- 172.21.15.134:0/2310103416 reap_dead start 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 shutdown_connections 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 --2- 172.21.15.134:0/2310103416 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1528137220 0x7f15281355c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 >> 172.21.15.134:0/2310103416 conn(0x7f15280a6110 msgr2=0x7f152809f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 shutdown_connections 2024-09-16T15:11:59.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.356+0000 7f153035e640 1 -- 172.21.15.134:0/2310103416 wait complete. 2024-09-16T15:11:59.398 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:11:59.398 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:11:59.398 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:11:59.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.492+0000 7fc64b448640 10 monclient: get_monmap_and_config 2024-09-16T15:11:59.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.492+0000 7fc64b448640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.492+0000 7fc64b448640 10 monclient: monmap: 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:11:59.494237+0000 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:11:59.494237+0000 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.495 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.496 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:11:59.496 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:11:59.496 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.492+0000 7fc64b448640 1 Processor -- start 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 -- start start 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient: init 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient: _add_conns ranks=[6,9,5,7,0,10,8,1,4,3,2] 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-d con 0x7fc6440a1d30 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc644070e90 0x7fc6440a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-a con 0x7fc644070e90 addr v2:172.21.15.134:3300/0 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6440a2650 0x7fc644134b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-h con 0x7fc6440a2650 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6440a3880 con 0x7fc644070e90 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc644070540 con 0x7fc6440a2650 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440a3a80 con 0x7fc6440a1d30 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc64b448640 10 monclient(hunting): authenticate will time out at 1456.083722s 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52036/0 (socket says 172.21.15.134:52036) 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 learned_addr learned my addr 172.21.15.134:0/3021711652 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): get_auth_request con 0x7fc6440a1d30 auth_method 0 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> v2:172.21.15.134:3300/0 conn(0x7fc644070e90 0x7fc6440a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): get_auth_request con 0x7fc644070e90 auth_method 0 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.493+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6499e5640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6440a2650 0x7fc644134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 10 monclient(hunting): handle_auth_done global_id 35070 payload 274 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6440a2650 msgr2=0x7fc644134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6440a2650 0x7fc644134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 >> v2:172.21.15.134:3300/0 conn(0x7fc644070e90 msgr2=0x7fc6440a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> v2:172.21.15.134:3300/0 conn(0x7fc644070e90 0x7fc6440a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 monclient: found mon.noname-d 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644073250 con 0x7fc6440a1d30 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6499e5640 10 monclient: get_auth_request con 0x7fc6440a2650 auth_method 0 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6499e5640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6440a2650 0x7fc644134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:11:59.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.494+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> v2:172.21.15.134:3300/0 conn(0x7fc644070e90 0x7fc6440a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.496+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 msgr2=0x7fc64406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.496+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a130 comp rx=0 tx=0).ready entity=mon.6 client_cookie=50811da4577f183a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc64002ae90 con 0x7fc6440a1d30 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 msgr2=0x7fc64406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 msgr2=0x7fc64406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a130 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a130 comp rx=0 tx=0).stop 2024-09-16T15:11:59.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: dump: 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.510 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 msgr2=0x7fc64406f550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc6440a1d30 0x7fc64406f550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient: _add_conns ranks=[1,6,4,2,8,5,0,7,3] 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630004560 0x7fc630006970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): picked mon.f con 0x7fc630004560 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 0x7fc6300095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): picked mon.d con 0x7fc630007170 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc630009e50 0x7fc63000c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): picked mon.c con 0x7fc630009e50 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63000cbf0 con 0x7fc630004560 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc63000cd60 con 0x7fc630009e50 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc63000ced0 con 0x7fc630007170 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc637fff640 10 monclient(hunting): ms_handle_reset hunted mon [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 0x7fc6300095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): get_auth_request con 0x7fc630007170 auth_method 0 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc630009e50 0x7fc63000c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): get_auth_request con 0x7fc630009e50 auth_method 0 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630004560 0x7fc630006970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 10 monclient(hunting): get_auth_request con 0x7fc630004560 auth_method 0 2024-09-16T15:11:59.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6491e4640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient(hunting): handle_auth_done global_id 35070 payload 293 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc630009e50 msgr2=0x7fc63000c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc630009e50 0x7fc63000c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630004560 msgr2=0x7fc630006970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc630004560 0x7fc630006970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 monclient: found mon.d 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 1 -- 172.21.15.134:0/3021711652 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=2+}) v3 -- 0x7fc630003f10 con 0x7fc630007170 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc6489e3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc64b448640 5 monclient: authenticate success, global_id 35070 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.497+0000 7fc64b448640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc6489e3640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 msgr2=0x7fc6300095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 0x7fc6300095c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a7270 tx=0x7fc640009240 comp rx=0 tx=0).ready entity=mon.6 client_cookie=d872d7ccd5a3b6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc638009e40 con 0x7fc630007170 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 10 monclient: dump: 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc64b448640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 1 -- 172.21.15.134:0/3021711652 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6380044a0 con 0x7fc630007170 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc637fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc64b448640 10 monclient: get_monmap_and_config success 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.499+0000 7fc64b448640 4 set_mon_vals no callback set 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 10 monclient: shutdown 2024-09-16T15:11:59.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 -- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 msgr2=0x7fc6300095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 --2- 172.21.15.134:0/3021711652 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc630007170 0x7fc6300095c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6440a7270 tx=0x7fc640009240 comp rx=0 tx=0).stop 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc6499e5640 1 -- 172.21.15.134:0/3021711652 reap_dead start 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 -- 172.21.15.134:0/3021711652 shutdown_connections 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 -- 172.21.15.134:0/3021711652 >> 172.21.15.134:0/3021711652 conn(0x7fc64409abf0 msgr2=0x7fc6440a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 -- 172.21.15.134:0/3021711652 shutdown_connections 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.500+0000 7fc64b448640 1 -- 172.21.15.134:0/3021711652 wait complete. 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient: build_initial_monmap 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient: monmap: 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:11:59.517 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 1 Processor -- start 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 1 -- start start 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient: init 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient: _reopen_session rank -1 2024-09-16T15:11:59.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient: _add_conns ranks=[3,5,7,8,4,0,2,1,6] 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-g con 0x7fc630007170 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6441cb9f0 0x7fc64406af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.501+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-h con 0x7fc6441cb9f0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 0x7fc64406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): picked mon.noname-i con 0x7fc64406b4a0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64409d720 con 0x7fc630007170 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6441272c0 con 0x7fc6441cb9f0 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): start opening mon connection 2024-09-16T15:11:59.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc644125da0 con 0x7fc64406b4a0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): _renew_subs 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc64b448640 10 monclient(hunting): authenticate will time out at 1456.092032s 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 0x7fc64406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:33002/0 (socket says 172.21.15.134:33002) 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6441cb9f0 0x7fc64406af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 -- 172.21.15.134:0/3986404373 learned_addr learned my addr 172.21.15.134:0/3986404373 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): get_auth_request con 0x7fc630007170 auth_method 0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): get_auth_request con 0x7fc64406b4a0 auth_method 0 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): get_auth_request con 0x7fc6441cb9f0 auth_method 0 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6489e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient(hunting): handle_auth_done global_id 84135 payload 824 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 -- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 msgr2=0x7fc64406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 0x7fc64406b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 -- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6441cb9f0 msgr2=0x7fc64406af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6441cb9f0 0x7fc64406af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient: _finish_hunting 0 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 monclient: found mon.noname-g 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6491e4640 1 -- 172.21.15.134:0/3986404373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440a6cf0 con 0x7fc630007170 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.502+0000 7fc6499e5640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 0x7fc64406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc6491e4640 1 -- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 msgr2=0x7fc6441cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc6491e4640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc6400046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc640030dc0 con 0x7fc630007170 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: dump: 2024-09-16T15:11:59.522 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: _finish_auth 0 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64002ae90 con 0x7fc630007170 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc640032440 con 0x7fc630007170 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc635ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc635ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc635ffb640 10 monclient: dump: 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:11:59.524 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 5 monclient: authenticate success, global_id 84135 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 10 monclient: _renew_subs 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6440a67a0 con 0x7fc630007170 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.504+0000 7fc6377fe640 4 set_mon_vals no callback set 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 10 monclient: _renew_subs 2024-09-16T15:11:59.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.505+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64406cab0 con 0x7fc630007170 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc640039070 con 0x7fc630007170 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc64b448640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc64b448640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc64b448640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64409e340 con 0x7fc630007170 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc635ffb640 1 --2- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 0x7fc62803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(516..516 src has 1..516) v4 ==== 3943+0+0 (secure 0 0 0) 0x7fc64006a850 con 0x7fc630007170 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 0x7fc62803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.506+0000 7fc6489e3640 10 monclient: get_auth_request con 0x7fc628038880 auth_method 0 2024-09-16T15:11:59.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.508+0000 7fc6489e3640 1 --2- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 0x7fc62803ad40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc630003ee0 tx=0x7fc638008030 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:11:59.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.511+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc640003610 con 0x7fc630007170 2024-09-16T15:11:59.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.511+0000 7fc635ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:11:59.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.511+0000 7fc635ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:11:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.745+0000 7fc64b448640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:11:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.745+0000 7fc64b448640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:11:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.745+0000 7fc64b448640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:11:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.745+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fc644126d00 con 0x7fc630007170 2024-09-16T15:11:59.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.747+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.747+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=2,health=9,kv=12,logm=62} crc {auth=3066388895,config=2586702516,health=2045204522,kv=1090497770,logm=1631225519}) 2024-09-16T15:11:59.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=16,config=1,health=11,kv=12,logm=60} crc {auth=840028059,config=1228700967,health=4008132344,kv=1090497770,logm=3242561074}) 2024-09-16T15:11:59.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.748+0000 7fc635ffb640 1 -- 172.21.15.134:0/3986404373 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fc640037030 con 0x7fc630007170 2024-09-16T15:11:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.748+0000 7fc635ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:11:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.748+0000 7fc635ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=840028059,config=2586702516,health=820269119,kv=1090497770,logm=2752736899}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=12,logm=58} crc {auth=840028059,config=2586702516,health=820269119,kv=1090497770,logm=2046541319}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=15,config=2,health=11,kv=11,logm=61} crc {auth=1529127766,config=2586702516,health=2592800404,kv=3549148692,logm=831563376}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=16,config=2,health=11,kv=11,logm=60} crc {auth=840028059,config=2586702516,health=3418632627,kv=2983865385,logm=2338465103}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=3658104266,config=2586702516,health=820269119,kv=1090497770,logm=2222975767}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=12,logm=59} crc {auth=840028059,config=2586702516,health=3760039052,kv=1090497770,logm=2123416323}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.748+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=1520395282,config=2586702516,health=349125746,kv=1090497770,logm=928118049}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2513721280}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1226674466}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1689224172}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=881831994}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=813203221}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4183180711}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=519998434}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2952110719}) 2024-09-16T15:11:59.757 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:11:59.749+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3048916802}) 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 msgr2=0x7fc62803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 0x7fc62803ad40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc630003ee0 tx=0x7fc638008030 comp rx=0 tx=0).stop 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 10 monclient: shutdown 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 msgr2=0x7fc6441cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc6400046c0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 shutdown_connections 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc64406b4a0 0x7fc64406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc6441cb9f0 0x7fc64406af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> v2:172.21.15.162:6800/980547263 conn(0x7fc628038880 0x7fc62803ad40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 --2- 172.21.15.134:0/3986404373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc630007170 0x7fc6441cb4b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:11:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 >> 172.21.15.134:0/3986404373 conn(0x7fc64409abf0 msgr2=0x7fc64409bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:11:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 shutdown_connections 2024-09-16T15:11:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:11:59.751+0000 7fc64b448640 1 -- 172.21.15.134:0/3986404373 wait complete. 2024-09-16T15:11:59.764 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:12:00.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2625313739}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1579406740}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4286001401}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3837758057}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1472428904}) 2024-09-16T15:12:00.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3908398799}) 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3009773355}) 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=999764375}) 2024-09-16T15:12:00.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.017+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2518869122}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2116048894}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2772095581}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3087459012}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1612447514}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818558311}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=439215496}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=356232510}) 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1774660968}) 2024-09-16T15:12:00.030 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2547357815}) 2024-09-16T15:12:00.030 INFO:tasks.rados.rados.0.smithi162.stdout:2553: done (2 left) 2024-09-16T15:12:00.030 INFO:tasks.rados.rados.0.smithi162.stdout:2554: write oid 1 current snap is 249 2024-09-16T15:12:00.030 INFO:tasks.rados.rados.0.smithi162.stdout:2554: seq_num 903 ranges {776721=479501,2024887=693316} 2024-09-16T15:12:00.034 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.034 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3503466046}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3843292281}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2371477476}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3503990045}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.030+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.035 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3358029780}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2007871947}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1376672797}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2547438756}) 2024-09-16T15:12:00.036 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.031+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1633230168}) 2024-09-16T15:12:00.036 INFO:tasks.rados.rados.0.smithi162.stdout:2554: writing smithi16234619-1 from 776721 to 1256222 tid 1 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2554: writing smithi16234619-1 from 2024887 to 2718203 tid 2 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2555: write (excl) oid 10 current snap is 249 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2552: finishing write tid 1 to smithi16234619-47 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2552: finishing write tid 2 to smithi16234619-47 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2552: finishing write tid 3 to smithi16234619-47 2024-09-16T15:12:00.044 INFO:tasks.rados.rados.0.smithi162.stdout:2555: seq_num 904 ranges {793528=554750,2083369=436974,2958856=234672} 2024-09-16T15:12:00.052 INFO:tasks.rados.rados.0.smithi162.stdout:2555: writing smithi16234619-10 from 793528 to 1348278 tid 1 2024-09-16T15:12:00.059 INFO:tasks.rados.rados.0.smithi162.stdout:2555: writing smithi16234619-10 from 2083369 to 2520343 tid 2 2024-09-16T15:12:00.064 INFO:tasks.rados.rados.0.smithi162.stdout:2555: writing smithi16234619-10 from 2958856 to 3193528 tid 3 2024-09-16T15:12:00.064 INFO:tasks.rados.rados.0.smithi162.stdout:2556: snap_remove snap 227 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=555448611}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1384153462}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1562086691}) 2024-09-16T15:12:00.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3826792759}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4106129199}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4151569401}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2755600762}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1655853996}) 2024-09-16T15:12:00.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.062+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1869374517}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647268334}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=280571047}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1272060215}) 2024-09-16T15:12:00.206 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3753530749}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1727762254}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2574094395}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=625292284}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=839387408}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.198+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1237711687}) 2024-09-16T15:12:00.207 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2331927489}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2176160485}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2523601666}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4074658977}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=937663573}) 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.208 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4283198497}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3331185930}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892896930}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.199+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280340859}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3179549343}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2136820873}) 2024-09-16T15:12:00.209 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1450745299}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1496549286}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797976647}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1246256272}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3829857887}) 2024-09-16T15:12:00.210 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235235920}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.200+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3617168530}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3157038840}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3144383654}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=71385418}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1229786178}) 2024-09-16T15:12:00.211 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546321}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2311774790}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4164554741}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=841746268}) 2024-09-16T15:12:00.212 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.202+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3134172529}) 2024-09-16T15:12:00.233 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.209+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.233 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.209+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.209+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1598408231}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.209+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1721031228}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=697506786}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2024617152}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=447243449}) 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.234 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4282469904}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1440617871}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2727528729}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.210+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3904883960}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1431661348}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1887401040}) 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.235 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1135695232}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=257196907}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1909479637}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4207740957}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.211+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.212+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.212+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3453076231}) 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.212+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.212+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681199070}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.212+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1610027232}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=354060011}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2556921177}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2858914686}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2357803599}) 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2640988689}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3719543516}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1035143946}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543054901}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.213+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2308456384}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3892745154}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=696447766}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1524869924}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1366586978}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1411739493}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2464587681}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.214+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.215+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1623560228}) 2024-09-16T15:12:00.239 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.215+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.215+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432094042}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.215+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2089928076}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=160542767}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3196924756}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=549158795}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3735506966}) 2024-09-16T15:12:00.240 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432851870}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4114999713}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760254812}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3040240567}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.216+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2569670544}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3825431684}) 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.241 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3771718594}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3536565195}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3624046216}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4241052426}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1261027839}) 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.242 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3818228650}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1569889085}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.217+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1754043999}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2072340547}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2556279176}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1980365895}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.218+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.243 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2848505241}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3077977103}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3999616203}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=390212780}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2561731908}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.219+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3753662927}) 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.244 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2887201582}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=912727730}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4281506935}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=973203682}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=34797234}) 2024-09-16T15:12:00.245 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2495625127}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3091149365}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=684619883}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.220+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3083381404}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3691452413}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3728051042}) 2024-09-16T15:12:00.246 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3001736509}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2040118940}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=774595712}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1883899750}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1858430787}) 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.247 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608647725}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.221+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1290546301}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=37244140}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=37985809}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1777293168}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1012347377}) 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.248 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3542569673}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3196510499}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3211100874}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2412047925}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.222+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3268652524}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2142630735}) 2024-09-16T15:12:00.249 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3197791369}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3525256622}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4265945781}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4028478373}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.250 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2253267804}) 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=834759133}) 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2592686401}) 2024-09-16T15:12:00.251 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.224+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=570352800}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=333278719}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2907211406}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.261 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=245589100}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2809096064}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1249611185}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977325098}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3064767131}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1557302208}) 2024-09-16T15:12:00.262 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.232+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4222801995}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=857217498}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2436791710}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1021057864}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=580213530}) 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.263 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=419839463}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=858580729}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=20234751}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3339742031}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.233+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1271135339}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4071206714}) 2024-09-16T15:12:00.264 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=375481463}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2181833501}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1588600802}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=211989734}) 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.265 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.234+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3095532310}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3243657645}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467881702}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.235+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4161308631}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2233609358}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2326884967}) 2024-09-16T15:12:00.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1593320120}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=84493271}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3228106744}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2843261435}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2871030314}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103614493}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.236+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=980667244}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3981783905}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3040097925}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3501152606}) 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1184282412}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3492464160}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=218102340}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1547553397}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611601112}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.237+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1565593748}) 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.269 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=885474815}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1902686969}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3627552422}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3764487271}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4055000790}) 2024-09-16T15:12:00.270 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2073399926}) 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3789299459}) 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2735032961}) 2024-09-16T15:12:00.271 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.238+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1231807979}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1329097009}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1551386286}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1415506079}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3169656735}) 2024-09-16T15:12:00.367 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3142737881}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=917697236}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2011830560}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1160372820}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4224011132}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.368 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1939343910}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1191989665}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2587282768}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1995610182}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=240520645}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3611672311}) 2024-09-16T15:12:00.369 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2456805111}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462865078}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2058157915}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3545363873}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=384275508}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3069502300}) 2024-09-16T15:12:00.370 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3893141032}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1675222518}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4226114149}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3268831550}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=505880839}) 2024-09-16T15:12:00.371 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2593967431}) 2024-09-16T15:12:00.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=856467145}) 2024-09-16T15:12:00.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3885665547}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1670820258}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2071475049}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2737739198}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4089510734}) 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3172550668}) 2024-09-16T15:12:00.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=727827444}) 2024-09-16T15:12:00.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.650+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3745701580}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1614804873}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3710740131}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1415439751}) 2024-09-16T15:12:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=326767491}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2825624488}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=684168902}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2619570428}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809056122}) 2024-09-16T15:12:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:00.674+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=630008026}) 2024-09-16T15:12:00.765 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:12:00.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.858+0000 7f596729c640 10 monclient: get_monmap_and_config 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.858+0000 7f596729c640 10 monclient: build_initial_monmap 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.858+0000 7f596729c640 10 monclient: monmap: 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:00.860683+0000 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:00.860683+0000 2024-09-16T15:12:00.861 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 Processor -- start 2024-09-16T15:12:00.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 -- start start 2024-09-16T15:12:00.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient: init 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient: _add_conns ranks=[10,2,7,9,0,1,5,6,8,3,4] 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5960136080 0x7f5960136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): picked mon.noname-f con 0x7f5960136080 addr v2:172.21.15.162:3300/0 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): picked mon.noname-b con 0x7f59600a5700 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f59600a44c0 0x7f59600a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): picked mon.noname-i con 0x7f59600a44c0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596009bb00 con 0x7f5960136080 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596009b4b0 con 0x7f59600a5700 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f596009c6a0 con 0x7f59600a44c0 2024-09-16T15:12:00.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.859+0000 7f596729c640 10 monclient(hunting): authenticate will time out at 1457.450000s 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54346/0 (socket says 172.21.15.134:54346) 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 -- 172.21.15.134:0/455037194 learned_addr learned my addr 172.21.15.134:0/455037194 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): get_auth_request con 0x7f59600a5700 auth_method 0 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f59600a44c0 0x7f59600a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 1 --2- 172.21.15.134:0/455037194 >> v2:172.21.15.162:3300/0 conn(0x7f5960136080 0x7f5960136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): get_auth_request con 0x7f59600a44c0 auth_method 0 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): get_auth_request con 0x7f5960136080 auth_method 0 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient(hunting): handle_auth_done global_id 44984 payload 274 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 -- 172.21.15.134:0/455037194 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f59600a44c0 msgr2=0x7f59600a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:00.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f59600a44c0 0x7f59600a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 -- 172.21.15.134:0/455037194 >> v2:172.21.15.162:3300/0 conn(0x7f5960136080 msgr2=0x7f5960136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 --2- 172.21.15.134:0/455037194 >> v2:172.21.15.162:3300/0 conn(0x7f5960136080 0x7f5960136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient: _finish_hunting 0 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 monclient: found mon.noname-b 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.860+0000 7f5964837640 1 -- 172.21.15.134:0/455037194 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960073250 con 0x7f59600a5700 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5964837640 1 -- 172.21.15.134:0/455037194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 msgr2=0x7f59600a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5964837640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d0a21dfb444e132 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 1 -- 172.21.15.134:0/455037194 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f595002ae00 con 0x7f59600a5700 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: dump: 2024-09-16T15:12:00.873 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:00.874 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: _finish_auth 0 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 1 -- 172.21.15.134:0/455037194 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500307b0 con 0x7f59600a5700 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 5 monclient: authenticate success, global_id 44984 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:00.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 1 -- 172.21.15.134:0/455037194 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5950030dd0 con 0x7f59600a5700 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 4 set_mon_vals no callback set 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f5957fff640 10 monclient: dump: 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:00.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 10 monclient: shutdown 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 -- 172.21.15.134:0/455037194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 msgr2=0x7f59600a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 -- 172.21.15.134:0/455037194 shutdown_connections 2024-09-16T15:12:00.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f59600a44c0 0x7f59600a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 --2- 172.21.15.134:0/455037194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59600a5700 0x7f59600a3f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 --2- 172.21.15.134:0/455037194 >> v2:172.21.15.162:3300/0 conn(0x7f5960136080 0x7f5960136460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.863+0000 7f596729c640 1 -- 172.21.15.134:0/455037194 >> 172.21.15.134:0/455037194 conn(0x7f59600a99a0 msgr2=0x7f59600a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.864+0000 7f596729c640 1 -- 172.21.15.134:0/455037194 shutdown_connections 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.864+0000 7f596729c640 1 -- 172.21.15.134:0/455037194 wait complete. 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.864+0000 7f596729c640 10 monclient: build_initial_monmap 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.864+0000 7f596729c640 10 monclient: monmap: 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:00.878 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.864+0000 7f596729c640 1 Processor -- start 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 -- start start 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient: init 2024-09-16T15:12:00.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient: _add_conns ranks=[5,6,0,3,2,7,1,8,4] 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f59600a44c0 0x7f5960134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): picked mon.noname-h con 0x7f59600a44c0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): picked mon.noname-d con 0x7f59600a5700 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): picked mon.noname-a con 0x7f5960136080 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5960122070 con 0x7f5960136080 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5960120f40 con 0x7f59600a44c0 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:00.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f59601cc220 con 0x7f59600a5700 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f596729c640 10 monclient(hunting): authenticate will time out at 1457.455488s 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f59600a44c0 0x7f5960134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58206/0 (socket says 172.21.15.134:58206) 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52066/0 (socket says 172.21.15.134:52066) 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 1 -- 172.21.15.134:0/106218638 learned_addr learned my addr 172.21.15.134:0/106218638 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): get_auth_request con 0x7f5960136080 auth_method 0 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): get_auth_request con 0x7f59600a5700 auth_method 0 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): get_auth_request con 0x7f59600a44c0 auth_method 0 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965038640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient(hunting): handle_auth_done global_id 35079 payload 824 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 -- 172.21.15.134:0/106218638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f59600a44c0 msgr2=0x7f5960134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f59600a44c0 0x7f5960134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 -- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 msgr2=0x7f596011fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient: _finish_hunting 0 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 monclient: found mon.noname-d 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5964837640 1 -- 172.21.15.134:0/106218638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500094d0 con 0x7f59600a5700 2024-09-16T15:12:00.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.865+0000 7f5965839640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5964837640 1 -- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 msgr2=0x7f5960135470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5964837640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f596009b5f0 tx=0x7f59500046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5950003610 con 0x7f59600a5700 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: dump: 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:00.886 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:00.887 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: _finish_auth 0 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500037f0 con 0x7f59600a5700 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 5 monclient: authenticate success, global_id 35079 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 10 monclient: _renew_subs 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59601cb7a0 con 0x7f59600a5700 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5950003dd0 con 0x7f59600a5700 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:00.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f5955ffb640 10 monclient: dump: 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:00.889 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 10 monclient: _renew_subs 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f59577fe640 4 set_mon_vals no callback set 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.868+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59600a8f40 con 0x7f59600a5700 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5950039070 con 0x7f59600a5700 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f596729c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f596729c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f596729c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:00.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59601221b0 con 0x7f59600a5700 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f5955ffb640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(517..517 src has 1..517) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f595006a310 con 0x7f59600a5700 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f5965038640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.869+0000 7f5965038640 10 monclient: get_auth_request con 0x7f59440388b0 auth_method 0 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.870+0000 7f5965038640 1 -- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 msgr2=0x7f594403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.870+0000 7f5965038640 1 -- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 msgr2=0x7f594403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.870+0000 7f5965038640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f595c009a20 tx=0x7f595c009190 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.870+0000 7f5965038640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.874+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f59601221b0 con 0x7f59600a5700 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.874+0000 7f5955ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:00.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:00.874+0000 7f5955ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2324018737}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=704651721}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=794574338}) 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2066367467}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2951560339}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712626888}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3434678222}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2359470278}) 2024-09-16T15:12:01.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.020+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1323677390}) 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2870410684}) 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=140760271}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=55840312}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1498609855}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.024+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4162466690}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1024856641}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.029 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1629674462}) 2024-09-16T15:12:01.030 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.030 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812345377}) 2024-09-16T15:12:01.030 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.025+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3930578397}) 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=618082360}) 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=890441775}) 2024-09-16T15:12:01.038 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2008214639}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=683370273}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3557229635}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3668317923}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1718657152}) 2024-09-16T15:12:01.039 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=908608202}) 2024-09-16T15:12:01.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.034+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=820373815}) 2024-09-16T15:12:01.040 INFO:tasks.rados.rados.0.smithi162.stdout:2556: done (4 left) 2024-09-16T15:12:01.040 INFO:tasks.rados.rados.0.smithi162.stdout:2557: write oid 19 current snap is 249 2024-09-16T15:12:01.040 INFO:tasks.rados.rados.0.smithi162.stdout:2557: seq_num 905 ranges {410335=746352,1741851=696985,3102632=107703} 2024-09-16T15:12:01.043 INFO:tasks.rados.rados.0.smithi162.stdout:2557: writing smithi16234619-19 from 410335 to 1156687 tid 1 2024-09-16T15:12:01.051 INFO:tasks.rados.rados.0.smithi162.stdout:2557: writing smithi16234619-19 from 1741851 to 2438836 tid 2 2024-09-16T15:12:01.055 INFO:tasks.rados.rados.0.smithi162.stdout:2557: writing smithi16234619-19 from 3102632 to 3210335 tid 3 2024-09-16T15:12:01.055 INFO:tasks.rados.rados.0.smithi162.stdout:2558: snap_create 2024-09-16T15:12:01.055 INFO:tasks.rados.rados.0.smithi162.stdout:2552: finishing write tid 4 to smithi16234619-47 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: finishing write tid 5 to smithi16234619-47 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: oid 47 updating version 0 to 1113 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: oid 47 updating version 1113 to 1114 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: oid 47 updating version 1114 to 1115 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: oid 47 version 1115 is already newer than 1112 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1115 (ObjNum 902 snap 249 seq_num 902) dirty exists 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2552: left oid 47 (ObjNum 902 snap 249 seq_num 902) 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2551: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2551: finishing copy_from to smithi16234619-15 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 847 (ObjNum 739 snap 194 seq_num 739) dirty exists 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2554: finishing write tid 1 to smithi16234619-1 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2554: finishing write tid 2 to smithi16234619-1 2024-09-16T15:12:01.056 INFO:tasks.rados.rados.0.smithi162.stdout:2554: finishing write tid 3 to smithi16234619-1 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2554: finishing write tid 4 to smithi16234619-1 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2554: oid 1 updating version 0 to 916 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2554: oid 1 updating version 916 to 917 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2554: oid 1 version 917 is already newer than 915 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 917 (ObjNum 903 snap 249 seq_num 903) dirty exists 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2554: left oid 1 (ObjNum 903 snap 249 seq_num 903) 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: finishing write tid 1 to smithi16234619-10 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: finishing write tid 2 to smithi16234619-10 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: finishing write tid 3 to smithi16234619-10 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: finishing write tid 4 to smithi16234619-10 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: finishing write tid 5 to smithi16234619-10 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: oid 10 updating version 0 to 1217 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: oid 10 version 1217 is already newer than 1215 2024-09-16T15:12:01.057 INFO:tasks.rados.rados.0.smithi162.stdout:2555: oid 10 version 1217 is already newer than 1214 2024-09-16T15:12:01.058 INFO:tasks.rados.rados.0.smithi162.stdout:2555: oid 10 version 1217 is already newer than 1216 2024-09-16T15:12:01.058 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1217 (ObjNum 904 snap 249 seq_num 904) dirty exists 2024-09-16T15:12:01.058 INFO:tasks.rados.rados.0.smithi162.stdout:2555: left oid 10 (ObjNum 904 snap 249 seq_num 904) 2024-09-16T15:12:01.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.071+0000 7f5965038640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.071+0000 7f5965038640 10 monclient: get_auth_request con 0x7f59440388b0 auth_method 0 2024-09-16T15:12:01.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.073+0000 7f5965038640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f595c009a20 tx=0x7f595c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.093 INFO:tasks.rados.rados.0.smithi162.stdout:2557: finishing write tid 1 to smithi16234619-19 2024-09-16T15:12:01.095 INFO:tasks.rados.rados.0.smithi162.stdout:2557: finishing write tid 2 to smithi16234619-19 2024-09-16T15:12:01.096 INFO:tasks.rados.rados.0.smithi162.stdout:2557: finishing write tid 3 to smithi16234619-19 2024-09-16T15:12:01.096 INFO:tasks.rados.rados.0.smithi162.stdout:2557: finishing write tid 4 to smithi16234619-19 2024-09-16T15:12:01.096 INFO:tasks.rados.rados.0.smithi162.stdout:2557: finishing write tid 5 to smithi16234619-19 2024-09-16T15:12:01.096 INFO:tasks.rados.rados.0.smithi162.stdout:2557: oid 19 updating version 0 to 1038 2024-09-16T15:12:01.096 INFO:tasks.rados.rados.0.smithi162.stdout:2557: oid 19 updating version 1038 to 1040 2024-09-16T15:12:01.097 INFO:tasks.rados.rados.0.smithi162.stdout:2557: oid 19 updating version 1040 to 1041 2024-09-16T15:12:01.097 INFO:tasks.rados.rados.0.smithi162.stdout:2557: oid 19 version 1041 is already newer than 1039 2024-09-16T15:12:01.097 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1041 (ObjNum 905 snap 249 seq_num 905) dirty exists 2024-09-16T15:12:01.097 INFO:tasks.rados.rados.0.smithi162.stdout:2557: left oid 19 (ObjNum 905 snap 249 seq_num 905) 2024-09-16T15:12:01.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f596729c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:12:01.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f596729c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:01.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f596729c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:01.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f59601221b0 con 0x7f59600a5700 2024-09-16T15:12:01.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f5955ffb640 1 -- 172.21.15.134:0/106218638 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f59601221b0 con 0x7f59600a5700 2024-09-16T15:12:01.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f5955ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:01.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.121+0000 7f5955ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:01.124 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 msgr2=0x7f594403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f595c009a20 tx=0x7f595c008040 comp rx=0 tx=0).stop 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 10 monclient: shutdown 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 msgr2=0x7f5960135470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f596009b5f0 tx=0x7f59500046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 shutdown_connections 2024-09-16T15:12:01.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> v2:172.21.15.162:6800/980547263 conn(0x7f59440388b0 0x7f594403ad70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5960136080 0x7f596011fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f59600a5700 0x7f5960135470 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 --2- 172.21.15.134:0/106218638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f59600a44c0 0x7f5960134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 >> 172.21.15.134:0/106218638 conn(0x7f59600a99a0 msgr2=0x7f5960136460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 shutdown_connections 2024-09-16T15:12:01.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.123+0000 7f596729c640 1 -- 172.21.15.134:0/106218638 wait complete. 2024-09-16T15:12:01.136 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:12:01.136 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:12:01.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:12:01.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:12:01.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.231+0000 7fdbf9220640 10 monclient: get_monmap_and_config 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.231+0000 7fdbf9220640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.231+0000 7fdbf9220640 10 monclient: monmap: 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:01.233511+0000 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:01.233511+0000 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.234 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:01.235 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 Processor -- start 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 -- start start 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient: init 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient: _add_conns ranks=[6,4,7,2,9,5,0,3,10,8,1] 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-d con 0x7fdbf4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 0x7fdbf4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-c con 0x7fdbf4137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdbf4135dc0 0x7fdbf4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-i con 0x7fdbf4135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf409abc0 con 0x7fdbf4137220 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf409ba50 con 0x7fdbf4136e40 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40a53c0 con 0x7fdbf4135dc0 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf9220640 10 monclient(hunting): authenticate will time out at 1457.822688s 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 0x7fdbf4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52078/0 (socket says 172.21.15.134:52078) 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): get_auth_request con 0x7fdbf4137220 auth_method 0 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 1 -- 172.21.15.134:0/460631422 learned_addr learned my addr 172.21.15.134:0/460631422 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request con 0x7fdbf4136e40 auth_method 0 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.232+0000 7fdbf359e640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdbf4135dc0 0x7fdbf4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf359e640 10 monclient(hunting): get_auth_request con 0x7fdbf4135dc0 auth_method 0 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf359e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 10 monclient(hunting): handle_auth_done global_id 5413 payload 274 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 -- 172.21.15.134:0/460631422 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdbf4135dc0 msgr2=0x7fdbf4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdbf4135dc0 0x7fdbf4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 -- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 msgr2=0x7fdbf409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 monclient: found mon.noname-c 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf259c640 1 -- 172.21.15.134:0/460631422 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4073250 con 0x7fdbf4137220 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.233+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.235+0000 7fdbf259c640 1 -- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 msgr2=0x7fdbf4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.235+0000 7fdbf259c640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 0x7fdbf4137600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80097d0 tx=0x7fdbe8029fb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e434c5bef875a26e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 1 -- 172.21.15.134:0/460631422 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe802ad10 con 0x7fdbf4137220 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: dump: 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.244 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 5 monclient: authenticate success, global_id 5413 2024-09-16T15:12:01.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 1 -- 172.21.15.134:0/460631422 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe802ae90 con 0x7fdbf4137220 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 1 -- 172.21.15.134:0/460631422 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe8030c20 con 0x7fdbf4137220 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 4 set_mon_vals no callback set 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf1d9b640 10 monclient: dump: 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.246 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 10 monclient: shutdown 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 -- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 msgr2=0x7fdbf4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 0x7fdbf4137600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80097d0 tx=0x7fdbe8029fb0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 -- 172.21.15.134:0/460631422 shutdown_connections 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdbf4135dc0 0x7fdbf4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdbf4137220 0x7fdbf4137600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 --2- 172.21.15.134:0/460631422 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 -- 172.21.15.134:0/460631422 >> 172.21.15.134:0/460631422 conn(0x7fdbf40a6110 msgr2=0x7fdbf40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 -- 172.21.15.134:0/460631422 shutdown_connections 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.236+0000 7fdbf9220640 1 -- 172.21.15.134:0/460631422 wait complete. 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.237+0000 7fdbf9220640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.237+0000 7fdbf9220640 10 monclient: monmap: 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:01.248 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.237+0000 7fdbf9220640 1 Processor -- start 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.237+0000 7fdbf9220640 1 -- start start 2024-09-16T15:12:01.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.237+0000 7fdbf9220640 10 monclient: init 2024-09-16T15:12:01.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient: _add_conns ranks=[1,6,5,0,8,4,7,3,2] 2024-09-16T15:12:01.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbf4135dc0 0x7fdbf4134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-f con 0x7fdbf4135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-d con 0x7fdbf4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdbf4137220 0x7fdbf4121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): picked mon.noname-h con 0x7fdbf4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40a8a50 con 0x7fdbf4135dc0 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4136890 con 0x7fdbf4137220 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4136340 con 0x7fdbf4136e40 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf9220640 10 monclient(hunting): authenticate will time out at 1457.828317s 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52082/0 (socket says 172.21.15.134:52082) 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 -- 172.21.15.134:0/3662399761 learned_addr learned my addr 172.21.15.134:0/3662399761 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): get_auth_request con 0x7fdbf4136e40 auth_method 0 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdbf4137220 0x7fdbf4121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbf4135dc0 0x7fdbf4134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): get_auth_request con 0x7fdbf4137220 auth_method 0 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request con 0x7fdbf4135dc0 auth_method 0 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf359e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient(hunting): handle_auth_done global_id 35097 payload 824 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdbf4137220 msgr2=0x7fdbf4121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdbf4137220 0x7fdbf4121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbf4135dc0 msgr2=0x7fdbf4134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbf4135dc0 0x7fdbf4134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 monclient: found mon.noname-d 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf259c640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe40094d0 con 0x7fdbf4136e40 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.238+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdbf4135dc0 0x7fdbf4134a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.240+0000 7fdbf259c640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 msgr2=0x7fdbf4134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.240+0000 7fdbf259c640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80096f0 tx=0x7fdbe80046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe8003d60 con 0x7fdbf4136e40 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:12:01.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: dump: 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.262 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe803e8c0 con 0x7fdbf4136e40 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe803eea0 con 0x7fdbf4136e40 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 5 monclient: authenticate success, global_id 35097 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 10 monclient: _renew_subs 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8009400 con 0x7fdbf4136e40 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbdb7fe640 10 monclient: dump: 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.264 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf159a640 4 set_mon_vals no callback set 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 10 monclient: _renew_subs 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.241+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf4125210 con 0x7fdbf4136e40 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf9220640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf9220640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 0x7fdbf40a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4073620 con 0x7fdbf40a1a20 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdbe803ea30 con 0x7fdbf4136e40 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 0x7fdbf40a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient: get_auth_request con 0x7fdbf40a1a20 auth_method 0 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbdb7fe640 1 --2- 172.21.15.134:0/3662399761 >> v2:172.21.15.162:6800/980547263 conn(0x7fdbe0038880 0x7fdbe003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(518..518 src has 1..518) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fdbe806ab00 con 0x7fdbf4136e40 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_done global_id 74484 payload 824 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf2d9d640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbf4061ae0 con 0x7fdbf40a1a20 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf359e640 1 --2- 172.21.15.134:0/3662399761 >> v2:172.21.15.162:6800/980547263 conn(0x7fdbe0038880 0x7fdbe003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.242+0000 7fdbf359e640 10 monclient: get_auth_request con 0x7fdbe0038880 auth_method 0 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbf2d9d640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 msgr2=0x7fdbf40a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 0x7fdbf40a1e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011830 tx=0x7fdbe4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe40044a0 con 0x7fdbf40a1a20 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbdb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.244+0000 7fdbdb7fe640 10 monclient: dump: 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.267 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdbe400fc50 con 0x7fdbf40a1a20 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 msgr2=0x7fdbf40a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 0x7fdbf40a1e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011830 tx=0x7fdbe4011cf0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 msgr2=0x7fdbf40a1e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbdb7fe640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a1a20 0x7fdbf40a1e00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.245+0000 7fdbf359e640 1 --2- 172.21.15.134:0/3662399761 >> v2:172.21.15.162:6800/980547263 conn(0x7fdbe0038880 0x7fdbe003ad40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4123000 tx=0x7fdbec009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf9220640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf9220640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 0x7fdbf4121d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf9220640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4071b20 con 0x7fdbf40a2340 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 0x7fdbf4121d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient: get_auth_request con 0x7fdbf40a2340 auth_method 0 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 10 monclient(hunting): handle_auth_done global_id 74493 payload 824 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.251+0000 7fdbf2d9d640 1 -- 172.21.15.134:0/3662399761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdbf407a680 con 0x7fdbf40a2340 2024-09-16T15:12:01.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbf2d9d640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 msgr2=0x7fdbf4121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbf2d9d640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 0x7fdbf4121d60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011830 tx=0x7fdbe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdbe4024070 con 0x7fdbf40a2340 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbdb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbdb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.253+0000 7fdbdb7fe640 10 monclient: dump: 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.270 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fdbe4003b10 con 0x7fdbf40a2340 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 msgr2=0x7fdbf4121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 0x7fdbf4121d60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011830 tx=0x7fdbe40046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 msgr2=0x7fdbf4121d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbdb7fe640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdbf40a2340 0x7fdbf4121d60 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011830 tx=0x7fdbe40046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:01.272 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.273 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.274 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:01.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.278 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:01.280 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.283 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.284 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:01.285 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.286 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.287 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.289 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:01.291 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.292 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.294 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:01.294 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 >> v2:172.21.15.162:6800/980547263 conn(0x7fdbe0038880 msgr2=0x7fdbe003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 --2- 172.21.15.134:0/3662399761 >> v2:172.21.15.162:6800/980547263 conn(0x7fdbe0038880 0x7fdbe003ad40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4123000 tx=0x7fdbec009820 comp rx=0 tx=0).stop 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 10 monclient: shutdown 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 msgr2=0x7fdbf4134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf359e640 1 -- 172.21.15.134:0/3662399761 reap_dead start 2024-09-16T15:12:01.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80096f0 tx=0x7fdbe80046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 shutdown_connections 2024-09-16T15:12:01.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 --2- 172.21.15.134:0/3662399761 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdbf4136e40 0x7fdbf4134fb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 >> 172.21.15.134:0/3662399761 conn(0x7fdbf40a6110 msgr2=0x7fdbf406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 shutdown_connections 2024-09-16T15:12:01.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.254+0000 7fdbf9220640 1 -- 172.21.15.134:0/3662399761 wait complete. 2024-09-16T15:12:01.296 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:01.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:12:01.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.391+0000 7f9649e0d640 10 monclient: get_monmap_and_config 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.392+0000 7f9649e0d640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.392+0000 7f9649e0d640 10 monclient: monmap: 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:01.393960+0000 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:01.393960+0000 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.394 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:01.395 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.392+0000 7f9649e0d640 1 Processor -- start 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.392+0000 7f9649e0d640 1 -- start start 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.392+0000 7f9649e0d640 10 monclient: init 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient: _add_conns ranks=[6,8,5,7,1,4,0,2,10,3,9] 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-d con 0x7f9644136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-e con 0x7f9644137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 0x7f9644136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-h con 0x7f9644135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:01.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964409abc0 con 0x7f9644135dc0 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f964409ba50 con 0x7f9644136e40 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f96440a53c0 con 0x7f9644137220 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9649e0d640 10 monclient(hunting): authenticate will time out at 1457.983424s 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52088/0 (socket says 172.21.15.134:52088) 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 -- 172.21.15.134:0/683290067 learned_addr learned my addr 172.21.15.134:0/683290067 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56916/0 (socket says 172.21.15.134:56916) 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): get_auth_request con 0x7f9644136e40 auth_method 0 2024-09-16T15:12:01.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): get_auth_request con 0x7f9644137220 auth_method 0 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 0x7f9644136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 10 monclient(hunting): get_auth_request con 0x7f9644135dc0 auth_method 0 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient(hunting): handle_auth_done global_id 35106 payload 274 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 -- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 msgr2=0x7f9644137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 -- 172.21.15.134:0/683290067 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 msgr2=0x7f9644136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 0x7f9644136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 monclient: found mon.noname-d 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:12:01.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f96437fe640 1 -- 172.21.15.134:0/683290067 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644073250 con 0x7f9644136e40 2024-09-16T15:12:01.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9643fff640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 0x7f9644136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:01.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.393+0000 7f9642ffd640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96437fe640 1 -- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 msgr2=0x7f964409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96437fe640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e80 tx=0x7f963402a530 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9e97c59081fa7b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 1 -- 172.21.15.134:0/683290067 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9634007a70 con 0x7f9644136e40 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:12:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: dump: 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.412 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 1 -- 172.21.15.134:0/683290067 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963402d050 con 0x7f9644136e40 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f9649e0d640 5 monclient: authenticate success, global_id 35106 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f9649e0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f9649e0d640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 1 -- 172.21.15.134:0/683290067 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9634007bf0 con 0x7f9644136e40 2024-09-16T15:12:01.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f9649e0d640 4 set_mon_vals no callback set 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.396+0000 7f96427fc640 10 monclient: dump: 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.414 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 10 monclient: shutdown 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 -- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 msgr2=0x7f964409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e80 tx=0x7f963402a530 comp rx=0 tx=0).stop 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 -- 172.21.15.134:0/683290067 shutdown_connections 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9644135dc0 0x7f9644136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644137220 0x7f9644137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 --2- 172.21.15.134:0/683290067 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9644136e40 0x7f964409c150 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 -- 172.21.15.134:0/683290067 >> 172.21.15.134:0/683290067 conn(0x7f96440a6110 msgr2=0x7f96440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 -- 172.21.15.134:0/683290067 shutdown_connections 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 1 -- 172.21.15.134:0/683290067 wait complete. 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.397+0000 7f9649e0d640 10 monclient: monmap: 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 Processor -- start 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 -- start start 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient: init 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient: _add_conns ranks=[1,8,7,5,2,6,4,3,0] 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9644135dc0 0x7f96441cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-f con 0x7f9644135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-e con 0x7f9644136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9644137220 0x7f9644135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): picked mon.noname-i con 0x7f9644137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96441257d0 con 0x7f9644135dc0 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9644125a20 con 0x7f9644137220 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9644136850 con 0x7f9644136e40 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.398+0000 7f9649e0d640 10 monclient(hunting): authenticate will time out at 1457.988986s 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56932/0 (socket says 172.21.15.134:56932) 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 learned_addr learned my addr 172.21.15.134:0/762909571 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): get_auth_request con 0x7f9644136e40 auth_method 0 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9644135dc0 0x7f96441cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): get_auth_request con 0x7f9644135dc0 auth_method 0 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9644137220 0x7f9644135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): get_auth_request con 0x7f9644137220 auth_method 0 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient(hunting): handle_auth_done global_id 84176 payload 824 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9644137220 msgr2=0x7f9644135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9644137220 0x7f9644135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9644135dc0 msgr2=0x7f96441cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9644135dc0 0x7f96441cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 monclient: found mon.noname-e 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.399+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9638009400 con 0x7f9644136e40 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 msgr2=0x7f9644135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f963800e990 tx=0x7f963800ee50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 msgr2=0x7f9644135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 msgr2=0x7f9644135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f963800e990 tx=0x7f963800ee50 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f963800e990 tx=0x7f963800ee50 comp rx=0 tx=0).stop 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 msgr2=0x7f9644135010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9644136e40 0x7f9644135010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient: _add_conns ranks=[3,5,0,4,2,6] 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96300035d0 0x7f9630005a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): picked mon.noname-g con 0x7f96300035d0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9630006570 0x7f9630008a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): picked mon.noname-h con 0x7f9630006570 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 0x7f963000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f96300095a0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963000c650 con 0x7f96300095a0 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963000c790 con 0x7f96300035d0 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f963000c950 con 0x7f9630006570 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.401+0000 7f962bfff640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 0x7f963000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9630006570 0x7f9630008a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): get_auth_request con 0x7f96300095a0 auth_method 0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96300035d0 0x7f9630005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): get_auth_request con 0x7f9630006570 auth_method 0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): get_auth_request con 0x7f96300035d0 auth_method 0 2024-09-16T15:12:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9642ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient(hunting): handle_auth_done global_id 84176 payload 293 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9630006570 msgr2=0x7f9630008a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9630006570 0x7f9630008a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96300035d0 msgr2=0x7f9630005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96300035d0 0x7f9630005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 monclient: found mon.noname-a 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963000cbd0 con 0x7f96300095a0 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.402+0000 7f9643fff640 1 -- 172.21.15.134:0/762909571 reap_dead start 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9643fff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 msgr2=0x7f963000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9643fff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 0x7f963000ba60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f9638009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f963c00c070 con 0x7f96300095a0 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: dump: 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 5 monclient: authenticate success, global_id 84176 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c011d40 con 0x7f96300095a0 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 10 monclient: _renew_subs 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9634009b30 con 0x7f96300095a0 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f963c011eb0 con 0x7f96300095a0 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f962bfff640 10 monclient: dump: 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9641ffb640 4 set_mon_vals no callback set 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 10 monclient: _renew_subs 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.404+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9644136320 con 0x7f96300095a0 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f9649e0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440735d0 con 0x7f96441cb550 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f963c0118d0 con 0x7f96300095a0 2024-09-16T15:12:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient: get_auth_request con 0x7f96441cb550 auth_method 0 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 0x7f963003f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 10 monclient(hunting): handle_auth_done global_id 44993 payload 824 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f96437fe640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9644061ae0 con 0x7f96441cb550 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.405+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(518..518 src has 1..518) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f963c048e40 con 0x7f96300095a0 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.406+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 0x7f963003f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.406+0000 7f9642ffd640 10 monclient: get_auth_request con 0x7f963003d170 auth_method 0 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f96437fe640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 msgr2=0x7f9644135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9634033000 tx=0x7f9634004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9634007000 con 0x7f96441cb550 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 10 monclient: dump: 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.431 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9634003e00 con 0x7f96441cb550 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 msgr2=0x7f9644135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9634033000 tx=0x7f9634004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 msgr2=0x7f9644135010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9634033000 tx=0x7f9634004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:01.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.408+0000 7f9642ffd640 1 --2- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 0x7f963003f630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f963800ee90 tx=0x7f9638008030 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f9649e0d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f9649e0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9644125bb0 con 0x7f9644135550 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient: get_auth_request con 0x7f9644135550 auth_method 0 2024-09-16T15:12:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 10 monclient(hunting): handle_auth_done global_id 45002 payload 824 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.414+0000 7f96437fe640 1 -- 172.21.15.134:0/762909571 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f964407a680 con 0x7f9644135550 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f96437fe640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 msgr2=0x7f9644121a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f96437fe640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e20 tx=0x7f9634009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9634007600 con 0x7f9644135550 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 10 monclient: dump: 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.436 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f9634047a50 con 0x7f9644135550 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 msgr2=0x7f9644121a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e20 tx=0x7f9634009810 comp rx=0 tx=0).stop 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 msgr2=0x7f9644121a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f962bfff640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9634009e20 tx=0x7f9634009810 comp rx=0 tx=0).stop 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:01.437 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.438 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.439 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.440 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:01.441 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:01.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:01.443 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:01.444 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.447 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.448 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.452 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.453 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.455 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.459 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.459 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:01.459 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 msgr2=0x7f963003f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 0x7f963003f630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f963800ee90 tx=0x7f9638008030 comp rx=0 tx=0).stop 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 10 monclient: shutdown 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 msgr2=0x7f963000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 0x7f963000ba60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f963c006fd0 tx=0x7f9638009190 comp rx=0 tx=0).stop 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 shutdown_connections 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> v2:172.21.15.162:6800/980547263 conn(0x7f963003d170 0x7f963003f630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96300095a0 0x7f963000ba60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9644135550 0x7f9644121a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 --2- 172.21.15.134:0/762909571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96441cb550 0x7f9644135010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 >> 172.21.15.134:0/762909571 conn(0x7f96440a6110 msgr2=0x7f964406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 shutdown_connections 2024-09-16T15:12:01.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.418+0000 7f9649e0d640 1 -- 172.21.15.134:0/762909571 wait complete. 2024-09-16T15:12:01.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:01.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:12:01.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:12:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.556+0000 7fa87b11f640 10 monclient: get_monmap_and_config 2024-09-16T15:12:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.556+0000 7fa87b11f640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.556+0000 7fa87b11f640 10 monclient: monmap: 2024-09-16T15:12:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:01.558242+0000 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:01.558242+0000 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.560 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.560 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:01.560 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:01.560 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 Processor -- start 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 -- start start 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient: init 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient: _add_conns ranks=[7,10,1,5,8,6,3,0,9,4,2] 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874136e40 0x7fa87409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-i con 0x7fa874136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa874137220 0x7fa874137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-f con 0x7fa874137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa874135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa874073250 con 0x7fa874135dc0 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87409abc0 con 0x7fa874137220 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa87409ba50 con 0x7fa874136e40 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa87b11f640 10 monclient(hunting): authenticate will time out at 1458.147578s 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa874137220 0x7fa874137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874136e40 0x7fa87409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa8796bc640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa874135dc0 0x7fa874136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:60704/0 (socket says 172.21.15.134:60704) 2024-09-16T15:12:01.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa8796bc640 1 -- 172.21.15.134:0/3404192654 learned_addr learned my addr 172.21.15.134:0/3404192654 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): get_auth_request con 0x7fa874137220 auth_method 0 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): get_auth_request con 0x7fa874136e40 auth_method 0 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.557+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 10 monclient(hunting): handle_auth_done global_id 44866 payload 274 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 1 -- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874136e40 msgr2=0x7fa87409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 1 --2- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874136e40 0x7fa87409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 1 -- 172.21.15.134:0/3404192654 >> v1:172.21.15.162:6789/0 conn(0x7fa874135dc0 legacy=0x7fa874136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 1 monclient: found mon.noname-f 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:12:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.558+0000 7fa873fff640 1 -- 172.21.15.134:0/3404192654 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8740a53c0 con 0x7fa874137220 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.560+0000 7fa873fff640 1 -- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874137220 msgr2=0x7fa874137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.560+0000 7fa873fff640 1 --2- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874137220 0x7fa874137600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097a0 tx=0x7fa86002a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf83b903c3c9806 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 1 -- 172.21.15.134:0/3404192654 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa86002ad30 con 0x7fa874137220 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:12:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: dump: 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 1 -- 172.21.15.134:0/3404192654 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8600307b0 con 0x7fa874137220 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 1 -- 172.21.15.134:0/3404192654 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa860030d90 con 0x7fa874137220 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa8737fe640 10 monclient: dump: 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 5 monclient: authenticate success, global_id 44866 2024-09-16T15:12:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 4 set_mon_vals no callback set 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 10 monclient: shutdown 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 1 -- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874137220 msgr2=0x7fa874137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.561+0000 7fa87b11f640 1 --2- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874137220 0x7fa874137600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa8600097a0 tx=0x7fa86002a010 comp rx=0 tx=0).stop 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 -- 172.21.15.134:0/3404192654 shutdown_connections 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 --2- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa874137220 0x7fa874137600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 --2- 172.21.15.134:0/3404192654 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874136e40 0x7fa87409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 -- 172.21.15.134:0/3404192654 >> 172.21.15.134:0/3404192654 conn(0x7fa8740a6110 msgr2=0x7fa8740a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 -- 172.21.15.134:0/3404192654 shutdown_connections 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 -- 172.21.15.134:0/3404192654 wait complete. 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 10 monclient: monmap: 2024-09-16T15:12:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.577 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 Processor -- start 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 1 -- start start 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.562+0000 7fa87b11f640 10 monclient: init 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient: _add_conns ranks=[7,0,4,6,2,1,8,5,3] 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874135dc0 0x7fa8740a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-i con 0x7fa874135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 0x7fa874134ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-a con 0x7fa874136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa874137220 0x7fa874135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): picked mon.noname-c con 0x7fa874137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8741210b0 con 0x7fa874136e40 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa874136950 con 0x7fa874137220 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa874136390 con 0x7fa874135dc0 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa87b11f640 10 monclient(hunting): authenticate will time out at 1458.153543s 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 0x7fa874134ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa874137220 0x7fa874135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 0x7fa874134ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58248/0 (socket says 172.21.15.134:58248) 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 -- 172.21.15.134:0/107267055 learned_addr learned my addr 172.21.15.134:0/107267055 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): get_auth_request con 0x7fa874136e40 auth_method 0 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): get_auth_request con 0x7fa874137220 auth_method 0 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa8796bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa878ebb640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874135dc0 0x7fa8740a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa878ebb640 10 monclient(hunting): get_auth_request con 0x7fa874135dc0 auth_method 0 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa878ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa878ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa878ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient(hunting): handle_auth_done global_id 74502 payload 824 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874135dc0 msgr2=0x7fa8740a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa874135dc0 0x7fa8740a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa874137220 msgr2=0x7fa874135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa874137220 0x7fa874135430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 monclient: found mon.noname-a 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.563+0000 7fa873fff640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009400 con 0x7fa874136e40 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.564+0000 7fa8796bc640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa874137220 0x7fa874135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa873fff640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 msgr2=0x7fa874134ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa873fff640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 0x7fa874134ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa86002a050 tx=0x7fa8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa860004500 con 0x7fa874136e40 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: dump: 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.582 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860003610 con 0x7fa874136e40 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 5 monclient: authenticate success, global_id 74502 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa860003bf0 con 0x7fa874136e40 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 10 monclient: _renew_subs 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874122010 con 0x7fa874136e40 2024-09-16T15:12:01.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa872ffd640 4 set_mon_vals no callback set 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa8717fa640 10 monclient: dump: 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.584 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 10 monclient: _renew_subs 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.566+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8741212e0 con 0x7fa874136e40 2024-09-16T15:12:01.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa87b11f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa860003d50 con 0x7fa874136e40 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa87b11f640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 0x7fa87402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa874071b20 con 0x7fa8741cbb80 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 0x7fa87402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient: get_auth_request con 0x7fa8741cbb80 auth_method 0 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa8717fa640 1 --2- 172.21.15.134:0/107267055 >> v2:172.21.15.162:6800/980547263 conn(0x7fa858038880 0x7fa85803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(518..518 src has 1..518) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fa86006aab0 con 0x7fa874136e40 2024-09-16T15:12:01.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 10 monclient(hunting): handle_auth_done global_id 5431 payload 824 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa878ebb640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa874061ae0 con 0x7fa8741cbb80 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa8796bc640 1 --2- 172.21.15.134:0/107267055 >> v2:172.21.15.162:6800/980547263 conn(0x7fa858038880 0x7fa85803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.567+0000 7fa8796bc640 10 monclient: get_auth_request con 0x7fa858038880 auth_method 0 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa878ebb640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 msgr2=0x7fa87402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa878ebb640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 0x7fa87402b260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa868011590 tx=0x7fa868011a50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa86800fe70 con 0x7fa8741cbb80 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa8717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa8717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.569+0000 7fa8717fa640 10 monclient: dump: 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.587 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa86800fa00 con 0x7fa8741cbb80 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:01.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 msgr2=0x7fa87402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 0x7fa87402b260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa868011590 tx=0x7fa868011a50 comp rx=0 tx=0).stop 2024-09-16T15:12:01.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 msgr2=0x7fa87402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8717fa640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbb80 0x7fa87402b260 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa868011590 tx=0x7fa868011a50 comp rx=0 tx=0).stop 2024-09-16T15:12:01.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.570+0000 7fa8796bc640 1 --2- 172.21.15.134:0/107267055 >> v2:172.21.15.162:6800/980547263 conn(0x7fa858038880 0x7fa85803ad40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005c00 tx=0x7fa86c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa87b11f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa87b11f640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 0x7fa87402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa87b11f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa874122760 con 0x7fa8741cbf60 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 0x7fa87402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient: get_auth_request con 0x7fa8741cbf60 auth_method 0 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 10 monclient(hunting): handle_auth_done global_id 5440 payload 824 2024-09-16T15:12:01.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.576+0000 7fa878ebb640 1 -- 172.21.15.134:0/107267055 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa87407a680 con 0x7fa8741cbf60 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa878ebb640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 msgr2=0x7fa87402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa878ebb640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 0x7fa87402b7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa868008010 tx=0x7fa8680046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa868024070 con 0x7fa8741cbf60 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa8717fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa8717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.578+0000 7fa8717fa640 10 monclient: dump: 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.592 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7fa868004550 con 0x7fa8741cbf60 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 msgr2=0x7fa87402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 0x7fa87402b7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa868008010 tx=0x7fa8680046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 msgr2=0x7fa87402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8717fa640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa8741cbf60 0x7fa87402b7a0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa868008010 tx=0x7fa8680046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.595 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.596 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:01.597 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.598 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:01.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:01.602 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.605 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.607 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.608 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.609 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.610 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:01.611 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.613 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:01.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:01.615 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:01.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 >> v2:172.21.15.162:6800/980547263 conn(0x7fa858038880 msgr2=0x7fa85803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 --2- 172.21.15.134:0/107267055 >> v2:172.21.15.162:6800/980547263 conn(0x7fa858038880 0x7fa85803ad40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005c00 tx=0x7fa86c009040 comp rx=0 tx=0).stop 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 10 monclient: shutdown 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 msgr2=0x7fa874134ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 --2- 172.21.15.134:0/107267055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa874136e40 0x7fa874134ef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa86002a050 tx=0x7fa8600046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8796bc640 1 -- 172.21.15.134:0/107267055 reap_dead start 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa8796bc640 1 -- 172.21.15.134:0/107267055 reap_dead start 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 shutdown_connections 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 >> 172.21.15.134:0/107267055 conn(0x7fa8740a6110 msgr2=0x7fa87409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 shutdown_connections 2024-09-16T15:12:01.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.579+0000 7fa87b11f640 1 -- 172.21.15.134:0/107267055 wait complete. 2024-09-16T15:12:01.618 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:01.618 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:12:01.618 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4040208386}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=803103889}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1727857198}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=977165953}) 2024-09-16T15:12:01.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=894305826}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3596317795}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=837033910}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3848618588}) 2024-09-16T15:12:01.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.646+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3298505204}) 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 10 monclient: get_monmap_and_config 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 10 monclient: monmap: 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.717 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:01.717122+0000 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:01.717122+0000 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:01.718 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 1 Processor -- start 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 1 -- start start 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.715+0000 7f9390496640 10 monclient: init 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient: _add_conns ranks=[9,7,6,0,10,4,3,8,5,2,1] 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93880a1650 0x7f938809d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): picked mon.noname-a con 0x7f93880a1650 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1a30 0x7f93880a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): picked mon.noname-i con 0x7f93880a1a30 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 0x7f9388133840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): picked mon.noname-d con 0x7f9388133400 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938809b100 con 0x7f93880a1650 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f938809ca60 con 0x7f9388133400 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f938809c780 con 0x7f93880a1a30 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f9390496640 10 monclient(hunting): authenticate will time out at 1458.306501s 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93880a1650 0x7f938809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 0x7f9388133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93880a1650 0x7f938809d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58254/0 (socket says 172.21.15.134:58254) 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 -- 172.21.15.134:0/744204028 learned_addr learned my addr 172.21.15.134:0/744204028 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): get_auth_request con 0x7f93880a1650 auth_method 0 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): get_auth_request con 0x7f9388133400 auth_method 0 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938da31640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1a30 0x7f93880a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient(hunting): handle_auth_done global_id 74511 payload 274 2024-09-16T15:12:01.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 -- 172.21.15.134:0/744204028 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1a30 msgr2=0x7f93880a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1a30 0x7f93880a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 -- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 msgr2=0x7f9388133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 0x7f9388133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 monclient: found mon.noname-a 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938e232640 1 -- 172.21.15.134:0/744204028 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388073250 con 0x7f93880a1650 2024-09-16T15:12:01.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.716+0000 7f938ea33640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 0x7f9388133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.719+0000 7f938e232640 1 -- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93880a1650 msgr2=0x7f938809d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.719+0000 7f938e232640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93880a1650 0x7f938809d540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f937800aa10 tx=0x7f937802b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640b74cf48175fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.719+0000 7f938d230640 1 -- 172.21.15.134:0/744204028 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9378030b90 con 0x7f93880a1650 2024-09-16T15:12:01.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.719+0000 7f938d230640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: dump: 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.732 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 1 -- 172.21.15.134:0/744204028 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937802e050 con 0x7f93880a1650 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 5 monclient: authenticate success, global_id 74511 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:01.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 1 -- 172.21.15.134:0/744204028 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9378030d10 con 0x7f93880a1650 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 4 set_mon_vals no callback set 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f938d230640 10 monclient: dump: 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.734 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 10 monclient: shutdown 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 -- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93880a1650 msgr2=0x7f938809d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93880a1650 0x7f938809d540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f937800aa10 tx=0x7f937802b460 comp rx=0 tx=0).stop 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 -- 172.21.15.134:0/744204028 shutdown_connections 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1a30 0x7f93880a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9388133400 0x7f9388133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 --2- 172.21.15.134:0/744204028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93880a1650 0x7f938809d540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 -- 172.21.15.134:0/744204028 >> 172.21.15.134:0/744204028 conn(0x7f93880a2740 msgr2=0x7f93880a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 -- 172.21.15.134:0/744204028 shutdown_connections 2024-09-16T15:12:01.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.720+0000 7f9390496640 1 -- 172.21.15.134:0/744204028 wait complete. 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 10 monclient: monmap: 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.736 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 1 Processor -- start 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 1 -- start start 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 10 monclient: init 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.721+0000 7f9390496640 10 monclient: _add_conns ranks=[7,1,0,2,3,6,4,8,5] 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1650 0x7f93881caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): picked mon.noname-i con 0x7f93880a1650 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880a1a30 0x7f93881328f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): picked mon.noname-f con 0x7f93880a1a30 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): picked mon.noname-a con 0x7f9388133400 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:01.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93881cb4b0 con 0x7f9388133400 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388123a50 con 0x7f93880a1a30 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93880a55e0 con 0x7f93880a1650 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f9390496640 10 monclient(hunting): authenticate will time out at 1458.312062s 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58268/0 (socket says 172.21.15.134:58268) 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 learned_addr learned my addr 172.21.15.134:0/1813702174 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): get_auth_request con 0x7f9388133400 auth_method 0 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1650 0x7f93881caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880a1a30 0x7f93881328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): get_auth_request con 0x7f93880a1650 auth_method 0 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): get_auth_request con 0x7f93880a1a30 auth_method 0 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938da31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient(hunting): handle_auth_done global_id 74520 payload 824 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1650 msgr2=0x7f93881caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f93880a1650 0x7f93881caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880a1a30 msgr2=0x7f93881328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93880a1a30 0x7f93881328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 monclient: found mon.noname-a 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.722+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009a70 con 0x7f9388133400 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 msgr2=0x7f9388132e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f938ea33640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9380009e60 tx=0x7f938000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9380009cd0 con 0x7f9388133400 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f937effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f937effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.724+0000 7f937effd640 10 monclient: dump: 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.741 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 5 monclient: authenticate success, global_id 74520 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938000f040 con 0x7f9388133400 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 monclient: _renew_subs 2024-09-16T15:12:01.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937800a670 con 0x7f9388133400 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9380014d60 con 0x7f9388133400 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 10 monclient: dump: 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.743 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938ca2f640 4 set_mon_vals no callback set 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 monclient: _renew_subs 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881242f0 con 0x7f9388133400 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 0x7f938811c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9388067450 con 0x7f938811c290 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9380014480 con 0x7f9388133400 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938e232640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 0x7f938811c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938e232640 10 monclient: get_auth_request con 0x7f938811c290 auth_method 0 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938e232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938e232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.725+0000 7f938e232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f937effd640 1 --2- 172.21.15.134:0/1813702174 >> v2:172.21.15.162:6800/980547263 conn(0x7f936c038880 0x7f936c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938e232640 10 monclient(hunting): handle_auth_done global_id 35124 payload 824 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938e232640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9388061ae0 con 0x7f938811c290 2024-09-16T15:12:01.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(518..518 src has 1..518) v4 ==== 3975+0+0 (secure 0 0 0) 0x7f938004a790 con 0x7f9388133400 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938da31640 1 --2- 172.21.15.134:0/1813702174 >> v2:172.21.15.162:6800/980547263 conn(0x7f936c038880 0x7f936c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.726+0000 7f938da31640 10 monclient: get_auth_request con 0x7f936c038880 auth_method 0 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f938e232640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 msgr2=0x7f938811c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f938e232640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 0x7f938811c670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9378035000 tx=0x7f93780046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9378003a80 con 0x7f938811c290 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 10 monclient: dump: 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.746 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9378040070 con 0x7f938811c290 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 msgr2=0x7f938811c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 0x7f938811c670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9378035000 tx=0x7f93780046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 msgr2=0x7f938811c670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f937effd640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811c290 0x7f938811c670 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9378035000 tx=0x7f93780046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.728+0000 7f938da31640 1 --2- 172.21.15.134:0/1813702174 >> v2:172.21.15.162:6800/980547263 conn(0x7f936c038880 0x7f936c03ad40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f93880712b0 tx=0x7f9384008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.734+0000 7f9390496640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.734+0000 7f9390496640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 0x7f938806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.734+0000 7f9390496640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.734+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f93880677b0 con 0x7f938811cbb0 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.734+0000 7f938e232640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 0x7f938806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient: get_auth_request con 0x7f938811cbb0 auth_method 0 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 10 monclient(hunting): handle_auth_done global_id 35133 payload 824 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.735+0000 7f938e232640 1 -- 172.21.15.134:0/1813702174 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f938807a680 con 0x7f938811cbb0 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f938e232640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 msgr2=0x7f938806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f938e232640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 0x7f938806d250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f937802b980 tx=0x7f9378031000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9378049580 con 0x7f938811cbb0 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 10 monclient: dump: 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.751 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7f937804d070 con 0x7f938811cbb0 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 msgr2=0x7f938806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 0x7f938806d250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f937802b980 tx=0x7f9378031000 comp rx=0 tx=0).stop 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 msgr2=0x7f938806d250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:01.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f937effd640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f938811cbb0 0x7f938806d250 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f937802b980 tx=0x7f9378031000 comp rx=0 tx=0).stop 2024-09-16T15:12:01.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3192348079}) 2024-09-16T15:12:01.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.753 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:01.753 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:01.754 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.755 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.756 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:01.757 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:01.758 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.759 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.760 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.762 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:01.763 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.765 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:01.766 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.767 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:01.768 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.769 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:01.770 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.771 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:01.772 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.773 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:01.774 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:01.775 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:01.775 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:01.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:01.775 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:01.775 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=550763218}) 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=487699676}) 2024-09-16T15:12:01.776 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.737+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3492627472}) 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1619990046}) 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=600737612}) 2024-09-16T15:12:01.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 >> v2:172.21.15.162:6800/980547263 conn(0x7f936c038880 msgr2=0x7f936c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.777 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 --2- 172.21.15.134:0/1813702174 >> v2:172.21.15.162:6800/980547263 conn(0x7f936c038880 0x7f936c03ad40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f93880712b0 tx=0x7f9384008040 comp rx=0 tx=0).stop 2024-09-16T15:12:01.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 10 monclient: shutdown 2024-09-16T15:12:01.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 msgr2=0x7f9388132e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.778 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3969220189}) 2024-09-16T15:12:01.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9380009e60 tx=0x7f938000c490 comp rx=0 tx=0).stop 2024-09-16T15:12:01.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f938ea33640 1 -- 172.21.15.134:0/1813702174 reap_dead start 2024-09-16T15:12:01.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495782789}) 2024-09-16T15:12:01.779 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4237686472}) 2024-09-16T15:12:01.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 shutdown_connections 2024-09-16T15:12:01.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 --2- 172.21.15.134:0/1813702174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9388133400 0x7f9388132e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 >> 172.21.15.134:0/1813702174 conn(0x7f93880a2740 msgr2=0x7f93880a4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 shutdown_connections 2024-09-16T15:12:01.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.738+0000 7f9390496640 1 -- 172.21.15.134:0/1813702174 wait complete. 2024-09-16T15:12:01.781 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:01.781 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:12:01.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:12:01.906 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.906 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.906 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2791233860}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3564061301}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2344985019}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1040857099}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1498325055}) 2024-09-16T15:12:01.907 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=563062701}) 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2659731887}) 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=943905753}) 2024-09-16T15:12:01.908 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.903+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3433484880}) 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.909+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.909+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=58512875}) 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3482136863}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=743309740}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2190616670}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1224427782}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=787253946}) 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1272273055}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2586204396}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.910+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2236940509}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2975901997}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=434126855}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2909101839}) 2024-09-16T15:12:01.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1985600675}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=809375026}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3583176727}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.911+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1954436000}) 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.912+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.912+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2756688926}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.912+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=671644917}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986967253}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=678808216}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3910552702}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1044502386}) 2024-09-16T15:12:01.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3829739862}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2430030909}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3787477197}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2134677445}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.913+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1347750411}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3387131936}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1523377296}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.914+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=288261394}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3507400911}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4095414131}) 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2446024876}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=546750888}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379931338}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.915+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=586095992}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=523021541}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1665078648}) 2024-09-16T15:12:01.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=948231325}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3420513282}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3051800412}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2774890072}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1637621899}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1692449818}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.916+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=482086735}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1974681793}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2760696562}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=487069023}) 2024-09-16T15:12:01.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3611506958}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=721177792}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1814992985}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4089946673}) 2024-09-16T15:12:01.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1913398672}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.917+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=465770201}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3933920317}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1874428826}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2337905396}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4010127411}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3613583389}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1301343783}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1796382956}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2541833892}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1640601042}) 2024-09-16T15:12:01.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3387047174}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2632132678}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2156765700}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.918+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1330156637}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.947 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3508892849}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1564611494}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1076014311}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3215447537}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2903937902}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=378669405}) 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.948 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=205696726}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3778809546}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=920366747}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536916645}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=51703138}) 2024-09-16T15:12:01.949 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2589620437}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1675862029}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.919+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4039947624}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131744112}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3804188594}) 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.950 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2541947332}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1844744847}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426625348}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3942103793}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1680036116}) 2024-09-16T15:12:01.951 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=360943440}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.920+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=451440083}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1713103051}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2461795620}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=377004453}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.952 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3920658937}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=402115608}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3543413077}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4085720532}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822412866}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.921+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1484703240}) 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.953 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=578936118}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=540408634}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1383316042}) 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=950890801}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3000210215}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=312325482}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2987469954}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2504740435}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.923+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2562251555}) 2024-09-16T15:12:01.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=163959956}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279731544}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4193927035}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3499118020}) 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1331500224}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3752007928}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3159363149}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3863011008}) 2024-09-16T15:12:01.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.924+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1505131709}) 2024-09-16T15:12:01.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.926+0000 7ff5af908640 10 monclient: get_monmap_and_config 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.926+0000 7ff5af908640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.926+0000 7ff5af908640 10 monclient: monmap: 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:01.928256+0000 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:01.928256+0000 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.958 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:01.959 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1372955884,mon_config_key=3304248118,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=3212303637}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=46} crc {logm=1672574148,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4258366210,mon_config_key=3699183655,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=3896354504}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=1554554616,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1816077598,mon_config_key=2328141726,monmap=2838206317,osd_metadata=1958636322,osd_pg_creating=2886231791,osd_snap=1095581740}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=44} crc {logm=1672574148,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1372955884,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=3313854291}) 2024-09-16T15:12:01.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.926+0000 7ff5af908640 1 Processor -- start 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.926+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=7,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=47} crc {logm=1672574148,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1372955884,mon_config_key=507470931,monmap=2838206317,osd_metadata=1802263926,osd_pg_creating=2886231791,osd_snap=2928814830}) 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=2385455004,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1372955884,mon_config_key=1108115599,monmap=2838206317,osd_metadata=2834326749,osd_pg_creating=2886231791,osd_snap=1522374158}) 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=46} crc {logm=2492773158,mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1372955884,mon_config_key=2856605709,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2700091993}) 2024-09-16T15:12:01.964 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.964 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=7,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=1672574148,mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1623268177,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=2933903774}) 2024-09-16T15:12:01.964 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.927+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=8,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=49} crc {logm=2660004420,mdsmap=1531151339,mgr=2345524120,mgr_metadata=3299114077,mgrstat=1490618205,mon_config_key=4103464827,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=2886231791,osd_snap=3411757531}) 2024-09-16T15:12:01.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 -- start start 2024-09-16T15:12:01.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient: init 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient: _add_conns ranks=[7,2,4,0,3,10,6,5,1,8,9] 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 0x7ff5a806b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-i con 0x7ff5a80a1f60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-b con 0x7ff5a806cd30 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a8137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a80a2880 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806c9a0 con 0x7ff5a806cd30 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806f3b0 con 0x7ff5a80a2880 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806c2e0 con 0x7ff5a80a1f60 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.927+0000 7ff5af908640 10 monclient(hunting): authenticate will time out at 1458.517897s 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54388/0 (socket says 172.21.15.134:54388) 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 1 -- 172.21.15.134:0/412878129 learned_addr learned my addr 172.21.15.134:0/412878129 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): get_auth_request con 0x7ff5a806cd30 auth_method 0 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 0x7ff5a806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a8137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request con 0x7ff5a80a1f60 auth_method 0 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): get_auth_request con 0x7ff5a80a2880 auth_method 0 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.969 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.969 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.969 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=60,osdmap=40} crc {osd_snap=3791916034,osdmap=3684598471}) 2024-09-16T15:12:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_done global_id 54592 payload 274 2024-09-16T15:12:01.970 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.970 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.970 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=62,osdmap=38} crc {osd_snap=3991654359,osdmap=4178092528}) 2024-09-16T15:12:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 msgr2=0x7ff5a8137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a8137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.970 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.971 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 msgr2=0x7ff5a80a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 monclient: found mon.noname-i 2024-09-16T15:12:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.971 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=64,osdmap=36} crc {osd_snap=1734216422,osdmap=3206234248}) 2024-09-16T15:12:01.971 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:12:01.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/412878129 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8073250 con 0x7ff5a80a1f60 2024-09-16T15:12:01.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.928+0000 7ff5acea3640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.973 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.973 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=61,osdmap=39} crc {osd_snap=368009524,osdmap=2389425712}) 2024-09-16T15:12:01.973 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.973 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=64,osdmap=36} crc {osd_snap=111644796,osdmap=2483526222}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=62,osdmap=38} crc {osd_snap=3186013055,osdmap=1047085305}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=1340908714,osdmap=3083237526}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=63,osdmap=37} crc {osd_snap=4285640915,osdmap=1639175943}) 2024-09-16T15:12:01.974 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.928+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=64,osdmap=36} crc {osd_snap=2988236074,osdmap=3271461366}) 2024-09-16T15:12:01.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.931+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 msgr2=0x7ff5a806b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:01.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.931+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 0x7ff5a806b260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009830 tx=0x7ff5a402a000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=bca4c8e85a73db66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:01.975 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.975 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.975 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2831342818}) 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3133202684}) 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=31008791}) 2024-09-16T15:12:01.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 1 -- 172.21.15.134:0/412878129 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a402ad20 con 0x7ff5a80a1f60 2024-09-16T15:12:01.976 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.977 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.977 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1587102079}) 2024-09-16T15:12:01.977 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: dump: 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.979 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: _finish_auth 0 2024-09-16T15:12:01.979 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.979 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1866545955}) 2024-09-16T15:12:01.979 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 1 -- 172.21.15.134:0/412878129 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a40307b0 con 0x7ff5a80a1f60 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 1 -- 172.21.15.134:0/412878129 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a402aea0 con 0x7ff5a80a1f60 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff59ffff640 10 monclient: dump: 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:01.981 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.981 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.982 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=587747546}) 2024-09-16T15:12:01.982 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff5af908640 5 monclient: authenticate success, global_id 54592 2024-09-16T15:12:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff5af908640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.932+0000 7ff5af908640 4 set_mon_vals no callback set 2024-09-16T15:12:01.983 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.983 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=748824445}) 2024-09-16T15:12:01.983 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.983 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=768368214}) 2024-09-16T15:12:01.983 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.932+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1595159467}) 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 10 monclient: shutdown 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 -- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 msgr2=0x7ff5a806b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 0x7ff5a806b260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009830 tx=0x7ff5a402a000 comp rx=0 tx=0).stop 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 -- 172.21.15.134:0/412878129 shutdown_connections 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a8137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5a806cd30 0x7ff5a80a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 --2- 172.21.15.134:0/412878129 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff5a80a1f60 0x7ff5a806b260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 -- 172.21.15.134:0/412878129 >> 172.21.15.134:0/412878129 conn(0x7ff5a809ae30 msgr2=0x7ff5a80a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 -- 172.21.15.134:0/412878129 shutdown_connections 2024-09-16T15:12:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.933+0000 7ff5af908640 1 -- 172.21.15.134:0/412878129 wait complete. 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1797472705}) 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3821368097}) 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1076468767}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3619606331}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4061129110}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=313402949}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2323397164}) 2024-09-16T15:12:01.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4138932924}) 2024-09-16T15:12:01.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.933+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1062049795}) 2024-09-16T15:12:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.934+0000 7ff5af908640 10 monclient: build_initial_monmap 2024-09-16T15:12:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.934+0000 7ff5af908640 10 monclient: monmap: 2024-09-16T15:12:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:01.990 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.934+0000 7ff5af908640 1 Processor -- start 2024-09-16T15:12:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.934+0000 7ff5af908640 1 -- start start 2024-09-16T15:12:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.934+0000 7ff5af908640 10 monclient: init 2024-09-16T15:12:01.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1300543122}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2238565373}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.934+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=343195405}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2714021104}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2364534485}) 2024-09-16T15:12:01.992 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3433661285}) 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3961801854}) 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3893080529}) 2024-09-16T15:12:01.993 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.935+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3060727457}) 2024-09-16T15:12:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient: _add_conns ranks=[6,8,4,3,1,2,7,0,5] 2024-09-16T15:12:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 0x7ff5a81cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-d con 0x7ff5a806cd30 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 0x7ff5a8136c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-e con 0x7ff5a80a1f60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 0x7ff5a81cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 0x7ff5a81cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52134/0 (socket says 172.21.15.134:52134) 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 0x7ff5a8136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55262/0 (socket says 172.21.15.134:55262) 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 0x7ff5a8136c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56942/0 (socket says 172.21.15.134:56942) 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/2387686102 learned_addr learned my addr 172.21.15.134:0/2387686102 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a80a2880 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80a7090 con 0x7ff5a80a2880 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8074bf0 con 0x7ff5a806cd30 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a812ae10 con 0x7ff5a80a1f60 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5af908640 10 monclient(hunting): authenticate will time out at 1458.525856s 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request con 0x7ff5a806cd30 auth_method 0 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 10 monclient(hunting): get_auth_request con 0x7ff5a80a2880 auth_method 0 2024-09-16T15:12:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5adea5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 10 monclient(hunting): get_auth_request con 0x7ff5a80a1f60 auth_method 0 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5acea3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.935+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient(hunting): handle_auth_done global_id 5458 payload 824 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 msgr2=0x7ff5a8136c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 0x7ff5a8136c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 msgr2=0x7ff5a81cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 0x7ff5a81cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient: _finish_hunting 0 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 monclient: found mon.noname-c 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5adea5640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980094d0 con 0x7ff5a80a2880 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff5a806cd30 0x7ff5a81cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.936+0000 7ff5acea3640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a80a1f60 0x7ff5a8136c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4257290297}) 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3869844574}) 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1285802233}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3600568862}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=393977264}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3640486051}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=846180070}) 2024-09-16T15:12:02.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2865616762}) 2024-09-16T15:12:02.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.937+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2543104209}) 2024-09-16T15:12:02.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.938+0000 7ff5adea5640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 msgr2=0x7ff5a81371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.938+0000 7ff5adea5640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000edd0 tx=0x7ff5a000c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a000cd30 con 0x7ff5a80a2880 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:02.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: dump: 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.004 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.006 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 5 monclient: authenticate success, global_id 5458 2024-09-16T15:12:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a000f040 con 0x7ff5a80a2880 2024-09-16T15:12:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2320705987}) 2024-09-16T15:12:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a0014d60 con 0x7ff5a80a2880 2024-09-16T15:12:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 10 monclient: _renew_subs 2024-09-16T15:12:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4009400 con 0x7ff5a80a2880 2024-09-16T15:12:02.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=360570230}) 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59dffb640 10 monclient: dump: 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.009 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff59f7fe640 4 set_mon_vals no callback set 2024-09-16T15:12:02.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3405434213}) 2024-09-16T15:12:02.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3892228418}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2821742345}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1704979710}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.012 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4017554066}) 2024-09-16T15:12:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 10 monclient: _renew_subs 2024-09-16T15:12:02.013 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.939+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a812b3d0 con 0x7ff5a80a2880 2024-09-16T15:12:02.013 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1248540848}) 2024-09-16T15:12:02.013 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.939+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2963146269}) 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.940+0000 7ff5af908640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.940+0000 7ff5af908640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 0x7ff5a802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8073880 con 0x7ff5a811f3c0 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 0x7ff5a802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient: get_auth_request con 0x7ff5a811f3c0 auth_method 0 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_done global_id 84194 payload 824 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.941+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5a8061ae0 con 0x7ff5a811f3c0 2024-09-16T15:12:02.016 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.942+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff5a0014480 con 0x7ff5a80a2880 2024-09-16T15:12:02.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1741568312}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2894988450}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1131111093}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2216793490}) 2024-09-16T15:12:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.942+0000 7ff59dffb640 1 --2- 172.21.15.134:0/2387686102 >> v2:172.21.15.162:6800/980547263 conn(0x7ff58c038880 0x7ff58c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1820981700}) 2024-09-16T15:12:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.942+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(518..518 src has 1..518) v4 ==== 3975+0+0 (secure 0 0 0) 0x7ff5a004acc0 con 0x7ff5a80a2880 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=512805932}) 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1486892552}) 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=631744421}) 2024-09-16T15:12:02.021 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.942+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1792452805}) 2024-09-16T15:12:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.942+0000 7ff5acea3640 1 --2- 172.21.15.134:0/2387686102 >> v2:172.21.15.162:6800/980547263 conn(0x7ff58c038880 0x7ff58c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.942+0000 7ff5acea3640 10 monclient: get_auth_request con 0x7ff58c038880 auth_method 0 2024-09-16T15:12:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.943+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 msgr2=0x7ff5a802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.943+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 0x7ff5a802b260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4035040 tx=0x7ff5a40046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a40044e0 con 0x7ff5a811f3c0 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 10 monclient: dump: 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff5a4003820 con 0x7ff5a811f3c0 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 msgr2=0x7ff5a802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 0x7ff5a802b260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4035040 tx=0x7ff5a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 msgr2=0x7ff5a802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.944+0000 7ff59dffb640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f3c0 0x7ff5a802b260 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4035040 tx=0x7ff5a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=385079833}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3806196682}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.944+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=99} crc {osdmap=1651159515}) 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.027 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=97} crc {osdmap=2746399640}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2550230362}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1911056272}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=856549955}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=486853650}) 2024-09-16T15:12:02.028 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:12:01.945+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=98} crc {osdmap=3037323725}) 2024-09-16T15:12:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.945+0000 7ff5acea3640 1 --2- 172.21.15.134:0/2387686102 >> v2:172.21.15.162:6800/980547263 conn(0x7ff58c038880 0x7ff58c03ad40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8120240 tx=0x7ff598008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5af908640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5af908640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 0x7ff5a802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5af908640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806a3f0 con 0x7ff5a811f7a0 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 0x7ff5a802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient: get_auth_request con 0x7ff5a811f7a0 auth_method 0 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 10 monclient(hunting): handle_auth_done global_id 84203 payload 824 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.951+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/2387686102 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5a807a680 con 0x7ff5a811f7a0 2024-09-16T15:12:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.953+0000 7ff5ad6a4640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 msgr2=0x7ff5a802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.953+0000 7ff5ad6a4640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 0x7ff5a802b7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5a404c070 con 0x7ff5a811f7a0 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 10 monclient: dump: 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.032 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7ff5a4032440 con 0x7ff5a811f7a0 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 msgr2=0x7ff5a802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 0x7ff5a802b7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4008030 comp rx=0 tx=0).stop 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 msgr2=0x7ff5a802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff59dffb640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff5a811f7a0 0x7ff5a802b7a0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4008030 comp rx=0 tx=0).stop 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.039 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.040 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.041 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.042 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.043 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.044 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.045 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:02.046 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.047 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.048 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.049 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.050 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.051 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.052 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.053 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.054 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:02.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:02.056 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:02.057 INFO:tasks.rados.rados.0.smithi162.stdout:2551: done (5 left) 2024-09-16T15:12:02.057 INFO:tasks.rados.rados.0.smithi162.stdout:2552: done (4 left) 2024-09-16T15:12:02.058 INFO:tasks.rados.rados.0.smithi162.stdout:2554: done (3 left) 2024-09-16T15:12:02.058 INFO:tasks.rados.rados.0.smithi162.stdout:2555: done (2 left) 2024-09-16T15:12:02.058 INFO:tasks.rados.rados.0.smithi162.stdout:2557: done (1 left) 2024-09-16T15:12:02.058 INFO:tasks.rados.rados.0.smithi162.stdout:2558: done (0 left) 2024-09-16T15:12:02.058 INFO:tasks.rados.rados.0.smithi162.stdout:2559: snap_remove snap 241 2024-09-16T15:12:02.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 >> v2:172.21.15.162:6800/980547263 conn(0x7ff58c038880 msgr2=0x7ff58c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 --2- 172.21.15.134:0/2387686102 >> v2:172.21.15.162:6800/980547263 conn(0x7ff58c038880 0x7ff58c03ad40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8120240 tx=0x7ff598008040 comp rx=0 tx=0).stop 2024-09-16T15:12:02.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 10 monclient: shutdown 2024-09-16T15:12:02.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 msgr2=0x7ff5a81371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5adea5640 1 -- 172.21.15.134:0/2387686102 reap_dead start 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000edd0 tx=0x7ff5a000c490 comp rx=0 tx=0).stop 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 shutdown_connections 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 --2- 172.21.15.134:0/2387686102 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff5a80a2880 0x7ff5a81371c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 >> 172.21.15.134:0/2387686102 conn(0x7ff5a809ae30 msgr2=0x7ff5a80a0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 shutdown_connections 2024-09-16T15:12:02.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:01.954+0000 7ff5af908640 1 -- 172.21.15.134:0/2387686102 wait complete. 2024-09-16T15:12:02.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:02.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:12:02.060 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:12:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.157+0000 7f8bc58e9640 10 monclient: get_monmap_and_config 2024-09-16T15:12:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.157+0000 7f8bc58e9640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.157+0000 7f8bc58e9640 10 monclient: monmap: 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:02.159380+0000 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:02.159380+0000 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:02.161 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.157+0000 7f8bc58e9640 1 Processor -- start 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 1 -- start start 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient: init 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient: _add_conns ranks=[1,9,0,8,10,2,3,5,7,4,6] 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f8bc0136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8bc0137220 0x7f8bc0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-a con 0x7f8bc0137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8bc013c370 addr v1:172.21.15.134:6789/0 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc0073250 con 0x7f8bc013c370 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc009ba50 con 0x7f8bc0136e40 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc00a56e0 con 0x7f8bc0137220 2024-09-16T15:12:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bc58e9640 10 monclient(hunting): authenticate will time out at 1458.748583s 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8bc0137220 0x7f8bc0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbf7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8bc013c370 0x7f8bc013e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50814/0 (socket says 172.21.15.134:50814) 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/2669119306 learned_addr learned my addr 172.21.15.134:0/2669119306 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): get_auth_request con 0x7f8bc0137220 auth_method 0 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient(hunting): handle_auth_done global_id 74529 payload 274 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 1 -- 172.21.15.134:0/2669119306 >> v1:172.21.15.162:6789/0 conn(0x7f8bc0136e40 legacy=0x7f8bc009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 1 -- 172.21.15.134:0/2669119306 >> v1:172.21.15.134:6789/0 conn(0x7f8bc013c370 legacy=0x7f8bc013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 1 monclient: found mon.noname-a 2024-09-16T15:12:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.158+0000 7f8bbe7fc640 1 -- 172.21.15.134:0/2669119306 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00a53c0 con 0x7f8bc0137220 2024-09-16T15:12:02.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.161+0000 7f8bbe7fc640 1 -- 172.21.15.134:0/2669119306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc0137220 msgr2=0x7f8bc0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.161+0000 7f8bbe7fc640 1 --2- 172.21.15.134:0/2669119306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc0137220 0x7f8bc0137600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097a0 tx=0x7f8bb402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ab93ebc64be3ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 1 -- 172.21.15.134:0/2669119306 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb402ae60 con 0x7f8bc0137220 2024-09-16T15:12:02.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: dump: 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.172 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 5 monclient: authenticate success, global_id 74529 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 1 -- 172.21.15.134:0/2669119306 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40307b0 con 0x7f8bc0137220 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 4 set_mon_vals no callback set 2024-09-16T15:12:02.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 1 -- 172.21.15.134:0/2669119306 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb4030d90 con 0x7f8bc0137220 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bbdffb640 10 monclient: dump: 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.174 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 10 monclient: shutdown 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 -- 172.21.15.134:0/2669119306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc0137220 msgr2=0x7f8bc0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/2669119306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc0137220 0x7f8bc0137600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097a0 tx=0x7f8bb402a060 comp rx=0 tx=0).stop 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 -- 172.21.15.134:0/2669119306 shutdown_connections 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/2669119306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc0137220 0x7f8bc0137600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 -- 172.21.15.134:0/2669119306 >> 172.21.15.134:0/2669119306 conn(0x7f8bc00a6110 msgr2=0x7f8bc00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 -- 172.21.15.134:0/2669119306 shutdown_connections 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.162+0000 7f8bc58e9640 1 -- 172.21.15.134:0/2669119306 wait complete. 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.163+0000 7f8bc58e9640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.163+0000 7f8bc58e9640 10 monclient: monmap: 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.175 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.163+0000 7f8bc58e9640 1 Processor -- start 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.163+0000 7f8bc58e9640 1 -- start start 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.163+0000 7f8bc58e9640 10 monclient: init 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient: _add_conns ranks=[1,3,6,0,5,8,4,7,2] 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136e40 0x7f8bc011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-f con 0x7f8bc0136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc0137220 0x7f8bc01201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-g con 0x7f8bc0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): picked mon.noname-d con 0x7f8bc013c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00a7520 con 0x7f8bc0136e40 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0122790 con 0x7f8bc0137220 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0121000 con 0x7f8bc013c370 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bc58e9640 10 monclient(hunting): authenticate will time out at 1458.754258s 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52142/0 (socket says 172.21.15.134:52142) 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 learned_addr learned my addr 172.21.15.134:0/1976323871 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): get_auth_request con 0x7f8bc013c370 auth_method 0 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136e40 0x7f8bc011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 10 monclient(hunting): get_auth_request con 0x7f8bc0136e40 auth_method 0 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc0137220 0x7f8bc01201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 10 monclient(hunting): get_auth_request con 0x7f8bc0137220 auth_method 0 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient(hunting): handle_auth_done global_id 35151 payload 824 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc0137220 msgr2=0x7f8bc01201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc0137220 0x7f8bc01201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136e40 msgr2=0x7f8bc011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136e40 0x7f8bc011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 monclient: found mon.noname-d 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4009400 con 0x7f8bc013c370 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136e40 0x7f8bc011fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.164+0000 7f8bbe7fc640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc0137220 0x7f8bc01201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.166+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 msgr2=0x7f8bc01206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.166+0000 7f8bbf7fe640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800be70 tx=0x7f8bb8009f60 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb8013810 con 0x7f8bc013c370 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: dump: 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.181 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb80139d0 con 0x7f8bc013c370 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 5 monclient: authenticate success, global_id 35151 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 10 monclient: _renew_subs 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc0122480 con 0x7f8bc013c370 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb8012340 con 0x7f8bc013c370 2024-09-16T15:12:02.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 10 monclient: dump: 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.183 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 10 monclient: _renew_subs 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc013b510 con 0x7f8bc013c370 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bbd7fa640 4 set_mon_vals no callback set 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 0x7f8bc0135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.167+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8bb80133a0 con 0x7f8bc013c370 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0071b20 con 0x7f8bc0135840 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8ba7fff640 1 --2- 172.21.15.134:0/1976323871 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bac0388b0 0x7f8bac03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f8bb8049f90 con 0x7f8bc013c370 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 0x7f8bc0135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient: get_auth_request con 0x7f8bc0135840 auth_method 0 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbe7fc640 1 --2- 172.21.15.134:0/1976323871 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bac0388b0 0x7f8bac03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbe7fc640 10 monclient: get_auth_request con 0x7f8bac0388b0 auth_method 0 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_done global_id 44893 payload 824 2024-09-16T15:12:02.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.168+0000 7f8bbeffd640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bc01228d0 con 0x7f8bc0135840 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.170+0000 7f8bbeffd640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 msgr2=0x7f8bc0135c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.170+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 0x7f8bc0135c20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e4d0 tx=0x7f8bb000e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.170+0000 7f8bbe7fc640 1 --2- 172.21.15.134:0/1976323871 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bac0388b0 0x7f8bac03ad70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006b7b0 tx=0x7f8bb402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb00074a0 con 0x7f8bc0135840 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 10 monclient: dump: 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.188 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8bb0007600 con 0x7f8bc0135840 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:02.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 msgr2=0x7f8bc0135c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 0x7f8bc0135c20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e4d0 tx=0x7f8bb000e990 comp rx=0 tx=0).stop 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 msgr2=0x7f8bc0135c20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.171+0000 7f8ba7fff640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0135840 0x7f8bc0135c20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bc58e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 0x7f8bc009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bc58e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0061ae0 con 0x7f8bc0136160 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 0x7f8bc009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient: get_auth_request con 0x7f8bc0136160 auth_method 0 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 10 monclient(hunting): handle_auth_done global_id 44902 payload 824 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.178+0000 7f8bbeffd640 1 -- 172.21.15.134:0/1976323871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8bc007a680 con 0x7f8bc0136160 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bbeffd640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 msgr2=0x7f8bc009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bbeffd640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 0x7f8bc009d4f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009010 tx=0x7f8bb0001020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8bb0021070 con 0x7f8bc0136160 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 10 monclient: dump: 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f8bb00035f0 con 0x7f8bc0136160 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 msgr2=0x7f8bc009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 0x7f8bc009d4f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009010 tx=0x7f8bb0001020 comp rx=0 tx=0).stop 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 msgr2=0x7f8bc009d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8ba7fff640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc0136160 0x7f8bc009d4f0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009010 tx=0x7f8bb0001020 comp rx=0 tx=0).stop 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:02.193 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 7, 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.194 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.195 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.196 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.197 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:02.198 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.199 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.200 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.201 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:02.202 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.203 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.204 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.205 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.206 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.208 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:02.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.213 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:02.214 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bac0388b0 msgr2=0x7f8bac03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/1976323871 >> v2:172.21.15.162:6800/980547263 conn(0x7f8bac0388b0 0x7f8bac03ad70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8bc006b7b0 tx=0x7f8bb402e000 comp rx=0 tx=0).stop 2024-09-16T15:12:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 10 monclient: shutdown 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bbf7fe640 1 -- 172.21.15.134:0/1976323871 reap_dead start 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 msgr2=0x7f8bc01206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800be70 tx=0x7f8bb8009f60 comp rx=0 tx=0).stop 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 shutdown_connections 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 --2- 172.21.15.134:0/1976323871 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8bc013c370 0x7f8bc01206e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 >> 172.21.15.134:0/1976323871 conn(0x7f8bc00a6110 msgr2=0x7f8bc013b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 shutdown_connections 2024-09-16T15:12:02.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.181+0000 7f8bc58e9640 1 -- 172.21.15.134:0/1976323871 wait complete. 2024-09-16T15:12:02.216 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:02.216 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:12:02.217 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.312+0000 7f2ff6ae9640 10 monclient: get_monmap_and_config 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.313+0000 7f2ff6ae9640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.313+0000 7f2ff6ae9640 10 monclient: monmap: 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:02.314958+0000 2024-09-16T15:12:02.315 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:02.314958+0000 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:02.316 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:02.317 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.313+0000 7f2ff6ae9640 1 Processor -- start 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.313+0000 7f2ff6ae9640 1 -- start start 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.313+0000 7f2ff6ae9640 10 monclient: init 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient: _add_conns ranks=[3,8,2,1,9,6,10,7,0,4,5] 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0136e40 0x7f2ff009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-g con 0x7f2ff0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-e con 0x7f2ff0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-b con 0x7f2ff0135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009abc0 con 0x7f2ff0135dc0 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009ba50 con 0x7f2ff0136e40 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00a53c0 con 0x7f2ff0137220 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff6ae9640 10 monclient(hunting): authenticate will time out at 1458.904296s 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54404/0 (socket says 172.21.15.134:54404) 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56970/0 (socket says 172.21.15.134:56970) 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 -- 172.21.15.134:0/841368592 learned_addr learned my addr 172.21.15.134:0/841368592 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): get_auth_request con 0x7f2ff0135dc0 auth_method 0 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): get_auth_request con 0x7f2ff0137220 auth_method 0 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff4885640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0136e40 0x7f2ff009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff4885640 10 monclient(hunting): get_auth_request con 0x7f2ff0136e40 auth_method 0 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff4885640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff4885640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff4885640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient(hunting): handle_auth_done global_id 45020 payload 274 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 -- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 msgr2=0x7f2ff0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 -- 172.21.15.134:0/841368592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0136e40 msgr2=0x7f2ff009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0136e40 0x7f2ff009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 monclient: found mon.noname-b 2024-09-16T15:12:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2ff5086640 1 -- 172.21.15.134:0/841368592 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0073250 con 0x7f2ff0135dc0 2024-09-16T15:12:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.314+0000 7f2feffff640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff5086640 1 -- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 msgr2=0x7f2ff0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff5086640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80098a0 tx=0x7f2fd802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48883a5b2d3ec04d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 1 -- 172.21.15.134:0/841368592 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd802ade0 con 0x7f2ff0135dc0 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: dump: 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 1 -- 172.21.15.134:0/841368592 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd80307b0 con 0x7f2ff0135dc0 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff6ae9640 5 monclient: authenticate success, global_id 45020 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff6ae9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 1 -- 172.21.15.134:0/841368592 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd8030d90 con 0x7f2ff0135dc0 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2fef7fe640 10 monclient: dump: 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff6ae9640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.317+0000 7f2ff6ae9640 4 set_mon_vals no callback set 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 10 monclient: shutdown 2024-09-16T15:12:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 msgr2=0x7f2ff0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80098a0 tx=0x7f2fd802a080 comp rx=0 tx=0).stop 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/841368592 shutdown_connections 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ff0135dc0 0x7f2ff0136220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2ff0137220 0x7f2ff0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/841368592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0136e40 0x7f2ff009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/841368592 >> 172.21.15.134:0/841368592 conn(0x7f2ff00a6110 msgr2=0x7f2ff00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/841368592 shutdown_connections 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/841368592 wait complete. 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.318+0000 7f2ff6ae9640 10 monclient: monmap: 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 Processor -- start 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 -- start start 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient: init 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient: _add_conns ranks=[6,5,4,1,8,2,7,0,3] 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2ff0135dc0 0x7f2ff0135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-d con 0x7f2ff0135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2ff0136e40 0x7f2ff0121910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-h con 0x7f2ff0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): picked mon.noname-c con 0x7f2ff0137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0124290 con 0x7f2ff0137220 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01cc060 con 0x7f2ff0136e40 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0136880 con 0x7f2ff0135dc0 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.319+0000 7f2ff6ae9640 10 monclient(hunting): authenticate will time out at 1458.909916s 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2ff0135dc0 0x7f2ff0135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55274/0 (socket says 172.21.15.134:55274) 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): get_auth_request con 0x7f2ff0135dc0 auth_method 0 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 learned_addr learned my addr 172.21.15.134:0/1141883232 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): get_auth_request con 0x7f2ff0137220 auth_method 0 2024-09-16T15:12:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2ff0136e40 0x7f2ff0121910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff4885640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): get_auth_request con 0x7f2ff0136e40 auth_method 0 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient(hunting): handle_auth_done global_id 5467 payload 824 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2ff0135dc0 msgr2=0x7f2ff0135120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f2ff0135dc0 0x7f2ff0135120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2ff0136e40 msgr2=0x7f2ff0121910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f2ff0136e40 0x7f2ff0121910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 monclient: found mon.noname-c 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.320+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009a70 con 0x7f2ff0137220 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 msgr2=0x7f2ff0121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff5086640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80090b0 tx=0x7f2fd80046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd8003610 con 0x7f2ff0137220 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: dump: 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8003790 con 0x7f2ff0137220 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 5 monclient: authenticate success, global_id 5467 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fd8003d70 con 0x7f2ff0137220 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2fed7fa640 10 monclient: dump: 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2feeffd640 4 set_mon_vals no callback set 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 10 monclient: _renew_subs 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd80094d0 con 0x7f2ff0137220 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 10 monclient: _renew_subs 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.322+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff01234a0 con 0x7f2ff0137220 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2fd8039040 con 0x7f2ff0137220 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2ff6ae9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2fed7fa640 1 --2- 172.21.15.134:0/1141883232 >> v2:172.21.15.162:6800/980547263 conn(0x7f2fd00388b0 0x7f2fd003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 0x7f2ff0125210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0071b20 con 0x7f2ff0122b50 2024-09-16T15:12:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.323+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f2fd806a650 con 0x7f2ff0137220 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2ff4885640 1 --2- 172.21.15.134:0/1141883232 >> v2:172.21.15.162:6800/980547263 conn(0x7f2fd00388b0 0x7f2fd003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 0x7f2ff0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient: get_auth_request con 0x7f2ff0122b50 auth_method 0 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2ff4885640 10 monclient: get_auth_request con 0x7f2fd00388b0 auth_method 0 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 10 monclient(hunting): handle_auth_done global_id 84162 payload 824 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.324+0000 7f2feffff640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff0061ae0 con 0x7f2ff0122b50 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2ff4885640 1 --2- 172.21.15.134:0/1141883232 >> v2:172.21.15.162:6800/980547263 conn(0x7f2fd00388b0 0x7f2fd003ad70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8006fd0 tx=0x7f2fe8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2feffff640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 msgr2=0x7f2ff0125210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2feffff640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 0x7f2ff0125210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe000f8a0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fe000e040 con 0x7f2ff0122b50 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2fed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2fed7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.326+0000 7f2fed7fa640 10 monclient: dump: 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2fe000f040 con 0x7f2ff0122b50 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 msgr2=0x7f2ff0125210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 0x7f2ff0125210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe000f8a0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 msgr2=0x7f2ff0125210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.327+0000 7f2fed7fa640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff0122b50 0x7f2ff0125210 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe000f8a0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2ff6ae9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 0x7f2ff002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2ff6ae9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff007a680 con 0x7f2ff002b260 2024-09-16T15:12:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 0x7f2ff002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient: get_auth_request con 0x7f2ff002b260 auth_method 0 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.334+0000 7f2feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.335+0000 7f2feffff640 10 monclient(hunting): handle_auth_done global_id 84171 payload 824 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.335+0000 7f2feffff640 1 -- 172.21.15.134:0/1141883232 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2ff009b490 con 0x7f2ff002b260 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2feffff640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 msgr2=0x7f2ff002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2feffff640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 0x7f2ff002b670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe0004a40 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2fe0024070 con 0x7f2ff002b260 2024-09-16T15:12:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 10 monclient: dump: 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9195 (secure 0 0 0) 0x7f2fe001e660 con 0x7f2ff002b260 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 msgr2=0x7f2ff002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 0x7f2ff002b670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe0004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 msgr2=0x7f2ff002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.337+0000 7f2fed7fa640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff002b260 0x7f2ff002b670 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000f3e0 tx=0x7f2fe0004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:02.352 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.353 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:02.355 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.356 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:02.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:02.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:02.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:02.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.365 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.367 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.372 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.374 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.374 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:02.374 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 >> v2:172.21.15.162:6800/980547263 conn(0x7f2fd00388b0 msgr2=0x7f2fd003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/1141883232 >> v2:172.21.15.162:6800/980547263 conn(0x7f2fd00388b0 0x7f2fd003ad70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8006fd0 tx=0x7f2fe8008040 comp rx=0 tx=0).stop 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 10 monclient: shutdown 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 msgr2=0x7f2ff0121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff5086640 1 -- 172.21.15.134:0/1141883232 reap_dead start 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80090b0 tx=0x7f2fd80046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 shutdown_connections 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 --2- 172.21.15.134:0/1141883232 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2ff0137220 0x7f2ff0121e50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 >> 172.21.15.134:0/1141883232 conn(0x7f2ff00a6110 msgr2=0x7f2ff00a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 shutdown_connections 2024-09-16T15:12:02.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.338+0000 7f2ff6ae9640 1 -- 172.21.15.134:0/1141883232 wait complete. 2024-09-16T15:12:02.376 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:02.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:12:02.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:12:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.472+0000 7feed6861640 10 monclient: get_monmap_and_config 2024-09-16T15:12:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.472+0000 7feed6861640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.472+0000 7feed6861640 10 monclient: monmap: 2024-09-16T15:12:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:02.474238+0000 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:02.474238+0000 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:02.476 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 Processor -- start 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 -- start start 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient: init 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient: _add_conns ranks=[0,8,4,3,10,6,9,7,5,1,2] 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feed0136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 0x7feed0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): picked mon.noname-e con 0x7feed0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 0x7feed013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): picked mon.noname-c con 0x7feed013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed0073250 con 0x7feed0136e40 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feed00a56e0 con 0x7feed013c370 2024-09-16T15:12:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7feed009abc0 con 0x7feed0137220 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed6861640 10 monclient(hunting): authenticate will time out at 1459.063753s 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feecffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7feed0136e40 0x7feed009c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:50816/0 (socket says 172.21.15.134:50816) 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed4dfe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 0x7feed013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feecffff640 1 -- 172.21.15.134:0/2223364583 learned_addr learned my addr 172.21.15.134:0/2223364583 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feec7fff640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 0x7feed0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed4dfe640 10 monclient(hunting): get_auth_request con 0x7feed013c370 auth_method 0 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed4dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed4dfe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feed4dfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feec7fff640 10 monclient(hunting): get_auth_request con 0x7feed0137220 auth_method 0 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feec7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feec7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.473+0000 7feec7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient(hunting): handle_auth_done global_id 84221 payload 274 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 1 -- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 msgr2=0x7feed013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 0x7feed013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 1 -- 172.21.15.134:0/2223364583 >> v1:172.21.15.134:6789/0 conn(0x7feed0136e40 legacy=0x7feed009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 1 monclient: found mon.noname-e 2024-09-16T15:12:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feec7fff640 1 -- 172.21.15.134:0/2223364583 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed009ba50 con 0x7feed0137220 2024-09-16T15:12:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.474+0000 7feed4dfe640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 0x7feed013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.476+0000 7feec7fff640 1 -- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 msgr2=0x7feed0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.476+0000 7feec7fff640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 0x7feed0137600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097d0 tx=0x7feeb802a000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a87e44c4d1b32852 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 1 -- 172.21.15.134:0/2223364583 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feeb802ad20 con 0x7feed0137220 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: dump: 2024-09-16T15:12:02.487 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.488 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 5 monclient: authenticate success, global_id 84221 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 1 -- 172.21.15.134:0/2223364583 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb80307b0 con 0x7feed0137220 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 1 -- 172.21.15.134:0/2223364583 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feeb802aea0 con 0x7feed0137220 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 4 set_mon_vals no callback set 2024-09-16T15:12:02.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feecf7fe640 10 monclient: dump: 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.490 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 10 monclient: shutdown 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 -- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 msgr2=0x7feed0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 0x7feed0137600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097d0 tx=0x7feeb802a000 comp rx=0 tx=0).stop 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 -- 172.21.15.134:0/2223364583 shutdown_connections 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feed013c370 0x7feed013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 --2- 172.21.15.134:0/2223364583 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feed0137220 0x7feed0137600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 -- 172.21.15.134:0/2223364583 >> 172.21.15.134:0/2223364583 conn(0x7feed00a6110 msgr2=0x7feed00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 -- 172.21.15.134:0/2223364583 shutdown_connections 2024-09-16T15:12:02.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.477+0000 7feed6861640 1 -- 172.21.15.134:0/2223364583 wait complete. 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.478+0000 7feed6861640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.478+0000 7feed6861640 10 monclient: monmap: 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.492 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.478+0000 7feed6861640 1 Processor -- start 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.478+0000 7feed6861640 1 -- start start 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.478+0000 7feed6861640 10 monclient: init 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient: _add_conns ranks=[7,3,2,0,8,6,5,4,1] 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feed0136e40 0x7feed00a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): picked mon.noname-i con 0x7feed0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feed0137220 0x7feed0135db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): picked mon.noname-g con 0x7feed0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 0x7feed01362f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): picked mon.noname-b con 0x7feed013c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00a7af0 con 0x7feed013c370 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00a75a0 con 0x7feed0137220 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feed01221a0 con 0x7feed0136e40 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed6861640 10 monclient(hunting): authenticate will time out at 1459.069242s 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 0x7feed01362f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 0x7feed01362f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54418/0 (socket says 172.21.15.134:54418) 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 learned_addr learned my addr 172.21.15.134:0/3400604905 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): get_auth_request con 0x7feed013c370 auth_method 0 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feecffff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feed0136e40 0x7feed00a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feecffff640 10 monclient(hunting): get_auth_request con 0x7feed0136e40 auth_method 0 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feec7fff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feed0137220 0x7feed0135db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feec7fff640 10 monclient(hunting): get_auth_request con 0x7feed0137220 auth_method 0 2024-09-16T15:12:02.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feec7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feec7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feec7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient(hunting): handle_auth_done global_id 45029 payload 824 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feed0136e40 msgr2=0x7feed00a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feed0136e40 0x7feed00a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feed0137220 msgr2=0x7feed0135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feed0137220 0x7feed0135db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 monclient: found mon.noname-b 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.479+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009a70 con 0x7feed013c370 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 msgr2=0x7feed01362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed4dfe640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 0x7feed01362f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feec800efb0 tx=0x7feec800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec80148f0 con 0x7feed013c370 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: dump: 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.497 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800f040 con 0x7feed013c370 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed6861640 5 monclient: authenticate success, global_id 45029 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed6861640 10 monclient: _renew_subs 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed6861640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb8009400 con 0x7feed013c370 2024-09-16T15:12:02.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.481+0000 7feecd7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec80133e0 con 0x7feed013c370 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 10 monclient: dump: 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.499 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 10 monclient: _renew_subs 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feeceffd640 4 set_mon_vals no callback set 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed0121ec0 con 0x7feed013c370 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 0x7feed002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feed0071b20 con 0x7feed0120420 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7feec8014480 con 0x7feed013c370 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.482+0000 7feecd7fa640 1 --2- 172.21.15.134:0/3400604905 >> v2:172.21.15.162:6800/980547263 conn(0x7feeb0038880 0x7feeb003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 0x7feed002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient: get_auth_request con 0x7feed0120420 auth_method 0 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7feec804b0a0 con 0x7feed013c370 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feec7fff640 1 --2- 172.21.15.134:0/3400604905 >> v2:172.21.15.162:6800/980547263 conn(0x7feeb0038880 0x7feeb003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feec7fff640 10 monclient: get_auth_request con 0x7feeb0038880 auth_method 0 2024-09-16T15:12:02.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 10 monclient(hunting): handle_auth_done global_id 44816 payload 824 2024-09-16T15:12:02.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.483+0000 7feecffff640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feed0061ae0 con 0x7feed0120420 2024-09-16T15:12:02.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecffff640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 msgr2=0x7feed002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecffff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 0x7feed002b260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec000e9c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feec7fff640 1 --2- 172.21.15.134:0/3400604905 >> v2:172.21.15.162:6800/980547263 conn(0x7feeb0038880 0x7feeb003ad40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097d0 tx=0x7feeb802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec0004e30 con 0x7feed0120420 2024-09-16T15:12:02.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 10 monclient: dump: 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.505 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feec00060d0 con 0x7feed0120420 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 msgr2=0x7feed002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 0x7feed002b260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec000e9c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 msgr2=0x7feed002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.486+0000 7feecd7fa640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120420 0x7feed002b260 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec000e9c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feed6861640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:02.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feed6861640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 0x7feed002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feed6861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feed007a680 con 0x7feed0120800 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 0x7feed002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient: get_auth_request con 0x7feed0120800 auth_method 0 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 10 monclient(hunting): handle_auth_done global_id 44825 payload 824 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.492+0000 7feecffff640 1 -- 172.21.15.134:0/3400604905 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7feed00a53c0 con 0x7feed0120800 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.494+0000 7feecffff640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 msgr2=0x7feed002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.494+0000 7feecffff640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 0x7feed002b7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec0004940 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7feec0013890 con 0x7feed0120800 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 10 monclient: dump: 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.508 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9527 (secure 0 0 0) 0x7feec0013420 con 0x7feed0120800 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 msgr2=0x7feed002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 0x7feed002b7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec0004940 comp rx=0 tx=0).stop 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 msgr2=0x7feed002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feecd7fa640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feed0120800 0x7feed002b7a0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feec000e500 tx=0x7feec0004940 comp rx=0 tx=0).stop 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:02.510 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.511 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.512 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.513 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.514 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.515 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.516 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.517 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:02.518 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.519 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:02.520 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.521 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.522 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.523 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.524 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:02.525 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.530 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.532 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.532 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:02.532 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 >> v2:172.21.15.162:6800/980547263 conn(0x7feeb0038880 msgr2=0x7feeb003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 --2- 172.21.15.134:0/3400604905 >> v2:172.21.15.162:6800/980547263 conn(0x7feeb0038880 0x7feeb003ad40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feeb80097d0 tx=0x7feeb802e000 comp rx=0 tx=0).stop 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 10 monclient: shutdown 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 msgr2=0x7feed01362f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 --2- 172.21.15.134:0/3400604905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed013c370 0x7feed01362f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feec800efb0 tx=0x7feec800c490 comp rx=0 tx=0).stop 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 reap_dead start 2024-09-16T15:12:02.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed4dfe640 1 -- 172.21.15.134:0/3400604905 reap_dead start 2024-09-16T15:12:02.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 shutdown_connections 2024-09-16T15:12:02.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 >> 172.21.15.134:0/3400604905 conn(0x7feed00a6110 msgr2=0x7feed013a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 shutdown_connections 2024-09-16T15:12:02.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.495+0000 7feed6861640 1 -- 172.21.15.134:0/3400604905 wait complete. 2024-09-16T15:12:02.534 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:02.534 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:12:02.534 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.629+0000 7fbe34de2640 10 monclient: get_monmap_and_config 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.629+0000 7fbe34de2640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.629+0000 7fbe34de2640 10 monclient: monmap: 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:02.631693+0000 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:02.631693+0000 2024-09-16T15:12:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:02.633 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 Processor -- start 2024-09-16T15:12:02.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 -- start start 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient: init 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient: _add_conns ranks=[6,4,9,2,5,3,10,0,7,1,8] 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 0x7fbe3009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-d con 0x7fbe30136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-c con 0x7fbe30137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbe30135dc0 0x7fbe30136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-a con 0x7fbe30135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3009abc0 con 0x7fbe30135dc0 2024-09-16T15:12:02.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3009ba50 con 0x7fbe30137220 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300a53c0 con 0x7fbe30136e40 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.630+0000 7fbe34de2640 10 monclient(hunting): authenticate will time out at 1459.220992s 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 0x7fbe3009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbe30135dc0 0x7fbe30136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55302/0 (socket says 172.21.15.134:55302) 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request con 0x7fbe30136e40 auth_method 0 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 -- 172.21.15.134:0/626316140 learned_addr learned my addr 172.21.15.134:0/626316140 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): get_auth_request con 0x7fbe30137220 auth_method 0 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request con 0x7fbe30135dc0 auth_method 0 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.640 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient(hunting): handle_auth_done global_id 5485 payload 274 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 -- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 msgr2=0x7fbe3009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 0x7fbe3009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 -- 172.21.15.134:0/626316140 >> v2:172.21.15.134:3300/0 conn(0x7fbe30135dc0 msgr2=0x7fbe30136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 --2- 172.21.15.134:0/626316140 >> v2:172.21.15.134:3300/0 conn(0x7fbe30135dc0 0x7fbe30136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 monclient: found mon.noname-c 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe27fff640 1 -- 172.21.15.134:0/626316140 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30073250 con 0x7fbe30137220 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.631+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 0x7fbe3009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe27fff640 1 -- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 msgr2=0x7fbe30137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe27fff640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbe180098a0 tx=0x7fbe1802a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=cd64aefe64aa343d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 1 -- 172.21.15.134:0/626316140 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe1802adf0 con 0x7fbe30137220 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: dump: 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.642 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 5 monclient: authenticate success, global_id 5485 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 1 -- 172.21.15.134:0/626316140 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe180307b0 con 0x7fbe30137220 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 4 set_mon_vals no callback set 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 1 -- 172.21.15.134:0/626316140 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe18030d90 con 0x7fbe30137220 2024-09-16T15:12:02.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe2dd9b640 10 monclient: dump: 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.644 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 10 monclient: shutdown 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 -- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 msgr2=0x7fbe30137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbe180098a0 tx=0x7fbe1802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 -- 172.21.15.134:0/626316140 shutdown_connections 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30137220 0x7fbe30137600 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 --2- 172.21.15.134:0/626316140 >> v2:172.21.15.134:3300/0 conn(0x7fbe30135dc0 0x7fbe30136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 --2- 172.21.15.134:0/626316140 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30136e40 0x7fbe3009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.634+0000 7fbe34de2640 1 -- 172.21.15.134:0/626316140 >> 172.21.15.134:0/626316140 conn(0x7fbe300a6110 msgr2=0x7fbe300a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.635+0000 7fbe34de2640 1 -- 172.21.15.134:0/626316140 shutdown_connections 2024-09-16T15:12:02.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.635+0000 7fbe34de2640 1 -- 172.21.15.134:0/626316140 wait complete. 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.635+0000 7fbe34de2640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.635+0000 7fbe34de2640 10 monclient: monmap: 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.646 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.647 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.647 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.647 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.647 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.647 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 Processor -- start 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 -- start start 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient: init 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient: _add_conns ranks=[4,1,6,3,0,8,7,5,2] 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-c con 0x7fbe30135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30136e40 0x7fbe30134f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-f con 0x7fbe30136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe30135460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): picked mon.noname-d con 0x7fbe30137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30125090 con 0x7fbe30136e40 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301235d0 con 0x7fbe30135dc0 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30136340 con 0x7fbe30137220 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe34de2640 10 monclient(hunting): authenticate will time out at 1459.226574s 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55304/0 (socket says 172.21.15.134:55304) 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 learned_addr learned my addr 172.21.15.134:0/391217905 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request con 0x7fbe30135dc0 auth_method 0 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe30135460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request con 0x7fbe30137220 auth_method 0 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30136e40 0x7fbe30134f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_done global_id 5494 payload 824 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 msgr2=0x7fbe30135460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe30135460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30136e40 msgr2=0x7fbe30134f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30136e40 0x7fbe30134f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 monclient: found mon.noname-c 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20009400 con 0x7fbe30135dc0 2024-09-16T15:12:02.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe27fff640 10 monclient: get_auth_request con 0x7fbe30136e40 auth_method 0 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe30135460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.636+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe30136e40 0x7fbe30134f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ec10 tx=0x7fbe2000c520 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe20009760 con 0x7fbe30135dc0 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ec10 tx=0x7fbe2000c520 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000ec10 tx=0x7fbe2000c520 comp rx=0 tx=0).stop 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: dump: 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.659 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 msgr2=0x7fbe301cb220 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe30135dc0 0x7fbe301cb220 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient: _add_conns ranks=[2,3,6,5,7,0,4,1,8] 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 0x7fbe10006a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): picked mon.b con 0x7fbe100045f0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe100071e0 0x7fbe10009650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): picked mon.g con 0x7fbe100071e0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe1000c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): picked mon.d con 0x7fbe30137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1000cb40 con 0x7fbe100045f0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1000ccb0 con 0x7fbe100071e0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1000ce20 con 0x7fbe30137220 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe26ffd640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 0x7fbe10006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request con 0x7fbe100045f0 auth_method 0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe1000c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request con 0x7fbe30137220 auth_method 0 2024-09-16T15:12:02.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe100071e0 0x7fbe10009650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient(hunting): handle_auth_done global_id 5494 payload 293 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 msgr2=0x7fbe1000c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe1000c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe100071e0 msgr2=0x7fbe10009650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe100071e0 0x7fbe10009650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 monclient: found mon.b 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=2+}) v3 -- 0x7fbe10003f40 con 0x7fbe100045f0 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe27fff640 10 monclient: get_auth_request con 0x7fbe100071e0 auth_method 0 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe100071e0 0x7fbe10009650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:02.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 5 monclient: authenticate success, global_id 5494 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe30137220 0x7fbe1000c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 10 monclient: _renew_subs 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe180094d0 con 0x7fbe100045f0 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 10 monclient: _renew_subs 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.639+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30123b50 con 0x7fbe100045f0 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe34de2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 0x7fbe30122160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30071b20 con 0x7fbe30135460 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 0x7fbe30122160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient: get_auth_request con 0x7fbe30135460 auth_method 0 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.640+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.641+0000 7fbe27fff640 10 monclient(hunting): handle_auth_done global_id 54610 payload 824 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.641+0000 7fbe27fff640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe30061ae0 con 0x7fbe30135460 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe2e59c640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 msgr2=0x7fbe10006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe2e59c640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 0x7fbe10006a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe20019000 tx=0x7fbe20009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe2002a070 con 0x7fbe100045f0 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:12:02.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 10 monclient: dump: 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.667 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe20022040 con 0x7fbe100045f0 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe2d59a640 4 set_mon_vals no callback set 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.642+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbe20013700 con 0x7fbe100045f0 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 0x7fbe10043b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.2 v2:172.21.15.134:3301/0 4 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fbe2004baa0 con 0x7fbe100045f0 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 0x7fbe10043b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe2ed9d640 10 monclient: get_auth_request con 0x7fbe10041640 auth_method 0 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe27fff640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 msgr2=0x7fbe30122160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 0x7fbe30122160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe180046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe18030d90 con 0x7fbe30135460 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 10 monclient: dump: 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbe18030920 con 0x7fbe30135460 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 msgr2=0x7fbe30122160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 0x7fbe30122160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe180046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 msgr2=0x7fbe30122160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30135460 0x7fbe30122160 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe18034000 tx=0x7fbe180046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.643+0000 7fbe2ed9d640 1 -- 172.21.15.134:0/391217905 reap_dead start 2024-09-16T15:12:02.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.645+0000 7fbe2ed9d640 1 --2- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 0x7fbe10043b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe30123100 tx=0x7fbe2800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe34de2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe34de2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3007a680 con 0x7fbe30125490 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient: get_auth_request con 0x7fbe30125490 auth_method 0 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 10 monclient(hunting): handle_auth_done global_id 54619 payload 824 2024-09-16T15:12:02.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.650+0000 7fbe27fff640 1 -- 172.21.15.134:0/391217905 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbe3009b490 con 0x7fbe30125490 2024-09-16T15:12:02.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe27fff640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe27fff640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe18000c00 tx=0x7fbe18009240 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe1804c070 con 0x7fbe30125490 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 10 monclient: dump: 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.673 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9361 (secure 0 0 0) 0x7fbe18050410 con 0x7fbe30125490 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe18000c00 tx=0x7fbe18009240 comp rx=0 tx=0).stop 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 msgr2=0x7fbe301cb220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:02.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe26ffd640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe18000c00 tx=0x7fbe18009240 comp rx=0 tx=0).stop 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 98, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:02.675 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 8, 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.676 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.677 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:02.678 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:02.679 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:02.680 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.682 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.683 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:02.684 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:02.686 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:02.689 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.692 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:02.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:02.696 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:02.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 msgr2=0x7fbe10043b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 0x7fbe10043b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbe30123100 tx=0x7fbe2800b040 comp rx=0 tx=0).stop 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 10 monclient: shutdown 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 msgr2=0x7fbe10006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.653+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 0x7fbe10006a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe20019000 tx=0x7fbe20009190 comp rx=0 tx=0).stop 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 shutdown_connections 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe10041640 0x7fbe10043b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe100045f0 0x7fbe10006a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 --2- 172.21.15.134:0/391217905 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbe30125490 0x7fbe301cb220 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 >> 172.21.15.134:0/391217905 conn(0x7fbe300a6110 msgr2=0x7fbe3009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 shutdown_connections 2024-09-16T15:12:02.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.654+0000 7fbe34de2640 1 -- 172.21.15.134:0/391217905 wait complete. 2024-09-16T15:12:02.699 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:12:02.699 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'd'] 2024-09-16T15:12:02.699 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:12:02.699 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-16T15:12:02.699 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:12:02.699 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:12:02.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:12:02.711 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:12:02.711 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-16T15:12:02.711 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-16T15:12:02.711 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-16T15:12:02.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:12:02.765 INFO:tasks.ceph.mon.d:Stopped 2024-09-16T15:12:02.765 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-09-16T15:12:02.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:12:02.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.899+0000 7f170cd3e640 10 monclient: get_monmap_and_config 2024-09-16T15:12:02.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.899+0000 7f170cd3e640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.899+0000 7f170cd3e640 10 monclient: monmap: 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:02.901670+0000 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:02.901670+0000 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.903 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 Processor -- start 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 -- start start 2024-09-16T15:12:02.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient: init 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient: _add_conns ranks=[5,10,9,2,0,6,1,4,8,3,7] 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1708066040 0x7f1708070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-h con 0x7f1708066040 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f17080a1550 0x7f17080a1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-f con 0x7f17080a1550 addr v2:172.21.15.162:3300/0 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17080a1eb0 0x7f170813a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-a con 0x7f17080a1eb0 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1708066420 con 0x7f17080a1eb0 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170806fd40 con 0x7f17080a1550 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1708070020 con 0x7f1708066040 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.900+0000 7f170cd3e640 10 monclient(hunting): authenticate will time out at 1459.490982s 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17080a1eb0 0x7f170813a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17080a1eb0 0x7f170813a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58340/0 (socket says 172.21.15.134:58340) 2024-09-16T15:12:02.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 -- 172.21.15.134:0/4057395669 learned_addr learned my addr 172.21.15.134:0/4057395669 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): get_auth_request con 0x7f17080a1eb0 auth_method 0 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 1 --2- 172.21.15.134:0/4057395669 >> v2:172.21.15.162:3300/0 conn(0x7f17080a1550 0x7f17080a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1708066040 0x7f1708070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): get_auth_request con 0x7f17080a1550 auth_method 0 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): get_auth_request con 0x7f1708066040 auth_method 0 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1705d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f170659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient(hunting): handle_auth_done global_id 74547 payload 274 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 -- 172.21.15.134:0/4057395669 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1708066040 msgr2=0x7f1708070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1708066040 0x7f1708070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 -- 172.21.15.134:0/4057395669 >> v2:172.21.15.162:3300/0 conn(0x7f17080a1550 msgr2=0x7f17080a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:02.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 --2- 172.21.15.134:0/4057395669 >> v2:172.21.15.162:3300/0 conn(0x7f17080a1550 0x7f17080a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 monclient: found mon.noname-a 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.901+0000 7f1706d9d640 1 -- 172.21.15.134:0/4057395669 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708073250 con 0x7f17080a1eb0 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f1706d9d640 1 -- 172.21.15.134:0/4057395669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17080a1eb0 msgr2=0x7f170813a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f1706d9d640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17080a1eb0 0x7f170813a340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17080a00b0 tx=0x7f170002c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41afd407f657c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 1 -- 172.21.15.134:0/4057395669 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1700009a70 con 0x7f17080a1eb0 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: dump: 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.914 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170cd3e640 5 monclient: authenticate success, global_id 74547 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170cd3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 1 -- 172.21.15.134:0/4057395669 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1700009bf0 con 0x7f17080a1eb0 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170cd3e640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170cd3e640 4 set_mon_vals no callback set 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 1 -- 172.21.15.134:0/4057395669 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1700009d60 con 0x7f17080a1eb0 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.904+0000 7f170559a640 10 monclient: dump: 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.916 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 10 monclient: shutdown 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 -- 172.21.15.134:0/4057395669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17080a1eb0 msgr2=0x7f170813a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17080a1eb0 0x7f170813a340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17080a00b0 tx=0x7f170002c780 comp rx=0 tx=0).stop 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 -- 172.21.15.134:0/4057395669 shutdown_connections 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17080a1eb0 0x7f170813a340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 --2- 172.21.15.134:0/4057395669 >> v2:172.21.15.162:3300/0 conn(0x7f17080a1550 0x7f17080a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 --2- 172.21.15.134:0/4057395669 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1708066040 0x7f1708070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 -- 172.21.15.134:0/4057395669 >> 172.21.15.134:0/4057395669 conn(0x7f17080a6230 msgr2=0x7f17080a2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 -- 172.21.15.134:0/4057395669 shutdown_connections 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 1 -- 172.21.15.134:0/4057395669 wait complete. 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 10 monclient: build_initial_monmap 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.905+0000 7f170cd3e640 10 monclient: monmap: 2024-09-16T15:12:02.918 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:02.919 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 Processor -- start 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 -- start start 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient: init 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient: _add_conns ranks=[6,2,8,5,7,3,1,0,4] 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 0x7f17081c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-d con 0x7f1708066040 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 0x7f17081c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-b con 0x7f17080a1550 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): picked mon.noname-e con 0x7f17080a1eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:02.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17081a4050 con 0x7f17080a1550 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f17081a3e30 con 0x7f1708066040 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f17081a55c0 con 0x7f17080a1eb0 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170cd3e640 10 monclient(hunting): authenticate will time out at 1459.496761s 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170659c640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 msgr2=0x7f17081c73a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f170659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 0x7f17081c73a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1705d9b640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 msgr2=0x7f17081c78e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1705d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 0x7f17081c78e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:56996/0 (socket says 172.21.15.134:56996) 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 1 -- 172.21.15.134:0/3704529777 learned_addr learned my addr 172.21.15.134:0/3704529777 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): get_auth_request con 0x7f17080a1eb0 auth_method 0 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.906+0000 7f1706d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 10 monclient(hunting): handle_auth_done global_id 84230 payload 824 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 -- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 msgr2=0x7f17081c73a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 0x7f17081c73a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 -- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 msgr2=0x7f17081c78e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:02.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 0x7f17081c78e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:02.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:12:02.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 monclient: found mon.noname-e 2024-09-16T15:12:02.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:02.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:02.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.907+0000 7f1706d9d640 1 -- 172.21.15.134:0/3704529777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170000bd40 con 0x7f17080a1eb0 2024-09-16T15:12:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f1706d9d640 1 -- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 msgr2=0x7f17081a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f1706d9d640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f170002cca0 tx=0x7f1700004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1700003db0 con 0x7f17080a1eb0 2024-09-16T15:12:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: dump: 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.926 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17000044b0 con 0x7f17080a1eb0 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 5 monclient: authenticate success, global_id 84230 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 10 monclient: _renew_subs 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1708070520 con 0x7f17080a1eb0 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:02.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1700004620 con 0x7f17080a1eb0 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f16f6ffd640 10 monclient: dump: 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:02.928 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 10 monclient: _renew_subs 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17081a52e0 con 0x7f17080a1eb0 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.909+0000 7f1704d99640 4 set_mon_vals no callback set 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1700004040 con 0x7f17080a1eb0 2024-09-16T15:12:02.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f170cd3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f170cd3e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f170cd3e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170806edf0 con 0x7f17080a1eb0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f16f6ffd640 1 --2- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 0x7f16e803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.910+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f170006d450 con 0x7f17080a1eb0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.911+0000 7f170659c640 1 --2- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 0x7f16e803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.911+0000 7f170659c640 10 monclient: get_auth_request con 0x7f16e8038880 auth_method 0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.913+0000 7f170659c640 1 --2- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 0x7f16e803ad40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16f80094c0 tx=0x7f16f8009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.915+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f17000042f0 con 0x7f17080a1eb0 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.915+0000 7f16f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:02.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:02.915+0000 7f16f6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:03.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.140+0000 7f170cd3e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:12:03.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.140+0000 7f170cd3e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:03.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.140+0000 7f170cd3e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:03.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.140+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f17081a3260 con 0x7f17080a1eb0 2024-09-16T15:12:03.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.141+0000 7f16f6ffd640 1 -- 172.21.15.134:0/3704529777 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3195 (secure 0 0 0) 0x7f1700044070 con 0x7f17080a1eb0 2024-09-16T15:12:03.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.141+0000 7f16f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:03.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.141+0000 7f16f6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:03.144 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:12:03.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 msgr2=0x7f16e803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:03.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 0x7f16e803ad40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16f80094c0 tx=0x7f16f8009190 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 10 monclient: shutdown 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 msgr2=0x7f17081a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.142+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f170002cca0 tx=0x7f1700004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 shutdown_connections 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> v2:172.21.15.162:6800/980547263 conn(0x7f16e8038880 0x7f16e803ad40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f17080a1eb0 0x7f17081a15e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17080a1550 0x7f17081c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 --2- 172.21.15.134:0/3704529777 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1708066040 0x7f17081c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 >> 172.21.15.134:0/3704529777 conn(0x7f17080a6230 msgr2=0x7f17081b9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 shutdown_connections 2024-09-16T15:12:03.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:03.143+0000 7f170cd3e640 1 -- 172.21.15.134:0/3704529777 wait complete. 2024-09-16T15:12:06.157 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:12:06.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 10 monclient: get_monmap_and_config 2024-09-16T15:12:06.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 10 monclient: build_initial_monmap 2024-09-16T15:12:06.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 10 monclient: monmap: 2024-09-16T15:12:06.265 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:06.265 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:06.265117+0000 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:06.265117+0000 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:06.266 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:06.267 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:06.267 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:06.267 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 1 Processor -- start 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 1 -- start start 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.264+0000 7f25c4ca3640 10 monclient: init 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient: _add_conns ranks=[6,4,7,10,9,5,0,2,8,3,1] 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 0x7f25c0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-d con 0x7f25c0136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-c con 0x7f25c009bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f25c0136c60 0x7f25c01370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-i con 0x7f25c0136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c009b600 con 0x7f25c009bfe0 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00a5740 con 0x7f25c0136030 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00a52c0 con 0x7f25c0136c60 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25c4ca3640 10 monclient(hunting): authenticate will time out at 1462.854511s 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25be59c640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 msgr2=0x7f25c0136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25be59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 0x7f25c0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:36224/0 (socket says 172.21.15.134:36224) 2024-09-16T15:12:06.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 -- 172.21.15.134:0/1348408286 learned_addr learned my addr 172.21.15.134:0/1348408286 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): get_auth_request con 0x7f25c009bfe0 auth_method 0 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bed9d640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f25c0136c60 0x7f25c01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bed9d640 10 monclient(hunting): get_auth_request con 0x7f25c0136c60 auth_method 0 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient(hunting): handle_auth_done global_id 5503 payload 274 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 -- 172.21.15.134:0/1348408286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f25c0136c60 msgr2=0x7f25c01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:06.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f25c0136c60 0x7f25c01370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 -- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 msgr2=0x7f25c0136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 0x7f25c0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient: _finish_hunting 0 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 monclient: found mon.noname-c 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.265+0000 7f25bdd9b640 1 -- 172.21.15.134:0/1348408286 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0073250 con 0x7f25c009bfe0 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bdd9b640 1 -- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 msgr2=0x7f25c009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bdd9b640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098a0 tx=0x7f25a802a0a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=517c2cb218c768e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 1 -- 172.21.15.134:0/1348408286 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25a802ae00 con 0x7f25c009bfe0 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: dump: 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.278 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: _finish_auth 0 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: _check_auth_tickets 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 1 -- 172.21.15.134:0/1348408286 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80307b0 con 0x7f25c009bfe0 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25c4ca3640 5 monclient: authenticate success, global_id 5503 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25c4ca3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25c4ca3640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 1 -- 172.21.15.134:0/1348408286 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25a8030dd0 con 0x7f25c009bfe0 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25c4ca3640 4 set_mon_vals no callback set 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.268+0000 7f25bd59a640 10 monclient: dump: 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:06.280 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:06.281 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 10 monclient: shutdown 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 -- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 msgr2=0x7f25c009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098a0 tx=0x7f25a802a0a0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 -- 172.21.15.134:0/1348408286 shutdown_connections 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f25c0136c60 0x7f25c01370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c009c400 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/1348408286 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f25c0136030 0x7f25c0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 -- 172.21.15.134:0/1348408286 >> 172.21.15.134:0/1348408286 conn(0x7f25c00a9ea0 msgr2=0x7f25c00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 -- 172.21.15.134:0/1348408286 shutdown_connections 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.269+0000 7f25c4ca3640 1 -- 172.21.15.134:0/1348408286 wait complete. 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.270+0000 7f25c4ca3640 10 monclient: build_initial_monmap 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.270+0000 7f25c4ca3640 10 monclient: monmap: 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:06.282 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:06.283 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.270+0000 7f25c4ca3640 1 Processor -- start 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.270+0000 7f25c4ca3640 1 -- start start 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.270+0000 7f25c4ca3640 10 monclient: init 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient: _add_conns ranks=[4,2,1,6,8,7,5,3,0] 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-c con 0x7f25c009bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 0x7f25c01236b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-b con 0x7f25c0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c0136c60 0x7f25c0123bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): picked mon.noname-f con 0x7f25c0136c60 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0126dc0 con 0x7f25c0136c60 2024-09-16T15:12:06.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01270a0 con 0x7f25c0136030 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0124cd0 con 0x7f25c009bfe0 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25c4ca3640 10 monclient(hunting): authenticate will time out at 1462.860415s 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25bdd9b640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 msgr2=0x7f25c01236b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25bdd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 0x7f25c01236b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:36238/0 (socket says 172.21.15.134:36238) 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 -- 172.21.15.134:0/3421015794 learned_addr learned my addr 172.21.15.134:0/3421015794 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): get_auth_request con 0x7f25c009bfe0 auth_method 0 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:06.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient(hunting): handle_auth_done global_id 5512 payload 824 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 -- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 msgr2=0x7f25c01236b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 0x7f25c01236b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 -- 172.21.15.134:0/3421015794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c0136c60 msgr2=0x7f25c0123bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c0136c60 0x7f25c0123bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 monclient: found mon.noname-c 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.271+0000 7f25be59c640 1 -- 172.21.15.134:0/3421015794 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80094d0 con 0x7f25c009bfe0 2024-09-16T15:12:06.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25be59c640 1 -- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 msgr2=0x7f25c0123170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25be59c640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f25b800b550 tx=0x7f25b800ba10 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25b8003dd0 con 0x7f25c009bfe0 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: dump: 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:06.287 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b80054e0 con 0x7f25c009bfe0 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 5 monclient: authenticate success, global_id 5512 2024-09-16T15:12:06.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 10 monclient: _renew_subs 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c009fc70 con 0x7f25c009bfe0 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25b8005690 con 0x7f25c009bfe0 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25bcd99640 4 set_mon_vals no callback set 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25b6ffd640 10 monclient: dump: 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:06.289 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 10 monclient: _renew_subs 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.274+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00a0620 con 0x7f25c009bfe0 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f25b8005070 con 0x7f25c009bfe0 2024-09-16T15:12:06.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25c4ca3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25c4ca3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25c4ca3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c009c760 con 0x7f25c009bfe0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25b6ffd640 1 --2- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 0x7f25a003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f25b8049db0 con 0x7f25c009bfe0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25bdd9b640 1 --2- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 0x7f25a003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.275+0000 7f25bdd9b640 10 monclient: get_auth_request con 0x7f25a0038880 auth_method 0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.278+0000 7f25bdd9b640 1 --2- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 0x7f25a003ad40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a8034000 tx=0x7f25a802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:06.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.280+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f25b8005320 con 0x7f25c009bfe0 2024-09-16T15:12:06.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.280+0000 7f25b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:06.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.280+0000 7f25b6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25c4ca3640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25c4ca3640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25c4ca3640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f25c0135990 con 0x7f25c009bfe0 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25b6ffd640 1 -- 172.21.15.134:0/3421015794 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f25b8018b10 con 0x7f25c009bfe0 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25b6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.502+0000 7f25b6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:06.504 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 msgr2=0x7f25a003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 0x7f25a003ad40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a8034000 tx=0x7f25a802e000 comp rx=0 tx=0).stop 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 10 monclient: shutdown 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 msgr2=0x7f25c0123170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f25b800b550 tx=0x7f25b800ba10 comp rx=0 tx=0).stop 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 shutdown_connections 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> v2:172.21.15.162:6800/980547263 conn(0x7f25a0038880 0x7f25a003ad40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25c0136c60 0x7f25c0123bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25c0136030 0x7f25c01236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 --2- 172.21.15.134:0/3421015794 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f25c009bfe0 0x7f25c0123170 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:06.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 >> 172.21.15.134:0/3421015794 conn(0x7f25c00a9ea0 msgr2=0x7f25c009fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:06.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 shutdown_connections 2024-09-16T15:12:06.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:06.504+0000 7f25c4ca3640 1 -- 172.21.15.134:0/3421015794 wait complete. 2024-09-16T15:12:09.517 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 10 monclient: get_monmap_and_config 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 10 monclient: build_initial_monmap 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 10 monclient: monmap: 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:09.626 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:09.626020+0000 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:09.626020+0000 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:09.627 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:09.628 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:09.628 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:09.628 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 1 Processor -- start 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 1 -- start start 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.625+0000 7fdd82b9d640 10 monclient: init 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient: _add_conns ranks=[6,10,0,4,5,1,8,9,3,7,2] 2024-09-16T15:12:09.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 0x7fdd7c137220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-d con 0x7fdd7c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdd7c137760 0x7fdd7c0a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-f con 0x7fdd7c137760 addr v2:172.21.15.162:3300/0 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdd7c0a4200 addr v1:172.21.15.134:6789/0 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c073250 con 0x7fdd7c0a4200 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0a0f80 con 0x7fdd7c137760 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0a1260 con 0x7fdd7c136e40 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd82b9d640 10 monclient(hunting): authenticate will time out at 1466.215450s 2024-09-16T15:12:09.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd81b9b640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 msgr2=0x7fdd7c137220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd81b9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 0x7fdd7c137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8239c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fdd7c0a4200 0x7fdd7c13ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56586/0 (socket says 172.21.15.134:56586) 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8239c640 1 -- 172.21.15.134:0/792904191 learned_addr learned my addr 172.21.15.134:0/792904191 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 1 --2- 172.21.15.134:0/792904191 >> v2:172.21.15.162:3300/0 conn(0x7fdd7c137760 0x7fdd7c0a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): get_auth_request con 0x7fdd7c137760 auth_method 0 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:09.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.626+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 10 monclient(hunting): handle_auth_done global_id 44920 payload 274 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 1 -- 172.21.15.134:0/792904191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 msgr2=0x7fdd7c137220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 1 --2- 172.21.15.134:0/792904191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 0x7fdd7c137220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 1 -- 172.21.15.134:0/792904191 >> v1:172.21.15.134:6789/0 conn(0x7fdd7c0a4200 legacy=0x7fdd7c13ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 10 monclient: _finish_hunting 0 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 1 monclient: found mon.noname-f 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.162:3300/0 2024-09-16T15:12:09.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.627+0000 7fdd8139a640 1 -- 172.21.15.134:0/792904191 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0a09e0 con 0x7fdd7c137760 2024-09-16T15:12:09.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd8139a640 1 -- 172.21.15.134:0/792904191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7c137760 msgr2=0x7fdd7c0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:09.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd8139a640 1 --2- 172.21.15.134:0/792904191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7c137760 0x7fdd7c0a3cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0097a0 tx=0x7fdd6c02a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab1cf0cf45d540b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:09.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 1 -- 172.21.15.134:0/792904191 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd6c02ae30 con 0x7fdd7c137760 2024-09-16T15:12:09.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:09.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: dump: 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:09.648 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: _finish_auth 0 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: _check_auth_tickets 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 1 -- 172.21.15.134:0/792904191 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0307b0 con 0x7fdd7c137760 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:09.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 1 -- 172.21.15.134:0/792904191 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd6c030d90 con 0x7fdd7c137760 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd80b99640 10 monclient: dump: 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.650 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:09.651 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 5 monclient: authenticate success, global_id 44920 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 4 set_mon_vals no callback set 2024-09-16T15:12:09.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 10 monclient: shutdown 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 1 -- 172.21.15.134:0/792904191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7c137760 msgr2=0x7fdd7c0a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.630+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/792904191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7c137760 0x7fdd7c0a3cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0097a0 tx=0x7fdd6c02a030 comp rx=0 tx=0).stop 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 -- 172.21.15.134:0/792904191 shutdown_connections 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/792904191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7c137760 0x7fdd7c0a3cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/792904191 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdd7c136e40 0x7fdd7c137220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 -- 172.21.15.134:0/792904191 >> 172.21.15.134:0/792904191 conn(0x7fdd7c065e30 msgr2=0x7fdd7c066240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 -- 172.21.15.134:0/792904191 shutdown_connections 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 -- 172.21.15.134:0/792904191 wait complete. 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 10 monclient: build_initial_monmap 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 10 monclient: monmap: 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:09.653 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.631+0000 7fdd82b9d640 1 Processor -- start 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 -- start start 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient: init 2024-09-16T15:12:09.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient: _add_conns ranks=[3,7,4,2,0,5,8,6,1] 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 0x7fdd7c135c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-g con 0x7fdd7c0a4200 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd7c136e40 0x7fdd7c1361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-i con 0x7fdd7c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): picked mon.noname-c con 0x7fdd7c137760 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1cbf80 con 0x7fdd7c0a4200 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0a4850 con 0x7fdd7c137760 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0a8880 con 0x7fdd7c136e40 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:09.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd82b9d640 10 monclient(hunting): authenticate will time out at 1466.221486s 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:36254/0 (socket says 172.21.15.134:36254) 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 1 -- 172.21.15.134:0/427642431 learned_addr learned my addr 172.21.15.134:0/427642431 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): get_auth_request con 0x7fdd7c137760 auth_method 0 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd7c136e40 0x7fdd7c1361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd81b9b640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 0x7fdd7c135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): get_auth_request con 0x7fdd7c136e40 auth_method 0 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:09.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd81b9b640 10 monclient(hunting): get_auth_request con 0x7fdd7c0a4200 auth_method 0 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd81b9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd81b9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd81b9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8139a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.632+0000 7fdd8239c640 10 monclient(hunting): handle_auth_done global_id 5521 payload 824 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 -- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd7c136e40 msgr2=0x7fdd7c1361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:09.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd7c136e40 0x7fdd7c1361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 -- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 msgr2=0x7fdd7c135c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 0x7fdd7c135c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 10 monclient: _finish_hunting 0 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 monclient: found mon.noname-c 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd8239c640 1 -- 172.21.15.134:0/427642431 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009400 con 0x7fdd7c137760 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.633+0000 7fdd81b9b640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 0x7fdd7c135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd8239c640 1 -- 172.21.15.134:0/427642431 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 msgr2=0x7fdd7c11f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd8239c640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400caf0 tx=0x7fdd7400a060 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd7400acb0 con 0x7fdd7c137760 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: dump: 2024-09-16T15:12:09.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:09.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 5 monclient: authenticate success, global_id 5521 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7400ae50 con 0x7fdd7c137760 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd74012ce0 con 0x7fdd7c137760 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 10 monclient: _renew_subs 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c121290 con 0x7fdd7c137760 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd73fff640 4 set_mon_vals no callback set 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd727fc640 10 monclient: dump: 2024-09-16T15:12:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:09.661 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 10 monclient: _renew_subs 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.635+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c123640 con 0x7fdd7c137760 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdd74012400 con 0x7fdd7c137760 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd82b9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd82b9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd82b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c071c40 con 0x7fdd7c137760 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd727fc640 1 --2- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 0x7fdd6003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fdd7404a5a0 con 0x7fdd7c137760 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd81b9b640 1 --2- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 0x7fdd6003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.636+0000 7fdd81b9b640 10 monclient: get_auth_request con 0x7fdd600388b0 auth_method 0 2024-09-16T15:12:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.639+0000 7fdd81b9b640 1 --2- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 0x7fdd6003ad70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd78003980 tx=0x7fdd78008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.641+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdd740126b0 con 0x7fdd7c137760 2024-09-16T15:12:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.641+0000 7fdd727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.641+0000 7fdd727fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.864+0000 7fdd82b9d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:12:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.864+0000 7fdd82b9d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.864+0000 7fdd82b9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.864+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdd7c06a480 con 0x7fdd7c137760 2024-09-16T15:12:09.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.865+0000 7fdd727fc640 1 -- 172.21.15.134:0/427642431 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fdd74019300 con 0x7fdd7c137760 2024-09-16T15:12:09.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.865+0000 7fdd727fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:09.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.865+0000 7fdd727fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:09.867 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":98,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:12:09.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 msgr2=0x7fdd6003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 0x7fdd6003ad70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdd78003980 tx=0x7fdd78008040 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 10 monclient: shutdown 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 msgr2=0x7fdd7c11f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400caf0 tx=0x7fdd7400a060 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 shutdown_connections 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd7c137760 0x7fdd7c11f910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd7c136e40 0x7fdd7c1361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd600388b0 0x7fdd6003ad70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 --2- 172.21.15.134:0/427642431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7c0a4200 0x7fdd7c135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 >> 172.21.15.134:0/427642431 conn(0x7fdd7c065e30 msgr2=0x7fdd7c0a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 shutdown_connections 2024-09-16T15:12:09.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:09.867+0000 7fdd82b9d640 1 -- 172.21.15.134:0/427642431 wait complete. 2024-09-16T15:12:12.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.986+0000 7f3ef7c20640 10 monclient: get_monmap_and_config 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.986+0000 7f3ef7c20640 10 monclient: build_initial_monmap 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.986+0000 7f3ef7c20640 10 monclient: monmap: 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:12.987553+0000 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:12.987553+0000 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:12.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:12.989 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 Processor -- start 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 -- start start 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient: init 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient: _add_conns ranks=[2,9,10,3,8,0,7,5,6,4,1] 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-b con 0x7f3ef0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:12.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-a con 0x7f3ef009bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-f con 0x7f3ef0136c60 addr v2:172.21.15.162:3300/0 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef009b600 con 0x7f3ef009bfe0 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00a5740 con 0x7f3ef0136c60 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00a52c0 con 0x7f3ef0136030 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef7c20640 10 monclient(hunting): authenticate will time out at 1469.576969s 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.987+0000 7f3ef59bc640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 msgr2=0x7f3ef0136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:12.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef59bc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38848/0 (socket says 172.21.15.134:38848) 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 learned_addr learned my addr 172.21.15.134:0/2206560774 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ef009bfe0 auth_method 0 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ef0136c60 auth_method 0 2024-09-16T15:12:12.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:12.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:12.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 msgr2=0x7f3ef0136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ef009bfe0 auth_method 0 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:13.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:13.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ef0136c60 auth_method 0 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:13.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 msgr2=0x7f3ef0136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ef009bfe0 auth_method 0 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:13.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ef0136c60 auth_method 0 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:13.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:13.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:14.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 msgr2=0x7f3ef0136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:14.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:12:14.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:14.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ef009bfe0 auth_method 0 2024-09-16T15:12:14.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ef0136c60 auth_method 0 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:14.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:14.391+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): tick 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 monclient(hunting): continuing hunt 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 msgr2=0x7f3ef0136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ef0136030 0x7f3ef0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 msgr2=0x7f3ef0137110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:15.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.162:3300/0 conn(0x7f3ef0136c60 0x7f3ef0137110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 msgr2=0x7f3ef009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> v2:172.21.15.134:3300/0 conn(0x7f3ef009bfe0 0x7f3ef009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient: _add_conns ranks=[5,8,7,6,1,4,3,0] 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): picked mon.noname-h con 0x7f3ed40037a0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): picked mon.noname-e con 0x7f3ed4006240 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): picked mon.noname-i con 0x7f3ed4008dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef009b600 con 0x7f3ed40037a0 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00a5740 con 0x7f3ed4008dc0 2024-09-16T15:12:15.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 1 -- 172.21.15.134:0/2206560774 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00a52c0 con 0x7f3ed4006240 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ee77fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ed4006240 auth_method 0 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request con 0x7f3ed40037a0 auth_method 0 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:15.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ed4008dc0 auth_method 0 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:15.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:15.988+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ed4006240 auth_method 0 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ed4008dc0 auth_method 0 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request con 0x7f3ed40037a0 auth_method 0 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:16.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:16.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.189+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ed4006240 auth_method 0 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ed4008dc0 auth_method 0 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request con 0x7f3ed40037a0 auth_method 0 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:16.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:12:16.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:12:16.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:12:16.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:16.590+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:12:17.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ed4006240 auth_method 0 2024-09-16T15:12:17.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:17.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request con 0x7f3ed40037a0 auth_method 0 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ed4008dc0 auth_method 0 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient(hunting): handle_auth_done global_id 84239 payload 274 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 msgr2=0x7f3ed400b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 msgr2=0x7f3ed4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ed40037a0 0x7f3ed4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 monclient: found mon.noname-e 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0073250 con 0x7f3ed4006240 2024-09-16T15:12:17.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3ed4008dc0 0x7f3ed400b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:17.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.391+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2206560774 reap_dead start 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.394+0000 7f3ef51bb640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.394+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0018ac0 tx=0x7f3ee002bd00 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c8c728b273687989 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.394+0000 7f3ef49ba640 1 -- 172.21.15.134:0/2206560774 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ee002cc80 con 0x7f3ed4006240 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: dump: 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.403 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.404 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 1 -- 172.21.15.134:0/2206560774 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0030070 con 0x7f3ed4006240 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 5 monclient: authenticate success, global_id 84239 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 4 set_mon_vals no callback set 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 1 -- 172.21.15.134:0/2206560774 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ee002ce00 con 0x7f3ed4006240 2024-09-16T15:12:17.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef49ba640 10 monclient: dump: 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.406 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 10 monclient: shutdown 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 msgr2=0x7f3ed40086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.395+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0018ac0 tx=0x7f3ee002bd00 comp rx=0 tx=0).stop 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.396+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2206560774 shutdown_connections 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.396+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2206560774 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ed4006240 0x7f3ed40086d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.396+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2206560774 >> 172.21.15.134:0/2206560774 conn(0x7f3ef00a9ea0 msgr2=0x7f3ef00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.396+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2206560774 shutdown_connections 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.396+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2206560774 wait complete. 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.397+0000 7f3ef7c20640 10 monclient: build_initial_monmap 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.397+0000 7f3ef7c20640 10 monclient: monmap: 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:17.408 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.397+0000 7f3ef7c20640 1 Processor -- start 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.397+0000 7f3ef7c20640 1 -- start start 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.397+0000 7f3ef7c20640 10 monclient: init 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient: _add_conns ranks=[0,5,8,3,2,6,7,1,4] 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-a con 0x7f3ed4006240 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 0x7f3ef011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-h con 0x7f3ef01352b0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): picked mon.noname-e con 0x7f3ef011ffc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0121e90 con 0x7f3ed4006240 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef01cc210 con 0x7f3ef01352b0 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef01cb790 con 0x7f3ef011ffc0 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef7c20640 10 monclient(hunting): authenticate will time out at 1473.987165s 2024-09-16T15:12:17.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:34448/0 (socket says 172.21.15.134:34448) 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57632/0 (socket says 172.21.15.134:57632) 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2249617902 learned_addr learned my addr 172.21.15.134:0/2249617902 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request con 0x7f3ef011ffc0 auth_method 0 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request con 0x7f3ed4006240 auth_method 0 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 0x7f3ef011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request con 0x7f3ef01352b0 auth_method 0 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient(hunting): handle_auth_done global_id 84248 payload 824 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2249617902 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 msgr2=0x7f3ef011fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 0x7f3ef011fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 msgr2=0x7f3ef0134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 monclient: found mon.noname-e 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2249617902 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0018830 con 0x7f3ef011ffc0 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.398+0000 7f3ef51bb640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 0x7f3ef011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef61bd640 1 -- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 msgr2=0x7f3ef0120440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef61bd640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009610 tx=0x7f3ee8018790 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ee8018ee0 con 0x7f3ef011ffc0 2024-09-16T15:12:17.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: dump: 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.414 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee800f040 con 0x7f3ef011ffc0 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 5 monclient: authenticate success, global_id 84248 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 10 monclient: _renew_subs 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0122730 con 0x7f3ef011ffc0 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3ee8016c30 con 0x7f3ef011ffc0 2024-09-16T15:12:17.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee67fc640 10 monclient: dump: 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.416 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ee7fff640 4 set_mon_vals no callback set 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 10 monclient: _renew_subs 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.401+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00a2ec0 con 0x7f3ef011ffc0 2024-09-16T15:12:17.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3ee80167c0 con 0x7f3ef011ffc0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ee67fc640 1 --2- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 0x7f3ed403bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef7c20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef7c20640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef7c20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f3ee804bb70 con 0x7f3ef011ffc0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef00a7e10 con 0x7f3ef011ffc0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 0x7f3ed403bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.402+0000 7f3ef59bc640 10 monclient: get_auth_request con 0x7f3ed4039b10 auth_method 0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.405+0000 7f3ef59bc640 1 --2- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 0x7f3ed403bfd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3eec005010 tx=0x7f3eec005040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.407+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3ee8016a70 con 0x7f3ef011ffc0 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.407+0000 7f3ee67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.407+0000 7f3ee67fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ef7c20640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ef7c20640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ef7c20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3ef009c400 con 0x7f3ef011ffc0 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ee67fc640 1 -- 172.21.15.134:0/2249617902 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3184 (secure 0 0 0) 0x7f3ee8015330 con 0x7f3ef011ffc0 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ee67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.633+0000 7f3ee67fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:17.635 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":102,"quorum":[0,1,3,4,5,7,8],"quorum_names":["a","f","g","c","h","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 msgr2=0x7f3ed403bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 0x7f3ed403bfd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3eec005010 tx=0x7f3eec005040 comp rx=0 tx=0).stop 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 10 monclient: shutdown 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 msgr2=0x7f3ef0120440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009610 tx=0x7f3ee8018790 comp rx=0 tx=0).stop 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 shutdown_connections 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> v2:172.21.15.162:6800/980547263 conn(0x7f3ed4039b10 0x7f3ed403bfd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3ef011ffc0 0x7f3ef0120440 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3ef01352b0 0x7f3ef011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 --2- 172.21.15.134:0/2249617902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ed4006240 0x7f3ef0134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 >> 172.21.15.134:0/2249617902 conn(0x7f3ef00a9ea0 msgr2=0x7f3ef01bddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 shutdown_connections 2024-09-16T15:12:17.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.635+0000 7f3ef7c20640 1 -- 172.21.15.134:0/2249617902 wait complete. 2024-09-16T15:12:17.647 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-09-16T15:12:17.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-09-16T15:12:17.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:12:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.746+0000 7f8b98965640 10 monclient: get_monmap_and_config 2024-09-16T15:12:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.746+0000 7f8b98965640 10 monclient: build_initial_monmap 2024-09-16T15:12:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.746+0000 7f8b98965640 10 monclient: monmap: 2024-09-16T15:12:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:17.748 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:17.747644+0000 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:17.747644+0000 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:17.749 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:17.750 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 Processor -- start 2024-09-16T15:12:17.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 -- start start 2024-09-16T15:12:17.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient: init 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient: _add_conns ranks=[5,4,9,6,3,7,1,10,2,0,8] 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b94136e40 0x7f8b9409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-h con 0x7f8b94136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-c con 0x7f8b94137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b94135dc0 0x7f8b94136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-a con 0x7f8b94135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:17.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9409abc0 con 0x7f8b94135dc0 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9409ba50 con 0x7f8b94137220 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940a53c0 con 0x7f8b94136e40 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.747+0000 7f8b98965640 10 monclient(hunting): authenticate will time out at 1474.336936s 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b94135dc0 0x7f8b94136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b94135dc0 0x7f8b94136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57638/0 (socket says 172.21.15.134:57638) 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56908/0 (socket says 172.21.15.134:56908) 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1196960304 learned_addr learned my addr 172.21.15.134:0/1196960304 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:17.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): get_auth_request con 0x7f8b94135dc0 auth_method 0 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): get_auth_request con 0x7f8b94137220 auth_method 0 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b9259c640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b94136e40 0x7f8b9409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b9259c640 10 monclient(hunting): get_auth_request con 0x7f8b94136e40 auth_method 0 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b9259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_done global_id 74565 payload 274 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1196960304 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b94136e40 msgr2=0x7f8b9409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b94136e40 0x7f8b9409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 msgr2=0x7f8b94137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 monclient: found mon.noname-a 2024-09-16T15:12:17.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1196960304 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94073250 con 0x7f8b94135dc0 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.748+0000 7f8b91d9b640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94135dc0 msgr2=0x7f8b94136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94135dc0 0x7f8b94136220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e50 tx=0x7f8b8c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded92b668bffb5da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 1 -- 172.21.15.134:0/1196960304 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c007a70 con 0x7f8b94135dc0 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: dump: 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.758 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 1 -- 172.21.15.134:0/1196960304 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c02d050 con 0x7f8b94135dc0 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 5 monclient: authenticate success, global_id 74565 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 1 -- 172.21.15.134:0/1196960304 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b8c007bf0 con 0x7f8b94135dc0 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 4 set_mon_vals no callback set 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b9159a640 10 monclient: dump: 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.760 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 10 monclient: shutdown 2024-09-16T15:12:17.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 -- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94135dc0 msgr2=0x7f8b94136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94135dc0 0x7f8b94136220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e50 tx=0x7f8b8c02a560 comp rx=0 tx=0).stop 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 -- 172.21.15.134:0/1196960304 shutdown_connections 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94135dc0 0x7f8b94136220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b94137220 0x7f8b94137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 --2- 172.21.15.134:0/1196960304 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b94136e40 0x7f8b9409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 -- 172.21.15.134:0/1196960304 >> 172.21.15.134:0/1196960304 conn(0x7f8b940a6110 msgr2=0x7f8b940a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 -- 172.21.15.134:0/1196960304 shutdown_connections 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.751+0000 7f8b98965640 1 -- 172.21.15.134:0/1196960304 wait complete. 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.752+0000 7f8b98965640 10 monclient: build_initial_monmap 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.752+0000 7f8b98965640 10 monclient: monmap: 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:17.762 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.752+0000 7f8b98965640 1 Processor -- start 2024-09-16T15:12:17.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 -- start start 2024-09-16T15:12:17.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient: init 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient: _add_conns ranks=[2,7,3,5,0,4,6,1,8] 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b94135dc0 0x7f8b94134920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-b con 0x7f8b94135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-i con 0x7f8b94136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 0x7f8b94121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): picked mon.noname-g con 0x7f8b94137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940a8930 con 0x7f8b94135dc0 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941367f0 con 0x7f8b94137220 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941362c0 con 0x7f8b94136e40 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b98965640 10 monclient(hunting): authenticate will time out at 1474.342447s 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b9259c640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b94135dc0 msgr2=0x7f8b94134920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b9259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b94135dc0 0x7f8b94134920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 0x7f8b94121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 0x7f8b94121820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:42488/0 (socket says 172.21.15.134:42488) 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:53322/0 (socket says 172.21.15.134:53322) 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1245777629 learned_addr learned my addr 172.21.15.134:0/1245777629 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): get_auth_request con 0x7f8b94137220 auth_method 0 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): get_auth_request con 0x7f8b94136e40 auth_method 0 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b92d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.753+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 10 monclient(hunting): handle_auth_done global_id 54655 payload 824 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 msgr2=0x7f8b94121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 0x7f8b94121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b94135dc0 msgr2=0x7f8b94134920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b94135dc0 0x7f8b94134920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 monclient: found mon.noname-i 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b91d9b640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009400 con 0x7f8b94136e40 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.754+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b94137220 0x7f8b94121820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:17.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b91d9b640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 msgr2=0x7f8b94134e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b91d9b640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e870 tx=0x7f8b8800ed30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b88013470 con 0x7f8b94136e40 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: dump: 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.773 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b880135f0 con 0x7f8b94136e40 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 5 monclient: authenticate success, global_id 54655 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 10 monclient: _renew_subs 2024-09-16T15:12:17.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b88013bd0 con 0x7f8b94136e40 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c009b30 con 0x7f8b94136e40 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b82ffd640 10 monclient: dump: 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.775 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b90d99640 4 set_mon_vals no callback set 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 10 monclient: _renew_subs 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.756+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b941251d0 con 0x7f8b94136e40 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b98965640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b98965640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 0x7f8b94122140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94123420 con 0x7f8b94121d60 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b91d9b640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e870 tx=0x7f8b8800ed30 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f8b8804a2c0 osd_map(519..519 src has 1..519) v4 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8b88013d30 con 0x7f8b94136e40 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 0x7f8b94122140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient: get_auth_request con 0x7f8b94121d60 auth_method 0 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b82ffd640 1 --2- 172.21.15.134:0/1245777629 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b74038880 0x7f8b7403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 10 monclient(hunting): handle_auth_done global_id 74574 payload 824 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b9259c640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b94061ae0 con 0x7f8b94121d60 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1245777629 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b74038880 0x7f8b7403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.757+0000 7f8b92d9d640 10 monclient: get_auth_request con 0x7f8b74038880 auth_method 0 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.759+0000 7f8b9259c640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 msgr2=0x7f8b94122140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.759+0000 7f8b9259c640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 0x7f8b94122140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d4f0 tx=0x7f8b8400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b840044a0 con 0x7f8b94121d60 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 10 monclient: dump: 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.778 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8b84004030 con 0x7f8b94121d60 2024-09-16T15:12:17.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 msgr2=0x7f8b94122140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 0x7f8b94122140 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d4f0 tx=0x7f8b8400d9b0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 msgr2=0x7f8b94122140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b82ffd640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94121d60 0x7f8b94122140 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d4f0 tx=0x7f8b8400d9b0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.760+0000 7f8b92d9d640 1 --2- 172.21.15.134:0/1245777629 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b74038880 0x7f8b7403ad40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b94074fb0 tx=0x7f8b8c009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b98965640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b98965640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 0x7f8b940a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b98965640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94124e90 con 0x7f8b94122680 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 0x7f8b940a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient: get_auth_request con 0x7f8b94122680 auth_method 0 2024-09-16T15:12:17.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 10 monclient(hunting): handle_auth_done global_id 74583 payload 824 2024-09-16T15:12:17.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.766+0000 7f8b9259c640 1 -- 172.21.15.134:0/1245777629 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b9407a680 con 0x7f8b94122680 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b9259c640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 msgr2=0x7f8b940a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b9259c640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 0x7f8b940a1a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b84017000 tx=0x7f8b84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b84003610 con 0x7f8b94122680 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 10 monclient: dump: 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.782 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.783 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9176 (secure 0 0 0) 0x7f8b8401db90 con 0x7f8b94122680 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 msgr2=0x7f8b940a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 0x7f8b940a1a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b84017000 tx=0x7f8b84004730 comp rx=0 tx=0).stop 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 msgr2=0x7f8b940a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b94122680 0x7f8b940a1a20 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b84017000 tx=0x7f8b84004730 comp rx=0 tx=0).stop 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:17.784 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.785 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.786 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:17.788 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.789 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:17.790 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:17.798 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.801 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.803 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.805 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:17.806 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.807 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:17.807 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:17.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b74038880 msgr2=0x7f8b7403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 --2- 172.21.15.134:0/1245777629 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b74038880 0x7f8b7403ad40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b94074fb0 tx=0x7f8b8c009000 comp rx=0 tx=0).stop 2024-09-16T15:12:17.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 10 monclient: shutdown 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 msgr2=0x7f8b94134e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e870 tx=0x7f8b8800ed30 comp rx=0 tx=0).stop 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b92d9d640 1 -- 172.21.15.134:0/1245777629 reap_dead start 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b82ffd640 1 -- 172.21.15.134:0/1245777629 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (unknown 0 0 0) 0x7f8b8804a2c0 con 0x7f8b94136e40 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 shutdown_connections 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 --2- 172.21.15.134:0/1245777629 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b94136e40 0x7f8b94134e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 >> 172.21.15.134:0/1245777629 conn(0x7f8b940a6110 msgr2=0x7f8b9406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 shutdown_connections 2024-09-16T15:12:17.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.769+0000 7f8b98965640 1 -- 172.21.15.134:0/1245777629 wait complete. 2024-09-16T15:12:17.809 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:17.810 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon c 2024-09-16T15:12:17.810 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.905+0000 7fad8291f640 10 monclient: get_monmap_and_config 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.905+0000 7fad8291f640 10 monclient: build_initial_monmap 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.906+0000 7fad8291f640 10 monclient: monmap: 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:17.906904+0000 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:17.906904+0000 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.907 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:17.908 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.906+0000 7fad8291f640 1 Processor -- start 2024-09-16T15:12:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.906+0000 7fad8291f640 1 -- start start 2024-09-16T15:12:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.906+0000 7fad8291f640 10 monclient: init 2024-09-16T15:12:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:17.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient: _add_conns ranks=[1,9,4,6,3,0,5,7,10,2,8] 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fad7c136eb0 addr v1:172.21.15.162:6789/0 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-a con 0x7fad7c137290 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-c con 0x7fad7c13c450 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c073250 con 0x7fad7c136eb0 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7c09c610 con 0x7fad7c137290 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0a5560 con 0x7fad7c13c450 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad8291f640 10 monclient(hunting): authenticate will time out at 1474.496465s 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57656/0 (socket says 172.21.15.134:57656) 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 1 -- 172.21.15.134:0/2794587126 learned_addr learned my addr 172.21.15.134:0/2794587126 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56916/0 (socket says 172.21.15.134:56916) 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): get_auth_request con 0x7fad7c137290 auth_method 0 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): get_auth_request con 0x7fad7c13c450 auth_method 0 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient(hunting): handle_auth_done global_id 5539 payload 274 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 -- 172.21.15.134:0/2794587126 >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 msgr2=0x7fad7c1376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 --2- 172.21.15.134:0/2794587126 >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 -- 172.21.15.134:0/2794587126 >> v1:172.21.15.162:6789/0 conn(0x7fad7c136eb0 legacy=0x7fad7c09bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 monclient: found mon.noname-c 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad80ebc640 1 -- 172.21.15.134:0/2794587126 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c09b240 con 0x7fad7c13c450 2024-09-16T15:12:17.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.907+0000 7fad73fff640 1 --2- 172.21.15.134:0/2794587126 >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:17.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad80ebc640 1 -- 172.21.15.134:0/2794587126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 msgr2=0x7fad7c13e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad80ebc640 1 --2- 172.21.15.134:0/2794587126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad74009dc0 tx=0x7fad7402a510 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b736bf37f9afb74f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 1 -- 172.21.15.134:0/2794587126 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad74007bc0 con 0x7fad7c13c450 2024-09-16T15:12:17.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: dump: 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.920 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 1 -- 172.21.15.134:0/2794587126 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7402d050 con 0x7fad7c13c450 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad8291f640 5 monclient: authenticate success, global_id 5539 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad8291f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad8291f640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 1 -- 172.21.15.134:0/2794587126 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad74007d40 con 0x7fad7c13c450 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad8291f640 4 set_mon_vals no callback set 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:17.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.910+0000 7fad7b7fe640 10 monclient: dump: 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.922 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 10 monclient: shutdown 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 -- 172.21.15.134:0/2794587126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 msgr2=0x7fad7c13e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 --2- 172.21.15.134:0/2794587126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad74009dc0 tx=0x7fad7402a510 comp rx=0 tx=0).stop 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 -- 172.21.15.134:0/2794587126 shutdown_connections 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 --2- 172.21.15.134:0/2794587126 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c13c450 0x7fad7c13e840 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 --2- 172.21.15.134:0/2794587126 >> v2:172.21.15.134:3300/0 conn(0x7fad7c137290 0x7fad7c1376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 -- 172.21.15.134:0/2794587126 >> 172.21.15.134:0/2794587126 conn(0x7fad7c0a99a0 msgr2=0x7fad7c0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 -- 172.21.15.134:0/2794587126 shutdown_connections 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.911+0000 7fad8291f640 1 -- 172.21.15.134:0/2794587126 wait complete. 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient: build_initial_monmap 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient: monmap: 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:17.923 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 Processor -- start 2024-09-16T15:12:17.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 -- start start 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient: init 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient: _add_conns ranks=[6,7,8,3,5,4,2,1,0] 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fad7c136eb0 0x7fad7c1356e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-d con 0x7fad7c136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fad7c137290 0x7fad7c135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-i con 0x7fad7c137290 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): picked mon.noname-e con 0x7fad7c13c450 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c122140 con 0x7fad7c136eb0 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c13b4d0 con 0x7fad7c137290 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad7bfff640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fad7c136eb0 msgr2=0x7fad7c1356e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0a7810 con 0x7fad7c13c450 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad8291f640 10 monclient(hunting): authenticate will time out at 1474.501860s 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad7bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fad7c136eb0 0x7fad7c1356e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:34452/0 (socket says 172.21.15.134:34452) 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 learned_addr learned my addr 172.21.15.134:0/1023288744 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 10 monclient(hunting): get_auth_request con 0x7fad7c13c450 auth_method 0 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.912+0000 7fad80ebc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fad7c137290 0x7fad7c135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 10 monclient(hunting): get_auth_request con 0x7fad7c137290 auth_method 0 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient(hunting): handle_auth_done global_id 84257 payload 824 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fad7c137290 msgr2=0x7fad7c135c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fad7c137290 0x7fad7c135c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fad7c136eb0 msgr2=0x7fad7c1356e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fad7c136eb0 0x7fad7c1356e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient: _finish_hunting 0 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 monclient: found mon.noname-e 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640094d0 con 0x7fad7c13c450 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.913+0000 7fad73fff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fad7c137290 0x7fad7c135c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 msgr2=0x7fad7c136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad80ebc640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad7402aa30 tx=0x7fad74004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad74007ea0 con 0x7fad7c13c450 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: dump: 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.928 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: _finish_auth 0 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: _check_auth_tickets 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 5 monclient: authenticate success, global_id 84257 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740044b0 con 0x7fad7c13c450 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 10 monclient: _renew_subs 2024-09-16T15:12:17.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad74009a70 con 0x7fad7c13c450 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad74004620 con 0x7fad7c13c450 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad7affd640 4 set_mon_vals no callback set 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad797fa640 10 monclient: dump: 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.930 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 10 monclient: _renew_subs 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.915+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c121e60 con 0x7fad7c13c450 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fad74004040 con 0x7fad7c13c450 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad8291f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad8291f640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 0x7fad7c061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c121ff0 con 0x7fad7c0610b0 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 0x7fad7c061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient: get_auth_request con 0x7fad7c0610b0 auth_method 0 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad797fa640 1 --2- 172.21.15.134:0/1023288744 >> v2:172.21.15.162:6800/980547263 conn(0x7fad5c038870 0x7fad5c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fad7406b030 con 0x7fad7c13c450 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad73fff640 1 --2- 172.21.15.134:0/1023288744 >> v2:172.21.15.162:6800/980547263 conn(0x7fad5c038870 0x7fad5c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 10 monclient(hunting): handle_auth_done global_id 5548 payload 824 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad7bfff640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad7c061ae0 con 0x7fad7c0610b0 2024-09-16T15:12:17.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.916+0000 7fad73fff640 10 monclient: get_auth_request con 0x7fad5c038870 auth_method 0 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad7bfff640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 msgr2=0x7fad7c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad7bfff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 0x7fad7c061490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c380 tx=0x7fad6c00c840 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad6c0075d0 con 0x7fad7c0610b0 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 10 monclient: dump: 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.935 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad73fff640 1 --2- 172.21.15.134:0/1023288744 >> v2:172.21.15.162:6800/980547263 conn(0x7fad5c038870 0x7fad5c03ad30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0a2670 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fad6c0057c0 con 0x7fad7c0610b0 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:17.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 msgr2=0x7fad7c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 0x7fad7c061490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c380 tx=0x7fad6c00c840 comp rx=0 tx=0).stop 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 msgr2=0x7fad7c061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.919+0000 7fad797fa640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c0610b0 0x7fad7c061490 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad8291f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad8291f640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 0x7fad7c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad8291f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c071b20 con 0x7fad7c02b260 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 0x7fad7c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient: get_auth_request con 0x7fad7c02b260 auth_method 0 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:17.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 10 monclient(hunting): handle_auth_done global_id 5557 payload 824 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.925+0000 7fad7bfff640 1 -- 172.21.15.134:0/1023288744 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fad7c0a5cf0 con 0x7fad7c02b260 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad7bfff640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 msgr2=0x7fad7c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad7bfff640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 0x7fad7c02b670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c380 tx=0x7fad6c0037a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fad6c005e80 con 0x7fad7c02b260 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 10 monclient: dump: 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:17.938 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9340 (secure 0 0 0) 0x7fad6c01a2a0 con 0x7fad7c02b260 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 msgr2=0x7fad7c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 0x7fad7c02b670 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c380 tx=0x7fad6c0037a0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 msgr2=0x7fad7c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:17.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad797fa640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fad7c02b260 0x7fad7c02b670 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c380 tx=0x7fad6c0037a0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:17.940 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.941 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.942 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:17.943 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:17.944 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.947 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.948 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.949 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.950 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.952 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.953 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.956 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.957 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:17.958 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:17.959 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:17.960 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:17.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 >> v2:172.21.15.162:6800/980547263 conn(0x7fad5c038870 msgr2=0x7fad5c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 --2- 172.21.15.134:0/1023288744 >> v2:172.21.15.162:6800/980547263 conn(0x7fad5c038870 0x7fad5c03ad30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0a2670 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 10 monclient: shutdown 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 msgr2=0x7fad7c136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad80ebc640 1 -- 172.21.15.134:0/1023288744 reap_dead start 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fad7402aa30 tx=0x7fad74004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 shutdown_connections 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 --2- 172.21.15.134:0/1023288744 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fad7c13c450 0x7fad7c136160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.928+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 >> 172.21.15.134:0/1023288744 conn(0x7fad7c0a99a0 msgr2=0x7fad7c13a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.929+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 shutdown_connections 2024-09-16T15:12:17.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:17.929+0000 7fad8291f640 1 -- 172.21.15.134:0/1023288744 wait complete. 2024-09-16T15:12:17.962 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:17.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-09-16T15:12:17.963 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.058+0000 7f9796d9d640 10 monclient: get_monmap_and_config 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.058+0000 7f9796d9d640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.059+0000 7f9796d9d640 10 monclient: monmap: 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:18.059915+0000 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:18.059915+0000 2024-09-16T15:12:18.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:18.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.059+0000 7f9796d9d640 1 Processor -- start 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.059+0000 7f9796d9d640 1 -- start start 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.059+0000 7f9796d9d640 10 monclient: init 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient: _add_conns ranks=[4,7,1,10,9,6,5,2,0,3,8] 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-c con 0x7f9790135de0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 0x7f97901365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-i con 0x7f97901361c0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f97900a3ce0 addr v1:172.21.15.162:6789/0 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9790073250 con 0x7f97900a3ce0 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f979009b620 con 0x7f9790135de0 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f97900a5540 con 0x7f97901361c0 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9796d9d640 10 monclient(hunting): authenticate will time out at 1474.649336s 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56932/0 (socket says 172.21.15.134:56932) 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 -- 172.21.15.134:0/3855086017 learned_addr learned my addr 172.21.15.134:0/3855086017 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): get_auth_request con 0x7f9790135de0 auth_method 0 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f979559a640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 0x7f97901365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient(hunting): handle_auth_done global_id 5566 payload 274 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 -- 172.21.15.134:0/3855086017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 msgr2=0x7f97901365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 0x7f97901365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 -- 172.21.15.134:0/3855086017 >> v1:172.21.15.162:6789/0 conn(0x7f97900a3ce0 legacy=0x7f97900a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 monclient: found mon.noname-c 2024-09-16T15:12:18.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f9795d9b640 1 -- 172.21.15.134:0/3855086017 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900a4e80 con 0x7f9790135de0 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f979559a640 10 monclient: get_auth_request con 0x7f97901361c0 auth_method 0 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.060+0000 7f979559a640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 0x7f97901365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9795d9b640 1 -- 172.21.15.134:0/3855086017 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 msgr2=0x7f979009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9795d9b640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c02a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2a7010ab1ced0142 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 1 -- 172.21.15.134:0/3855086017 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f978c0307b0 con 0x7f9790135de0 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: dump: 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.074 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 1 -- 172.21.15.134:0/3855086017 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c030930 con 0x7f9790135de0 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9796d9d640 5 monclient: authenticate success, global_id 5566 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9796d9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9796d9d640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:18.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 1 -- 172.21.15.134:0/3855086017 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f978c0313a0 con 0x7f9790135de0 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9796d9d640 4 set_mon_vals no callback set 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.063+0000 7f9794d99640 10 monclient: dump: 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 10 monclient: shutdown 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 -- 172.21.15.134:0/3855086017 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 msgr2=0x7f979009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 -- 172.21.15.134:0/3855086017 shutdown_connections 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f97901361c0 0x7f97901365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 --2- 172.21.15.134:0/3855086017 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f979009bf00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 -- 172.21.15.134:0/3855086017 >> 172.21.15.134:0/3855086017 conn(0x7f9790066070 msgr2=0x7f9790066480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 -- 172.21.15.134:0/3855086017 shutdown_connections 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 1 -- 172.21.15.134:0/3855086017 wait complete. 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.064+0000 7f9796d9d640 10 monclient: monmap: 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:18.078 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 Processor -- start 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 -- start start 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient: init 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient: _add_conns ranks=[8,4,6,2,5,7,3,0,1] 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f97900a3ce0 0x7f97901cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-e con 0x7f97900a3ce0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f97901cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-c con 0x7f9790135de0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f97901361c0 0x7f97901cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): picked mon.noname-d con 0x7f97901361c0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9790135530 con 0x7f9790135de0 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f97900a7370 con 0x7f97901361c0 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f97900a3160 con 0x7f97900a3ce0 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9796d9d640 10 monclient(hunting): authenticate will time out at 1474.654632s 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979659c640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f97901361c0 msgr2=0x7f97901cbb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f97901361c0 0x7f97901cbb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f97901cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f97901cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56934/0 (socket says 172.21.15.134:56934) 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f97900a3ce0 0x7f97901cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 1 -- 172.21.15.134:0/4026890878 learned_addr learned my addr 172.21.15.134:0/4026890878 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): get_auth_request con 0x7f9790135de0 auth_method 0 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): get_auth_request con 0x7f97900a3ce0 auth_method 0 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 10 monclient(hunting): handle_auth_done global_id 5575 payload 824 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f97900a3ce0 msgr2=0x7f97901cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.065+0000 7f979559a640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f97900a3ce0 0x7f97901cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f97901361c0 msgr2=0x7f97901cbb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f97901361c0 0x7f97901cbb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 1 monclient: found mon.noname-c 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f979559a640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c0094d0 con 0x7f9790135de0 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.066+0000 7f9795d9b640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f97900a3ce0 0x7f97901cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f979559a640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 msgr2=0x7f97901cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f979559a640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f97901cb650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f978000c7f0 tx=0x7f978000ccb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9780007ca0 con 0x7f9790135de0 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: dump: 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.083 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9780007e40 con 0x7f9790135de0 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 5 monclient: authenticate success, global_id 5575 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 10 monclient: _renew_subs 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9790121ce0 con 0x7f9790135de0 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9780011860 con 0x7f9790135de0 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f97867fc640 10 monclient: dump: 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.085 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9787fff640 4 set_mon_vals no callback set 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 10 monclient: _renew_subs 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.068+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900a4940 con 0x7f9790135de0 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9796d9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9796d9d640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 0x7f9790134770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9790069fe0 con 0x7f9790134390 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f97800113f0 con 0x7f9790135de0 2024-09-16T15:12:18.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 0x7f9790134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient: get_auth_request con 0x7f9790134390 auth_method 0 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f97867fc640 1 --2- 172.21.15.134:0/4026890878 >> v2:172.21.15.162:6800/980547263 conn(0x7f9774038880 0x7f977403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 10 monclient(hunting): handle_auth_done global_id 84275 payload 824 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.069+0000 7f9795d9b640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9790073810 con 0x7f9790134390 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.070+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f978004a190 con 0x7f9790135de0 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.070+0000 7f979659c640 1 --2- 172.21.15.134:0/4026890878 >> v2:172.21.15.162:6800/980547263 conn(0x7f9774038880 0x7f977403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.070+0000 7f979659c640 10 monclient: get_auth_request con 0x7f9774038880 auth_method 0 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f9795d9b640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 msgr2=0x7f9790134770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f9795d9b640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 0x7f9790134770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f978c034000 tx=0x7f978c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f978c003c90 con 0x7f9790134390 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 10 monclient: dump: 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f978c003df0 con 0x7f9790134390 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 msgr2=0x7f9790134770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 0x7f9790134770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f978c034000 tx=0x7f978c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 msgr2=0x7f9790134770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f97867fc640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134390 0x7f9790134770 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f978c034000 tx=0x7f978c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.072+0000 7f979659c640 1 --2- 172.21.15.134:0/4026890878 >> v2:172.21.15.162:6800/980547263 conn(0x7f9774038880 0x7f977403ad40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97900a3520 tx=0x7f9788007d50 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9796d9d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9796d9d640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 0x7f979009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9796d9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9790061ae0 con 0x7f9790134cb0 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 0x7f979009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient: get_auth_request con 0x7f9790134cb0 auth_method 0 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 10 monclient(hunting): handle_auth_done global_id 84284 payload 824 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.078+0000 7f9795d9b640 1 -- 172.21.15.134:0/4026890878 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f979007a680 con 0x7f9790134cb0 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9795d9b640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 msgr2=0x7f979009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9795d9b640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 0x7f979009d7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f978c02a110 tx=0x7f978c008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f978c048c30 con 0x7f9790134cb0 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 10 monclient: dump: 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9174 (secure 0 0 0) 0x7f978c048d90 con 0x7f9790134cb0 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 msgr2=0x7f979009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 0x7f979009d7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f978c02a110 tx=0x7f978c008030 comp rx=0 tx=0).stop 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 msgr2=0x7f979009d7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f97867fc640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9790134cb0 0x7f979009d7d0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f978c02a110 tx=0x7f978c008030 comp rx=0 tx=0).stop 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:18.095 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.096 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.097 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:18.098 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.099 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:18.103 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:18.105 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.107 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.109 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.110 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.112 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.114 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:18.115 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 >> v2:172.21.15.162:6800/980547263 conn(0x7f9774038880 msgr2=0x7f977403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 1 --2- 172.21.15.134:0/4026890878 >> v2:172.21.15.162:6800/980547263 conn(0x7f9774038880 0x7f977403ad40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97900a3520 tx=0x7f9788007d50 comp rx=0 tx=0).stop 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 10 monclient: shutdown 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 msgr2=0x7f97901cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f9796d9d640 1 --2- 172.21.15.134:0/4026890878 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9790135de0 0x7f97901cb650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f978000c7f0 tx=0x7f978000ccb0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.081+0000 7f979659c640 1 -- 172.21.15.134:0/4026890878 reap_dead start 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.082+0000 7f979659c640 1 -- 172.21.15.134:0/4026890878 reap_dead start 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.082+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 shutdown_connections 2024-09-16T15:12:18.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.082+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 >> 172.21.15.134:0/4026890878 conn(0x7f9790066070 msgr2=0x7f97901beb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.082+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 shutdown_connections 2024-09-16T15:12:18.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.082+0000 7f9796d9d640 1 -- 172.21.15.134:0/4026890878 wait complete. 2024-09-16T15:12:18.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:18.118 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-09-16T15:12:18.118 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:12:18.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.215+0000 7f6833632640 10 monclient: get_monmap_and_config 2024-09-16T15:12:18.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.215+0000 7f6833632640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.215+0000 7f6833632640 10 monclient: monmap: 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:18.216262+0000 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:18.216262+0000 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.217 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.218 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:18.218 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:18.218 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.215+0000 7f6833632640 1 Processor -- start 2024-09-16T15:12:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 -- start start 2024-09-16T15:12:18.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient: init 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient: _add_conns ranks=[9,4,8,2,7,0,6,3,1,5,10] 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): picked mon.noname-a con 0x7f682c136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 0x7f682c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): picked mon.noname-c con 0x7f682c137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 0x7f682c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): picked mon.noname-e con 0x7f682c135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682c09abc0 con 0x7f682c136e40 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f682c09ba50 con 0x7f682c137220 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0a53c0 con 0x7f682c135dc0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6833632640 10 monclient(hunting): authenticate will time out at 1474.805608s 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 0x7f682c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57660/0 (socket says 172.21.15.134:57660) 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 1 -- 172.21.15.134:0/3876413482 learned_addr learned my addr 172.21.15.134:0/3876413482 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): get_auth_request con 0x7f682c136e40 auth_method 0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 0x7f682c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): get_auth_request con 0x7f682c135dc0 auth_method 0 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): get_auth_request con 0x7f682c137220 auth_method 0 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6830bcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 10 monclient(hunting): handle_auth_done global_id 84293 payload 274 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 1 -- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 msgr2=0x7f682c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.216+0000 7f6831bcf640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 0x7f682c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 1 -- 172.21.15.134:0/3876413482 >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 msgr2=0x7f682c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 1 --2- 172.21.15.134:0/3876413482 >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 1 monclient: found mon.noname-e 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6831bcf640 1 -- 172.21.15.134:0/3876413482 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c073250 con 0x7f682c135dc0 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f68313ce640 1 --2- 172.21.15.134:0/3876413482 >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.217+0000 7f6830bcd640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 0x7f682c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6831bcf640 1 -- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 msgr2=0x7f682c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6831bcf640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 0x7f682c136220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6824009dc0 tx=0x7f682402a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=96304a5e2005fec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 1 -- 172.21.15.134:0/3876413482 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6824007bc0 con 0x7f682c135dc0 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: dump: 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.226 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 1 -- 172.21.15.134:0/3876413482 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682402d050 con 0x7f682c135dc0 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6833632640 5 monclient: authenticate success, global_id 84293 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6833632640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6833632640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 1 -- 172.21.15.134:0/3876413482 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6824007d80 con 0x7f682c135dc0 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f6833632640 4 set_mon_vals no callback set 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.219+0000 7f681ffff640 10 monclient: dump: 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.228 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 10 monclient: shutdown 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 -- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 msgr2=0x7f682c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 0x7f682c136220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6824009dc0 tx=0x7f682402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 -- 172.21.15.134:0/3876413482 shutdown_connections 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f682c135dc0 0x7f682c136220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 --2- 172.21.15.134:0/3876413482 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f682c137220 0x7f682c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 --2- 172.21.15.134:0/3876413482 >> v2:172.21.15.134:3300/0 conn(0x7f682c136e40 0x7f682c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 -- 172.21.15.134:0/3876413482 >> 172.21.15.134:0/3876413482 conn(0x7f682c0a6110 msgr2=0x7f682c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 -- 172.21.15.134:0/3876413482 shutdown_connections 2024-09-16T15:12:18.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.220+0000 7f6833632640 1 -- 172.21.15.134:0/3876413482 wait complete. 2024-09-16T15:12:18.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient: monmap: 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.235 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 Processor -- start 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 -- start start 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient: init 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient: _add_conns ranks=[2,1,0,5,4,6,8,7,3] 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c135dc0 0x7f682c09f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): picked mon.noname-b con 0x7f682c135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c136e40 0x7f682c134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): picked mon.noname-f con 0x7f682c136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): picked mon.noname-a con 0x7f682c137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c1364c0 con 0x7f682c137220 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c121e80 con 0x7f682c136e40 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c122440 con 0x7f682c135dc0 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f6833632640 10 monclient(hunting): authenticate will time out at 1474.810924s 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f68313ce640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c135dc0 msgr2=0x7f682c09f2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.221+0000 7f68313ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c135dc0 0x7f682c09f2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57670/0 (socket says 172.21.15.134:57670) 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 learned_addr learned my addr 172.21.15.134:0/4039242801 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): get_auth_request con 0x7f682c137220 auth_method 0 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6830bcd640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c136e40 0x7f682c134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient(hunting): handle_auth_done global_id 74610 payload 824 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c135dc0 msgr2=0x7f682c09f2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c135dc0 0x7f682c09f2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c136e40 msgr2=0x7f682c134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c136e40 0x7f682c134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 monclient: found mon.noname-a 2024-09-16T15:12:18.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200094d0 con 0x7f682c137220 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6830bcd640 10 monclient: get_auth_request con 0x7f682c136e40 auth_method 0 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.222+0000 7f6830bcd640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c136e40 0x7f682c134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 msgr2=0x7f682c1352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6831bcf640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f682402a9f0 tx=0x7f6824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6824007ea0 con 0x7f682c137220 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: dump: 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.239 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 5 monclient: authenticate success, global_id 74610 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68240044b0 con 0x7f682c137220 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 10 monclient: _renew_subs 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6828009400 con 0x7f682c137220 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6824004620 con 0x7f682c137220 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681dffb640 10 monclient: dump: 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.241 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f681f7fe640 4 set_mon_vals no callback set 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 10 monclient: _renew_subs 2024-09-16T15:12:18.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.224+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6824009a70 con 0x7f682c137220 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f6833632640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f6833632640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 0x7f682c061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c071b20 con 0x7f682c0610b0 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6824004040 con 0x7f682c137220 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f681dffb640 1 --2- 172.21.15.134:0/4039242801 >> v2:172.21.15.162:6800/980547263 conn(0x7f6810038880 0x7f681003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.225+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f682406b870 con 0x7f682c137220 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 0x7f682c061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient: get_auth_request con 0x7f682c0610b0 auth_method 0 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f6830bcd640 1 --2- 172.21.15.134:0/4039242801 >> v2:172.21.15.162:6800/980547263 conn(0x7f6810038880 0x7f681003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f6830bcd640 10 monclient: get_auth_request con 0x7f6810038880 auth_method 0 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 10 monclient(hunting): handle_auth_done global_id 44929 payload 824 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.226+0000 7f68313ce640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f682c061ae0 con 0x7f682c0610b0 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f68313ce640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 msgr2=0x7f682c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f68313ce640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 0x7f682c061490 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6820006c80 tx=0x7f682000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f6830bcd640 1 --2- 172.21.15.134:0/4039242801 >> v2:172.21.15.162:6800/980547263 conn(0x7f6810038880 0x7f681003ad40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b30 tx=0x7f6828008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f682000cc70 con 0x7f682c0610b0 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f681dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f681dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.228+0000 7f681dffb640 10 monclient: dump: 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.244 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f682000cdd0 con 0x7f682c0610b0 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 msgr2=0x7f682c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 0x7f682c061490 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6820006c80 tx=0x7f682000ed40 comp rx=0 tx=0).stop 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 msgr2=0x7f682c061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.229+0000 7f681dffb640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c0610b0 0x7f682c061490 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f6833632640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f6833632640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 0x7f682c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f6833632640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c065ca0 con 0x7f682c02b260 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 0x7f682c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient: get_auth_request con 0x7f682c02b260 auth_method 0 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.235+0000 7f68313ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.236+0000 7f68313ce640 10 monclient(hunting): handle_auth_done global_id 44938 payload 824 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.236+0000 7f68313ce640 1 -- 172.21.15.134:0/4039242801 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f682c07a680 con 0x7f682c02b260 2024-09-16T15:12:18.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f68313ce640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 msgr2=0x7f682c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f68313ce640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 0x7f682c02b670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6820019000 tx=0x7f6820004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6820023070 con 0x7f682c02b260 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 10 monclient: dump: 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.249 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9340 (secure 0 0 0) 0x7f6820013680 con 0x7f682c02b260 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 msgr2=0x7f682c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 0x7f682c02b670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6820019000 tx=0x7f6820004730 comp rx=0 tx=0).stop 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 msgr2=0x7f682c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.238+0000 7f681dffb640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f682c02b260 0x7f682c02b670 secure :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6820019000 tx=0x7f6820004730 comp rx=0 tx=0).stop 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:18.251 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.252 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.253 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:18.254 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.255 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.257 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:18.259 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.261 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.263 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.266 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.268 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.269 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.270 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.271 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:18.272 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 >> v2:172.21.15.162:6800/980547263 conn(0x7f6810038880 msgr2=0x7f681003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 --2- 172.21.15.134:0/4039242801 >> v2:172.21.15.162:6800/980547263 conn(0x7f6810038880 0x7f681003ad40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b30 tx=0x7f6828008040 comp rx=0 tx=0).stop 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 10 monclient: shutdown 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 msgr2=0x7f682c1352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f682402a9f0 tx=0x7f6824004a40 comp rx=0 tx=0).stop 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6831bcf640 1 -- 172.21.15.134:0/4039242801 reap_dead start 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 shutdown_connections 2024-09-16T15:12:18.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 --2- 172.21.15.134:0/4039242801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c137220 0x7f682c1352a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 >> 172.21.15.134:0/4039242801 conn(0x7f682c0a6110 msgr2=0x7f682c0a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 shutdown_connections 2024-09-16T15:12:18.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.239+0000 7f6833632640 1 -- 172.21.15.134:0/4039242801 wait complete. 2024-09-16T15:12:18.274 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:18.274 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-09-16T15:12:18.274 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.370+0000 7f533b43a640 10 monclient: get_monmap_and_config 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.370+0000 7f533b43a640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.370+0000 7f533b43a640 10 monclient: monmap: 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:18.371802+0000 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:18.371802+0000 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.372 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:18.373 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.371+0000 7f533b43a640 1 Processor -- start 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.371+0000 7f533b43a640 1 -- start start 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.371+0000 7f533b43a640 10 monclient: init 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient: _add_conns ranks=[6,8,5,10,4,3,9,7,1,0,2] 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 0x7f533409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-d con 0x7f5334136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-e con 0x7f5334137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 0x7f5334136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-h con 0x7f5334135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f533409abc0 con 0x7f5334135dc0 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f533409ba50 con 0x7f5334136e40 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f53340a53c0 con 0x7f5334137220 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f533b43a640 10 monclient(hunting): authenticate will time out at 1474.961174s 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53391d6640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 msgr2=0x7f533409c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53391d6640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 0x7f533409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:34484/0 (socket says 172.21.15.134:34484) 2024-09-16T15:12:18.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 -- 172.21.15.134:0/3457141050 learned_addr learned my addr 172.21.15.134:0/3457141050 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): get_auth_request con 0x7f5334137220 auth_method 0 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 0x7f5334136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 10 monclient(hunting): get_auth_request con 0x7f5334135dc0 auth_method 0 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient(hunting): handle_auth_done global_id 84302 payload 274 2024-09-16T15:12:18.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 -- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 msgr2=0x7f533409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 0x7f533409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 -- 172.21.15.134:0/3457141050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 msgr2=0x7f5334136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 0x7f5334136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 monclient: found mon.noname-e 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53389d5640 1 -- 172.21.15.134:0/3457141050 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334073250 con 0x7f5334137220 2024-09-16T15:12:18.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.372+0000 7f53399d7640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 0x7f5334136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:12:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f53389d5640 1 -- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 msgr2=0x7f5334137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f53389d5640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53240098a0 tx=0x7f532402a0a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=a6632dfdd936bba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 1 -- 172.21.15.134:0/3457141050 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f532402ae00 con 0x7f5334137220 2024-09-16T15:12:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: dump: 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 1 -- 172.21.15.134:0/3457141050 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53240307b0 con 0x7f5334137220 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f533b43a640 5 monclient: authenticate success, global_id 84302 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f533b43a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f533b43a640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 1 -- 172.21.15.134:0/3457141050 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5324030d90 con 0x7f5334137220 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f533b43a640 4 set_mon_vals no callback set 2024-09-16T15:12:18.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.375+0000 7f532bfff640 10 monclient: dump: 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 10 monclient: shutdown 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 -- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 msgr2=0x7f5334137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53240098a0 tx=0x7f532402a0a0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 -- 172.21.15.134:0/3457141050 shutdown_connections 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334135dc0 0x7f5334136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5334137220 0x7f5334137600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 --2- 172.21.15.134:0/3457141050 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f5334136e40 0x7f533409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 -- 172.21.15.134:0/3457141050 >> 172.21.15.134:0/3457141050 conn(0x7f53340a6110 msgr2=0x7f53340a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 -- 172.21.15.134:0/3457141050 shutdown_connections 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 1 -- 172.21.15.134:0/3457141050 wait complete. 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.376+0000 7f533b43a640 10 monclient: monmap: 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.391 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 Processor -- start 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 -- start start 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient: init 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient: _add_conns ranks=[7,4,5,2,8,3,1,6,0] 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5334135dc0 0x7f53341cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-i con 0x7f5334135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:18.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 0x7f5334134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-c con 0x7f5334136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334137220 0x7f53341354d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): picked mon.noname-h con 0x7f5334137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53341cb760 con 0x7f5334136e40 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53341254a0 con 0x7f5334137220 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f53340a89a0 con 0x7f5334135dc0 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f533b43a640 10 monclient(hunting): authenticate will time out at 1474.966844s 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 0x7f5334134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 0x7f5334134f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56944/0 (socket says 172.21.15.134:56944) 2024-09-16T15:12:18.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 1 -- 172.21.15.134:0/1866679388 learned_addr learned my addr 172.21.15.134:0/1866679388 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 10 monclient(hunting): get_auth_request con 0x7f5334136e40 auth_method 0 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.377+0000 7f53389d5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53399d7640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334137220 0x7f53341354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient(hunting): handle_auth_done global_id 5593 payload 824 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5334135dc0 0x7f53341cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5334135dc0 msgr2=0x7f53341cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5334135dc0 0x7f53341cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334137220 msgr2=0x7f53341354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334137220 0x7f53341354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 monclient: found mon.noname-c 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53389d5640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240094d0 con 0x7f5334136e40 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53399d7640 10 monclient: get_auth_request con 0x7f5334137220 auth_method 0 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53391d6640 10 monclient: get_auth_request con 0x7f5334135dc0 auth_method 0 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53399d7640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5334137220 0x7f53341354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.378+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5334135dc0 0x7f53341cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f53389d5640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 msgr2=0x7f5334134f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f53389d5640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 0x7f5334134f90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5324035040 tx=0x7f53240046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5324030e20 con 0x7f5334136e40 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: dump: 2024-09-16T15:12:18.395 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.396 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f533b43a640 5 monclient: authenticate success, global_id 5593 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5324003a80 con 0x7f5334136e40 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5324003bf0 con 0x7f5334136e40 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f533b43a640 10 monclient: _renew_subs 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f533b43a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5334136320 con 0x7f5334136e40 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f532b7fe640 4 set_mon_vals no callback set 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.380+0000 7f5329ffb640 10 monclient: dump: 2024-09-16T15:12:18.397 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 10 monclient: _renew_subs 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5334136850 con 0x7f5334136e40 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5324039040 con 0x7f5334136e40 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 0x7f5334121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5334071b20 con 0x7f5334121990 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f5329ffb640 1 --2- 172.21.15.134:0/1866679388 >> v2:172.21.15.162:6800/980547263 conn(0x7f53180388b0 0x7f531803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f532406a8d0 con 0x7f5334136e40 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 0x7f5334121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.381+0000 7f53391d6640 10 monclient: get_auth_request con 0x7f5334121990 auth_method 0 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53399d7640 1 --2- 172.21.15.134:0/1866679388 >> v2:172.21.15.162:6800/980547263 conn(0x7f53180388b0 0x7f531803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53399d7640 10 monclient: get_auth_request con 0x7f53180388b0 auth_method 0 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 10 monclient(hunting): handle_auth_done global_id 84207 payload 824 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.382+0000 7f53391d6640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5334061ae0 con 0x7f5334121990 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f53399d7640 1 --2- 172.21.15.134:0/1866679388 >> v2:172.21.15.162:6800/980547263 conn(0x7f53180388b0 0x7f531803ad70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f532c0094c0 tx=0x7f532c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f53391d6640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 msgr2=0x7f5334121d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 0x7f5334121d70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f533000c6f0 tx=0x7f533000cbb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5330003bc0 con 0x7f5334121990 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f5329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f5329ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.384+0000 7f5329ffb640 10 monclient: dump: 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.400 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5330003d20 con 0x7f5334121990 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 msgr2=0x7f5334121d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 0x7f5334121d70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f533000c6f0 tx=0x7f533000cbb0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 msgr2=0x7f5334121d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.385+0000 7f5329ffb640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5334121990 0x7f5334121d70 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f533000c6f0 tx=0x7f533000cbb0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f533b43a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f533b43a640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 0x7f53340a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f533b43a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53341251c0 con 0x7f53341222b0 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 0x7f53340a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient: get_auth_request con 0x7f53341222b0 auth_method 0 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 10 monclient(hunting): handle_auth_done global_id 84216 payload 824 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.391+0000 7f53391d6640 1 -- 172.21.15.134:0/1866679388 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f533407a680 con 0x7f53341222b0 2024-09-16T15:12:18.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.393+0000 7f53391d6640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 msgr2=0x7f53340a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.393+0000 7f53391d6640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 0x7f53340a1a20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5330014000 tx=0x7f53300059d0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5330021070 con 0x7f53341222b0 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 10 monclient: dump: 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.405 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9174 (secure 0 0 0) 0x7f533001c110 con 0x7f53341222b0 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 msgr2=0x7f53340a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 0x7f53340a1a20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5330014000 tx=0x7f53300059d0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 msgr2=0x7f53340a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f5329ffb640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53341222b0 0x7f53340a1a20 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5330014000 tx=0x7f53300059d0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:18.407 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.408 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.409 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:18.410 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:18.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.413 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.417 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.420 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.423 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.425 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.426 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.427 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:18.428 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:18.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 >> v2:172.21.15.162:6800/980547263 conn(0x7f53180388b0 msgr2=0x7f531803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 --2- 172.21.15.134:0/1866679388 >> v2:172.21.15.162:6800/980547263 conn(0x7f53180388b0 0x7f531803ad70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f532c0094c0 tx=0x7f532c00b040 comp rx=0 tx=0).stop 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 10 monclient: shutdown 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 msgr2=0x7f5334134f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 --2- 172.21.15.134:0/1866679388 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5334136e40 0x7f5334134f90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5324035040 tx=0x7f53240046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f53399d7640 1 -- 172.21.15.134:0/1866679388 reap_dead start 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f53399d7640 1 -- 172.21.15.134:0/1866679388 reap_dead start 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 shutdown_connections 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 >> 172.21.15.134:0/1866679388 conn(0x7f53340a6110 msgr2=0x7f533406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 shutdown_connections 2024-09-16T15:12:18.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.394+0000 7f533b43a640 1 -- 172.21.15.134:0/1866679388 wait complete. 2024-09-16T15:12:18.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:18.431 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-09-16T15:12:18.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.526+0000 7f045e219640 10 monclient: get_monmap_and_config 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.526+0000 7f045e219640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.527+0000 7f045e219640 10 monclient: monmap: 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:18.527929+0000 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:18.527929+0000 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.528 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:18.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.527+0000 7f045e219640 1 Processor -- start 2024-09-16T15:12:18.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.527+0000 7f045e219640 1 -- start start 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.527+0000 7f045e219640 10 monclient: init 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient: _add_conns ranks=[9,6,1,10,8,3,2,4,7,5,0] 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0458136eb0 0x7f045809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): picked mon.noname-a con 0x7f0458136eb0 addr v2:172.21.15.134:3300/0 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 0x7f0458137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): picked mon.noname-d con 0x7f0458137290 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0458133d10 addr v1:172.21.15.162:6789/0 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0458073250 con 0x7f0458133d10 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04580a5560 con 0x7f0458136eb0 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f045809b240 con 0x7f0458137290 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f045e219640 10 monclient(hunting): authenticate will time out at 1475.117272s 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f0456ffd640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 msgr2=0x7f0458137690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 0x7f0458137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0458136eb0 0x7f045809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0458136eb0 0x7f045809bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57684/0 (socket says 172.21.15.134:57684) 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 -- 172.21.15.134:0/1601638194 learned_addr learned my addr 172.21.15.134:0/1601638194 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): get_auth_request con 0x7f0458136eb0 auth_method 0 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient(hunting): handle_auth_done global_id 74619 payload 274 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 -- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 msgr2=0x7f0458137690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 --2- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 0x7f0458137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 -- 172.21.15.134:0/1601638194 >> v1:172.21.15.162:6789/0 conn(0x7f0458133d10 legacy=0x7f0458134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 monclient: found mon.noname-a 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.528+0000 7f04577fe640 1 -- 172.21.15.134:0/1601638194 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045809b520 con 0x7f0458136eb0 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04577fe640 1 -- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458136eb0 msgr2=0x7f045809bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04577fe640 1 --2- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458136eb0 0x7f045809bf00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0448009870 tx=0x7f044802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db89282da53898fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 1 -- 172.21.15.134:0/1601638194 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04480307b0 con 0x7f0458136eb0 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: dump: 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.540 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 1 -- 172.21.15.134:0/1601638194 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448030930 con 0x7f0458136eb0 2024-09-16T15:12:18.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f045e219640 5 monclient: authenticate success, global_id 74619 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f045e219640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 1 -- 172.21.15.134:0/1601638194 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f04480313a0 con 0x7f0458136eb0 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f04567fc640 10 monclient: dump: 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.542 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f045e219640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.531+0000 7f045e219640 4 set_mon_vals no callback set 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 10 monclient: shutdown 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 -- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458136eb0 msgr2=0x7f045809bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 --2- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458136eb0 0x7f045809bf00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0448009870 tx=0x7f044802a110 comp rx=0 tx=0).stop 2024-09-16T15:12:18.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 -- 172.21.15.134:0/1601638194 shutdown_connections 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 --2- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458137290 0x7f0458137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 --2- 172.21.15.134:0/1601638194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458136eb0 0x7f045809bf00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 -- 172.21.15.134:0/1601638194 >> 172.21.15.134:0/1601638194 conn(0x7f04580a99a0 msgr2=0x7f04580a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 -- 172.21.15.134:0/1601638194 shutdown_connections 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 1 -- 172.21.15.134:0/1601638194 wait complete. 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.532+0000 7f045e219640 10 monclient: monmap: 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:18.544 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 Processor -- start 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 -- start start 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient: init 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient: _add_conns ranks=[2,6,0,4,1,7,5,3,8] 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0458133d10 0x7f04581cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): picked mon.noname-b con 0x7f0458133d10 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:18.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458136eb0 0x7f0458133060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): picked mon.noname-d con 0x7f0458136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f04577fe640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0458133d10 msgr2=0x7f04581cb800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): picked mon.noname-a con 0x7f0458137290 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581cbd40 con 0x7f0458137290 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0458120180 con 0x7f0458133d10 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f04577fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0458133d10 0x7f04581cb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f04581339d0 con 0x7f0458136eb0 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f045e219640 10 monclient(hunting): authenticate will time out at 1475.122889s 2024-09-16T15:12:18.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0456ffd640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458136eb0 msgr2=0x7f0458133060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458136eb0 0x7f0458133060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57698/0 (socket says 172.21.15.134:57698) 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 learned_addr learned my addr 172.21.15.134:0/3609375668 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 10 monclient(hunting): get_auth_request con 0x7f0458137290 auth_method 0 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.533+0000 7f0457fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient(hunting): handle_auth_done global_id 74628 payload 824 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458136eb0 msgr2=0x7f0458133060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0458136eb0 0x7f0458133060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0458133d10 msgr2=0x7f04581cb800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0458133d10 0x7f04581cb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 monclient: found mon.noname-a 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.534+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04480094d0 con 0x7f0458137290 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 msgr2=0x7f045809d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f0457fff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f045000c7b0 tx=0x7f045000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0450004960 con 0x7f0458137290 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: dump: 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.548 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450004ae0 con 0x7f0458137290 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 5 monclient: authenticate success, global_id 74628 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 10 monclient: _renew_subs 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04580a77f0 con 0x7f0458137290 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0450004c50 con 0x7f0458137290 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f043ffff640 10 monclient: dump: 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.550 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f0455ffb640 4 set_mon_vals no callback set 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 10 monclient: _renew_subs 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:12:18.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.536+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458120a20 con 0x7f0458137290 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f045e219640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f045000aa60 con 0x7f0458137290 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f045e219640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 0x7f045809d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0458073880 con 0x7f04581335a0 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f043ffff640 1 --2- 172.21.15.134:0/3609375668 >> v2:172.21.15.162:6800/980547263 conn(0x7f0444038880 0x7f044403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(519..519 src has 1..519) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f045004a2e0 con 0x7f0458137290 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 0x7f045809d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient: get_auth_request con 0x7f04581335a0 auth_method 0 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f0456ffd640 1 --2- 172.21.15.134:0/3609375668 >> v2:172.21.15.162:6800/980547263 conn(0x7f0444038880 0x7f044403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.537+0000 7f0456ffd640 10 monclient: get_auth_request con 0x7f0444038880 auth_method 0 2024-09-16T15:12:18.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.538+0000 7f04577fe640 10 monclient(hunting): handle_auth_done global_id 44879 payload 824 2024-09-16T15:12:18.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.538+0000 7f04577fe640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0458061ae0 con 0x7f04581335a0 2024-09-16T15:12:18.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f04577fe640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 msgr2=0x7f045809d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f04577fe640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 0x7f045809d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f04480046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f0456ffd640 1 --2- 172.21.15.134:0/3609375668 >> v2:172.21.15.162:6800/980547263 conn(0x7f0444038880 0x7f044403ad40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f045809ec30 tx=0x7f044c006cb0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0448004250 con 0x7f04581335a0 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f043ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f043ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.540+0000 7f043ffff640 10 monclient: dump: 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.556 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f04480043b0 con 0x7f04581335a0 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 msgr2=0x7f045809d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 0x7f045809d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f04480046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 msgr2=0x7f045809d9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.541+0000 7f043ffff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f04581335a0 0x7f045809d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f045e219640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:18.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f045e219640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 0x7f04581341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f045e219640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0458071b20 con 0x7f045809df00 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 0x7f04581341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient: get_auth_request con 0x7f045809df00 auth_method 0 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 10 monclient(hunting): handle_auth_done global_id 44888 payload 824 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.547+0000 7f04577fe640 1 -- 172.21.15.134:0/3609375668 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f045807a680 con 0x7f045809df00 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.549+0000 7f04577fe640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 msgr2=0x7f04581341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.549+0000 7f04577fe640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 0x7f04581341f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f044802afd0 tx=0x7f0448009240 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f044803c8b0 con 0x7f045809df00 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 10 monclient: dump: 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.559 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9506 (secure 0 0 0) 0x7f044803c440 con 0x7f045809df00 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 msgr2=0x7f04581341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 0x7f04581341f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f044802afd0 tx=0x7f0448009240 comp rx=0 tx=0).stop 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 msgr2=0x7f04581341f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f043ffff640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f045809df00 0x7f04581341f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.560 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.561 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.562 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:18.563 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.564 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:18.565 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:18.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.569 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.572 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.574 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.576 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.577 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:18.578 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.579 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.580 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:18.581 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:18.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 >> v2:172.21.15.162:6800/980547263 conn(0x7f0444038880 msgr2=0x7f044403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 --2- 172.21.15.134:0/3609375668 >> v2:172.21.15.162:6800/980547263 conn(0x7f0444038880 0x7f044403ad40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f045809ec30 tx=0x7f044c006cb0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 10 monclient: shutdown 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 msgr2=0x7f045809d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f045000c7b0 tx=0x7f045000cc70 comp rx=0 tx=0).stop 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f0457fff640 1 -- 172.21.15.134:0/3609375668 reap_dead start 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 shutdown_connections 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 --2- 172.21.15.134:0/3609375668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0458137290 0x7f045809d480 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.550+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 >> 172.21.15.134:0/3609375668 conn(0x7f04580a99a0 msgr2=0x7f045806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.551+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 shutdown_connections 2024-09-16T15:12:18.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.551+0000 7f045e219640 1 -- 172.21.15.134:0/3609375668 wait complete. 2024-09-16T15:12:18.583 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:18.583 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-09-16T15:12:18.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:2559: done (0 left) 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:2560: copy_from oid 36 from oid 48 current snap is 250 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:2561: rollback oid 50 current snap is 250 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 238 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:2562: write oid 34 current snap is 250 2024-09-16T15:12:18.673 INFO:tasks.rados.rados.0.smithi162.stdout:2562: seq_num 906 ranges {427142=421601,1400333=440643,2613702=450326} 2024-09-16T15:12:18.678 INFO:tasks.rados.rados.0.smithi162.stdout:2562: writing smithi16234619-34 from 427142 to 848743 tid 1 2024-09-16T15:12:18.684 INFO:tasks.rados.rados.0.smithi162.stdout:2562: writing smithi16234619-34 from 1400333 to 1840976 tid 2 2024-09-16T15:12:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2562: writing smithi16234619-34 from 2613702 to 3064028 tid 3 2024-09-16T15:12:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2563: snap_create 2024-09-16T15:12:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2560: finishing copy_from to smithi16234619-36 2024-09-16T15:12:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2560: got expected ENOENT (src dne) 2024-09-16T15:12:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2560: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:12:18.693 INFO:tasks.rados.rados.0.smithi162.stdout:2561: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.698+0000 7fe0500e9640 10 monclient: get_monmap_and_config 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.698+0000 7fe0500e9640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.699+0000 7fe0500e9640 10 monclient: monmap: 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:12:18.699910+0000 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:12:18.699910+0000 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.700 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:12:18.701 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.699+0000 7fe0500e9640 1 Processor -- start 2024-09-16T15:12:18.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.699+0000 7fe0500e9640 1 -- start start 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.699+0000 7fe0500e9640 10 monclient: init 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient: _add_conns ranks=[6,7,2,8,10,5,3,0,1,9,4] 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 0x7fe0481342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-d con 0x7fe048133f10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-i con 0x7fe048134830 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 0x7fe0481351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-b con 0x7fe048071bd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480a3d50 con 0x7fe048071bd0 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe04806f630 con 0x7fe048133f10 2024-09-16T15:12:18.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480a1e10 con 0x7fe048134830 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe0500e9640 10 monclient(hunting): authenticate will time out at 1475.289324s 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04de85640 1 -- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 msgr2=0x7fe0481342f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3303/0 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04de85640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 0x7fe0481342f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04e686640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 msgr2=0x7fe0481351f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04e686640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 0x7fe0481351f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:53362/0 (socket says 172.21.15.134:53362) 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 1 -- 172.21.15.134:0/2699846445 learned_addr learned my addr 172.21.15.134:0/2699846445 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): get_auth_request con 0x7fe048134830 auth_method 0 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 10 monclient(hunting): handle_auth_done global_id 54673 payload 274 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 1 -- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 msgr2=0x7fe0481342f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.700+0000 7fe04d684640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 0x7fe0481342f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 1 -- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 msgr2=0x7fe0481351f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 0x7fe0481351f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 1 monclient: found mon.noname-i 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:12:18.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.701+0000 7fe04d684640 1 -- 172.21.15.134:0/2699846445 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048073250 con 0x7fe048134830 2024-09-16T15:12:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.703+0000 7fe04d684640 1 -- 172.21.15.134:0/2699846445 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 msgr2=0x7fe048134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.703+0000 7fe04d684640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0380098a0 tx=0x7fe03802a0b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=4970815e4cb9d119 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 1 -- 172.21.15.134:0/2699846445 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0380307b0 con 0x7fe048134830 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: dump: 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.712 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 1 -- 172.21.15.134:0/2699846445 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038030930 con 0x7fe048134830 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe0500e9640 5 monclient: authenticate success, global_id 54673 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe0500e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 1 -- 172.21.15.134:0/2699846445 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0380313a0 con 0x7fe048134830 2024-09-16T15:12:18.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe04ce83640 10 monclient: dump: 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.714 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe0500e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.704+0000 7fe0500e9640 4 set_mon_vals no callback set 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 10 monclient: shutdown 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 -- 172.21.15.134:0/2699846445 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 msgr2=0x7fe048134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0380098a0 tx=0x7fe03802a0b0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 -- 172.21.15.134:0/2699846445 shutdown_connections 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048071bd0 0x7fe0481351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048134830 0x7fe048134cb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 --2- 172.21.15.134:0/2699846445 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe048133f10 0x7fe0481342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 -- 172.21.15.134:0/2699846445 >> 172.21.15.134:0/2699846445 conn(0x7fe0480a7410 msgr2=0x7fe0480a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 -- 172.21.15.134:0/2699846445 shutdown_connections 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 1 -- 172.21.15.134:0/2699846445 wait complete. 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 10 monclient: build_initial_monmap 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.705+0000 7fe0500e9640 10 monclient: monmap: 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:12:18.716 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 Processor -- start 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 -- start start 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient: init 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient: _add_conns ranks=[7,0,2,3,6,5,4,1,8] 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 0x7fe048132d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-i con 0x7fe048071bd0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048133f10 0x7fe048133240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-a con 0x7fe048133f10 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048134830 0x7fe04811da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): picked mon.noname-b con 0x7fe048134830 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481ca010 con 0x7fe048133f10 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe048120100 con 0x7fe048134830 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481206c0 con 0x7fe048071bd0 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe0500e9640 10 monclient(hunting): authenticate will time out at 1475.295833s 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04e686640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048134830 msgr2=0x7fe04811da60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04e686640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048134830 0x7fe04811da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:12:18.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04de85640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 0x7fe048132d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04d684640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048133f10 0x7fe048133240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04de85640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 0x7fe048132d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:53378/0 (socket says 172.21.15.134:53378) 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04d684640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048133f10 0x7fe048133240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57706/0 (socket says 172.21.15.134:57706) 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.706+0000 7fe04de85640 1 -- 172.21.15.134:0/800203009 learned_addr learned my addr 172.21.15.134:0/800203009 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): get_auth_request con 0x7fe048071bd0 auth_method 0 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): get_auth_request con 0x7fe048133f10 auth_method 0 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient(hunting): handle_auth_done global_id 54682 payload 824 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048134830 msgr2=0x7fe04811da60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe048134830 0x7fe04811da60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048133f10 msgr2=0x7fe048133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe048133f10 0x7fe048133240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient: _finish_hunting 0 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 monclient: found mon.noname-i 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.707+0000 7fe04de85640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380094d0 con 0x7fe048071bd0 2024-09-16T15:12:18.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe04de85640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 msgr2=0x7fe048132d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe04de85640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 0x7fe048132d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b580 tx=0x7fe04400ba40 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe0440092c0 con 0x7fe048071bd0 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: dump: 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.721 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe0500e9640 5 monclient: authenticate success, global_id 54682 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044003dd0 con 0x7fe048071bd0 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe0500e9640 10 monclient: _renew_subs 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe0500e9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0480a1730 con 0x7fe048071bd0 2024-09-16T15:12:18.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe044010480 con 0x7fe048071bd0 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03ffff640 4 set_mon_vals no callback set 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.709+0000 7fe03e7fc640 10 monclient: dump: 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 10 monclient: _renew_subs 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:12:18.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0480a1220 con 0x7fe048071bd0 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2561: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2561: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1051 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: finishing write tid 1 to smithi16234619-34 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: finishing write tid 2 to smithi16234619-34 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: finishing write tid 3 to smithi16234619-34 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: finishing write tid 4 to smithi16234619-34 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: finishing write tid 5 to smithi16234619-34 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: oid 34 updating version 0 to 1052 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: oid 34 updating version 1052 to 1054 2024-09-16T15:12:18.726 INFO:tasks.rados.rados.0.smithi162.stdout:2562: oid 34 version 1054 is already newer than 1053 2024-09-16T15:12:18.727 INFO:tasks.rados.rados.0.smithi162.stdout:2562: oid 34 updating version 1054 to 1055 2024-09-16T15:12:18.727 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1055 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:12:18.727 INFO:tasks.rados.rados.0.smithi162.stdout:2562: left oid 34 (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:12:18.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 0x7fe04811e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.710+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe048073880 con 0x7fe04811dfa0 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe0440050f0 con 0x7fe048071bd0 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 0x7fe04811e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient: get_auth_request con 0x7fe04811dfa0 auth_method 0 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe03e7fc640 1 --2- 172.21.15.134:0/800203009 >> v2:172.21.15.162:6800/980547263 conn(0x7fe02c038880 0x7fe02c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(520..520 src has 1..520) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fe044049f50 con 0x7fe048071bd0 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04e686640 1 --2- 172.21.15.134:0/800203009 >> v2:172.21.15.162:6800/980547263 conn(0x7fe02c038880 0x7fe02c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04e686640 10 monclient: get_auth_request con 0x7fe02c038880 auth_method 0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 10 monclient(hunting): handle_auth_done global_id 54691 payload 824 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.711+0000 7fe04d684640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe048061ae0 con 0x7fe04811dfa0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.713+0000 7fe04d684640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 msgr2=0x7fe04811e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.713+0000 7fe04d684640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 0x7fe04811e380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe038034000 tx=0x7fe0380046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.713+0000 7fe04e686640 1 --2- 172.21.15.134:0/800203009 >> v2:172.21.15.162:6800/980547263 conn(0x7fe02c038880 0x7fe02c03ad40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0480a84d0 tx=0x7fe040009230 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe038003a80 con 0x7fe04811dfa0 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 10 monclient: dump: 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.731 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.732 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe038003be0 con 0x7fe04811dfa0 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 msgr2=0x7fe04811e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 0x7fe04811e380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe038034000 tx=0x7fe0380046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 msgr2=0x7fe04811e380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.714+0000 7fe03e7fc640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04811dfa0 0x7fe04811e380 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe038034000 tx=0x7fe0380046c0 comp rx=0 tx=0).stop 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe0500e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe0500e9640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 0x7fe04809b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe0500e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe048065df0 con 0x7fe04809b310 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 0x7fe04809b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:12:18.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient: get_auth_request con 0x7fe04809b310 auth_method 0 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.721+0000 7fe04d684640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.722+0000 7fe04d684640 10 monclient(hunting): handle_auth_done global_id 54700 payload 824 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.722+0000 7fe04d684640 1 -- 172.21.15.134:0/800203009 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe04807a680 con 0x7fe04809b310 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe04d684640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 msgr2=0x7fe04809b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe04d684640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 0x7fe04809b720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe038000c00 tx=0x7fe038008030 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe038048e40 con 0x7fe04809b310 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:12:18.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 10 monclient: dump: 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:12:18.735 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9340 (secure 0 0 0) 0x7fe03804c070 con 0x7fe04809b310 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 msgr2=0x7fe04809b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 0x7fe04809b720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe038000c00 tx=0x7fe038008030 comp rx=0 tx=0).stop 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 msgr2=0x7fe04809b720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.724+0000 7fe03e7fc640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe04809b310 0x7fe04809b720 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe038000c00 tx=0x7fe038008030 comp rx=0 tx=0).stop 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.736 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 102, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:12:18.737 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.738 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:12:18.739 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:12:18.740 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.741 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:12:18.743 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.745 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.746 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.747 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:12:18.749 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:12:18.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:12:18.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:12:18.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:12:18.757 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:12:18.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 >> v2:172.21.15.162:6800/980547263 conn(0x7fe02c038880 msgr2=0x7fe02c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 --2- 172.21.15.134:0/800203009 >> v2:172.21.15.162:6800/980547263 conn(0x7fe02c038880 0x7fe02c03ad40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0480a84d0 tx=0x7fe040009230 comp rx=0 tx=0).stop 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 10 monclient: shutdown 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 msgr2=0x7fe048132d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 --2- 172.21.15.134:0/800203009 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe048071bd0 0x7fe048132d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b580 tx=0x7fe04400ba40 comp rx=0 tx=0).stop 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe04e686640 1 -- 172.21.15.134:0/800203009 reap_dead start 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe04e686640 1 -- 172.21.15.134:0/800203009 reap_dead start 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 shutdown_connections 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 >> 172.21.15.134:0/800203009 conn(0x7fe0480a7410 msgr2=0x7fe0481b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 shutdown_connections 2024-09-16T15:12:18.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:12:18.725+0000 7fe0500e9640 1 -- 172.21.15.134:0/800203009 wait complete. 2024-09-16T15:12:18.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-09-16T15:12:18.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:12:19.025 INFO:tasks.rados.rados.0.smithi162.stdout:2560: done (3 left) 2024-09-16T15:12:19.025 INFO:tasks.rados.rados.0.smithi162.stdout:2561: done (2 left) 2024-09-16T15:12:19.025 INFO:tasks.rados.rados.0.smithi162.stdout:2562: done (1 left) 2024-09-16T15:12:19.025 INFO:tasks.rados.rados.0.smithi162.stdout:2563: done (0 left) 2024-09-16T15:12:19.025 INFO:tasks.rados.rados.0.smithi162.stdout:2564: read oid 15 snap -1 2024-09-16T15:12:19.026 INFO:tasks.rados.rados.0.smithi162.stdout:2564: expect (ObjNum 739 snap 194 seq_num 739) 2024-09-16T15:12:19.026 INFO:tasks.rados.rados.0.smithi162.stdout:2565: rollback oid 28 current snap is 251 2024-09-16T15:12:19.026 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 248 2024-09-16T15:12:19.026 INFO:tasks.rados.rados.0.smithi162.stdout:2566: write oid 50 current snap is 251 2024-09-16T15:12:19.026 INFO:tasks.rados.rados.0.smithi162.stdout:2566: seq_num 907 ranges {443949=496850,1458815=584301,2641125=602824} 2024-09-16T15:12:19.030 INFO:tasks.rados.rados.0.smithi162.stdout:2566: writing smithi16234619-50 from 443949 to 940799 tid 1 2024-09-16T15:12:19.036 INFO:tasks.rados.rados.0.smithi162.stdout:2566: writing smithi16234619-50 from 1458815 to 2043116 tid 2 2024-09-16T15:12:19.043 INFO:tasks.rados.rados.0.smithi162.stdout:2566: writing smithi16234619-50 from 2641125 to 3243949 tid 3 2024-09-16T15:12:19.043 INFO:tasks.rados.rados.0.smithi162.stdout:2567: write oid 44 current snap is 251 2024-09-16T15:12:19.044 INFO:tasks.rados.rados.0.smithi162.stdout:2567: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2024-09-16T15:12:19.049 INFO:tasks.rados.rados.0.smithi162.stdout:2567: writing smithi16234619-44 from 460756 to 1032855 tid 1 2024-09-16T15:12:19.055 INFO:tasks.rados.rados.0.smithi162.stdout:2567: writing smithi16234619-44 from 1517297 to 1961609 tid 2 2024-09-16T15:12:19.062 INFO:tasks.rados.rados.0.smithi162.stdout:2567: writing smithi16234619-44 from 2668548 to 3141418 tid 3 2024-09-16T15:12:19.062 INFO:tasks.rados.rados.0.smithi162.stdout:2568: delete oid 16 current snap is 251 2024-09-16T15:12:19.062 INFO:tasks.rados.rados.0.smithi162.stdout:2564: expect (ObjNum 739 snap 194 seq_num 739) 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2565: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1131 (ObjNum 129 snap 0 seq_num 864475736) dirty dne 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: finishing write tid 1 to smithi16234619-50 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: finishing write tid 2 to smithi16234619-50 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: finishing write tid 3 to smithi16234619-50 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: finishing write tid 4 to smithi16234619-50 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: finishing write tid 5 to smithi16234619-50 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: oid 50 updating version 0 to 1059 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: oid 50 updating version 1059 to 1060 2024-09-16T15:12:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2566: oid 50 version 1060 is already newer than 1058 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2566: oid 50 version 1060 is already newer than 1057 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1060 (ObjNum 907 snap 251 seq_num 907) dirty exists 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2566: left oid 50 (ObjNum 907 snap 251 seq_num 907) 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: finishing write tid 1 to smithi16234619-44 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: finishing write tid 2 to smithi16234619-44 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: finishing write tid 3 to smithi16234619-44 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2564: done (4 left) 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2565: done (3 left) 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2566: done (2 left) 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2568: done (1 left) 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2569: rollback oid 23 current snap is 251 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 250 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: finishing write tid 4 to smithi16234619-44 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: finishing write tid 5 to smithi16234619-44 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: oid 44 updating version 0 to 971 2024-09-16T15:12:19.179 INFO:tasks.rados.rados.0.smithi162.stdout:2567: oid 44 version 971 is already newer than 970 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2567: oid 44 updating version 971 to 972 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2567: oid 44 updating version 972 to 973 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 973 (ObjNum 908 snap 251 seq_num 908) dirty exists 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2567: left oid 44 (ObjNum 908 snap 251 seq_num 908) 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2567: done (1 left) 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2570: snap_create 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:2569: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:12:19.180 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 597 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:12:20.033 INFO:tasks.rados.rados.0.smithi162.stdout:2569: done (1 left) 2024-09-16T15:12:20.034 INFO:tasks.rados.rados.0.smithi162.stdout:2570: done (0 left) 2024-09-16T15:12:20.034 INFO:tasks.rados.rados.0.smithi162.stdout:2571: delete oid 42 current snap is 252 2024-09-16T15:12:20.037 INFO:tasks.rados.rados.0.smithi162.stdout:2571: done (0 left) 2024-09-16T15:12:20.037 INFO:tasks.rados.rados.0.smithi162.stdout:2572: write (excl) oid 25 current snap is 252 2024-09-16T15:12:20.037 INFO:tasks.rados.rados.0.smithi162.stdout:2572: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2024-09-16T15:12:20.044 INFO:tasks.rados.rados.0.smithi162.stdout:2572: writing smithi16234619-25 from 511177 to 1309023 tid 1 2024-09-16T15:12:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:2572: writing smithi16234619-25 from 1976390 to 2568029 tid 2 2024-09-16T15:12:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:2572: writing smithi16234619-25 from 3034464 to 3311177 tid 3 2024-09-16T15:12:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:2573: write oid 28 current snap is 252 2024-09-16T15:12:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:2573: seq_num 912 ranges {527984=473095,1634872=451650,2661887=634311} 2024-09-16T15:12:20.060 INFO:tasks.rados.rados.0.smithi162.stdout:2573: writing smithi16234619-28 from 527984 to 1001079 tid 1 2024-09-16T15:12:20.066 INFO:tasks.rados.rados.0.smithi162.stdout:2573: writing smithi16234619-28 from 1634872 to 2086522 tid 2 2024-09-16T15:12:20.072 INFO:tasks.rados.rados.0.smithi162.stdout:2573: writing smithi16234619-28 from 2661887 to 3296198 tid 3 2024-09-16T15:12:20.072 INFO:tasks.rados.rados.0.smithi162.stdout:2574: snap_create 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: finishing write tid 1 to smithi16234619-25 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: finishing write tid 2 to smithi16234619-25 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: finishing write tid 3 to smithi16234619-25 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: finishing write tid 4 to smithi16234619-25 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: finishing write tid 5 to smithi16234619-25 2024-09-16T15:12:20.097 INFO:tasks.rados.rados.0.smithi162.stdout:2572: oid 25 updating version 0 to 1064 2024-09-16T15:12:20.098 INFO:tasks.rados.rados.0.smithi162.stdout:2572: oid 25 updating version 1064 to 1065 2024-09-16T15:12:20.098 INFO:tasks.rados.rados.0.smithi162.stdout:2572: oid 25 version 1065 is already newer than 1063 2024-09-16T15:12:20.098 INFO:tasks.rados.rados.0.smithi162.stdout:2572: oid 25 version 1065 is already newer than 1062 2024-09-16T15:12:20.098 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 1065 (ObjNum 911 snap 252 seq_num 911) dirty exists 2024-09-16T15:12:20.098 INFO:tasks.rados.rados.0.smithi162.stdout:2572: left oid 25 (ObjNum 911 snap 252 seq_num 911) 2024-09-16T15:12:20.118 INFO:tasks.rados.rados.0.smithi162.stdout:2573: finishing write tid 1 to smithi16234619-28 2024-09-16T15:12:20.118 INFO:tasks.rados.rados.0.smithi162.stdout:2573: finishing write tid 2 to smithi16234619-28 2024-09-16T15:12:20.118 INFO:tasks.rados.rados.0.smithi162.stdout:2573: finishing write tid 3 to smithi16234619-28 2024-09-16T15:12:20.118 INFO:tasks.rados.rados.0.smithi162.stdout:2573: finishing write tid 4 to smithi16234619-28 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: finishing write tid 5 to smithi16234619-28 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: oid 28 updating version 0 to 1139 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: oid 28 version 1139 is already newer than 1138 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: oid 28 version 1139 is already newer than 1137 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: oid 28 version 1139 is already newer than 1136 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1139 (ObjNum 912 snap 252 seq_num 912) dirty exists 2024-09-16T15:12:20.119 INFO:tasks.rados.rados.0.smithi162.stdout:2573: left oid 28 (ObjNum 912 snap 252 seq_num 912) 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2572: done (2 left) 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2573: done (1 left) 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2574: done (0 left) 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2575: read oid 32 snap 235 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2575: expect (ObjNum 858 snap 227 seq_num 858) 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2576: rollback oid 4 current snap is 253 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 252 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:2577: rollback oid 3 current snap is 253 2024-09-16T15:12:21.044 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 251 2024-09-16T15:12:21.045 INFO:tasks.rados.rados.0.smithi162.stdout:2578: delete oid 5 current snap is 253 2024-09-16T15:12:21.047 INFO:tasks.rados.rados.0.smithi162.stdout:2577: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:12:21.048 INFO:tasks.rados.rados.0.smithi162.stdout:2577: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:12:21.048 INFO:tasks.rados.rados.0.smithi162.stdout:2577: finishing rollback tid 2 to smithi16234619-3 2024-09-16T15:12:21.049 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 124 (ObjNum 900 snap 247 seq_num 900) dirty exists 2024-09-16T15:12:21.049 INFO:tasks.rados.rados.0.smithi162.stdout:2577: done (3 left) 2024-09-16T15:12:21.049 INFO:tasks.rados.rados.0.smithi162.stdout:2578: done (2 left) 2024-09-16T15:12:21.049 INFO:tasks.rados.rados.0.smithi162.stdout:2579: snap_create 2024-09-16T15:12:21.051 INFO:tasks.rados.rados.0.smithi162.stdout:2575: expect (ObjNum 858 snap 227 seq_num 858) 2024-09-16T15:12:21.204 INFO:tasks.rados.rados.0.smithi162.stdout:2576: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:12:21.204 INFO:tasks.rados.rados.0.smithi162.stdout:2576: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:12:21.204 INFO:tasks.rados.rados.0.smithi162.stdout:2576: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:12:21.204 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 920 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:12:22.306 INFO:tasks.rados.rados.0.smithi162.stdout:2575: done (2 left) 2024-09-16T15:12:22.306 INFO:tasks.rados.rados.0.smithi162.stdout:2576: done (1 left) 2024-09-16T15:12:22.306 INFO:tasks.rados.rados.0.smithi162.stdout:2579: done (0 left) 2024-09-16T15:12:22.306 INFO:tasks.rados.rados.0.smithi162.stdout:2580: write oid 30 current snap is 254 2024-09-16T15:12:22.306 INFO:tasks.rados.rados.0.smithi162.stdout:2580: seq_num 914 ranges {561598=739946,1751836=455319,2716733=644865} 2024-09-16T15:12:22.313 INFO:tasks.rados.rados.0.smithi162.stdout:2580: writing smithi16234619-30 from 561598 to 1301544 tid 1 2024-09-16T15:12:22.318 INFO:tasks.rados.rados.0.smithi162.stdout:2580: writing smithi16234619-30 from 1751836 to 2207155 tid 2 2024-09-16T15:12:22.324 INFO:tasks.rados.rados.0.smithi162.stdout:2580: writing smithi16234619-30 from 2716733 to 3361598 tid 3 2024-09-16T15:12:22.324 INFO:tasks.rados.rados.0.smithi162.stdout:2581: delete oid 44 current snap is 254 2024-09-16T15:12:22.328 INFO:tasks.rados.rados.0.smithi162.stdout:2581: done (1 left) 2024-09-16T15:12:22.329 INFO:tasks.rados.rados.0.smithi162.stdout:2582: write oid 47 current snap is 254 2024-09-16T15:12:22.329 INFO:tasks.rados.rados.0.smithi162.stdout:2582: seq_num 916 ranges {595212=490444,1868800=458988,2887932=507280} 2024-09-16T15:12:22.334 INFO:tasks.rados.rados.0.smithi162.stdout:2582: writing smithi16234619-47 from 595212 to 1085656 tid 1 2024-09-16T15:12:22.341 INFO:tasks.rados.rados.0.smithi162.stdout:2582: writing smithi16234619-47 from 1868800 to 2327788 tid 2 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:2582: writing smithi16234619-47 from 2887932 to 3395212 tid 3 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:2583: rollback oid 39 current snap is 254 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 251 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:2584: read oid 1 snap -1 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:2584: expect (ObjNum 903 snap 249 seq_num 903) 2024-09-16T15:12:22.347 INFO:tasks.rados.rados.0.smithi162.stdout:2585: snap_remove snap 242 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2585: done (4 left) 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2586: rollback oid 18 current snap is 254 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 252 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2583: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2583: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2583: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1217 (ObjNum 888 snap 246 seq_num 888) dirty exists 2024-09-16T15:12:23.859 INFO:tasks.rados.rados.0.smithi162.stdout:2584: expect (ObjNum 903 snap 249 seq_num 903) 2024-09-16T15:12:23.963 INFO:tasks.rados.rados.0.smithi162.stdout:2580: finishing write tid 1 to smithi16234619-30 2024-09-16T15:12:23.963 INFO:tasks.rados.rados.0.smithi162.stdout:2580: finishing write tid 2 to smithi16234619-30 2024-09-16T15:12:23.963 INFO:tasks.rados.rados.0.smithi162.stdout:2580: finishing write tid 3 to smithi16234619-30 2024-09-16T15:12:23.963 INFO:tasks.rados.rados.0.smithi162.stdout:2580: finishing write tid 4 to smithi16234619-30 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: finishing write tid 5 to smithi16234619-30 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: oid 30 updating version 0 to 1141 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: oid 30 updating version 1141 to 1142 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: oid 30 version 1142 is already newer than 1140 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: oid 30 updating version 1142 to 1143 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1143 (ObjNum 914 snap 254 seq_num 914) dirty exists 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: left oid 30 (ObjNum 914 snap 254 seq_num 914) 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2582: finishing write tid 1 to smithi16234619-47 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2580: done (4 left) 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2583: done (3 left) 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2584: done (2 left) 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2587: rollback oid 50 current snap is 254 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 249 2024-09-16T15:12:23.964 INFO:tasks.rados.rados.0.smithi162.stdout:2582: finishing write tid 2 to smithi16234619-47 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: finishing write tid 3 to smithi16234619-47 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: finishing write tid 4 to smithi16234619-47 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: finishing write tid 5 to smithi16234619-47 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: oid 47 updating version 0 to 1118 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: oid 47 updating version 1118 to 1119 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: oid 47 version 1119 is already newer than 1116 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: oid 47 version 1119 is already newer than 1117 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1119 (ObjNum 916 snap 254 seq_num 916) dirty exists 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: left oid 47 (ObjNum 916 snap 254 seq_num 916) 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2586: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 0 (ObjNum 209 snap 0 seq_num 1841270819) dirty dne 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2582: done (2 left) 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2586: done (1 left) 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2588: copy_from oid 4 from oid 24 current snap is 254 2024-09-16T15:12:23.965 INFO:tasks.rados.rados.0.smithi162.stdout:2589: snap_remove snap 235 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2589: done (2 left) 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2590: read oid 16 snap -1 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2590: expect deleted 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2591: rollback oid 33 current snap is 254 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 247 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2587: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2587: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2587: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1075 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2588: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:2588: finishing copy_from to smithi16234619-4 2024-09-16T15:12:24.021 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1137 (ObjNum 846 snap 225 seq_num 846) dirty exists 2024-09-16T15:12:24.022 INFO:tasks.rados.rados.0.smithi162.stdout:2587: done (3 left) 2024-09-16T15:12:24.022 INFO:tasks.rados.rados.0.smithi162.stdout:2588: done (2 left) 2024-09-16T15:12:24.022 INFO:tasks.rados.rados.0.smithi162.stdout:2592: snap_remove snap 234 2024-09-16T15:12:25.023 INFO:tasks.rados.rados.0.smithi162.stdout:2592: done (2 left) 2024-09-16T15:12:25.023 INFO:tasks.rados.rados.0.smithi162.stdout:2593: write oid 45 current snap is 254 2024-09-16T15:12:25.023 INFO:tasks.rados.rados.0.smithi162.stdout:2593: seq_num 917 ranges {612019=565693,1810929=602646,3082649=329370} 2024-09-16T15:12:25.030 INFO:tasks.rados.rados.0.smithi162.stdout:2593: writing smithi16234619-45 from 612019 to 1177712 tid 1 2024-09-16T15:12:25.037 INFO:tasks.rados.rados.0.smithi162.stdout:2593: writing smithi16234619-45 from 1810929 to 2413575 tid 2 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:2593: writing smithi16234619-45 from 3082649 to 3412019 tid 3 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:2594: snap_create 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:2591: finishing rollback tid 0 to smithi16234619-33 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:2591: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:2591: finishing rollback tid 2 to smithi16234619-33 2024-09-16T15:12:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 814 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:12:25.071 INFO:tasks.rados.rados.0.smithi162.stdout:2593: finishing write tid 1 to smithi16234619-45 2024-09-16T15:12:25.071 INFO:tasks.rados.rados.0.smithi162.stdout:2593: finishing write tid 2 to smithi16234619-45 2024-09-16T15:12:25.071 INFO:tasks.rados.rados.0.smithi162.stdout:2593: finishing write tid 3 to smithi16234619-45 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: finishing write tid 4 to smithi16234619-45 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: finishing write tid 5 to smithi16234619-45 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: oid 45 updating version 0 to 1264 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: oid 45 version 1264 is already newer than 1262 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: oid 45 version 1264 is already newer than 1263 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: oid 45 version 1264 is already newer than 1261 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1264 (ObjNum 917 snap 254 seq_num 917) dirty exists 2024-09-16T15:12:25.074 INFO:tasks.rados.rados.0.smithi162.stdout:2593: left oid 45 (ObjNum 917 snap 254 seq_num 917) 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2590: done (3 left) 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2591: done (2 left) 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2593: done (1 left) 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2594: done (0 left) 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2595: copy_from oid 42 from oid 34 current snap is 255 2024-09-16T15:12:26.657 INFO:tasks.rados.rados.0.smithi162.stdout:2596: read oid 35 snap -1 2024-09-16T15:12:26.658 INFO:tasks.rados.rados.0.smithi162.stdout:2596: expect deleted 2024-09-16T15:12:26.658 INFO:tasks.rados.rados.0.smithi162.stdout:2597: snap_create 2024-09-16T15:12:26.696 INFO:tasks.rados.rados.0.smithi162.stdout:2595: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:12:26.699 INFO:tasks.rados.rados.0.smithi162.stdout:2595: finishing copy_from to smithi16234619-42 2024-09-16T15:12:26.699 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1055 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:12:27.667 INFO:tasks.rados.rados.0.smithi162.stdout:2595: done (2 left) 2024-09-16T15:12:27.667 INFO:tasks.rados.rados.0.smithi162.stdout:2596: done (1 left) 2024-09-16T15:12:27.667 INFO:tasks.rados.rados.0.smithi162.stdout:2597: done (0 left) 2024-09-16T15:12:27.667 INFO:tasks.rados.rados.0.smithi162.stdout:2598: snap_remove snap 254 2024-09-16T15:12:28.671 INFO:tasks.rados.rados.0.smithi162.stdout:2598: done (0 left) 2024-09-16T15:12:28.672 INFO:tasks.rados.rados.0.smithi162.stdout:2599: read oid 43 snap -1 2024-09-16T15:12:28.672 INFO:tasks.rados.rados.0.smithi162.stdout:2599: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:28.672 INFO:tasks.rados.rados.0.smithi162.stdout:2600: write oid 20 current snap is 256 2024-09-16T15:12:28.672 INFO:tasks.rados.rados.0.smithi162.stdout:2600: seq_num 918 ranges {628826=640942,1869411=462657,2826425=534649} 2024-09-16T15:12:28.678 INFO:tasks.rados.rados.0.smithi162.stdout:2600: writing smithi16234619-20 from 628826 to 1269768 tid 1 2024-09-16T15:12:28.685 INFO:tasks.rados.rados.0.smithi162.stdout:2600: writing smithi16234619-20 from 1869411 to 2332068 tid 2 2024-09-16T15:12:28.690 INFO:tasks.rados.rados.0.smithi162.stdout:2600: writing smithi16234619-20 from 2826425 to 3361074 tid 3 2024-09-16T15:12:28.690 INFO:tasks.rados.rados.0.smithi162.stdout:2601: rollback oid 8 current snap is 256 2024-09-16T15:12:28.690 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 255 2024-09-16T15:12:28.690 INFO:tasks.rados.rados.0.smithi162.stdout:2599: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:28.821 INFO:tasks.rados.rados.0.smithi162.stdout:2601: finishing rollback tid 0 to smithi16234619-8 2024-09-16T15:12:28.821 INFO:tasks.rados.rados.0.smithi162.stdout:2601: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2601: finishing rollback tid 2 to smithi16234619-8 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 908 (ObjNum 867 snap 235 seq_num 867) dirty exists 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: finishing write tid 1 to smithi16234619-20 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: finishing write tid 2 to smithi16234619-20 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: finishing write tid 3 to smithi16234619-20 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: finishing write tid 4 to smithi16234619-20 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: finishing write tid 5 to smithi16234619-20 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: oid 20 updating version 0 to 816 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: oid 20 version 816 is already newer than 815 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: oid 20 updating version 816 to 818 2024-09-16T15:12:28.822 INFO:tasks.rados.rados.0.smithi162.stdout:2600: oid 20 version 818 is already newer than 817 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 818 (ObjNum 918 snap 256 seq_num 918) dirty exists 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2600: left oid 20 (ObjNum 918 snap 256 seq_num 918) 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2599: done (2 left) 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2600: done (1 left) 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2601: done (0 left) 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2602: read oid 25 snap -1 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2602: expect (ObjNum 911 snap 252 seq_num 911) 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2603: write (excl) oid 2 current snap is 256 2024-09-16T15:12:28.823 INFO:tasks.rados.rados.0.smithi162.stdout:2603: seq_num 919 ranges {645633=716191,1927893=606315,3137495=308138} 2024-09-16T15:12:28.828 INFO:tasks.rados.rados.0.smithi162.stdout:2603: writing smithi16234619-2 from 645633 to 1361824 tid 1 2024-09-16T15:12:28.836 INFO:tasks.rados.rados.0.smithi162.stdout:2603: writing smithi16234619-2 from 1927893 to 2534208 tid 2 2024-09-16T15:12:28.840 INFO:tasks.rados.rados.0.smithi162.stdout:2603: writing smithi16234619-2 from 3137495 to 3445633 tid 3 2024-09-16T15:12:28.840 INFO:tasks.rados.rados.0.smithi162.stdout:2604: read oid 13 snap -1 2024-09-16T15:12:28.840 INFO:tasks.rados.rados.0.smithi162.stdout:2604: expect deleted 2024-09-16T15:12:28.841 INFO:tasks.rados.rados.0.smithi162.stdout:2605: delete oid 1 current snap is 256 2024-09-16T15:12:28.841 INFO:tasks.rados.rados.0.smithi162.stdout:2602: expect (ObjNum 911 snap 252 seq_num 911) 2024-09-16T15:12:29.071 INFO:tasks.rados.rados.0.smithi162.stdout:2603: finishing write tid 1 to smithi16234619-2 2024-09-16T15:12:29.071 INFO:tasks.rados.rados.0.smithi162.stdout:2603: finishing write tid 2 to smithi16234619-2 2024-09-16T15:12:29.071 INFO:tasks.rados.rados.0.smithi162.stdout:2603: finishing write tid 3 to smithi16234619-2 2024-09-16T15:12:29.071 INFO:tasks.rados.rados.0.smithi162.stdout:2603: finishing write tid 4 to smithi16234619-2 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2602: done (3 left) 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2604: done (2 left) 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2605: done (1 left) 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2606: write (excl) oid 39 current snap is 256 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: finishing write tid 5 to smithi16234619-2 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: oid 2 updating version 0 to 1269 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: oid 2 version 1269 is already newer than 1268 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: oid 2 version 1269 is already newer than 1267 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: oid 2 version 1269 is already newer than 1266 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1269 (ObjNum 919 snap 256 seq_num 919) dirty exists 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2603: left oid 2 (ObjNum 919 snap 256 seq_num 919) 2024-09-16T15:12:29.072 INFO:tasks.rados.rados.0.smithi162.stdout:2606: seq_num 921 ranges {679247=583042,2044857=609984,3192341=286906} 2024-09-16T15:12:29.080 INFO:tasks.rados.rados.0.smithi162.stdout:2606: writing smithi16234619-39 from 679247 to 1262289 tid 1 2024-09-16T15:12:29.089 INFO:tasks.rados.rados.0.smithi162.stdout:2606: writing smithi16234619-39 from 2044857 to 2654841 tid 2 2024-09-16T15:12:29.093 INFO:tasks.rados.rados.0.smithi162.stdout:2606: writing smithi16234619-39 from 3192341 to 3479247 tid 3 2024-09-16T15:12:29.093 INFO:tasks.rados.rados.0.smithi162.stdout:2603: done (1 left) 2024-09-16T15:12:29.094 INFO:tasks.rados.rados.0.smithi162.stdout:2607: copy_from oid 33 from oid 23 current snap is 256 2024-09-16T15:12:29.094 INFO:tasks.rados.rados.0.smithi162.stdout:2608: write (excl) oid 1 current snap is 256 2024-09-16T15:12:29.094 INFO:tasks.rados.rados.0.smithi162.stdout:2608: seq_num 922 ranges {696054=658291,2103339=753642} 2024-09-16T15:12:29.101 INFO:tasks.rados.rados.0.smithi162.stdout:2608: writing smithi16234619-1 from 696054 to 1354345 tid 1 2024-09-16T15:12:29.109 INFO:tasks.rados.rados.0.smithi162.stdout:2608: writing smithi16234619-1 from 2103339 to 2856981 tid 2 2024-09-16T15:12:29.109 INFO:tasks.rados.rados.0.smithi162.stdout:2609: snap_create 2024-09-16T15:12:29.109 INFO:tasks.rados.rados.0.smithi162.stdout:2607: finishing copy_from to smithi16234619-33 2024-09-16T15:12:29.109 INFO:tasks.rados.rados.0.smithi162.stdout:2607: got expected ENOENT (src dne) 2024-09-16T15:12:29.110 INFO:tasks.rados.rados.0.smithi162.stdout:2607: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:12:29.124 INFO:tasks.rados.rados.0.smithi162.stdout:2606: finishing write tid 1 to smithi16234619-39 2024-09-16T15:12:29.125 INFO:tasks.rados.rados.0.smithi162.stdout:2606: finishing write tid 2 to smithi16234619-39 2024-09-16T15:12:29.125 INFO:tasks.rados.rados.0.smithi162.stdout:2606: finishing write tid 3 to smithi16234619-39 2024-09-16T15:12:29.125 INFO:tasks.rados.rados.0.smithi162.stdout:2606: finishing write tid 4 to smithi16234619-39 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: finishing write tid 5 to smithi16234619-39 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: oid 39 updating version 0 to 1246 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: oid 39 version 1246 is already newer than 1245 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: oid 39 updating version 1246 to 1247 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: oid 39 version 1247 is already newer than 1244 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1247 (ObjNum 921 snap 256 seq_num 921) dirty exists 2024-09-16T15:12:29.126 INFO:tasks.rados.rados.0.smithi162.stdout:2606: left oid 39 (ObjNum 921 snap 256 seq_num 921) 2024-09-16T15:12:29.138 INFO:tasks.rados.rados.0.smithi162.stdout:2608: finishing write tid 1 to smithi16234619-1 2024-09-16T15:12:29.138 INFO:tasks.rados.rados.0.smithi162.stdout:2608: finishing write tid 2 to smithi16234619-1 2024-09-16T15:12:29.138 INFO:tasks.rados.rados.0.smithi162.stdout:2608: finishing write tid 3 to smithi16234619-1 2024-09-16T15:12:29.138 INFO:tasks.rados.rados.0.smithi162.stdout:2608: finishing write tid 4 to smithi16234619-1 2024-09-16T15:12:29.138 INFO:tasks.rados.rados.0.smithi162.stdout:2608: oid 1 updating version 0 to 1139 2024-09-16T15:12:29.139 INFO:tasks.rados.rados.0.smithi162.stdout:2608: oid 1 updating version 1139 to 1141 2024-09-16T15:12:29.139 INFO:tasks.rados.rados.0.smithi162.stdout:2608: oid 1 version 1141 is already newer than 1140 2024-09-16T15:12:29.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1141 (ObjNum 922 snap 256 seq_num 922) dirty exists 2024-09-16T15:12:29.139 INFO:tasks.rados.rados.0.smithi162.stdout:2608: left oid 1 (ObjNum 922 snap 256 seq_num 922) 2024-09-16T15:12:30.065 INFO:tasks.rados.rados.0.smithi162.stdout:2606: done (3 left) 2024-09-16T15:12:30.065 INFO:tasks.rados.rados.0.smithi162.stdout:2607: done (2 left) 2024-09-16T15:12:30.066 INFO:tasks.rados.rados.0.smithi162.stdout:2608: done (1 left) 2024-09-16T15:12:30.066 INFO:tasks.rados.rados.0.smithi162.stdout:2609: done (0 left) 2024-09-16T15:12:30.066 INFO:tasks.rados.rados.0.smithi162.stdout:2610: delete oid 41 current snap is 257 2024-09-16T15:12:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:2610: done (0 left) 2024-09-16T15:12:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:2611: write (excl) oid 4 current snap is 257 2024-09-16T15:12:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:2611: seq_num 924 ranges {729668=408789,1820303=757311,3274610=255058} 2024-09-16T15:12:30.074 INFO:tasks.rados.rados.0.smithi162.stdout:2611: writing smithi16234619-4 from 729668 to 1138457 tid 1 2024-09-16T15:12:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:2611: writing smithi16234619-4 from 1820303 to 2577614 tid 2 2024-09-16T15:12:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:2611: writing smithi16234619-4 from 3274610 to 3529668 tid 3 2024-09-16T15:12:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:2612: read oid 24 snap -1 2024-09-16T15:12:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:2612: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:12:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:2613: write (excl) oid 11 current snap is 257 2024-09-16T15:12:30.088 INFO:tasks.rados.rados.0.smithi162.stdout:2613: seq_num 925 ranges {746475=484038,1762432=617322,2785680=529906} 2024-09-16T15:12:30.094 INFO:tasks.rados.rados.0.smithi162.stdout:2613: writing smithi16234619-11 from 746475 to 1230513 tid 1 2024-09-16T15:12:30.100 INFO:tasks.rados.rados.0.smithi162.stdout:2613: writing smithi16234619-11 from 1762432 to 2379754 tid 2 2024-09-16T15:12:30.106 INFO:tasks.rados.rados.0.smithi162.stdout:2613: writing smithi16234619-11 from 2785680 to 3315586 tid 3 2024-09-16T15:12:30.106 INFO:tasks.rados.rados.0.smithi162.stdout:2614: delete oid 1 current snap is 257 2024-09-16T15:12:30.106 INFO:tasks.rados.rados.0.smithi162.stdout:2612: expect (ObjNum 846 snap 225 seq_num 846) 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: finishing write tid 1 to smithi16234619-4 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: finishing write tid 2 to smithi16234619-4 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: finishing write tid 3 to smithi16234619-4 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: finishing write tid 4 to smithi16234619-4 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: finishing write tid 5 to smithi16234619-4 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: oid 4 updating version 0 to 1143 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: oid 4 updating version 1143 to 1144 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: oid 4 updating version 1144 to 1145 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: oid 4 version 1145 is already newer than 1142 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1145 (ObjNum 924 snap 257 seq_num 924) dirty exists 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2611: left oid 4 (ObjNum 924 snap 257 seq_num 924) 2024-09-16T15:12:30.281 INFO:tasks.rados.rados.0.smithi162.stdout:2613: finishing write tid 1 to smithi16234619-11 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2611: done (3 left) 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2612: done (2 left) 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2614: done (1 left) 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2615: read oid 3 snap -1 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2615: expect (ObjNum 900 snap 247 seq_num 900) 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2616: snap_create 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: finishing write tid 2 to smithi16234619-11 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: finishing write tid 3 to smithi16234619-11 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: finishing write tid 4 to smithi16234619-11 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: finishing write tid 5 to smithi16234619-11 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: oid 11 updating version 0 to 597 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: oid 11 version 597 is already newer than 596 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: oid 11 version 597 is already newer than 595 2024-09-16T15:12:30.282 INFO:tasks.rados.rados.0.smithi162.stdout:2613: oid 11 version 597 is already newer than 594 2024-09-16T15:12:30.283 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 597 (ObjNum 925 snap 257 seq_num 925) dirty exists 2024-09-16T15:12:30.283 INFO:tasks.rados.rados.0.smithi162.stdout:2613: left oid 11 (ObjNum 925 snap 257 seq_num 925) 2024-09-16T15:12:30.283 INFO:tasks.rados.rados.0.smithi162.stdout:2615: expect (ObjNum 900 snap 247 seq_num 900) 2024-09-16T15:12:31.073 INFO:tasks.rados.rados.0.smithi162.stdout:2613: done (2 left) 2024-09-16T15:12:31.073 INFO:tasks.rados.rados.0.smithi162.stdout:2615: done (1 left) 2024-09-16T15:12:31.073 INFO:tasks.rados.rados.0.smithi162.stdout:2616: done (0 left) 2024-09-16T15:12:31.073 INFO:tasks.rados.rados.0.smithi162.stdout:2617: delete oid 40 current snap is 258 2024-09-16T15:12:31.081 INFO:tasks.rados.rados.0.smithi162.stdout:2617: done (0 left) 2024-09-16T15:12:31.081 INFO:tasks.rados.rados.0.smithi162.stdout:2618: read oid 46 snap -1 2024-09-16T15:12:31.081 INFO:tasks.rados.rados.0.smithi162.stdout:2618: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:12:31.081 INFO:tasks.rados.rados.0.smithi162.stdout:2619: write (excl) oid 26 current snap is 258 2024-09-16T15:12:31.081 INFO:tasks.rados.rados.0.smithi162.stdout:2619: seq_num 928 ranges {796896=426138,1654231=764649,2984302=480075} 2024-09-16T15:12:31.087 INFO:tasks.rados.rados.0.smithi162.stdout:2619: writing smithi16234619-26 from 796896 to 1223034 tid 1 2024-09-16T15:12:31.094 INFO:tasks.rados.rados.0.smithi162.stdout:2619: writing smithi16234619-26 from 1654231 to 2418880 tid 2 2024-09-16T15:12:31.100 INFO:tasks.rados.rados.0.smithi162.stdout:2619: writing smithi16234619-26 from 2984302 to 3464377 tid 3 2024-09-16T15:12:31.100 INFO:tasks.rados.rados.0.smithi162.stdout:2620: read oid 27 snap -1 2024-09-16T15:12:31.100 INFO:tasks.rados.rados.0.smithi162.stdout:2620: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:12:31.100 INFO:tasks.rados.rados.0.smithi162.stdout:2621: snap_remove snap 256 2024-09-16T15:12:32.091 INFO:tasks.rados.rados.0.smithi162.stdout:2621: done (3 left) 2024-09-16T15:12:32.092 INFO:tasks.rados.rados.0.smithi162.stdout:2622: snap_create 2024-09-16T15:12:32.092 INFO:tasks.rados.rados.0.smithi162.stdout:2618: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:12:32.201 INFO:tasks.rados.rados.0.smithi162.stdout:2620: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:12:32.336 INFO:tasks.rados.rados.0.smithi162.stdout:2619: finishing write tid 1 to smithi16234619-26 2024-09-16T15:12:32.336 INFO:tasks.rados.rados.0.smithi162.stdout:2619: finishing write tid 2 to smithi16234619-26 2024-09-16T15:12:32.336 INFO:tasks.rados.rados.0.smithi162.stdout:2619: finishing write tid 3 to smithi16234619-26 2024-09-16T15:12:32.336 INFO:tasks.rados.rados.0.smithi162.stdout:2619: finishing write tid 4 to smithi16234619-26 2024-09-16T15:12:32.336 INFO:tasks.rados.rados.0.smithi162.stdout:2619: finishing write tid 5 to smithi16234619-26 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:2619: oid 26 updating version 0 to 1272 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:2619: oid 26 updating version 1272 to 1274 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:2619: oid 26 version 1274 is already newer than 1271 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:2619: oid 26 version 1274 is already newer than 1273 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1274 (ObjNum 928 snap 258 seq_num 928) dirty exists 2024-09-16T15:12:32.337 INFO:tasks.rados.rados.0.smithi162.stdout:2619: left oid 26 (ObjNum 928 snap 258 seq_num 928) 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2618: done (3 left) 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2619: done (2 left) 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2620: done (1 left) 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2622: done (0 left) 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2623: rollback oid 19 current snap is 259 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 249 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2624: read oid 40 snap 258 2024-09-16T15:12:33.106 INFO:tasks.rados.rados.0.smithi162.stdout:2624: expect deleted 2024-09-16T15:12:33.107 INFO:tasks.rados.rados.0.smithi162.stdout:2625: rollback oid 48 current snap is 259 2024-09-16T15:12:33.107 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 48 to 250 2024-09-16T15:12:33.107 INFO:tasks.rados.rados.0.smithi162.stdout:2626: write oid 17 current snap is 259 2024-09-16T15:12:33.107 INFO:tasks.rados.rados.0.smithi162.stdout:2626: seq_num 929 ranges {413703=501387,1596360=624660,2895372=691347} 2024-09-16T15:12:33.111 INFO:tasks.rados.rados.0.smithi162.stdout:2626: writing smithi16234619-17 from 413703 to 915090 tid 1 2024-09-16T15:12:33.118 INFO:tasks.rados.rados.0.smithi162.stdout:2626: writing smithi16234619-17 from 1596360 to 2221020 tid 2 2024-09-16T15:12:33.125 INFO:tasks.rados.rados.0.smithi162.stdout:2626: writing smithi16234619-17 from 2895372 to 3586719 tid 3 2024-09-16T15:12:33.125 INFO:tasks.rados.rados.0.smithi162.stdout:2627: delete oid 29 current snap is 259 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:2625: finishing rollback tid 1 to smithi16234619-48 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 0 (ObjNum 129 snap 0 seq_num 864421376) dirty dne 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:2623: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:2623: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:2623: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:12:33.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1091 (ObjNum 905 snap 249 seq_num 905) dirty exists 2024-09-16T15:12:33.128 INFO:tasks.rados.rados.0.smithi162.stdout:2623: done (4 left) 2024-09-16T15:12:33.129 INFO:tasks.rados.rados.0.smithi162.stdout:2624: done (3 left) 2024-09-16T15:12:33.129 INFO:tasks.rados.rados.0.smithi162.stdout:2625: done (2 left) 2024-09-16T15:12:33.129 INFO:tasks.rados.rados.0.smithi162.stdout:2627: done (1 left) 2024-09-16T15:12:33.129 INFO:tasks.rados.rados.0.smithi162.stdout:2628: delete oid 1 current snap is 259 2024-09-16T15:12:33.130 INFO:tasks.rados.rados.0.smithi162.stdout:2628: done (1 left) 2024-09-16T15:12:33.131 INFO:tasks.rados.rados.0.smithi162.stdout:2629: snap_create 2024-09-16T15:12:33.154 INFO:tasks.rados.rados.0.smithi162.stdout:2626: finishing write tid 1 to smithi16234619-17 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: finishing write tid 2 to smithi16234619-17 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: finishing write tid 3 to smithi16234619-17 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: finishing write tid 4 to smithi16234619-17 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: finishing write tid 5 to smithi16234619-17 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: oid 17 updating version 0 to 1278 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: oid 17 version 1278 is already newer than 1277 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: oid 17 updating version 1278 to 1279 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: oid 17 updating version 1279 to 1280 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1280 (ObjNum 929 snap 259 seq_num 929) dirty exists 2024-09-16T15:12:33.156 INFO:tasks.rados.rados.0.smithi162.stdout:2626: left oid 17 (ObjNum 929 snap 259 seq_num 929) 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2626: done (1 left) 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2629: done (0 left) 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2630: read oid 7 snap -1 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2630: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2631: read oid 30 snap -1 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2631: expect (ObjNum 914 snap 254 seq_num 914) 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2632: rollback oid 38 current snap is 260 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 252 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2633: copy_from oid 9 from oid 49 current snap is 260 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2634: write oid 46 current snap is 260 2024-09-16T15:12:34.023 INFO:tasks.rados.rados.0.smithi162.stdout:2634: seq_num 932 ranges {464124=727134,1771806=771987,2977641=641516} 2024-09-16T15:12:34.032 INFO:tasks.rados.rados.0.smithi162.stdout:2634: writing smithi16234619-46 from 464124 to 1191258 tid 1 2024-09-16T15:12:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2634: writing smithi16234619-46 from 1771806 to 2543793 tid 2 2024-09-16T15:12:34.047 INFO:tasks.rados.rados.0.smithi162.stdout:2634: writing smithi16234619-46 from 2977641 to 3619157 tid 3 2024-09-16T15:12:34.047 INFO:tasks.rados.rados.0.smithi162.stdout:2635: delete oid 39 current snap is 260 2024-09-16T15:12:34.047 INFO:tasks.rados.rados.0.smithi162.stdout:2632: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:12:34.048 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 963 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:12:34.048 INFO:tasks.rados.rados.0.smithi162.stdout:2631: expect (ObjNum 914 snap 254 seq_num 914) 2024-09-16T15:12:34.187 INFO:tasks.rados.rados.0.smithi162.stdout:2630: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:34.348 INFO:tasks.rados.rados.0.smithi162.stdout:2633: finishing copy_from racing read to smithi16234619-9 2024-09-16T15:12:34.348 INFO:tasks.rados.rados.0.smithi162.stdout:2633: finishing copy_from to smithi16234619-9 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: finishing write tid 1 to smithi16234619-46 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: finishing write tid 2 to smithi16234619-46 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: finishing write tid 3 to smithi16234619-46 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: finishing write tid 4 to smithi16234619-46 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: finishing write tid 5 to smithi16234619-46 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: oid 46 updating version 0 to 598 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: oid 46 updating version 598 to 599 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: oid 46 updating version 599 to 601 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: oid 46 version 601 is already newer than 600 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 601 (ObjNum 932 snap 260 seq_num 932) dirty exists 2024-09-16T15:12:34.349 INFO:tasks.rados.rados.0.smithi162.stdout:2634: left oid 46 (ObjNum 932 snap 260 seq_num 932) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2630: done (5 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2631: done (4 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2632: done (3 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2633: done (2 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2634: done (1 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2635: done (0 left) 2024-09-16T15:12:34.350 INFO:tasks.rados.rados.0.smithi162.stdout:2636: snap_create 2024-09-16T15:12:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2636: done (0 left) 2024-09-16T15:12:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2637: read oid 33 snap -1 2024-09-16T15:12:35.108 INFO:tasks.rados.rados.0.smithi162.stdout:2637: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:12:35.109 INFO:tasks.rados.rados.0.smithi162.stdout:2638: copy_from oid 24 from oid 36 current snap is 261 2024-09-16T15:12:35.109 INFO:tasks.rados.rados.0.smithi162.stdout:2639: read oid 39 snap -1 2024-09-16T15:12:35.109 INFO:tasks.rados.rados.0.smithi162.stdout:2639: expect deleted 2024-09-16T15:12:35.109 INFO:tasks.rados.rados.0.smithi162.stdout:2640: write oid 30 current snap is 261 2024-09-16T15:12:35.109 INFO:tasks.rados.rados.0.smithi162.stdout:2640: seq_num 934 ranges {497738=477632,1772417=775656,3316134=381604} 2024-09-16T15:12:35.114 INFO:tasks.rados.rados.0.smithi162.stdout:2640: writing smithi16234619-30 from 497738 to 975370 tid 1 2024-09-16T15:12:35.122 INFO:tasks.rados.rados.0.smithi162.stdout:2640: writing smithi16234619-30 from 1772417 to 2548073 tid 2 2024-09-16T15:12:35.129 INFO:tasks.rados.rados.0.smithi162.stdout:2640: writing smithi16234619-30 from 3316134 to 3697738 tid 3 2024-09-16T15:12:35.129 INFO:tasks.rados.rados.0.smithi162.stdout:2641: copy_from oid 4 from oid 12 current snap is 261 2024-09-16T15:12:35.129 INFO:tasks.rados.rados.0.smithi162.stdout:2637: expect (ObjNum 765 snap 198 seq_num 765) 2024-09-16T15:12:35.295 INFO:tasks.rados.rados.0.smithi162.stdout:2640: finishing write tid 1 to smithi16234619-30 2024-09-16T15:12:35.295 INFO:tasks.rados.rados.0.smithi162.stdout:2640: finishing write tid 2 to smithi16234619-30 2024-09-16T15:12:35.295 INFO:tasks.rados.rados.0.smithi162.stdout:2638: finishing copy_from racing read to smithi16234619-24 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: finishing write tid 3 to smithi16234619-30 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: finishing write tid 4 to smithi16234619-30 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: finishing write tid 5 to smithi16234619-30 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: oid 30 updating version 0 to 1145 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: oid 30 updating version 1145 to 1146 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: oid 30 version 1146 is already newer than 1144 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: oid 30 updating version 1146 to 1147 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1147 (ObjNum 934 snap 261 seq_num 934) dirty exists 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2640: left oid 30 (ObjNum 934 snap 261 seq_num 934) 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2638: finishing copy_from to smithi16234619-24 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 838 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2641: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:12:35.296 INFO:tasks.rados.rados.0.smithi162.stdout:2641: finishing copy_from to smithi16234619-4 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 892 (ObjNum 802 snap 216 seq_num 802) dirty exists 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2637: done (4 left) 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2638: done (3 left) 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2639: done (2 left) 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2640: done (1 left) 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2641: done (0 left) 2024-09-16T15:12:35.297 INFO:tasks.rados.rados.0.smithi162.stdout:2642: snap_remove snap 255 2024-09-16T15:12:36.116 INFO:tasks.rados.rados.0.smithi162.stdout:2642: done (0 left) 2024-09-16T15:12:36.116 INFO:tasks.rados.rados.0.smithi162.stdout:2643: copy_from oid 48 from oid 6 current snap is 261 2024-09-16T15:12:36.116 INFO:tasks.rados.rados.0.smithi162.stdout:2644: read oid 32 snap -1 2024-09-16T15:12:36.116 INFO:tasks.rados.rados.0.smithi162.stdout:2644: expect (ObjNum 872 snap 237 seq_num 872) 2024-09-16T15:12:36.117 INFO:tasks.rados.rados.0.smithi162.stdout:2645: read oid 3 snap -1 2024-09-16T15:12:36.117 INFO:tasks.rados.rados.0.smithi162.stdout:2645: expect (ObjNum 900 snap 247 seq_num 900) 2024-09-16T15:12:36.117 INFO:tasks.rados.rados.0.smithi162.stdout:2646: snap_remove snap 257 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2646: done (3 left) 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2647: read oid 41 snap 258 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2647: expect deleted 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2648: snap_create 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2643: finishing copy_from to smithi16234619-48 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2643: got expected ENOENT (src dne) 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2643: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:12:37.132 INFO:tasks.rados.rados.0.smithi162.stdout:2645: expect (ObjNum 900 snap 247 seq_num 900) 2024-09-16T15:12:37.236 INFO:tasks.rados.rados.0.smithi162.stdout:2644: expect (ObjNum 872 snap 237 seq_num 872) 2024-09-16T15:12:38.136 INFO:tasks.rados.rados.0.smithi162.stdout:2643: done (4 left) 2024-09-16T15:12:38.136 INFO:tasks.rados.rados.0.smithi162.stdout:2644: done (3 left) 2024-09-16T15:12:38.136 INFO:tasks.rados.rados.0.smithi162.stdout:2645: done (2 left) 2024-09-16T15:12:38.136 INFO:tasks.rados.rados.0.smithi162.stdout:2647: done (1 left) 2024-09-16T15:12:38.137 INFO:tasks.rados.rados.0.smithi162.stdout:2648: done (0 left) 2024-09-16T15:12:38.137 INFO:tasks.rados.rados.0.smithi162.stdout:2649: rollback oid 37 current snap is 262 2024-09-16T15:12:38.137 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 247 2024-09-16T15:12:38.137 INFO:tasks.rados.rados.0.smithi162.stdout:2650: delete oid 36 current snap is 262 2024-09-16T15:12:38.140 INFO:tasks.rados.rados.0.smithi162.stdout:2650: done (1 left) 2024-09-16T15:12:38.140 INFO:tasks.rados.rados.0.smithi162.stdout:2651: delete oid 12 current snap is 262 2024-09-16T15:12:38.140 INFO:tasks.rados.rados.0.smithi162.stdout:2649: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:12:38.141 INFO:tasks.rados.rados.0.smithi162.stdout:2649: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:12:38.141 INFO:tasks.rados.rados.0.smithi162.stdout:2649: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:12:38.141 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1263 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2649: done (1 left) 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2651: done (0 left) 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2652: copy_from oid 48 from oid 34 current snap is 262 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2653: rollback oid 11 current snap is 262 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 252 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2654: write oid 46 current snap is 262 2024-09-16T15:12:38.144 INFO:tasks.rados.rados.0.smithi162.stdout:2654: seq_num 937 ranges {548159=419732,1664216=522983,2714756=614229} 2024-09-16T15:12:38.149 INFO:tasks.rados.rados.0.smithi162.stdout:2654: writing smithi16234619-46 from 548159 to 967891 tid 1 2024-09-16T15:12:38.156 INFO:tasks.rados.rados.0.smithi162.stdout:2654: writing smithi16234619-46 from 1664216 to 2187199 tid 2 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:2654: writing smithi16234619-46 from 2714756 to 3328985 tid 3 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:2655: snap_create 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:2653: finishing rollback tid 0 to smithi16234619-11 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:2653: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:2653: finishing rollback tid 2 to smithi16234619-11 2024-09-16T15:12:38.163 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 600 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:12:38.181 INFO:tasks.rados.rados.0.smithi162.stdout:2652: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:12:38.192 INFO:tasks.rados.rados.0.smithi162.stdout:2652: finishing copy_from to smithi16234619-48 2024-09-16T15:12:38.192 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1055 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:12:38.203 INFO:tasks.rados.rados.0.smithi162.stdout:2654: finishing write tid 1 to smithi16234619-46 2024-09-16T15:12:38.203 INFO:tasks.rados.rados.0.smithi162.stdout:2654: finishing write tid 2 to smithi16234619-46 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: finishing write tid 3 to smithi16234619-46 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: finishing write tid 4 to smithi16234619-46 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: finishing write tid 5 to smithi16234619-46 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: oid 46 updating version 0 to 604 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: oid 46 version 604 is already newer than 602 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: oid 46 version 604 is already newer than 603 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: oid 46 updating version 604 to 605 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 605 (ObjNum 937 snap 262 seq_num 937) dirty exists 2024-09-16T15:12:38.205 INFO:tasks.rados.rados.0.smithi162.stdout:2654: left oid 46 (ObjNum 937 snap 262 seq_num 937) 2024-09-16T15:12:39.024 INFO:tasks.rados.rados.0.smithi162.stdout:2652: done (3 left) 2024-09-16T15:12:39.024 INFO:tasks.rados.rados.0.smithi162.stdout:2653: done (2 left) 2024-09-16T15:12:39.024 INFO:tasks.rados.rados.0.smithi162.stdout:2654: done (1 left) 2024-09-16T15:12:39.024 INFO:tasks.rados.rados.0.smithi162.stdout:2655: done (0 left) 2024-09-16T15:12:39.024 INFO:tasks.rados.rados.0.smithi162.stdout:2656: snap_create 2024-09-16T15:12:40.035 INFO:tasks.rados.rados.0.smithi162.stdout:2656: done (0 left) 2024-09-16T15:12:40.035 INFO:tasks.rados.rados.0.smithi162.stdout:2657: delete oid 26 current snap is 264 2024-09-16T15:12:40.039 INFO:tasks.rados.rados.0.smithi162.stdout:2657: done (0 left) 2024-09-16T15:12:40.039 INFO:tasks.rados.rados.0.smithi162.stdout:2658: snap_create 2024-09-16T15:12:41.042 INFO:tasks.rados.rados.0.smithi162.stdout:2658: done (0 left) 2024-09-16T15:12:41.042 INFO:tasks.rados.rados.0.smithi162.stdout:2659: snap_remove snap 248 2024-09-16T15:12:42.046 INFO:tasks.rados.rados.0.smithi162.stdout:2659: done (0 left) 2024-09-16T15:12:42.046 INFO:tasks.rados.rados.0.smithi162.stdout:2660: read oid 7 snap 261 2024-09-16T15:12:42.046 INFO:tasks.rados.rados.0.smithi162.stdout:2660: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:42.046 INFO:tasks.rados.rados.0.smithi162.stdout:2661: delete oid 2 current snap is 265 2024-09-16T15:12:42.055 INFO:tasks.rados.rados.0.smithi162.stdout:2660: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:42.218 INFO:tasks.rados.rados.0.smithi162.stdout:2660: done (1 left) 2024-09-16T15:12:42.218 INFO:tasks.rados.rados.0.smithi162.stdout:2661: done (0 left) 2024-09-16T15:12:42.218 INFO:tasks.rados.rados.0.smithi162.stdout:2662: snap_remove snap 262 2024-09-16T15:12:43.457 INFO:tasks.rados.rados.0.smithi162.stdout:2662: done (0 left) 2024-09-16T15:12:43.457 INFO:tasks.rados.rados.0.smithi162.stdout:2663: snap_create 2024-09-16T15:12:44.028 INFO:tasks.rados.rados.0.smithi162.stdout:2663: done (0 left) 2024-09-16T15:12:44.028 INFO:tasks.rados.rados.0.smithi162.stdout:2664: copy_from oid 11 from oid 37 current snap is 266 2024-09-16T15:12:44.028 INFO:tasks.rados.rados.0.smithi162.stdout:2665: write oid 4 current snap is 266 2024-09-16T15:12:44.028 INFO:tasks.rados.rados.0.smithi162.stdout:2665: seq_num 940 ranges {598580=645479,1723309=786663,3080672=448045} 2024-09-16T15:12:44.035 INFO:tasks.rados.rados.0.smithi162.stdout:2665: writing smithi16234619-4 from 598580 to 1244059 tid 1 2024-09-16T15:12:44.044 INFO:tasks.rados.rados.0.smithi162.stdout:2665: writing smithi16234619-4 from 1723309 to 2509972 tid 2 2024-09-16T15:12:44.050 INFO:tasks.rados.rados.0.smithi162.stdout:2665: writing smithi16234619-4 from 3080672 to 3528717 tid 3 2024-09-16T15:12:44.051 INFO:tasks.rados.rados.0.smithi162.stdout:2666: write (excl) oid 49 current snap is 266 2024-09-16T15:12:44.051 INFO:tasks.rados.rados.0.smithi162.stdout:2666: seq_num 941 ranges {615387=720728,1781791=530321,3108095=707292} 2024-09-16T15:12:44.059 INFO:tasks.rados.rados.0.smithi162.stdout:2666: writing smithi16234619-49 from 615387 to 1336115 tid 1 2024-09-16T15:12:44.065 INFO:tasks.rados.rados.0.smithi162.stdout:2666: writing smithi16234619-49 from 1781791 to 2312112 tid 2 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2666: writing smithi16234619-49 from 3108095 to 3815387 tid 3 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2667: copy_from oid 47 from oid 22 current snap is 266 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2668: read oid 21 snap -1 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2668: expect (ObjNum 832 snap 224 seq_num 832) 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2669: rollback oid 16 current snap is 266 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 258 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2670: read oid 36 snap -1 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2670: expect deleted 2024-09-16T15:12:44.074 INFO:tasks.rados.rados.0.smithi162.stdout:2671: copy_from oid 31 from oid 9 current snap is 266 2024-09-16T15:12:44.075 INFO:tasks.rados.rados.0.smithi162.stdout:2672: write (excl) oid 42 current snap is 266 2024-09-16T15:12:44.075 INFO:tasks.rados.rados.0.smithi162.stdout:2672: seq_num 942 ranges {632194=795977,2123920=790332,3419165=413029} 2024-09-16T15:12:44.081 INFO:tasks.rados.rados.0.smithi162.stdout:2672: writing smithi16234619-42 from 632194 to 1428171 tid 1 2024-09-16T15:12:44.089 INFO:tasks.rados.rados.0.smithi162.stdout:2672: writing smithi16234619-42 from 2123920 to 2914252 tid 2 2024-09-16T15:12:44.095 INFO:tasks.rados.rados.0.smithi162.stdout:2672: writing smithi16234619-42 from 3419165 to 3832194 tid 3 2024-09-16T15:12:44.095 INFO:tasks.rados.rados.0.smithi162.stdout:2673: write oid 14 current snap is 266 2024-09-16T15:12:44.095 INFO:tasks.rados.rados.0.smithi162.stdout:2673: seq_num 943 ranges {649001=587579,1898755=533990,3162941=686060} 2024-09-16T15:12:44.102 INFO:tasks.rados.rados.0.smithi162.stdout:2673: writing smithi16234619-14 from 649001 to 1236580 tid 1 2024-09-16T15:12:44.108 INFO:tasks.rados.rados.0.smithi162.stdout:2673: writing smithi16234619-14 from 1898755 to 2432745 tid 2 2024-09-16T15:12:44.116 INFO:tasks.rados.rados.0.smithi162.stdout:2673: writing smithi16234619-14 from 3162941 to 3849001 tid 3 2024-09-16T15:12:44.116 INFO:tasks.rados.rados.0.smithi162.stdout:2674: snap_create 2024-09-16T15:12:44.117 INFO:tasks.rados.rados.0.smithi162.stdout:2668: expect (ObjNum 832 snap 224 seq_num 832) 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:2669: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 974 (ObjNum 129 snap 0 seq_num 1838704592) dirty dne 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:2664: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:2664: finishing copy_from to smithi16234619-11 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1263 (ObjNum 721 snap 190 seq_num 721) dirty exists 2024-09-16T15:12:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:2667: finishing copy_from to smithi16234619-47 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2667: got expected ENOENT (src dne) 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2667: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: finishing write tid 1 to smithi16234619-4 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: finishing write tid 2 to smithi16234619-4 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: finishing write tid 3 to smithi16234619-4 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: finishing write tid 4 to smithi16234619-4 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: finishing write tid 5 to smithi16234619-4 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: oid 4 updating version 0 to 1149 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: oid 4 version 1149 is already newer than 1148 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: oid 4 version 1149 is already newer than 1147 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: oid 4 updating version 1149 to 1150 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1150 (ObjNum 940 snap 266 seq_num 940) dirty exists 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2665: left oid 4 (ObjNum 940 snap 266 seq_num 940) 2024-09-16T15:12:44.248 INFO:tasks.rados.rados.0.smithi162.stdout:2666: finishing write tid 1 to smithi16234619-49 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: finishing write tid 2 to smithi16234619-49 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: finishing write tid 3 to smithi16234619-49 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: finishing write tid 4 to smithi16234619-49 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: finishing write tid 5 to smithi16234619-49 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: oid 49 updating version 0 to 809 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: oid 49 updating version 809 to 811 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: oid 49 version 811 is already newer than 810 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: oid 49 updating version 811 to 812 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 812 (ObjNum 941 snap 266 seq_num 941) dirty exists 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2666: left oid 49 (ObjNum 941 snap 266 seq_num 941) 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2672: finishing write tid 1 to smithi16234619-42 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2672: finishing write tid 2 to smithi16234619-42 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2672: finishing write tid 3 to smithi16234619-42 2024-09-16T15:12:44.249 INFO:tasks.rados.rados.0.smithi162.stdout:2672: finishing write tid 4 to smithi16234619-42 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: finishing write tid 5 to smithi16234619-42 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: oid 42 updating version 0 to 1275 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: oid 42 version 1275 is already newer than 1273 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: oid 42 updating version 1275 to 1276 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: oid 42 version 1276 is already newer than 1274 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1276 (ObjNum 942 snap 266 seq_num 942) dirty exists 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2672: left oid 42 (ObjNum 942 snap 266 seq_num 942) 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2671: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2671: finishing copy_from to smithi16234619-31 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2673: finishing write tid 1 to smithi16234619-14 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2673: finishing write tid 2 to smithi16234619-14 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2673: finishing write tid 3 to smithi16234619-14 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2673: finishing write tid 4 to smithi16234619-14 2024-09-16T15:12:44.250 INFO:tasks.rados.rados.0.smithi162.stdout:2673: finishing write tid 5 to smithi16234619-14 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:2673: oid 14 updating version 0 to 1110 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:2673: oid 14 version 1110 is already newer than 1108 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:2673: oid 14 version 1110 is already newer than 1107 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:2673: oid 14 version 1110 is already newer than 1109 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1110 (ObjNum 943 snap 266 seq_num 943) dirty exists 2024-09-16T15:12:44.251 INFO:tasks.rados.rados.0.smithi162.stdout:2673: left oid 14 (ObjNum 943 snap 266 seq_num 943) 2024-09-16T15:12:45.044 INFO:tasks.rados.rados.0.smithi162.stdout:2664: done (10 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2665: done (9 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2666: done (8 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2667: done (7 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2668: done (6 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2669: done (5 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2670: done (4 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2671: done (3 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2672: done (2 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2673: done (1 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2674: done (0 left) 2024-09-16T15:12:45.052 INFO:tasks.rados.rados.0.smithi162.stdout:2675: rollback oid 10 current snap is 267 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 230 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2676: read oid 5 snap 252 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2676: expect (ObjNum 817 snap 221 seq_num 817) 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2677: read oid 26 snap -1 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2677: expect deleted 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2678: write (excl) oid 8 current snap is 267 2024-09-16T15:12:45.053 INFO:tasks.rados.rados.0.smithi162.stdout:2678: seq_num 944 ranges {665808=662828,1957237=677648,3074011=609486} 2024-09-16T15:12:45.054 INFO:tasks.rados.rados.0.smithi162.stdout:2678: writing smithi16234619-8 from 665808 to 1328636 tid 1 2024-09-16T15:12:45.059 INFO:tasks.rados.rados.0.smithi162.stdout:2678: writing smithi16234619-8 from 1957237 to 2634885 tid 2 2024-09-16T15:12:45.065 INFO:tasks.rados.rados.0.smithi162.stdout:2678: writing smithi16234619-8 from 3074011 to 3683497 tid 3 2024-09-16T15:12:45.065 INFO:tasks.rados.rados.0.smithi162.stdout:2679: rollback oid 17 current snap is 267 2024-09-16T15:12:45.065 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 266 2024-09-16T15:12:45.065 INFO:tasks.rados.rados.0.smithi162.stdout:2676: expect (ObjNum 817 snap 221 seq_num 817) 2024-09-16T15:12:45.184 INFO:tasks.rados.rados.0.smithi162.stdout:2675: finishing rollback tid 0 to smithi16234619-10 2024-09-16T15:12:45.184 INFO:tasks.rados.rados.0.smithi162.stdout:2675: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2675: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1314 (ObjNum 765 snap 198 seq_num 765) dirty exists 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2679: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2679: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2679: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1318 (ObjNum 929 snap 259 seq_num 929) dirty exists 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2678: finishing write tid 1 to smithi16234619-8 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2675: done (4 left) 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2676: done (3 left) 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2677: done (2 left) 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2679: done (1 left) 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2680: rollback oid 38 current snap is 267 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 263 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2678: finishing write tid 2 to smithi16234619-8 2024-09-16T15:12:45.185 INFO:tasks.rados.rados.0.smithi162.stdout:2678: finishing write tid 3 to smithi16234619-8 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: finishing write tid 4 to smithi16234619-8 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: finishing write tid 5 to smithi16234619-8 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: oid 8 updating version 0 to 912 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: oid 8 version 912 is already newer than 911 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: oid 8 version 912 is already newer than 910 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: oid 8 version 912 is already newer than 909 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 912 (ObjNum 944 snap 267 seq_num 944) dirty exists 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: left oid 8 (ObjNum 944 snap 267 seq_num 944) 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2678: done (1 left) 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2681: write (excl) oid 40 current snap is 267 2024-09-16T15:12:45.186 INFO:tasks.rados.rados.0.smithi162.stdout:2681: seq_num 945 ranges {682615=738077,2015719=537659,3217787=420758} 2024-09-16T15:12:45.192 INFO:tasks.rados.rados.0.smithi162.stdout:2681: writing smithi16234619-40 from 682615 to 1420692 tid 1 2024-09-16T15:12:45.197 INFO:tasks.rados.rados.0.smithi162.stdout:2681: writing smithi16234619-40 from 2015719 to 2553378 tid 2 2024-09-16T15:12:45.203 INFO:tasks.rados.rados.0.smithi162.stdout:2681: writing smithi16234619-40 from 3217787 to 3638545 tid 3 2024-09-16T15:12:45.204 INFO:tasks.rados.rados.0.smithi162.stdout:2682: write (excl) oid 7 current snap is 267 2024-09-16T15:12:45.204 INFO:tasks.rados.rados.0.smithi162.stdout:2680: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:12:45.204 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 963 (ObjNum 97 snap 0 seq_num 866437040) dirty dne 2024-09-16T15:12:45.204 INFO:tasks.rados.rados.0.smithi162.stdout:2682: seq_num 946 ranges {699422=413326,1557848=681317,3012504=748383} 2024-09-16T15:12:45.209 INFO:tasks.rados.rados.0.smithi162.stdout:2682: writing smithi16234619-7 from 699422 to 1112748 tid 1 2024-09-16T15:12:45.215 INFO:tasks.rados.rados.0.smithi162.stdout:2682: writing smithi16234619-7 from 1557848 to 2239165 tid 2 2024-09-16T15:12:45.223 INFO:tasks.rados.rados.0.smithi162.stdout:2682: writing smithi16234619-7 from 3012504 to 3760887 tid 3 2024-09-16T15:12:45.223 INFO:tasks.rados.rados.0.smithi162.stdout:2680: done (2 left) 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2683: read oid 43 snap -1 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2683: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2684: read oid 45 snap 251 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2684: expect (ObjNum 881 snap 245 seq_num 881) 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2685: write oid 5 current snap is 267 2024-09-16T15:12:45.224 INFO:tasks.rados.rados.0.smithi162.stdout:2685: seq_num 947 ranges {716229=488575,1616330=541328,2756280=559655} 2024-09-16T15:12:45.230 INFO:tasks.rados.rados.0.smithi162.stdout:2685: writing smithi16234619-5 from 716229 to 1204804 tid 1 2024-09-16T15:12:45.235 INFO:tasks.rados.rados.0.smithi162.stdout:2685: writing smithi16234619-5 from 1616330 to 2157658 tid 2 2024-09-16T15:12:45.242 INFO:tasks.rados.rados.0.smithi162.stdout:2685: writing smithi16234619-5 from 2756280 to 3315935 tid 3 2024-09-16T15:12:45.242 INFO:tasks.rados.rados.0.smithi162.stdout:2686: write oid 28 current snap is 267 2024-09-16T15:12:45.242 INFO:tasks.rados.rados.0.smithi162.stdout:2684: expect (ObjNum 881 snap 245 seq_num 881) 2024-09-16T15:12:45.400 INFO:tasks.rados.rados.0.smithi162.stdout:2683: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:45.525 INFO:tasks.rados.rados.0.smithi162.stdout:2681: finishing write tid 1 to smithi16234619-40 2024-09-16T15:12:45.525 INFO:tasks.rados.rados.0.smithi162.stdout:2685: finishing write tid 1 to smithi16234619-5 2024-09-16T15:12:45.525 INFO:tasks.rados.rados.0.smithi162.stdout:2685: finishing write tid 2 to smithi16234619-5 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: finishing write tid 3 to smithi16234619-5 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: finishing write tid 4 to smithi16234619-5 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: finishing write tid 5 to smithi16234619-5 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: oid 5 updating version 0 to 1280 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: oid 5 version 1280 is already newer than 1278 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: oid 5 version 1280 is already newer than 1277 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: oid 5 version 1280 is already newer than 1279 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1280 (ObjNum 947 snap 267 seq_num 947) dirty exists 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2685: left oid 5 (ObjNum 947 snap 267 seq_num 947) 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2681: finishing write tid 2 to smithi16234619-40 2024-09-16T15:12:45.526 INFO:tasks.rados.rados.0.smithi162.stdout:2686: seq_num 948 ranges {733036=563824,2074812=684986,3183703=749333} 2024-09-16T15:12:45.532 INFO:tasks.rados.rados.0.smithi162.stdout:2686: writing smithi16234619-28 from 733036 to 1296860 tid 1 2024-09-16T15:12:45.540 INFO:tasks.rados.rados.0.smithi162.stdout:2686: writing smithi16234619-28 from 2074812 to 2759798 tid 2 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2686: writing smithi16234619-28 from 3183703 to 3933036 tid 3 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2683: done (5 left) 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2684: done (4 left) 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2685: done (3 left) 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2687: delete oid 33 current snap is 267 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2681: finishing write tid 3 to smithi16234619-40 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2682: finishing write tid 1 to smithi16234619-7 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2681: finishing write tid 4 to smithi16234619-40 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2682: finishing write tid 2 to smithi16234619-7 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2682: finishing write tid 3 to smithi16234619-7 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2682: finishing write tid 4 to smithi16234619-7 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2681: finishing write tid 5 to smithi16234619-40 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2681: oid 40 updating version 0 to 903 2024-09-16T15:12:45.547 INFO:tasks.rados.rados.0.smithi162.stdout:2681: oid 40 version 903 is already newer than 902 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2681: oid 40 updating version 903 to 904 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2681: oid 40 updating version 904 to 905 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 905 (ObjNum 945 snap 267 seq_num 945) dirty exists 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2681: left oid 40 (ObjNum 945 snap 267 seq_num 945) 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: finishing write tid 5 to smithi16234619-7 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: oid 7 updating version 0 to 914 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: oid 7 updating version 914 to 915 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: oid 7 version 915 is already newer than 913 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: oid 7 updating version 915 to 916 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 916 (ObjNum 946 snap 267 seq_num 946) dirty exists 2024-09-16T15:12:45.548 INFO:tasks.rados.rados.0.smithi162.stdout:2682: left oid 7 (ObjNum 946 snap 267 seq_num 946) 2024-09-16T15:12:45.549 INFO:tasks.rados.rados.0.smithi162.stdout:2681: done (3 left) 2024-09-16T15:12:45.549 INFO:tasks.rados.rados.0.smithi162.stdout:2682: done (2 left) 2024-09-16T15:12:45.549 INFO:tasks.rados.rados.0.smithi162.stdout:2687: done (1 left) 2024-09-16T15:12:45.549 INFO:tasks.rados.rados.0.smithi162.stdout:2688: snap_create 2024-09-16T15:12:45.587 INFO:tasks.rados.rados.0.smithi162.stdout:2686: finishing write tid 1 to smithi16234619-28 2024-09-16T15:12:45.588 INFO:tasks.rados.rados.0.smithi162.stdout:2686: finishing write tid 2 to smithi16234619-28 2024-09-16T15:12:45.588 INFO:tasks.rados.rados.0.smithi162.stdout:2686: finishing write tid 3 to smithi16234619-28 2024-09-16T15:12:45.588 INFO:tasks.rados.rados.0.smithi162.stdout:2686: finishing write tid 4 to smithi16234619-28 2024-09-16T15:12:45.588 INFO:tasks.rados.rados.0.smithi162.stdout:2686: finishing write tid 5 to smithi16234619-28 2024-09-16T15:12:45.588 INFO:tasks.rados.rados.0.smithi162.stdout:2686: oid 28 updating version 0 to 1151 2024-09-16T15:12:45.589 INFO:tasks.rados.rados.0.smithi162.stdout:2686: oid 28 version 1151 is already newer than 1148 2024-09-16T15:12:45.589 INFO:tasks.rados.rados.0.smithi162.stdout:2686: oid 28 version 1151 is already newer than 1149 2024-09-16T15:12:45.589 INFO:tasks.rados.rados.0.smithi162.stdout:2686: oid 28 version 1151 is already newer than 1150 2024-09-16T15:12:45.589 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1151 (ObjNum 948 snap 267 seq_num 948) dirty exists 2024-09-16T15:12:45.589 INFO:tasks.rados.rados.0.smithi162.stdout:2686: left oid 28 (ObjNum 948 snap 267 seq_num 948) 2024-09-16T15:12:46.660 INFO:tasks.rados.rados.0.smithi162.stdout:2686: done (1 left) 2024-09-16T15:12:46.660 INFO:tasks.rados.rados.0.smithi162.stdout:2688: done (0 left) 2024-09-16T15:12:46.660 INFO:tasks.rados.rados.0.smithi162.stdout:2689: snap_remove snap 260 2024-09-16T15:12:47.660 INFO:tasks.rados.rados.0.smithi162.stdout:2689: done (0 left) 2024-09-16T15:12:47.660 INFO:tasks.rados.rados.0.smithi162.stdout:2690: copy_from oid 11 from oid 32 current snap is 268 2024-09-16T15:12:47.660 INFO:tasks.rados.rados.0.smithi162.stdout:2691: rollback oid 4 current snap is 268 2024-09-16T15:12:47.660 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 238 2024-09-16T15:12:47.661 INFO:tasks.rados.rados.0.smithi162.stdout:2692: snap_remove snap 267 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2692: done (2 left) 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2693: copy_from oid 1 from oid 44 current snap is 268 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2691: finishing rollback tid 0 to smithi16234619-4 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2691: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2691: finishing rollback tid 2 to smithi16234619-4 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1153 (ObjNum 829 snap 224 seq_num 829) dirty exists 2024-09-16T15:12:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:2690: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:2690: finishing copy_from to smithi16234619-11 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 898 (ObjNum 872 snap 237 seq_num 872) dirty exists 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:2690: done (2 left) 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:2691: done (1 left) 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:2694: write oid 3 current snap is 268 2024-09-16T15:12:49.059 INFO:tasks.rados.rados.0.smithi162.stdout:2694: seq_num 950 ranges {766650=714322,2075423=688655,3522196=444454} 2024-09-16T15:12:49.066 INFO:tasks.rados.rados.0.smithi162.stdout:2694: writing smithi16234619-3 from 766650 to 1480972 tid 1 2024-09-16T15:12:49.074 INFO:tasks.rados.rados.0.smithi162.stdout:2694: writing smithi16234619-3 from 2075423 to 2764078 tid 2 2024-09-16T15:12:49.081 INFO:tasks.rados.rados.0.smithi162.stdout:2694: writing smithi16234619-3 from 3522196 to 3966650 tid 3 2024-09-16T15:12:49.081 INFO:tasks.rados.rados.0.smithi162.stdout:2695: rollback oid 21 current snap is 268 2024-09-16T15:12:49.081 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 251 2024-09-16T15:12:49.081 INFO:tasks.rados.rados.0.smithi162.stdout:2693: finishing copy_from to smithi16234619-1 2024-09-16T15:12:49.082 INFO:tasks.rados.rados.0.smithi162.stdout:2693: got expected ENOENT (src dne) 2024-09-16T15:12:49.082 INFO:tasks.rados.rados.0.smithi162.stdout:2693: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:12:49.082 INFO:tasks.rados.rados.0.smithi162.stdout:2693: done (2 left) 2024-09-16T15:12:49.082 INFO:tasks.rados.rados.0.smithi162.stdout:2696: snap_remove snap 250 2024-09-16T15:12:50.112 INFO:tasks.rados.rados.0.smithi162.stdout:2696: done (2 left) 2024-09-16T15:12:50.112 INFO:tasks.rados.rados.0.smithi162.stdout:2697: write oid 39 current snap is 268 2024-09-16T15:12:50.112 INFO:tasks.rados.rados.0.smithi162.stdout:2697: seq_num 951 ranges {783457=505924,1850258=548666,2865972=721096} 2024-09-16T15:12:50.119 INFO:tasks.rados.rados.0.smithi162.stdout:2697: writing smithi16234619-39 from 783457 to 1289381 tid 1 2024-09-16T15:12:50.125 INFO:tasks.rados.rados.0.smithi162.stdout:2697: writing smithi16234619-39 from 1850258 to 2398924 tid 2 2024-09-16T15:12:50.133 INFO:tasks.rados.rados.0.smithi162.stdout:2697: writing smithi16234619-39 from 2865972 to 3587068 tid 3 2024-09-16T15:12:50.133 INFO:tasks.rados.rados.0.smithi162.stdout:2698: rollback oid 47 current snap is 268 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 251 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2695: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2695: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2695: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1332 (ObjNum 832 snap 224 seq_num 832) dirty exists 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: finishing write tid 1 to smithi16234619-3 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: finishing write tid 2 to smithi16234619-3 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: finishing write tid 3 to smithi16234619-3 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: finishing write tid 4 to smithi16234619-3 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: finishing write tid 5 to smithi16234619-3 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: oid 3 updating version 0 to 130 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: oid 3 version 130 is already newer than 129 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: oid 3 version 130 is already newer than 128 2024-09-16T15:12:50.134 INFO:tasks.rados.rados.0.smithi162.stdout:2694: oid 3 updating version 130 to 131 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 131 (ObjNum 950 snap 268 seq_num 950) dirty exists 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2694: left oid 3 (ObjNum 950 snap 268 seq_num 950) 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2694: done (3 left) 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2695: done (2 left) 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2699: rollback oid 7 current snap is 268 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 252 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2700: rollback oid 19 current snap is 268 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 259 2024-09-16T15:12:50.135 INFO:tasks.rados.rados.0.smithi162.stdout:2701: snap_remove snap 238 2024-09-16T15:12:51.124 INFO:tasks.rados.rados.0.smithi162.stdout:2701: done (4 left) 2024-09-16T15:12:51.124 INFO:tasks.rados.rados.0.smithi162.stdout:2699: finishing rollback tid 0 to smithi16234619-7 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2700: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2699: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2700: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2700: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1128 (ObjNum 905 snap 249 seq_num 905) dirty exists 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2699: finishing rollback tid 2 to smithi16234619-7 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 919 (ObjNum 851 snap 227 seq_num 851) dirty exists 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2698: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2698: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2698: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1122 (ObjNum 902 snap 249 seq_num 902) dirty exists 2024-09-16T15:12:51.125 INFO:tasks.rados.rados.0.smithi162.stdout:2697: finishing write tid 1 to smithi16234619-39 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: finishing write tid 2 to smithi16234619-39 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: finishing write tid 3 to smithi16234619-39 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: finishing write tid 4 to smithi16234619-39 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: finishing write tid 5 to smithi16234619-39 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: oid 39 updating version 0 to 1291 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: oid 39 version 1291 is already newer than 1289 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: oid 39 version 1291 is already newer than 1290 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: oid 39 updating version 1291 to 1292 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1292 (ObjNum 951 snap 268 seq_num 951) dirty exists 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2697: left oid 39 (ObjNum 951 snap 268 seq_num 951) 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2702: read oid 12 snap -1 2024-09-16T15:12:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:2702: expect deleted 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2697: done (4 left) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2698: done (3 left) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2699: done (2 left) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2700: done (1 left) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2703: rollback oid 19 current snap is 268 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 251 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2704: read oid 49 snap 252 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2704: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2705: read oid 26 snap 230 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2705: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2706: rollback oid 24 current snap is 268 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 261 2024-09-16T15:12:51.127 INFO:tasks.rados.rados.0.smithi162.stdout:2707: copy_from oid 35 from oid 45 current snap is 268 2024-09-16T15:12:51.128 INFO:tasks.rados.rados.0.smithi162.stdout:2708: write oid 30 current snap is 268 2024-09-16T15:12:51.128 INFO:tasks.rados.rados.0.smithi162.stdout:2708: seq_num 952 ranges {400264=581173,1508740=491524} 2024-09-16T15:12:51.129 INFO:tasks.rados.rados.0.smithi162.stdout:2708: writing smithi16234619-30 from 400264 to 981437 tid 1 2024-09-16T15:12:51.136 INFO:tasks.rados.rados.0.smithi162.stdout:2708: writing smithi16234619-30 from 1508740 to 2000264 tid 2 2024-09-16T15:12:51.136 INFO:tasks.rados.rados.0.smithi162.stdout:2709: write oid 20 current snap is 268 2024-09-16T15:12:51.136 INFO:tasks.rados.rados.0.smithi162.stdout:2709: seq_num 953 ranges {417071=656422,1567222=449849} 2024-09-16T15:12:51.143 INFO:tasks.rados.rados.0.smithi162.stdout:2709: writing smithi16234619-20 from 417071 to 1073493 tid 1 2024-09-16T15:12:51.149 INFO:tasks.rados.rados.0.smithi162.stdout:2709: writing smithi16234619-20 from 1567222 to 2017071 tid 2 2024-09-16T15:12:51.149 INFO:tasks.rados.rados.0.smithi162.stdout:2710: write (excl) oid 11 current snap is 268 2024-09-16T15:12:51.149 INFO:tasks.rados.rados.0.smithi162.stdout:2710: seq_num 954 ranges {433878=731671,1909351=124527} 2024-09-16T15:12:51.155 INFO:tasks.rados.rados.0.smithi162.stdout:2710: writing smithi16234619-11 from 433878 to 1165549 tid 1 2024-09-16T15:12:51.161 INFO:tasks.rados.rados.0.smithi162.stdout:2710: writing smithi16234619-11 from 1909351 to 2033878 tid 2 2024-09-16T15:12:51.161 INFO:tasks.rados.rados.0.smithi162.stdout:2711: delete oid 10 current snap is 268 2024-09-16T15:12:51.161 INFO:tasks.rados.rados.0.smithi162.stdout:2704: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:51.296 INFO:tasks.rados.rados.0.smithi162.stdout:2703: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:12:51.296 INFO:tasks.rados.rados.0.smithi162.stdout:2705: expect (ObjNum 713 snap 185 seq_num 713) 2024-09-16T15:12:51.548 INFO:tasks.rados.rados.0.smithi162.stdout:2703: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2703: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1135 (ObjNum 905 snap 249 seq_num 905) dirty exists 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2706: finishing rollback tid 0 to smithi16234619-24 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2706: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2706: finishing rollback tid 2 to smithi16234619-24 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1348 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: finishing write tid 1 to smithi16234619-30 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: finishing write tid 2 to smithi16234619-30 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: finishing write tid 3 to smithi16234619-30 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: finishing write tid 4 to smithi16234619-30 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: oid 30 updating version 0 to 1153 2024-09-16T15:12:51.549 INFO:tasks.rados.rados.0.smithi162.stdout:2708: oid 30 updating version 1153 to 1154 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2708: oid 30 version 1154 is already newer than 1152 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1154 (ObjNum 952 snap 268 seq_num 952) dirty exists 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2708: left oid 30 (ObjNum 952 snap 268 seq_num 952) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2709: finishing write tid 1 to smithi16234619-20 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2709: finishing write tid 2 to smithi16234619-20 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2702: done (9 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2703: done (8 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2704: done (7 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2705: done (6 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2706: done (5 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2708: done (4 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2711: done (3 left) 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2712: delete oid 38 current snap is 268 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2709: finishing write tid 3 to smithi16234619-20 2024-09-16T15:12:51.550 INFO:tasks.rados.rados.0.smithi162.stdout:2709: finishing write tid 4 to smithi16234619-20 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2709: oid 20 updating version 0 to 822 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2709: oid 20 version 822 is already newer than 820 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2709: oid 20 version 822 is already newer than 821 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 822 (ObjNum 953 snap 268 seq_num 953) dirty exists 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2709: left oid 20 (ObjNum 953 snap 268 seq_num 953) 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: finishing write tid 1 to smithi16234619-11 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: finishing write tid 2 to smithi16234619-11 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: finishing write tid 3 to smithi16234619-11 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: finishing write tid 4 to smithi16234619-11 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: oid 11 updating version 0 to 1266 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: oid 11 version 1266 is already newer than 1265 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: oid 11 version 1266 is already newer than 1264 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1266 (ObjNum 954 snap 268 seq_num 954) dirty exists 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2710: left oid 11 (ObjNum 954 snap 268 seq_num 954) 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2707: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:12:51.551 INFO:tasks.rados.rados.0.smithi162.stdout:2707: finishing copy_from to smithi16234619-35 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1264 (ObjNum 917 snap 254 seq_num 917) dirty exists 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2707: done (3 left) 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2709: done (2 left) 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2710: done (1 left) 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2712: done (0 left) 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2713: write oid 8 current snap is 268 2024-09-16T15:12:51.552 INFO:tasks.rados.rados.0.smithi162.stdout:2713: seq_num 957 ranges {484299=557418,1684797=399502} 2024-09-16T15:12:51.555 INFO:tasks.rados.rados.0.smithi162.stdout:2713: writing smithi16234619-8 from 484299 to 1041717 tid 1 2024-09-16T15:12:51.561 INFO:tasks.rados.rados.0.smithi162.stdout:2713: writing smithi16234619-8 from 1684797 to 2084299 tid 2 2024-09-16T15:12:51.561 INFO:tasks.rados.rados.0.smithi162.stdout:2714: write (excl) oid 25 current snap is 268 2024-09-16T15:12:51.561 INFO:tasks.rados.rados.0.smithi162.stdout:2714: seq_num 958 ranges {501106=749020,1743279=357827} 2024-09-16T15:12:51.568 INFO:tasks.rados.rados.0.smithi162.stdout:2714: writing smithi16234619-25 from 501106 to 1250126 tid 1 2024-09-16T15:12:51.572 INFO:tasks.rados.rados.0.smithi162.stdout:2714: writing smithi16234619-25 from 1743279 to 2101106 tid 2 2024-09-16T15:12:51.573 INFO:tasks.rados.rados.0.smithi162.stdout:2715: snap_create 2024-09-16T15:12:51.582 INFO:tasks.rados.rados.0.smithi162.stdout:2713: finishing write tid 1 to smithi16234619-8 2024-09-16T15:12:51.589 INFO:tasks.rados.rados.0.smithi162.stdout:2713: finishing write tid 2 to smithi16234619-8 2024-09-16T15:12:51.591 INFO:tasks.rados.rados.0.smithi162.stdout:2713: finishing write tid 3 to smithi16234619-8 2024-09-16T15:12:51.591 INFO:tasks.rados.rados.0.smithi162.stdout:2713: finishing write tid 4 to smithi16234619-8 2024-09-16T15:12:51.591 INFO:tasks.rados.rados.0.smithi162.stdout:2713: oid 8 updating version 0 to 921 2024-09-16T15:12:51.591 INFO:tasks.rados.rados.0.smithi162.stdout:2713: oid 8 version 921 is already newer than 920 2024-09-16T15:12:51.592 INFO:tasks.rados.rados.0.smithi162.stdout:2713: oid 8 updating version 921 to 922 2024-09-16T15:12:51.592 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 922 (ObjNum 957 snap 268 seq_num 957) dirty exists 2024-09-16T15:12:51.592 INFO:tasks.rados.rados.0.smithi162.stdout:2713: left oid 8 (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:12:51.609 INFO:tasks.rados.rados.0.smithi162.stdout:2714: finishing write tid 1 to smithi16234619-25 2024-09-16T15:12:51.609 INFO:tasks.rados.rados.0.smithi162.stdout:2714: finishing write tid 2 to smithi16234619-25 2024-09-16T15:12:51.611 INFO:tasks.rados.rados.0.smithi162.stdout:2714: finishing write tid 3 to smithi16234619-25 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:2714: finishing write tid 4 to smithi16234619-25 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:2714: oid 25 updating version 0 to 1145 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:2714: oid 25 version 1145 is already newer than 1143 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:2714: oid 25 version 1145 is already newer than 1144 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 1145 (ObjNum 958 snap 268 seq_num 958) dirty exists 2024-09-16T15:12:51.612 INFO:tasks.rados.rados.0.smithi162.stdout:2714: left oid 25 (ObjNum 958 snap 268 seq_num 958) 2024-09-16T15:12:52.133 INFO:tasks.rados.rados.0.smithi162.stdout:2713: done (2 left) 2024-09-16T15:12:52.133 INFO:tasks.rados.rados.0.smithi162.stdout:2714: done (1 left) 2024-09-16T15:12:52.133 INFO:tasks.rados.rados.0.smithi162.stdout:2715: done (0 left) 2024-09-16T15:12:52.133 INFO:tasks.rados.rados.0.smithi162.stdout:2716: write (excl) oid 14 current snap is 269 2024-09-16T15:12:52.133 INFO:tasks.rados.rados.0.smithi162.stdout:2716: seq_num 959 ranges {517913=424269,1401761=446989} 2024-09-16T15:12:52.138 INFO:tasks.rados.rados.0.smithi162.stdout:2716: writing smithi16234619-14 from 517913 to 942182 tid 1 2024-09-16T15:12:52.143 INFO:tasks.rados.rados.0.smithi162.stdout:2716: writing smithi16234619-14 from 1401761 to 1848750 tid 2 2024-09-16T15:12:52.143 INFO:tasks.rados.rados.0.smithi162.stdout:2717: snap_remove snap 253 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2717: done (1 left) 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2718: read oid 50 snap -1 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2718: expect (ObjNum 811 snap 218 seq_num 811) 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2719: read oid 26 snap -1 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2719: expect deleted 2024-09-16T15:12:53.143 INFO:tasks.rados.rados.0.smithi162.stdout:2720: write (excl) oid 28 current snap is 269 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: finishing write tid 1 to smithi16234619-14 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: finishing write tid 2 to smithi16234619-14 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: finishing write tid 3 to smithi16234619-14 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: finishing write tid 4 to smithi16234619-14 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: oid 14 updating version 0 to 1148 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: oid 14 updating version 1148 to 1149 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: oid 14 version 1149 is already newer than 1147 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1149 (ObjNum 959 snap 269 seq_num 959) dirty exists 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2716: left oid 14 (ObjNum 959 snap 269 seq_num 959) 2024-09-16T15:12:53.144 INFO:tasks.rados.rados.0.smithi162.stdout:2720: seq_num 960 ranges {534720=499518,1460243=674477} 2024-09-16T15:12:53.148 INFO:tasks.rados.rados.0.smithi162.stdout:2720: writing smithi16234619-28 from 534720 to 1034238 tid 1 2024-09-16T15:12:53.155 INFO:tasks.rados.rados.0.smithi162.stdout:2720: writing smithi16234619-28 from 1460243 to 2134720 tid 2 2024-09-16T15:12:53.155 INFO:tasks.rados.rados.0.smithi162.stdout:2716: done (3 left) 2024-09-16T15:12:53.155 INFO:tasks.rados.rados.0.smithi162.stdout:2721: copy_from oid 48 from oid 35 current snap is 269 2024-09-16T15:12:53.155 INFO:tasks.rados.rados.0.smithi162.stdout:2719: done (3 left) 2024-09-16T15:12:53.156 INFO:tasks.rados.rados.0.smithi162.stdout:2722: read oid 14 snap 268 2024-09-16T15:12:53.156 INFO:tasks.rados.rados.0.smithi162.stdout:2722: expect (ObjNum 943 snap 266 seq_num 943) 2024-09-16T15:12:53.156 INFO:tasks.rados.rados.0.smithi162.stdout:2723: write oid 37 current snap is 269 2024-09-16T15:12:53.156 INFO:tasks.rados.rados.0.smithi162.stdout:2723: seq_num 961 ranges {551527=574767,1918725=232802} 2024-09-16T15:12:53.162 INFO:tasks.rados.rados.0.smithi162.stdout:2723: writing smithi16234619-37 from 551527 to 1126294 tid 1 2024-09-16T15:12:53.168 INFO:tasks.rados.rados.0.smithi162.stdout:2723: writing smithi16234619-37 from 1918725 to 2151527 tid 2 2024-09-16T15:12:53.168 INFO:tasks.rados.rados.0.smithi162.stdout:2724: snap_remove snap 263 2024-09-16T15:12:54.029 INFO:tasks.rados.rados.0.smithi162.stdout:2724: done (5 left) 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2725: read oid 34 snap -1 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2725: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2726: read oid 20 snap -1 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2726: expect (ObjNum 953 snap 268 seq_num 953) 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2727: copy_from oid 33 from oid 2 current snap is 269 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2728: read oid 39 snap -1 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2728: expect (ObjNum 951 snap 268 seq_num 951) 2024-09-16T15:12:54.030 INFO:tasks.rados.rados.0.smithi162.stdout:2729: snap_remove snap 264 2024-09-16T15:12:55.041 INFO:tasks.rados.rados.0.smithi162.stdout:2729: done (9 left) 2024-09-16T15:12:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:2730: write oid 44 current snap is 269 2024-09-16T15:12:55.042 INFO:tasks.rados.rados.0.smithi162.stdout:2730: seq_num 962 ranges {568334=650016,1860854=307480} 2024-09-16T15:12:55.048 INFO:tasks.rados.rados.0.smithi162.stdout:2730: writing smithi16234619-44 from 568334 to 1218350 tid 1 2024-09-16T15:12:55.054 INFO:tasks.rados.rados.0.smithi162.stdout:2730: writing smithi16234619-44 from 1860854 to 2168334 tid 2 2024-09-16T15:12:55.054 INFO:tasks.rados.rados.0.smithi162.stdout:2731: snap_remove snap 247 2024-09-16T15:12:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:2731: done (10 left) 2024-09-16T15:12:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:2732: read oid 46 snap 258 2024-09-16T15:12:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:2732: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:12:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:2733: copy_from oid 25 from oid 15 current snap is 269 2024-09-16T15:12:56.097 INFO:tasks.rados.rados.0.smithi162.stdout:2718: expect (ObjNum 811 snap 218 seq_num 811) 2024-09-16T15:12:56.365 INFO:tasks.rados.rados.0.smithi162.stdout:2722: expect (ObjNum 943 snap 266 seq_num 943) 2024-09-16T15:12:56.523 INFO:tasks.rados.rados.0.smithi162.stdout:2723: finishing write tid 1 to smithi16234619-37 2024-09-16T15:12:56.523 INFO:tasks.rados.rados.0.smithi162.stdout:2723: finishing write tid 2 to smithi16234619-37 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: finishing write tid 3 to smithi16234619-37 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: finishing write tid 4 to smithi16234619-37 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: oid 37 updating version 0 to 1313 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: oid 37 version 1313 is already newer than 1312 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: oid 37 updating version 1313 to 1314 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1314 (ObjNum 961 snap 269 seq_num 961) dirty exists 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: left oid 37 (ObjNum 961 snap 269 seq_num 961) 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2720: finishing write tid 1 to smithi16234619-28 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2718: done (11 left) 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2722: done (10 left) 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2723: done (9 left) 2024-09-16T15:12:56.524 INFO:tasks.rados.rados.0.smithi162.stdout:2734: snap_remove snap 265 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2734: done (9 left) 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2735: rollback oid 13 current snap is 269 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 13 to 261 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: finishing write tid 2 to smithi16234619-28 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: finishing write tid 3 to smithi16234619-28 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: finishing write tid 4 to smithi16234619-28 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: oid 28 updating version 0 to 1156 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: oid 28 version 1156 is already newer than 1155 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: oid 28 updating version 1156 to 1157 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1157 (ObjNum 960 snap 269 seq_num 960) dirty exists 2024-09-16T15:12:57.103 INFO:tasks.rados.rados.0.smithi162.stdout:2720: left oid 28 (ObjNum 960 snap 269 seq_num 960) 2024-09-16T15:12:57.104 INFO:tasks.rados.rados.0.smithi162.stdout:2721: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:12:57.104 INFO:tasks.rados.rados.0.smithi162.stdout:2721: finishing copy_from to smithi16234619-48 2024-09-16T15:12:57.104 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1264 (ObjNum 917 snap 254 seq_num 917) dirty exists 2024-09-16T15:12:57.104 INFO:tasks.rados.rados.0.smithi162.stdout:2725: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:12:57.210 INFO:tasks.rados.rados.0.smithi162.stdout:2726: expect (ObjNum 953 snap 268 seq_num 953) 2024-09-16T15:12:57.289 INFO:tasks.rados.rados.0.smithi162.stdout:2727: finishing copy_from to smithi16234619-33 2024-09-16T15:12:57.289 INFO:tasks.rados.rados.0.smithi162.stdout:2727: got expected ENOENT (src dne) 2024-09-16T15:12:57.289 INFO:tasks.rados.rados.0.smithi162.stdout:2727: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:12:57.289 INFO:tasks.rados.rados.0.smithi162.stdout:2728: expect (ObjNum 951 snap 268 seq_num 951) 2024-09-16T15:12:57.417 INFO:tasks.rados.rados.0.smithi162.stdout:2730: finishing write tid 1 to smithi16234619-44 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: finishing write tid 2 to smithi16234619-44 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: finishing write tid 3 to smithi16234619-44 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: finishing write tid 4 to smithi16234619-44 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: oid 44 updating version 0 to 1266 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: oid 44 updating version 1266 to 1267 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: oid 44 version 1267 is already newer than 1265 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1267 (ObjNum 962 snap 269 seq_num 962) dirty exists 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2730: left oid 44 (ObjNum 962 snap 269 seq_num 962) 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2720: done (9 left) 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2721: done (8 left) 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2725: done (7 left) 2024-09-16T15:12:57.418 INFO:tasks.rados.rados.0.smithi162.stdout:2726: done (6 left) 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2727: done (5 left) 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2728: done (4 left) 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2730: done (3 left) 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2736: read oid 7 snap -1 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2736: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2737: read oid 12 snap -1 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2737: expect deleted 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2738: snap_create 2024-09-16T15:12:57.419 INFO:tasks.rados.rados.0.smithi162.stdout:2732: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:12:57.525 INFO:tasks.rados.rados.0.smithi162.stdout:2733: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:12:57.525 INFO:tasks.rados.rados.0.smithi162.stdout:2733: finishing copy_from to smithi16234619-25 2024-09-16T15:12:57.525 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 847 (ObjNum 739 snap 194 seq_num 739) dirty exists 2024-09-16T15:12:57.525 INFO:tasks.rados.rados.0.smithi162.stdout:2735: finishing rollback tid 1 to smithi16234619-13 2024-09-16T15:12:57.525 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:12:57.526 INFO:tasks.rados.rados.0.smithi162.stdout:2736: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2732: done (5 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2733: done (4 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2735: done (3 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2736: done (2 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2737: done (1 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2738: done (0 left) 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2739: copy_from oid 32 from oid 23 current snap is 270 2024-09-16T15:12:58.103 INFO:tasks.rados.rados.0.smithi162.stdout:2740: snap_create 2024-09-16T15:12:58.104 INFO:tasks.rados.rados.0.smithi162.stdout:2739: finishing copy_from to smithi16234619-32 2024-09-16T15:12:58.104 INFO:tasks.rados.rados.0.smithi162.stdout:2739: got expected ENOENT (src dne) 2024-09-16T15:12:58.104 INFO:tasks.rados.rados.0.smithi162.stdout:2739: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:12:59.030 INFO:tasks.rados.rados.0.smithi162.stdout:2739: done (1 left) 2024-09-16T15:12:59.030 INFO:tasks.rados.rados.0.smithi162.stdout:2740: done (0 left) 2024-09-16T15:12:59.030 INFO:tasks.rados.rados.0.smithi162.stdout:2741: read oid 43 snap -1 2024-09-16T15:12:59.030 INFO:tasks.rados.rados.0.smithi162.stdout:2741: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:59.030 INFO:tasks.rados.rados.0.smithi162.stdout:2742: delete oid 6 current snap is 271 2024-09-16T15:12:59.042 INFO:tasks.rados.rados.0.smithi162.stdout:2741: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2741: done (1 left) 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2742: done (0 left) 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2743: read oid 2 snap 269 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2743: expect deleted 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2744: write (excl) oid 28 current snap is 271 2024-09-16T15:12:59.169 INFO:tasks.rados.rados.0.smithi162.stdout:2744: seq_num 964 ranges {601948=400514,1577818=624130} 2024-09-16T15:12:59.174 INFO:tasks.rados.rados.0.smithi162.stdout:2744: writing smithi16234619-28 from 601948 to 1002462 tid 1 2024-09-16T15:12:59.180 INFO:tasks.rados.rados.0.smithi162.stdout:2744: writing smithi16234619-28 from 1577818 to 2201948 tid 2 2024-09-16T15:12:59.181 INFO:tasks.rados.rados.0.smithi162.stdout:2745: snap_create 2024-09-16T15:12:59.206 INFO:tasks.rados.rados.0.smithi162.stdout:2744: finishing write tid 1 to smithi16234619-28 2024-09-16T15:12:59.207 INFO:tasks.rados.rados.0.smithi162.stdout:2744: finishing write tid 2 to smithi16234619-28 2024-09-16T15:12:59.207 INFO:tasks.rados.rados.0.smithi162.stdout:2744: finishing write tid 3 to smithi16234619-28 2024-09-16T15:12:59.207 INFO:tasks.rados.rados.0.smithi162.stdout:2744: finishing write tid 4 to smithi16234619-28 2024-09-16T15:12:59.207 INFO:tasks.rados.rados.0.smithi162.stdout:2744: oid 28 updating version 0 to 1158 2024-09-16T15:12:59.208 INFO:tasks.rados.rados.0.smithi162.stdout:2744: oid 28 updating version 1158 to 1160 2024-09-16T15:12:59.208 INFO:tasks.rados.rados.0.smithi162.stdout:2744: oid 28 version 1160 is already newer than 1159 2024-09-16T15:12:59.208 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1160 (ObjNum 964 snap 271 seq_num 964) dirty exists 2024-09-16T15:12:59.208 INFO:tasks.rados.rados.0.smithi162.stdout:2744: left oid 28 (ObjNum 964 snap 271 seq_num 964) 2024-09-16T15:13:00.114 INFO:tasks.rados.rados.0.smithi162.stdout:2743: done (2 left) 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:2744: done (1 left) 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:2745: done (0 left) 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:2746: rollback oid 34 current snap is 272 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 258 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:2747: write (excl) oid 29 current snap is 272 2024-09-16T15:13:00.115 INFO:tasks.rados.rados.0.smithi162.stdout:2747: seq_num 965 ranges {618755=475763,1636300=457996} 2024-09-16T15:13:00.120 INFO:tasks.rados.rados.0.smithi162.stdout:2747: writing smithi16234619-29 from 618755 to 1094518 tid 1 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:2747: writing smithi16234619-29 from 1636300 to 2094296 tid 2 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:2748: delete oid 48 current snap is 272 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:2746: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:2746: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:2746: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:13:00.126 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1193 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:13:00.129 INFO:tasks.rados.rados.0.smithi162.stdout:2746: done (2 left) 2024-09-16T15:13:00.129 INFO:tasks.rados.rados.0.smithi162.stdout:2748: done (1 left) 2024-09-16T15:13:00.129 INFO:tasks.rados.rados.0.smithi162.stdout:2749: write (excl) oid 49 current snap is 272 2024-09-16T15:13:00.129 INFO:tasks.rados.rados.0.smithi162.stdout:2749: seq_num 967 ranges {652369=742614,2153264=99105} 2024-09-16T15:13:00.137 INFO:tasks.rados.rados.0.smithi162.stdout:2749: writing smithi16234619-49 from 652369 to 1394983 tid 1 2024-09-16T15:13:00.142 INFO:tasks.rados.rados.0.smithi162.stdout:2749: writing smithi16234619-49 from 2153264 to 2252369 tid 2 2024-09-16T15:13:00.142 INFO:tasks.rados.rados.0.smithi162.stdout:2750: snap_create 2024-09-16T15:13:00.146 INFO:tasks.rados.rados.0.smithi162.stdout:2747: finishing write tid 1 to smithi16234619-29 2024-09-16T15:13:00.146 INFO:tasks.rados.rados.0.smithi162.stdout:2747: finishing write tid 2 to smithi16234619-29 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: finishing write tid 3 to smithi16234619-29 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: finishing write tid 4 to smithi16234619-29 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: oid 29 updating version 0 to 1338 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: oid 29 version 1338 is already newer than 1337 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: oid 29 version 1338 is already newer than 1336 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1338 (ObjNum 965 snap 272 seq_num 965) dirty exists 2024-09-16T15:13:00.149 INFO:tasks.rados.rados.0.smithi162.stdout:2747: left oid 29 (ObjNum 965 snap 272 seq_num 965) 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: finishing write tid 1 to smithi16234619-49 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: finishing write tid 2 to smithi16234619-49 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: finishing write tid 3 to smithi16234619-49 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: finishing write tid 4 to smithi16234619-49 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: oid 49 updating version 0 to 814 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: oid 49 updating version 814 to 815 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: oid 49 version 815 is already newer than 813 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 815 (ObjNum 967 snap 272 seq_num 967) dirty exists 2024-09-16T15:13:00.162 INFO:tasks.rados.rados.0.smithi162.stdout:2749: left oid 49 (ObjNum 967 snap 272 seq_num 967) 2024-09-16T15:13:01.134 INFO:tasks.rados.rados.0.smithi162.stdout:2747: done (2 left) 2024-09-16T15:13:01.134 INFO:tasks.rados.rados.0.smithi162.stdout:2749: done (1 left) 2024-09-16T15:13:01.134 INFO:tasks.rados.rados.0.smithi162.stdout:2750: done (0 left) 2024-09-16T15:13:01.134 INFO:tasks.rados.rados.0.smithi162.stdout:2751: copy_from oid 2 from oid 14 current snap is 273 2024-09-16T15:13:01.134 INFO:tasks.rados.rados.0.smithi162.stdout:2752: read oid 27 snap -1 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2752: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2753: read oid 18 snap -1 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2753: expect deleted 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2754: rollback oid 1 current snap is 273 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 261 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2755: delete oid 41 current snap is 273 2024-09-16T15:13:01.135 INFO:tasks.rados.rados.0.smithi162.stdout:2753: done (4 left) 2024-09-16T15:13:01.136 INFO:tasks.rados.rados.0.smithi162.stdout:2755: done (3 left) 2024-09-16T15:13:01.136 INFO:tasks.rados.rados.0.smithi162.stdout:2756: snap_create 2024-09-16T15:13:01.138 INFO:tasks.rados.rados.0.smithi162.stdout:2752: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:01.276 INFO:tasks.rados.rados.0.smithi162.stdout:2754: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:13:01.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1146 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-16T15:13:01.276 INFO:tasks.rados.rados.0.smithi162.stdout:2751: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:13:01.276 INFO:tasks.rados.rados.0.smithi162.stdout:2751: finishing copy_from to smithi16234619-2 2024-09-16T15:13:01.276 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1149 (ObjNum 959 snap 269 seq_num 959) dirty exists 2024-09-16T15:13:02.132 INFO:tasks.rados.rados.0.smithi162.stdout:2751: done (3 left) 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2752: done (2 left) 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2754: done (1 left) 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2756: done (0 left) 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2757: read oid 7 snap -1 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2757: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:02.133 INFO:tasks.rados.rados.0.smithi162.stdout:2758: snap_remove snap 268 2024-09-16T15:13:04.030 INFO:tasks.rados.rados.0.smithi162.stdout:2758: done (1 left) 2024-09-16T15:13:04.030 INFO:tasks.rados.rados.0.smithi162.stdout:2759: write oid 9 current snap is 274 2024-09-16T15:13:04.030 INFO:tasks.rados.rados.0.smithi162.stdout:2759: seq_num 969 ranges {685983=493112,1870228=415755} 2024-09-16T15:13:04.036 INFO:tasks.rados.rados.0.smithi162.stdout:2759: writing smithi16234619-9 from 685983 to 1179095 tid 1 2024-09-16T15:13:04.043 INFO:tasks.rados.rados.0.smithi162.stdout:2759: writing smithi16234619-9 from 1870228 to 2285983 tid 2 2024-09-16T15:13:04.043 INFO:tasks.rados.rados.0.smithi162.stdout:2760: delete oid 44 current snap is 274 2024-09-16T15:13:04.043 INFO:tasks.rados.rados.0.smithi162.stdout:2757: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2759: finishing write tid 1 to smithi16234619-9 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2759: finishing write tid 2 to smithi16234619-9 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2757: done (2 left) 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2760: done (1 left) 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2761: snap_create 2024-09-16T15:13:04.215 INFO:tasks.rados.rados.0.smithi162.stdout:2759: finishing write tid 3 to smithi16234619-9 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:2759: finishing write tid 4 to smithi16234619-9 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:2759: oid 9 updating version 0 to 1268 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:2759: oid 9 version 1268 is already newer than 1266 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:2759: oid 9 version 1268 is already newer than 1267 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1268 (ObjNum 969 snap 274 seq_num 969) dirty exists 2024-09-16T15:13:04.216 INFO:tasks.rados.rados.0.smithi162.stdout:2759: left oid 9 (ObjNum 969 snap 274 seq_num 969) 2024-09-16T15:13:05.080 INFO:tasks.rados.rados.0.smithi162.stdout:2759: done (1 left) 2024-09-16T15:13:05.080 INFO:tasks.rados.rados.0.smithi162.stdout:2761: done (0 left) 2024-09-16T15:13:05.080 INFO:tasks.rados.rados.0.smithi162.stdout:2762: delete oid 25 current snap is 275 2024-09-16T15:13:05.084 INFO:tasks.rados.rados.0.smithi162.stdout:2762: done (0 left) 2024-09-16T15:13:05.084 INFO:tasks.rados.rados.0.smithi162.stdout:2763: snap_remove snap 272 2024-09-16T15:13:06.332 INFO:tasks.rados.rados.0.smithi162.stdout:2763: done (0 left) 2024-09-16T15:13:06.333 INFO:tasks.rados.rados.0.smithi162.stdout:2764: snap_remove snap 274 2024-09-16T15:13:07.347 INFO:tasks.rados.rados.0.smithi162.stdout:2764: done (0 left) 2024-09-16T15:13:07.348 INFO:tasks.rados.rados.0.smithi162.stdout:2765: delete oid 9 current snap is 275 2024-09-16T15:13:07.352 INFO:tasks.rados.rados.0.smithi162.stdout:2765: done (0 left) 2024-09-16T15:13:07.352 INFO:tasks.rados.rados.0.smithi162.stdout:2766: read oid 28 snap -1 2024-09-16T15:13:07.352 INFO:tasks.rados.rados.0.smithi162.stdout:2766: expect (ObjNum 964 snap 271 seq_num 964) 2024-09-16T15:13:07.352 INFO:tasks.rados.rados.0.smithi162.stdout:2767: read oid 1 snap -1 2024-09-16T15:13:07.352 INFO:tasks.rados.rados.0.smithi162.stdout:2767: expect deleted 2024-09-16T15:13:07.353 INFO:tasks.rados.rados.0.smithi162.stdout:2768: write (excl) oid 3 current snap is 275 2024-09-16T15:13:07.353 INFO:tasks.rados.rados.0.smithi162.stdout:2768: seq_num 973 ranges {753211=794108,1987803=365408} 2024-09-16T15:13:07.360 INFO:tasks.rados.rados.0.smithi162.stdout:2768: writing smithi16234619-3 from 753211 to 1547319 tid 1 2024-09-16T15:13:07.365 INFO:tasks.rados.rados.0.smithi162.stdout:2768: writing smithi16234619-3 from 1987803 to 2353211 tid 2 2024-09-16T15:13:07.365 INFO:tasks.rados.rados.0.smithi162.stdout:2769: write oid 50 current snap is 275 2024-09-16T15:13:07.365 INFO:tasks.rados.rados.0.smithi162.stdout:2769: seq_num 974 ranges {770018=585710,2046285=323733} 2024-09-16T15:13:07.372 INFO:tasks.rados.rados.0.smithi162.stdout:2769: writing smithi16234619-50 from 770018 to 1355728 tid 1 2024-09-16T15:13:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2769: writing smithi16234619-50 from 2046285 to 2370018 tid 2 2024-09-16T15:13:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2770: read oid 13 snap -1 2024-09-16T15:13:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2770: expect deleted 2024-09-16T15:13:07.378 INFO:tasks.rados.rados.0.smithi162.stdout:2771: snap_remove snap 261 2024-09-16T15:13:08.352 INFO:tasks.rados.rados.0.smithi162.stdout:2771: done (5 left) 2024-09-16T15:13:08.352 INFO:tasks.rados.rados.0.smithi162.stdout:2772: write oid 2 current snap is 275 2024-09-16T15:13:08.352 INFO:tasks.rados.rados.0.smithi162.stdout:2772: seq_num 975 ranges {786825=660959,2104767=282058} 2024-09-16T15:13:08.360 INFO:tasks.rados.rados.0.smithi162.stdout:2772: writing smithi16234619-2 from 786825 to 1447784 tid 1 2024-09-16T15:13:08.366 INFO:tasks.rados.rados.0.smithi162.stdout:2772: writing smithi16234619-2 from 2104767 to 2386825 tid 2 2024-09-16T15:13:08.366 INFO:tasks.rados.rados.0.smithi162.stdout:2773: read oid 27 snap 258 2024-09-16T15:13:08.366 INFO:tasks.rados.rados.0.smithi162.stdout:2773: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:08.366 INFO:tasks.rados.rados.0.smithi162.stdout:2774: delete oid 18 current snap is 275 2024-09-16T15:13:08.366 INFO:tasks.rados.rados.0.smithi162.stdout:2766: expect (ObjNum 964 snap 271 seq_num 964) 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: finishing write tid 1 to smithi16234619-3 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: finishing write tid 2 to smithi16234619-3 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: finishing write tid 3 to smithi16234619-3 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: finishing write tid 4 to smithi16234619-3 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: oid 3 updating version 0 to 145 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: oid 3 updating version 145 to 146 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: oid 3 updating version 146 to 147 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 147 (ObjNum 973 snap 275 seq_num 973) dirty exists 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2768: left oid 3 (ObjNum 973 snap 275 seq_num 973) 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2769: finishing write tid 1 to smithi16234619-50 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2766: done (7 left) 2024-09-16T15:13:08.486 INFO:tasks.rados.rados.0.smithi162.stdout:2767: done (6 left) 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2768: done (5 left) 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2770: done (4 left) 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2774: done (3 left) 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2775: copy_from oid 35 from oid 8 current snap is 275 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: finishing write tid 2 to smithi16234619-50 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: finishing write tid 3 to smithi16234619-50 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: finishing write tid 4 to smithi16234619-50 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: oid 50 updating version 0 to 1207 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: oid 50 updating version 1207 to 1208 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: oid 50 updating version 1208 to 1209 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1209 (ObjNum 974 snap 275 seq_num 974) dirty exists 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2769: left oid 50 (ObjNum 974 snap 275 seq_num 974) 2024-09-16T15:13:08.487 INFO:tasks.rados.rados.0.smithi162.stdout:2773: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: finishing write tid 1 to smithi16234619-2 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: finishing write tid 2 to smithi16234619-2 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: finishing write tid 3 to smithi16234619-2 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: finishing write tid 4 to smithi16234619-2 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: oid 2 updating version 0 to 1423 2024-09-16T15:13:08.621 INFO:tasks.rados.rados.0.smithi162.stdout:2772: oid 2 version 1423 is already newer than 1418 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2772: oid 2 updating version 1423 to 1424 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1424 (ObjNum 975 snap 275 seq_num 975) dirty exists 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2772: left oid 2 (ObjNum 975 snap 275 seq_num 975) 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2775: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2775: finishing copy_from to smithi16234619-35 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 922 (ObjNum 957 snap 268 seq_num 957) dirty exists 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2769: done (3 left) 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2772: done (2 left) 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2773: done (1 left) 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2775: done (0 left) 2024-09-16T15:13:08.622 INFO:tasks.rados.rados.0.smithi162.stdout:2776: delete oid 42 current snap is 275 2024-09-16T15:13:08.625 INFO:tasks.rados.rados.0.smithi162.stdout:2776: done (0 left) 2024-09-16T15:13:08.625 INFO:tasks.rados.rados.0.smithi162.stdout:2777: copy_from oid 46 from oid 34 current snap is 275 2024-09-16T15:13:08.625 INFO:tasks.rados.rados.0.smithi162.stdout:2778: snap_create 2024-09-16T15:13:08.661 INFO:tasks.rados.rados.0.smithi162.stdout:2777: finishing copy_from racing read to smithi16234619-46 2024-09-16T15:13:08.662 INFO:tasks.rados.rados.0.smithi162.stdout:2777: finishing copy_from to smithi16234619-46 2024-09-16T15:13:08.662 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1193 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:13:09.030 INFO:tasks.rados.rados.0.smithi162.stdout:2777: done (1 left) 2024-09-16T15:13:09.030 INFO:tasks.rados.rados.0.smithi162.stdout:2778: done (0 left) 2024-09-16T15:13:09.031 INFO:tasks.rados.rados.0.smithi162.stdout:2779: copy_from oid 28 from oid 18 current snap is 276 2024-09-16T15:13:09.031 INFO:tasks.rados.rados.0.smithi162.stdout:2780: write oid 16 current snap is 276 2024-09-16T15:13:09.031 INFO:tasks.rados.rados.0.smithi162.stdout:2780: seq_num 978 ranges {437246=486706,1363860=623668} 2024-09-16T15:13:09.035 INFO:tasks.rados.rados.0.smithi162.stdout:2780: writing smithi16234619-16 from 437246 to 923952 tid 1 2024-09-16T15:13:09.043 INFO:tasks.rados.rados.0.smithi162.stdout:2780: writing smithi16234619-16 from 1363860 to 1987528 tid 2 2024-09-16T15:13:09.044 INFO:tasks.rados.rados.0.smithi162.stdout:2781: snap_remove snap 258 2024-09-16T15:13:10.041 INFO:tasks.rados.rados.0.smithi162.stdout:2781: done (2 left) 2024-09-16T15:13:10.041 INFO:tasks.rados.rados.0.smithi162.stdout:2782: write oid 9 current snap is 276 2024-09-16T15:13:10.041 INFO:tasks.rados.rados.0.smithi162.stdout:2782: seq_num 979 ranges {454053=561955,1422342=483679,2368404=85649} 2024-09-16T15:13:10.046 INFO:tasks.rados.rados.0.smithi162.stdout:2782: writing smithi16234619-9 from 454053 to 1016008 tid 1 2024-09-16T15:13:10.051 INFO:tasks.rados.rados.0.smithi162.stdout:2782: writing smithi16234619-9 from 1422342 to 1906021 tid 2 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2782: writing smithi16234619-9 from 2368404 to 2454053 tid 3 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2783: delete oid 42 current snap is 276 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2779: finishing copy_from to smithi16234619-28 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2779: got expected ENOENT (src dne) 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2779: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2780: finishing write tid 1 to smithi16234619-16 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2780: finishing write tid 2 to smithi16234619-16 2024-09-16T15:13:10.055 INFO:tasks.rados.rados.0.smithi162.stdout:2780: finishing write tid 3 to smithi16234619-16 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:2780: finishing write tid 4 to smithi16234619-16 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:2780: oid 16 updating version 0 to 1270 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:2780: oid 16 version 1270 is already newer than 1269 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:2780: oid 16 updating version 1270 to 1271 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1271 (ObjNum 978 snap 276 seq_num 978) dirty exists 2024-09-16T15:13:10.056 INFO:tasks.rados.rados.0.smithi162.stdout:2780: left oid 16 (ObjNum 978 snap 276 seq_num 978) 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2779: done (3 left) 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2780: done (2 left) 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2783: done (1 left) 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2784: read oid 44 snap 270 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2784: expect (ObjNum 962 snap 269 seq_num 962) 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2785: write oid 50 current snap is 276 2024-09-16T15:13:10.060 INFO:tasks.rados.rados.0.smithi162.stdout:2785: seq_num 981 ranges {487667=428806,1655659=770995} 2024-09-16T15:13:10.065 INFO:tasks.rados.rados.0.smithi162.stdout:2785: writing smithi16234619-50 from 487667 to 916473 tid 1 2024-09-16T15:13:10.074 INFO:tasks.rados.rados.0.smithi162.stdout:2785: writing smithi16234619-50 from 1655659 to 2426654 tid 2 2024-09-16T15:13:10.074 INFO:tasks.rados.rados.0.smithi162.stdout:2786: rollback oid 10 current snap is 276 2024-09-16T15:13:10.074 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 251 2024-09-16T15:13:10.074 INFO:tasks.rados.rados.0.smithi162.stdout:2784: expect (ObjNum 962 snap 269 seq_num 962) 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2786: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2786: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1444 (ObjNum 904 snap 249 seq_num 904) dirty exists 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2782: finishing write tid 1 to smithi16234619-9 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2782: finishing write tid 2 to smithi16234619-9 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2782: finishing write tid 3 to smithi16234619-9 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2782: finishing write tid 4 to smithi16234619-9 2024-09-16T15:13:10.145 INFO:tasks.rados.rados.0.smithi162.stdout:2782: finishing write tid 5 to smithi16234619-9 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: oid 9 updating version 0 to 1270 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: oid 9 updating version 1270 to 1273 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: oid 9 version 1273 is already newer than 1272 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: oid 9 version 1273 is already newer than 1271 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1273 (ObjNum 979 snap 276 seq_num 979) dirty exists 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: left oid 9 (ObjNum 979 snap 276 seq_num 979) 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2785: finishing write tid 1 to smithi16234619-50 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2785: finishing write tid 2 to smithi16234619-50 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2785: finishing write tid 3 to smithi16234619-50 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2782: done (3 left) 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2784: done (2 left) 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2786: done (1 left) 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:2787: rollback oid 5 current snap is 276 2024-09-16T15:13:10.146 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 230 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: finishing write tid 4 to smithi16234619-50 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: oid 50 updating version 0 to 1235 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: oid 50 updating version 1235 to 1237 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: oid 50 version 1237 is already newer than 1236 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1237 (ObjNum 981 snap 276 seq_num 981) dirty exists 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: left oid 50 (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2785: done (1 left) 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2788: delete oid 19 current snap is 276 2024-09-16T15:13:10.147 INFO:tasks.rados.rados.0.smithi162.stdout:2788: done (1 left) 2024-09-16T15:13:10.148 INFO:tasks.rados.rados.0.smithi162.stdout:2789: snap_create 2024-09-16T15:13:10.149 INFO:tasks.rados.rados.0.smithi162.stdout:2787: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:13:10.149 INFO:tasks.rados.rados.0.smithi162.stdout:2787: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:13:10.151 INFO:tasks.rados.rados.0.smithi162.stdout:2787: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:13:10.151 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1359 (ObjNum 817 snap 221 seq_num 817) dirty exists 2024-09-16T15:13:11.050 INFO:tasks.rados.rados.0.smithi162.stdout:2787: done (1 left) 2024-09-16T15:13:11.051 INFO:tasks.rados.rados.0.smithi162.stdout:2789: done (0 left) 2024-09-16T15:13:11.051 INFO:tasks.rados.rados.0.smithi162.stdout:2790: snap_create 2024-09-16T15:13:12.056 INFO:tasks.rados.rados.0.smithi162.stdout:2790: done (0 left) 2024-09-16T15:13:12.056 INFO:tasks.rados.rados.0.smithi162.stdout:2791: read oid 40 snap -1 2024-09-16T15:13:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2791: expect (ObjNum 945 snap 267 seq_num 945) 2024-09-16T15:13:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2792: write oid 18 current snap is 278 2024-09-16T15:13:12.057 INFO:tasks.rados.rados.0.smithi162.stdout:2792: seq_num 983 ranges {521281=579304,1656270=774664} 2024-09-16T15:13:12.062 INFO:tasks.rados.rados.0.smithi162.stdout:2792: writing smithi16234619-18 from 521281 to 1100585 tid 1 2024-09-16T15:13:12.070 INFO:tasks.rados.rados.0.smithi162.stdout:2792: writing smithi16234619-18 from 1656270 to 2430934 tid 2 2024-09-16T15:13:12.070 INFO:tasks.rados.rados.0.smithi162.stdout:2793: read oid 29 snap -1 2024-09-16T15:13:12.070 INFO:tasks.rados.rados.0.smithi162.stdout:2793: expect (ObjNum 965 snap 272 seq_num 965) 2024-09-16T15:13:12.070 INFO:tasks.rados.rados.0.smithi162.stdout:2794: write oid 25 current snap is 278 2024-09-16T15:13:12.070 INFO:tasks.rados.rados.0.smithi162.stdout:2794: seq_num 984 ranges {538088=654553,1714752=634675} 2024-09-16T15:13:12.076 INFO:tasks.rados.rados.0.smithi162.stdout:2794: writing smithi16234619-25 from 538088 to 1192641 tid 1 2024-09-16T15:13:12.082 INFO:tasks.rados.rados.0.smithi162.stdout:2794: writing smithi16234619-25 from 1714752 to 2349427 tid 2 2024-09-16T15:13:12.082 INFO:tasks.rados.rados.0.smithi162.stdout:2795: snap_create 2024-09-16T15:13:12.083 INFO:tasks.rados.rados.0.smithi162.stdout:2791: expect (ObjNum 945 snap 267 seq_num 945) 2024-09-16T15:13:12.208 INFO:tasks.rados.rados.0.smithi162.stdout:2793: expect (ObjNum 965 snap 272 seq_num 965) 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: finishing write tid 1 to smithi16234619-25 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2792: finishing write tid 1 to smithi16234619-18 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: finishing write tid 2 to smithi16234619-25 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: finishing write tid 3 to smithi16234619-25 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: finishing write tid 4 to smithi16234619-25 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: oid 25 updating version 0 to 1241 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: oid 25 version 1241 is already newer than 1240 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:2794: oid 25 updating version 1241 to 1242 2024-09-16T15:13:12.280 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 1242 (ObjNum 984 snap 278 seq_num 984) dirty exists 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2794: left oid 25 (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: finishing write tid 2 to smithi16234619-18 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: finishing write tid 3 to smithi16234619-18 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: finishing write tid 4 to smithi16234619-18 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: oid 18 updating version 0 to 923 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: oid 18 updating version 923 to 924 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: oid 18 updating version 924 to 925 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 925 (ObjNum 983 snap 278 seq_num 983) dirty exists 2024-09-16T15:13:12.281 INFO:tasks.rados.rados.0.smithi162.stdout:2792: left oid 18 (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2791: done (4 left) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2792: done (3 left) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2793: done (2 left) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2794: done (1 left) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2795: done (0 left) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2796: read oid 5 snap -1 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2796: expect (ObjNum 817 snap 221 seq_num 817) 2024-09-16T15:13:13.672 INFO:tasks.rados.rados.0.smithi162.stdout:2797: snap_create 2024-09-16T15:13:13.674 INFO:tasks.rados.rados.0.smithi162.stdout:2796: expect (ObjNum 817 snap 221 seq_num 817) 2024-09-16T15:13:14.036 INFO:tasks.rados.rados.0.smithi162.stdout:2796: done (1 left) 2024-09-16T15:13:14.036 INFO:tasks.rados.rados.0.smithi162.stdout:2797: done (0 left) 2024-09-16T15:13:14.036 INFO:tasks.rados.rados.0.smithi162.stdout:2798: write (excl) oid 21 current snap is 280 2024-09-16T15:13:14.036 INFO:tasks.rados.rados.0.smithi162.stdout:2798: seq_num 985 ranges {554895=729802,1773234=778333} 2024-09-16T15:13:14.042 INFO:tasks.rados.rados.0.smithi162.stdout:2798: writing smithi16234619-21 from 554895 to 1284697 tid 1 2024-09-16T15:13:14.049 INFO:tasks.rados.rados.0.smithi162.stdout:2798: writing smithi16234619-21 from 1773234 to 2551567 tid 2 2024-09-16T15:13:14.050 INFO:tasks.rados.rados.0.smithi162.stdout:2799: snap_create 2024-09-16T15:13:14.081 INFO:tasks.rados.rados.0.smithi162.stdout:2798: finishing write tid 1 to smithi16234619-21 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: finishing write tid 2 to smithi16234619-21 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: finishing write tid 3 to smithi16234619-21 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: finishing write tid 4 to smithi16234619-21 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: oid 21 updating version 0 to 1446 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: oid 21 updating version 1446 to 1447 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: oid 21 updating version 1447 to 1448 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1448 (ObjNum 985 snap 280 seq_num 985) dirty exists 2024-09-16T15:13:14.082 INFO:tasks.rados.rados.0.smithi162.stdout:2798: left oid 21 (ObjNum 985 snap 280 seq_num 985) 2024-09-16T15:13:15.041 INFO:tasks.rados.rados.0.smithi162.stdout:2798: done (1 left) 2024-09-16T15:13:15.041 INFO:tasks.rados.rados.0.smithi162.stdout:2799: done (0 left) 2024-09-16T15:13:15.041 INFO:tasks.rados.rados.0.smithi162.stdout:2800: rollback oid 27 current snap is 281 2024-09-16T15:13:15.041 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 27 to 269 2024-09-16T15:13:15.041 INFO:tasks.rados.rados.0.smithi162.stdout:2801: write (excl) oid 35 current snap is 281 2024-09-16T15:13:15.042 INFO:tasks.rados.rados.0.smithi162.stdout:2801: seq_num 986 ranges {571702=405051,1431716=638344,2560365=11337} 2024-09-16T15:13:15.046 INFO:tasks.rados.rados.0.smithi162.stdout:2801: writing smithi16234619-35 from 571702 to 976753 tid 1 2024-09-16T15:13:15.052 INFO:tasks.rados.rados.0.smithi162.stdout:2801: writing smithi16234619-35 from 1431716 to 2070060 tid 2 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2801: writing smithi16234619-35 from 2560365 to 2571702 tid 3 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2802: copy_from oid 11 from oid 9 current snap is 281 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2800: finishing rollback tid 0 to smithi16234619-27 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2800: finishing rollback tid 1 to smithi16234619-27 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2800: finishing rollback tid 2 to smithi16234619-27 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 1276 (ObjNum 892 snap 246 seq_num 892) dirty exists 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2800: done (2 left) 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2803: copy_from oid 14 from oid 42 current snap is 281 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2804: write (excl) oid 15 current snap is 281 2024-09-16T15:13:15.056 INFO:tasks.rados.rados.0.smithi162.stdout:2804: seq_num 987 ranges {588509=480300,1773845=782002} 2024-09-16T15:13:15.062 INFO:tasks.rados.rados.0.smithi162.stdout:2804: writing smithi16234619-15 from 588509 to 1068809 tid 1 2024-09-16T15:13:15.070 INFO:tasks.rados.rados.0.smithi162.stdout:2804: writing smithi16234619-15 from 1773845 to 2555847 tid 2 2024-09-16T15:13:15.070 INFO:tasks.rados.rados.0.smithi162.stdout:2805: delete oid 4 current snap is 281 2024-09-16T15:13:15.070 INFO:tasks.rados.rados.0.smithi162.stdout:2803: finishing copy_from to smithi16234619-14 2024-09-16T15:13:15.070 INFO:tasks.rados.rados.0.smithi162.stdout:2803: got expected ENOENT (src dne) 2024-09-16T15:13:15.070 INFO:tasks.rados.rados.0.smithi162.stdout:2803: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:13:15.074 INFO:tasks.rados.rados.0.smithi162.stdout:2803: done (4 left) 2024-09-16T15:13:15.074 INFO:tasks.rados.rados.0.smithi162.stdout:2805: done (3 left) 2024-09-16T15:13:15.074 INFO:tasks.rados.rados.0.smithi162.stdout:2806: read oid 38 snap -1 2024-09-16T15:13:15.074 INFO:tasks.rados.rados.0.smithi162.stdout:2806: expect deleted 2024-09-16T15:13:15.074 INFO:tasks.rados.rados.0.smithi162.stdout:2807: snap_remove snap 271 2024-09-16T15:13:16.672 INFO:tasks.rados.rados.0.smithi162.stdout:2807: done (4 left) 2024-09-16T15:13:16.672 INFO:tasks.rados.rados.0.smithi162.stdout:2808: snap_remove snap 251 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2808: done (4 left) 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2809: copy_from oid 31 from oid 32 current snap is 281 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: finishing write tid 1 to smithi16234619-35 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: finishing write tid 2 to smithi16234619-35 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: finishing write tid 3 to smithi16234619-35 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: finishing write tid 4 to smithi16234619-35 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: finishing write tid 5 to smithi16234619-35 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: oid 35 updating version 0 to 1275 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: oid 35 version 1275 is already newer than 1272 2024-09-16T15:13:17.678 INFO:tasks.rados.rados.0.smithi162.stdout:2801: oid 35 version 1275 is already newer than 1274 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2801: oid 35 version 1275 is already newer than 1273 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1275 (ObjNum 986 snap 281 seq_num 986) dirty exists 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2801: left oid 35 (ObjNum 986 snap 281 seq_num 986) 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2802: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2802: finishing copy_from to smithi16234619-11 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1273 (ObjNum 979 snap 276 seq_num 979) dirty exists 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: finishing write tid 1 to smithi16234619-15 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: finishing write tid 2 to smithi16234619-15 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: finishing write tid 3 to smithi16234619-15 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: finishing write tid 4 to smithi16234619-15 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: oid 15 updating version 0 to 1162 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: oid 15 updating version 1162 to 1163 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:2804: oid 15 version 1163 is already newer than 1161 2024-09-16T15:13:17.679 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 1163 (ObjNum 987 snap 281 seq_num 987) dirty exists 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2804: left oid 15 (ObjNum 987 snap 281 seq_num 987) 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2801: done (4 left) 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2802: done (3 left) 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2804: done (2 left) 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2806: done (1 left) 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2810: write oid 34 current snap is 281 2024-09-16T15:13:17.680 INFO:tasks.rados.rados.0.smithi162.stdout:2810: seq_num 989 ranges {622123=747151,2007162=614961} 2024-09-16T15:13:17.685 INFO:tasks.rados.rados.0.smithi162.stdout:2810: writing smithi16234619-34 from 622123 to 1369274 tid 1 2024-09-16T15:13:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:2810: writing smithi16234619-34 from 2007162 to 2622123 tid 2 2024-09-16T15:13:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:2811: rollback oid 41 current snap is 281 2024-09-16T15:13:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 41 to 280 2024-09-16T15:13:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:2812: write (excl) oid 28 current snap is 281 2024-09-16T15:13:17.692 INFO:tasks.rados.rados.0.smithi162.stdout:2812: seq_num 990 ranges {638930=422400,1665644=645682} 2024-09-16T15:13:17.697 INFO:tasks.rados.rados.0.smithi162.stdout:2812: writing smithi16234619-28 from 638930 to 1061330 tid 1 2024-09-16T15:13:17.704 INFO:tasks.rados.rados.0.smithi162.stdout:2812: writing smithi16234619-28 from 1665644 to 2311326 tid 2 2024-09-16T15:13:17.704 INFO:tasks.rados.rados.0.smithi162.stdout:2813: copy_from oid 44 from oid 26 current snap is 281 2024-09-16T15:13:17.704 INFO:tasks.rados.rados.0.smithi162.stdout:2811: finishing rollback tid 1 to smithi16234619-41 2024-09-16T15:13:17.705 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 808 (ObjNum 129 snap 0 seq_num 864803312) dirty dne 2024-09-16T15:13:17.705 INFO:tasks.rados.rados.0.smithi162.stdout:2811: done (4 left) 2024-09-16T15:13:17.705 INFO:tasks.rados.rados.0.smithi162.stdout:2814: write (excl) oid 48 current snap is 281 2024-09-16T15:13:17.705 INFO:tasks.rados.rados.0.smithi162.stdout:2814: seq_num 991 ranges {655737=497649,1607773=789340} 2024-09-16T15:13:17.710 INFO:tasks.rados.rados.0.smithi162.stdout:2814: writing smithi16234619-48 from 655737 to 1153386 tid 1 2024-09-16T15:13:17.719 INFO:tasks.rados.rados.0.smithi162.stdout:2814: writing smithi16234619-48 from 1607773 to 2397113 tid 2 2024-09-16T15:13:17.719 INFO:tasks.rados.rados.0.smithi162.stdout:2815: rollback oid 14 current snap is 281 2024-09-16T15:13:17.719 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 276 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2813: finishing copy_from to smithi16234619-44 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2813: got expected ENOENT (src dne) 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2813: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2813: done (5 left) 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2816: write oid 5 current snap is 281 2024-09-16T15:13:17.720 INFO:tasks.rados.rados.0.smithi162.stdout:2816: seq_num 992 ranges {672544=572898,1666255=532998} 2024-09-16T15:13:17.727 INFO:tasks.rados.rados.0.smithi162.stdout:2816: writing smithi16234619-5 from 672544 to 1245442 tid 1 2024-09-16T15:13:17.732 INFO:tasks.rados.rados.0.smithi162.stdout:2816: writing smithi16234619-5 from 1666255 to 2199253 tid 2 2024-09-16T15:13:17.733 INFO:tasks.rados.rados.0.smithi162.stdout:2817: write (excl) oid 3 current snap is 281 2024-09-16T15:13:17.733 INFO:tasks.rados.rados.0.smithi162.stdout:2812: finishing write tid 1 to smithi16234619-28 2024-09-16T15:13:17.733 INFO:tasks.rados.rados.0.smithi162.stdout:2817: seq_num 993 ranges {689351=648147,2124737=564614} 2024-09-16T15:13:17.740 INFO:tasks.rados.rados.0.smithi162.stdout:2817: writing smithi16234619-3 from 689351 to 1337498 tid 1 2024-09-16T15:13:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:2817: writing smithi16234619-3 from 2124737 to 2689351 tid 2 2024-09-16T15:13:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:2818: write oid 20 current snap is 281 2024-09-16T15:13:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:2818: seq_num 994 ranges {706158=723396,2183219=522939} 2024-09-16T15:13:17.755 INFO:tasks.rados.rados.0.smithi162.stdout:2818: writing smithi16234619-20 from 706158 to 1429554 tid 1 2024-09-16T15:13:17.762 INFO:tasks.rados.rados.0.smithi162.stdout:2818: writing smithi16234619-20 from 2183219 to 2706158 tid 2 2024-09-16T15:13:17.762 INFO:tasks.rados.rados.0.smithi162.stdout:2819: snap_remove snap 280 2024-09-16T15:13:18.686 INFO:tasks.rados.rados.0.smithi162.stdout:2819: done (8 left) 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2820: copy_from oid 36 from oid 22 current snap is 281 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2809: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: finishing write tid 2 to smithi16234619-28 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2809: finishing copy_from to smithi16234619-31 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 898 (ObjNum 872 snap 237 seq_num 872) dirty exists 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: finishing write tid 3 to smithi16234619-28 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: finishing write tid 4 to smithi16234619-28 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: oid 28 updating version 0 to 1166 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: oid 28 version 1166 is already newer than 1165 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: oid 28 version 1166 is already newer than 1164 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1166 (ObjNum 990 snap 281 seq_num 990) dirty exists 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2812: left oid 28 (ObjNum 990 snap 281 seq_num 990) 2024-09-16T15:13:18.687 INFO:tasks.rados.rados.0.smithi162.stdout:2810: finishing write tid 1 to smithi16234619-34 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2814: finishing write tid 1 to smithi16234619-48 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: finishing write tid 2 to smithi16234619-34 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: finishing write tid 3 to smithi16234619-34 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2814: finishing write tid 2 to smithi16234619-48 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2814: finishing write tid 3 to smithi16234619-48 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: finishing write tid 4 to smithi16234619-34 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: oid 34 updating version 0 to 1265 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: oid 34 version 1265 is already newer than 1260 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: oid 34 updating version 1265 to 1266 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1266 (ObjNum 989 snap 281 seq_num 989) dirty exists 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2810: left oid 34 (ObjNum 989 snap 281 seq_num 989) 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2815: finishing rollback tid 0 to smithi16234619-14 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2815: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2814: finishing write tid 4 to smithi16234619-48 2024-09-16T15:13:18.688 INFO:tasks.rados.rados.0.smithi162.stdout:2814: oid 48 updating version 0 to 1278 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2814: oid 48 updating version 1278 to 1279 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2814: oid 48 updating version 1279 to 1280 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1280 (ObjNum 991 snap 281 seq_num 991) dirty exists 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2814: left oid 48 (ObjNum 991 snap 281 seq_num 991) 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2815: finishing rollback tid 2 to smithi16234619-14 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1270 (ObjNum 959 snap 269 seq_num 959) dirty exists 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: finishing write tid 1 to smithi16234619-5 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: finishing write tid 2 to smithi16234619-5 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: finishing write tid 3 to smithi16234619-5 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: finishing write tid 4 to smithi16234619-5 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: oid 5 updating version 0 to 1375 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: oid 5 version 1375 is already newer than 1374 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: oid 5 version 1375 is already newer than 1373 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1375 (ObjNum 992 snap 281 seq_num 992) dirty exists 2024-09-16T15:13:18.689 INFO:tasks.rados.rados.0.smithi162.stdout:2816: left oid 5 (ObjNum 992 snap 281 seq_num 992) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: finishing write tid 1 to smithi16234619-3 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2809: done (8 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2810: done (7 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2812: done (6 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2814: done (5 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2815: done (4 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2816: done (3 left) 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2821: write oid 38 current snap is 281 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: finishing write tid 2 to smithi16234619-3 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: finishing write tid 3 to smithi16234619-3 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: finishing write tid 4 to smithi16234619-3 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: oid 3 updating version 0 to 158 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: oid 3 version 158 is already newer than 157 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: oid 3 updating version 158 to 159 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 159 (ObjNum 993 snap 281 seq_num 993) dirty exists 2024-09-16T15:13:18.690 INFO:tasks.rados.rados.0.smithi162.stdout:2817: left oid 3 (ObjNum 993 snap 281 seq_num 993) 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: finishing write tid 1 to smithi16234619-20 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: finishing write tid 2 to smithi16234619-20 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: finishing write tid 3 to smithi16234619-20 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: finishing write tid 4 to smithi16234619-20 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: oid 20 updating version 0 to 823 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: oid 20 updating version 823 to 824 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: oid 20 updating version 824 to 825 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 825 (ObjNum 994 snap 281 seq_num 994) dirty exists 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2818: left oid 20 (ObjNum 994 snap 281 seq_num 994) 2024-09-16T15:13:18.691 INFO:tasks.rados.rados.0.smithi162.stdout:2821: seq_num 995 ranges {722965=798645,2125348=597617} 2024-09-16T15:13:18.693 INFO:tasks.rados.rados.0.smithi162.stdout:2821: writing smithi16234619-38 from 722965 to 1521610 tid 1 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2821: writing smithi16234619-38 from 2125348 to 2722965 tid 2 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2817: done (3 left) 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2818: done (2 left) 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2822: copy_from oid 37 from oid 6 current snap is 281 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2820: finishing copy_from to smithi16234619-36 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2820: got expected ENOENT (src dne) 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2820: finishing copy_from racing read to smithi16234619-36 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2820: done (2 left) 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2823: write (excl) oid 23 current snap is 281 2024-09-16T15:13:18.701 INFO:tasks.rados.rados.0.smithi162.stdout:2823: seq_num 996 ranges {739772=590247,1900183=540336} 2024-09-16T15:13:18.708 INFO:tasks.rados.rados.0.smithi162.stdout:2823: writing smithi16234619-23 from 739772 to 1330019 tid 1 2024-09-16T15:13:18.714 INFO:tasks.rados.rados.0.smithi162.stdout:2823: writing smithi16234619-23 from 1900183 to 2440519 tid 2 2024-09-16T15:13:18.714 INFO:tasks.rados.rados.0.smithi162.stdout:2824: read oid 5 snap -1 2024-09-16T15:13:18.715 INFO:tasks.rados.rados.0.smithi162.stdout:2824: expect (ObjNum 992 snap 281 seq_num 992) 2024-09-16T15:13:18.715 INFO:tasks.rados.rados.0.smithi162.stdout:2825: read oid 42 snap 275 2024-09-16T15:13:18.715 INFO:tasks.rados.rados.0.smithi162.stdout:2825: expect deleted 2024-09-16T15:13:18.715 INFO:tasks.rados.rados.0.smithi162.stdout:2826: snap_remove snap 270 2024-09-16T15:13:19.033 INFO:tasks.rados.rados.0.smithi162.stdout:2824: expect (ObjNum 992 snap 281 seq_num 992) 2024-09-16T15:13:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:2821: finishing write tid 1 to smithi16234619-38 2024-09-16T15:13:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:2822: finishing copy_from to smithi16234619-37 2024-09-16T15:13:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:2822: got expected ENOENT (src dne) 2024-09-16T15:13:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:2822: finishing copy_from racing read to smithi16234619-37 2024-09-16T15:13:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:2821: finishing write tid 2 to smithi16234619-38 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: finishing write tid 3 to smithi16234619-38 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: finishing write tid 4 to smithi16234619-38 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: oid 38 updating version 0 to 1277 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: oid 38 version 1277 is already newer than 1276 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: oid 38 updating version 1277 to 1278 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1278 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: left oid 38 (ObjNum 995 snap 281 seq_num 995) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2823: finishing write tid 1 to smithi16234619-23 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2821: done (5 left) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2822: done (4 left) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2824: done (3 left) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2825: done (2 left) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2826: done (1 left) 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2827: read oid 40 snap -1 2024-09-16T15:13:19.174 INFO:tasks.rados.rados.0.smithi162.stdout:2827: expect (ObjNum 945 snap 267 seq_num 945) 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2828: delete oid 6 current snap is 281 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: finishing write tid 2 to smithi16234619-23 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: finishing write tid 3 to smithi16234619-23 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: finishing write tid 4 to smithi16234619-23 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: oid 23 updating version 0 to 599 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: oid 23 updating version 599 to 600 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: oid 23 version 600 is already newer than 598 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 600 (ObjNum 996 snap 281 seq_num 996) dirty exists 2024-09-16T15:13:19.175 INFO:tasks.rados.rados.0.smithi162.stdout:2823: left oid 23 (ObjNum 996 snap 281 seq_num 996) 2024-09-16T15:13:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2823: done (2 left) 2024-09-16T15:13:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2828: done (1 left) 2024-09-16T15:13:19.178 INFO:tasks.rados.rados.0.smithi162.stdout:2829: snap_create 2024-09-16T15:13:19.182 INFO:tasks.rados.rados.0.smithi162.stdout:2827: expect (ObjNum 945 snap 267 seq_num 945) 2024-09-16T15:13:20.050 INFO:tasks.rados.rados.0.smithi162.stdout:2827: done (1 left) 2024-09-16T15:13:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:2829: done (0 left) 2024-09-16T15:13:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:2830: read oid 25 snap -1 2024-09-16T15:13:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:2830: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:13:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:2831: delete oid 31 current snap is 282 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:2831: done (1 left) 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:2832: rollback oid 6 current snap is 282 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 279 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:2833: rollback oid 45 current snap is 282 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 230 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:2834: write (excl) oid 50 current snap is 282 2024-09-16T15:13:20.054 INFO:tasks.rados.rados.0.smithi162.stdout:2834: seq_num 999 ranges {790193=415994,1959276=404016} 2024-09-16T15:13:20.061 INFO:tasks.rados.rados.0.smithi162.stdout:2834: writing smithi16234619-50 from 790193 to 1206187 tid 1 2024-09-16T15:13:20.067 INFO:tasks.rados.rados.0.smithi162.stdout:2834: writing smithi16234619-50 from 1959276 to 2363292 tid 2 2024-09-16T15:13:20.067 INFO:tasks.rados.rados.0.smithi162.stdout:2835: delete oid 30 current snap is 282 2024-09-16T15:13:20.067 INFO:tasks.rados.rados.0.smithi162.stdout:2832: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:13:20.067 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 965 (ObjNum 33 snap 0 seq_num 851388208) dirty dne 2024-09-16T15:13:20.068 INFO:tasks.rados.rados.0.smithi162.stdout:2830: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2833: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2833: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2833: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1468 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: finishing write tid 1 to smithi16234619-50 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: finishing write tid 2 to smithi16234619-50 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: finishing write tid 3 to smithi16234619-50 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: finishing write tid 4 to smithi16234619-50 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: oid 50 updating version 0 to 1286 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: oid 50 updating version 1286 to 1288 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: oid 50 version 1288 is already newer than 1287 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1288 (ObjNum 999 snap 282 seq_num 999) dirty exists 2024-09-16T15:13:20.148 INFO:tasks.rados.rados.0.smithi162.stdout:2834: left oid 50 (ObjNum 999 snap 282 seq_num 999) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2830: done (4 left) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2832: done (3 left) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2833: done (2 left) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2834: done (1 left) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2835: done (0 left) 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2836: rollback oid 36 current snap is 282 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 36 to 269 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2837: rollback oid 37 current snap is 282 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 278 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2838: copy_from oid 32 from oid 2 current snap is 282 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2839: copy_from oid 13 from oid 50 current snap is 282 2024-09-16T15:13:20.149 INFO:tasks.rados.rados.0.smithi162.stdout:2840: snap_create 2024-09-16T15:13:20.150 INFO:tasks.rados.rados.0.smithi162.stdout:2836: finishing rollback tid 1 to smithi16234619-36 2024-09-16T15:13:20.150 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1097 (ObjNum 97 snap 0 seq_num 865745760) dirty dne 2024-09-16T15:13:20.151 INFO:tasks.rados.rados.0.smithi162.stdout:2837: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:13:20.151 INFO:tasks.rados.rados.0.smithi162.stdout:2837: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:13:20.154 INFO:tasks.rados.rados.0.smithi162.stdout:2837: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:13:20.154 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1385 (ObjNum 961 snap 269 seq_num 961) dirty exists 2024-09-16T15:13:20.196 INFO:tasks.rados.rados.0.smithi162.stdout:2838: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:13:20.199 INFO:tasks.rados.rados.0.smithi162.stdout:2838: finishing copy_from to smithi16234619-32 2024-09-16T15:13:20.199 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1424 (ObjNum 975 snap 275 seq_num 975) dirty exists 2024-09-16T15:13:20.202 INFO:tasks.rados.rados.0.smithi162.stdout:2839: finishing copy_from racing read to smithi16234619-13 2024-09-16T15:13:20.206 INFO:tasks.rados.rados.0.smithi162.stdout:2839: finishing copy_from to smithi16234619-13 2024-09-16T15:13:20.206 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1288 (ObjNum 999 snap 282 seq_num 999) dirty exists 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2836: done (4 left) 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2837: done (3 left) 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2838: done (2 left) 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2839: done (1 left) 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2840: done (0 left) 2024-09-16T15:13:21.667 INFO:tasks.rados.rados.0.smithi162.stdout:2841: delete oid 15 current snap is 283 2024-09-16T15:13:21.672 INFO:tasks.rados.rados.0.smithi162.stdout:2841: done (0 left) 2024-09-16T15:13:21.672 INFO:tasks.rados.rados.0.smithi162.stdout:2842: snap_create 2024-09-16T15:13:22.679 INFO:tasks.rados.rados.0.smithi162.stdout:2842: done (0 left) 2024-09-16T15:13:22.679 INFO:tasks.rados.rados.0.smithi162.stdout:2843: rollback oid 46 current snap is 284 2024-09-16T15:13:22.679 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 249 2024-09-16T15:13:22.679 INFO:tasks.rados.rados.0.smithi162.stdout:2844: delete oid 20 current snap is 284 2024-09-16T15:13:22.684 INFO:tasks.rados.rados.0.smithi162.stdout:2844: done (1 left) 2024-09-16T15:13:22.684 INFO:tasks.rados.rados.0.smithi162.stdout:2845: write oid 33 current snap is 284 2024-09-16T15:13:22.684 INFO:tasks.rados.rados.0.smithi162.stdout:2845: seq_num 1003 ranges {457421=716990,1676851=411354,2677497=179924} 2024-09-16T15:13:22.690 INFO:tasks.rados.rados.0.smithi162.stdout:2845: writing smithi16234619-33 from 457421 to 1174411 tid 1 2024-09-16T15:13:22.695 INFO:tasks.rados.rados.0.smithi162.stdout:2845: writing smithi16234619-33 from 1676851 to 2088205 tid 2 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:2845: writing smithi16234619-33 from 2677497 to 2857421 tid 3 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:2846: snap_create 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:2843: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:2843: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:2843: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:13:22.700 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1196 (ObjNum 856 snap 227 seq_num 856) dirty exists 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: finishing write tid 1 to smithi16234619-33 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: finishing write tid 2 to smithi16234619-33 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: finishing write tid 3 to smithi16234619-33 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: finishing write tid 4 to smithi16234619-33 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: finishing write tid 5 to smithi16234619-33 2024-09-16T15:13:22.721 INFO:tasks.rados.rados.0.smithi162.stdout:2845: oid 33 updating version 0 to 830 2024-09-16T15:13:22.722 INFO:tasks.rados.rados.0.smithi162.stdout:2845: oid 33 version 830 is already newer than 829 2024-09-16T15:13:22.722 INFO:tasks.rados.rados.0.smithi162.stdout:2845: oid 33 version 830 is already newer than 828 2024-09-16T15:13:22.722 INFO:tasks.rados.rados.0.smithi162.stdout:2845: oid 33 version 830 is already newer than 827 2024-09-16T15:13:22.722 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 830 (ObjNum 1003 snap 284 seq_num 1003) dirty exists 2024-09-16T15:13:22.722 INFO:tasks.rados.rados.0.smithi162.stdout:2845: left oid 33 (ObjNum 1003 snap 284 seq_num 1003) 2024-09-16T15:13:23.677 INFO:tasks.rados.rados.0.smithi162.stdout:2843: done (2 left) 2024-09-16T15:13:23.677 INFO:tasks.rados.rados.0.smithi162.stdout:2845: done (1 left) 2024-09-16T15:13:23.677 INFO:tasks.rados.rados.0.smithi162.stdout:2846: done (0 left) 2024-09-16T15:13:23.677 INFO:tasks.rados.rados.0.smithi162.stdout:2847: read oid 30 snap -1 2024-09-16T15:13:23.677 INFO:tasks.rados.rados.0.smithi162.stdout:2847: expect deleted 2024-09-16T15:13:23.678 INFO:tasks.rados.rados.0.smithi162.stdout:2848: delete oid 32 current snap is 285 2024-09-16T15:13:23.681 INFO:tasks.rados.rados.0.smithi162.stdout:2847: done (1 left) 2024-09-16T15:13:23.681 INFO:tasks.rados.rados.0.smithi162.stdout:2848: done (0 left) 2024-09-16T15:13:23.681 INFO:tasks.rados.rados.0.smithi162.stdout:2849: write (excl) oid 47 current snap is 285 2024-09-16T15:13:23.681 INFO:tasks.rados.rados.0.smithi162.stdout:2849: seq_num 1005 ranges {491035=583841,1510168=698670,2732343=158692} 2024-09-16T15:13:23.686 INFO:tasks.rados.rados.0.smithi162.stdout:2849: writing smithi16234619-47 from 491035 to 1074876 tid 1 2024-09-16T15:13:23.694 INFO:tasks.rados.rados.0.smithi162.stdout:2849: writing smithi16234619-47 from 1510168 to 2208838 tid 2 2024-09-16T15:13:23.698 INFO:tasks.rados.rados.0.smithi162.stdout:2849: writing smithi16234619-47 from 2732343 to 2891035 tid 3 2024-09-16T15:13:23.698 INFO:tasks.rados.rados.0.smithi162.stdout:2850: snap_create 2024-09-16T15:13:23.734 INFO:tasks.rados.rados.0.smithi162.stdout:2849: finishing write tid 1 to smithi16234619-47 2024-09-16T15:13:23.735 INFO:tasks.rados.rados.0.smithi162.stdout:2849: finishing write tid 2 to smithi16234619-47 2024-09-16T15:13:23.735 INFO:tasks.rados.rados.0.smithi162.stdout:2849: finishing write tid 3 to smithi16234619-47 2024-09-16T15:13:23.739 INFO:tasks.rados.rados.0.smithi162.stdout:2849: finishing write tid 4 to smithi16234619-47 2024-09-16T15:13:23.739 INFO:tasks.rados.rados.0.smithi162.stdout:2849: finishing write tid 5 to smithi16234619-47 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:2849: oid 47 updating version 0 to 1125 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:2849: oid 47 version 1125 is already newer than 1124 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:2849: oid 47 version 1125 is already newer than 1123 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:2849: oid 47 updating version 1125 to 1126 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1126 (ObjNum 1005 snap 285 seq_num 1005) dirty exists 2024-09-16T15:13:23.740 INFO:tasks.rados.rados.0.smithi162.stdout:2849: left oid 47 (ObjNum 1005 snap 285 seq_num 1005) 2024-09-16T15:13:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:2849: done (1 left) 2024-09-16T15:13:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:2850: done (0 left) 2024-09-16T15:13:24.400 INFO:tasks.rados.rados.0.smithi162.stdout:2851: snap_create 2024-09-16T15:13:25.408 INFO:tasks.rados.rados.0.smithi162.stdout:2851: done (0 left) 2024-09-16T15:13:25.408 INFO:tasks.rados.rados.0.smithi162.stdout:2852: snap_create 2024-09-16T15:13:26.417 INFO:tasks.rados.rados.0.smithi162.stdout:2852: done (0 left) 2024-09-16T15:13:26.417 INFO:tasks.rados.rados.0.smithi162.stdout:2853: write (excl) oid 11 current snap is 288 2024-09-16T15:13:26.417 INFO:tasks.rados.rados.0.smithi162.stdout:2853: seq_num 1006 ranges {507842=659090,1568650=558681,2759766=148076} 2024-09-16T15:13:26.423 INFO:tasks.rados.rados.0.smithi162.stdout:2853: writing smithi16234619-11 from 507842 to 1166932 tid 1 2024-09-16T15:13:26.428 INFO:tasks.rados.rados.0.smithi162.stdout:2853: writing smithi16234619-11 from 1568650 to 2127331 tid 2 2024-09-16T15:13:26.433 INFO:tasks.rados.rados.0.smithi162.stdout:2853: writing smithi16234619-11 from 2759766 to 2907842 tid 3 2024-09-16T15:13:26.433 INFO:tasks.rados.rados.0.smithi162.stdout:2854: copy_from oid 34 from oid 20 current snap is 288 2024-09-16T15:13:26.433 INFO:tasks.rados.rados.0.smithi162.stdout:2855: snap_create 2024-09-16T15:13:26.433 INFO:tasks.rados.rados.0.smithi162.stdout:2854: finishing copy_from to smithi16234619-34 2024-09-16T15:13:26.434 INFO:tasks.rados.rados.0.smithi162.stdout:2854: got expected ENOENT (src dne) 2024-09-16T15:13:26.434 INFO:tasks.rados.rados.0.smithi162.stdout:2854: finishing copy_from racing read to smithi16234619-34 2024-09-16T15:13:26.458 INFO:tasks.rados.rados.0.smithi162.stdout:2853: finishing write tid 1 to smithi16234619-11 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: finishing write tid 2 to smithi16234619-11 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: finishing write tid 3 to smithi16234619-11 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: finishing write tid 4 to smithi16234619-11 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: finishing write tid 5 to smithi16234619-11 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: oid 11 updating version 0 to 1274 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: oid 11 updating version 1274 to 1275 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: oid 11 updating version 1275 to 1276 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: oid 11 updating version 1276 to 1277 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1277 (ObjNum 1006 snap 288 seq_num 1006) dirty exists 2024-09-16T15:13:26.459 INFO:tasks.rados.rados.0.smithi162.stdout:2853: left oid 11 (ObjNum 1006 snap 288 seq_num 1006) 2024-09-16T15:13:27.421 INFO:tasks.rados.rados.0.smithi162.stdout:2853: done (2 left) 2024-09-16T15:13:27.421 INFO:tasks.rados.rados.0.smithi162.stdout:2854: done (1 left) 2024-09-16T15:13:27.421 INFO:tasks.rados.rados.0.smithi162.stdout:2855: done (0 left) 2024-09-16T15:13:27.422 INFO:tasks.rados.rados.0.smithi162.stdout:2856: snap_remove snap 281 2024-09-16T15:13:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:2856: done (0 left) 2024-09-16T15:13:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:2857: write (excl) oid 5 current snap is 289 2024-09-16T15:13:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:2857: seq_num 1007 ranges {524649=734339,1910779=702339} 2024-09-16T15:13:28.617 INFO:tasks.rados.rados.0.smithi162.stdout:2857: writing smithi16234619-5 from 524649 to 1258988 tid 1 2024-09-16T15:13:28.625 INFO:tasks.rados.rados.0.smithi162.stdout:2857: writing smithi16234619-5 from 1910779 to 2613118 tid 2 2024-09-16T15:13:28.626 INFO:tasks.rados.rados.0.smithi162.stdout:2858: snap_create 2024-09-16T15:13:28.652 INFO:tasks.rados.rados.0.smithi162.stdout:2857: finishing write tid 1 to smithi16234619-5 2024-09-16T15:13:28.653 INFO:tasks.rados.rados.0.smithi162.stdout:2857: finishing write tid 2 to smithi16234619-5 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: finishing write tid 3 to smithi16234619-5 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: finishing write tid 4 to smithi16234619-5 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: oid 5 updating version 0 to 1392 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: oid 5 version 1392 is already newer than 1390 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: oid 5 version 1392 is already newer than 1391 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1392 (ObjNum 1007 snap 289 seq_num 1007) dirty exists 2024-09-16T15:13:28.657 INFO:tasks.rados.rados.0.smithi162.stdout:2857: left oid 5 (ObjNum 1007 snap 289 seq_num 1007) 2024-09-16T15:13:29.036 INFO:tasks.rados.rados.0.smithi162.stdout:2857: done (1 left) 2024-09-16T15:13:29.037 INFO:tasks.rados.rados.0.smithi162.stdout:2858: done (0 left) 2024-09-16T15:13:29.037 INFO:tasks.rados.rados.0.smithi162.stdout:2859: read oid 27 snap -1 2024-09-16T15:13:29.037 INFO:tasks.rados.rados.0.smithi162.stdout:2859: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:29.037 INFO:tasks.rados.rados.0.smithi162.stdout:2860: snap_remove snap 259 2024-09-16T15:13:30.043 INFO:tasks.rados.rados.0.smithi162.stdout:2860: done (1 left) 2024-09-16T15:13:30.043 INFO:tasks.rados.rados.0.smithi162.stdout:2861: read oid 7 snap -1 2024-09-16T15:13:30.043 INFO:tasks.rados.rados.0.smithi162.stdout:2861: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:30.043 INFO:tasks.rados.rados.0.smithi162.stdout:2862: write (excl) oid 19 current snap is 290 2024-09-16T15:13:30.043 INFO:tasks.rados.rados.0.smithi162.stdout:2862: seq_num 1008 ranges {541456=409588,1569261=562350,2814612=126844} 2024-09-16T15:13:30.048 INFO:tasks.rados.rados.0.smithi162.stdout:2862: writing smithi16234619-19 from 541456 to 951044 tid 1 2024-09-16T15:13:30.055 INFO:tasks.rados.rados.0.smithi162.stdout:2862: writing smithi16234619-19 from 1569261 to 2131611 tid 2 2024-09-16T15:13:30.059 INFO:tasks.rados.rados.0.smithi162.stdout:2862: writing smithi16234619-19 from 2814612 to 2941456 tid 3 2024-09-16T15:13:30.059 INFO:tasks.rados.rados.0.smithi162.stdout:2863: read oid 32 snap 289 2024-09-16T15:13:30.059 INFO:tasks.rados.rados.0.smithi162.stdout:2863: expect deleted 2024-09-16T15:13:30.059 INFO:tasks.rados.rados.0.smithi162.stdout:2864: snap_remove snap 283 2024-09-16T15:13:31.048 INFO:tasks.rados.rados.0.smithi162.stdout:2864: done (4 left) 2024-09-16T15:13:31.048 INFO:tasks.rados.rados.0.smithi162.stdout:2865: write oid 38 current snap is 290 2024-09-16T15:13:31.048 INFO:tasks.rados.rados.0.smithi162.stdout:2865: seq_num 1009 ranges {558263=484837,1627743=706008} 2024-09-16T15:13:31.053 INFO:tasks.rados.rados.0.smithi162.stdout:2865: writing smithi16234619-38 from 558263 to 1043100 tid 1 2024-09-16T15:13:31.060 INFO:tasks.rados.rados.0.smithi162.stdout:2865: writing smithi16234619-38 from 1627743 to 2333751 tid 2 2024-09-16T15:13:31.060 INFO:tasks.rados.rados.0.smithi162.stdout:2866: snap_remove snap 269 2024-09-16T15:13:32.048 INFO:tasks.rados.rados.0.smithi162.stdout:2866: done (5 left) 2024-09-16T15:13:32.048 INFO:tasks.rados.rados.0.smithi162.stdout:2867: copy_from oid 47 from oid 18 current snap is 290 2024-09-16T15:13:32.049 INFO:tasks.rados.rados.0.smithi162.stdout:2859: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:32.200 INFO:tasks.rados.rados.0.smithi162.stdout:2861: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:32.361 INFO:tasks.rados.rados.0.smithi162.stdout:2862: finishing write tid 1 to smithi16234619-19 2024-09-16T15:13:32.361 INFO:tasks.rados.rados.0.smithi162.stdout:2862: finishing write tid 2 to smithi16234619-19 2024-09-16T15:13:32.361 INFO:tasks.rados.rados.0.smithi162.stdout:2862: finishing write tid 3 to smithi16234619-19 2024-09-16T15:13:32.361 INFO:tasks.rados.rados.0.smithi162.stdout:2862: finishing write tid 4 to smithi16234619-19 2024-09-16T15:13:32.361 INFO:tasks.rados.rados.0.smithi162.stdout:2862: finishing write tid 5 to smithi16234619-19 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: oid 19 updating version 0 to 1292 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: oid 19 updating version 1292 to 1295 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: oid 19 version 1295 is already newer than 1294 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: oid 19 version 1295 is already newer than 1293 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1295 (ObjNum 1008 snap 290 seq_num 1008) dirty exists 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: left oid 19 (ObjNum 1008 snap 290 seq_num 1008) 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2865: finishing write tid 1 to smithi16234619-38 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2859: done (5 left) 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2861: done (4 left) 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2862: done (3 left) 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2863: done (2 left) 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2868: snap_create 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2865: finishing write tid 2 to smithi16234619-38 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2865: finishing write tid 3 to smithi16234619-38 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2865: finishing write tid 4 to smithi16234619-38 2024-09-16T15:13:32.362 INFO:tasks.rados.rados.0.smithi162.stdout:2865: oid 38 updating version 0 to 1279 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:2865: oid 38 updating version 1279 to 1281 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:2865: oid 38 version 1281 is already newer than 1280 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1281 (ObjNum 1009 snap 290 seq_num 1009) dirty exists 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:2865: left oid 38 (ObjNum 1009 snap 290 seq_num 1009) 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:2867: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:2867: finishing copy_from to smithi16234619-47 2024-09-16T15:13:32.363 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 925 (ObjNum 983 snap 278 seq_num 983) dirty exists 2024-09-16T15:13:33.069 INFO:tasks.rados.rados.0.smithi162.stdout:2865: done (2 left) 2024-09-16T15:13:33.069 INFO:tasks.rados.rados.0.smithi162.stdout:2867: done (1 left) 2024-09-16T15:13:33.069 INFO:tasks.rados.rados.0.smithi162.stdout:2868: done (0 left) 2024-09-16T15:13:33.069 INFO:tasks.rados.rados.0.smithi162.stdout:2869: rollback oid 49 current snap is 291 2024-09-16T15:13:33.070 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 266 2024-09-16T15:13:33.070 INFO:tasks.rados.rados.0.smithi162.stdout:2870: snap_remove snap 290 2024-09-16T15:13:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:2870: done (1 left) 2024-09-16T15:13:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:2871: read oid 20 snap -1 2024-09-16T15:13:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:2871: expect deleted 2024-09-16T15:13:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:2872: copy_from oid 14 from oid 4 current snap is 291 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2869: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2869: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2869: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 818 (ObjNum 941 snap 266 seq_num 941) dirty exists 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2869: done (2 left) 2024-09-16T15:13:34.040 INFO:tasks.rados.rados.0.smithi162.stdout:2873: snap_create 2024-09-16T15:13:34.045 INFO:tasks.rados.rados.0.smithi162.stdout:2872: finishing copy_from to smithi16234619-14 2024-09-16T15:13:34.045 INFO:tasks.rados.rados.0.smithi162.stdout:2872: got expected ENOENT (src dne) 2024-09-16T15:13:34.046 INFO:tasks.rados.rados.0.smithi162.stdout:2872: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2871: done (2 left) 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2872: done (1 left) 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2873: done (0 left) 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2874: rollback oid 29 current snap is 292 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 273 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2875: write (excl) oid 13 current snap is 292 2024-09-16T15:13:35.040 INFO:tasks.rados.rados.0.smithi162.stdout:2875: seq_num 1010 ranges {575070=560086,1686225=566019,2869458=105612} 2024-09-16T15:13:35.046 INFO:tasks.rados.rados.0.smithi162.stdout:2875: writing smithi16234619-13 from 575070 to 1135156 tid 1 2024-09-16T15:13:35.052 INFO:tasks.rados.rados.0.smithi162.stdout:2875: writing smithi16234619-13 from 1686225 to 2252244 tid 2 2024-09-16T15:13:35.056 INFO:tasks.rados.rados.0.smithi162.stdout:2875: writing smithi16234619-13 from 2869458 to 2975070 tid 3 2024-09-16T15:13:35.057 INFO:tasks.rados.rados.0.smithi162.stdout:2876: snap_create 2024-09-16T15:13:35.060 INFO:tasks.rados.rados.0.smithi162.stdout:2874: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:13:35.060 INFO:tasks.rados.rados.0.smithi162.stdout:2874: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:13:35.076 INFO:tasks.rados.rados.0.smithi162.stdout:2874: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:13:35.076 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1410 (ObjNum 965 snap 272 seq_num 965) dirty exists 2024-09-16T15:13:35.077 INFO:tasks.rados.rados.0.smithi162.stdout:2875: finishing write tid 1 to smithi16234619-13 2024-09-16T15:13:35.082 INFO:tasks.rados.rados.0.smithi162.stdout:2875: finishing write tid 2 to smithi16234619-13 2024-09-16T15:13:35.085 INFO:tasks.rados.rados.0.smithi162.stdout:2875: finishing write tid 3 to smithi16234619-13 2024-09-16T15:13:35.085 INFO:tasks.rados.rados.0.smithi162.stdout:2875: finishing write tid 4 to smithi16234619-13 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: finishing write tid 5 to smithi16234619-13 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: oid 13 updating version 0 to 1414 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: oid 13 version 1414 is already newer than 1413 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: oid 13 version 1414 is already newer than 1412 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: oid 13 updating version 1414 to 1415 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1415 (ObjNum 1010 snap 292 seq_num 1010) dirty exists 2024-09-16T15:13:35.086 INFO:tasks.rados.rados.0.smithi162.stdout:2875: left oid 13 (ObjNum 1010 snap 292 seq_num 1010) 2024-09-16T15:13:36.042 INFO:tasks.rados.rados.0.smithi162.stdout:2874: done (2 left) 2024-09-16T15:13:36.042 INFO:tasks.rados.rados.0.smithi162.stdout:2875: done (1 left) 2024-09-16T15:13:36.042 INFO:tasks.rados.rados.0.smithi162.stdout:2876: done (0 left) 2024-09-16T15:13:36.042 INFO:tasks.rados.rados.0.smithi162.stdout:2877: copy_from oid 13 from oid 24 current snap is 293 2024-09-16T15:13:36.043 INFO:tasks.rados.rados.0.smithi162.stdout:2878: copy_from oid 32 from oid 22 current snap is 293 2024-09-16T15:13:36.043 INFO:tasks.rados.rados.0.smithi162.stdout:2879: snap_remove snap 291 2024-09-16T15:13:37.052 INFO:tasks.rados.rados.0.smithi162.stdout:2879: done (2 left) 2024-09-16T15:13:37.052 INFO:tasks.rados.rados.0.smithi162.stdout:2880: write (excl) oid 11 current snap is 293 2024-09-16T15:13:37.052 INFO:tasks.rados.rados.0.smithi162.stdout:2880: seq_num 1011 ranges {591877=635335,1628354=709677} 2024-09-16T15:13:37.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: writing smithi16234619-11 from 591877 to 1227212 tid 1 2024-09-16T15:13:37.065 INFO:tasks.rados.rados.0.smithi162.stdout:2880: writing smithi16234619-11 from 1628354 to 2338031 tid 2 2024-09-16T15:13:37.065 INFO:tasks.rados.rados.0.smithi162.stdout:2881: write (excl) oid 14 current snap is 293 2024-09-16T15:13:37.065 INFO:tasks.rados.rados.0.smithi162.stdout:2878: finishing copy_from to smithi16234619-32 2024-09-16T15:13:37.065 INFO:tasks.rados.rados.0.smithi162.stdout:2878: got expected ENOENT (src dne) 2024-09-16T15:13:37.066 INFO:tasks.rados.rados.0.smithi162.stdout:2878: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:13:37.066 INFO:tasks.rados.rados.0.smithi162.stdout:2877: finishing copy_from racing read to smithi16234619-13 2024-09-16T15:13:37.066 INFO:tasks.rados.rados.0.smithi162.stdout:2877: finishing copy_from to smithi16234619-13 2024-09-16T15:13:37.066 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1348 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:13:37.066 INFO:tasks.rados.rados.0.smithi162.stdout:2881: seq_num 1012 ranges {608684=426937,1803189=569688,2924304=84380} 2024-09-16T15:13:37.070 INFO:tasks.rados.rados.0.smithi162.stdout:2881: writing smithi16234619-14 from 608684 to 1035621 tid 1 2024-09-16T15:13:37.077 INFO:tasks.rados.rados.0.smithi162.stdout:2881: writing smithi16234619-14 from 1803189 to 2372877 tid 2 2024-09-16T15:13:37.081 INFO:tasks.rados.rados.0.smithi162.stdout:2881: writing smithi16234619-14 from 2924304 to 3008684 tid 3 2024-09-16T15:13:37.081 INFO:tasks.rados.rados.0.smithi162.stdout:2877: done (3 left) 2024-09-16T15:13:37.081 INFO:tasks.rados.rados.0.smithi162.stdout:2878: done (2 left) 2024-09-16T15:13:37.081 INFO:tasks.rados.rados.0.smithi162.stdout:2882: write (excl) oid 1 current snap is 293 2024-09-16T15:13:37.082 INFO:tasks.rados.rados.0.smithi162.stdout:2882: seq_num 1013 ranges {625491=502186,1861671=713346} 2024-09-16T15:13:37.087 INFO:tasks.rados.rados.0.smithi162.stdout:2882: writing smithi16234619-1 from 625491 to 1127677 tid 1 2024-09-16T15:13:37.095 INFO:tasks.rados.rados.0.smithi162.stdout:2882: writing smithi16234619-1 from 1861671 to 2575017 tid 2 2024-09-16T15:13:37.095 INFO:tasks.rados.rados.0.smithi162.stdout:2883: snap_remove snap 292 2024-09-16T15:13:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:2883: done (3 left) 2024-09-16T15:13:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:2884: snap_create 2024-09-16T15:13:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:2880: finishing write tid 1 to smithi16234619-11 2024-09-16T15:13:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:2880: finishing write tid 2 to smithi16234619-11 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: finishing write tid 3 to smithi16234619-11 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: finishing write tid 4 to smithi16234619-11 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: oid 11 updating version 0 to 1280 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: oid 11 version 1280 is already newer than 1278 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: oid 11 version 1280 is already newer than 1279 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1280 (ObjNum 1011 snap 293 seq_num 1011) dirty exists 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2880: left oid 11 (ObjNum 1011 snap 293 seq_num 1011) 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: finishing write tid 1 to smithi16234619-14 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: finishing write tid 2 to smithi16234619-14 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: finishing write tid 3 to smithi16234619-14 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: finishing write tid 4 to smithi16234619-14 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2882: finishing write tid 1 to smithi16234619-1 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2882: finishing write tid 2 to smithi16234619-1 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: finishing write tid 5 to smithi16234619-14 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: oid 14 updating version 0 to 1322 2024-09-16T15:13:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:2881: oid 14 version 1322 is already newer than 1320 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2881: oid 14 version 1322 is already newer than 1319 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2881: oid 14 version 1322 is already newer than 1321 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1322 (ObjNum 1012 snap 293 seq_num 1012) dirty exists 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2881: left oid 14 (ObjNum 1012 snap 293 seq_num 1012) 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: finishing write tid 3 to smithi16234619-1 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: finishing write tid 4 to smithi16234619-1 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: oid 1 updating version 0 to 1282 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: oid 1 version 1282 is already newer than 1281 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: oid 1 updating version 1282 to 1283 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1283 (ObjNum 1013 snap 293 seq_num 1013) dirty exists 2024-09-16T15:13:38.059 INFO:tasks.rados.rados.0.smithi162.stdout:2882: left oid 1 (ObjNum 1013 snap 293 seq_num 1013) 2024-09-16T15:13:39.039 INFO:tasks.rados.rados.0.smithi162.stdout:2880: done (3 left) 2024-09-16T15:13:39.039 INFO:tasks.rados.rados.0.smithi162.stdout:2881: done (2 left) 2024-09-16T15:13:39.039 INFO:tasks.rados.rados.0.smithi162.stdout:2882: done (1 left) 2024-09-16T15:13:39.039 INFO:tasks.rados.rados.0.smithi162.stdout:2884: done (0 left) 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:2885: rollback oid 18 current snap is 294 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 278 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:2886: rollback oid 14 current snap is 294 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 14 to 293 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:2887: write (excl) oid 7 current snap is 294 2024-09-16T15:13:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:2887: seq_num 1014 ranges {642298=577435,1920153=573357,2979150=63148} 2024-09-16T15:13:39.046 INFO:tasks.rados.rados.0.smithi162.stdout:2887: writing smithi16234619-7 from 642298 to 1219733 tid 1 2024-09-16T15:13:39.052 INFO:tasks.rados.rados.0.smithi162.stdout:2887: writing smithi16234619-7 from 1920153 to 2493510 tid 2 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2887: writing smithi16234619-7 from 2979150 to 3042298 tid 3 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2888: write (excl) oid 41 current snap is 294 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2886: finishing rollback tid 0 to smithi16234619-14 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2885: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2886: finishing rollback tid 1 to smithi16234619-14 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2886: finishing rollback tid 2 to smithi16234619-14 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1328 (ObjNum 1012 snap 293 seq_num 1012) dirty exists 2024-09-16T15:13:39.056 INFO:tasks.rados.rados.0.smithi162.stdout:2888: seq_num 1015 ranges {659105=652684,1862282=717015} 2024-09-16T15:13:39.063 INFO:tasks.rados.rados.0.smithi162.stdout:2888: writing smithi16234619-41 from 659105 to 1311789 tid 1 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2888: writing smithi16234619-41 from 1862282 to 2579297 tid 2 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2886: done (3 left) 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2889: read oid 12 snap -1 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2889: expect deleted 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2890: copy_from oid 39 from oid 36 current snap is 294 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2891: copy_from oid 50 from oid 24 current snap is 294 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2892: read oid 20 snap 277 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2892: expect (ObjNum 953 snap 268 seq_num 953) 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2893: copy_from oid 23 from oid 6 current snap is 294 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2894: read oid 26 snap -1 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2894: expect deleted 2024-09-16T15:13:39.071 INFO:tasks.rados.rados.0.smithi162.stdout:2895: delete oid 34 current snap is 294 2024-09-16T15:13:39.077 INFO:tasks.rados.rados.0.smithi162.stdout:2892: expect (ObjNum 953 snap 268 seq_num 953) 2024-09-16T15:13:39.158 INFO:tasks.rados.rados.0.smithi162.stdout:2893: finishing copy_from to smithi16234619-23 2024-09-16T15:13:39.158 INFO:tasks.rados.rados.0.smithi162.stdout:2893: got expected ENOENT (src dne) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2893: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2890: finishing copy_from to smithi16234619-39 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2890: got expected ENOENT (src dne) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2890: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2885: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2885: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 929 (ObjNum 983 snap 278 seq_num 983) dirty exists 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2887: finishing write tid 1 to smithi16234619-7 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2885: done (9 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2889: done (8 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2890: done (7 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2892: done (6 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2893: done (5 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2894: done (4 left) 2024-09-16T15:13:39.159 INFO:tasks.rados.rados.0.smithi162.stdout:2895: done (3 left) 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2896: snap_create 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: finishing write tid 2 to smithi16234619-7 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: finishing write tid 3 to smithi16234619-7 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: finishing write tid 4 to smithi16234619-7 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: finishing write tid 5 to smithi16234619-7 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: oid 7 updating version 0 to 930 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: oid 7 updating version 930 to 932 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: oid 7 updating version 932 to 933 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: oid 7 version 933 is already newer than 931 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 933 (ObjNum 1014 snap 294 seq_num 1014) dirty exists 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2887: left oid 7 (ObjNum 1014 snap 294 seq_num 1014) 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2888: finishing write tid 1 to smithi16234619-41 2024-09-16T15:13:39.160 INFO:tasks.rados.rados.0.smithi162.stdout:2888: finishing write tid 2 to smithi16234619-41 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: finishing write tid 3 to smithi16234619-41 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: finishing write tid 4 to smithi16234619-41 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: oid 41 updating version 0 to 819 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: oid 41 updating version 819 to 820 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: oid 41 updating version 820 to 821 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 821 (ObjNum 1015 snap 294 seq_num 1015) dirty exists 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2888: left oid 41 (ObjNum 1015 snap 294 seq_num 1015) 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2891: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:2891: finishing copy_from to smithi16234619-50 2024-09-16T15:13:39.161 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1348 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2887: done (3 left) 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2888: done (2 left) 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2891: done (1 left) 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2896: done (0 left) 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2897: write oid 5 current snap is 295 2024-09-16T15:13:40.070 INFO:tasks.rados.rados.0.smithi162.stdout:2897: seq_num 1017 ranges {692719=403182,1579246=720684,2828713=264006} 2024-09-16T15:13:40.076 INFO:tasks.rados.rados.0.smithi162.stdout:2897: writing smithi16234619-5 from 692719 to 1095901 tid 1 2024-09-16T15:13:40.082 INFO:tasks.rados.rados.0.smithi162.stdout:2897: writing smithi16234619-5 from 1579246 to 2299930 tid 2 2024-09-16T15:13:40.087 INFO:tasks.rados.rados.0.smithi162.stdout:2897: writing smithi16234619-5 from 2828713 to 3092719 tid 3 2024-09-16T15:13:40.087 INFO:tasks.rados.rados.0.smithi162.stdout:2898: read oid 1 snap -1 2024-09-16T15:13:40.087 INFO:tasks.rados.rados.0.smithi162.stdout:2898: expect (ObjNum 1013 snap 293 seq_num 1013) 2024-09-16T15:13:40.087 INFO:tasks.rados.rados.0.smithi162.stdout:2899: write oid 27 current snap is 295 2024-09-16T15:13:40.087 INFO:tasks.rados.rados.0.smithi162.stdout:2899: seq_num 1018 ranges {709526=478431,1637728=464342,2856136=253390} 2024-09-16T15:13:40.094 INFO:tasks.rados.rados.0.smithi162.stdout:2899: writing smithi16234619-27 from 709526 to 1187957 tid 1 2024-09-16T15:13:40.099 INFO:tasks.rados.rados.0.smithi162.stdout:2899: writing smithi16234619-27 from 1637728 to 2102070 tid 2 2024-09-16T15:13:40.105 INFO:tasks.rados.rados.0.smithi162.stdout:2899: writing smithi16234619-27 from 2856136 to 3109526 tid 3 2024-09-16T15:13:40.105 INFO:tasks.rados.rados.0.smithi162.stdout:2900: read oid 15 snap 230 2024-09-16T15:13:40.105 INFO:tasks.rados.rados.0.smithi162.stdout:2900: expect deleted 2024-09-16T15:13:40.105 INFO:tasks.rados.rados.0.smithi162.stdout:2901: write (excl) oid 10 current snap is 295 2024-09-16T15:13:40.105 INFO:tasks.rados.rados.0.smithi162.stdout:2898: expect (ObjNum 1013 snap 293 seq_num 1013) 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: finishing write tid 1 to smithi16234619-5 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: finishing write tid 2 to smithi16234619-5 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: finishing write tid 3 to smithi16234619-5 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: finishing write tid 4 to smithi16234619-5 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: finishing write tid 5 to smithi16234619-5 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: oid 5 updating version 0 to 1425 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: oid 5 updating version 1425 to 1428 2024-09-16T15:13:40.206 INFO:tasks.rados.rados.0.smithi162.stdout:2897: oid 5 version 1428 is already newer than 1426 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2897: oid 5 version 1428 is already newer than 1427 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1428 (ObjNum 1017 snap 295 seq_num 1017) dirty exists 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2897: left oid 5 (ObjNum 1017 snap 295 seq_num 1017) 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: finishing write tid 1 to smithi16234619-27 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: finishing write tid 2 to smithi16234619-27 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: finishing write tid 3 to smithi16234619-27 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: finishing write tid 4 to smithi16234619-27 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: finishing write tid 5 to smithi16234619-27 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: oid 27 updating version 0 to 1284 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: oid 27 updating version 1284 to 1285 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: oid 27 updating version 1285 to 1286 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: oid 27 updating version 1286 to 1287 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 1287 (ObjNum 1018 snap 295 seq_num 1018) dirty exists 2024-09-16T15:13:40.207 INFO:tasks.rados.rados.0.smithi162.stdout:2899: left oid 27 (ObjNum 1018 snap 295 seq_num 1018) 2024-09-16T15:13:40.208 INFO:tasks.rados.rados.0.smithi162.stdout:2901: seq_num 1019 ranges {726333=670033,2096210=724353} 2024-09-16T15:13:40.213 INFO:tasks.rados.rados.0.smithi162.stdout:2901: writing smithi16234619-10 from 726333 to 1396366 tid 1 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2901: writing smithi16234619-10 from 2096210 to 2820563 tid 2 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2897: done (4 left) 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2898: done (3 left) 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2899: done (2 left) 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2900: done (1 left) 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2902: read oid 13 snap -1 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2902: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2903: write (excl) oid 42 current snap is 295 2024-09-16T15:13:40.223 INFO:tasks.rados.rados.0.smithi162.stdout:2903: seq_num 1020 ranges {743140=745282,2154692=468011} 2024-09-16T15:13:40.232 INFO:tasks.rados.rados.0.smithi162.stdout:2903: writing smithi16234619-42 from 743140 to 1488422 tid 1 2024-09-16T15:13:40.239 INFO:tasks.rados.rados.0.smithi162.stdout:2903: writing smithi16234619-42 from 2154692 to 2622703 tid 2 2024-09-16T15:13:40.239 INFO:tasks.rados.rados.0.smithi162.stdout:2904: copy_from oid 18 from oid 16 current snap is 295 2024-09-16T15:13:40.239 INFO:tasks.rados.rados.0.smithi162.stdout:2902: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:13:40.410 INFO:tasks.rados.rados.0.smithi162.stdout:2901: finishing write tid 1 to smithi16234619-10 2024-09-16T15:13:40.410 INFO:tasks.rados.rados.0.smithi162.stdout:2901: finishing write tid 2 to smithi16234619-10 2024-09-16T15:13:40.410 INFO:tasks.rados.rados.0.smithi162.stdout:2901: finishing write tid 3 to smithi16234619-10 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: finishing write tid 4 to smithi16234619-10 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: oid 10 updating version 0 to 1492 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: oid 10 version 1492 is already newer than 1491 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: oid 10 version 1492 is already newer than 1490 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1492 (ObjNum 1019 snap 295 seq_num 1019) dirty exists 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: left oid 10 (ObjNum 1019 snap 295 seq_num 1019) 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2904: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2903: finishing write tid 1 to smithi16234619-42 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2903: finishing write tid 2 to smithi16234619-42 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2903: finishing write tid 3 to smithi16234619-42 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2901: done (3 left) 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2902: done (2 left) 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2905: read oid 8 snap 273 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2905: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:13:40.411 INFO:tasks.rados.rados.0.smithi162.stdout:2906: read oid 24 snap -1 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2906: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2907: rollback oid 7 current snap is 295 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 279 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: finishing write tid 4 to smithi16234619-42 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: oid 42 updating version 0 to 1431 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: oid 42 version 1431 is already newer than 1430 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: oid 42 version 1431 is already newer than 1429 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1431 (ObjNum 1020 snap 295 seq_num 1020) dirty exists 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: left oid 42 (ObjNum 1020 snap 295 seq_num 1020) 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2904: finishing copy_from to smithi16234619-18 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1271 (ObjNum 978 snap 276 seq_num 978) dirty exists 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2903: done (4 left) 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2904: done (3 left) 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2908: copy_from oid 12 from oid 36 current snap is 295 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2909: snap_create 2024-09-16T15:13:40.412 INFO:tasks.rados.rados.0.smithi162.stdout:2908: finishing copy_from to smithi16234619-12 2024-09-16T15:13:40.413 INFO:tasks.rados.rados.0.smithi162.stdout:2908: got expected ENOENT (src dne) 2024-09-16T15:13:40.413 INFO:tasks.rados.rados.0.smithi162.stdout:2908: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:13:40.413 INFO:tasks.rados.rados.0.smithi162.stdout:2906: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:13:40.582 INFO:tasks.rados.rados.0.smithi162.stdout:2905: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:13:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:2907: finishing rollback tid 0 to smithi16234619-7 2024-09-16T15:13:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:2907: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:13:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:2907: finishing rollback tid 2 to smithi16234619-7 2024-09-16T15:13:40.665 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1274 (ObjNum 851 snap 227 seq_num 851) dirty exists 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2905: done (4 left) 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2906: done (3 left) 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2907: done (2 left) 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2908: done (1 left) 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2909: done (0 left) 2024-09-16T15:13:41.078 INFO:tasks.rados.rados.0.smithi162.stdout:2910: snap_create 2024-09-16T15:13:42.104 INFO:tasks.rados.rados.0.smithi162.stdout:2910: done (0 left) 2024-09-16T15:13:42.105 INFO:tasks.rados.rados.0.smithi162.stdout:2911: write oid 12 current snap is 297 2024-09-16T15:13:42.105 INFO:tasks.rados.rados.0.smithi162.stdout:2911: seq_num 1021 ranges {759947=420531,1813174=728022} 2024-09-16T15:13:42.111 INFO:tasks.rados.rados.0.smithi162.stdout:2911: writing smithi16234619-12 from 759947 to 1180478 tid 1 2024-09-16T15:13:42.119 INFO:tasks.rados.rados.0.smithi162.stdout:2911: writing smithi16234619-12 from 1813174 to 2541196 tid 2 2024-09-16T15:13:42.119 INFO:tasks.rados.rados.0.smithi162.stdout:2912: rollback oid 23 current snap is 297 2024-09-16T15:13:42.119 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 289 2024-09-16T15:13:42.119 INFO:tasks.rados.rados.0.smithi162.stdout:2913: write oid 29 current snap is 297 2024-09-16T15:13:42.119 INFO:tasks.rados.rados.0.smithi162.stdout:2913: seq_num 1022 ranges {776754=495780,1871656=471680,2965828=210926} 2024-09-16T15:13:42.126 INFO:tasks.rados.rados.0.smithi162.stdout:2913: writing smithi16234619-29 from 776754 to 1272534 tid 1 2024-09-16T15:13:42.133 INFO:tasks.rados.rados.0.smithi162.stdout:2913: writing smithi16234619-29 from 1871656 to 2343336 tid 2 2024-09-16T15:13:42.138 INFO:tasks.rados.rados.0.smithi162.stdout:2913: writing smithi16234619-29 from 2965828 to 3176754 tid 3 2024-09-16T15:13:42.138 INFO:tasks.rados.rados.0.smithi162.stdout:2914: copy_from oid 15 from oid 30 current snap is 297 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2912: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2912: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2912: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 603 (ObjNum 996 snap 281 seq_num 996) dirty exists 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2912: done (3 left) 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2915: rollback oid 38 current snap is 297 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 287 2024-09-16T15:13:42.139 INFO:tasks.rados.rados.0.smithi162.stdout:2916: snap_remove snap 293 2024-09-16T15:13:43.123 INFO:tasks.rados.rados.0.smithi162.stdout:2916: done (4 left) 2024-09-16T15:13:43.123 INFO:tasks.rados.rados.0.smithi162.stdout:2917: write (excl) oid 18 current snap is 297 2024-09-16T15:13:43.123 INFO:tasks.rados.rados.0.smithi162.stdout:2917: seq_num 1023 ranges {793561=571029,1813785=731691,2993251=200310} 2024-09-16T15:13:43.133 INFO:tasks.rados.rados.0.smithi162.stdout:2917: writing smithi16234619-18 from 793561 to 1364590 tid 1 2024-09-16T15:13:43.137 INFO:tasks.rados.rados.0.smithi162.stdout:2917: writing smithi16234619-18 from 1813785 to 2545476 tid 2 2024-09-16T15:13:43.140 INFO:tasks.rados.rados.0.smithi162.stdout:2917: writing smithi16234619-18 from 2993251 to 3193561 tid 3 2024-09-16T15:13:43.140 INFO:tasks.rados.rados.0.smithi162.stdout:2918: read oid 9 snap -1 2024-09-16T15:13:43.141 INFO:tasks.rados.rados.0.smithi162.stdout:2918: expect (ObjNum 979 snap 276 seq_num 979) 2024-09-16T15:13:43.141 INFO:tasks.rados.rados.0.smithi162.stdout:2919: snap_remove snap 295 2024-09-16T15:13:44.681 INFO:tasks.rados.rados.0.smithi162.stdout:2919: done (6 left) 2024-09-16T15:13:44.681 INFO:tasks.rados.rados.0.smithi162.stdout:2920: snap_create 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2914: finishing copy_from to smithi16234619-15 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2914: got expected ENOENT (src dne) 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2914: finishing copy_from racing read to smithi16234619-15 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2915: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2915: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2915: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1284 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: finishing write tid 1 to smithi16234619-12 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: finishing write tid 2 to smithi16234619-12 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: finishing write tid 3 to smithi16234619-12 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: finishing write tid 4 to smithi16234619-12 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: oid 12 updating version 0 to 1428 2024-09-16T15:13:44.682 INFO:tasks.rados.rados.0.smithi162.stdout:2911: oid 12 version 1428 is already newer than 1427 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2911: oid 12 version 1428 is already newer than 1426 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1428 (ObjNum 1021 snap 297 seq_num 1021) dirty exists 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2911: left oid 12 (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: finishing write tid 1 to smithi16234619-29 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: finishing write tid 2 to smithi16234619-29 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: finishing write tid 3 to smithi16234619-29 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: finishing write tid 4 to smithi16234619-29 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: finishing write tid 5 to smithi16234619-29 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: oid 29 updating version 0 to 1433 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: oid 29 updating version 1433 to 1434 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: oid 29 updating version 1434 to 1435 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:2913: oid 29 updating version 1435 to 1436 2024-09-16T15:13:44.683 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1436 (ObjNum 1022 snap 297 seq_num 1022) dirty exists 2024-09-16T15:13:44.684 INFO:tasks.rados.rados.0.smithi162.stdout:2913: left oid 29 (ObjNum 1022 snap 297 seq_num 1022) 2024-09-16T15:13:44.684 INFO:tasks.rados.rados.0.smithi162.stdout:2918: expect (ObjNum 979 snap 276 seq_num 979) 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: finishing write tid 1 to smithi16234619-18 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: finishing write tid 2 to smithi16234619-18 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: finishing write tid 3 to smithi16234619-18 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: finishing write tid 4 to smithi16234619-18 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: finishing write tid 5 to smithi16234619-18 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: oid 18 updating version 0 to 1276 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: oid 18 updating version 1276 to 1277 2024-09-16T15:13:44.765 INFO:tasks.rados.rados.0.smithi162.stdout:2917: oid 18 version 1277 is already newer than 1275 2024-09-16T15:13:44.766 INFO:tasks.rados.rados.0.smithi162.stdout:2917: oid 18 updating version 1277 to 1278 2024-09-16T15:13:44.766 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1278 (ObjNum 1023 snap 297 seq_num 1023) dirty exists 2024-09-16T15:13:44.766 INFO:tasks.rados.rados.0.smithi162.stdout:2917: left oid 18 (ObjNum 1023 snap 297 seq_num 1023) 2024-09-16T15:13:45.730 INFO:tasks.rados.rados.0.smithi162.stdout:2911: done (6 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2913: done (5 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2914: done (4 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2915: done (3 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2917: done (2 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2918: done (1 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2920: done (0 left) 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2921: rollback oid 39 current snap is 298 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 39 to 249 2024-09-16T15:13:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:2922: delete oid 27 current snap is 298 2024-09-16T15:13:45.735 INFO:tasks.rados.rados.0.smithi162.stdout:2922: done (1 left) 2024-09-16T15:13:45.735 INFO:tasks.rados.rados.0.smithi162.stdout:2923: read oid 27 snap 294 2024-09-16T15:13:45.735 INFO:tasks.rados.rados.0.smithi162.stdout:2923: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:45.735 INFO:tasks.rados.rados.0.smithi162.stdout:2924: read oid 19 snap 277 2024-09-16T15:13:45.735 INFO:tasks.rados.rados.0.smithi162.stdout:2924: expect deleted 2024-09-16T15:13:45.736 INFO:tasks.rados.rados.0.smithi162.stdout:2925: rollback oid 28 current snap is 298 2024-09-16T15:13:45.736 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 288 2024-09-16T15:13:45.736 INFO:tasks.rados.rados.0.smithi162.stdout:2926: snap_remove snap 297 2024-09-16T15:13:46.744 INFO:tasks.rados.rados.0.smithi162.stdout:2926: done (4 left) 2024-09-16T15:13:46.744 INFO:tasks.rados.rados.0.smithi162.stdout:2927: read oid 18 snap 282 2024-09-16T15:13:46.744 INFO:tasks.rados.rados.0.smithi162.stdout:2927: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2924: done (4 left) 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2928: read oid 7 snap -1 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2928: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2929: copy_from oid 26 from oid 38 current snap is 298 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2930: copy_from oid 42 from oid 43 current snap is 298 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2931: read oid 29 snap -1 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2931: expect (ObjNum 1022 snap 297 seq_num 1022) 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2932: write oid 5 current snap is 298 2024-09-16T15:13:46.746 INFO:tasks.rados.rados.0.smithi162.stdout:2932: seq_num 1025 ranges {427175=721527,1930749=735360} 2024-09-16T15:13:46.752 INFO:tasks.rados.rados.0.smithi162.stdout:2932: writing smithi16234619-5 from 427175 to 1148702 tid 1 2024-09-16T15:13:46.761 INFO:tasks.rados.rados.0.smithi162.stdout:2932: writing smithi16234619-5 from 1930749 to 2666109 tid 2 2024-09-16T15:13:46.762 INFO:tasks.rados.rados.0.smithi162.stdout:2933: read oid 50 snap 277 2024-09-16T15:13:46.762 INFO:tasks.rados.rados.0.smithi162.stdout:2933: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:13:46.762 INFO:tasks.rados.rados.0.smithi162.stdout:2934: write oid 9 current snap is 298 2024-09-16T15:13:46.762 INFO:tasks.rados.rados.0.smithi162.stdout:2923: expect (ObjNum 892 snap 246 seq_num 892) 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2925: finishing rollback tid 0 to smithi16234619-28 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2925: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2925: finishing rollback tid 2 to smithi16234619-28 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1171 (ObjNum 990 snap 281 seq_num 990) dirty exists 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2921: finishing rollback tid 0 to smithi16234619-39 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2921: finishing rollback tid 1 to smithi16234619-39 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2921: finishing rollback tid 2 to smithi16234619-39 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1446 (ObjNum 888 snap 246 seq_num 888) dirty exists 2024-09-16T15:13:46.900 INFO:tasks.rados.rados.0.smithi162.stdout:2934: seq_num 1026 ranges {443982=796776,1989231=479018,2959167=284815} 2024-09-16T15:13:46.906 INFO:tasks.rados.rados.0.smithi162.stdout:2934: writing smithi16234619-9 from 443982 to 1240758 tid 1 2024-09-16T15:13:46.913 INFO:tasks.rados.rados.0.smithi162.stdout:2934: writing smithi16234619-9 from 1989231 to 2468249 tid 2 2024-09-16T15:13:46.917 INFO:tasks.rados.rados.0.smithi162.stdout:2934: writing smithi16234619-9 from 2959167 to 3243982 tid 3 2024-09-16T15:13:46.918 INFO:tasks.rados.rados.0.smithi162.stdout:2921: done (10 left) 2024-09-16T15:13:46.918 INFO:tasks.rados.rados.0.smithi162.stdout:2923: done (9 left) 2024-09-16T15:13:46.918 INFO:tasks.rados.rados.0.smithi162.stdout:2925: done (8 left) 2024-09-16T15:13:46.918 INFO:tasks.rados.rados.0.smithi162.stdout:2935: write (excl) oid 14 current snap is 298 2024-09-16T15:13:46.918 INFO:tasks.rados.rados.0.smithi162.stdout:2935: seq_num 1027 ranges {460789=588378,1764066=739029,3219296=41493} 2024-09-16T15:13:46.923 INFO:tasks.rados.rados.0.smithi162.stdout:2935: writing smithi16234619-14 from 460789 to 1049167 tid 1 2024-09-16T15:13:46.931 INFO:tasks.rados.rados.0.smithi162.stdout:2935: writing smithi16234619-14 from 1764066 to 2503095 tid 2 2024-09-16T15:13:46.935 INFO:tasks.rados.rados.0.smithi162.stdout:2935: writing smithi16234619-14 from 3219296 to 3260789 tid 3 2024-09-16T15:13:46.935 INFO:tasks.rados.rados.0.smithi162.stdout:2936: snap_create 2024-09-16T15:13:46.935 INFO:tasks.rados.rados.0.smithi162.stdout:2927: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:13:47.014 INFO:tasks.rados.rados.0.smithi162.stdout:2931: expect (ObjNum 1022 snap 297 seq_num 1022) 2024-09-16T15:13:47.150 INFO:tasks.rados.rados.0.smithi162.stdout:2928: expect (ObjNum 851 snap 227 seq_num 851) 2024-09-16T15:13:47.309 INFO:tasks.rados.rados.0.smithi162.stdout:2933: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:13:47.484 INFO:tasks.rados.rados.0.smithi162.stdout:2930: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:13:47.484 INFO:tasks.rados.rados.0.smithi162.stdout:2929: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:13:47.484 INFO:tasks.rados.rados.0.smithi162.stdout:2929: finishing copy_from to smithi16234619-26 2024-09-16T15:13:47.484 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1284 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:13:47.484 INFO:tasks.rados.rados.0.smithi162.stdout:2930: finishing copy_from to smithi16234619-42 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 803 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: finishing write tid 1 to smithi16234619-5 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: finishing write tid 2 to smithi16234619-5 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: finishing write tid 3 to smithi16234619-5 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: finishing write tid 4 to smithi16234619-5 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: oid 5 updating version 0 to 1452 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: oid 5 updating version 1452 to 1454 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: oid 5 version 1454 is already newer than 1453 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1454 (ObjNum 1025 snap 298 seq_num 1025) dirty exists 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2932: left oid 5 (ObjNum 1025 snap 298 seq_num 1025) 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2934: finishing write tid 1 to smithi16234619-9 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2934: finishing write tid 2 to smithi16234619-9 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2934: finishing write tid 3 to smithi16234619-9 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2934: finishing write tid 4 to smithi16234619-9 2024-09-16T15:13:47.485 INFO:tasks.rados.rados.0.smithi162.stdout:2934: finishing write tid 5 to smithi16234619-9 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2934: oid 9 updating version 0 to 1289 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2934: oid 9 updating version 1289 to 1292 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2934: oid 9 version 1292 is already newer than 1290 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2934: oid 9 version 1292 is already newer than 1291 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1292 (ObjNum 1026 snap 298 seq_num 1026) dirty exists 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2934: left oid 9 (ObjNum 1026 snap 298 seq_num 1026) 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: finishing write tid 1 to smithi16234619-14 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: finishing write tid 2 to smithi16234619-14 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: finishing write tid 3 to smithi16234619-14 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: finishing write tid 4 to smithi16234619-14 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: finishing write tid 5 to smithi16234619-14 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: oid 14 updating version 0 to 1349 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: oid 14 updating version 1349 to 1352 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: oid 14 version 1352 is already newer than 1350 2024-09-16T15:13:47.486 INFO:tasks.rados.rados.0.smithi162.stdout:2935: oid 14 version 1352 is already newer than 1351 2024-09-16T15:13:47.487 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1352 (ObjNum 1027 snap 298 seq_num 1027) dirty exists 2024-09-16T15:13:47.487 INFO:tasks.rados.rados.0.smithi162.stdout:2935: left oid 14 (ObjNum 1027 snap 298 seq_num 1027) 2024-09-16T15:13:47.744 INFO:tasks.rados.rados.0.smithi162.stdout:2927: done (9 left) 2024-09-16T15:13:47.744 INFO:tasks.rados.rados.0.smithi162.stdout:2928: done (8 left) 2024-09-16T15:13:47.744 INFO:tasks.rados.rados.0.smithi162.stdout:2929: done (7 left) 2024-09-16T15:13:47.744 INFO:tasks.rados.rados.0.smithi162.stdout:2930: done (6 left) 2024-09-16T15:13:47.744 INFO:tasks.rados.rados.0.smithi162.stdout:2931: done (5 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2932: done (4 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2933: done (3 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2934: done (2 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2935: done (1 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2936: done (0 left) 2024-09-16T15:13:47.745 INFO:tasks.rados.rados.0.smithi162.stdout:2937: snap_remove snap 289 2024-09-16T15:13:48.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:13:48.760 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-16T15:13:48.761 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:13:48.761 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:13:48.764 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:13:48.764 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-16T15:13:48.764 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-16T15:13:48.764 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-16T15:13:48.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-16T15:13:48.767 INFO:tasks.ceph.mon.d:Started 2024-09-16T15:13:48.767 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:13:48.767 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.878+0000 7ff6de91f640 10 monclient: get_monmap_and_config 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.878+0000 7ff6de91f640 10 monclient: build_initial_monmap 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.878+0000 7ff6de91f640 10 monclient: monmap: 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:13:48.879780+0000 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:13:48.879780+0000 2024-09-16T15:13:48.880 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:13:48.881 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.879+0000 7ff6de91f640 1 Processor -- start 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.879+0000 7ff6de91f640 1 -- start start 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.879+0000 7ff6de91f640 10 monclient: init 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient: _reopen_session rank -1 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient: _add_conns ranks=[6,10,7,3,1,8,4,5,2,9,0] 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-d con 0x7ff6d8136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:13:48.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 0x7ff6d809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-f con 0x7ff6d809bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 0x7ff6d81370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-i con 0x7ff6d8136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d809b600 con 0x7ff6d809bfe0 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80a5740 con 0x7ff6d8136030 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80a52c0 con 0x7ff6d8136c60 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6de91f640 10 monclient(hunting): authenticate will time out at 1565.469441s 2024-09-16T15:13:48.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:35662/0 (socket says 172.21.15.134:35662) 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 -- 172.21.15.134:0/2535759836 learned_addr learned my addr 172.21.15.134:0/2535759836 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 10 monclient(hunting): get_auth_request con 0x7ff6d8136030 auth_method 0 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 -- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 msgr2=0x7ff6d8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 -- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 msgr2=0x7ff6d8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/2535759836 >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 0x7ff6d809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 0x7ff6d81370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): get_auth_request con 0x7ff6d809bfe0 auth_method 0 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): get_auth_request con 0x7ff6d8136c60 auth_method 0 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:48.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.880+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_done global_id 54709 payload 274 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 -- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 msgr2=0x7ff6d8136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 -- 172.21.15.134:0/2535759836 >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 msgr2=0x7ff6d809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/2535759836 >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 0x7ff6d809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 10 monclient: _finish_hunting 0 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 monclient: found mon.noname-i 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6dcebc640 1 -- 172.21.15.134:0/2535759836 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8073250 con 0x7ff6d8136c60 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.881+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/2535759836 >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 0x7ff6d809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6dcebc640 1 -- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 msgr2=0x7ff6d81370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 0x7ff6d81370e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009d90 tx=0x7ff6d002a490 comp rx=0 tx=0).ready entity=mon.7 client_cookie=7631f38e3a14142d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:48.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 1 -- 172.21.15.134:0/2535759836 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0007bc0 con 0x7ff6d8136c60 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: dump: 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:48.892 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 1 -- 172.21.15.134:0/2535759836 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d002d050 con 0x7ff6d8136c60 2024-09-16T15:13:48.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 1 -- 172.21.15.134:0/2535759836 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0007d80 con 0x7ff6d8136c60 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6de91f640 5 monclient: authenticate success, global_id 54709 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6de91f640 10 monclient: get_monmap_and_config success 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6de91f640 4 set_mon_vals no callback set 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.884+0000 7ff6d6ffd640 10 monclient: dump: 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:48.894 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 10 monclient: shutdown 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 -- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 msgr2=0x7ff6d81370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:48.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 0x7ff6d81370e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009d90 tx=0x7ff6d002a490 comp rx=0 tx=0).stop 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 -- 172.21.15.134:0/2535759836 shutdown_connections 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136c60 0x7ff6d81370e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 --2- 172.21.15.134:0/2535759836 >> v2:172.21.15.162:3300/0 conn(0x7ff6d809bfe0 0x7ff6d809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 --2- 172.21.15.134:0/2535759836 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d8136030 0x7ff6d8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 -- 172.21.15.134:0/2535759836 >> 172.21.15.134:0/2535759836 conn(0x7ff6d80a9ea0 msgr2=0x7ff6d80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 -- 172.21.15.134:0/2535759836 shutdown_connections 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 1 -- 172.21.15.134:0/2535759836 wait complete. 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.885+0000 7ff6de91f640 10 monclient: build_initial_monmap 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.886+0000 7ff6de91f640 10 monclient: monmap: 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:13:48.896 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:13:48.897 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.886+0000 7ff6de91f640 1 Processor -- start 2024-09-16T15:13:48.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.886+0000 7ff6de91f640 1 -- start start 2024-09-16T15:13:48.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.886+0000 7ff6de91f640 10 monclient: init 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient: _reopen_session rank -1 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient: _add_conns ranks=[6,7,1,4,0,2,8,3,5] 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-d con 0x7ff6d809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 0x7ff6d8134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-i con 0x7ff6d8136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 0x7ff6d8135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): picked mon.noname-f con 0x7ff6d8136c60 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81263e0 con 0x7ff6d8136c60 2024-09-16T15:13:48.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8126630 con 0x7ff6d809bfe0 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:35676/0 (socket says 172.21.15.134:35676) 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d809fc90 con 0x7ff6d8136030 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6de91f640 10 monclient(hunting): authenticate will time out at 1565.476433s 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 -- 172.21.15.134:0/1516157133 learned_addr learned my addr 172.21.15.134:0/1516157133 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 10 monclient(hunting): get_auth_request con 0x7ff6d809bfe0 auth_method 0 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 msgr2=0x7ff6d81cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:13:48.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 msgr2=0x7ff6d81cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 0x7ff6d8134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 0x7ff6d8135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): get_auth_request con 0x7ff6d8136030 auth_method 0 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): get_auth_request con 0x7ff6d8136c60 auth_method 0 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:48.906 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.887+0000 7ff6dcebc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 10 monclient(hunting): handle_auth_done global_id 54718 payload 824 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 msgr2=0x7ff6d81cb0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 msgr2=0x7ff6d8135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 0x7ff6d8135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 monclient: found mon.noname-i 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:13:48.907 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6d77fe640 1 -- 172.21.15.134:0/1516157133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009400 con 0x7ff6d8136030 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.888+0000 7ff6dcebc640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 0x7ff6d8135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d77fe640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 msgr2=0x7ff6d8134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 0x7ff6d8134f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e930 tx=0x7ff6cc00edf0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6cc00cc10 con 0x7ff6d8136030 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: dump: 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.908 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:13:48.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc00cd90 con 0x7ff6d8136030 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 5 monclient: authenticate success, global_id 54718 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 10 monclient: _renew_subs 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0009a70 con 0x7ff6d8136030 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6cc013d50 con 0x7ff6d8136030 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d4ff9640 10 monclient: dump: 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.910 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 10 monclient: _renew_subs 2024-09-16T15:13:48.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d809f8b0 con 0x7ff6d8136030 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.890+0000 7ff6d67fc640 4 set_mon_vals no callback set 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6de91f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6de91f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6de91f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8126770 con 0x7ff6d8136030 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff6cc013470 con 0x7ff6d8136030 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6d4ff9640 1 --2- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 0x7ff6c403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.891+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(607..607 src has 1..607) v4 ==== 3931+0+0 (secure 0 0 0) 0x7ff6cc04a490 con 0x7ff6d8136030 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.892+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 0x7ff6c403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.892+0000 7ff6d7fff640 10 monclient: get_auth_request con 0x7ff6c40388b0 auth_method 0 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.894+0000 7ff6d7fff640 1 --2- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 0x7ff6c403ad70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80091d0 tx=0x7ff6c8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.897+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff6d8126770 con 0x7ff6d8136030 2024-09-16T15:13:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.897+0000 7ff6d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:48.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:48.897+0000 7ff6d4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:13:49.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.125+0000 7ff6de91f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:13:49.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.125+0000 7ff6de91f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:13:49.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.125+0000 7ff6de91f640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:13:49.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.125+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff6d8126770 con 0x7ff6d8136030 2024-09-16T15:13:49.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.125+0000 7ff6d4ff9640 1 -- 172.21.15.134:0/1516157133 <== mon.7 v2:172.21.15.162:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3185 (secure 0 0 0) 0x7ff6d8126770 con 0x7ff6d8136030 2024-09-16T15:13:49.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.126+0000 7ff6d4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:13:49.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.126+0000 7ff6d4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:13:49.127 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":102,"quorum":[0,1,3,4,5,7,8],"quorum_names":["a","f","g","c","h","i","e"],"quorum_leader_name":"a","quorum_age":92,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 msgr2=0x7ff6c403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 0x7ff6c403ad70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80091d0 tx=0x7ff6c8008040 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 10 monclient: shutdown 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 msgr2=0x7ff6d8134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 0x7ff6d8134f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00e930 tx=0x7ff6cc00edf0 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 shutdown_connections 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c40388b0 0x7ff6c403ad70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6d8136c60 0x7ff6d8135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff6d8136030 0x7ff6d8134f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 --2- 172.21.15.134:0/1516157133 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6d809bfe0 0x7ff6d81cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 >> 172.21.15.134:0/1516157133 conn(0x7ff6d80a9ea0 msgr2=0x7ff6d806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 shutdown_connections 2024-09-16T15:13:49.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:49.128+0000 7ff6de91f640 1 -- 172.21.15.134:0/1516157133 wait complete. 2024-09-16T15:13:49.337 INFO:tasks.rados.rados.0.smithi162.stdout:2937: done (0 left) 2024-09-16T15:13:49.337 INFO:tasks.rados.rados.0.smithi162.stdout:2938: read oid 3 snap -1 2024-09-16T15:13:49.337 INFO:tasks.rados.rados.0.smithi162.stdout:2938: expect (ObjNum 993 snap 281 seq_num 993) 2024-09-16T15:13:49.338 INFO:tasks.rados.rados.0.smithi162.stdout:2939: write (excl) oid 1 current snap is 299 2024-09-16T15:13:49.338 INFO:tasks.rados.rados.0.smithi162.stdout:2939: seq_num 1028 ranges {477596=663627,1706195=482687,2614013=567041} 2024-09-16T15:13:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:2939: writing smithi16234619-1 from 477596 to 1141223 tid 1 2024-09-16T15:13:49.350 INFO:tasks.rados.rados.0.smithi162.stdout:2939: writing smithi16234619-1 from 1706195 to 2188882 tid 2 2024-09-16T15:13:49.356 INFO:tasks.rados.rados.0.smithi162.stdout:2939: writing smithi16234619-1 from 2614013 to 3181054 tid 3 2024-09-16T15:13:49.356 INFO:tasks.rados.rados.0.smithi162.stdout:2940: rollback oid 43 current snap is 299 2024-09-16T15:13:49.356 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 276 2024-09-16T15:13:49.356 INFO:tasks.rados.rados.0.smithi162.stdout:2938: expect (ObjNum 993 snap 281 seq_num 993) 2024-09-16T15:13:49.506 INFO:tasks.rados.rados.0.smithi162.stdout:2940: finishing rollback tid 0 to smithi16234619-43 2024-09-16T15:13:49.506 INFO:tasks.rados.rados.0.smithi162.stdout:2940: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:13:49.506 INFO:tasks.rados.rados.0.smithi162.stdout:2940: finishing rollback tid 2 to smithi16234619-43 2024-09-16T15:13:49.506 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1129 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: finishing write tid 1 to smithi16234619-1 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: finishing write tid 2 to smithi16234619-1 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2938: done (2 left) 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2940: done (1 left) 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2941: copy_from oid 48 from oid 33 current snap is 299 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: finishing write tid 3 to smithi16234619-1 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: finishing write tid 4 to smithi16234619-1 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: finishing write tid 5 to smithi16234619-1 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: oid 1 updating version 0 to 1295 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: oid 1 version 1295 is already newer than 1294 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: oid 1 version 1295 is already newer than 1293 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: oid 1 updating version 1295 to 1296 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1296 (ObjNum 1028 snap 299 seq_num 1028) dirty exists 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: left oid 1 (ObjNum 1028 snap 299 seq_num 1028) 2024-09-16T15:13:49.507 INFO:tasks.rados.rados.0.smithi162.stdout:2939: done (1 left) 2024-09-16T15:13:49.508 INFO:tasks.rados.rados.0.smithi162.stdout:2942: snap_create 2024-09-16T15:13:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:2941: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:13:49.542 INFO:tasks.rados.rados.0.smithi162.stdout:2941: finishing copy_from to smithi16234619-48 2024-09-16T15:13:49.542 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 830 (ObjNum 1003 snap 284 seq_num 1003) dirty exists 2024-09-16T15:13:50.340 INFO:tasks.rados.rados.0.smithi162.stdout:2941: done (1 left) 2024-09-16T15:13:50.340 INFO:tasks.rados.rados.0.smithi162.stdout:2942: done (0 left) 2024-09-16T15:13:50.340 INFO:tasks.rados.rados.0.smithi162.stdout:2943: read oid 12 snap -1 2024-09-16T15:13:50.340 INFO:tasks.rados.rados.0.smithi162.stdout:2943: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:13:50.340 INFO:tasks.rados.rados.0.smithi162.stdout:2944: snap_create 2024-09-16T15:13:50.350 INFO:tasks.rados.rados.0.smithi162.stdout:2943: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:13:51.346 INFO:tasks.rados.rados.0.smithi162.stdout:2943: done (1 left) 2024-09-16T15:13:51.346 INFO:tasks.rados.rados.0.smithi162.stdout:2944: done (0 left) 2024-09-16T15:13:51.346 INFO:tasks.rados.rados.0.smithi162.stdout:2945: snap_create 2024-09-16T15:13:52.140 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.247+0000 7f59267ec640 10 monclient: get_monmap_and_config 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.247+0000 7f59267ec640 10 monclient: build_initial_monmap 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.247+0000 7f59267ec640 10 monclient: monmap: 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:13:52.248818+0000 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:13:52.248818+0000 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.249 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:13:52.250 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.248+0000 7f59267ec640 1 Processor -- start 2024-09-16T15:13:52.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.248+0000 7f59267ec640 1 -- start start 2024-09-16T15:13:52.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.248+0000 7f59267ec640 10 monclient: init 2024-09-16T15:13:52.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient: _add_conns ranks=[10,4,1,6,3,2,0,8,7,5,9] 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5920133a10 0x7f592009cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-f con 0x7f5920133a10 addr v2:172.21.15.162:3300/0 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-c con 0x7f5920133df0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5920099070 addr v1:172.21.15.162:6789/0 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5920073250 con 0x7f5920099070 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200a3ca0 con 0x7f5920133a10 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f592009bcf0 con 0x7f5920133df0 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:52.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f59267ec640 10 monclient(hunting): authenticate will time out at 1568.838232s 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40280/0 (socket says 172.21.15.134:40280) 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 -- 172.21.15.134:0/481239678 learned_addr learned my addr 172.21.15.134:0/481239678 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): get_auth_request con 0x7f5920133df0 auth_method 0 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient(hunting): handle_auth_done global_id 5602 payload 274 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 -- 172.21.15.134:0/481239678 >> v2:172.21.15.162:3300/0 conn(0x7f5920133a10 msgr2=0x7f592009cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:13:52.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 --2- 172.21.15.134:0/481239678 >> v2:172.21.15.162:3300/0 conn(0x7f5920133a10 0x7f592009cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 -- 172.21.15.134:0/481239678 >> v1:172.21.15.162:6789/0 conn(0x7f5920099070 legacy=0x7f5920099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 monclient: found mon.noname-c 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.249+0000 7f591f7fe640 1 -- 172.21.15.134:0/481239678 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592009bfd0 con 0x7f5920133df0 2024-09-16T15:13:52.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591f7fe640 1 -- 172.21.15.134:0/481239678 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 msgr2=0x7f59201341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:13:52.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591f7fe640 1 --2- 172.21.15.134:0/481239678 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f591400bb10 tx=0x7f591402c7a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ba574618861d648 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:52.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 1 -- 172.21.15.134:0/481239678 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5914031c70 con 0x7f5920133df0 2024-09-16T15:13:52.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:52.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: dump: 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:52.262 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: _finish_auth 0 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: _check_auth_tickets 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f59267ec640 5 monclient: authenticate success, global_id 5602 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f59267ec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 1 -- 172.21.15.134:0/481239678 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591402f050 con 0x7f5920133df0 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f59267ec640 10 monclient: get_monmap_and_config success 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 1 -- 172.21.15.134:0/481239678 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5914031e30 con 0x7f5920133df0 2024-09-16T15:13:52.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f59267ec640 4 set_mon_vals no callback set 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.252+0000 7f591effd640 10 monclient: dump: 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:52.264 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 10 monclient: shutdown 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 -- 172.21.15.134:0/481239678 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 msgr2=0x7f59201341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 --2- 172.21.15.134:0/481239678 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f591400bb10 tx=0x7f591402c7a0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 -- 172.21.15.134:0/481239678 shutdown_connections 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 --2- 172.21.15.134:0/481239678 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133df0 0x7f59201341f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 --2- 172.21.15.134:0/481239678 >> v2:172.21.15.162:3300/0 conn(0x7f5920133a10 0x7f592009cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 -- 172.21.15.134:0/481239678 >> 172.21.15.134:0/481239678 conn(0x7f59200a2e20 msgr2=0x7f59200a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 -- 172.21.15.134:0/481239678 shutdown_connections 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.253+0000 7f59267ec640 1 -- 172.21.15.134:0/481239678 wait complete. 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.254+0000 7f59267ec640 10 monclient: build_initial_monmap 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.254+0000 7f59267ec640 10 monclient: monmap: 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:13:52.266 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.254+0000 7f59267ec640 1 Processor -- start 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.254+0000 7f59267ec640 1 -- start start 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.254+0000 7f59267ec640 10 monclient: init 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient: _reopen_session rank -1 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient: _add_conns ranks=[5,4,7,8,0,2,3,6,1] 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5920099070 0x7f59201c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-h con 0x7f5920099070 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-c con 0x7f5920133a10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:13:52.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5920133df0 0x7f5920130ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): picked mon.noname-i con 0x7f5920133df0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59200998d0 con 0x7f5920133a10 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59201c8120 con 0x7f5920099070 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f592012be90 con 0x7f5920133df0 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f59267ec640 10 monclient(hunting): authenticate will time out at 1568.844174s 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40290/0 (socket says 172.21.15.134:40290) 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 -- 172.21.15.134:0/973904231 learned_addr learned my addr 172.21.15.134:0/973904231 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): get_auth_request con 0x7f5920133a10 auth_method 0 2024-09-16T15:13:52.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5920133df0 0x7f5920130ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5920099070 0x7f59201c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): get_auth_request con 0x7f5920133df0 auth_method 0 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): get_auth_request con 0x7f5920099070 auth_method 0 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:52.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f5924d89640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient(hunting): handle_auth_done global_id 5611 payload 824 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 -- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5920133df0 msgr2=0x7f5920130ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5920133df0 0x7f5920130ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 -- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5920099070 msgr2=0x7f59201c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5920099070 0x7f59201c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 monclient: found mon.noname-c 2024-09-16T15:13:52.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.255+0000 7f591f7fe640 1 -- 172.21.15.134:0/973904231 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591400b770 con 0x7f5920133a10 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.257+0000 7f591f7fe640 1 -- 172.21.15.134:0/973904231 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 msgr2=0x7f59201309b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.257+0000 7f591f7fe640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f591402ccc0 tx=0x7f59140046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5914003a80 con 0x7f5920133a10 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: dump: 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.271 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: _finish_auth 0 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 5 monclient: authenticate success, global_id 5611 2024-09-16T15:13:52.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914003be0 con 0x7f5920133a10 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 10 monclient: _renew_subs 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592012c450 con 0x7f5920133a10 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5914003d50 con 0x7f5920133a10 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 10 monclient: dump: 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:13:52.273 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591e7fc640 4 set_mon_vals no callback set 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 10 monclient: _renew_subs 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5920071c40 con 0x7f5920133a10 2024-09-16T15:13:52.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.258+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5914003610 con 0x7f5920133a10 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f591cff9640 1 --2- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 0x7f590c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(610..610 src has 1..610) v4 ==== 3947+0+0 (secure 0 0 0) 0x7f591406d560 con 0x7f5920133a10 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f59267ec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f59267ec640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f59267ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59201346e0 con 0x7f5920133a10 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f591ffff640 1 --2- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 0x7f590c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.259+0000 7f591ffff640 10 monclient: get_auth_request con 0x7f590c038880 auth_method 0 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.261+0000 7f591ffff640 1 --2- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 0x7f590c03ad40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5910009990 tx=0x7f5910009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:13:52.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.264+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f59140038c0 con 0x7f5920133a10 2024-09-16T15:13:52.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.264+0000 7f591cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:13:52.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.264+0000 7f591cff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:13:52.359 INFO:tasks.rados.rados.0.smithi162.stdout:2945: done (0 left) 2024-09-16T15:13:52.359 INFO:tasks.rados.rados.0.smithi162.stdout:2946: snap_create 2024-09-16T15:13:52.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.491+0000 7f59267ec640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:13:52.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.491+0000 7f59267ec640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:13:52.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.491+0000 7f59267ec640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:13:52.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.491+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f59201341f0 con 0x7f5920133a10 2024-09-16T15:13:52.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.492+0000 7f591cff9640 1 -- 172.21.15.134:0/973904231 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3185 (secure 0 0 0) 0x7f59140323a0 con 0x7f5920133a10 2024-09-16T15:13:52.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.492+0000 7f591cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:13:52.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.492+0000 7f591cff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:13:52.494 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":102,"quorum":[0,1,3,4,5,7,8],"quorum_names":["a","f","g","c","h","i","e"],"quorum_leader_name":"a","quorum_age":95,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 msgr2=0x7f590c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 0x7f590c03ad40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5910009990 tx=0x7f5910009110 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 10 monclient: shutdown 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 msgr2=0x7f59201309b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f591402ccc0 tx=0x7f59140046c0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 shutdown_connections 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> v2:172.21.15.162:6800/980547263 conn(0x7f590c038880 0x7f590c03ad40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5920133df0 0x7f5920130ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5920133a10 0x7f59201309b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 --2- 172.21.15.134:0/973904231 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5920099070 0x7f59201c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:52.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 >> 172.21.15.134:0/973904231 conn(0x7f59200a2e20 msgr2=0x7f5920098570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:13:52.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 shutdown_connections 2024-09-16T15:13:52.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:52.494+0000 7f59267ec640 1 -- 172.21.15.134:0/973904231 wait complete. 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2946: done (0 left) 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2947: read oid 8 snap -1 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2947: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2948: rollback oid 34 current snap is 303 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 249 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2949: rollback oid 20 current snap is 303 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 230 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:2950: rollback oid 43 current snap is 303 2024-09-16T15:13:53.357 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 282 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2951: read oid 18 snap 279 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2951: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2952: read oid 31 snap -1 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2952: expect deleted 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2953: read oid 2 snap -1 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2953: expect (ObjNum 975 snap 275 seq_num 975) 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2954: rollback oid 49 current snap is 303 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 275 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2955: write oid 48 current snap is 303 2024-09-16T15:13:53.358 INFO:tasks.rados.rados.0.smithi162.stdout:2955: seq_num 1029 ranges {494403=738876,1764677=626345,3041436=252967} 2024-09-16T15:13:53.365 INFO:tasks.rados.rados.0.smithi162.stdout:2955: writing smithi16234619-48 from 494403 to 1233279 tid 1 2024-09-16T15:13:53.372 INFO:tasks.rados.rados.0.smithi162.stdout:2955: writing smithi16234619-48 from 1764677 to 2391022 tid 2 2024-09-16T15:13:53.377 INFO:tasks.rados.rados.0.smithi162.stdout:2955: writing smithi16234619-48 from 3041436 to 3294403 tid 3 2024-09-16T15:13:53.377 INFO:tasks.rados.rados.0.smithi162.stdout:2956: read oid 12 snap -1 2024-09-16T15:13:53.377 INFO:tasks.rados.rados.0.smithi162.stdout:2956: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:13:53.377 INFO:tasks.rados.rados.0.smithi162.stdout:2957: snap_create 2024-09-16T15:13:53.377 INFO:tasks.rados.rados.0.smithi162.stdout:2947: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:13:53.463 INFO:tasks.rados.rados.0.smithi162.stdout:2953: expect (ObjNum 975 snap 275 seq_num 975) 2024-09-16T15:13:53.591 INFO:tasks.rados.rados.0.smithi162.stdout:2948: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:13:53.591 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1330 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2949: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2949: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 832 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2954: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2954: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2950: finishing rollback tid 0 to smithi16234619-43 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2950: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2950: finishing rollback tid 2 to smithi16234619-43 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1132 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:13:53.592 INFO:tasks.rados.rados.0.smithi162.stdout:2951: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:13:53.669 INFO:tasks.rados.rados.0.smithi162.stdout:2954: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:13:53.669 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 824 (ObjNum 967 snap 272 seq_num 967) dirty exists 2024-09-16T15:13:53.669 INFO:tasks.rados.rados.0.smithi162.stdout:2956: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: finishing write tid 1 to smithi16234619-48 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: finishing write tid 2 to smithi16234619-48 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: finishing write tid 3 to smithi16234619-48 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: finishing write tid 4 to smithi16234619-48 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: finishing write tid 5 to smithi16234619-48 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: oid 48 updating version 0 to 1300 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: oid 48 version 1300 is already newer than 1298 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: oid 48 version 1300 is already newer than 1297 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: oid 48 version 1300 is already newer than 1299 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1300 (ObjNum 1029 snap 303 seq_num 1029) dirty exists 2024-09-16T15:13:53.774 INFO:tasks.rados.rados.0.smithi162.stdout:2955: left oid 48 (ObjNum 1029 snap 303 seq_num 1029) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2947: done (10 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2948: done (9 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2949: done (8 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2950: done (7 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2951: done (6 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2952: done (5 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2953: done (4 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2954: done (3 left) 2024-09-16T15:13:54.045 INFO:tasks.rados.rados.0.smithi162.stdout:2955: done (2 left) 2024-09-16T15:13:54.046 INFO:tasks.rados.rados.0.smithi162.stdout:2956: done (1 left) 2024-09-16T15:13:54.046 INFO:tasks.rados.rados.0.smithi162.stdout:2957: done (0 left) 2024-09-16T15:13:54.046 INFO:tasks.rados.rados.0.smithi162.stdout:2958: write (excl) oid 35 current snap is 304 2024-09-16T15:13:54.046 INFO:tasks.rados.rados.0.smithi162.stdout:2958: seq_num 1030 ranges {511210=414125,1423159=486356,2668859=589585} 2024-09-16T15:13:54.050 INFO:tasks.rados.rados.0.smithi162.stdout:2958: writing smithi16234619-35 from 511210 to 925335 tid 1 2024-09-16T15:13:54.055 INFO:tasks.rados.rados.0.smithi162.stdout:2958: writing smithi16234619-35 from 1423159 to 1909515 tid 2 2024-09-16T15:13:54.063 INFO:tasks.rados.rados.0.smithi162.stdout:2958: writing smithi16234619-35 from 2668859 to 3258444 tid 3 2024-09-16T15:13:54.063 INFO:tasks.rados.rados.0.smithi162.stdout:2959: snap_remove snap 275 2024-09-16T15:13:55.507 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.613+0000 7f3b5f4d0640 10 monclient: get_monmap_and_config 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.613+0000 7f3b5f4d0640 10 monclient: build_initial_monmap 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.613+0000 7f3b5f4d0640 10 monclient: monmap: 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:13:55.614781+0000 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:13:55.614781+0000 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:13:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:13:55.616 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.614+0000 7f3b5f4d0640 1 Processor -- start 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.614+0000 7f3b5f4d0640 1 -- start start 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.614+0000 7f3b5f4d0640 10 monclient: init 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient: _add_conns ranks=[4,2,10,1,3,8,5,7,6,0,9] 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-c con 0x7f3b58136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-b con 0x7f3b5809bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-f con 0x7f3b58136c60 addr v2:172.21.15.162:3300/0 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5809b600 con 0x7f3b58136c60 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a5740 con 0x7f3b5809bfe0 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a52c0 con 0x7f3b58136030 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5f4d0640 10 monclient(hunting): authenticate will time out at 1572.204166s 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:56600/0 (socket says 172.21.15.134:56600) 2024-09-16T15:13:55.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 learned_addr learned my addr 172.21.15.134:0/687760628 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b5809bfe0 auth_method 0 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b58136030 auth_method 0 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b58136c60 auth_method 0 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b58136030 auth_method 0 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b5809bfe0 auth_method 0 2024-09-16T15:13:55.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b58136c60 auth_method 0 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:55.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:55.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:55.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:55.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:55.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:55.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b58136030 auth_method 0 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:56.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b5809bfe0 auth_method 0 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:56.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b58136c60 auth_method 0 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:56.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:56.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b5809bfe0 auth_method 0 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b58136030 auth_method 0 2024-09-16T15:13:57.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:13:57.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b58136c60 auth_method 0 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.018+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:57.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:57.019+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): tick 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 monclient(hunting): continuing hunt 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 msgr2=0x7f3b58136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3b58136030 0x7f3b58136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 msgr2=0x7f3b5809c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b5809bfe0 0x7f3b5809c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 msgr2=0x7f3b581370e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.162:3300/0 conn(0x7f3b58136c60 0x7f3b581370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:13:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient: _add_conns ranks=[7,5,8,1,0,9,6,3] 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-i con 0x7f3b44003810 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-h con 0x7f3b44006200 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-e con 0x7f3b44008d80 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5809b600 con 0x7f3b44006200 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a5740 con 0x7f3b44003810 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a52c0 con 0x7f3b44008d80 2024-09-16T15:13:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b4effd640 10 monclient(hunting): _renew_subs 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b44008d80 auth_method 0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b44003810 auth_method 0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44006200 auth_method 0 2024-09-16T15:13:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.615+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:13:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b44008d80 auth_method 0 2024-09-16T15:13:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b44003810 auth_method 0 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44006200 auth_method 0 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:58.816+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:13:59.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:59.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b44008d80 auth_method 0 2024-09-16T15:13:59.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b44003810 auth_method 0 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:59.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44006200 auth_method 0 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:13:59.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:13:59.217+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:00.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b44008d80 auth_method 0 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44006200 auth_method 0 2024-09-16T15:14:00.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b44003810 auth_method 0 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:00.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:00.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:00.019+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2959: done (1 left) 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2960: copy_from oid 27 from oid 41 current snap is 304 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: finishing write tid 1 to smithi16234619-35 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: finishing write tid 2 to smithi16234619-35 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: finishing write tid 3 to smithi16234619-35 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: finishing write tid 4 to smithi16234619-35 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: finishing write tid 5 to smithi16234619-35 2024-09-16T15:14:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:2958: oid 35 updating version 0 to 1287 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2958: oid 35 updating version 1287 to 1288 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2958: oid 35 version 1288 is already newer than 1286 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2958: oid 35 version 1288 is already newer than 1285 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1288 (ObjNum 1030 snap 304 seq_num 1030) dirty exists 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2958: left oid 35 (ObjNum 1030 snap 304 seq_num 1030) 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2958: done (1 left) 2024-09-16T15:14:01.194 INFO:tasks.rados.rados.0.smithi162.stdout:2961: snap_remove snap 266 2024-09-16T15:14:01.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): tick 2024-09-16T15:14:01.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 monclient(hunting): continuing hunt 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 msgr2=0x7f3b4400b210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44008d80 0x7f3b4400b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 msgr2=0x7f3b44005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3b44003810 0x7f3b44005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 msgr2=0x7f3b44008690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3b44006200 0x7f3b44008690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient: _add_conns ranks=[3,6,9,0,1] 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 reap_dead start 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b4400e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-g con 0x7f3b4400be70 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-d con 0x7f3b44003bf0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.134:3300/0 conn(0x7f3b44009740 0x7f3b44008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): picked mon.noname-a con 0x7f3b44009740 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5809b600 con 0x7f3b44009740 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a5740 con 0x7f3b4400be70 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 1 -- 172.21.15.134:0/687760628 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3b580a52c0 con 0x7f3b44003bf0 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b4effd640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> v2:172.21.15.134:3300/0 conn(0x7f3b44009740 0x7f3b44008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b44009740 auth_method 0 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.615+0000 7f3b5da6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44003bf0 auth_method 0 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b4400e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b4400be70 auth_method 0 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 msgr2=0x7f3b44009200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 msgr2=0x7f3b44009200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient(hunting): handle_auth_done global_id 74646 payload 274 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 msgr2=0x7f3b44009200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:01.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 msgr2=0x7f3b4400e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b4400e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 monclient: found mon.noname-a 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.616+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b58073250 con 0x7f3b44009740 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5da6d640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 msgr2=0x7f3b44008630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b44008630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b580a3af0 tx=0x7f3b5002c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bcaced57099527 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 1 -- 172.21.15.134:0/687760628 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b5002cd90 con 0x7f3b44009740 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:01.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: dump: 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.630 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: _finish_auth 0 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 1 -- 172.21.15.134:0/687760628 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b50030070 con 0x7f3b44009740 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 5 monclient: authenticate success, global_id 74646 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 1 -- 172.21.15.134:0/687760628 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b50036ce0 con 0x7f3b44009740 2024-09-16T15:14:01.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 4 set_mon_vals no callback set 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b4ffff640 10 monclient: dump: 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.632 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 10 monclient: shutdown 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 msgr2=0x7f3b44008630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b44008630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b580a3af0 tx=0x7f3b5002c150 comp rx=0 tx=0).stop 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/687760628 shutdown_connections 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b44008630 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3b44003bf0 0x7f3b44009200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.619+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/687760628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b4400e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.620+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/687760628 >> 172.21.15.134:0/687760628 conn(0x7f3b580a9ea0 msgr2=0x7f3b580a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.620+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/687760628 shutdown_connections 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.620+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/687760628 wait complete. 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.620+0000 7f3b5f4d0640 10 monclient: build_initial_monmap 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.620+0000 7f3b5f4d0640 10 monclient: monmap: 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:01.634 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 Processor -- start 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 -- start start 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient: init 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient: _add_conns ranks=[8,0,3,1,2,4,6,7,5] 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44003bf0 0x7f3b58134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-e con 0x7f3b44003bf0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-a con 0x7f3b44009740 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b581cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): picked mon.noname-g con 0x7f3b4400be70 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58120af0 con 0x7f3b44009740 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58122150 con 0x7f3b4400be70 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58122430 con 0x7f3b44003bf0 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5f4d0640 10 monclient(hunting): authenticate will time out at 1578.210728s 2024-09-16T15:14:01.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44003bf0 0x7f3b58134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37098/0 (socket says 172.21.15.134:37098) 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/2962166918 learned_addr learned my addr 172.21.15.134:0/2962166918 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request con 0x7f3b44009740 auth_method 0 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request con 0x7f3b44003bf0 auth_method 0 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5ca6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5d26c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.621+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b581cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request con 0x7f3b4400be70 auth_method 0 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5da6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5da6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5da6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 10 monclient(hunting): handle_auth_done global_id 74655 payload 824 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44003bf0 msgr2=0x7f3b58134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44003bf0 0x7f3b58134d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.638 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/2962166918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 msgr2=0x7f3b581cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b581cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 10 monclient: _finish_hunting 0 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 monclient: found mon.noname-a 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/2962166918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500099f0 con 0x7f3b44009740 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.622+0000 7f3b5da6d640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b581cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5ca6b640 1 -- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 msgr2=0x7f3b58135290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5ca6b640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b48016aa0 tx=0x7f3b48016f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b4801f070 con 0x7f3b44009740 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: dump: 2024-09-16T15:14:01.639 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.640 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: _finish_auth 0 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4800c040 con 0x7f3b44009740 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 5 monclient: authenticate success, global_id 74655 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 10 monclient: _renew_subs 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b4801a2e0 con 0x7f3b44009740 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b581bf750 con 0x7f3b44009740 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4dffb640 10 monclient: dump: 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.642 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b4f7fe640 4 set_mon_vals no callback set 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 10 monclient: _renew_subs 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.624+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b580a8170 con 0x7f3b44009740 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3b48017c90 con 0x7f3b44009740 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b5f4d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b5f4d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b5f4d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b580a7cd0 con 0x7f3b44009740 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b4dffb640 1 --2- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 0x7f3b4403cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.625+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(614..614 src has 1..614) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f3b4804b320 con 0x7f3b44009740 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.626+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 0x7f3b4403cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.626+0000 7f3b5d26c640 10 monclient: get_auth_request con 0x7f3b4403a8b0 auth_method 0 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.628+0000 7f3b5d26c640 1 --2- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 0x7f3b4403cd70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b54006fd0 tx=0x7f3b54008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.630+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3b48015770 con 0x7f3b44009740 2024-09-16T15:14:01.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.630+0000 7f3b4dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:01.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.630+0000 7f3b4dffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:01.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.856+0000 7f3b5f4d0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:14:01.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b5f4d0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:01.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b5f4d0640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:01.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3b5809c400 con 0x7f3b44009740 2024-09-16T15:14:01.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b4dffb640 1 -- 172.21.15.134:0/2962166918 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f3b4801a6c0 con 0x7f3b44009740 2024-09-16T15:14:01.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b4dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:01.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.857+0000 7f3b4dffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:01.859 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 msgr2=0x7f3b4403cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 0x7f3b4403cd70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b54006fd0 tx=0x7f3b54008040 comp rx=0 tx=0).stop 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 10 monclient: shutdown 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 msgr2=0x7f3b58135290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b48016aa0 tx=0x7f3b48016f60 comp rx=0 tx=0).stop 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 shutdown_connections 2024-09-16T15:14:01.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> v2:172.21.15.162:6800/980547263 conn(0x7f3b4403a8b0 0x7f3b4403cd70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b4400be70 0x7f3b581cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b44009740 0x7f3b58135290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 --2- 172.21.15.134:0/2962166918 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3b44003bf0 0x7f3b58134d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 >> 172.21.15.134:0/2962166918 conn(0x7f3b580a9ea0 msgr2=0x7f3b5806bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 shutdown_connections 2024-09-16T15:14:01.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.859+0000 7f3b5f4d0640 1 -- 172.21.15.134:0/2962166918 wait complete. 2024-09-16T15:14:01.871 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:14:01.871 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:14:01.872 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.967+0000 7f9f4a34c640 10 monclient: get_monmap_and_config 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.967+0000 7f9f4a34c640 10 monclient: build_initial_monmap 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.967+0000 7f9f4a34c640 10 monclient: monmap: 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:01.968803+0000 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:01.968803+0000 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:01.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.968+0000 7f9f4a34c640 1 Processor -- start 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.968+0000 7f9f4a34c640 1 -- start start 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.968+0000 7f9f4a34c640 10 monclient: init 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient: _add_conns ranks=[2,3,4,5,10,6,0,8,9,1,7] 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-b con 0x7f9f44136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f44137220 0x7f9f44137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-g con 0x7f9f44137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-c con 0x7f9f44135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4409abc0 con 0x7f9f44136e40 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4409ba50 con 0x7f9f44137220 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440a53c0 con 0x7f9f44135dc0 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f4a34c640 10 monclient(hunting): authenticate will time out at 1578.558279s 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55368/0 (socket says 172.21.15.134:55368) 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:43154/0 (socket says 172.21.15.134:43154) 2024-09-16T15:14:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 -- 172.21.15.134:0/2534846647 learned_addr learned my addr 172.21.15.134:0/2534846647 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): get_auth_request con 0x7f9f44135dc0 auth_method 0 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 10 monclient(hunting): get_auth_request con 0x7f9f44136e40 auth_method 0 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 msgr2=0x7f9f4409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 msgr2=0x7f9f4409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f43fff640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f3bfff640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f44137220 0x7f9f44137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f3bfff640 10 monclient(hunting): get_auth_request con 0x7f9f44137220 auth_method 0 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient(hunting): handle_auth_done global_id 5620 payload 274 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f44137220 msgr2=0x7f9f44137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f44137220 0x7f9f44137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 msgr2=0x7f9f4409c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient: _finish_hunting 0 2024-09-16T15:14:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 monclient: found mon.noname-c 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.969+0000 7f9f488e9640 1 -- 172.21.15.134:0/2534846647 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44073250 con 0x7f9f44135dc0 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f488e9640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 msgr2=0x7f9f44136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f488e9640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009e80 tx=0x7f9f3c02a530 comp rx=0 tx=0).ready entity=mon.4 client_cookie=141800c41754daeb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 1 -- 172.21.15.134:0/2534846647 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f3c007a70 con 0x7f9f44135dc0 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: dump: 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f4a34c640 5 monclient: authenticate success, global_id 5620 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f4a34c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 1 -- 172.21.15.134:0/2534846647 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c02d050 con 0x7f9f44135dc0 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f4a34c640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f4a34c640 4 set_mon_vals no callback set 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 1 -- 172.21.15.134:0/2534846647 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f3c007bf0 con 0x7f9f44135dc0 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.972+0000 7f9f437fe640 10 monclient: dump: 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.988 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 10 monclient: shutdown 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 -- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 msgr2=0x7f9f44136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009e80 tx=0x7f9f3c02a530 comp rx=0 tx=0).stop 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 -- 172.21.15.134:0/2534846647 shutdown_connections 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44135dc0 0x7f9f44136220 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f44137220 0x7f9f44137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/2534846647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f44136e40 0x7f9f4409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 -- 172.21.15.134:0/2534846647 >> 172.21.15.134:0/2534846647 conn(0x7f9f440a6110 msgr2=0x7f9f440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 -- 172.21.15.134:0/2534846647 shutdown_connections 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 1 -- 172.21.15.134:0/2534846647 wait complete. 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.973+0000 7f9f4a34c640 10 monclient: build_initial_monmap 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient: monmap: 2024-09-16T15:14:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 Processor -- start 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 -- start start 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient: init 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient: _add_conns ranks=[1,5,4,7,0,3,8,6,2] 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f44135dc0 0x7f9f44134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-f con 0x7f9f44135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f44136e40 0x7f9f441354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-h con 0x7f9f44136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 0x7f9f44121a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): picked mon.noname-c con 0x7f9f44137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441250c0 con 0x7f9f44135dc0 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44136330 con 0x7f9f44137220 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441cc160 con 0x7f9f44136e40 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f4a34c640 10 monclient(hunting): authenticate will time out at 1578.563897s 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f488e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 0x7f9f44121a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f488e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 0x7f9f44121a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55370/0 (socket says 172.21.15.134:55370) 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.974+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 learned_addr learned my addr 172.21.15.134:0/1906541839 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:01.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): get_auth_request con 0x7f9f44137220 auth_method 0 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f44136e40 0x7f9f441354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f44135dc0 0x7f9f44134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): get_auth_request con 0x7f9f44136e40 auth_method 0 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 10 monclient(hunting): get_auth_request con 0x7f9f44135dc0 auth_method 0 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient(hunting): handle_auth_done global_id 5629 payload 824 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f44136e40 msgr2=0x7f9f441354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f44136e40 0x7f9f441354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f44135dc0 msgr2=0x7f9f44134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f44135dc0 0x7f9f44134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient: _finish_hunting 0 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 monclient: found mon.noname-c 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009b30 con 0x7f9f44137220 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.975+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f44135dc0 0x7f9f44134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 msgr2=0x7f9f44121a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f488e9640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 0x7f9f44121a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009e80 tx=0x7f9f3c004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f3c004790 con 0x7f9f44137220 2024-09-16T15:14:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: dump: 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c007a70 con 0x7f9f44137220 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 5 monclient: authenticate success, global_id 5629 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 10 monclient: _renew_subs 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44123ce0 con 0x7f9f44137220 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f3c007be0 con 0x7f9f44137220 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f417fa640 10 monclient: dump: 2024-09-16T15:14:01.998 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:01.999 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f42ffd640 4 set_mon_vals no callback set 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 10 monclient: _renew_subs 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.977+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f440a89c0 con 0x7f9f44137220 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f4a34c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 0x7f9f44121f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44071b20 con 0x7f9f441cb490 2024-09-16T15:14:02.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9f3c02ad50 con 0x7f9f44137220 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 0x7f9f44121f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient: get_auth_request con 0x7f9f441cb490 auth_method 0 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f417fa640 1 --2- 172.21.15.134:0/1906541839 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f24038880 0x7f9f2403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(614..614 src has 1..614) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f9f3c06aab0 con 0x7f9f44137220 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 10 monclient(hunting): handle_auth_done global_id 74664 payload 824 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f43fff640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f44061ae0 con 0x7f9f441cb490 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f3bfff640 1 --2- 172.21.15.134:0/1906541839 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f24038880 0x7f9f2403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.978+0000 7f9f3bfff640 10 monclient: get_auth_request con 0x7f9f24038880 auth_method 0 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.980+0000 7f9f43fff640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 msgr2=0x7f9f44121f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.980+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 0x7f9f44121f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400f550 tx=0x7f9f3400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f34014cf0 con 0x7f9f441cb490 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 10 monclient: dump: 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9f34014e50 con 0x7f9f441cb490 2024-09-16T15:14:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 msgr2=0x7f9f44121f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 0x7f9f44121f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400f550 tx=0x7f9f3400fa10 comp rx=0 tx=0).stop 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 msgr2=0x7f9f44121f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f417fa640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441cb490 0x7f9f44121f90 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400f550 tx=0x7f9f3400fa10 comp rx=0 tx=0).stop 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.981+0000 7f9f3bfff640 1 --2- 172.21.15.134:0/1906541839 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f24038880 0x7f9f2403ad40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007920 tx=0x7f9f2c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f4a34c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 0x7f9f440a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f4a34c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4407a680 con 0x7f9f441224d0 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 0x7f9f440a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient: get_auth_request con 0x7f9f441224d0 auth_method 0 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 10 monclient(hunting): handle_auth_done global_id 74673 payload 824 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.987+0000 7f9f43fff640 1 -- 172.21.15.134:0/1906541839 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f4409b490 con 0x7f9f441224d0 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.989+0000 7f9f43fff640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 msgr2=0x7f9f440a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f43fff640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 0x7f9f440a1a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9f3401a000 tx=0x7f9f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f3401e890 con 0x7f9f441224d0 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 10 monclient: dump: 2024-09-16T15:14:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9198 (secure 0 0 0) 0x7f9f3401e420 con 0x7f9f441224d0 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 msgr2=0x7f9f440a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 0x7f9f440a1a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9f3401a000 tx=0x7f9f340046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 msgr2=0x7f9f440a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f417fa640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f441224d0 0x7f9f440a1a20 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9f3401a000 tx=0x7f9f340046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:02.011 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:02.012 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.014 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:02.015 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:02.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:02.020 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:02.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:02.022 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:02.023 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.024 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.028 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.030 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.032 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:02.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:02.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:02.035 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:02.036 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:02.037 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:02.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f24038880 msgr2=0x7f9f2403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/1906541839 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f24038880 0x7f9f2403ad40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007920 tx=0x7f9f2c008040 comp rx=0 tx=0).stop 2024-09-16T15:14:02.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 10 monclient: shutdown 2024-09-16T15:14:02.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 msgr2=0x7f9f44121a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 --2- 172.21.15.134:0/1906541839 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f44137220 0x7f9f44121a50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009e80 tx=0x7f9f3c004a40 comp rx=0 tx=0).stop 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 reap_dead start 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f488e9640 1 -- 172.21.15.134:0/1906541839 reap_dead start 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 shutdown_connections 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 >> 172.21.15.134:0/1906541839 conn(0x7f9f440a6110 msgr2=0x7f9f4406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 shutdown_connections 2024-09-16T15:14:02.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:01.990+0000 7f9f4a34c640 1 -- 172.21.15.134:0/1906541839 wait complete. 2024-09-16T15:14:02.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:02.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:14:02.040 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:14:02.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.136+0000 7fa2e6b76640 10 monclient: get_monmap_and_config 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.136+0000 7fa2e6b76640 10 monclient: build_initial_monmap 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.136+0000 7fa2e6b76640 10 monclient: monmap: 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:02.137481+0000 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:02.137481+0000 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:02.138 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:02.139 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 Processor -- start 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 -- start start 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient: init 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient: _add_conns ranks=[7,4,8,5,1,9,2,0,3,6,10] 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa2e0136e40 0x7fa2e009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-i con 0x7fa2e0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-c con 0x7fa2e0137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 0x7fa2e0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-e con 0x7fa2e0135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e009abc0 con 0x7fa2e0137220 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e009ba50 con 0x7fa2e0136e40 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00a53c0 con 0x7fa2e0135dc0 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e6b76640 10 monclient(hunting): authenticate will time out at 1578.726850s 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2e5113640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 0x7fa2e0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55376/0 (socket says 172.21.15.134:55376) 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 1 -- 172.21.15.134:0/3798398923 learned_addr learned my addr 172.21.15.134:0/3798398923 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 10 monclient(hunting): get_auth_request con 0x7fa2e0137220 auth_method 0 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.137+0000 7fa2d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): get_auth_request con 0x7fa2e0135dc0 auth_method 0 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e4912640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa2e0136e40 0x7fa2e009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e4912640 10 monclient(hunting): get_auth_request con 0x7fa2e0136e40 auth_method 0 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient(hunting): handle_auth_done global_id 84320 payload 274 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 -- 172.21.15.134:0/3798398923 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa2e0136e40 msgr2=0x7fa2e009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa2e0136e40 0x7fa2e009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 -- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 msgr2=0x7fa2e0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient: _finish_hunting 0 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 monclient: found mon.noname-e 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2e5113640 1 -- 172.21.15.134:0/3798398923 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0073250 con 0x7fa2e0135dc0 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.138+0000 7fa2d7fff640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:02.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.140+0000 7fa2e5113640 1 -- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 msgr2=0x7fa2e0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:02.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.140+0000 7fa2e5113640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 0x7fa2e0136220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009dc0 tx=0x7fa2d802a480 comp rx=0 tx=0).ready entity=mon.8 client_cookie=6f14f89ed74af526 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 1 -- 172.21.15.134:0/3798398923 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d8007bc0 con 0x7fa2e0135dc0 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: dump: 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.152 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 5 monclient: authenticate success, global_id 84320 2024-09-16T15:14:02.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 1 -- 172.21.15.134:0/3798398923 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d802d050 con 0x7fa2e0135dc0 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 4 set_mon_vals no callback set 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 1 -- 172.21.15.134:0/3798398923 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d8007d80 con 0x7fa2e0135dc0 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2d77fe640 10 monclient: dump: 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 10 monclient: shutdown 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 -- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 msgr2=0x7fa2e0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 0x7fa2e0136220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009dc0 tx=0x7fa2d802a480 comp rx=0 tx=0).stop 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 -- 172.21.15.134:0/3798398923 shutdown_connections 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0135dc0 0x7fa2e0136220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa2e0137220 0x7fa2e0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/3798398923 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fa2e0136e40 0x7fa2e009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 -- 172.21.15.134:0/3798398923 >> 172.21.15.134:0/3798398923 conn(0x7fa2e00a6110 msgr2=0x7fa2e00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 -- 172.21.15.134:0/3798398923 shutdown_connections 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.141+0000 7fa2e6b76640 1 -- 172.21.15.134:0/3798398923 wait complete. 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.142+0000 7fa2e6b76640 10 monclient: build_initial_monmap 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.142+0000 7fa2e6b76640 10 monclient: monmap: 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:02.156 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.142+0000 7fa2e6b76640 1 Processor -- start 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.142+0000 7fa2e6b76640 1 -- start start 2024-09-16T15:14:02.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.142+0000 7fa2e6b76640 10 monclient: init 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient: _add_conns ranks=[2,5,8,1,3,7,4,0,6] 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-b con 0x7fa2e0135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa2e0136e40 0x7fa2e0121910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-h con 0x7fa2e0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 0x7fa2e0121e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): picked mon.noname-e con 0x7fa2e0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0124290 con 0x7fa2e0135dc0 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e01cc060 con 0x7fa2e0136e40 2024-09-16T15:14:02.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0136880 con 0x7fa2e0137220 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e6b76640 10 monclient(hunting): authenticate will time out at 1578.732362s 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 0x7fa2e0121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:43160/0 (socket says 172.21.15.134:43160) 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 learned_addr learned my addr 172.21.15.134:0/1428076471 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 10 monclient(hunting): get_auth_request con 0x7fa2e0135dc0 auth_method 0 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): get_auth_request con 0x7fa2e0137220 auth_method 0 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 msgr2=0x7fa2e0135120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 msgr2=0x7fa2e0135120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2d7fff640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa2e0136e40 0x7fa2e0121910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2d7fff640 10 monclient(hunting): get_auth_request con 0x7fa2e0136e40 auth_method 0 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient(hunting): handle_auth_done global_id 84329 payload 824 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa2e0136e40 msgr2=0x7fa2e0121910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa2e0136e40 0x7fa2e0121910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 msgr2=0x7fa2e0135120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0135dc0 0x7fa2e0135120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient: _finish_hunting 0 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 monclient: found mon.noname-e 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.143+0000 7fa2e5113640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094d0 con 0x7fa2e0137220 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.145+0000 7fa2e5113640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 msgr2=0x7fa2e0121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.145+0000 7fa2e5113640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 0x7fa2e0121e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8031000 tx=0x7fa2d8004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d80044b0 con 0x7fa2e0137220 2024-09-16T15:14:02.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: dump: 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.162 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8004630 con 0x7fa2e0137220 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 5 monclient: authenticate success, global_id 84329 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 10 monclient: _renew_subs 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d8009a70 con 0x7fa2e0137220 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:02.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2d80047a0 con 0x7fa2e0137220 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d57fa640 10 monclient: dump: 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:02.164 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 10 monclient: _renew_subs 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e01234a0 con 0x7fa2e0137220 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.146+0000 7fa2d6ffd640 4 set_mon_vals no callback set 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa2d8004040 con 0x7fa2e0137220 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e6b76640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0071b20 con 0x7fa2e0122b50 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e0122b50 auth_method 0 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2d57fa640 1 --2- 172.21.15.134:0/1428076471 >> v2:172.21.15.162:6800/980547263 conn(0x7fa2cc0388b0 0x7fa2cc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(614..614 src has 1..614) v4 ==== 3943+0+0 (secure 0 0 0) 0x7fa2d806b350 con 0x7fa2e0137220 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2d7fff640 1 --2- 172.21.15.134:0/1428076471 >> v2:172.21.15.162:6800/980547263 conn(0x7fa2cc0388b0 0x7fa2cc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.147+0000 7fa2d7fff640 10 monclient: get_auth_request con 0x7fa2cc0388b0 auth_method 0 2024-09-16T15:14:02.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.150+0000 7fa2d7fff640 1 --2- 172.21.15.134:0/1428076471 >> v2:172.21.15.162:6800/980547263 conn(0x7fa2cc0388b0 0x7fa2cc03ad70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00a06c0 tx=0x7fa2d0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:2961: done (1 left) 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:2962: rollback oid 47 current snap is 304 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 276 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:2960: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:2960: finishing copy_from to smithi16234619-27 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 821 (ObjNum 1015 snap 294 seq_num 1015) dirty exists 2024-09-16T15:14:02.189 INFO:tasks.rados.rados.0.smithi162.stdout:2960: done (1 left) 2024-09-16T15:14:02.190 INFO:tasks.rados.rados.0.smithi162.stdout:2963: delete oid 43 current snap is 304 2024-09-16T15:14:02.202 INFO:tasks.rados.rados.0.smithi162.stdout:2962: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:14:02.202 INFO:tasks.rados.rados.0.smithi162.stdout:2962: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:14:02.202 INFO:tasks.rados.rados.0.smithi162.stdout:2962: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:14:02.203 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1135 (ObjNum 902 snap 249 seq_num 902) dirty exists 2024-09-16T15:14:02.207 INFO:tasks.rados.rados.0.smithi162.stdout:2962: done (1 left) 2024-09-16T15:14:02.207 INFO:tasks.rados.rados.0.smithi162.stdout:2963: done (0 left) 2024-09-16T15:14:02.207 INFO:tasks.rados.rados.0.smithi162.stdout:2964: delete oid 4 current snap is 304 2024-09-16T15:14:02.214 INFO:tasks.rados.rados.0.smithi162.stdout:2964: done (0 left) 2024-09-16T15:14:02.214 INFO:tasks.rados.rados.0.smithi162.stdout:2965: snap_create 2024-09-16T15:14:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e0122b50 auth_method 0 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:02.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.348+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e0122b50 auth_method 0 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:14:02.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:02.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:02.749+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:03.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e0122b50 auth_method 0 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:03.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:03.550+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:04.049 INFO:tasks.rados.rados.0.smithi162.stdout:2965: done (0 left) 2024-09-16T15:14:04.049 INFO:tasks.rados.rados.0.smithi162.stdout:2966: write (excl) oid 43 current snap is 305 2024-09-16T15:14:04.049 INFO:tasks.rados.rados.0.smithi162.stdout:2966: seq_num 1033 ranges {561631=639872,1882252=633683,3034775=326856} 2024-09-16T15:14:04.056 INFO:tasks.rados.rados.0.smithi162.stdout:2966: writing smithi16234619-43 from 561631 to 1201503 tid 1 2024-09-16T15:14:04.065 INFO:tasks.rados.rados.0.smithi162.stdout:2966: writing smithi16234619-43 from 1882252 to 2515935 tid 2 2024-09-16T15:14:04.070 INFO:tasks.rados.rados.0.smithi162.stdout:2966: writing smithi16234619-43 from 3034775 to 3361631 tid 3 2024-09-16T15:14:04.070 INFO:tasks.rados.rados.0.smithi162.stdout:2967: delete oid 48 current snap is 305 2024-09-16T15:14:04.087 INFO:tasks.rados.rados.0.smithi162.stdout:2967: done (1 left) 2024-09-16T15:14:04.087 INFO:tasks.rados.rados.0.smithi162.stdout:2968: snap_create 2024-09-16T15:14:04.100 INFO:tasks.rados.rados.0.smithi162.stdout:2966: finishing write tid 1 to smithi16234619-43 2024-09-16T15:14:04.102 INFO:tasks.rados.rados.0.smithi162.stdout:2966: finishing write tid 2 to smithi16234619-43 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: finishing write tid 3 to smithi16234619-43 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: finishing write tid 4 to smithi16234619-43 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: finishing write tid 5 to smithi16234619-43 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: oid 43 updating version 0 to 1139 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: oid 43 updating version 1139 to 1140 2024-09-16T15:14:04.103 INFO:tasks.rados.rados.0.smithi162.stdout:2966: oid 43 version 1140 is already newer than 1137 2024-09-16T15:14:04.104 INFO:tasks.rados.rados.0.smithi162.stdout:2966: oid 43 version 1140 is already newer than 1138 2024-09-16T15:14:04.104 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1140 (ObjNum 1033 snap 305 seq_num 1033) dirty exists 2024-09-16T15:14:04.104 INFO:tasks.rados.rados.0.smithi162.stdout:2966: left oid 43 (ObjNum 1033 snap 305 seq_num 1033) 2024-09-16T15:14:05.065 INFO:tasks.rados.rados.0.smithi162.stdout:2966: done (1 left) 2024-09-16T15:14:05.065 INFO:tasks.rados.rados.0.smithi162.stdout:2968: done (0 left) 2024-09-16T15:14:05.065 INFO:tasks.rados.rados.0.smithi162.stdout:2969: snap_create 2024-09-16T15:14:05.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.143+0000 7fa2c7fff640 10 monclient: tick 2024-09-16T15:14:05.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.143+0000 7fa2c7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.143+0000 7fa2c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.143+0000 7fa2c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e0122b50 auth_method 0 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 10 monclient(hunting): handle_auth_done global_id 94106 payload 824 2024-09-16T15:14:05.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.152+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2e0061ae0 con 0x7fa2e0122b50 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.154+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.154+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006fd0 tx=0x7fa2dc017b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2dc015e20 con 0x7fa2e0122b50 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 10 monclient: dump: 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.156 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.157 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa2dc01c070 con 0x7fa2e0122b50 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006fd0 tx=0x7fa2dc017b10 comp rx=0 tx=0).stop 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 msgr2=0x7fa2e0125210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.155+0000 7fa2d57fa640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e0122b50 0x7fa2e0125210 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006fd0 tx=0x7fa2dc017b10 comp rx=0 tx=0).stop 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e6b76640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 0x7fa2e002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e6b76640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e007a680 con 0x7fa2e002b260 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 0x7fa2e002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient: get_auth_request con 0x7fa2e002b260 auth_method 0 2024-09-16T15:14:05.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 10 monclient(hunting): handle_auth_done global_id 94115 payload 824 2024-09-16T15:14:05.165 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.162+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2e009b490 con 0x7fa2e002b260 2024-09-16T15:14:05.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2e4912640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 msgr2=0x7fa2e002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2e4912640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 0x7fa2e002b670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc021000 tx=0x7fa2dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa2dc027ca0 con 0x7fa2e002b260 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 10 monclient: dump: 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.167 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fa2dc027e00 con 0x7fa2e002b260 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 msgr2=0x7fa2e002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 0x7fa2e002b670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc021000 tx=0x7fa2dc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 msgr2=0x7fa2e002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2d57fa640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2e002b260 0x7fa2e002b670 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc021000 tx=0x7fa2dc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:05.169 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.171 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:05.172 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:05.173 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.174 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:05.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.176 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.177 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.178 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.179 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.180 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.181 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.182 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.183 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:05.184 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.185 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:05.186 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.187 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.188 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.189 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.190 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.191 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.192 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.193 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.193 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.193 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:05.193 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 >> v2:172.21.15.162:6800/980547263 conn(0x7fa2cc0388b0 msgr2=0x7fa2cc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.165+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/1428076471 >> v2:172.21.15.162:6800/980547263 conn(0x7fa2cc0388b0 0x7fa2cc03ad70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00a06c0 tx=0x7fa2d0008040 comp rx=0 tx=0).stop 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 10 monclient: shutdown 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 msgr2=0x7fa2e0121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e5113640 1 -- 172.21.15.134:0/1428076471 reap_dead start 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 0x7fa2e0121e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8031000 tx=0x7fa2d8004a40 comp rx=0 tx=0).stop 2024-09-16T15:14:05.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 shutdown_connections 2024-09-16T15:14:05.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 --2- 172.21.15.134:0/1428076471 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa2e0137220 0x7fa2e0121e50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 >> 172.21.15.134:0/1428076471 conn(0x7fa2e00a6110 msgr2=0x7fa2e00a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 shutdown_connections 2024-09-16T15:14:05.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.166+0000 7fa2e6b76640 1 -- 172.21.15.134:0/1428076471 wait complete. 2024-09-16T15:14:05.195 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:05.195 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:14:05.195 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.297+0000 7ff45ce48640 10 monclient: get_monmap_and_config 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.297+0000 7ff45ce48640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.297+0000 7ff45ce48640 10 monclient: monmap: 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:05.298825+0000 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:05.298825+0000 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.299 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:05.300 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.298+0000 7ff45ce48640 1 Processor -- start 2024-09-16T15:14:05.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.298+0000 7ff45ce48640 1 -- start start 2024-09-16T15:14:05.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.298+0000 7ff45ce48640 10 monclient: init 2024-09-16T15:14:05.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient: _add_conns ranks=[0,1,5,3,10,6,7,2,8,9,4] 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff458136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff458137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 0x7ff45813f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-h con 0x7ff45813d380 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff458073250 con 0x7ff458136e40 2024-09-16T15:14:05.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45809ba50 con 0x7ff458137220 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff45809c860 con 0x7ff45813d380 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45ce48640 10 monclient(hunting): authenticate will time out at 1581.888228s 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45659c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff458136e40 0x7ff45809c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:43578/0 (socket says 172.21.15.134:43578) 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff45659c640 1 -- 172.21.15.134:0/968548701 learned_addr learned my addr 172.21.15.134:0/968548701 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 1 --2- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 0x7ff45813f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): get_auth_request con 0x7ff45813d380 auth_method 0 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient(hunting): handle_auth_done global_id 44924 payload 274 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 1 -- 172.21.15.134:0/968548701 >> v1:172.21.15.162:6789/0 conn(0x7ff458137220 legacy=0x7ff458137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 1 -- 172.21.15.134:0/968548701 >> v1:172.21.15.134:6789/0 conn(0x7ff458136e40 legacy=0x7ff45809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 1 monclient: found mon.noname-h 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:14:05.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.299+0000 7ff456d9d640 1 -- 172.21.15.134:0/968548701 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4580a53c0 con 0x7ff45813d380 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.302+0000 7ff456d9d640 1 -- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 msgr2=0x7ff45813f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.302+0000 7ff456d9d640 1 --2- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 0x7ff45813f770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff450009dc0 tx=0x7ff45002a510 comp rx=0 tx=0).ready entity=mon.5 client_cookie=4747ea7567c5362e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.302+0000 7ff45559a640 1 -- 172.21.15.134:0/968548701 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff450007ca0 con 0x7ff45813d380 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.302+0000 7ff45559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: dump: 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.314 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.315 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 5 monclient: authenticate success, global_id 44924 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 1 -- 172.21.15.134:0/968548701 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45002d050 con 0x7ff45813d380 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:05.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 4 set_mon_vals no callback set 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 1 -- 172.21.15.134:0/968548701 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff450007e20 con 0x7ff45813d380 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45559a640 10 monclient: dump: 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 10 monclient: shutdown 2024-09-16T15:14:05.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 -- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 msgr2=0x7ff45813f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 --2- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 0x7ff45813f770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff450009dc0 tx=0x7ff45002a510 comp rx=0 tx=0).stop 2024-09-16T15:14:05.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 -- 172.21.15.134:0/968548701 shutdown_connections 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 --2- 172.21.15.134:0/968548701 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff45813d380 0x7ff45813f770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 -- 172.21.15.134:0/968548701 >> 172.21.15.134:0/968548701 conn(0x7ff4580a6110 msgr2=0x7ff4580a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 -- 172.21.15.134:0/968548701 shutdown_connections 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.303+0000 7ff45ce48640 1 -- 172.21.15.134:0/968548701 wait complete. 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.304+0000 7ff45ce48640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.304+0000 7ff45ce48640 10 monclient: monmap: 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:05.320 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.321 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.304+0000 7ff45ce48640 1 Processor -- start 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.304+0000 7ff45ce48640 1 -- start start 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.304+0000 7ff45ce48640 10 monclient: init 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient: _add_conns ranks=[1,3,0,7,5,8,2,6,4] 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff458136e40 0x7ff458136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-f con 0x7ff458136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff458137220 0x7ff458139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-g con 0x7ff458137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): picked mon.noname-a con 0x7ff45813d380 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:05.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580a6810 con 0x7ff45813d380 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff458137920 con 0x7ff458136e40 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff458121190 con 0x7ff458137220 2024-09-16T15:14:05.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45ce48640 10 monclient(hunting): authenticate will time out at 1581.894397s 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37150/0 (socket says 172.21.15.134:37150) 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 learned_addr learned my addr 172.21.15.134:0/1587863456 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): get_auth_request con 0x7ff45813d380 auth_method 0 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff458136e40 0x7ff458136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff458137220 0x7ff458139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): get_auth_request con 0x7ff458136e40 auth_method 0 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 10 monclient(hunting): get_auth_request con 0x7ff458137220 auth_method 0 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient(hunting): handle_auth_done global_id 74682 payload 824 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff458137220 msgr2=0x7ff458139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff458137220 0x7ff458139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff458136e40 msgr2=0x7ff458136020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff458136e40 0x7ff458136020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 monclient: found mon.noname-a 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450009a70 con 0x7ff45813d380 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.305+0000 7ff455d9b640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff458137220 0x7ff458139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 msgr2=0x7ff45813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff456d9d640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff450009ee0 tx=0x7ff450004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff450007e30 con 0x7ff45813d380 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: dump: 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.328 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4500046c0 con 0x7ff45813d380 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 5 monclient: authenticate success, global_id 74682 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 10 monclient: _renew_subs 2024-09-16T15:14:05.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:05.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45813bad0 con 0x7ff45813d380 2024-09-16T15:14:05.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff450004830 con 0x7ff45813d380 2024-09-16T15:14:05.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:05.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff446ffd640 10 monclient: dump: 2024-09-16T15:14:05.332 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.332 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.332 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.333 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 10 monclient: _renew_subs 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff454d99640 4 set_mon_vals no callback set 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff458136560 con 0x7ff45813d380 2024-09-16T15:14:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.308+0000 7ff45ce48640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45ce48640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 0x7ff45813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff458071b20 con 0x7ff45813aa10 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff450004250 con 0x7ff45813d380 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 0x7ff45813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient: get_auth_request con 0x7ff45813aa10 auth_method 0 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff446ffd640 1 --2- 172.21.15.134:0/1587863456 >> v2:172.21.15.162:6800/980547263 conn(0x7ff438038880 0x7ff43803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(618..618 src has 1..618) v4 ==== 3915+0+0 (secure 0 0 0) 0x7ff45006b150 con 0x7ff45813d380 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 10 monclient(hunting): handle_auth_done global_id 5647 payload 824 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff45659c640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff458061ae0 con 0x7ff45813aa10 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff455d9b640 1 --2- 172.21.15.134:0/1587863456 >> v2:172.21.15.162:6800/980547263 conn(0x7ff438038880 0x7ff43803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.309+0000 7ff455d9b640 10 monclient: get_auth_request con 0x7ff438038880 auth_method 0 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff45659c640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 msgr2=0x7ff45813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff45659c640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 0x7ff45813adf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff44800ea70 tx=0x7ff44800ef30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff44800cc50 con 0x7ff45813aa10 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff446ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.311+0000 7ff446ffd640 10 monclient: dump: 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.337 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff44800cdb0 con 0x7ff45813aa10 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 msgr2=0x7ff45813adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 0x7ff45813adf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff44800ea70 tx=0x7ff44800ef30 comp rx=0 tx=0).stop 2024-09-16T15:14:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 msgr2=0x7ff45813adf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff446ffd640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45813aa10 0x7ff45813adf0 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff44800ea70 tx=0x7ff44800ef30 comp rx=0 tx=0).stop 2024-09-16T15:14:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.312+0000 7ff455d9b640 1 --2- 172.21.15.134:0/1587863456 >> v2:172.21.15.162:6800/980547263 conn(0x7ff438038880 0x7ff43803ad40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff45813c1b0 tx=0x7ff44c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45ce48640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45ce48640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 0x7ff45809e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45ce48640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff45807a680 con 0x7ff45809e520 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 0x7ff45809e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient: get_auth_request con 0x7ff45809e520 auth_method 0 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 10 monclient(hunting): handle_auth_done global_id 5656 payload 824 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.318+0000 7ff45659c640 1 -- 172.21.15.134:0/1587863456 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff45809abc0 con 0x7ff45809e520 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.320+0000 7ff45659c640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 msgr2=0x7ff45809e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.320+0000 7ff45659c640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 0x7ff45809e930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff448017000 tx=0x7ff448004940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.320+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff448015cc0 con 0x7ff45809e520 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 10 monclient: dump: 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.344 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7ff448015850 con 0x7ff45809e520 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 msgr2=0x7ff45809e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 0x7ff45809e930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff448017000 tx=0x7ff448004940 comp rx=0 tx=0).stop 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 msgr2=0x7ff45809e930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff446ffd640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff45809e520 0x7ff45809e930 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff448017000 tx=0x7ff448004940 comp rx=0 tx=0).stop 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:05.346 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:05.347 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.348 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.349 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:05.350 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.351 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:05.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.353 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:05.353 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.353 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:05.353 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.354 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.356 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.356 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.356 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.356 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.357 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.357 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.357 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.357 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.357 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.358 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.359 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:05.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.364 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.370 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.374 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:05.375 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.376 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.377 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.378 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:05.378 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:05.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 >> v2:172.21.15.162:6800/980547263 conn(0x7ff438038880 msgr2=0x7ff43803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 --2- 172.21.15.134:0/1587863456 >> v2:172.21.15.162:6800/980547263 conn(0x7ff438038880 0x7ff43803ad40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff45813c1b0 tx=0x7ff44c009040 comp rx=0 tx=0).stop 2024-09-16T15:14:05.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 10 monclient: shutdown 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 msgr2=0x7ff45813a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff456d9d640 1 -- 172.21.15.134:0/1587863456 reap_dead start 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff450009ee0 tx=0x7ff450004a40 comp rx=0 tx=0).stop 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 shutdown_connections 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 --2- 172.21.15.134:0/1587863456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff45813d380 0x7ff45813a4d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 >> 172.21.15.134:0/1587863456 conn(0x7ff4580a6110 msgr2=0x7ff4580a7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 shutdown_connections 2024-09-16T15:14:05.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.321+0000 7ff45ce48640 1 -- 172.21.15.134:0/1587863456 wait complete. 2024-09-16T15:14:05.382 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:05.382 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:14:05.382 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.479+0000 7fb29e0f0640 10 monclient: get_monmap_and_config 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.479+0000 7fb29e0f0640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.479+0000 7fb29e0f0640 10 monclient: monmap: 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.481 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:05.480670+0000 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:05.480670+0000 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.482 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:05.483 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.480+0000 7fb29e0f0640 1 Processor -- start 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.480+0000 7fb29e0f0640 1 -- start start 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.480+0000 7fb29e0f0640 10 monclient: init 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient: _add_conns ranks=[1,9,4,8,3,6,5,2,7,10,0] 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb298136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb298137220 0x7fb298137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-a con 0x7fb298137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:05.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-c con 0x7fb29813c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb298073250 con 0x7fb298136e40 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980a56e0 con 0x7fb298137220 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb29809abc0 con 0x7fb29813c370 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb29e0f0640 10 monclient(hunting): authenticate will time out at 1582.070371s 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb298137220 0x7fb298137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55402/0 (socket says 172.21.15.134:55402) 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb298137220 0x7fb298137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:37156/0 (socket says 172.21.15.134:37156) 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 1 -- 172.21.15.134:0/1038644475 learned_addr learned my addr 172.21.15.134:0/1038644475 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): get_auth_request con 0x7fb29813c370 auth_method 0 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): get_auth_request con 0x7fb298137220 auth_method 0 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient(hunting): handle_auth_done global_id 74691 payload 274 2024-09-16T15:14:05.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 -- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 msgr2=0x7fb29813e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 -- 172.21.15.134:0/1038644475 >> v1:172.21.15.162:6789/0 conn(0x7fb298136e40 legacy=0x7fb29809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 monclient: found mon.noname-a 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb28ffff640 1 -- 172.21.15.134:0/1038644475 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29809ba50 con 0x7fb298137220 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.481+0000 7fb297fff640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:05.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb28ffff640 1 -- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb298137220 msgr2=0x7fb298137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb28ffff640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb298137220 0x7fb298137600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51df23e259dea18d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 1 -- 172.21.15.134:0/1038644475 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb28002ad30 con 0x7fb298137220 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: dump: 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.498 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.499 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb29e0f0640 5 monclient: authenticate success, global_id 74691 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb29e0f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 1 -- 172.21.15.134:0/1038644475 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2800307b0 con 0x7fb298137220 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb29e0f0640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 1 -- 172.21.15.134:0/1038644475 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb280030d90 con 0x7fb298137220 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb29e0f0640 4 set_mon_vals no callback set 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.484+0000 7fb296ffd640 10 monclient: dump: 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.501 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 10 monclient: shutdown 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 -- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb298137220 msgr2=0x7fb298137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb298137220 0x7fb298137600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2800097a0 tx=0x7fb28002a010 comp rx=0 tx=0).stop 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 -- 172.21.15.134:0/1038644475 shutdown_connections 2024-09-16T15:14:05.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb29813c370 0x7fb29813e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/1038644475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb298137220 0x7fb298137600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 -- 172.21.15.134:0/1038644475 >> 172.21.15.134:0/1038644475 conn(0x7fb2980a6110 msgr2=0x7fb2980a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 -- 172.21.15.134:0/1038644475 shutdown_connections 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 1 -- 172.21.15.134:0/1038644475 wait complete. 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.485+0000 7fb29e0f0640 10 monclient: monmap: 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.504 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.505 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 Processor -- start 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 -- start start 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient: init 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient: _add_conns ranks=[3,7,8,6,5,2,0,1,4] 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb298136e40 0x7fb298135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-g con 0x7fb298136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:05.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb298137220 0x7fb2981363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-i con 0x7fb298137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 0x7fb2981cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): picked mon.noname-e con 0x7fb29813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298122920 con 0x7fb298136e40 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980a7a60 con 0x7fb298137220 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980a74a0 con 0x7fb29813c370 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb29e0f0640 10 monclient(hunting): authenticate will time out at 1582.075766s 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 0x7fb2981cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 0x7fb2981cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52920/0 (socket says 172.21.15.134:52920) 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 learned_addr learned my addr 172.21.15.134:0/3484105208 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): get_auth_request con 0x7fb29813c370 auth_method 0 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb297fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb298137220 0x7fb2981363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.486+0000 7fb2977fe640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb298136e40 0x7fb298135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 10 monclient(hunting): get_auth_request con 0x7fb298137220 auth_method 0 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb2977fe640 10 monclient(hunting): get_auth_request con 0x7fb298136e40 auth_method 0 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb2977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb2977fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb2977fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 10 monclient(hunting): handle_auth_done global_id 84338 payload 824 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb298137220 msgr2=0x7fb2981363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb298137220 0x7fb2981363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb298136e40 msgr2=0x7fb298135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb298136e40 0x7fb298135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 monclient: found mon.noname-e 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290009b30 con 0x7fb29813c370 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb298137220 0x7fb2981363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.487+0000 7fb2977fe640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb298136e40 0x7fb298135e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 msgr2=0x7fb2981cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb297fff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 0x7fb2981cb570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ef90 tx=0x7fb29000c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb290018070 con 0x7fb29813c370 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: dump: 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.512 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 5 monclient: authenticate success, global_id 84338 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29000f040 con 0x7fb29813c370 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 10 monclient: _renew_subs 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb280009400 con 0x7fb29813c370 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb290014d60 con 0x7fb29813c370 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb2967fc640 4 set_mon_vals no callback set 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb294ff9640 10 monclient: dump: 2024-09-16T15:14:05.514 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.515 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 10 monclient: _renew_subs 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.489+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb298121fa0 con 0x7fb29813c370 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb29000cd30 con 0x7fb29813c370 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb294ff9640 1 --2- 172.21.15.134:0/3484105208 >> v2:172.21.15.162:6800/980547263 conn(0x7fb278038880 0x7fb27803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb29e0f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 0x7fb2981cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb298071b20 con 0x7fb2980a2c20 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(618..618 src has 1..618) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fb29004a260 con 0x7fb29813c370 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 0x7fb2981cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient: get_auth_request con 0x7fb2980a2c20 auth_method 0 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb2977fe640 1 --2- 172.21.15.134:0/3484105208 >> v2:172.21.15.162:6800/980547263 conn(0x7fb278038880 0x7fb27803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb2977fe640 10 monclient: get_auth_request con 0x7fb278038880 auth_method 0 2024-09-16T15:14:05.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 10 monclient(hunting): handle_auth_done global_id 94110 payload 824 2024-09-16T15:14:05.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.490+0000 7fb28ffff640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb298061ae0 con 0x7fb2980a2c20 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb28ffff640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 msgr2=0x7fb2981cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 0x7fb2981cbab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb280034000 tx=0x7fb2800046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb2800044a0 con 0x7fb2980a2c20 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb2977fe640 1 --2- 172.21.15.134:0/3484105208 >> v2:172.21.15.162:6800/980547263 conn(0x7fb278038880 0x7fb27803ad40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb288003eb0 tx=0x7fb288009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 10 monclient: dump: 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.521 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb280030d40 con 0x7fb2980a2c20 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 msgr2=0x7fb2981cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 0x7fb2981cbab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb280034000 tx=0x7fb2800046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 msgr2=0x7fb2981cbab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.493+0000 7fb294ff9640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb2980a2c20 0x7fb2981cbab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.499+0000 7fb29e0f0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.499+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 0x7fb2981cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.499+0000 7fb29e0f0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.499+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb298122620 con 0x7fb298121040 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 0x7fb2981cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient: get_auth_request con 0x7fb298121040 auth_method 0 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 10 monclient(hunting): handle_auth_done global_id 94119 payload 824 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.500+0000 7fb28ffff640 1 -- 172.21.15.134:0/3484105208 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb29807a680 con 0x7fb298121040 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb28ffff640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 msgr2=0x7fb2981cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb28ffff640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 0x7fb2981cbff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280008030 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb280032440 con 0x7fb298121040 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 10 monclient: dump: 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.524 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.525 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fb280003b40 con 0x7fb298121040 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 msgr2=0x7fb2981cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 0x7fb2981cbff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280008030 comp rx=0 tx=0).stop 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 msgr2=0x7fb2981cbff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.502+0000 7fb294ff9640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb298121040 0x7fb2981cbff0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb280000c00 tx=0x7fb280008030 comp rx=0 tx=0).stop 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:05.527 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.528 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.529 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.530 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.531 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:05.532 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.533 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.534 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.538 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.539 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.540 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.541 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.546 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.547 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.549 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:05.550 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.551 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.553 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.554 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:05.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:05.556 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:05.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 >> v2:172.21.15.162:6800/980547263 conn(0x7fb278038880 msgr2=0x7fb27803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/3484105208 >> v2:172.21.15.162:6800/980547263 conn(0x7fb278038880 0x7fb27803ad40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb288003eb0 tx=0x7fb288009040 comp rx=0 tx=0).stop 2024-09-16T15:14:05.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 10 monclient: shutdown 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 msgr2=0x7fb2981cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 --2- 172.21.15.134:0/3484105208 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb29813c370 0x7fb2981cb570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb29000ef90 tx=0x7fb29000c490 comp rx=0 tx=0).stop 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 reap_dead start 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb297fff640 1 -- 172.21.15.134:0/3484105208 reap_dead start 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 shutdown_connections 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 >> 172.21.15.134:0/3484105208 conn(0x7fb2980a6110 msgr2=0x7fb29813a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 shutdown_connections 2024-09-16T15:14:05.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.503+0000 7fb29e0f0640 1 -- 172.21.15.134:0/3484105208 wait complete. 2024-09-16T15:14:05.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:05.560 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:14:05.560 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:14:05.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.656+0000 7fce90879640 10 monclient: get_monmap_and_config 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.656+0000 7fce90879640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.656+0000 7fce90879640 10 monclient: monmap: 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:05.657392+0000 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:05.657392+0000 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.658 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:05.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 Processor -- start 2024-09-16T15:14:05.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 -- start start 2024-09-16T15:14:05.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient: init 2024-09-16T15:14:05.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient: _add_conns ranks=[1,6,7,4,10,9,5,2,0,8,3] 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fce8c135090 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): picked mon.noname-d con 0x7fce8c071530 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fce8c139ef0 0x7fce8c13c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): picked mon.noname-i con 0x7fce8c139ef0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c073250 con 0x7fce8c135090 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c135b80 con 0x7fce8c071530 2024-09-16T15:14:05.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0a3550 con 0x7fce8c139ef0 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce90879640 10 monclient(hunting): authenticate will time out at 1582.246863s 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57114/0 (socket says 172.21.15.134:57114) 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 1 -- 172.21.15.134:0/1945916837 learned_addr learned my addr 172.21.15.134:0/1945916837 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 10 monclient(hunting): get_auth_request con 0x7fce8c071530 auth_method 0 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.657+0000 7fce89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce8ad9d640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fce8c139ef0 0x7fce8c13c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce8ad9d640 10 monclient(hunting): get_auth_request con 0x7fce8c139ef0 auth_method 0 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient(hunting): handle_auth_done global_id 94128 payload 274 2024-09-16T15:14:05.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 1 -- 172.21.15.134:0/1945916837 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fce8c139ef0 msgr2=0x7fce8c13c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fce8c139ef0 0x7fce8c13c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 1 -- 172.21.15.134:0/1945916837 >> v1:172.21.15.162:6789/0 conn(0x7fce8c135090 legacy=0x7fce8c135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 1 monclient: found mon.noname-d 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.658+0000 7fce89d9b640 1 -- 172.21.15.134:0/1945916837 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0a3330 con 0x7fce8c071530 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.660+0000 7fce89d9b640 1 -- 172.21.15.134:0/1945916837 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 msgr2=0x7fce8c071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.660+0000 7fce89d9b640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce740098a0 tx=0x7fce7402a0b0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=6422a5942cc047bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 1 -- 172.21.15.134:0/1945916837 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7402add0 con 0x7fce8c071530 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: dump: 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.671 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.672 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 1 -- 172.21.15.134:0/1945916837 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740307b0 con 0x7fce8c071530 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 1 -- 172.21.15.134:0/1945916837 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce74030dd0 con 0x7fce8c071530 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 5 monclient: authenticate success, global_id 94128 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 4 set_mon_vals no callback set 2024-09-16T15:14:05.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce8959a640 10 monclient: dump: 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.674 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 10 monclient: shutdown 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 -- 172.21.15.134:0/1945916837 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 msgr2=0x7fce8c071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce740098a0 tx=0x7fce7402a0b0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 -- 172.21.15.134:0/1945916837 shutdown_connections 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fce8c139ef0 0x7fce8c13c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 --2- 172.21.15.134:0/1945916837 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c071530 0x7fce8c071950 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 -- 172.21.15.134:0/1945916837 >> 172.21.15.134:0/1945916837 conn(0x7fce8c0a7410 msgr2=0x7fce8c0a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 -- 172.21.15.134:0/1945916837 shutdown_connections 2024-09-16T15:14:05.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.661+0000 7fce90879640 1 -- 172.21.15.134:0/1945916837 wait complete. 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.662+0000 7fce90879640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.662+0000 7fce90879640 10 monclient: monmap: 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:05.677 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.662+0000 7fce90879640 1 Processor -- start 2024-09-16T15:14:05.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.662+0000 7fce90879640 1 -- start start 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.662+0000 7fce90879640 10 monclient: init 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient: _add_conns ranks=[5,6,0,4,3,1,8,7,2] 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c071530 0x7fce8c1c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): picked mon.noname-h con 0x7fce8c071530 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 0x7fce8c133860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): picked mon.noname-d con 0x7fce8c135090 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce8c139ef0 0x7fce8c133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): picked mon.noname-a con 0x7fce8c139ef0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:05.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1c9840 con 0x7fce8c139ef0 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c11fce0 con 0x7fce8c071530 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c11ff30 con 0x7fce8c135090 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce90879640 10 monclient(hunting): authenticate will time out at 1582.252383s 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 0x7fce8c133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce8c139ef0 0x7fce8c133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 0x7fce8c133860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57122/0 (socket says 172.21.15.134:57122) 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 -- 172.21.15.134:0/264885951 learned_addr learned my addr 172.21.15.134:0/264885951 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): get_auth_request con 0x7fce8c135090 auth_method 0 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): get_auth_request con 0x7fce8c139ef0 auth_method 0 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce8a59c640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c071530 0x7fce8c1c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient(hunting): handle_auth_done global_id 94137 payload 824 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c071530 msgr2=0x7fce8c1c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c071530 0x7fce8c1c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce8c139ef0 msgr2=0x7fce8c133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce8c139ef0 0x7fce8c133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 monclient: found mon.noname-d 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:05.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.663+0000 7fce89d9b640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740094d0 con 0x7fce8c135090 2024-09-16T15:14:05.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.665+0000 7fce89d9b640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 msgr2=0x7fce8c133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.665+0000 7fce89d9b640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 0x7fce8c133860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce740090b0 tx=0x7fce740046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce740044b0 con 0x7fce8c135090 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: dump: 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.690 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74030d40 con 0x7fce8c135090 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 5 monclient: authenticate success, global_id 94137 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 10 monclient: _renew_subs 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c0a4c20 con 0x7fce8c135090 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7403e450 con 0x7fce8c135090 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 10 monclient: dump: 2024-09-16T15:14:05.692 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.693 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce88d99640 4 set_mon_vals no callback set 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 10 monclient: _renew_subs 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c0a6ad0 con 0x7fce8c135090 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fce74003610 con 0x7fce8c135090 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.666+0000 7fce90879640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 0x7fce8c138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c06a3f0 con 0x7fce8c120070 2024-09-16T15:14:05.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce82ffd640 1 --2- 172.21.15.134:0/264885951 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 0x7fce8c138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient: get_auth_request con 0x7fce8c120070 auth_method 0 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(618..618 src has 1..618) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fce7406ac70 con 0x7fce8c135090 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 10 monclient(hunting): handle_auth_done global_id 84347 payload 824 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8a59c640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce8c061ae0 con 0x7fce8c120070 2024-09-16T15:14:05.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8ad9d640 1 --2- 172.21.15.134:0/264885951 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.667+0000 7fce8ad9d640 10 monclient: get_auth_request con 0x7fce6c038880 auth_method 0 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce8a59c640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 msgr2=0x7fce8c138e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce8a59c640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 0x7fce8c138e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce8400dbb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce8400de00 con 0x7fce8c120070 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce82ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce82ffd640 10 monclient: dump: 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.696 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.697 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.669+0000 7fce8ad9d640 1 --2- 172.21.15.134:0/264885951 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06fd20 tx=0x7fce7c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fce8400bb40 con 0x7fce8c120070 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 msgr2=0x7fce8c138e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 0x7fce8c138e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce8400dbb0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 msgr2=0x7fce8c138e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.670+0000 7fce82ffd640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c120070 0x7fce8c138e80 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce8400dbb0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce90879640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce90879640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 0x7fce8c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce90879640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c07a680 con 0x7fce8c02b260 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 0x7fce8c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient: get_auth_request con 0x7fce8c02b260 auth_method 0 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 10 monclient(hunting): handle_auth_done global_id 84356 payload 824 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.676+0000 7fce8a59c640 1 -- 172.21.15.134:0/264885951 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce8c06f5e0 con 0x7fce8c02b260 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce8a59c640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 msgr2=0x7fce8c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce8a59c640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 0x7fce8c02b670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce84003a70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce84020040 con 0x7fce8c02b260 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce82ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce82ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.678+0000 7fce82ffd640 10 monclient: dump: 2024-09-16T15:14:05.701 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.702 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fce8400bec0 con 0x7fce8c02b260 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 msgr2=0x7fce8c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 0x7fce8c02b670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce84003a70 comp rx=0 tx=0).stop 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 msgr2=0x7fce8c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce82ffd640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce8c02b260 0x7fce8c02b670 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d6f0 tx=0x7fce84003a70 comp rx=0 tx=0).stop 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.723 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.724 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.725 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.726 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.731 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.732 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.733 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:05.733 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:05.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 msgr2=0x7fce6c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 --2- 172.21.15.134:0/264885951 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06fd20 tx=0x7fce7c008040 comp rx=0 tx=0).stop 2024-09-16T15:14:05.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 10 monclient: shutdown 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 msgr2=0x7fce8c133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 --2- 172.21.15.134:0/264885951 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c135090 0x7fce8c133860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce740090b0 tx=0x7fce740046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce8ad9d640 1 -- 172.21.15.134:0/264885951 reap_dead start 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce8ad9d640 1 -- 172.21.15.134:0/264885951 reap_dead start 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 shutdown_connections 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 >> 172.21.15.134:0/264885951 conn(0x7fce8c0a7410 msgr2=0x7fce8c0a5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 shutdown_connections 2024-09-16T15:14:05.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.679+0000 7fce90879640 1 -- 172.21.15.134:0/264885951 wait complete. 2024-09-16T15:14:05.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:05.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:14:05.736 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.831+0000 7f0d44d84640 10 monclient: get_monmap_and_config 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.831+0000 7f0d44d84640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.831+0000 7f0d44d84640 10 monclient: monmap: 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:05.833355+0000 2024-09-16T15:14:05.834 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:05.833355+0000 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.835 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 Processor -- start 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 -- start start 2024-09-16T15:14:05.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient: init 2024-09-16T15:14:05.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient: _add_conns ranks=[2,6,5,1,3,10,7,0,4,8,9] 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-b con 0x7f0d40136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-d con 0x7f0d40137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0d40135dc0 0x7f0d40136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-h con 0x7f0d40135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4009abc0 con 0x7f0d40136e40 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4009ba50 con 0x7f0d40135dc0 2024-09-16T15:14:05.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400a53c0 con 0x7f0d40137220 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d44d84640 10 monclient(hunting): authenticate will time out at 1582.422719s 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57134/0 (socket says 172.21.15.134:57134) 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:43256/0 (socket says 172.21.15.134:43256) 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2289555309 learned_addr learned my addr 172.21.15.134:0/2289555309 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request con 0x7f0d40137220 auth_method 0 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): get_auth_request con 0x7f0d40136e40 auth_method 0 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.832+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.849 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3ed9d640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0d40135dc0 0x7f0d40136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3ed9d640 10 monclient(hunting): get_auth_request con 0x7f0d40135dc0 auth_method 0 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_done global_id 94146 payload 274 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2289555309 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0d40135dc0 msgr2=0x7f0d40136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0d40135dc0 0x7f0d40136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 msgr2=0x7f0d4009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 monclient: found mon.noname-d 2024-09-16T15:14:05.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2289555309 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40073250 con 0x7f0d40137220 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.833+0000 7f0d3e59c640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.835+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 msgr2=0x7f0d40137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.835+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098a0 tx=0x7f0d3002a080 comp rx=0 tx=0).ready entity=mon.6 client_cookie=938535c84f098852 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 1 -- 172.21.15.134:0/2289555309 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d3002ade0 con 0x7f0d40137220 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: dump: 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.851 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.852 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 1 -- 172.21.15.134:0/2289555309 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d300307b0 con 0x7f0d40137220 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 5 monclient: authenticate success, global_id 94146 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:05.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 1 -- 172.21.15.134:0/2289555309 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d30030d90 con 0x7f0d40137220 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 4 set_mon_vals no callback set 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d3d59a640 10 monclient: dump: 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.854 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 10 monclient: shutdown 2024-09-16T15:14:05.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 -- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 msgr2=0x7f0d40137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d300098a0 tx=0x7f0d3002a080 comp rx=0 tx=0).stop 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 -- 172.21.15.134:0/2289555309 shutdown_connections 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0d40135dc0 0x7f0d40136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f0d40137220 0x7f0d40137600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2289555309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d40136e40 0x7f0d4009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 -- 172.21.15.134:0/2289555309 >> 172.21.15.134:0/2289555309 conn(0x7f0d400a6110 msgr2=0x7f0d400a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 -- 172.21.15.134:0/2289555309 shutdown_connections 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.836+0000 7f0d44d84640 1 -- 172.21.15.134:0/2289555309 wait complete. 2024-09-16T15:14:05.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.837+0000 7f0d44d84640 10 monclient: build_initial_monmap 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.837+0000 7f0d44d84640 10 monclient: monmap: 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:05.857 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.837+0000 7f0d44d84640 1 Processor -- start 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.837+0000 7f0d44d84640 1 -- start start 2024-09-16T15:14:05.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.837+0000 7f0d44d84640 10 monclient: init 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient: _add_conns ranks=[8,3,7,1,6,4,0,5,2] 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-e con 0x7f0d40135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d40136e40 0x7f0d40134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-g con 0x7f0d40136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0d40137220 0x7f0d40121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): picked mon.noname-i con 0x7f0d40137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400a8a50 con 0x7f0d40136e40 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40136890 con 0x7f0d40137220 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40136340 con 0x7f0d40135dc0 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d44d84640 10 monclient(hunting): authenticate will time out at 1582.428459s 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52940/0 (socket says 172.21.15.134:52940) 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 -- 172.21.15.134:0/2748765652 learned_addr learned my addr 172.21.15.134:0/2748765652 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): get_auth_request con 0x7f0d40135dc0 auth_method 0 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d40136e40 0x7f0d40134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request con 0x7f0d40136e40 auth_method 0 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0d40137220 0x7f0d40121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): get_auth_request con 0x7f0d40137220 auth_method 0 2024-09-16T15:14:05.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient(hunting): handle_auth_done global_id 84365 payload 824 2024-09-16T15:14:05.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0d40137220 msgr2=0x7f0d40121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0d40137220 0x7f0d40121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d40136e40 msgr2=0x7f0d40134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0d40136e40 0x7f0d40134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 monclient: found mon.noname-e 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.838+0000 7f0d3e59c640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34009400 con 0x7f0d40135dc0 2024-09-16T15:14:05.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d3e59c640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 msgr2=0x7f0d40134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d3e59c640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400ebe0 tx=0x7f0d3400c520 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d34009760 con 0x7f0d40135dc0 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: dump: 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.865 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.866 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: _finish_auth 0 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 5 monclient: authenticate success, global_id 84365 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3400ce40 con 0x7f0d40135dc0 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 10 monclient: _renew_subs 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d300094d0 con 0x7f0d40135dc0 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d3400eee0 con 0x7f0d40135dc0 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d3cd99640 4 set_mon_vals no callback set 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d2effd640 10 monclient: dump: 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.868 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.869 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 10 monclient: _renew_subs 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.841+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d40125210 con 0x7f0d40135dc0 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3e59c640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400ebe0 tx=0x7f0d3400c520 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f0d34013470 mgrmap(e 5) v1 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0d34013470 con 0x7f0d40135dc0 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d44d84640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 0x7f0d400a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40073620 con 0x7f0d400a1a20 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d2effd640 1 --2- 172.21.15.134:0/2748765652 >> v2:172.21.15.162:6800/980547263 conn(0x7f0d20038880 0x7f0d2003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 0x7f0d400a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient: get_auth_request con 0x7f0d400a1a20 auth_method 0 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3ed9d640 1 --2- 172.21.15.134:0/2748765652 >> v2:172.21.15.162:6800/980547263 conn(0x7f0d20038880 0x7f0d2003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3ed9d640 10 monclient: get_auth_request con 0x7f0d20038880 auth_method 0 2024-09-16T15:14:05.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_done global_id 44983 payload 824 2024-09-16T15:14:05.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.842+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d40061ae0 con 0x7f0d400a1a20 2024-09-16T15:14:05.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d3ed9d640 1 --2- 172.21.15.134:0/2748765652 >> v2:172.21.15.162:6800/980547263 conn(0x7f0d20038880 0x7f0d2003ad40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d40123000 tx=0x7f0d38009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 msgr2=0x7f0d400a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 0x7f0d400a1e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d30034000 tx=0x7f0d300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d30030d40 con 0x7f0d400a1a20 2024-09-16T15:14:05.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 10 monclient: dump: 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0d30003610 con 0x7f0d400a1a20 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 msgr2=0x7f0d400a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 0x7f0d400a1e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d30034000 tx=0x7f0d300046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 msgr2=0x7f0d400a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.845+0000 7f0d2effd640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a1a20 0x7f0d400a1e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.851+0000 7f0d44d84640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.851+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 0x7f0d40121d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.851+0000 7f0d44d84640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.851+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40071b20 con 0x7f0d400a2340 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 0x7f0d40121d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient: get_auth_request con 0x7f0d400a2340 auth_method 0 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:05.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 10 monclient(hunting): handle_auth_done global_id 44992 payload 824 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.852+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2748765652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d4007a680 con 0x7f0d400a2340 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.854+0000 7f0d3dd9b640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 msgr2=0x7f0d40121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.854+0000 7f0d3dd9b640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 0x7f0d40121d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d30033000 tx=0x7f0d30009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0d30032440 con 0x7f0d400a2340 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 10 monclient: dump: 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:05.879 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f0d3003dad0 con 0x7f0d400a2340 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 msgr2=0x7f0d40121d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 0x7f0d40121d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d30033000 tx=0x7f0d30009240 comp rx=0 tx=0).stop 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 msgr2=0x7f0d40121d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:05.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0d400a2340 0x7f0d40121d60 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d30033000 tx=0x7f0d30009240 comp rx=0 tx=0).stop 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:05.882 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:05.883 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.884 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.885 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:05.886 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:05.887 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:05.888 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.889 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:05.891 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.893 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.894 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.900 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.906 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:05.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:05.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:05.909 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:05.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:05.911 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 >> v2:172.21.15.162:6800/980547263 conn(0x7f0d20038880 msgr2=0x7f0d2003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2748765652 >> v2:172.21.15.162:6800/980547263 conn(0x7f0d20038880 0x7f0d2003ad40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d40123000 tx=0x7f0d38009820 comp rx=0 tx=0).stop 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 10 monclient: shutdown 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 msgr2=0x7f0d40134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 1 --2- 172.21.15.134:0/2748765652 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0d40135dc0 0x7f0d40134a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400ebe0 tx=0x7f0d3400c520 comp rx=0 tx=0).stop 2024-09-16T15:14:05.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d3ed9d640 1 -- 172.21.15.134:0/2748765652 reap_dead start 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d2effd640 1 -- 172.21.15.134:0/2748765652 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(618..618 src has 1..618) v4 ==== 3915+0+0 (unknown 0 0 0) 0x7f0d3404a490 con 0x7f0d40135dc0 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d3ed9d640 1 -- 172.21.15.134:0/2748765652 reap_dead start 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.855+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 shutdown_connections 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.856+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 >> 172.21.15.134:0/2748765652 conn(0x7f0d400a6110 msgr2=0x7f0d4006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.856+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 shutdown_connections 2024-09-16T15:14:05.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:05.856+0000 7f0d44d84640 1 -- 172.21.15.134:0/2748765652 wait complete. 2024-09-16T15:14:05.914 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:05.914 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:14:05.915 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.010+0000 7f04309e2640 10 monclient: get_monmap_and_config 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.010+0000 7f04309e2640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.010+0000 7f04309e2640 10 monclient: monmap: 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.013 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:06.012459+0000 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:06.012459+0000 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.014 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 Processor -- start 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 -- start start 2024-09-16T15:14:06.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient: init 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient: _add_conns ranks=[1,4,8,10,7,5,3,9,0,2,6] 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f042c136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-c con 0x7f042c137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-e con 0x7f042c13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c073250 con 0x7f042c136e40 2024-09-16T15:14:06.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0a56e0 con 0x7f042c137220 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f042c09abc0 con 0x7f042c13c370 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f04309e2640 10 monclient(hunting): authenticate will time out at 1582.601787s 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:55414/0 (socket says 172.21.15.134:55414) 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 1 -- 172.21.15.134:0/4067179256 learned_addr learned my addr 172.21.15.134:0/4067179256 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 10 monclient(hunting): get_auth_request con 0x7f042c137220 auth_method 0 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f0429d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f042ad9d640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f042ad9d640 10 monclient(hunting): get_auth_request con 0x7f042c13c370 auth_method 0 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f042ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f042ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.011+0000 7f042ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient(hunting): handle_auth_done global_id 5674 payload 274 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 1 -- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 msgr2=0x7f042c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 1 -- 172.21.15.134:0/4067179256 >> v1:172.21.15.162:6789/0 conn(0x7f042c136e40 legacy=0x7f042c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 1 monclient: found mon.noname-c 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f0429d9b640 1 -- 172.21.15.134:0/4067179256 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c09ba50 con 0x7f042c137220 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.012+0000 7f042ad9d640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.014+0000 7f0429d9b640 1 -- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 msgr2=0x7f042c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.014+0000 7f0429d9b640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f042002a0d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=423ee838d7055461 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 1 -- 172.21.15.134:0/4067179256 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f042002adf0 con 0x7f042c137220 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: dump: 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.028 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 1 -- 172.21.15.134:0/4067179256 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200307b0 con 0x7f042c137220 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 5 monclient: authenticate success, global_id 5674 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 1 -- 172.21.15.134:0/4067179256 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0420030d90 con 0x7f042c137220 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f042959a640 10 monclient: dump: 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.030 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 4 set_mon_vals no callback set 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 10 monclient: shutdown 2024-09-16T15:14:06.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 -- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 msgr2=0x7f042c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f042002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 -- 172.21.15.134:0/4067179256 shutdown_connections 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 --2- 172.21.15.134:0/4067179256 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f042c137220 0x7f042c137600 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 -- 172.21.15.134:0/4067179256 >> 172.21.15.134:0/4067179256 conn(0x7f042c0a6110 msgr2=0x7f042c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 -- 172.21.15.134:0/4067179256 shutdown_connections 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.015+0000 7f04309e2640 1 -- 172.21.15.134:0/4067179256 wait complete. 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.016+0000 7f04309e2640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.016+0000 7f04309e2640 10 monclient: monmap: 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:06.032 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.033 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.016+0000 7f04309e2640 1 Processor -- start 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.016+0000 7f04309e2640 1 -- start start 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.016+0000 7f04309e2640 10 monclient: init 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient: _add_conns ranks=[7,3,8,4,0,5,2,1,6] 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f042c136e40 0x7f042c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-i con 0x7f042c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c137220 0x7f042c1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-g con 0x7f042c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): picked mon.noname-e con 0x7f042c13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:06.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c122280 con 0x7f042c137220 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f042c122840 con 0x7f042c136e40 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0a7af0 con 0x7f042c13c370 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f04309e2640 10 monclient(hunting): authenticate will time out at 1582.607310s 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52960/0 (socket says 172.21.15.134:52960) 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 learned_addr learned my addr 172.21.15.134:0/1810708798 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): get_auth_request con 0x7f042c13c370 auth_method 0 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f0429d9b640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c137220 0x7f042c1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f0429d9b640 10 monclient(hunting): get_auth_request con 0x7f042c137220 auth_method 0 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f0429d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f0429d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f0429d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f042c136e40 0x7f042c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient(hunting): handle_auth_done global_id 84383 payload 824 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f042c136e40 msgr2=0x7f042c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f042c136e40 0x7f042c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c137220 msgr2=0x7f042c1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c137220 0x7f042c1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 monclient: found mon.noname-e 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009a70 con 0x7f042c13c370 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042a59c640 10 monclient: get_auth_request con 0x7f042c136e40 auth_method 0 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.017+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f042c136e40 0x7f042c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.019+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 msgr2=0x7f042c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.019+0000 7f042ad9d640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0424009e10 tx=0x7f042400c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0424009cb0 con 0x7f042c13c370 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: dump: 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.038 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400f040 con 0x7f042c13c370 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 5 monclient: authenticate success, global_id 84383 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 monclient: _renew_subs 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04200094d0 con 0x7f042c13c370 2024-09-16T15:14:06.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0424014d60 con 0x7f042c13c370 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 10 monclient: dump: 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.040 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f0428d99640 4 set_mon_vals no callback set 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 monclient: _renew_subs 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c121fa0 con 0x7f042c13c370 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0424014480 con 0x7f042c13c370 2024-09-16T15:14:06.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 0x7f042c135c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.020+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c071b20 con 0x7f042c135890 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f041affd640 1 --2- 172.21.15.134:0/1810708798 >> v2:172.21.15.162:6800/980547263 conn(0x7f040c038880 0x7f040c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(618..618 src has 1..618) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f042404af60 con 0x7f042c13c370 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 0x7f042c135c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient: get_auth_request con 0x7f042c135890 auth_method 0 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f0429d9b640 1 --2- 172.21.15.134:0/1810708798 >> v2:172.21.15.162:6800/980547263 conn(0x7f040c038880 0x7f040c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f0429d9b640 10 monclient: get_auth_request con 0x7f040c038880 auth_method 0 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 10 monclient(hunting): handle_auth_done global_id 84288 payload 824 2024-09-16T15:14:06.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.021+0000 7f042a59c640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f042c061ae0 con 0x7f042c135890 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.023+0000 7f0429d9b640 1 --2- 172.21.15.134:0/1810708798 >> v2:172.21.15.162:6800/980547263 conn(0x7f040c038880 0x7f040c03ad40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f042002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.023+0000 7f042a59c640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 msgr2=0x7f042c135c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.023+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 0x7f042c135c70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c00d9b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f041c012c80 con 0x7f042c135890 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 10 monclient: dump: 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.047 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f041c012de0 con 0x7f042c135890 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 msgr2=0x7f042c135c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 0x7f042c135c70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c00d9b0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 msgr2=0x7f042c135c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.024+0000 7f041affd640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c135890 0x7f042c135c70 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c00d9b0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f04309e2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f04309e2640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 0x7f042c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f04309e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0677b0 con 0x7f042c1361b0 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 0x7f042c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient: get_auth_request con 0x7f042c1361b0 auth_method 0 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.030+0000 7f042a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.031+0000 7f042a59c640 10 monclient(hunting): handle_auth_done global_id 84297 payload 824 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.031+0000 7f042a59c640 1 -- 172.21.15.134:0/1810708798 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f042c07a680 con 0x7f042c1361b0 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f042a59c640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 msgr2=0x7f042c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f042a59c640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 0x7f042c09d4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c003760 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f041c028070 con 0x7f042c1361b0 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f041affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f041affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:06.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.033+0000 7f041affd640 10 monclient: dump: 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.051 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f041c020040 con 0x7f042c1361b0 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 msgr2=0x7f042c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 0x7f042c09d4f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c003760 comp rx=0 tx=0).stop 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 msgr2=0x7f042c09d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f041affd640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f042c1361b0 0x7f042c09d4f0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d4f0 tx=0x7f041c003760 comp rx=0 tx=0).stop 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:06.053 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.054 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.055 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.056 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.057 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.058 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:06.059 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:06.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:06.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.068 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.069 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.071 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:06.072 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.073 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.075 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.076 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.078 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:06.078 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:06.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 >> v2:172.21.15.162:6800/980547263 conn(0x7f040c038880 msgr2=0x7f040c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 --2- 172.21.15.134:0/1810708798 >> v2:172.21.15.162:6800/980547263 conn(0x7f040c038880 0x7f040c03ad40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f042002e000 comp rx=0 tx=0).stop 2024-09-16T15:14:06.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 10 monclient: shutdown 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 msgr2=0x7f042c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0424009e10 tx=0x7f042400c490 comp rx=0 tx=0).stop 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f042ad9d640 1 -- 172.21.15.134:0/1810708798 reap_dead start 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 shutdown_connections 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 --2- 172.21.15.134:0/1810708798 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f042c13c370 0x7f042c1206e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 >> 172.21.15.134:0/1810708798 conn(0x7f042c0a6110 msgr2=0x7f042c13b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 shutdown_connections 2024-09-16T15:14:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.034+0000 7f04309e2640 1 -- 172.21.15.134:0/1810708798 wait complete. 2024-09-16T15:14:06.081 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:06.081 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:14:06.081 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:14:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:2969: done (0 left) 2024-09-16T15:14:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:2970: read oid 4 snap -1 2024-09-16T15:14:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:2970: expect deleted 2024-09-16T15:14:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:2971: delete oid 29 current snap is 307 2024-09-16T15:14:06.082 INFO:tasks.rados.rados.0.smithi162.stdout:2970: done (1 left) 2024-09-16T15:14:06.083 INFO:tasks.rados.rados.0.smithi162.stdout:2971: done (0 left) 2024-09-16T15:14:06.083 INFO:tasks.rados.rados.0.smithi162.stdout:2972: snap_create 2024-09-16T15:14:06.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 10 monclient: get_monmap_and_config 2024-09-16T15:14:06.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 10 monclient: monmap: 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:06.180071+0000 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:06.180071+0000 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.181 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:06.182 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 1 Processor -- start 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 1 -- start start 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.178+0000 7f8c24bed640 10 monclient: init 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient: _add_conns ranks=[6,9,8,4,2,5,7,1,0,3,10] 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-d con 0x7f8c20136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-a con 0x7f8c20137290 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-e con 0x7f8c20133d10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200a5560 con 0x7f8c20137290 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2009b240 con 0x7f8c20136eb0 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2009b520 con 0x7f8c20133d10 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c24bed640 10 monclient(hunting): authenticate will time out at 1582.769412s 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54230/0 (socket says 172.21.15.134:54230) 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:49702/0 (socket says 172.21.15.134:49702) 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 1 -- 172.21.15.134:0/2063091603 learned_addr learned my addr 172.21.15.134:0/2063091603 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58966/0 (socket says 172.21.15.134:58966) 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): get_auth_request con 0x7f8c20133d10 auth_method 0 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request con 0x7f8c20137290 auth_method 0 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): get_auth_request con 0x7f8c20136eb0 auth_method 0 2024-09-16T15:14:06.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_done global_id 94155 payload 274 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 -- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 msgr2=0x7f8c20134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 -- 172.21.15.134:0/2063091603 >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 msgr2=0x7f8c20137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/2063091603 >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 monclient: found mon.noname-d 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1e59c640 1 -- 172.21.15.134:0/2063091603 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20073250 con 0x7f8c20136eb0 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1ed9d640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.179+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/2063091603 >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1e59c640 1 -- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 msgr2=0x7f8c2009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c08029fd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2ddb53437d024e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 1 -- 172.21.15.134:0/2063091603 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c0802ad30 con 0x7f8c20136eb0 2024-09-16T15:14:06.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: dump: 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.199 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 1 -- 172.21.15.134:0/2063091603 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080307b0 con 0x7f8c20136eb0 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c24bed640 5 monclient: authenticate success, global_id 94155 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c24bed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:06.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c24bed640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 1 -- 172.21.15.134:0/2063091603 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c08030d90 con 0x7f8c20136eb0 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c24bed640 4 set_mon_vals no callback set 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.182+0000 7f8c1d59a640 10 monclient: dump: 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.201 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.202 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 10 monclient: shutdown 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 -- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 msgr2=0x7f8c2009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c08029fd0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 -- 172.21.15.134:0/2063091603 shutdown_connections 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f8c20133d10 0x7f8c20134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 --2- 172.21.15.134:0/2063091603 >> v2:172.21.15.134:3300/0 conn(0x7f8c20137290 0x7f8c20137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 --2- 172.21.15.134:0/2063091603 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8c20136eb0 0x7f8c2009bf00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 -- 172.21.15.134:0/2063091603 >> 172.21.15.134:0/2063091603 conn(0x7f8c200a99a0 msgr2=0x7f8c200a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 -- 172.21.15.134:0/2063091603 shutdown_connections 2024-09-16T15:14:06.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.183+0000 7f8c24bed640 1 -- 172.21.15.134:0/2063091603 wait complete. 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient: monmap: 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:06.204 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 Processor -- start 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 -- start start 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient: init 2024-09-16T15:14:06.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient: _add_conns ranks=[0,2,3,4,7,6,8,5,1] 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-a con 0x7f8c20133d10 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c20136eb0 0x7f8c20132e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-b con 0x7f8c20136eb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c20137290 0x7f8c200a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): picked mon.noname-g con 0x7f8c20137290 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200a2ba0 con 0x7f8c20133d10 2024-09-16T15:14:06.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200a7610 con 0x7f8c20136eb0 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20136430 con 0x7f8c20137290 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c24bed640 10 monclient(hunting): authenticate will time out at 1582.774917s 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c1e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.184+0000 7f8c1dd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c20136eb0 0x7f8c20132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58976/0 (socket says 172.21.15.134:58976) 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 -- 172.21.15.134:0/277418612 learned_addr learned my addr 172.21.15.134:0/277418612 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): get_auth_request con 0x7f8c20133d10 auth_method 0 2024-09-16T15:14:06.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request con 0x7f8c20136eb0 auth_method 0 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient(hunting): handle_auth_done global_id 74718 payload 824 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c20137290 msgr2=0x7f8c200a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1ed9d640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c20137290 0x7f8c200a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c20137290 0x7f8c200a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c20136eb0 msgr2=0x7f8c20132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c20136eb0 0x7f8c20132e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 monclient: found mon.noname-a 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1e59c640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18009b30 con 0x7f8c20133d10 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.185+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c20136eb0 0x7f8c20132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c1e59c640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 msgr2=0x7f8c20132940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c08035040 tx=0x7f8c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c08003610 con 0x7f8c20133d10 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: dump: 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.210 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.211 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080037b0 con 0x7f8c20133d10 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 5 monclient: authenticate success, global_id 74718 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 10 monclient: _renew_subs 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10009400 con 0x7f8c20133d10 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c08003d90 con 0x7f8c20133d10 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c16ffd640 10 monclient: dump: 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.213 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c1cd99640 4 set_mon_vals no callback set 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 10 monclient: _renew_subs 2024-09-16T15:14:06.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.187+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08009400 con 0x7f8c20133d10 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1e59c640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c08035040 tx=0x7f8c080046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f8c08039040 mgrmap(e 5) v1 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c24bed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8c08039040 con 0x7f8c20133d10 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c24bed640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 0x7f8c200a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2004e300 con 0x7f8c200a0ec0 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c16ffd640 1 --2- 172.21.15.134:0/277418612 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c00038880 0x7f8c0003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 0x7f8c200a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1dd9b640 10 monclient: get_auth_request con 0x7f8c200a0ec0 auth_method 0 2024-09-16T15:14:06.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.188+0000 7f8c1ed9d640 1 --2- 172.21.15.134:0/277418612 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c00038880 0x7f8c0003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1ed9d640 10 monclient: get_auth_request con 0x7f8c00038880 auth_method 0 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_done global_id 44942 payload 824 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.189+0000 7f8c1dd9b640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c2011fe20 con 0x7f8c200a0ec0 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c1ed9d640 1 --2- 172.21.15.134:0/277418612 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c00038880 0x7f8c0003ad40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c200a03d0 tx=0x7f8c1800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c1dd9b640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 msgr2=0x7f8c200a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 0x7f8c200a12a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009e90 tx=0x7f8c1000f610 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c10014ac0 con 0x7f8c200a0ec0 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c16ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.191+0000 7f8c16ffd640 10 monclient: dump: 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.217 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8c10014c20 con 0x7f8c200a0ec0 2024-09-16T15:14:06.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:06.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 msgr2=0x7f8c200a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 0x7f8c200a12a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009e90 tx=0x7f8c1000f610 comp rx=0 tx=0).stop 2024-09-16T15:14:06.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 msgr2=0x7f8c200a12a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.192+0000 7f8c16ffd640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a0ec0 0x7f8c200a12a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c24bed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c24bed640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 0x7f8c20134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c24bed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20071b20 con 0x7f8c200a17e0 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 0x7f8c20134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient: get_auth_request con 0x7f8c200a17e0 auth_method 0 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 10 monclient(hunting): handle_auth_done global_id 44951 payload 824 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.198+0000 7f8c1dd9b640 1 -- 172.21.15.134:0/277418612 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8c20061ae0 con 0x7f8c200a17e0 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c1dd9b640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 msgr2=0x7f8c20134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c1dd9b640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 0x7f8c20134180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000fbb0 tx=0x7f8c100046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8c10022bd0 con 0x7f8c200a17e0 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 10 monclient: dump: 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.222 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f8c10022d30 con 0x7f8c200a17e0 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 msgr2=0x7f8c20134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 0x7f8c20134180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000fbb0 tx=0x7f8c100046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 msgr2=0x7f8c20134180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.201+0000 7f8c16ffd640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8c200a17e0 0x7f8c20134180 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000fbb0 tx=0x7f8c100046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:06.224 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:06.225 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:06.226 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:06.227 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:06.228 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.229 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:06.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.232 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.238 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.240 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:06.241 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.242 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:06.243 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.244 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:06.245 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.246 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.248 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.249 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.250 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.251 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.251 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.251 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:06.251 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:06.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c00038880 msgr2=0x7f8c0003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 --2- 172.21.15.134:0/277418612 >> v2:172.21.15.162:6800/980547263 conn(0x7f8c00038880 0x7f8c0003ad40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c200a03d0 tx=0x7f8c1800b040 comp rx=0 tx=0).stop 2024-09-16T15:14:06.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 10 monclient: shutdown 2024-09-16T15:14:06.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 msgr2=0x7f8c20132940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c08035040 tx=0x7f8c080046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c1ed9d640 1 -- 172.21.15.134:0/277418612 reap_dead start 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c16ffd640 1 -- 172.21.15.134:0/277418612 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(619..619 src has 1..619) v4 ==== 3915+0+0 (unknown 0 0 0) 0x7f8c0806a4d0 con 0x7f8c20133d10 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 shutdown_connections 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 --2- 172.21.15.134:0/277418612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c20133d10 0x7f8c20132940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 >> 172.21.15.134:0/277418612 conn(0x7f8c200a99a0 msgr2=0x7f8c2006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 shutdown_connections 2024-09-16T15:14:06.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.202+0000 7f8c24bed640 1 -- 172.21.15.134:0/277418612 wait complete. 2024-09-16T15:14:06.253 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:06.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:14:06.254 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:14:06.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.350+0000 7f7b56145640 10 monclient: get_monmap_and_config 2024-09-16T15:14:06.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.350+0000 7f7b56145640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.350+0000 7f7b56145640 10 monclient: monmap: 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:06.352292+0000 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:06.352292+0000 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.353 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:06.354 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 Processor -- start 2024-09-16T15:14:06.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 -- start start 2024-09-16T15:14:06.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient: init 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient: _add_conns ranks=[3,5,0,6,2,1,9,10,8,7,4] 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 0x7f7b5009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-g con 0x7f7b50136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7b50137220 0x7f7b50137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-h con 0x7f7b50137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7b50135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b50073250 con 0x7f7b50135dc0 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5009abc0 con 0x7f7b50136e40 2024-09-16T15:14:06.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5009ba50 con 0x7f7b50137220 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b56145640 10 monclient(hunting): authenticate will time out at 1582.941601s 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b4ffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7b50135dc0 0x7f7b50136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32934/0 (socket says 172.21.15.134:32934) 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b4ffff640 1 -- 172.21.15.134:0/4135014721 learned_addr learned my addr 172.21.15.134:0/4135014721 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 0x7f7b5009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.351+0000 7f7b4effd640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7b50137220 0x7f7b50137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request con 0x7f7b50136e40 auth_method 0 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): get_auth_request con 0x7f7b50137220 auth_method 0 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_done global_id 84306 payload 274 2024-09-16T15:14:06.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7b50137220 msgr2=0x7f7b50137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7b50137220 0x7f7b50137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/4135014721 >> v1:172.21.15.134:6789/0 conn(0x7f7b50135dc0 legacy=0x7f7b50136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 1 monclient: found mon.noname-g 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.352+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/4135014721 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500a53c0 con 0x7f7b50136e40 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 msgr2=0x7f7b5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 0x7f7b5009c150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b400097d0 tx=0x7f7b4002a000 comp rx=0 tx=0).ready entity=mon.3 client_cookie=4a9bb45d3e4f0d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 1 -- 172.21.15.134:0/4135014721 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b4002ad20 con 0x7f7b50136e40 2024-09-16T15:14:06.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: dump: 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.366 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 1 -- 172.21.15.134:0/4135014721 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b400307b0 con 0x7f7b50136e40 2024-09-16T15:14:06.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 5 monclient: authenticate success, global_id 84306 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 1 -- 172.21.15.134:0/4135014721 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b4002aea0 con 0x7f7b50136e40 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 4 set_mon_vals no callback set 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b4e7fc640 10 monclient: dump: 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.368 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 10 monclient: shutdown 2024-09-16T15:14:06.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 -- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 msgr2=0x7f7b5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 0x7f7b5009c150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b400097d0 tx=0x7f7b4002a000 comp rx=0 tx=0).stop 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 -- 172.21.15.134:0/4135014721 shutdown_connections 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7b50137220 0x7f7b50137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 --2- 172.21.15.134:0/4135014721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b50136e40 0x7f7b5009c150 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 -- 172.21.15.134:0/4135014721 >> 172.21.15.134:0/4135014721 conn(0x7f7b500a6110 msgr2=0x7f7b500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 -- 172.21.15.134:0/4135014721 shutdown_connections 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.355+0000 7f7b56145640 1 -- 172.21.15.134:0/4135014721 wait complete. 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.356+0000 7f7b56145640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.356+0000 7f7b56145640 10 monclient: monmap: 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:06.370 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.356+0000 7f7b56145640 1 Processor -- start 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.356+0000 7f7b56145640 1 -- start start 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.356+0000 7f7b56145640 10 monclient: init 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient: _add_conns ranks=[7,0,4,2,3,6,8,1,5] 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135dc0 0x7f7b501349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-i con 0x7f7b50135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-a con 0x7f7b50136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7b50137220 0x7f7b5011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): picked mon.noname-c con 0x7f7b50137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500a8ae0 con 0x7f7b50136e40 2024-09-16T15:14:06.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50136930 con 0x7f7b50137220 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b501363e0 con 0x7f7b50135dc0 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b56145640 10 monclient(hunting): authenticate will time out at 1582.947346s 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58980/0 (socket says 172.21.15.134:58980) 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 -- 172.21.15.134:0/591834604 learned_addr learned my addr 172.21.15.134:0/591834604 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): get_auth_request con 0x7f7b50136e40 auth_method 0 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7b50137220 0x7f7b5011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): get_auth_request con 0x7f7b50137220 auth_method 0 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135dc0 0x7f7b501349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request con 0x7f7b50135dc0 auth_method 0 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient(hunting): handle_auth_done global_id 74727 payload 824 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135dc0 msgr2=0x7f7b501349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135dc0 0x7f7b501349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7b50137220 msgr2=0x7f7b5011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7b50137220 0x7f7b5011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 monclient: found mon.noname-a 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4effd640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40009400 con 0x7f7b50136e40 2024-09-16T15:14:06.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.357+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135dc0 0x7f7b501349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.359+0000 7f7b4effd640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 msgr2=0x7f7b50134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.359+0000 7f7b4effd640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e8d0 tx=0x7f7b4400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b440094f0 con 0x7f7b50136e40 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: dump: 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.381 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.382 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 5 monclient: authenticate success, global_id 74727 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44013470 con 0x7f7b50136e40 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b44013a50 con 0x7f7b50136e40 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b37fff640 10 monclient: dump: 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.384 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 10 monclient: _renew_subs 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b4dffb640 4 set_mon_vals no callback set 2024-09-16T15:14:06.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b50122410 con 0x7f7b50136e40 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 10 monclient: _renew_subs 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.360+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b50120ef0 con 0x7f7b50136e40 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7b44013c30 con 0x7f7b50136e40 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b56145640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b56145640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 0x7f7b5002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50071b20 con 0x7f7b50135470 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b37fff640 1 --2- 172.21.15.134:0/591834604 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b3c038880 0x7f7b3c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(619..619 src has 1..619) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f7b4404a3d0 con 0x7f7b50136e40 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 0x7f7b5002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient: get_auth_request con 0x7f7b50135470 auth_method 0 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4ffff640 1 --2- 172.21.15.134:0/591834604 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b3c038880 0x7f7b3c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4ffff640 10 monclient: get_auth_request con 0x7f7b3c038880 auth_method 0 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_done global_id 54781 payload 824 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.361+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b50061ae0 con 0x7f7b50135470 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b4ffff640 1 --2- 172.21.15.134:0/591834604 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b3c038880 0x7f7b3c03ad40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b480055a0 tx=0x7f7b48005530 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 msgr2=0x7f7b5002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 0x7f7b5002b260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006b550 tx=0x7f7b400046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b40004460 con 0x7f7b50135470 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 10 monclient: dump: 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.388 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7b40003610 con 0x7f7b50135470 2024-09-16T15:14:06.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 msgr2=0x7f7b5002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 0x7f7b5002b260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006b550 tx=0x7f7b400046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 msgr2=0x7f7b5002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.364+0000 7f7b37fff640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b50135470 0x7f7b5002b260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.370+0000 7f7b56145640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.370+0000 7f7b56145640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 0x7f7b5002bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.370+0000 7f7b56145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.370+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5007a680 con 0x7f7b5002b7a0 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 0x7f7b5002bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient: get_auth_request con 0x7f7b5002b7a0 auth_method 0 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 10 monclient(hunting): handle_auth_done global_id 54790 payload 824 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.371+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/591834604 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b5009b490 con 0x7f7b5002b7a0 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b4f7fe640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 msgr2=0x7f7b5002bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b4f7fe640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 0x7f7b5002bb80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000c00 tx=0x7f7b40009190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b4004b040 con 0x7f7b5002b7a0 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b37fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b37fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.373+0000 7f7b37fff640 10 monclient: dump: 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.392 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.393 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.395 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f7b40051430 con 0x7f7b5002b7a0 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 msgr2=0x7f7b5002bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 0x7f7b5002bb80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b40000c00 tx=0x7f7b40009190 comp rx=0 tx=0).stop 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 msgr2=0x7f7b5002bb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:06.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b37fff640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b5002b7a0 0x7f7b5002bb80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:06.397 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:14:06.398 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.399 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.400 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:06.401 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:06.402 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.403 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.404 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.405 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.406 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.407 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.408 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:06.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.411 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:06.412 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:06.413 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.414 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.415 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.416 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.418 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:06.419 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:06.420 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.421 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:06.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:06.423 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b3c038880 msgr2=0x7f7b3c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 --2- 172.21.15.134:0/591834604 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b3c038880 0x7f7b3c03ad40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b480055a0 tx=0x7f7b48005530 comp rx=0 tx=0).stop 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 10 monclient: shutdown 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 msgr2=0x7f7b50134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e8d0 tx=0x7f7b4400ed90 comp rx=0 tx=0).stop 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b4ffff640 1 -- 172.21.15.134:0/591834604 reap_dead start 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 shutdown_connections 2024-09-16T15:14:06.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 --2- 172.21.15.134:0/591834604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b50136e40 0x7f7b50134f30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 >> 172.21.15.134:0/591834604 conn(0x7f7b500a6110 msgr2=0x7f7b5009f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 shutdown_connections 2024-09-16T15:14:06.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.374+0000 7f7b56145640 1 -- 172.21.15.134:0/591834604 wait complete. 2024-09-16T15:14:06.426 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:06.426 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:14:06.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.523+0000 7f4dcafc7640 10 monclient: get_monmap_and_config 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.523+0000 7f4dcafc7640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.523+0000 7f4dcafc7640 10 monclient: monmap: 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:06.525786+0000 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:06.525786+0000 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.526 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:06.527 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.524+0000 7f4dcafc7640 1 Processor -- start 2024-09-16T15:14:06.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.524+0000 7f4dcafc7640 1 -- start start 2024-09-16T15:14:06.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.524+0000 7f4dcafc7640 10 monclient: init 2024-09-16T15:14:06.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient: _add_conns ranks=[9,5,4,2,6,8,1,10,0,7,3] 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dc4136030 0x7f4dc4136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-a con 0x7f4dc4136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4dc409bfe0 0x7f4dc409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-h con 0x7f4dc409bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 0x7f4dc41370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-c con 0x7f4dc4136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc409b600 con 0x7f4dc4136030 2024-09-16T15:14:06.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc40a5740 con 0x7f4dc4136c60 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc40a52c0 con 0x7f4dc409bfe0 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dcafc7640 10 monclient(hunting): authenticate will time out at 1583.115250s 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dc4136030 0x7f4dc4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dc4136030 0x7f4dc4136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58988/0 (socket says 172.21.15.134:58988) 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 -- 172.21.15.134:0/3981808060 learned_addr learned my addr 172.21.15.134:0/3981808060 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): get_auth_request con 0x7f4dc4136030 auth_method 0 2024-09-16T15:14:06.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 0x7f4dc41370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): get_auth_request con 0x7f4dc4136c60 auth_method 0 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4dc409bfe0 0x7f4dc409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): get_auth_request con 0x7f4dc409bfe0 auth_method 0 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_done global_id 74736 payload 274 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 -- 172.21.15.134:0/3981808060 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4dc409bfe0 msgr2=0x7f4dc409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4dc409bfe0 0x7f4dc409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 -- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 msgr2=0x7f4dc41370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 0x7f4dc41370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 monclient: found mon.noname-a 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc8d63640 1 -- 172.21.15.134:0/3981808060 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc4073250 con 0x7f4dc4136030 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.525+0000 7f4dc9564640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 0x7f4dc41370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:06.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc8d63640 1 -- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 msgr2=0x7f4dc4136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc4136410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009870 tx=0x7f4db402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1c44f337780b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 1 -- 172.21.15.134:0/3981808060 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4db402ae00 con 0x7f4dc4136030 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: dump: 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.540 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dcafc7640 5 monclient: authenticate success, global_id 74736 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dcafc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 1 -- 172.21.15.134:0/3981808060 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db40307b0 con 0x7f4dc4136030 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dcafc7640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dcafc7640 4 set_mon_vals no callback set 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 1 -- 172.21.15.134:0/3981808060 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4db4030d90 con 0x7f4dc4136030 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.542 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.528+0000 7f4dc37fe640 10 monclient: dump: 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.543 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 10 monclient: shutdown 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 -- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 msgr2=0x7f4dc4136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc4136410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009870 tx=0x7f4db402a0a0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 -- 172.21.15.134:0/3981808060 shutdown_connections 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4dc4136c60 0x7f4dc41370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4dc409bfe0 0x7f4dc409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/3981808060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc4136410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 -- 172.21.15.134:0/3981808060 >> 172.21.15.134:0/3981808060 conn(0x7f4dc40a9ea0 msgr2=0x7f4dc40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 -- 172.21.15.134:0/3981808060 shutdown_connections 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 1 -- 172.21.15.134:0/3981808060 wait complete. 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 10 monclient: build_initial_monmap 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.529+0000 7f4dcafc7640 10 monclient: monmap: 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:06.545 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:06.546 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 Processor -- start 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 -- start start 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient: init 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient: _add_conns ranks=[3,0,8,2,6,5,7,1,4] 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dc409bfe0 0x7f4dc411fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-g con 0x7f4dc409bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-a con 0x7f4dc4136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:06.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4dc4136c60 0x7f4dc4120500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): picked mon.noname-e con 0x7f4dc4136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc4135700 con 0x7f4dc4136030 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc4120fc0 con 0x7f4dc409bfe0 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc409fd70 con 0x7f4dc4136c60 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dcafc7640 10 monclient(hunting): authenticate will time out at 1583.120769s 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58996/0 (socket says 172.21.15.134:58996) 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 1 -- 172.21.15.134:0/11585767 learned_addr learned my addr 172.21.15.134:0/11585767 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): get_auth_request con 0x7f4dc4136030 auth_method 0 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4dc4136c60 0x7f4dc4120500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 10 monclient(hunting): get_auth_request con 0x7f4dc4136c60 auth_method 0 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dc409bfe0 0x7f4dc411fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc8d63640 10 monclient(hunting): get_auth_request con 0x7f4dc409bfe0 auth_method 0 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.530+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc9564640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:06.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc8d63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 10 monclient(hunting): handle_auth_done global_id 74745 payload 824 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 -- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4dc4136c60 msgr2=0x7f4dc4120500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4dc4136c60 0x7f4dc4120500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 -- 172.21.15.134:0/11585767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dc409bfe0 msgr2=0x7f4dc411fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dc409bfe0 0x7f4dc411fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 monclient: found mon.noname-a 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.531+0000 7f4dc3fff640 1 -- 172.21.15.134:0/11585767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc009400 con 0x7f4dc4136030 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc3fff640 1 -- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 msgr2=0x7f4dc411ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc3fff640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4db800efc0 tx=0x7f4db800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4db8009e40 con 0x7f4dc4136030 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: dump: 2024-09-16T15:14:06.554 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.555 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db800f040 con 0x7f4dc4136030 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 5 monclient: authenticate success, global_id 74745 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 10 monclient: _renew_subs 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db40094d0 con 0x7f4dc4136030 2024-09-16T15:14:06.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4db8014d60 con 0x7f4dc4136030 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc17fa640 10 monclient: dump: 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:06.557 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 10 monclient: _renew_subs 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dc2ffd640 4 set_mon_vals no callback set 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.533+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc40a8220 con 0x7f4dc4136030 2024-09-16T15:14:06.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dc3fff640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4db800efc0 tx=0x7f4db800c490 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f4db8014480 mgrmap(e 5) v1 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4db8014480 con 0x7f4dc4136030 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dc17fa640 1 --2- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 0x7f4db003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dcafc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dcafc7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dcafc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc41bed10 con 0x7f4dc4136030 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 0x7f4db003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.534+0000 7f4dc8d63640 10 monclient: get_auth_request con 0x7f4db0038880 auth_method 0 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.537+0000 7f4dc8d63640 1 --2- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 0x7f4db003ad40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40a51e0 tx=0x7f4db402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:06.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:06.539+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(619..619 src has 1..619) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f4db804adc0 con 0x7f4dc4136030 2024-09-16T15:14:07.077 INFO:tasks.rados.rados.0.smithi162.stdout:2972: done (0 left) 2024-09-16T15:14:07.077 INFO:tasks.rados.rados.0.smithi162.stdout:2973: read oid 46 snap -1 2024-09-16T15:14:07.077 INFO:tasks.rados.rados.0.smithi162.stdout:2973: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:14:07.077 INFO:tasks.rados.rados.0.smithi162.stdout:2974: write (excl) oid 32 current snap is 308 2024-09-16T15:14:07.078 INFO:tasks.rados.rados.0.smithi162.stdout:2974: seq_num 1036 ranges {612052=581972,1657698=497363,2717044=695008} 2024-09-16T15:14:07.083 INFO:tasks.rados.rados.0.smithi162.stdout:2974: writing smithi16234619-32 from 612052 to 1194024 tid 1 2024-09-16T15:14:07.089 INFO:tasks.rados.rados.0.smithi162.stdout:2974: writing smithi16234619-32 from 1657698 to 2155061 tid 2 2024-09-16T15:14:07.096 INFO:tasks.rados.rados.0.smithi162.stdout:2974: writing smithi16234619-32 from 2717044 to 3412052 tid 3 2024-09-16T15:14:07.096 INFO:tasks.rados.rados.0.smithi162.stdout:2975: snap_remove snap 230 2024-09-16T15:14:07.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.363+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4dc41bed10 con 0x7f4dc4136030 2024-09-16T15:14:07.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.363+0000 7f4dc17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:07.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.363+0000 7f4dc17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:07.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.589+0000 7f4dcafc7640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:14:07.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.589+0000 7f4dcafc7640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:14:07.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.589+0000 7f4dcafc7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:07.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.589+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4dc41bed10 con 0x7f4dc4136030 2024-09-16T15:14:07.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f4dc17fa640 1 -- 172.21.15.134:0/11585767 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4dc41bed10 con 0x7f4dc4136030 2024-09-16T15:14:07.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f4dc17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:14:07.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f4dc17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=2,health=12,kv=12,logm=55} crc {auth=1542570681,config=2586702516,health=871165309,kv=1090497770,logm=2063295236}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=18,config=2,health=11,kv=12,logm=57} crc {auth=803565618,config=2586702516,health=2707396398,kv=1090497770,logm=889499991}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=11,kv=10,logm=60} crc {auth=2670384256,config=2586702516,health=3509097966,kv=3324725102,logm=1219811593}) 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.590+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.596 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=17,config=2,health=11,kv=12,logm=58} crc {auth=2383440312,config=2586702516,health=3078229769,kv=1090497770,logm=3810737577}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=18,config=2,health=11,kv=12,logm=57} crc {auth=2790380171,config=2586702516,health=3509097966,kv=1090497770,logm=4066399715}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=19,config=2,health=12,kv=12,logm=55} crc {auth=1542570681,config=2586702516,health=871165309,kv=1090497770,logm=3868775736}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=17,config=2,health=9,kv=12,logm=60} crc {auth=2025343558,config=2586702516,health=1705500206,kv=1090497770,logm=823053379}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=940993761,config=2586702516,health=4238543781,kv=1090497770,logm=1793819121}) 2024-09-16T15:14:07.597 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.591+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=18,config=2,health=12,kv=12,logm=56} crc {auth=477055332,config=2586702516,health=871165309,kv=1090497770,logm=2044881312}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2453405336}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1255888054}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4001608538}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=565139533}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.598 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1531282754}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1484896947}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3037753019}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696074289}) 2024-09-16T15:14:07.599 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.592+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2915850658}) 2024-09-16T15:14:07.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 msgr2=0x7f4db003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:07.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 0x7f4db003ad40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40a51e0 tx=0x7f4db402e000 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 10 monclient: shutdown 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 msgr2=0x7f4dc411ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4db800efc0 tx=0x7f4db800c490 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 shutdown_connections 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4dc4136c60 0x7f4dc4120500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dc4136030 0x7f4dc411ffc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db0038880 0x7f4db003ad40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 --2- 172.21.15.134:0/11585767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dc409bfe0 0x7f4dc411fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 >> 172.21.15.134:0/11585767 conn(0x7f4dc40a9ea0 msgr2=0x7f4dc40a8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 shutdown_connections 2024-09-16T15:14:07.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:07.593+0000 7f4dcafc7640 1 -- 172.21.15.134:0/11585767 wait complete. 2024-09-16T15:14:07.616 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=705058120}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3666502887}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1324867152}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3970941720}) 2024-09-16T15:14:07.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2638292498}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1050010416}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=311272361}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=816174688}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.594+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3399009922}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3752369597}) 2024-09-16T15:14:07.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=825032339}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4178963678}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1243810748}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2194516126}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=207988044}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4022015947}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3739675977}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.597+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=600725411}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2730774659}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=900522255}) 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=589215744}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4039248554}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1380849361}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=11054195}) 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.623 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=995306585}) 2024-09-16T15:14:07.623 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649478008}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.598+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2794776088}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1025471003}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=313043493}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.624 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1645702039}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1206310445}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2338483500}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3438979722}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1829550517}) 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.625 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=224669293}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.600+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3856476937}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1208605384}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2462908250}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2194329561}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2639640019}) 2024-09-16T15:14:07.626 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3228634342}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2683776888}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2081819681}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3724883181}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.602+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3442859466}) 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.627 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1551316765}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3486542048}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3801997383}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=198881331}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.628 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3383264512}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1103552284}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2539564811}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3503932421}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.603+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3988254842}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069558817}) 2024-09-16T15:14:07.629 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4258766095}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1494846554}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3578823822}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1775305910}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3412686104}) 2024-09-16T15:14:07.630 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2996790224}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1366079278}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.605+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=349828200}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4015127549}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3382232936}) 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.631 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1281297687}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3347751583}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1939774180}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3447731371}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2524743900}) 2024-09-16T15:14:07.632 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.633 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3369952727}) 2024-09-16T15:14:07.633 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.606+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1042986504}) 2024-09-16T15:14:07.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38994843}) 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2870912555}) 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1695798661}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3283657435}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=755758695}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117291388}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2882337572}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3194267667}) 2024-09-16T15:14:07.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.608+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=771771730}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3747583037}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=672399735}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3164506982}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2383042622}) 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=396177292}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2189754256}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1229119848}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3698407579}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.609+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3115522211}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3791739788}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2666338639}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=997353728}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1489704336}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2757328762}) 2024-09-16T15:14:07.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3023962255}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2956117140}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150467616}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.611+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2238579596}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1768342368}) 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3431353335}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3433173998}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=723929052}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2948599304}) 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1096337206}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=131699346}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2377282662}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.612+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=522703709}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3444759450}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=397594972}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.613+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2840078708}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=810026883}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1539445691}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1072737362}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=768056197}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2483394351}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.614+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=183705956}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2144240994}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1536500996}) 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.663 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1696675797}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=545047110}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3937221735}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1216289690}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.664 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1203330972}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1169077529}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.615+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2595537277}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2258108334}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3502861823}) 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.665 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=163395734}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.616+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=846881847}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1473205127}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=636826028}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.666 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3492726288}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=551346214}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.617+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3859600586}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=693350958}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=868572330}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1489957886}) 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.667 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2526831749}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3916936181}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470613587}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=571246236}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3827713881}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.618+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3816176044}) 2024-09-16T15:14:07.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1467220172}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2625780933}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=930239845}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1652562978}) 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2831447807}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.619+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3589807518}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=512288278}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2529559970}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.620+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531257974}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1583192071}) 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2579383901}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=556893549}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1959490206}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1579557220}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2141991073}) 2024-09-16T15:14:07.671 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1525811915}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1370306819}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.621+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1771167930}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1582917217}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=848418664}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3386771840}) 2024-09-16T15:14:07.672 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3481472952}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=211353244}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3139367207}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2018928067}) 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:07.673 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.622+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761787423}) 2024-09-16T15:14:07.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:07.623+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3181156589}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1824893064}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3987077301}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1353308740}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3952446570}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=21905533}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1470183573}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.075 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4160884479}) 2024-09-16T15:14:08.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461967112}) 2024-09-16T15:14:08.076 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.070+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2516997934}) 2024-09-16T15:14:08.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1400981505}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2901422836}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1685989407}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2165860118}) 2024-09-16T15:14:08.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.075+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=922865443}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1534845659}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3318865353}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=250475376}) 2024-09-16T15:14:08.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.076+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2581190284}) 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2975: done (2 left) 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2976: read oid 23 snap -1 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2976: expect (ObjNum 996 snap 281 seq_num 996) 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2977: read oid 50 snap -1 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2977: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:08.082 INFO:tasks.rados.rados.0.smithi162.stdout:2978: snap_remove snap 294 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1924750450}) 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4155133499}) 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=172816259}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.087+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=178093728}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2332910162}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1016139961}) 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3386633631}) 2024-09-16T15:14:08.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1161358446}) 2024-09-16T15:14:08.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.088+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316181777}) 2024-09-16T15:14:08.478 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=400888312}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1601053673}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.474+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2285788681}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1423027213}) 2024-09-16T15:14:08.479 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3612883640}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1300954760}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2055710301}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=645292254}) 2024-09-16T15:14:08.480 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.475+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1213787599}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3217738005}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1996417573}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=655253839}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3156134328}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4079009813}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3780001617}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2781458759}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3716246782}) 2024-09-16T15:14:08.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2031227827}) 2024-09-16T15:14:08.635 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:14:08.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2608508984}) 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3802601869}) 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=610147041}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1710691724}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3457995177}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3120814702}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=837689046}) 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2208318713}) 2024-09-16T15:14:08.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.680+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=133110857}) 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2244815623}) 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3655706415}) 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3915841596}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1419794916}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3502243972}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3454029886}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3170250387}) 2024-09-16T15:14:08.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1312076087}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.681+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=237713417}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=510961195}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=375251774}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3604624419}) 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3640707631}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532711418}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4122436813}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4034636789}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4034913449}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.683+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3798477030}) 2024-09-16T15:14:08.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332575306}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2188868716}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4232553064}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1601578158}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3331364909}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2311754786}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3112948385}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2729512525}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.684+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1428789339}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1378992590}) 2024-09-16T15:14:08.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1992859279}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=402393398}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3677981409}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=787380402}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=744076497}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1550376449}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3385826716}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.686+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1508586733}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1274159921}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=602766403}) 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=477588360}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1573836560}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3152800904}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3460873769}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1174444624}) 2024-09-16T15:14:08.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2715201549}) 2024-09-16T15:14:08.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.687+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3934654457}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=180600304}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1818490344}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3971864708}) 2024-09-16T15:14:08.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3349291592}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.695+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3423642128}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1487203412}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3408979484}) 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1350919889}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.696+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3411738131}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1835227108}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4229403412}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2104418468}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=267451738}) 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2340973485}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3894516018}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3631595280}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2649768489}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.697+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=630590944}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3973360678}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2020871416}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3345583669}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2771076498}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=847217894}) 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3724690620}) 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1153665740}) 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=20087088}) 2024-09-16T15:14:08.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:08.699+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=914597301}) 2024-09-16T15:14:08.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.734+0000 7f6adc55e640 10 monclient: get_monmap_and_config 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.734+0000 7f6adc55e640 10 monclient: build_initial_monmap 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.734+0000 7f6adc55e640 10 monclient: monmap: 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:08.736604+0000 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:08.736604+0000 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:08.737 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:08.738 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 Processor -- start 2024-09-16T15:14:08.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 -- start start 2024-09-16T15:14:08.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient: init 2024-09-16T15:14:08.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:08.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient: _add_conns ranks=[9,6,3,2,5,7,0,10,4,1,8] 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ad4136030 0x7f6ad4136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-a con 0x7f6ad4136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-d con 0x7f6ad409bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ad4136c60 0x7f6ad41370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-g con 0x7f6ad4136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad409b600 con 0x7f6ad4136030 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40a5740 con 0x7f6ad4136c60 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40a52c0 con 0x7f6ad409bfe0 2024-09-16T15:14:08.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.735+0000 7f6adc55e640 10 monclient(hunting): authenticate will time out at 1585.325903s 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ad4136030 0x7f6ad4136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:49722/0 (socket says 172.21.15.134:49722) 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ad4136030 0x7f6ad4136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59010/0 (socket says 172.21.15.134:59010) 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 1 -- 172.21.15.134:0/2933327253 learned_addr learned my addr 172.21.15.134:0/2933327253 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): get_auth_request con 0x7f6ad409bfe0 auth_method 0 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): get_auth_request con 0x7f6ad4136030 auth_method 0 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6adaafb640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ad4136c60 0x7f6ad41370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6adaafb640 10 monclient(hunting): get_auth_request con 0x7f6ad4136c60 auth_method 0 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6adaafb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6adaafb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6adaafb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_done global_id 74754 payload 274 2024-09-16T15:14:08.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 -- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 msgr2=0x7f6ad409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 -- 172.21.15.134:0/2933327253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ad4136c60 msgr2=0x7f6ad41370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ad4136c60 0x7f6ad41370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient: _finish_hunting 0 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 monclient: found mon.noname-a 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ada2fa640 1 -- 172.21.15.134:0/2933327253 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4073250 con 0x7f6ad4136030 2024-09-16T15:14:08.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.736+0000 7f6ad9af9640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ada2fa640 1 -- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 msgr2=0x7f6ad4136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4136410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad0029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c9e5ff6b12037b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 1 -- 172.21.15.134:0/2933327253 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ad002ad30 con 0x7f6ad4136030 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: dump: 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.750 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: _finish_auth 0 2024-09-16T15:14:08.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: _check_auth_tickets 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 5 monclient: authenticate success, global_id 74754 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 1 -- 172.21.15.134:0/2933327253 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad00307b0 con 0x7f6ad4136030 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 4 set_mon_vals no callback set 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 1 -- 172.21.15.134:0/2933327253 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ad0030dd0 con 0x7f6ad4136030 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6ad92f8640 10 monclient: dump: 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:08.752 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 10 monclient: shutdown 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:08.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 -- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 msgr2=0x7f6ad4136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4136410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad0029fd0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 -- 172.21.15.134:0/2933327253 shutdown_connections 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ad4136c60 0x7f6ad41370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6ad409bfe0 0x7f6ad409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 --2- 172.21.15.134:0/2933327253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4136410 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 -- 172.21.15.134:0/2933327253 >> 172.21.15.134:0/2933327253 conn(0x7f6ad40a9ea0 msgr2=0x7f6ad40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.739+0000 7f6adc55e640 1 -- 172.21.15.134:0/2933327253 shutdown_connections 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 1 -- 172.21.15.134:0/2933327253 wait complete. 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 10 monclient: build_initial_monmap 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 10 monclient: monmap: 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:08.754 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 1 Processor -- start 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 1 -- start start 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.740+0000 7f6adc55e640 10 monclient: init 2024-09-16T15:14:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient: _add_conns ranks=[8,0,1,4,7,5,3,2,6] 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-e con 0x7f6ad409bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-a con 0x7f6ad4136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad4136c60 0x7f6ad4135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): picked mon.noname-f con 0x7f6ad4136c60 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad41cb7e0 con 0x7f6ad4136030 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40a8130 con 0x7f6ad4136c60 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:08.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40a7c90 con 0x7f6ad409bfe0 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adc55e640 10 monclient(hunting): authenticate will time out at 1585.331340s 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54244/0 (socket says 172.21.15.134:54244) 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 -- 172.21.15.134:0/3377787372 learned_addr learned my addr 172.21.15.134:0/3377787372 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): get_auth_request con 0x7f6ad409bfe0 auth_method 0 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): get_auth_request con 0x7f6ad4136030 auth_method 0 2024-09-16T15:14:08.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adaafb640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad4136c60 0x7f6ad4135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adaafb640 10 monclient(hunting): get_auth_request con 0x7f6ad4136c60 auth_method 0 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adaafb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adaafb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6adaafb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:08.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient(hunting): handle_auth_done global_id 84410 payload 824 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 -- 172.21.15.134:0/3377787372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad4136c60 msgr2=0x7f6ad4135340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad4136c60 0x7f6ad4135340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 -- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 msgr2=0x7f6ad4134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient: _finish_hunting 0 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 monclient: found mon.noname-e 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ada2fa640 1 -- 172.21.15.134:0/3377787372 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40094d0 con 0x7f6ad409bfe0 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.741+0000 7f6ad9af9640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.743+0000 7f6ada2fa640 1 -- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 msgr2=0x7f6ad41beb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:08.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.743+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ad0003610 con 0x7f6ad409bfe0 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: dump: 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:08.760 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad00037b0 con 0x7f6ad409bfe0 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 5 monclient: authenticate success, global_id 84410 2024-09-16T15:14:08.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 10 monclient: _renew_subs 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad0009400 con 0x7f6ad409bfe0 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ad0003d90 con 0x7f6ad409bfe0 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6ad8af7640 4 set_mon_vals no callback set 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 10 monclient: dump: 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:08.762 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 10 monclient: _renew_subs 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad4121f20 con 0x7f6ad409bfe0 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6ada2fa640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad00046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f6ad006a4a0 osd_map(621..621 src has 1..621) v4 2024-09-16T15:14:08.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.744+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6ad0039040 con 0x7f6ad409bfe0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6adc55e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6adc55e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6adc55e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad40679b0 con 0x7f6ad409bfe0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6acaffd640 1 --2- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 0x7f6ab803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6ad9af9640 1 --2- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 0x7f6ab803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.745+0000 7f6ad9af9640 10 monclient: get_auth_request con 0x7f6ab8038880 auth_method 0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.747+0000 7f6ad9af9640 1 --2- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 0x7f6ab803ad40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6ad41bf630 tx=0x7f6ac4009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:08.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:08.749+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(621..621 src has 1..621) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f6ad006a4a0 con 0x7f6ad409bfe0 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2129411602}) 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=569613245}) 2024-09-16T15:14:09.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2163751046}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3134610226}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1180760531}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=500979266}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3427627314}) 2024-09-16T15:14:09.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358432931}) 2024-09-16T15:14:09.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.038+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1187547258}) 2024-09-16T15:14:09.049 INFO:tasks.rados.rados.0.smithi162.stdout:2978: done (4 left) 2024-09-16T15:14:09.049 INFO:tasks.rados.rados.0.smithi162.stdout:2979: copy_from oid 7 from oid 45 current snap is 308 2024-09-16T15:14:09.049 INFO:tasks.rados.rados.0.smithi162.stdout:2973: expect (ObjNum 856 snap 227 seq_num 856) 2024-09-16T15:14:09.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2976626241}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4276336337}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4189705672}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3757959329}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2268642729}) 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2242476959}) 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.047+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.048+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=592336057}) 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.048+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.048+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4092245029}) 2024-09-16T15:14:09.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.048+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3458667699}) 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3283198677}) 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=623691421}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4049951379}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3946047895}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3471609234}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2963844645}) 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=635311533}) 2024-09-16T15:14:09.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942300097}) 2024-09-16T15:14:09.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.054+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1989135793}) 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3288562496}) 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1470726730}) 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=117868481}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4012441349}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764303903}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=88070579}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=941239755}) 2024-09-16T15:14:09.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3047080452}) 2024-09-16T15:14:09.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.096+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3673615239}) 2024-09-16T15:14:09.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=291502874}) 2024-09-16T15:14:09.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=231176401}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=228121046}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1154745981}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3559646929}) 2024-09-16T15:14:09.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466091586}) 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1686185705}) 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3338604117}) 2024-09-16T15:14:09.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.102+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=503963178}) 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: finishing write tid 1 to smithi16234619-32 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: finishing write tid 2 to smithi16234619-32 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: finishing write tid 3 to smithi16234619-32 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: finishing write tid 4 to smithi16234619-32 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: finishing write tid 5 to smithi16234619-32 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: oid 32 updating version 0 to 1430 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: oid 32 updating version 1430 to 1432 2024-09-16T15:14:09.167 INFO:tasks.rados.rados.0.smithi162.stdout:2974: oid 32 version 1432 is already newer than 1431 2024-09-16T15:14:09.168 INFO:tasks.rados.rados.0.smithi162.stdout:2974: oid 32 version 1432 is already newer than 1429 2024-09-16T15:14:09.168 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1432 (ObjNum 1036 snap 308 seq_num 1036) dirty exists 2024-09-16T15:14:09.168 INFO:tasks.rados.rados.0.smithi162.stdout:2974: left oid 32 (ObjNum 1036 snap 308 seq_num 1036) 2024-09-16T15:14:09.168 INFO:tasks.rados.rados.0.smithi162.stdout:2976: expect (ObjNum 996 snap 281 seq_num 996) 2024-09-16T15:14:09.256 INFO:tasks.rados.rados.0.smithi162.stdout:2977: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:09.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3676458907}) 2024-09-16T15:14:09.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3741667326}) 2024-09-16T15:14:09.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2820712177}) 2024-09-16T15:14:09.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.301 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.301 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3102100036}) 2024-09-16T15:14:09.301 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.301 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.301 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=454083775}) 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=524083825}) 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.302 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963808543}) 2024-09-16T15:14:09.303 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.303 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3311191944}) 2024-09-16T15:14:09.303 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.293+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682467415}) 2024-09-16T15:14:09.425 INFO:tasks.rados.rados.0.smithi162.stdout:2979: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2979: finishing copy_from to smithi16234619-7 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1468 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2973: done (4 left) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2974: done (3 left) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2976: done (2 left) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2977: done (1 left) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2979: done (0 left) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2980: copy_from oid 2 from oid 31 current snap is 308 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2981: read oid 39 snap 304 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2981: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2982: read oid 32 snap -1 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2982: expect (ObjNum 1036 snap 308 seq_num 1036) 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2983: rollback oid 17 current snap is 308 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 278 2024-09-16T15:14:09.426 INFO:tasks.rados.rados.0.smithi162.stdout:2984: snap_create 2024-09-16T15:14:09.429 INFO:tasks.rados.rados.0.smithi162.stdout:2980: finishing copy_from to smithi16234619-2 2024-09-16T15:14:09.430 INFO:tasks.rados.rados.0.smithi162.stdout:2980: got expected ENOENT (src dne) 2024-09-16T15:14:09.430 INFO:tasks.rados.rados.0.smithi162.stdout:2980: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:14:09.431 INFO:tasks.rados.rados.0.smithi162.stdout:2981: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3439207490}) 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.489 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=666757449}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=868682737}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2528581165}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=336143679}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=375671893}) 2024-09-16T15:14:09.490 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3455866958}) 2024-09-16T15:14:09.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1362105225}) 2024-09-16T15:14:09.491 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.486+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1071467053}) 2024-09-16T15:14:09.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2432468734}) 2024-09-16T15:14:09.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2257502687}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1456833376}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3046312544}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634201804}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.503 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3369867510}) 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1268511617}) 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3220674721}) 2024-09-16T15:14:09.504 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.499+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=942209957}) 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2757908024}) 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1518671923}) 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.513 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=972247148}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2051933347}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3782073548}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=268800198}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2214138507}) 2024-09-16T15:14:09.514 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=347028275}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.507+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=522719449}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3068916587}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3639576986}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2085741234}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2269108683}) 2024-09-16T15:14:09.515 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2156415478}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=576256409}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=461073837}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2647234730}) 2024-09-16T15:14:09.516 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.508+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3051628409}) 2024-09-16T15:14:09.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.573+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6ad0045d00 con 0x7f6ad409bfe0 2024-09-16T15:14:09.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.573+0000 7f6acaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:09.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.573+0000 7f6acaffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:09.633 INFO:tasks.rados.rados.0.smithi162.stdout:2983: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:14:09.633 INFO:tasks.rados.rados.0.smithi162.stdout:2983: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:14:09.633 INFO:tasks.rados.rados.0.smithi162.stdout:2983: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:14:09.633 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1531 (ObjNum 929 snap 259 seq_num 929) dirty exists 2024-09-16T15:14:09.633 INFO:tasks.rados.rados.0.smithi162.stdout:2982: expect (ObjNum 1036 snap 308 seq_num 1036) 2024-09-16T15:14:09.668 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3777906186}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=42506757}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2202207178}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2612689656}) 2024-09-16T15:14:09.669 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1439631033}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2370215169}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1776511186}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2498601965}) 2024-09-16T15:14:09.670 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.665+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1203979094}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693370146}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4017480414}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=712371436}) 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3996935073}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1934435249}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3617711817}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1231834172}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273430807}) 2024-09-16T15:14:09.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:09.672+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242815693}) 2024-09-16T15:14:09.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6adc55e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:14:09.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6adc55e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6adc55e640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6ad409c400 con 0x7f6ad409bfe0 2024-09-16T15:14:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6acaffd640 1 -- 172.21.15.134:0/3377787372 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f6ad0037020 con 0x7f6ad409bfe0 2024-09-16T15:14:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6acaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.797+0000 7f6acaffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:09.801 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":110,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:14:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 msgr2=0x7f6ab803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 0x7f6ab803ad40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6ad41bf630 tx=0x7f6ac4009240 comp rx=0 tx=0).stop 2024-09-16T15:14:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 10 monclient: shutdown 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 msgr2=0x7f6ad41beb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097a0 tx=0x7f6ad00046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.799+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 shutdown_connections 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ab8038880 0x7f6ab803ad40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ad4136c60 0x7f6ad4135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ad4136030 0x7f6ad4134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 --2- 172.21.15.134:0/3377787372 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6ad409bfe0 0x7f6ad41beb90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 >> 172.21.15.134:0/3377787372 conn(0x7f6ad40a9ea0 msgr2=0x7f6ad40a8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 shutdown_connections 2024-09-16T15:14:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.800+0000 7f6adc55e640 1 -- 172.21.15.134:0/3377787372 wait complete. 2024-09-16T15:14:09.813 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:14:09.813 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:14:09.813 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:14:09.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: get_monmap_and_config 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: build_initial_monmap 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: monmap: 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:09.950618+0000 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:09.950618+0000 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:09.953 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 Processor -- start 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 -- start start 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: init 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient: _add_conns ranks=[0,5,1,10,6,3,8,7,2,9,4] 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7650066070 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f76501388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): picked mon.noname-h con 0x7f76501364f0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7650138e20 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7650073250 con 0x7f7650066070 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7650066450 con 0x7f7650138e20 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f765009ec30 con 0x7f76501364f0 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7658776640 10 monclient(hunting): authenticate will time out at 1586.539886s 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.949+0000 7f7656512640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7650066070 0x7f7650099e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32950/0 (socket says 172.21.15.134:32950) 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7656512640 1 -- 172.21.15.134:0/2784242271 learned_addr learned my addr 172.21.15.134:0/2784242271 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 1 --2- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f76501388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): get_auth_request con 0x7f76501364f0 auth_method 0 2024-09-16T15:14:09.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient(hunting): handle_auth_done global_id 44978 payload 274 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 1 -- 172.21.15.134:0/2784242271 >> v1:172.21.15.162:6789/0 conn(0x7f7650138e20 legacy=0x7f765013b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 1 -- 172.21.15.134:0/2784242271 >> v1:172.21.15.134:6789/0 conn(0x7f7650066070 legacy=0x7f7650099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient: _finish_hunting 0 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 1 monclient: found mon.noname-h 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.950+0000 7f7655d11640 1 -- 172.21.15.134:0/2784242271 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500994b0 con 0x7f76501364f0 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655d11640 1 -- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 msgr2=0x7f76501388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655d11640 1 --2- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f76501388e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76400097d0 tx=0x7f764002a040 comp rx=0 tx=0).ready entity=mon.5 client_cookie=2a13699218a2e8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 1 -- 172.21.15.134:0/2784242271 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f764002ae40 con 0x7f76501364f0 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: dump: 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: _finish_auth 0 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: _check_auth_tickets 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 1 -- 172.21.15.134:0/2784242271 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76400307b0 con 0x7f76501364f0 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 1 -- 172.21.15.134:0/2784242271 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7640030d90 con 0x7f76501364f0 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7658776640 5 monclient: authenticate success, global_id 44978 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7658776640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7658776640 4 set_mon_vals no callback set 2024-09-16T15:14:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7655510640 10 monclient: dump: 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.953+0000 7f7658776640 10 monclient: shutdown 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 -- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 msgr2=0x7f76501388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 --2- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f76501388e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76400097d0 tx=0x7f764002a040 comp rx=0 tx=0).stop 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 -- 172.21.15.134:0/2784242271 shutdown_connections 2024-09-16T15:14:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 --2- 172.21.15.134:0/2784242271 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f76501388e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 -- 172.21.15.134:0/2784242271 >> 172.21.15.134:0/2784242271 conn(0x7f76500a57a0 msgr2=0x7f76500a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 -- 172.21.15.134:0/2784242271 shutdown_connections 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 1 -- 172.21.15.134:0/2784242271 wait complete. 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 10 monclient: build_initial_monmap 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.954+0000 7f7658776640 10 monclient: monmap: 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 Processor -- start 2024-09-16T15:14:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 -- start start 2024-09-16T15:14:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient: init 2024-09-16T15:14:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient: _add_conns ranks=[0,5,1,6,7,2,8,4,3] 2024-09-16T15:14:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 0x7f76501c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): picked mon.noname-a con 0x7f7650066070 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f7650134850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): picked mon.noname-h con 0x7f76501364f0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7650138e20 0x7f7650134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): picked mon.noname-f con 0x7f7650138e20 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501c8770 con 0x7f7650066070 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500a2500 con 0x7f7650138e20 2024-09-16T15:14:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f765012e2d0 con 0x7f76501364f0 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7658776640 10 monclient(hunting): authenticate will time out at 1586.545633s 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 0x7f76501c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 0x7f76501c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59038/0 (socket says 172.21.15.134:59038) 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 -- 172.21.15.134:0/1437765026 learned_addr learned my addr 172.21.15.134:0/1437765026 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): get_auth_request con 0x7f7650066070 auth_method 0 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient(hunting): handle_auth_done global_id 74772 payload 824 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 msgr2=0x7f7650134850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f76501364f0 0x7f7650134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7650138e20 msgr2=0x7f7650134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7650138e20 0x7f7650134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient: _finish_hunting 0 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 monclient: found mon.noname-a 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.955+0000 7f7656512640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009400 con 0x7f7650066070 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7656512640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 msgr2=0x7f76501c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:09.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7656512640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 0x7f76501c8230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c460 tx=0x7f764c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f764c005cf0 con 0x7f7650066070 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: dump: 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 5 monclient: authenticate success, global_id 74772 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c005e90 con 0x7f7650066070 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f764c0074e0 con 0x7f7650066070 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7654d0f640 4 set_mon_vals no callback set 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7646ffd640 10 monclient: dump: 2024-09-16T15:14:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 10 monclient: _renew_subs 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500a2020 con 0x7f7650066070 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 10 monclient: _renew_subs 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.958+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765012bf70 con 0x7f7650066070 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7658776640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:09.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f764c007640 con 0x7f7650066070 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7658776640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 0x7f765012a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650072400 con 0x7f765012a440 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 0x7f765012a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient: get_auth_request con 0x7f765012a440 auth_method 0 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:09.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7646ffd640 1 --2- 172.21.15.134:0/1437765026 >> v2:172.21.15.162:6800/980547263 conn(0x7f7634038880 0x7f763403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(622..622 src has 1..622) v4 ==== 3987+0+0 (secure 0 0 0) 0x7f764c04bf10 con 0x7f7650066070 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 10 monclient(hunting): handle_auth_done global_id 74781 payload 824 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7655d11640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f765012da30 con 0x7f765012a440 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7656d13640 1 --2- 172.21.15.134:0/1437765026 >> v2:172.21.15.162:6800/980547263 conn(0x7f7634038880 0x7f763403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.959+0000 7f7656d13640 10 monclient: get_auth_request con 0x7f7634038880 auth_method 0 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7655d11640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 msgr2=0x7f765012a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7655d11640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 0x7f765012a820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f765006ad90 tx=0x7f7640030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7640004460 con 0x7f765012a440 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7646ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7646ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:09.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.961+0000 7f7646ffd640 10 monclient: dump: 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.984 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7640003610 con 0x7f765012a440 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 msgr2=0x7f765012a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 0x7f765012a820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f765006ad90 tx=0x7f7640030ce0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 msgr2=0x7f765012a820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:09.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7646ffd640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012a440 0x7f765012a820 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f765006ad90 tx=0x7f7640030ce0 comp rx=0 tx=0).stop 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.962+0000 7f7656d13640 1 --2- 172.21.15.134:0/1437765026 >> v2:172.21.15.162:6800/980547263 conn(0x7f7634038880 0x7f763403ad40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f765012d960 tx=0x7f7648009230 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7658776640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7658776640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 0x7f765009c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7658776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650073880 con 0x7f765012ad60 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 0x7f765009c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient: get_auth_request con 0x7f765012ad60 auth_method 0 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:09.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 10 monclient(hunting): handle_auth_done global_id 74790 payload 824 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.969+0000 7f7655d11640 1 -- 172.21.15.134:0/1437765026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7650061ae0 con 0x7f765012ad60 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.970+0000 7f7655d11640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 msgr2=0x7f765009c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.970+0000 7f7655d11640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 0x7f765009c880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f764002a080 tx=0x7f7640008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.970+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f764003d8b0 con 0x7f765012ad60 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 10 monclient: dump: 2024-09-16T15:14:09.987 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:09.988 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9198 (secure 0 0 0) 0x7f764003d440 con 0x7f765012ad60 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 msgr2=0x7f765009c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 0x7f765009c880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f764002a080 tx=0x7f7640008030 comp rx=0 tx=0).stop 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 msgr2=0x7f765009c880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:09.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7646ffd640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f765012ad60 0x7f765009c880 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f764002a080 tx=0x7f7640008030 comp rx=0 tx=0).stop 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 9, 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.001 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.002 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.003 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.004 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.005 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.008 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.013 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.015 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 >> v2:172.21.15.162:6800/980547263 conn(0x7f7634038880 msgr2=0x7f763403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 --2- 172.21.15.134:0/1437765026 >> v2:172.21.15.162:6800/980547263 conn(0x7f7634038880 0x7f763403ad40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f765012d960 tx=0x7f7648009230 comp rx=0 tx=0).stop 2024-09-16T15:14:10.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 10 monclient: shutdown 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 msgr2=0x7f76501c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 --2- 172.21.15.134:0/1437765026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7650066070 0x7f76501c8230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f764c00c460 tx=0x7f764c00c920 comp rx=0 tx=0).stop 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7656d13640 1 -- 172.21.15.134:0/1437765026 reap_dead start 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7656d13640 1 -- 172.21.15.134:0/1437765026 reap_dead start 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 shutdown_connections 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 >> 172.21.15.134:0/1437765026 conn(0x7f76500a57a0 msgr2=0x7f7650135990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 shutdown_connections 2024-09-16T15:14:10.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:09.971+0000 7f7658776640 1 -- 172.21.15.134:0/1437765026 wait complete. 2024-09-16T15:14:10.018 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:14:10.018 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4004448875}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2168502696}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3486625292}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=217672929}) 2024-09-16T15:14:10.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1956294657}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=94178710}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=534040363}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360756063}) 2024-09-16T15:14:10.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.037+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1343207058}) 2024-09-16T15:14:10.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1055718956}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3238173705}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2437352918}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3788185513}) 2024-09-16T15:14:10.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4129658084}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3421038002}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1462288171}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3100314254}) 2024-09-16T15:14:10.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.100+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3238249485}) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2980: done (4 left) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2981: done (3 left) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2982: done (2 left) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2983: done (1 left) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2984: done (0 left) 2024-09-16T15:14:10.106 INFO:tasks.rados.rados.0.smithi162.stdout:2985: snap_create 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3889237620}) 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=630215717}) 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4174439817}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=530748818}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=242736057}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225790531}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=403505558}) 2024-09-16T15:14:10.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912204078}) 2024-09-16T15:14:10.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.105+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=748236504}) 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.118+0000 7f7b34e20640 10 monclient: get_monmap_and_config 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.118+0000 7f7b34e20640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.118+0000 7f7b34e20640 10 monclient: monmap: 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:10.119756+0000 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:10.119756+0000 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:10.121 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.119+0000 7f7b34e20640 1 Processor -- start 2024-09-16T15:14:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.119+0000 7f7b34e20640 1 -- start start 2024-09-16T15:14:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.119+0000 7f7b34e20640 10 monclient: init 2024-09-16T15:14:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient: _add_conns ranks=[3,6,2,10,9,0,4,1,8,7,5] 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 0x7f7b3009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-g con 0x7f7b30136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 0x7f7b30137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-d con 0x7f7b30137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-b con 0x7f7b30135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3009abc0 con 0x7f7b30135dc0 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3009ba50 con 0x7f7b30136e40 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300a53c0 con 0x7f7b30137220 2024-09-16T15:14:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b34e20640 10 monclient(hunting): authenticate will time out at 1586.709089s 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 0x7f7b30137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54160/0 (socket says 172.21.15.134:54160) 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/157432746 learned_addr learned my addr 172.21.15.134:0/157432746 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): get_auth_request con 0x7f7b30135dc0 auth_method 0 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request con 0x7f7b30137220 auth_method 0 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2e59c640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 0x7f7b3009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_done global_id 94142 payload 274 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 msgr2=0x7f7b30137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 0x7f7b30137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/157432746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 msgr2=0x7f7b3009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 0x7f7b3009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 monclient: found mon.noname-b 2024-09-16T15:14:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/157432746 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30073250 con 0x7f7b30135dc0 2024-09-16T15:14:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2e59c640 10 monclient: get_auth_request con 0x7f7b30136e40 auth_method 0 2024-09-16T15:14:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 0x7f7b30137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.120+0000 7f7b2e59c640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 0x7f7b3009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 msgr2=0x7f7b30136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009e80 tx=0x7f7b2802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b462a8bda0e0857f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 1 -- 172.21.15.134:0/157432746 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b28007a70 con 0x7f7b30135dc0 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:10.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: dump: 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 1 -- 172.21.15.134:0/157432746 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2802d050 con 0x7f7b30135dc0 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b34e20640 5 monclient: authenticate success, global_id 94142 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b34e20640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 1 -- 172.21.15.134:0/157432746 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b28007bf0 con 0x7f7b30135dc0 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b2d59a640 10 monclient: dump: 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b34e20640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.123+0000 7f7b34e20640 4 set_mon_vals no callback set 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 10 monclient: shutdown 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 -- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 msgr2=0x7f7b30136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009e80 tx=0x7f7b2802a530 comp rx=0 tx=0).stop 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 -- 172.21.15.134:0/157432746 shutdown_connections 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30135dc0 0x7f7b30136220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7b30137220 0x7f7b30137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 --2- 172.21.15.134:0/157432746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30136e40 0x7f7b3009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 -- 172.21.15.134:0/157432746 >> 172.21.15.134:0/157432746 conn(0x7f7b300a6110 msgr2=0x7f7b300a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 -- 172.21.15.134:0/157432746 shutdown_connections 2024-09-16T15:14:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 1 -- 172.21.15.134:0/157432746 wait complete. 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.124+0000 7f7b34e20640 10 monclient: monmap: 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 Processor -- start 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 -- start start 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient: init 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient: _add_conns ranks=[8,7,0,6,2,5,3,4,1] 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 0x7f7b30134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-e con 0x7f7b30135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b30136e40 0x7f7b30134fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-i con 0x7f7b30136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30137220 0x7f7b30121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): picked mon.noname-a con 0x7f7b30137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300a8a10 con 0x7f7b30137220 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30136880 con 0x7f7b30136e40 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30136330 con 0x7f7b30135dc0 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b34e20640 10 monclient(hunting): authenticate will time out at 1586.714881s 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 0x7f7b30134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 0x7f7b30134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54284/0 (socket says 172.21.15.134:54284) 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 1 -- 172.21.15.134:0/3663371763 learned_addr learned my addr 172.21.15.134:0/3663371763 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 10 monclient(hunting): get_auth_request con 0x7f7b30135dc0 auth_method 0 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2e59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.125+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30137220 0x7f7b30121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): get_auth_request con 0x7f7b30137220 auth_method 0 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b30136e40 0x7f7b30134fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request con 0x7f7b30136e40 auth_method 0 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient(hunting): handle_auth_done global_id 84419 payload 824 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b30136e40 msgr2=0x7f7b30134fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f7b30136e40 0x7f7b30134fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30137220 msgr2=0x7f7b30121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30137220 0x7f7b30121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 monclient: found mon.noname-e 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2e59c640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b24009400 con 0x7f7b30135dc0 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.126+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30137220 0x7f7b30121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b2e59c640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 msgr2=0x7f7b30134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b2e59c640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 0x7f7b30134aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c7f0 tx=0x7f7b2000cbf0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b2000ce40 con 0x7f7b30135dc0 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: dump: 2024-09-16T15:14:10.144 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.145 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 5 monclient: authenticate success, global_id 84419 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200113f0 con 0x7f7b30135dc0 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 10 monclient: _renew_subs 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b28009b30 con 0x7f7b30135dc0 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b200119d0 con 0x7f7b30135dc0 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b1effd640 10 monclient: dump: 2024-09-16T15:14:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.147 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b2cd99640 4 set_mon_vals no callback set 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 10 monclient: _renew_subs 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.128+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30125780 con 0x7f7b30135dc0 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7b20011560 con 0x7f7b30135dc0 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b34e20640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b34e20640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 0x7f7b3002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30071b20 con 0x7f7b30122510 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 0x7f7b3002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient: get_auth_request con 0x7f7b30122510 auth_method 0 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b1effd640 1 --2- 172.21.15.134:0/3663371763 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b10038880 0x7f7b1003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f7b2004a220 con 0x7f7b30135dc0 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_done global_id 94151 payload 824 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.129+0000 7f7b2dd9b640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b30061ae0 con 0x7f7b30122510 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.130+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/3663371763 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b10038880 0x7f7b1003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.130+0000 7f7b2ed9d640 10 monclient: get_auth_request con 0x7f7b10038880 auth_method 0 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b2dd9b640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 msgr2=0x7f7b3002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 0x7f7b3002b260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b30122f30 tx=0x7f7b2400f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b24011e80 con 0x7f7b30122510 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 10 monclient: dump: 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.150 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7b2400fe70 con 0x7f7b30122510 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 msgr2=0x7f7b3002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 0x7f7b3002b260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b30122f30 tx=0x7f7b2400f610 comp rx=0 tx=0).stop 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 msgr2=0x7f7b3002b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b1effd640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b30122510 0x7f7b3002b260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.132+0000 7f7b2ed9d640 1 --2- 172.21.15.134:0/3663371763 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b10038880 0x7f7b1003ad40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009e50 tx=0x7f7b28009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b34e20640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b34e20640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 0x7f7b3002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b34e20640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3007a680 con 0x7f7b301228f0 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 0x7f7b3002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient: get_auth_request con 0x7f7b301228f0 auth_method 0 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 10 monclient(hunting): handle_auth_done global_id 94160 payload 824 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.138+0000 7f7b2dd9b640 1 -- 172.21.15.134:0/3663371763 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b3009b490 con 0x7f7b301228f0 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b2dd9b640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 msgr2=0x7f7b3002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b2dd9b640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 0x7f7b3002b7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b2401b000 tx=0x7f7b240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7b2402c070 con 0x7f7b301228f0 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 10 monclient: dump: 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f7b24024040 con 0x7f7b301228f0 2024-09-16T15:14:10.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 msgr2=0x7f7b3002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 0x7f7b3002b7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b2401b000 tx=0x7f7b240046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 msgr2=0x7f7b3002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b1effd640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b301228f0 0x7f7b3002b7a0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b2401b000 tx=0x7f7b240046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 9, 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.164 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.167 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.171 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.173 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.177 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.177 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.177 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.177 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b10038880 msgr2=0x7f7b1003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 --2- 172.21.15.134:0/3663371763 >> v2:172.21.15.162:6800/980547263 conn(0x7f7b10038880 0x7f7b1003ad40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009e50 tx=0x7f7b28009000 comp rx=0 tx=0).stop 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 10 monclient: shutdown 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 msgr2=0x7f7b30134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 --2- 172.21.15.134:0/3663371763 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7b30135dc0 0x7f7b30134aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c7f0 tx=0x7f7b2000cbf0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/3663371763 reap_dead start 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b2ed9d640 1 -- 172.21.15.134:0/3663371763 reap_dead start 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 shutdown_connections 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 >> 172.21.15.134:0/3663371763 conn(0x7f7b300a6110 msgr2=0x7f7b300a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 shutdown_connections 2024-09-16T15:14:10.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.141+0000 7f7b34e20640 1 -- 172.21.15.134:0/3663371763 wait complete. 2024-09-16T15:14:10.179 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.179 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:14:10.179 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:14:10.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.315+0000 7f960a2b1640 10 monclient: get_monmap_and_config 2024-09-16T15:14:10.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.315+0000 7f960a2b1640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.315+0000 7f960a2b1640 10 monclient: monmap: 2024-09-16T15:14:10.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:10.316092+0000 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:10.316092+0000 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.317 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:10.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 Processor -- start 2024-09-16T15:14:10.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 -- start start 2024-09-16T15:14:10.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient: init 2024-09-16T15:14:10.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient: _add_conns ranks=[9,8,0,1,7,3,10,5,4,6,2] 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 0x7f960409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-a con 0x7f9604136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-e con 0x7f9604137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9604135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9604073250 con 0x7f9604135dc0 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960409abc0 con 0x7f9604136e40 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f960409ba50 con 0x7f9604137220 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960a2b1640 10 monclient(hunting): authenticate will time out at 1586.905603s 2024-09-16T15:14:10.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 0x7f960409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54292/0 (socket says 172.21.15.134:54292) 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 -- 172.21.15.134:0/3395151995 learned_addr learned my addr 172.21.15.134:0/3395151995 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): get_auth_request con 0x7f9604136e40 auth_method 0 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f960884e640 1 --1- 172.21.15.134:0/3395151995 >> v1:172.21.15.134:6789/0 conn(0x7f9604135dc0 0x7f9604136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32962/0 (socket says 172.21.15.134:32962) 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): get_auth_request con 0x7f9604137220 auth_method 0 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 10 monclient(hunting): handle_auth_done global_id 84428 payload 274 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 -- 172.21.15.134:0/3395151995 >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 msgr2=0x7f960409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 --2- 172.21.15.134:0/3395151995 >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 0x7f960409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.316+0000 7f95fbfff640 1 -- 172.21.15.134:0/3395151995 >> v1:172.21.15.134:6789/0 conn(0x7f9604135dc0 legacy=0x7f9604136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f95fbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f95fbfff640 1 monclient: found mon.noname-e 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f95fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f95fbfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f95fbfff640 1 -- 172.21.15.134:0/3395151995 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040a53c0 con 0x7f9604137220 2024-09-16T15:14:10.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.317+0000 7f9603fff640 1 --2- 172.21.15.134:0/3395151995 >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 0x7f960409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f95fbfff640 1 -- 172.21.15.134:0/3395151995 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 msgr2=0x7f9604137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f95fbfff640 1 --2- 172.21.15.134:0/3395151995 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098a0 tx=0x7f95ec02a110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c2e1f9ce782a3995 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 1 -- 172.21.15.134:0/3395151995 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95ec02ae30 con 0x7f9604137220 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 10 monclient: dump: 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f960a2b1640 5 monclient: authenticate success, global_id 84428 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f96037fe640 1 -- 172.21.15.134:0/3395151995 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec0307b0 con 0x7f9604137220 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.319+0000 7f960a2b1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f96037fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 4 set_mon_vals no callback set 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f96037fe640 1 -- 172.21.15.134:0/3395151995 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95ec030d90 con 0x7f9604137220 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f96037fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f96037fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f96037fe640 10 monclient: dump: 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 10 monclient: shutdown 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 -- 172.21.15.134:0/3395151995 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 msgr2=0x7f9604137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 --2- 172.21.15.134:0/3395151995 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098a0 tx=0x7f95ec02a110 comp rx=0 tx=0).stop 2024-09-16T15:14:10.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 -- 172.21.15.134:0/3395151995 shutdown_connections 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 --2- 172.21.15.134:0/3395151995 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9604137220 0x7f9604137600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 --2- 172.21.15.134:0/3395151995 >> v2:172.21.15.134:3300/0 conn(0x7f9604136e40 0x7f960409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 -- 172.21.15.134:0/3395151995 >> 172.21.15.134:0/3395151995 conn(0x7f96040a6110 msgr2=0x7f96040a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 -- 172.21.15.134:0/3395151995 shutdown_connections 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.320+0000 7f960a2b1640 1 -- 172.21.15.134:0/3395151995 wait complete. 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.321+0000 7f960a2b1640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.321+0000 7f960a2b1640 10 monclient: monmap: 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:10.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.321+0000 7f960a2b1640 1 Processor -- start 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.321+0000 7f960a2b1640 1 -- start start 2024-09-16T15:14:10.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.321+0000 7f960a2b1640 10 monclient: init 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient: _add_conns ranks=[4,2,3,0,8,5,6,1,7] 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9604135dc0 0x7f9604134d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-c con 0x7f9604135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-b con 0x7f9604136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9604137220 0x7f960411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): picked mon.noname-g con 0x7f9604137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9604121ec0 con 0x7f9604136e40 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9604122110 con 0x7f9604137220 2024-09-16T15:14:10.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9604136910 con 0x7f9604135dc0 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960a2b1640 10 monclient(hunting): authenticate will time out at 1586.911228s 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9604135dc0 0x7f9604134d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54184/0 (socket says 172.21.15.134:54184) 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 -- 172.21.15.134:0/2102507626 learned_addr learned my addr 172.21.15.134:0/2102507626 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): get_auth_request con 0x7f9604136e40 auth_method 0 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): get_auth_request con 0x7f9604135dc0 auth_method 0 2024-09-16T15:14:10.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960884e640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9604137220 0x7f960411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient(hunting): handle_auth_done global_id 94169 payload 824 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9604135dc0 msgr2=0x7f9604134d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9604135dc0 0x7f9604134d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9604137220 msgr2=0x7f960411fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9604137220 0x7f960411fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 monclient: found mon.noname-b 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f95fbfff640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009400 con 0x7f9604136e40 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960884e640 10 monclient: get_auth_request con 0x7f9604137220 auth_method 0 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.322+0000 7f960884e640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9604137220 0x7f960411fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f95fbfff640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 msgr2=0x7f9604135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f95fbfff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0090b0 tx=0x7f95ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95ec003610 con 0x7f9604136e40 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f96017fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.324+0000 7f96017fa640 10 monclient: dump: 2024-09-16T15:14:10.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 5 monclient: authenticate success, global_id 94169 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 monclient: _renew_subs 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec0037b0 con 0x7f9604136e40 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec0094d0 con 0x7f9604136e40 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95ec003d90 con 0x7f9604136e40 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f9602ffd640 4 set_mon_vals no callback set 2024-09-16T15:14:10.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f96017fa640 10 monclient: dump: 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.343 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 monclient: _renew_subs 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96041363e0 con 0x7f9604136e40 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 0x7f960402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.325+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9604071b20 con 0x7f96041222a0 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f95fbfff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0090b0 tx=0x7f95ec0046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f95ec06aaa0 osd_map(623..623 src has 1..623) v4 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 0x7f960402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient: get_auth_request con 0x7f96041222a0 auth_method 0 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f95ec039070 con 0x7f9604136e40 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f96017fa640 1 --2- 172.21.15.134:0/2102507626 >> v2:172.21.15.162:6800/980547263 conn(0x7f95e4038880 0x7f95e403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 10 monclient(hunting): handle_auth_done global_id 5710 payload 824 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f9603fff640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9604061ae0 con 0x7f96041222a0 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f960884e640 1 --2- 172.21.15.134:0/2102507626 >> v2:172.21.15.162:6800/980547263 conn(0x7f95e4038880 0x7f95e403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.326+0000 7f960884e640 10 monclient: get_auth_request con 0x7f95e4038880 auth_method 0 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f9603fff640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 msgr2=0x7f960402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f9603fff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 0x7f960402b260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f400f610 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4014ac0 con 0x7f96041222a0 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 10 monclient: dump: 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f960884e640 1 --2- 172.21.15.134:0/2102507626 >> v2:172.21.15.162:6800/980547263 conn(0x7f95e4038880 0x7f95e403ad40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007fd0 tx=0x7f95fc009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f95f4014c20 con 0x7f96041222a0 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 msgr2=0x7f960402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 0x7f960402b260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f400f610 comp rx=0 tx=0).stop 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 msgr2=0x7f960402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.328+0000 7f96017fa640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f96041222a0 0x7f960402b260 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f400f610 comp rx=0 tx=0).stop 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.334+0000 7f960a2b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.334+0000 7f960a2b1640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 0x7f960402bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.334+0000 7f960a2b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.334+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f960407a680 con 0x7f960402b7a0 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 0x7f960402bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient: get_auth_request con 0x7f960402b7a0 auth_method 0 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 10 monclient(hunting): handle_auth_done global_id 5719 payload 824 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.335+0000 7f9603fff640 1 -- 172.21.15.134:0/2102507626 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f960409b490 con 0x7f960402b7a0 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f9603fff640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 msgr2=0x7f960402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f9603fff640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 0x7f960402bbb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f40046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95f4022e10 con 0x7f960402b7a0 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 10 monclient: dump: 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.350 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7f95f4026070 con 0x7f960402b7a0 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:10.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 msgr2=0x7f960402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 0x7f960402bbb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f40046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 msgr2=0x7f960402bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.337+0000 7f96017fa640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f960402b7a0 0x7f960402bbb0 secure :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f95f4011c00 tx=0x7f95f40046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:10.352 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:10.353 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:10.354 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.355 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:10.356 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.357 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.358 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.359 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.360 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.361 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.362 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.363 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.364 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.365 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.366 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.372 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.373 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.374 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 >> v2:172.21.15.162:6800/980547263 conn(0x7f95e4038880 msgr2=0x7f95e403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 --2- 172.21.15.134:0/2102507626 >> v2:172.21.15.162:6800/980547263 conn(0x7f95e4038880 0x7f95e403ad40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007fd0 tx=0x7f95fc009040 comp rx=0 tx=0).stop 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 10 monclient: shutdown 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 msgr2=0x7f9604135250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 --2- 172.21.15.134:0/2102507626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9604136e40 0x7f9604135250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0090b0 tx=0x7f95ec0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960884e640 1 -- 172.21.15.134:0/2102507626 reap_dead start 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f96017fa640 1 -- 172.21.15.134:0/2102507626 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (unknown 0 0 0) 0x7f95ec06aaa0 con 0x7f9604136e40 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960884e640 1 -- 172.21.15.134:0/2102507626 reap_dead start 2024-09-16T15:14:10.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 shutdown_connections 2024-09-16T15:14:10.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 >> 172.21.15.134:0/2102507626 conn(0x7f96040a6110 msgr2=0x7f960409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 shutdown_connections 2024-09-16T15:14:10.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.338+0000 7f960a2b1640 1 -- 172.21.15.134:0/2102507626 wait complete. 2024-09-16T15:14:10.377 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.377 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:14:10.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2477084630}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1394630035}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3662946284}) 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.407 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3381138460}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3673240730}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=406532549}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1833574513}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434449185}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.353+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=224464088}) 2024-09-16T15:14:10.408 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.354+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.354+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3990425353}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=921488252}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3750849439}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1113137352}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.409 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3988543337}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1726052263}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2634447357}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1841204919}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.355+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3676429496}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=788369319}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.410 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3100111052}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3886551586}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2629316573}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088037678}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290292340}) 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.411 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1503080839}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407855737}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.356+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1913069249}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3066550170}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3115756451}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4200600323}) 2024-09-16T15:14:10.412 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2921000040}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3309262652}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1186635017}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1399736812}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3604005373}) 2024-09-16T15:14:10.413 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.357+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1368831697}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.358+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.358+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.358+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=7333167}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.358+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3827463055}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=35898473}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1964972279}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.414 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3424067518}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=621617796}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1462598121}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1542129633}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.359+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1892276312}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4149737916}) 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.415 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1034423336}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2548677832}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2775357942}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3274792592}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4142282360}) 2024-09-16T15:14:10.416 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3306132782}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=124507641}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.360+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1646140353}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1340876036}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=351477184}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.417 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1487834189}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1425998064}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4159748534}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=662498787}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=766435593}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4268943103}) 2024-09-16T15:14:10.418 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.361+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=77990009}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2396240355}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3174020790}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=945899777}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2963416969}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.419 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3278549484}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3091822328}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3700029683}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1316686188}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.362+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3057476773}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.363+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.363+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.363+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4112853751}) 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.363+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.420 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.363+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4144977410}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3392193334}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4265861212}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4257910073}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1759542377}) 2024-09-16T15:14:10.421 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=892381177}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1571183541}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.364+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2309260224}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=640998217}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4091505209}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.422 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=138737274}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=28948660}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=830829348}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4209045315}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=372370749}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836378627}) 2024-09-16T15:14:10.423 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.365+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2457899463}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=503160555}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1606506942}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3554067053}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2082270340}) 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.424 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249632249}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2889538693}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3807718269}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4019138388}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.366+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3708034263}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3341686299}) 2024-09-16T15:14:10.425 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=372992562}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2197833309}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3178883328}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1862525910}) 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.426 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3115488862}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2768674396}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4226575233}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.367+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1081527403}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1291718511}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=606849538}) 2024-09-16T15:14:10.427 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3691205611}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3922928048}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238192895}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1087031990}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.428 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1342625336}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1482509246}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.369+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3488928916}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2606662545}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3265510856}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2998749886}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.429 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2251280112}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1651930584}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1657169282}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2298209464}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2663409110}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.370+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3638390917}) 2024-09-16T15:14:10.430 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=289672952}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3171973800}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2407097773}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3632034531}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.431 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1130747071}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1085308134}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2145611432}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3000086493}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.371+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2828083127}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2038787566}) 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.432 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3312900440}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1569431523}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3103435386}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2272897426}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.372+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3055129948}) 2024-09-16T15:14:10.433 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=858398848}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2588215877}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463450532}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1605110573}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1060018156}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3905019189}) 2024-09-16T15:14:10.434 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=644689244}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.373+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3368264587}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2094763949}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2578630452}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1539955198}) 2024-09-16T15:14:10.435 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=676840133}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2403314706}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3897509827}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3735361630}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.374+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4057851044}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.436 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2042197281}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=965434927}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=313864669}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=603853475}) 2024-09-16T15:14:10.437 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.375+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3592768243}) 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=774677140}) 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=492824590}) 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.493+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2555917766}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1935408509}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=369849572}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1385292489}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1335940730}) 2024-09-16T15:14:10.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325650029}) 2024-09-16T15:14:10.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.494+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4171449600}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.515+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.515+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.515+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=55} crc {mdsmap=1531151339,mgr=1087392413,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1060406612,mon_config_key=3884657887,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3588172181}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.515+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.515+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=52} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1403071389,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=2734703586}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=52} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1060406612,mon_config_key=2125290127,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=2860563177}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=13,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=3140514009,mgr=1248521450,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1060406612,mon_config_key=3500614978,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=1859489471}) 2024-09-16T15:14:10.519 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3741057474,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723899,osd_pg_creating=1888939473,osd_snap=1433077607}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=17,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=53} crc {mdsmap=1531151339,mgr=897917685,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1060406612,mon_config_key=1797494094,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939474,osd_snap=1225116132}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=55} crc {mdsmap=1531151339,mgr=3105783184,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=4007214249,mon_config_key=3711370568,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=698394196}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=14,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=57} crc {mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1246853027,mon_config_key=946731908,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1888939473,osd_snap=3146360857}) 2024-09-16T15:14:10.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:10.516+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=16,monmap=2,osd_metadata=5,osd_snap=56} crc {mdsmap=3140514009,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1060406612,mon_config_key=3388801388,monmap=2200900527,osd_metadata=2788994666,osd_snap=3590670105}) 2024-09-16T15:14:10.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.543+0000 7fcb64708640 10 monclient: get_monmap_and_config 2024-09-16T15:14:10.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.543+0000 7fcb64708640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.543+0000 7fcb64708640 10 monclient: monmap: 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:10.544470+0000 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:10.544470+0000 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.545 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.546 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.546 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.546 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:10.546 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:10.546 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 Processor -- start 2024-09-16T15:14:10.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 -- start start 2024-09-16T15:14:10.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient: init 2024-09-16T15:14:10.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient: _add_conns ranks=[0,5,2,4,10,7,9,1,3,8,6] 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcb5c136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c137220 0x7fcb5c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-h con 0x7fcb5c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 0x7fcb5c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-b con 0x7fcb5c13c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c073250 con 0x7fcb5c136e40 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0a56e0 con 0x7fcb5c13c370 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c09abc0 con 0x7fcb5c137220 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb64708640 10 monclient(hunting): authenticate will time out at 1587.133778s 2024-09-16T15:14:10.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb624a4640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcb5c136e40 0x7fcb5c09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32974/0 (socket says 172.21.15.134:32974) 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb624a4640 1 -- 172.21.15.134:0/2802804741 learned_addr learned my addr 172.21.15.134:0/2802804741 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 0x7fcb5c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb62ca5640 10 monclient(hunting): get_auth_request con 0x7fcb5c13c370 auth_method 0 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb62ca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb62ca5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.544+0000 7fcb62ca5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c137220 0x7fcb5c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request con 0x7fcb5c137220 auth_method 0 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb61ca3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb61ca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_done global_id 94178 payload 274 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2802804741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c137220 msgr2=0x7fcb5c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c137220 0x7fcb5c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2802804741 >> v1:172.21.15.134:6789/0 conn(0x7fcb5c136e40 legacy=0x7fcb5c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 1 monclient: found mon.noname-b 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.545+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2802804741 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c09ba50 con 0x7fcb5c13c370 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.547+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 msgr2=0x7fcb5c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.547+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 0x7fcb5c13e760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009dc0 tx=0x7fcb5402a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e343524b24b1565 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 1 -- 172.21.15.134:0/2802804741 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb54007bc0 con 0x7fcb5c13c370 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: dump: 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.559 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 1 -- 172.21.15.134:0/2802804741 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5402d050 con 0x7fcb5c13c370 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 5 monclient: authenticate success, global_id 94178 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 1 -- 172.21.15.134:0/2802804741 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb54007d40 con 0x7fcb5c13c370 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 4 set_mon_vals no callback set 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb614a2640 10 monclient: dump: 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.561 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 10 monclient: shutdown 2024-09-16T15:14:10.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 -- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 msgr2=0x7fcb5c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 0x7fcb5c13e760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009dc0 tx=0x7fcb5402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 -- 172.21.15.134:0/2802804741 shutdown_connections 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c13c370 0x7fcb5c13e760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 --2- 172.21.15.134:0/2802804741 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c137220 0x7fcb5c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 -- 172.21.15.134:0/2802804741 >> 172.21.15.134:0/2802804741 conn(0x7fcb5c0a6110 msgr2=0x7fcb5c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 -- 172.21.15.134:0/2802804741 shutdown_connections 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.548+0000 7fcb64708640 1 -- 172.21.15.134:0/2802804741 wait complete. 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.549+0000 7fcb64708640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.549+0000 7fcb64708640 10 monclient: monmap: 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:10.563 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.549+0000 7fcb64708640 1 Processor -- start 2024-09-16T15:14:10.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.549+0000 7fcb64708640 1 -- start start 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.549+0000 7fcb64708640 10 monclient: init 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient: _add_conns ranks=[2,6,5,7,8,0,3,4,1] 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 0x7fcb5c1cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-b con 0x7fcb5c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c137220 0x7fcb5c1cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-d con 0x7fcb5c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c13c370 0x7fcb5c0a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): picked mon.noname-h con 0x7fcb5c13c370 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0a7410 con 0x7fcb5c136e40 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c122440 con 0x7fcb5c13c370 2024-09-16T15:14:10.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c120d20 con 0x7fcb5c137220 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb64708640 10 monclient(hunting): authenticate will time out at 1587.139226s 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 0x7fcb5c1cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 0x7fcb5c1cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54196/0 (socket says 172.21.15.134:54196) 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 -- 172.21.15.134:0/2375666008 learned_addr learned my addr 172.21.15.134:0/2375666008 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): get_auth_request con 0x7fcb5c136e40 auth_method 0 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c137220 0x7fcb5c1cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request con 0x7fcb5c137220 auth_method 0 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c13c370 0x7fcb5c0a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): get_auth_request con 0x7fcb5c13c370 auth_method 0 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb62ca5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient(hunting): handle_auth_done global_id 94187 payload 824 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c137220 msgr2=0x7fcb5c1cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c137220 0x7fcb5c1cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c13c370 msgr2=0x7fcb5c0a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fcb5c13c370 0x7fcb5c0a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 monclient: found mon.noname-b 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.550+0000 7fcb624a4640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009a70 con 0x7fcb5c136e40 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb624a4640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 msgr2=0x7fcb5c1cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb624a4640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 0x7fcb5c1cb770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800c460 tx=0x7fcb5800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb58005cf0 con 0x7fcb5c136e40 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 10 monclient: dump: 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.569 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb58005e90 con 0x7fcb5c136e40 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb64708640 5 monclient: authenticate success, global_id 94187 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb64708640 10 monclient: _renew_subs 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb64708640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.552+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c13b410 con 0x7fcb5c136e40 2024-09-16T15:14:10.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb580074e0 con 0x7fcb5c136e40 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 10 monclient: dump: 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.571 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb60ca1640 4 set_mon_vals no callback set 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 10 monclient: _renew_subs 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c120120 con 0x7fcb5c136e40 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 0x7fcb5c09d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1225d0 con 0x7fcb5c09d4f0 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fcb58007070 con 0x7fcb5c136e40 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 0x7fcb5c09d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient: get_auth_request con 0x7fcb5c09d4f0 auth_method 0 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.553+0000 7fcb52ffd640 1 --2- 172.21.15.134:0/2375666008 >> v2:172.21.15.162:6800/980547263 conn(0x7fcb400387b0 0x7fcb4003ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.554+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fcb58049bc0 con 0x7fcb5c136e40 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.554+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_done global_id 94191 payload 824 2024-09-16T15:14:10.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.554+0000 7fcb61ca3640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb5c061ae0 con 0x7fcb5c09d4f0 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.554+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2375666008 >> v2:172.21.15.162:6800/980547263 conn(0x7fcb400387b0 0x7fcb4003ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.554+0000 7fcb62ca5640 10 monclient: get_auth_request con 0x7fcb400387b0 auth_method 0 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb61ca3640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 msgr2=0x7fcb5c09d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 0x7fcb5c09d8d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c13be90 tx=0x7fcb4c010c10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb4c010e60 con 0x7fcb5c09d4f0 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 10 monclient: dump: 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.574 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcb4c004450 con 0x7fcb5c09d4f0 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 msgr2=0x7fcb5c09d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb62ca5640 1 --2- 172.21.15.134:0/2375666008 >> v2:172.21.15.162:6800/980547263 conn(0x7fcb400387b0 0x7fcb4003ac70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009dc0 tx=0x7fcb5402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 0x7fcb5c09d8d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c13be90 tx=0x7fcb4c010c10 comp rx=0 tx=0).stop 2024-09-16T15:14:10.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 msgr2=0x7fcb5c09d8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.556+0000 7fcb52ffd640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09d4f0 0x7fcb5c09d8d0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c13be90 tx=0x7fcb4c010c10 comp rx=0 tx=0).stop 2024-09-16T15:14:10.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.562+0000 7fcb64708640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:10.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.562+0000 7fcb64708640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 0x7fcb5c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.562+0000 7fcb64708640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.562+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c071b20 con 0x7fcb5c09de10 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 0x7fcb5c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient: get_auth_request con 0x7fcb5c09de10 auth_method 0 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 10 monclient(hunting): handle_auth_done global_id 94200 payload 824 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.563+0000 7fcb61ca3640 1 -- 172.21.15.134:0/2375666008 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb5c07a680 con 0x7fcb5c09de10 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb61ca3640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 msgr2=0x7fcb5c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb61ca3640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 0x7fcb5c0a2a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c01f460 tx=0x7fcb4c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcb4c024070 con 0x7fcb5c09de10 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 10 monclient: dump: 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fcb4c00ecb0 con 0x7fcb5c09de10 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 msgr2=0x7fcb5c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 0x7fcb5c0a2a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c01f460 tx=0x7fcb4c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 msgr2=0x7fcb5c0a2a60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.565+0000 7fcb52ffd640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fcb5c09de10 0x7fcb5c0a2a60 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c01f460 tx=0x7fcb4c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.580 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:10.581 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.582 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:10.584 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.585 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.586 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.587 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.588 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.589 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.590 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.591 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.592 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.593 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.594 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.595 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.596 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.597 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.599 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.601 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.602 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.602 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 >> v2:172.21.15.162:6800/980547263 conn(0x7fcb400387b0 msgr2=0x7fcb4003ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 --2- 172.21.15.134:0/2375666008 >> v2:172.21.15.162:6800/980547263 conn(0x7fcb400387b0 0x7fcb4003ac70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009dc0 tx=0x7fcb5402f000 comp rx=0 tx=0).stop 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 10 monclient: shutdown 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 msgr2=0x7fcb5c1cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 --2- 172.21.15.134:0/2375666008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb5c136e40 0x7fcb5c1cb770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800c460 tx=0x7fcb5800c920 comp rx=0 tx=0).stop 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2375666008 reap_dead start 2024-09-16T15:14:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb62ca5640 1 -- 172.21.15.134:0/2375666008 reap_dead start 2024-09-16T15:14:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 shutdown_connections 2024-09-16T15:14:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 >> 172.21.15.134:0/2375666008 conn(0x7fcb5c0a6110 msgr2=0x7fcb5c13a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 shutdown_connections 2024-09-16T15:14:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.566+0000 7fcb64708640 1 -- 172.21.15.134:0/2375666008 wait complete. 2024-09-16T15:14:10.604 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.604 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:14:10.604 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.701+0000 7fb243053640 10 monclient: get_monmap_and_config 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.701+0000 7fb243053640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.701+0000 7fb243053640 10 monclient: monmap: 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:10.702644+0000 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:10.702644+0000 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.703 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:10.704 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.702+0000 7fb243053640 1 Processor -- start 2024-09-16T15:14:10.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.702+0000 7fb243053640 1 -- start start 2024-09-16T15:14:10.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.702+0000 7fb243053640 10 monclient: init 2024-09-16T15:14:10.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.702+0000 7fb243053640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient: _add_conns ranks=[0,9,1,5,7,10,6,2,8,3,4] 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb23c136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb23c137220 0x7fb23c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): picked mon.noname-a con 0x7fb23c137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb23c13c370 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c073250 con 0x7fb23c136e40 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c09ba50 con 0x7fb23c13c370 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0a56e0 con 0x7fb23c137220 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb243053640 10 monclient(hunting): authenticate will time out at 1587.292089s 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb23c137220 0x7fb23c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb240def640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb23c136e40 0x7fb23c09c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32976/0 (socket says 172.21.15.134:32976) 2024-09-16T15:14:10.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb240def640 1 -- 172.21.15.134:0/88967367 learned_addr learned my addr 172.21.15.134:0/88967367 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb23c137220 0x7fb23c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59088/0 (socket says 172.21.15.134:59088) 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): get_auth_request con 0x7fb23c137220 auth_method 0 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient(hunting): handle_auth_done global_id 74817 payload 274 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 -- 172.21.15.134:0/88967367 >> v1:172.21.15.162:6789/0 conn(0x7fb23c13c370 legacy=0x7fb23c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 -- 172.21.15.134:0/88967367 >> v1:172.21.15.134:6789/0 conn(0x7fb23c136e40 legacy=0x7fb23c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 monclient: found mon.noname-a 2024-09-16T15:14:10.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:14:10.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.703+0000 7fb23bfff640 1 -- 172.21.15.134:0/88967367 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0a53c0 con 0x7fb23c137220 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23bfff640 1 -- 172.21.15.134:0/88967367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb23c137220 msgr2=0x7fb23c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23bfff640 1 --2- 172.21.15.134:0/88967367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb23c137220 0x7fb23c137600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2280097a0 tx=0x7fb22802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ef1a25877f6886 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 1 -- 172.21.15.134:0/88967367 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb22802ae60 con 0x7fb23c137220 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: dump: 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.715 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb243053640 5 monclient: authenticate success, global_id 74817 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb243053640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 1 -- 172.21.15.134:0/88967367 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2280307b0 con 0x7fb23c137220 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb243053640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb243053640 4 set_mon_vals no callback set 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 1 -- 172.21.15.134:0/88967367 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb228030d90 con 0x7fb23c137220 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.706+0000 7fb23b7fe640 10 monclient: dump: 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.717 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 10 monclient: shutdown 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 -- 172.21.15.134:0/88967367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb23c137220 msgr2=0x7fb23c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 --2- 172.21.15.134:0/88967367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb23c137220 0x7fb23c137600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2280097a0 tx=0x7fb22802a060 comp rx=0 tx=0).stop 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 -- 172.21.15.134:0/88967367 shutdown_connections 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 --2- 172.21.15.134:0/88967367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb23c137220 0x7fb23c137600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 -- 172.21.15.134:0/88967367 >> 172.21.15.134:0/88967367 conn(0x7fb23c0a6110 msgr2=0x7fb23c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 -- 172.21.15.134:0/88967367 shutdown_connections 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 1 -- 172.21.15.134:0/88967367 wait complete. 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.707+0000 7fb243053640 10 monclient: monmap: 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:10.719 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 Processor -- start 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 -- start start 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient: init 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient: _add_conns ranks=[3,2,8,0,5,1,7,4,6] 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb23c136e40 0x7fb23c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): picked mon.noname-g con 0x7fb23c136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 0x7fb23c1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): picked mon.noname-b con 0x7fb23c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 0x7fb23c1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): picked mon.noname-e con 0x7fb23c13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c1225b0 con 0x7fb23c137220 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0a7940 con 0x7fb23c136e40 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0a7410 con 0x7fb23c13c370 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb243053640 10 monclient(hunting): authenticate will time out at 1587.297731s 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 0x7fb23c1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 0x7fb23c1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 0x7fb23c1201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54210/0 (socket says 172.21.15.134:54210) 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): get_auth_request con 0x7fb23c13c370 auth_method 0 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 1 -- 172.21.15.134:0/1725841023 learned_addr learned my addr 172.21.15.134:0/1725841023 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): get_auth_request con 0x7fb23c137220 auth_method 0 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb2415f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.708+0000 7fb23bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb23c136e40 0x7fb23c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 10 monclient(hunting): get_auth_request con 0x7fb23c136e40 auth_method 0 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 10 monclient(hunting): handle_auth_done global_id 84437 payload 824 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb23c136e40 msgr2=0x7fb23c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb23c136e40 0x7fb23c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 msgr2=0x7fb23c1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 0x7fb23c1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 monclient: found mon.noname-e 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009400 con 0x7fb23c13c370 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb23c136e40 0x7fb23c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.709+0000 7fb23bfff640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb23c137220 0x7fb23c1201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 msgr2=0x7fb23c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2415f0640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 0x7fb23c1206e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb23400be00 tx=0x7fb23400bef0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb23400ce80 con 0x7fb23c13c370 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: dump: 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.724 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2340097f0 con 0x7fb23c13c370 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb234014d80 con 0x7fb23c13c370 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 5 monclient: authenticate success, global_id 84437 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 10 monclient: _renew_subs 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c13a8f0 con 0x7fb23c13c370 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb23affd640 4 set_mon_vals no callback set 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb2397fa640 10 monclient: dump: 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.726 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 10 monclient: _renew_subs 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:10.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.711+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c120d20 con 0x7fb23c13c370 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb2340148f0 con 0x7fb23c13c370 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb243053640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb243053640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 0x7fb23c1358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c071b20 con 0x7fb23c1354e0 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 0x7fb23c1358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb2397fa640 1 --2- 172.21.15.134:0/1725841023 >> v2:172.21.15.162:6800/980547263 conn(0x7fb220038880 0x7fb22003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient: get_auth_request con 0x7fb23c1354e0 auth_method 0 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fb23404a040 con 0x7fb23c13c370 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 10 monclient(hunting): handle_auth_done global_id 84446 payload 824 2024-09-16T15:14:10.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb240def640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb23c061ae0 con 0x7fb23c1354e0 2024-09-16T15:14:10.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb23bfff640 1 --2- 172.21.15.134:0/1725841023 >> v2:172.21.15.162:6800/980547263 conn(0x7fb220038880 0x7fb22003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.712+0000 7fb23bfff640 10 monclient: get_auth_request con 0x7fb220038880 auth_method 0 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb240def640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 msgr2=0x7fb23c1358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 0x7fb23c1358c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb23000e520 tx=0x7fb23000e9e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb230005cf0 con 0x7fb23c1354e0 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 10 monclient: dump: 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.731 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.732 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb23bfff640 1 --2- 172.21.15.134:0/1725841023 >> v2:172.21.15.162:6800/980547263 conn(0x7fb220038880 0x7fb22003ad40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb22802a0a0 tx=0x7fb22802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb230005e50 con 0x7fb23c1354e0 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 msgr2=0x7fb23c1358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 0x7fb23c1358c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb23000e520 tx=0x7fb23000e9e0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 msgr2=0x7fb23c1358c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.715+0000 7fb2397fa640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c1354e0 0x7fb23c1358c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb243053640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb243053640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 0x7fb23c09e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb243053640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c07a680 con 0x7fb23c135e00 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 0x7fb23c09e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient: get_auth_request con 0x7fb23c135e00 auth_method 0 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 10 monclient(hunting): handle_auth_done global_id 84455 payload 824 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.721+0000 7fb240def640 1 -- 172.21.15.134:0/1725841023 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb23c09abc0 con 0x7fb23c135e00 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb240def640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 msgr2=0x7fb23c09e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb240def640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 0x7fb23c09e520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb230017000 tx=0x7fb230004940 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb230007180 con 0x7fb23c135e00 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:10.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 10 monclient: dump: 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.735 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fb230013420 con 0x7fb23c135e00 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 msgr2=0x7fb23c09e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 0x7fb23c09e520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb230017000 tx=0x7fb230004940 comp rx=0 tx=0).stop 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 msgr2=0x7fb23c09e520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2397fa640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c135e00 0x7fb23c09e520 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb230017000 tx=0x7fb230004940 comp rx=0 tx=0).stop 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:10.736 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:10.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:10.739 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.740 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:10.741 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.742 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.744 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.745 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.746 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.748 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.751 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.753 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:14:10.757 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.757 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.757 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.757 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 >> v2:172.21.15.162:6800/980547263 conn(0x7fb220038880 msgr2=0x7fb22003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 --2- 172.21.15.134:0/1725841023 >> v2:172.21.15.162:6800/980547263 conn(0x7fb220038880 0x7fb22003ad40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb22802a0a0 tx=0x7fb22802e000 comp rx=0 tx=0).stop 2024-09-16T15:14:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 10 monclient: shutdown 2024-09-16T15:14:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 msgr2=0x7fb23c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 --2- 172.21.15.134:0/1725841023 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb23c13c370 0x7fb23c1206e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb23400be00 tx=0x7fb23400bef0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 reap_dead start 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb2415f0640 1 -- 172.21.15.134:0/1725841023 reap_dead start 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 shutdown_connections 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 >> 172.21.15.134:0/1725841023 conn(0x7fb23c0a6110 msgr2=0x7fb23c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 shutdown_connections 2024-09-16T15:14:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.724+0000 7fb243053640 1 -- 172.21.15.134:0/1725841023 wait complete. 2024-09-16T15:14:10.759 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.759 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:14:10.759 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:14:10.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.856+0000 7fa686f3e640 10 monclient: get_monmap_and_config 2024-09-16T15:14:10.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.856+0000 7fa686f3e640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.856+0000 7fa686f3e640 10 monclient: monmap: 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:10.857278+0000 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:10.857278+0000 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.858 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:10.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 Processor -- start 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 -- start start 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient: init 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient: _add_conns ranks=[0,9,4,3,5,1,2,8,7,6,10] 2024-09-16T15:14:10.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa680136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-a con 0x7fa680137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-c con 0x7fa68013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa680073250 con 0x7fa680136e40 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6800a56e0 con 0x7fa680137220 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa68009abc0 con 0x7fa68013c370 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa686f3e640 10 monclient(hunting): authenticate will time out at 1587.446588s 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa684cda640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa680136e40 0x7fa68009c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:32986/0 (socket says 172.21.15.134:32986) 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:49850/0 (socket says 172.21.15.134:49850) 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59092/0 (socket says 172.21.15.134:59092) 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa684cda640 1 -- 172.21.15.134:0/1853640912 learned_addr learned my addr 172.21.15.134:0/1853640912 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): get_auth_request con 0x7fa68013c370 auth_method 0 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): get_auth_request con 0x7fa680137220 auth_method 0 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient(hunting): handle_auth_done global_id 5728 payload 274 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 -- 172.21.15.134:0/1853640912 >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 msgr2=0x7fa680137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 --2- 172.21.15.134:0/1853640912 >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 -- 172.21.15.134:0/1853640912 >> v1:172.21.15.134:6789/0 conn(0x7fa680136e40 legacy=0x7fa68009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 1 monclient: found mon.noname-c 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.857+0000 7fa6854db640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:10.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.858+0000 7fa6854db640 1 -- 172.21.15.134:0/1853640912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68009ba50 con 0x7fa68013c370 2024-09-16T15:14:10.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.858+0000 7fa67ffff640 1 --2- 172.21.15.134:0/1853640912 >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa6854db640 1 -- 172.21.15.134:0/1853640912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 msgr2=0x7fa68013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa6854db640 1 --2- 172.21.15.134:0/1853640912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa678009d90 tx=0x7fa67802a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=271ef3ab25a39bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 1 -- 172.21.15.134:0/1853640912 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa678007bc0 con 0x7fa68013c370 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: dump: 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.872 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.873 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 1 -- 172.21.15.134:0/1853640912 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67802d050 con 0x7fa68013c370 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa686f3e640 5 monclient: authenticate success, global_id 5728 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa686f3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa686f3e640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa686f3e640 4 set_mon_vals no callback set 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 1 -- 172.21.15.134:0/1853640912 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa678007d40 con 0x7fa68013c370 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.860+0000 7fa67f7fe640 10 monclient: dump: 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.874 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 10 monclient: shutdown 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 -- 172.21.15.134:0/1853640912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 msgr2=0x7fa68013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 --2- 172.21.15.134:0/1853640912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa678009d90 tx=0x7fa67802a4e0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 -- 172.21.15.134:0/1853640912 shutdown_connections 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 --2- 172.21.15.134:0/1853640912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa68013e760 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 --2- 172.21.15.134:0/1853640912 >> v2:172.21.15.134:3300/0 conn(0x7fa680137220 0x7fa680137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 -- 172.21.15.134:0/1853640912 >> 172.21.15.134:0/1853640912 conn(0x7fa6800a6110 msgr2=0x7fa6800a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 -- 172.21.15.134:0/1853640912 shutdown_connections 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.861+0000 7fa686f3e640 1 -- 172.21.15.134:0/1853640912 wait complete. 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 10 monclient: build_initial_monmap 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 10 monclient: monmap: 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:10.876 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 1 Processor -- start 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 1 -- start start 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 10 monclient: init 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.862+0000 7fa686f3e640 10 monclient: _add_conns ranks=[5,0,4,8,6,7,3,1,2] 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa680136e40 0x7fa6801cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-h con 0x7fa680136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 0x7fa6801cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-a con 0x7fa680137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa680139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): picked mon.noname-c con 0x7fa68013c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa680136480 con 0x7fa680137220 2024-09-16T15:14:10.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa680121c80 con 0x7fa68013c370 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa680121ed0 con 0x7fa680136e40 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa686f3e640 10 monclient(hunting): authenticate will time out at 1587.452069s 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 0x7fa6801cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa680139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 0x7fa6801cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59098/0 (socket says 172.21.15.134:59098) 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 1 -- 172.21.15.134:0/3699654865 learned_addr learned my addr 172.21.15.134:0/3699654865 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): get_auth_request con 0x7fa680137220 auth_method 0 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): get_auth_request con 0x7fa68013c370 auth_method 0 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa680136e40 0x7fa6801cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): get_auth_request con 0x7fa680136e40 auth_method 0 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa684cda640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient(hunting): handle_auth_done global_id 5737 payload 824 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa680136e40 msgr2=0x7fa6801cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa680136e40 0x7fa6801cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 msgr2=0x7fa6801cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 0x7fa6801cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient: _finish_hunting 0 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 monclient: found mon.noname-c 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c009400 con 0x7fa68013c370 2024-09-16T15:14:10.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.863+0000 7fa67ffff640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa680137220 0x7fa6801cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 msgr2=0x7fa680139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa6854db640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa680139d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa68009cb70 tx=0x7fa678004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa678007ea0 con 0x7fa68013c370 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: dump: 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.882 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6780044b0 con 0x7fa68013c370 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 5 monclient: authenticate success, global_id 5737 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 10 monclient: _renew_subs 2024-09-16T15:14:10.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa678004620 con 0x7fa68013c370 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa678009a70 con 0x7fa68013c370 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67effd640 4 set_mon_vals no callback set 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa67d7fa640 10 monclient: dump: 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.884 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 10 monclient: _renew_subs 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.865+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6800a78d0 con 0x7fa68013c370 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa678004040 con 0x7fa68013c370 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa67d7fa640 1 --2- 172.21.15.134:0/3699654865 >> v2:172.21.15.162:6800/980547263 conn(0x7fa6640388b0 0x7fa66403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa686f3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa686f3e640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 0x7fa68013a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fa67806af90 con 0x7fa68013c370 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.866+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa680071b20 con 0x7fa680122060 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 0x7fa68013a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa684cda640 1 --2- 172.21.15.134:0/3699654865 >> v2:172.21.15.162:6800/980547263 conn(0x7fa6640388b0 0x7fa66403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient: get_auth_request con 0x7fa680122060 auth_method 0 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa684cda640 10 monclient: get_auth_request con 0x7fa6640388b0 auth_method 0 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 10 monclient(hunting): handle_auth_done global_id 45010 payload 824 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.867+0000 7fa67ffff640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa680061ae0 con 0x7fa680122060 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa684cda640 1 --2- 172.21.15.134:0/3699654865 >> v2:172.21.15.162:6800/980547263 conn(0x7fa6640388b0 0x7fa66403ad70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa674007c90 tx=0x7fa674007be0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67ffff640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 msgr2=0x7fa68013a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67ffff640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 0x7fa68013a2d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00f470 tx=0x7fa66c00f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66c003a80 con 0x7fa680122060 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.869+0000 7fa67d7fa640 10 monclient: dump: 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.887 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa66c003be0 con 0x7fa680122060 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:10.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 msgr2=0x7fa68013a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 0x7fa68013a2d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00f470 tx=0x7fa66c00f930 comp rx=0 tx=0).stop 2024-09-16T15:14:10.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 msgr2=0x7fa68013a2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.870+0000 7fa67d7fa640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa680122060 0x7fa68013a2d0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00f470 tx=0x7fa66c00f930 comp rx=0 tx=0).stop 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa686f3e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa686f3e640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 0x7fa68013ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa686f3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68007a680 con 0x7fa68013a810 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 0x7fa68013ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient: get_auth_request con 0x7fa68013a810 auth_method 0 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 10 monclient(hunting): handle_auth_done global_id 45019 payload 824 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.876+0000 7fa67ffff640 1 -- 172.21.15.134:0/3699654865 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa6800a53c0 con 0x7fa68013a810 2024-09-16T15:14:10.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.878+0000 7fa67ffff640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 msgr2=0x7fa68013ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.878+0000 7fa67ffff640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 0x7fa68013ac20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00f470 tx=0x7fa66c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa66c02c070 con 0x7fa68013a810 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 10 monclient: dump: 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:10.892 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7fa66c024040 con 0x7fa68013a810 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 msgr2=0x7fa68013ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 0x7fa68013ac20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00f470 tx=0x7fa66c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 msgr2=0x7fa68013ac20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:10.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa67d7fa640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa68013a810 0x7fa68013ac20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:10.894 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.896 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:10.897 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:10.898 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:10.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:10.905 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:10.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.911 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:10.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:10.913 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.914 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:10.915 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:10.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 >> v2:172.21.15.162:6800/980547263 conn(0x7fa6640388b0 msgr2=0x7fa66403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 --2- 172.21.15.134:0/3699654865 >> v2:172.21.15.162:6800/980547263 conn(0x7fa6640388b0 0x7fa66403ad70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa674007c90 tx=0x7fa674007be0 comp rx=0 tx=0).stop 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 10 monclient: shutdown 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 msgr2=0x7fa680139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 --2- 172.21.15.134:0/3699654865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa68013c370 0x7fa680139d90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa68009cb70 tx=0x7fa678004a40 comp rx=0 tx=0).stop 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 reap_dead start 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa6854db640 1 -- 172.21.15.134:0/3699654865 reap_dead start 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 shutdown_connections 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 >> 172.21.15.134:0/3699654865 conn(0x7fa6800a6110 msgr2=0x7fa68006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 shutdown_connections 2024-09-16T15:14:10.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:10.879+0000 7fa686f3e640 1 -- 172.21.15.134:0/3699654865 wait complete. 2024-09-16T15:14:10.917 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:10.918 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:14:10.918 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:14:11.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.014+0000 7fdfaa98a640 10 monclient: get_monmap_and_config 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.014+0000 7fdfaa98a640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.014+0000 7fdfaa98a640 10 monclient: monmap: 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:11.015514+0000 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:11.015514+0000 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:11.016 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:11.017 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 Processor -- start 2024-09-16T15:14:11.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 -- start start 2024-09-16T15:14:11.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient: init 2024-09-16T15:14:11.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient: _add_conns ranks=[2,5,3,6,7,10,4,1,9,8,0] 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-b con 0x7fdfa4136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdfa4137220 0x7fdfa4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-h con 0x7fdfa4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4135dc0 0x7fdfa4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-g con 0x7fdfa4135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa409abc0 con 0x7fdfa4136e40 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa409ba50 con 0x7fdfa4135dc0 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40a53c0 con 0x7fdfa4137220 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.015+0000 7fdfaa98a640 10 monclient(hunting): authenticate will time out at 1587.604977s 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54214/0 (socket says 172.21.15.134:54214) 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 -- 172.21.15.134:0/1065412898 learned_addr learned my addr 172.21.15.134:0/1065412898 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): get_auth_request con 0x7fdfa4136e40 auth_method 0 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdfa4137220 0x7fdfa4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa8f27640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4135dc0 0x7fdfa4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request con 0x7fdfa4137220 auth_method 0 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdf9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa8f27640 10 monclient(hunting): get_auth_request con 0x7fdfa4135dc0 auth_method 0 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa8f27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa8f27640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa8f27640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_done global_id 94205 payload 274 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 -- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdfa4137220 msgr2=0x7fdfa4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdfa4137220 0x7fdfa4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 -- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4135dc0 msgr2=0x7fdfa4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4135dc0 0x7fdfa4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 monclient: found mon.noname-b 2024-09-16T15:14:11.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:11.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.016+0000 7fdfa3fff640 1 -- 172.21.15.134:0/1065412898 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4073250 con 0x7fdfa4136e40 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa3fff640 1 -- 172.21.15.134:0/1065412898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 msgr2=0x7fdfa409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf940098a0 tx=0x7fdf9402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b2608ec4fa1ead server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 1 -- 172.21.15.134:0/1065412898 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf9402ade0 con 0x7fdfa4136e40 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: dump: 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.030 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 1 -- 172.21.15.134:0/1065412898 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf940307b0 con 0x7fdfa4136e40 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 5 monclient: authenticate success, global_id 94205 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 1 -- 172.21.15.134:0/1065412898 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf94030d90 con 0x7fdfa4136e40 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 4 set_mon_vals no callback set 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfa37fe640 10 monclient: dump: 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.032 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 10 monclient: shutdown 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 -- 172.21.15.134:0/1065412898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 msgr2=0x7fdfa409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf940098a0 tx=0x7fdf9402a080 comp rx=0 tx=0).stop 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 -- 172.21.15.134:0/1065412898 shutdown_connections 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4135dc0 0x7fdfa4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdfa4137220 0x7fdfa4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/1065412898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4136e40 0x7fdfa409c150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 -- 172.21.15.134:0/1065412898 >> 172.21.15.134:0/1065412898 conn(0x7fdfa40a6110 msgr2=0x7fdfa40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.019+0000 7fdfaa98a640 1 -- 172.21.15.134:0/1065412898 shutdown_connections 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 1 -- 172.21.15.134:0/1065412898 wait complete. 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 10 monclient: monmap: 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.034 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 1 Processor -- start 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 1 -- start start 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.020+0000 7fdfaa98a640 10 monclient: init 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient: _add_conns ranks=[2,3,7,4,6,8,0,5,1] 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 0x7fdfa411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-b con 0x7fdfa4135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4136e40 0x7fdfa41350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-g con 0x7fdfa4136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfa4137220 0x7fdfa41cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): picked mon.noname-i con 0x7fdfa4137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4122940 con 0x7fdfa4135dc0 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4120c20 con 0x7fdfa4136e40 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4122280 con 0x7fdfa4137220 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfaa98a640 10 monclient(hunting): authenticate will time out at 1587.610380s 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 0x7fdfa411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 0x7fdfa411fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54228/0 (socket says 172.21.15.134:54228) 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 -- 172.21.15.134:0/2809829357 learned_addr learned my addr 172.21.15.134:0/2809829357 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): get_auth_request con 0x7fdfa4135dc0 auth_method 0 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4136e40 0x7fdfa41350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request con 0x7fdfa4136e40 auth_method 0 2024-09-16T15:14:11.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfa4137220 0x7fdfa41cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): get_auth_request con 0x7fdfa4137220 auth_method 0 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa8f27640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient(hunting): handle_auth_done global_id 94214 payload 824 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfa4137220 msgr2=0x7fdfa41cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdfa4137220 0x7fdfa41cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4136e40 msgr2=0x7fdfa41350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa4136e40 0x7fdfa41350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 monclient: found mon.noname-b 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.021+0000 7fdfa3fff640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940094d0 con 0x7fdfa4135dc0 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.023+0000 7fdfa3fff640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 msgr2=0x7fdfa411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.023+0000 7fdfa3fff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 0x7fdfa411fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf940090b0 tx=0x7fdf940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf940044b0 con 0x7fdfa4135dc0 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:11.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: dump: 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.040 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 5 monclient: authenticate success, global_id 94214 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94030d40 con 0x7fdfa4135dc0 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf94032440 con 0x7fdfa4135dc0 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa2ffd640 4 set_mon_vals no callback set 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfa17fa640 10 monclient: dump: 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.041 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 10 monclient: _renew_subs 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:11.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa4122560 con 0x7fdfa4135dc0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 10 monclient: _renew_subs 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.024+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa40a2cb0 con 0x7fdfa4135dc0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfaa98a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdf94039040 con 0x7fdfa4135dc0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 0x7fdfa402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40a2ea0 con 0x7fdfa41355f0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfa17fa640 1 --2- 172.21.15.134:0/2809829357 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf84038880 0x7fdf8403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 0x7fdfa402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(623..623 src has 1..623) v4 ==== 3915+0+0 (secure 0 0 0) 0x7fdf9406a8b0 con 0x7fdfa4135dc0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient: get_auth_request con 0x7fdfa41355f0 auth_method 0 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfa8f27640 1 --2- 172.21.15.134:0/2809829357 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf84038880 0x7fdf8403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdfa8f27640 10 monclient: get_auth_request con 0x7fdf84038880 auth_method 0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_done global_id 84360 payload 824 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.025+0000 7fdf9bfff640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfa4061ae0 con 0x7fdfa41355f0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.027+0000 7fdf9bfff640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 msgr2=0x7fdfa402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.027+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 0x7fdfa402b260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4120fd0 tx=0x7fdf8c010c80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.027+0000 7fdfa8f27640 1 --2- 172.21.15.134:0/2809829357 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf84038880 0x7fdf8403ad40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdfa406b550 tx=0x7fdf9c009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf8c010e70 con 0x7fdfa41355f0 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 10 monclient: dump: 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.045 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdf8c004450 con 0x7fdfa41355f0 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 msgr2=0x7fdfa402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 0x7fdfa402b260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4120fd0 tx=0x7fdf8c010c80 comp rx=0 tx=0).stop 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 msgr2=0x7fdfa402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.028+0000 7fdfa17fa640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa41355f0 0x7fdfa402b260 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4120fd0 tx=0x7fdf8c010c80 comp rx=0 tx=0).stop 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdfaa98a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 0x7fdfa402bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdfaa98a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa4071b20 con 0x7fdfa402b7a0 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 0x7fdfa402bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient: get_auth_request con 0x7fdfa402b7a0 auth_method 0 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.034+0000 7fdf9bfff640 10 monclient(hunting): handle_auth_done global_id 84369 payload 824 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.035+0000 7fdf9bfff640 1 -- 172.21.15.134:0/2809829357 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdfa407a680 con 0x7fdfa402b7a0 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdf9bfff640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 msgr2=0x7fdfa402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdf9bfff640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 0x7fdfa402bbb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c019000 tx=0x7fdf8c0046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf8c024070 con 0x7fdfa402b7a0 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 10 monclient: dump: 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.049 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9197 (secure 0 0 0) 0x7fdf8c01f2e0 con 0x7fdfa402b7a0 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:11.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 msgr2=0x7fdfa402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 0x7fdfa402bbb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c019000 tx=0x7fdf8c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 msgr2=0x7fdfa402bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.037+0000 7fdfa17fa640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfa402b7a0 0x7fdfa402bbb0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c019000 tx=0x7fdf8c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 10, 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:11.056 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.058 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.059 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:11.062 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:11.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.066 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.067 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.068 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.069 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:11.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.072 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:11.073 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf84038880 msgr2=0x7fdf8403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/2809829357 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf84038880 0x7fdf8403ad40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdfa406b550 tx=0x7fdf9c009820 comp rx=0 tx=0).stop 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 10 monclient: shutdown 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 msgr2=0x7fdfa411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 --2- 172.21.15.134:0/2809829357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfa4135dc0 0x7fdfa411fc60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf940090b0 tx=0x7fdf940046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfa8f27640 1 -- 172.21.15.134:0/2809829357 reap_dead start 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfa8f27640 1 -- 172.21.15.134:0/2809829357 reap_dead start 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 shutdown_connections 2024-09-16T15:14:11.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 >> 172.21.15.134:0/2809829357 conn(0x7fdfa40a6110 msgr2=0x7fdfa409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 shutdown_connections 2024-09-16T15:14:11.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.038+0000 7fdfaa98a640 1 -- 172.21.15.134:0/2809829357 wait complete. 2024-09-16T15:14:11.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:11.075 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:14:11.075 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=80,osdmap=20} crc {osd_snap=4264768740,osdmap=2076197381}) 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=80,osdmap=20} crc {osd_snap=3861237146,osdmap=1845544850}) 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.079 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=77,osdmap=23} crc {osd_snap=783832981,osdmap=3827367888}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=79,osdmap=21} crc {osd_snap=2082808880,osdmap=470402784}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=80,osdmap=20} crc {osd_snap=3074261728,osdmap=2098839961}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=76,osdmap=24} crc {osd_snap=3177064404,osdmap=2460083301}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=1494171858,osdmap=2159255361}) 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.080 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=839789599,osdmap=260218505}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.069+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=78,osdmap=22} crc {osd_snap=2311063820,osdmap=1062785707}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2829298551}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2712478614}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=954660203}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=599303174}) 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=474607933}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1660594902}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2202132246}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=737896891}) 2024-09-16T15:14:11.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.074+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1177799415}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1103072202}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4033309235}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2114342819}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=862051577}) 2024-09-16T15:14:11.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4017422863}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3340621116}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3500666461}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3928579367}) 2024-09-16T15:14:11.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.103+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=971316019}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3319984719}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3101254037}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1304498528}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1789987648}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2817277699}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2789025351}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3591177866}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=641803709}) 2024-09-16T15:14:11.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.110+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1708400939}) 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:2985: done (0 left) 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:2986: rollback oid 32 current snap is 310 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 306 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:2987: rollback oid 45 current snap is 310 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 306 2024-09-16T15:14:11.114 INFO:tasks.rados.rados.0.smithi162.stdout:2988: snap_create 2024-09-16T15:14:11.115 INFO:tasks.rados.rados.0.smithi162.stdout:2986: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:14:11.116 INFO:tasks.rados.rados.0.smithi162.stdout:2986: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:14:11.116 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1434 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:14:11.117 INFO:tasks.rados.rados.0.smithi162.stdout:2987: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:14:11.120 INFO:tasks.rados.rados.0.smithi162.stdout:2987: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:14:11.120 INFO:tasks.rados.rados.0.smithi162.stdout:2987: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:14:11.120 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1535 (ObjNum 834 snap 224 seq_num 834) dirty exists 2024-09-16T15:14:11.123 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1636089569}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=380702601}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2949265819}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=251952901}) 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3437354950}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2349200143}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2269108996}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.121+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3088461520}) 2024-09-16T15:14:11.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.122+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2297661589}) 2024-09-16T15:14:11.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 10 monclient: get_monmap_and_config 2024-09-16T15:14:11.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 10 monclient: monmap: 2024-09-16T15:14:11.186 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:11.186280+0000 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:11.186280+0000 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.187 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.188 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.188 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:11.188 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:11.188 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 1 Processor -- start 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 1 -- start start 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.185+0000 7f9e197ae640 10 monclient: init 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient: _add_conns ranks=[8,10,1,7,0,3,2,6,9,5,4] 2024-09-16T15:14:11.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-e con 0x7f9e14136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e14137220 0x7f9e14137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-f con 0x7f9e14137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9e14135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e14073250 con 0x7f9e14135dc0 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1409abc0 con 0x7f9e14137220 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1409ba50 con 0x7f9e14136e40 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e197ae640 10 monclient(hunting): authenticate will time out at 1587.775455s 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54334/0 (socket says 172.21.15.134:54334) 2024-09-16T15:14:11.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 -- 172.21.15.134:0/2194526231 learned_addr learned my addr 172.21.15.134:0/2194526231 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request con 0x7f9e14136e40 auth_method 0 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_done global_id 84464 payload 274 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e127fc640 1 --2- 172.21.15.134:0/2194526231 >> v2:172.21.15.162:3300/0 conn(0x7f9e14137220 0x7f9e14137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 -- 172.21.15.134:0/2194526231 >> v2:172.21.15.162:3300/0 conn(0x7f9e14137220 msgr2=0x7f9e14137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/2194526231 >> v2:172.21.15.162:3300/0 conn(0x7f9e14137220 0x7f9e14137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 -- 172.21.15.134:0/2194526231 >> v1:172.21.15.162:6789/0 conn(0x7f9e14135dc0 legacy=0x7f9e14136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 monclient: found mon.noname-e 2024-09-16T15:14:11.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:11.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.186+0000 7f9e12ffd640 1 -- 172.21.15.134:0/2194526231 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e140a53c0 con 0x7f9e14136e40 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e12ffd640 1 -- 172.21.15.134:0/2194526231 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 msgr2=0x7f9e1409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/2194526231 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e040098a0 tx=0x7f9e0402a110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3c9fa38936d563bb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 1 -- 172.21.15.134:0/2194526231 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e040307b0 con 0x7f9e14136e40 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: dump: 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.199 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.200 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 1 -- 172.21.15.134:0/2194526231 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e04030930 con 0x7f9e14136e40 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e197ae640 5 monclient: authenticate success, global_id 84464 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e197ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 1 -- 172.21.15.134:0/2194526231 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e040313a0 con 0x7f9e14136e40 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e197ae640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e197ae640 4 set_mon_vals no callback set 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:11.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.189+0000 7f9e11ffb640 10 monclient: dump: 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.202 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 10 monclient: shutdown 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 -- 172.21.15.134:0/2194526231 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 msgr2=0x7f9e1409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 --2- 172.21.15.134:0/2194526231 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e040098a0 tx=0x7f9e0402a110 comp rx=0 tx=0).stop 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 -- 172.21.15.134:0/2194526231 shutdown_connections 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 --2- 172.21.15.134:0/2194526231 >> v2:172.21.15.162:3300/0 conn(0x7f9e14137220 0x7f9e14137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 --2- 172.21.15.134:0/2194526231 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9e14136e40 0x7f9e1409c150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 -- 172.21.15.134:0/2194526231 >> 172.21.15.134:0/2194526231 conn(0x7f9e140a6110 msgr2=0x7f9e140a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 -- 172.21.15.134:0/2194526231 shutdown_connections 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 1 -- 172.21.15.134:0/2194526231 wait complete. 2024-09-16T15:14:11.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.190+0000 7f9e197ae640 10 monclient: monmap: 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.204 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 Processor -- start 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 -- start start 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient: init 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient: _add_conns ranks=[2,6,0,5,8,3,7,1,4] 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e14135dc0 0x7f9e14134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-b con 0x7f9e14135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 0x7f9e14135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-d con 0x7f9e14136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 0x7f9e141cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): picked mon.noname-a con 0x7f9e14137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e14120d20 con 0x7f9e14137220 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141225f0 con 0x7f9e14135dc0 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9e141364e0 con 0x7f9e14136e40 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.191+0000 7f9e197ae640 10 monclient(hunting): authenticate will time out at 1587.780957s 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 0x7f9e14135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 0x7f9e141cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 0x7f9e141cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59102/0 (socket says 172.21.15.134:59102) 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 0x7f9e14135450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:49790/0 (socket says 172.21.15.134:49790) 2024-09-16T15:14:11.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 1 -- 172.21.15.134:0/1086899647 learned_addr learned my addr 172.21.15.134:0/1086899647 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e14135dc0 0x7f9e14134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): get_auth_request con 0x7f9e14137220 auth_method 0 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): get_auth_request con 0x7f9e14136e40 auth_method 0 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request con 0x7f9e14135dc0 auth_method 0 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient(hunting): handle_auth_done global_id 94209 payload 824 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e14135dc0 msgr2=0x7f9e14134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e14135dc0 0x7f9e14134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 msgr2=0x7f9e141cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 0x7f9e141cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 monclient: found mon.noname-d 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e127fc640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040094d0 con 0x7f9e14136e40 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e137fe640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e14137220 0x7f9e141cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.192+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e14135dc0 0x7f9e14134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e127fc640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 msgr2=0x7f9e14135450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e127fc640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 0x7f9e14135450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c5c0 tx=0x7f9e0800ca80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e08011c80 con 0x7f9e14136e40 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:11.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: dump: 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 5 monclient: authenticate success, global_id 94209 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08011e00 con 0x7f9e14136e40 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 10 monclient: _renew_subs 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e140a3560 con 0x7f9e14136e40 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e080107c0 con 0x7f9e14136e40 2024-09-16T15:14:11.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9dfb7fe640 10 monclient: dump: 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.212 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e117fa640 4 set_mon_vals no callback set 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 10 monclient: _renew_subs 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.194+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e14122310 con 0x7f9e14136e40 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9e197ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9e197ae640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 0x7f9e1402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e14071b20 con 0x7f9e14120130 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9e08010350 con 0x7f9e14136e40 2024-09-16T15:14:11.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9dfb7fe640 1 --2- 172.21.15.134:0/1086899647 >> v2:172.21.15.162:6800/980547263 conn(0x7f9e000388e0 0x7f9e0003ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.195+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(624..624 src has 1..624) v4 ==== 3915+0+0 (secure 0 0 0) 0x7f9e08049b80 con 0x7f9e14136e40 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 0x7f9e1402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient: get_auth_request con 0x7f9e14120130 auth_method 0 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e137fe640 1 --2- 172.21.15.134:0/1086899647 >> v2:172.21.15.162:6800/980547263 conn(0x7f9e000388e0 0x7f9e0003ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e137fe640 10 monclient: get_auth_request con 0x7f9e000388e0 auth_method 0 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_done global_id 45023 payload 824 2024-09-16T15:14:11.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.196+0000 7f9e12ffd640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e14061ae0 con 0x7f9e14120130 2024-09-16T15:14:11.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9e137fe640 1 --2- 172.21.15.134:0/1086899647 >> v2:172.21.15.162:6800/980547263 conn(0x7f9e000388e0 0x7f9e0003ada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9e140a30e0 tx=0x7f9e0c003de0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9e12ffd640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 msgr2=0x7f9e1402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 0x7f9e1402b260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e1406b4b0 tx=0x7f9e040046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e04004550 con 0x7f9e14120130 2024-09-16T15:14:11.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9dfb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.198+0000 7f9dfb7fe640 10 monclient: dump: 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9e04003820 con 0x7f9e14120130 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 msgr2=0x7f9e1402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 0x7f9e1402b260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e1406b4b0 tx=0x7f9e040046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 msgr2=0x7f9e1402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.199+0000 7f9dfb7fe640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e14120130 0x7f9e1402b260 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e1406b4b0 tx=0x7f9e040046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e197ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e197ae640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 0x7f9e1402bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e197ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1407a680 con 0x7f9e1402b7a0 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 0x7f9e1402bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient: get_auth_request con 0x7f9e1402b7a0 auth_method 0 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 10 monclient(hunting): handle_auth_done global_id 45032 payload 824 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.205+0000 7f9e12ffd640 1 -- 172.21.15.134:0/1086899647 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e1409b490 con 0x7f9e1402b7a0 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9e12ffd640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 msgr2=0x7f9e1402bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9e12ffd640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 0x7f9e1402bb80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e0402a150 tx=0x7f9e04008030 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9e0404b2f0 con 0x7f9e1402b7a0 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 10 monclient: dump: 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.221 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9529 (secure 0 0 0) 0x7f9e0403cbe0 con 0x7f9e1402b7a0 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 msgr2=0x7f9e1402bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 0x7f9e1402bb80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e0402a150 tx=0x7f9e04008030 comp rx=0 tx=0).stop 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 msgr2=0x7f9e1402bb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.208+0000 7f9dfb7fe640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9e1402b7a0 0x7f9e1402bb80 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e0402a150 tx=0x7f9e04008030 comp rx=0 tx=0).stop 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:11.223 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 11, 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.224 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.225 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:11.226 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.227 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:11.228 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.229 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.232 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:11.234 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.235 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.239 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.242 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.243 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.245 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.245 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.245 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:11.245 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 >> v2:172.21.15.162:6800/980547263 conn(0x7f9e000388e0 msgr2=0x7f9e0003ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 --2- 172.21.15.134:0/1086899647 >> v2:172.21.15.162:6800/980547263 conn(0x7f9e000388e0 0x7f9e0003ada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9e140a30e0 tx=0x7f9e0c003de0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 10 monclient: shutdown 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 msgr2=0x7f9e14135450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 --2- 172.21.15.134:0/1086899647 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9e14136e40 0x7f9e14135450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c5c0 tx=0x7f9e0800ca80 comp rx=0 tx=0).stop 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e137fe640 1 -- 172.21.15.134:0/1086899647 reap_dead start 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e137fe640 1 -- 172.21.15.134:0/1086899647 reap_dead start 2024-09-16T15:14:11.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 shutdown_connections 2024-09-16T15:14:11.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 >> 172.21.15.134:0/1086899647 conn(0x7f9e140a6110 msgr2=0x7f9e1409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 shutdown_connections 2024-09-16T15:14:11.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.209+0000 7f9e197ae640 1 -- 172.21.15.134:0/1086899647 wait complete. 2024-09-16T15:14:11.247 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:11.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:14:11.247 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:14:11.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.383+0000 7ff2f5c61640 10 monclient: get_monmap_and_config 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.383+0000 7ff2f5c61640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.383+0000 7ff2f5c61640 10 monclient: monmap: 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:11.384446+0000 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:11.384446+0000 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:11.385 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:11.386 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 Processor -- start 2024-09-16T15:14:11.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 -- start start 2024-09-16T15:14:11.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient: init 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient: _add_conns ranks=[2,8,9,0,6,7,1,5,10,3,4] 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-b con 0x7ff2f0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-e con 0x7ff2f0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-a con 0x7ff2f0135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f009abc0 con 0x7ff2f0135dc0 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f009ba50 con 0x7ff2f0136e40 2024-09-16T15:14:11.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00a53c0 con 0x7ff2f0137220 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2f5c61640 10 monclient(hunting): authenticate will time out at 1587.973754s 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54350/0 (socket says 172.21.15.134:54350) 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:59118/0 (socket says 172.21.15.134:59118) 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54266/0 (socket says 172.21.15.134:54266) 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 1 -- 172.21.15.134:0/1779334524 learned_addr learned my addr 172.21.15.134:0/1779334524 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 10 monclient(hunting): get_auth_request con 0x7ff2f0137220 auth_method 0 2024-09-16T15:14:11.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request con 0x7ff2f0136e40 auth_method 0 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 10 monclient(hunting): get_auth_request con 0x7ff2f0135dc0 auth_method 0 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.384+0000 7ff2effff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_done global_id 94232 payload 274 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 msgr2=0x7ff2f0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/1779334524 >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 msgr2=0x7ff2f0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/1779334524 >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 monclient: found mon.noname-b 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:14:11.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/1779334524 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0073250 con 0x7ff2f0136e40 2024-09-16T15:14:11.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2eeffd640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:11.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.385+0000 7ff2effff640 1 --2- 172.21.15.134:0/1779334524 >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:11.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 msgr2=0x7ff2f009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00097d0 tx=0x7ff2e002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a100a717dfcdec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 1 -- 172.21.15.134:0/1779334524 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e002ad20 con 0x7ff2f0136e40 2024-09-16T15:14:11.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: dump: 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.400 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 1 -- 172.21.15.134:0/1779334524 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e00307b0 con 0x7ff2f0136e40 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 5 monclient: authenticate success, global_id 94232 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 1 -- 172.21.15.134:0/1779334524 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e002aea0 con 0x7ff2f0136e40 2024-09-16T15:14:11.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 4 set_mon_vals no callback set 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2ee7fc640 10 monclient: dump: 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.402 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 10 monclient: shutdown 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 -- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 msgr2=0x7ff2f009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00097d0 tx=0x7ff2e002a000 comp rx=0 tx=0).stop 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 -- 172.21.15.134:0/1779334524 shutdown_connections 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/1779334524 >> v2:172.21.15.134:3300/0 conn(0x7ff2f0135dc0 0x7ff2f0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff2f0137220 0x7ff2f0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/1779334524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0136e40 0x7ff2f009c150 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 -- 172.21.15.134:0/1779334524 >> 172.21.15.134:0/1779334524 conn(0x7ff2f00a6110 msgr2=0x7ff2f00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 -- 172.21.15.134:0/1779334524 shutdown_connections 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.388+0000 7ff2f5c61640 1 -- 172.21.15.134:0/1779334524 wait complete. 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.389+0000 7ff2f5c61640 10 monclient: build_initial_monmap 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.389+0000 7ff2f5c61640 10 monclient: monmap: 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:11.404 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.389+0000 7ff2f5c61640 1 Processor -- start 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.389+0000 7ff2f5c61640 1 -- start start 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.389+0000 7ff2f5c61640 10 monclient: init 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient: _add_conns ranks=[6,4,7,2,5,0,3,8,1] 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 0x7ff2f01cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-d con 0x7ff2f0135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 0x7ff2f0134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-c con 0x7ff2f0136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0137220 0x7ff2f0135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): picked mon.noname-i con 0x7ff2f0137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0125990 con 0x7ff2f0136e40 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0124390 con 0x7ff2f0135dc0 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01368c0 con 0x7ff2f0137220 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2f5c61640 10 monclient(hunting): authenticate will time out at 1587.979426s 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 0x7ff2f01cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 0x7ff2f0134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 0x7ff2f01cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:49798/0 (socket says 172.21.15.134:49798) 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/116407466 learned_addr learned my addr 172.21.15.134:0/116407466 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request con 0x7ff2f0135dc0 auth_method 0 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): get_auth_request con 0x7ff2f0136e40 auth_method 0 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2effff640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0137220 0x7ff2f0135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient(hunting): handle_auth_done global_id 5746 payload 824 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0137220 msgr2=0x7ff2f0135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0137220 0x7ff2f0135450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 msgr2=0x7ff2f01cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 0x7ff2f01cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 monclient: found mon.noname-c 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2eeffd640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8009a70 con 0x7ff2f0136e40 2024-09-16T15:14:11.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.390+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff2f0135dc0 0x7ff2f01cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2eeffd640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 msgr2=0x7ff2f0134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2eeffd640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 0x7ff2f0134f10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e860 tx=0x7ff2e400ed20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e4004030 con 0x7ff2f0136e40 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: dump: 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.409 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: _finish_auth 0 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e40041b0 con 0x7ff2f0136e40 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 5 monclient: authenticate success, global_id 5746 2024-09-16T15:14:11.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 10 monclient: _renew_subs 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e40094d0 con 0x7ff2f0136e40 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e4004320 con 0x7ff2f0136e40 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2d7fff640 10 monclient: dump: 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.411 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 10 monclient: _renew_subs 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e0009400 con 0x7ff2f0136e40 2024-09-16T15:14:11.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.393+0000 7ff2edffb640 4 set_mon_vals no callback set 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2f5c61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 0x7ff2f0121910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0071b20 con 0x7ff2f0125060 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff2e400cb50 con 0x7ff2f0136e40 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2d7fff640 1 --2- 172.21.15.134:0/116407466 >> v2:172.21.15.162:6800/980547263 conn(0x7ff2dc038880 0x7ff2dc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(624..624 src has 1..624) v4 ==== 3915+0+0 (secure 0 0 0) 0x7ff2e404a2e0 con 0x7ff2f0136e40 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 0x7ff2f0121910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient: get_auth_request con 0x7ff2f0125060 auth_method 0 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2effff640 1 --2- 172.21.15.134:0/116407466 >> v2:172.21.15.162:6800/980547263 conn(0x7ff2dc038880 0x7ff2dc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2effff640 10 monclient: get_auth_request con 0x7ff2dc038880 auth_method 0 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_done global_id 54817 payload 824 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.394+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2f0061ae0 con 0x7ff2f0125060 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.397+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 msgr2=0x7ff2f0121910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.397+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 0x7ff2f0121910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0000c00 tx=0x7ff2e00046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.397+0000 7ff2effff640 1 --2- 172.21.15.134:0/116407466 >> v2:172.21.15.162:6800/980547263 conn(0x7ff2dc038880 0x7ff2dc03ad40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00a8be0 tx=0x7ff2e800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e0003c90 con 0x7ff2f0125060 2024-09-16T15:14:11.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:14:11.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 10 monclient: dump: 2024-09-16T15:14:11.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff2e0003df0 con 0x7ff2f0125060 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 msgr2=0x7ff2f0121910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 0x7ff2f0121910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0000c00 tx=0x7ff2e00046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 msgr2=0x7ff2f0121910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.399+0000 7ff2d7fff640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0125060 0x7ff2f0121910 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0000c00 tx=0x7ff2e00046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.405+0000 7ff2f5c61640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.405+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 0x7ff2f0122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.405+0000 7ff2f5c61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.405+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f007a680 con 0x7ff2f0121e50 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 0x7ff2f0122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:11.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient: get_auth_request con 0x7ff2f0121e50 auth_method 0 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 10 monclient(hunting): handle_auth_done global_id 54826 payload 824 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.406+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/116407466 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2f009b490 con 0x7ff2f0121e50 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2ef7fe640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 msgr2=0x7ff2f0122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2ef7fe640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 0x7ff2f0122260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0033000 tx=0x7ff2e0009190 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff2e004c040 con 0x7ff2f0121e50 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:14:11.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 10 monclient: dump: 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:11.420 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9363 (secure 0 0 0) 0x7ff2e003dd30 con 0x7ff2f0121e50 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 msgr2=0x7ff2f0122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 0x7ff2f0122260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0033000 tx=0x7ff2e0009190 comp rx=0 tx=0).stop 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 msgr2=0x7ff2f0122260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2d7fff640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff2f0121e50 0x7ff2f0122260 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0033000 tx=0x7ff2e0009190 comp rx=0 tx=0).stop 2024-09-16T15:14:11.421 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 110, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:14:11.422 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 11, 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.423 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.424 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:11.425 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.426 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.427 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:11.428 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:11.430 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.432 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.433 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.434 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.437 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.438 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:11.439 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:11.440 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:11.441 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.442 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:11.443 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:11.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 >> v2:172.21.15.162:6800/980547263 conn(0x7ff2dc038880 msgr2=0x7ff2dc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/116407466 >> v2:172.21.15.162:6800/980547263 conn(0x7ff2dc038880 0x7ff2dc03ad40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00a8be0 tx=0x7ff2e800b040 comp rx=0 tx=0).stop 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 10 monclient: shutdown 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 msgr2=0x7ff2f0134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 --2- 172.21.15.134:0/116407466 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff2f0136e40 0x7ff2f0134f10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e860 tx=0x7ff2e400ed20 comp rx=0 tx=0).stop 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2effff640 1 -- 172.21.15.134:0/116407466 reap_dead start 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2effff640 1 -- 172.21.15.134:0/116407466 reap_dead start 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 shutdown_connections 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.409+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 >> 172.21.15.134:0/116407466 conn(0x7ff2f00a6110 msgr2=0x7ff2f006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.410+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 shutdown_connections 2024-09-16T15:14:11.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:11.410+0000 7ff2f5c61640 1 -- 172.21.15.134:0/116407466 wait complete. 2024-09-16T15:14:11.446 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:14:11.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i', 'a', 'b'] 2024-09-16T15:14:11.446 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['d'] 2024-09-16T15:14:11.446 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-16T15:14:11.446 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:14:11.446 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-16T15:14:11.446 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.455+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1116639168}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=897816137}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3733205874}) 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3574183385}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3717375702}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2482848412}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2902503359}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2518324746}) 2024-09-16T15:14:11.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:14:11.456+0000 7f12e0f68640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=502109921}) 2024-09-16T15:14:11.461 INFO:tasks.ceph.mon.i:Stopped 2024-09-16T15:14:11.462 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-16T15:14:11.462 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:14:11.462 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:14:11.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:14:11.474 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:14:11.474 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-16T15:14:11.474 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:14:11.474 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:14:11.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:14:11.524 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:14:11.524 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-16T15:14:11.525 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-16T15:14:11.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2986: done (2 left) 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2987: done (1 left) 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2988: done (0 left) 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2989: copy_from oid 27 from oid 13 current snap is 311 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2990: write oid 6 current snap is 311 2024-09-16T15:14:26.194 INFO:tasks.rados.rados.0.smithi162.stdout:2990: seq_num 1037 ranges {628859=657221,1716180=641021,3144467=284392} 2024-09-16T15:14:26.200 INFO:tasks.rados.rados.0.smithi162.stdout:2990: writing smithi16234619-6 from 628859 to 1286080 tid 1 2024-09-16T15:14:26.206 INFO:tasks.rados.rados.0.smithi162.stdout:2990: writing smithi16234619-6 from 1716180 to 2357201 tid 2 2024-09-16T15:14:26.212 INFO:tasks.rados.rados.0.smithi162.stdout:2990: writing smithi16234619-6 from 3144467 to 3428859 tid 3 2024-09-16T15:14:26.212 INFO:tasks.rados.rados.0.smithi162.stdout:2991: write (excl) oid 24 current snap is 311 2024-09-16T15:14:26.212 INFO:tasks.rados.rados.0.smithi162.stdout:2991: seq_num 1038 ranges {645666=732470,2174662=501032,3288243=157423} 2024-09-16T15:14:26.219 INFO:tasks.rados.rados.0.smithi162.stdout:2991: writing smithi16234619-24 from 645666 to 1378136 tid 1 2024-09-16T15:14:26.226 INFO:tasks.rados.rados.0.smithi162.stdout:2991: writing smithi16234619-24 from 2174662 to 2675694 tid 2 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2991: writing smithi16234619-24 from 3288243 to 3445666 tid 3 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2992: rollback oid 41 current snap is 311 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 41 to 277 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2993: rollback oid 38 current snap is 311 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 284 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2994: read oid 25 snap -1 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2994: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:26.231 INFO:tasks.rados.rados.0.smithi162.stdout:2995: copy_from oid 2 from oid 11 current snap is 311 2024-09-16T15:14:26.232 INFO:tasks.rados.rados.0.smithi162.stdout:2996: copy_from oid 45 from oid 3 current snap is 311 2024-09-16T15:14:26.232 INFO:tasks.rados.rados.0.smithi162.stdout:2997: snap_remove snap 299 2024-09-16T15:14:26.525 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-16T15:14:26.526 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-16T15:14:26.526 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-16T15:14:26.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:14:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.629+0000 7fe9b6ab1640 10 monclient: get_monmap_and_config 2024-09-16T15:14:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.629+0000 7fe9b6ab1640 10 monclient: build_initial_monmap 2024-09-16T15:14:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.629+0000 7fe9b6ab1640 10 monclient: monmap: 2024-09-16T15:14:26.630 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:26.630302+0000 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:26.630302+0000 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:26.631 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:26.632 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:26.632 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:26.632 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:26.632 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 Processor -- start 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 -- start start 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient: init 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient: _add_conns ranks=[2,10,9,6,0,5,7,4,8,3,1] 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-b con 0x7fe9b0136080 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-f con 0x7fe9b00a5700 addr v2:172.21.15.162:3300/0 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-a con 0x7fe9b00a44c0 addr v2:172.21.15.134:3300/0 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:26.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b009bb00 con 0x7fe9b00a44c0 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b009b4b0 con 0x7fe9b00a5700 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009c6a0 con 0x7fe9b0136080 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b6ab1640 10 monclient(hunting): authenticate will time out at 1603.219659s 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b484d640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 msgr2=0x7fe9b0136460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b504e640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 msgr2=0x7fe9b00a4950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b504e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.630+0000 7fe9b484d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:47974/0 (socket says 172.21.15.134:47974) 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 learned_addr learned my addr 172.21.15.134:0/3310465972 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe9b00a5700 auth_method 0 2024-09-16T15:14:26.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:26.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.631+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:26.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.831+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 msgr2=0x7fe9b00a4950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:26.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.831+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 msgr2=0x7fe9b0136460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:26.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.831+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:26.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.831+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:26.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe9b00a5700 auth_method 0 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:26.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:26.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:27.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.232+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 msgr2=0x7fe9b00a4950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:27.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.232+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 msgr2=0x7fe9b0136460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:27.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.232+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:27.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.232+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:27.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe9b00a5700 auth_method 0 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:27.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:27.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:28.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.033+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 msgr2=0x7fe9b00a4950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:28.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.033+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 msgr2=0x7fe9b0136460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:28.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.033+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:28.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.033+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe9b00a5700 auth_method 0 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:28.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:28.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:29.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): tick 2024-09-16T15:14:29.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:14:29.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:14:29.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 msgr2=0x7fe9b0136460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b0136080 0x7fe9b0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 msgr2=0x7fe9b00a3f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.162:3300/0 conn(0x7fe9b00a5700 0x7fe9b00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 msgr2=0x7fe9b00a4950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> v2:172.21.15.134:3300/0 conn(0x7fe9b00a44c0 0x7fe9b00a4950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient: _add_conns ranks=[6,3,7,8,4,0,5,1] 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-d con 0x7fe9940037c0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-g con 0x7fe994006260 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:29.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-i con 0x7fe994008de0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009bb00 con 0x7fe994006260 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009b4b0 con 0x7fe9940037c0 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009c6a0 con 0x7fe994008de0 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9ae7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 msgr2=0x7fe99400b270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 10 monclient(hunting): get_auth_request con 0x7fe9940037c0 auth_method 0 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:29.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe994006260 auth_method 0 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:29.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:29.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:29.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.631+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 msgr2=0x7fe99400b270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 10 monclient(hunting): get_auth_request con 0x7fe9940037c0 auth_method 0 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:29.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe994006260 auth_method 0 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:29.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:29.832+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 msgr2=0x7fe99400b270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 10 monclient(hunting): get_auth_request con 0x7fe9940037c0 auth_method 0 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:30.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe994006260 auth_method 0 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:30.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:30.233+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:14:31.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 msgr2=0x7fe99400b270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:31.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 10 monclient(hunting): get_auth_request con 0x7fe9940037c0 auth_method 0 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe994006260 auth_method 0 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:31.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:31.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:14:31.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:14:31.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:31.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:31.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:31.034+0000 7fe9affff640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:14:31.599 INFO:tasks.rados.rados.0.smithi162.stdout:2992: finishing rollback tid 0 to smithi16234619-41 2024-09-16T15:14:31.599 INFO:tasks.rados.rados.0.smithi162.stdout:2992: finishing rollback tid 1 to smithi16234619-41 2024-09-16T15:14:31.599 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 826 (ObjNum 49 snap 0 seq_num 1838624023) dirty dne 2024-09-16T15:14:31.599 INFO:tasks.rados.rados.0.smithi162.stdout:2994: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2991: finishing write tid 1 to smithi16234619-24 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2991: finishing write tid 2 to smithi16234619-24 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2992: done (8 left) 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2994: done (7 left) 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2997: done (6 left) 2024-09-16T15:14:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:2998: snap_remove snap 288 2024-09-16T15:14:32.617 INFO:tasks.rados.rados.0.smithi162.stdout:2998: done (6 left) 2024-09-16T15:14:32.617 INFO:tasks.rados.rados.0.smithi162.stdout:2999: copy_from oid 44 from oid 14 current snap is 311 2024-09-16T15:14:32.617 INFO:tasks.rados.rados.0.smithi162.stdout:2991: finishing write tid 3 to smithi16234619-24 2024-09-16T15:14:32.617 INFO:tasks.rados.rados.0.smithi162.stdout:2991: finishing write tid 4 to smithi16234619-24 2024-09-16T15:14:32.617 INFO:tasks.rados.rados.0.smithi162.stdout:2991: finishing write tid 5 to smithi16234619-24 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2991: oid 24 updating version 0 to 1538 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2991: oid 24 version 1538 is already newer than 1537 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2991: oid 24 updating version 1538 to 1540 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2991: oid 24 version 1540 is already newer than 1539 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1540 (ObjNum 1038 snap 311 seq_num 1038) dirty exists 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2991: left oid 24 (ObjNum 1038 snap 311 seq_num 1038) 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2989: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2989: finishing copy_from to smithi16234619-27 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 1348 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2990: finishing write tid 1 to smithi16234619-6 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2990: finishing write tid 2 to smithi16234619-6 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2990: finishing write tid 3 to smithi16234619-6 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2990: finishing write tid 4 to smithi16234619-6 2024-09-16T15:14:32.618 INFO:tasks.rados.rados.0.smithi162.stdout:2990: finishing write tid 5 to smithi16234619-6 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2990: oid 6 updating version 0 to 1289 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2990: oid 6 updating version 1289 to 1291 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2990: oid 6 updating version 1291 to 1292 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2990: oid 6 version 1292 is already newer than 1290 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1292 (ObjNum 1037 snap 311 seq_num 1037) dirty exists 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2990: left oid 6 (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2993: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2993: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2993: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1295 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2995: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2996: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:2996: finishing copy_from to smithi16234619-45 2024-09-16T15:14:32.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 159 (ObjNum 993 snap 281 seq_num 993) dirty exists 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2995: finishing copy_from to smithi16234619-2 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1280 (ObjNum 1011 snap 293 seq_num 1011) dirty exists 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2989: done (6 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2990: done (5 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2991: done (4 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2993: done (3 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2995: done (2 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:2996: done (1 left) 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:3000: write (excl) oid 40 current snap is 311 2024-09-16T15:14:32.620 INFO:tasks.rados.rados.0.smithi162.stdout:3000: seq_num 1039 ranges {662473=407719,1833144=644690,3199313=263160} 2024-09-16T15:14:32.623 INFO:tasks.rados.rados.0.smithi162.stdout:3000: writing smithi16234619-40 from 662473 to 1070192 tid 1 2024-09-16T15:14:32.631 INFO:tasks.rados.rados.0.smithi162.stdout:3000: writing smithi16234619-40 from 1833144 to 2477834 tid 2 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): tick 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 msgr2=0x7fe99400b270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe994008de0 0x7fe99400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 msgr2=0x7fe994005c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe9940037c0 0x7fe994005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 msgr2=0x7fe9940086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994006260 0x7fe9940086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 reap_dead start 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient: _add_conns ranks=[5,0,4,8,1] 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 reap_dead start 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-h con 0x7fe994003c00 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe9940097a0 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 0x7fe994008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): picked mon.noname-c con 0x7fe99400b2e0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b0073250 con 0x7fe9940097a0 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009bb00 con 0x7fe99400b2e0 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:32.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b009b4b0 con 0x7fe994003c00 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9ae7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9affff640 1 -- 172.21.15.134:0/3310465972 >> v1:172.21.15.134:6789/0 conn(0x7fe9940097a0 legacy=0x7fe994009bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 0x7fe994008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 msgr2=0x7fe994009260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b484d640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 msgr2=0x7fe994009260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): get_auth_request con 0x7fe99400b2e0 auth_method 0 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b484d640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:32.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.631+0000 7fe9b504e640 10 monclient(hunting): handle_auth_done global_id 5755 payload 274 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 msgr2=0x7fe994009260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> v1:172.21.15.134:6789/0 conn(0x7fe9940097a0 legacy=0x7fe994009bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 10 monclient: _finish_hunting 0 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 1 monclient: found mon.noname-c 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:32.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.632+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99400f050 con 0x7fe99400b2e0 2024-09-16T15:14:32.639 INFO:tasks.rados.rados.0.smithi162.stdout:3000: writing smithi16234619-40 from 3199313 to 3462473 tid 3 2024-09-16T15:14:32.639 INFO:tasks.rados.rados.0.smithi162.stdout:3001: read oid 22 snap -1 2024-09-16T15:14:32.639 INFO:tasks.rados.rados.0.smithi162.stdout:3001: expect deleted 2024-09-16T15:14:32.639 INFO:tasks.rados.rados.0.smithi162.stdout:3002: delete oid 42 current snap is 311 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.634+0000 7fe9b504e640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 msgr2=0x7fe994008670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.634+0000 7fe9b504e640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 0x7fe994008670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe9b009aef0 tx=0x7fe9a802aa70 comp rx=0 tx=0).ready entity=mon.4 client_cookie=34a98f72fd6c098f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 1 -- 172.21.15.134:0/3310465972 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9a802ed70 con 0x7fe99400b2e0 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: dump: 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:32.642 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:32.643 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: _finish_auth 0 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9b6ab1640 5 monclient: authenticate success, global_id 5755 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 1 -- 172.21.15.134:0/3310465972 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a802c380 con 0x7fe99400b2e0 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9b6ab1640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9b6ab1640 4 set_mon_vals no callback set 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 1 -- 172.21.15.134:0/3310465972 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9a802c960 con 0x7fe99400b2e0 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.635+0000 7fe9af7fe640 10 monclient: dump: 2024-09-16T15:14:32.644 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:32.645 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 10 monclient: shutdown 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 msgr2=0x7fe994008670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 0x7fe994008670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe9b009aef0 tx=0x7fe9a802aa70 comp rx=0 tx=0).stop 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/3310465972 shutdown_connections 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe99400b2e0 0x7fe994008670 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/3310465972 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe994003c00 0x7fe994009260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/3310465972 >> 172.21.15.134:0/3310465972 conn(0x7fe9b00a99a0 msgr2=0x7fe9b00a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/3310465972 shutdown_connections 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.636+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/3310465972 wait complete. 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.637+0000 7fe9b6ab1640 10 monclient: build_initial_monmap 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.637+0000 7fe9b6ab1640 10 monclient: monmap: 2024-09-16T15:14:32.646 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:32.647 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.637+0000 7fe9b6ab1640 1 Processor -- start 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 -- start start 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient: init 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient: _add_conns ranks=[3,4,8,0,6,1,2,7,5] 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 0x7fe9b009f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-g con 0x7fe994003c00 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe9940097a0 0x7fe9b0134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-c con 0x7fe9940097a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): picked mon.noname-e con 0x7fe99400b2e0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00a8dc0 con 0x7fe994003c00 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b0122910 con 0x7fe9940097a0 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00a8a90 con 0x7fe99400b2e0 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.638+0000 7fe9b6ab1640 10 monclient(hunting): authenticate will time out at 1609.227964s 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52612/0 (socket says 172.21.15.134:52612) 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 -- 172.21.15.134:0/2361193914 learned_addr learned my addr 172.21.15.134:0/2361193914 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): get_auth_request con 0x7fe99400b2e0 auth_method 0 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:32.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9affff640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe9940097a0 0x7fe9b0134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9affff640 10 monclient(hunting): get_auth_request con 0x7fe9940097a0 auth_method 0 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 0x7fe9b009f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 10 monclient(hunting): get_auth_request con 0x7fe994003c00 auth_method 0 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:32.650 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient(hunting): handle_auth_done global_id 84482 payload 824 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 -- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe9940097a0 msgr2=0x7fe9b0134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe9940097a0 0x7fe9b0134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 -- 172.21.15.134:0/2361193914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 msgr2=0x7fe9b009f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 0x7fe9b009f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient: _finish_hunting 0 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 monclient: found mon.noname-e 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b504e640 1 -- 172.21.15.134:0/2361193914 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8007b50 con 0x7fe99400b2e0 2024-09-16T15:14:32.651 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.639+0000 7fe9b484d640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 0x7fe9b009f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:14:32.652 INFO:tasks.rados.rados.0.smithi162.stdout:3001: done (3 left) 2024-09-16T15:14:32.652 INFO:tasks.rados.rados.0.smithi162.stdout:3002: done (2 left) 2024-09-16T15:14:32.652 INFO:tasks.rados.rados.0.smithi162.stdout:3003: write (excl) oid 23 current snap is 311 2024-09-16T15:14:32.652 INFO:tasks.rados.rados.0.smithi162.stdout:3003: seq_num 1041 ranges {696087=558217,1833755=648359,3137806=358281} 2024-09-16T15:14:32.652 INFO:tasks.rados.rados.0.smithi162.stdout:3003: writing smithi16234619-23 from 696087 to 1254304 tid 1 2024-09-16T15:14:32.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.641+0000 7fe9b504e640 1 -- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 msgr2=0x7fe9b0135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:32.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.641+0000 7fe9b504e640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00a6d80 tx=0x7fe9a8006190 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9a802cbc0 con 0x7fe99400b2e0 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: dump: 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:32.655 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: _finish_auth 0 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 5 monclient: authenticate success, global_id 84482 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a802cd40 con 0x7fe99400b2e0 2024-09-16T15:14:32.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe9a8037d30 con 0x7fe99400b2e0 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9aeffd640 4 set_mon_vals no callback set 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9ad7fa640 10 monclient: dump: 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:32.657 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 10 monclient: _renew_subs 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b0137790 con 0x7fe99400b2e0 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 10 monclient: _renew_subs 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:32.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.642+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b0122630 con 0x7fe99400b2e0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9b6ab1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9b6ab1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9b6ab1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b01cb520 con 0x7fe99400b2e0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe9a8037450 con 0x7fe99400b2e0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9ad7fa640 1 --2- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 0x7fe99403cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.643+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(627..627 src has 1..627) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fe9a806b180 con 0x7fe99400b2e0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.644+0000 7fe9b484d640 1 --2- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 0x7fe99403cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.644+0000 7fe9b484d640 10 monclient: get_auth_request con 0x7fe99403ab30 auth_method 0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.646+0000 7fe9b484d640 1 --2- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 0x7fe99403cff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4003890 tx=0x7fe9a4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.648+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fe9b01cb520 con 0x7fe99400b2e0 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.648+0000 7fe9ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:32.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.648+0000 7fe9ad7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:32.660 INFO:tasks.rados.rados.0.smithi162.stdout:3003: writing smithi16234619-23 from 1833755 to 2482114 tid 2 2024-09-16T15:14:32.661 INFO:tasks.rados.rados.0.smithi162.stdout:3003: writing smithi16234619-23 from 3137806 to 3496087 tid 3 2024-09-16T15:14:32.661 INFO:tasks.rados.rados.0.smithi162.stdout:3004: write oid 13 current snap is 311 2024-09-16T15:14:32.661 INFO:tasks.rados.rados.0.smithi162.stdout:3004: seq_num 1042 ranges {712894=749819,2008590=792017,3281582=231312} 2024-09-16T15:14:32.669 INFO:tasks.rados.rados.0.smithi162.stdout:3004: writing smithi16234619-13 from 712894 to 1462713 tid 1 2024-09-16T15:14:32.677 INFO:tasks.rados.rados.0.smithi162.stdout:3004: writing smithi16234619-13 from 2008590 to 2800607 tid 2 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3004: writing smithi16234619-13 from 3281582 to 3512894 tid 3 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3005: delete oid 17 current snap is 311 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: finishing write tid 1 to smithi16234619-40 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: finishing write tid 2 to smithi16234619-40 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: finishing write tid 3 to smithi16234619-40 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: finishing write tid 4 to smithi16234619-40 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: finishing write tid 5 to smithi16234619-40 2024-09-16T15:14:32.681 INFO:tasks.rados.rados.0.smithi162.stdout:3000: oid 40 updating version 0 to 1435 2024-09-16T15:14:32.682 INFO:tasks.rados.rados.0.smithi162.stdout:3000: oid 40 updating version 1435 to 1437 2024-09-16T15:14:32.682 INFO:tasks.rados.rados.0.smithi162.stdout:3000: oid 40 version 1437 is already newer than 1436 2024-09-16T15:14:32.682 INFO:tasks.rados.rados.0.smithi162.stdout:3000: oid 40 updating version 1437 to 1438 2024-09-16T15:14:32.682 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1438 (ObjNum 1039 snap 311 seq_num 1039) dirty exists 2024-09-16T15:14:32.682 INFO:tasks.rados.rados.0.smithi162.stdout:3000: left oid 40 (ObjNum 1039 snap 311 seq_num 1039) 2024-09-16T15:14:32.685 INFO:tasks.rados.rados.0.smithi162.stdout:3000: done (4 left) 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3005: done (3 left) 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3006: copy_from oid 3 from oid 1 current snap is 311 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3007: read oid 21 snap -1 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3007: expect (ObjNum 985 snap 280 seq_num 985) 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3008: rollback oid 10 current snap is 311 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 287 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3009: rollback oid 33 current snap is 311 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 302 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3010: read oid 18 snap -1 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3010: expect (ObjNum 1023 snap 297 seq_num 1023) 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:3011: delete oid 9 current snap is 311 2024-09-16T15:14:32.686 INFO:tasks.rados.rados.0.smithi162.stdout:2999: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:14:32.688 INFO:tasks.rados.rados.0.smithi162.stdout:2999: finishing copy_from to smithi16234619-44 2024-09-16T15:14:32.688 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1352 (ObjNum 1027 snap 298 seq_num 1027) dirty exists 2024-09-16T15:14:32.694 INFO:tasks.rados.rados.0.smithi162.stdout:3007: expect (ObjNum 985 snap 280 seq_num 985) 2024-09-16T15:14:32.815 INFO:tasks.rados.rados.0.smithi162.stdout:3009: finishing rollback tid 0 to smithi16234619-33 2024-09-16T15:14:32.815 INFO:tasks.rados.rados.0.smithi162.stdout:3009: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:14:32.815 INFO:tasks.rados.rados.0.smithi162.stdout:3009: finishing rollback tid 2 to smithi16234619-33 2024-09-16T15:14:32.816 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 835 (ObjNum 1003 snap 284 seq_num 1003) dirty exists 2024-09-16T15:14:32.816 INFO:tasks.rados.rados.0.smithi162.stdout:3010: expect (ObjNum 1023 snap 297 seq_num 1023) 2024-09-16T15:14:32.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.885+0000 7fe9b6ab1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:14:32.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.885+0000 7fe9b6ab1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:32.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.885+0000 7fe9b6ab1640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:14:32.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.885+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe9b01cb520 con 0x7fe99400b2e0 2024-09-16T15:14:32.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.886+0000 7fe9ad7fa640 1 -- 172.21.15.134:0/2361193914 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3178 (secure 0 0 0) 0x7fe9b01cb520 con 0x7fe99400b2e0 2024-09-16T15:14:32.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.886+0000 7fe9ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:14:32.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.886+0000 7fe9ad7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:32.888 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":114,"quorum":[1,3,4,5,6,8],"quorum_names":["f","g","c","h","d","e"],"quorum_leader_name":"f","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:14:32.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 msgr2=0x7fe99403cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 0x7fe99403cff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4003890 tx=0x7fe9a4008040 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 10 monclient: shutdown 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 msgr2=0x7fe9b0135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00a6d80 tx=0x7fe9a8006190 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 shutdown_connections 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe99400b2e0 0x7fe9b0135490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe9940097a0 0x7fe9b0134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> v2:172.21.15.162:6800/980547263 conn(0x7fe99403ab30 0x7fe99403cff0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 --2- 172.21.15.134:0/2361193914 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe994003c00 0x7fe9b009f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 >> 172.21.15.134:0/2361193914 conn(0x7fe9b00a99a0 msgr2=0x7fe9b00a81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 shutdown_connections 2024-09-16T15:14:32.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:32.888+0000 7fe9b6ab1640 1 -- 172.21.15.134:0/2361193914 wait complete. 2024-09-16T15:14:32.900 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-16T15:14:32.900 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-16T15:14:32.901 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3008: finishing rollback tid 0 to smithi16234619-10 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3008: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3008: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1568 (ObjNum 904 snap 249 seq_num 904) dirty exists 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3003: finishing write tid 1 to smithi16234619-23 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3003: finishing write tid 2 to smithi16234619-23 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3003: finishing write tid 3 to smithi16234619-23 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3003: finishing write tid 4 to smithi16234619-23 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:2999: done (8 left) 2024-09-16T15:14:32.948 INFO:tasks.rados.rados.0.smithi162.stdout:3007: done (7 left) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3008: done (6 left) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3009: done (5 left) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3010: done (4 left) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3011: done (3 left) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3012: read oid 10 snap -1 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3012: expect (ObjNum 904 snap 249 seq_num 904) 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3013: copy_from oid 26 from oid 20 current snap is 311 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: finishing write tid 5 to smithi16234619-23 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: oid 23 updating version 0 to 607 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: oid 23 version 607 is already newer than 606 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: oid 23 version 607 is already newer than 604 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: oid 23 version 607 is already newer than 605 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 607 (ObjNum 1041 snap 311 seq_num 1041) dirty exists 2024-09-16T15:14:32.949 INFO:tasks.rados.rados.0.smithi162.stdout:3003: left oid 23 (ObjNum 1041 snap 311 seq_num 1041) 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: finishing write tid 1 to smithi16234619-13 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3006: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: finishing write tid 2 to smithi16234619-13 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: finishing write tid 3 to smithi16234619-13 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3003: done (4 left) 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3014: delete oid 21 current snap is 311 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: finishing write tid 4 to smithi16234619-13 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: finishing write tid 5 to smithi16234619-13 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: oid 13 updating version 0 to 1508 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: oid 13 version 1508 is already newer than 1507 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: oid 13 version 1508 is already newer than 1506 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: oid 13 updating version 1508 to 1509 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1509 (ObjNum 1042 snap 311 seq_num 1042) dirty exists 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3004: left oid 13 (ObjNum 1042 snap 311 seq_num 1042) 2024-09-16T15:14:32.950 INFO:tasks.rados.rados.0.smithi162.stdout:3006: finishing copy_from to smithi16234619-3 2024-09-16T15:14:32.951 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1296 (ObjNum 1028 snap 299 seq_num 1028) dirty exists 2024-09-16T15:14:32.954 INFO:tasks.rados.rados.0.smithi162.stdout:3012: expect (ObjNum 904 snap 249 seq_num 904) 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.020+0000 7fc8b615e640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.020+0000 7fc8b615e640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.020+0000 7fc8b615e640 10 monclient: monmap: 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.021718+0000 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.021718+0000 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.022 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.023 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.021+0000 7fc8b615e640 1 Processor -- start 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.021+0000 7fc8b615e640 1 -- start start 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.021+0000 7fc8b615e640 10 monclient: init 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient: _add_conns ranks=[10,6,5,8,2,9,3,0,4,1,7] 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8b0136e40 0x7fc8b009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-f con 0x7fc8b0136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-d con 0x7fc8b0137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b0135dc0 0x7fc8b0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-h con 0x7fc8b0135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b009abc0 con 0x7fc8b0136e40 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b009ba50 con 0x7fc8b0135dc0 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00a53c0 con 0x7fc8b0137220 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8b615e640 10 monclient(hunting): authenticate will time out at 1609.611223s 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:45904/0 (socket says 172.21.15.134:45904) 2024-09-16T15:14:33.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8b0136e40 0x7fc8b009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1670578561 learned_addr learned my addr 172.21.15.134:0/1670578561 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request con 0x7fc8b0137220 auth_method 0 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b0135dc0 0x7fc8b0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request con 0x7fc8b0136e40 auth_method 0 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): get_auth_request con 0x7fc8b0135dc0 auth_method 0 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_done global_id 94227 payload 274 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1670578561 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b0135dc0 msgr2=0x7fc8b0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b0135dc0 0x7fc8b0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1670578561 >> v2:172.21.15.162:3300/0 conn(0x7fc8b0136e40 msgr2=0x7fc8b009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1670578561 >> v2:172.21.15.162:3300/0 conn(0x7fc8b0136e40 0x7fc8b009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 monclient: found mon.noname-d 2024-09-16T15:14:33.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:33.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.022+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1670578561 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b0073250 con 0x7fc8b0137220 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.025+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1670578561 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 msgr2=0x7fc8b0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.025+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40098a0 tx=0x7fc8a402a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=c49a37eebf369098 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 1 -- 172.21.15.134:0/1670578561 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a402adf0 con 0x7fc8b0137220 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:33.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: dump: 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.037 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 1 -- 172.21.15.134:0/1670578561 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a40307b0 con 0x7fc8b0137220 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8b615e640 5 monclient: authenticate success, global_id 94227 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8b615e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8b615e640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 1 -- 172.21.15.134:0/1670578561 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a4030d90 con 0x7fc8b0137220 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8b615e640 4 set_mon_vals no callback set 2024-09-16T15:14:33.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.026+0000 7fc8ae7fc640 10 monclient: dump: 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.039 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 10 monclient: shutdown 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 -- 172.21.15.134:0/1670578561 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 msgr2=0x7fc8b0137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40098a0 tx=0x7fc8a402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 -- 172.21.15.134:0/1670578561 shutdown_connections 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b0135dc0 0x7fc8b0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1670578561 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc8b0137220 0x7fc8b0137600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1670578561 >> v2:172.21.15.162:3300/0 conn(0x7fc8b0136e40 0x7fc8b009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 -- 172.21.15.134:0/1670578561 >> 172.21.15.134:0/1670578561 conn(0x7fc8b00a6110 msgr2=0x7fc8b00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 -- 172.21.15.134:0/1670578561 shutdown_connections 2024-09-16T15:14:33.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.027+0000 7fc8b615e640 1 -- 172.21.15.134:0/1670578561 wait complete. 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.028+0000 7fc8b615e640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.028+0000 7fc8b615e640 10 monclient: monmap: 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.041 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.028+0000 7fc8b615e640 1 Processor -- start 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.028+0000 7fc8b615e640 1 -- start start 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.028+0000 7fc8b615e640 10 monclient: init 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient: _add_conns ranks=[4,2,3,7,6,0,8,1,5] 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 0x7fc8b01cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-c con 0x7fc8b0135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0136e40 0x7fc8b0134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-b con 0x7fc8b0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8b0137220 0x7fc8b0135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): picked mon.noname-g con 0x7fc8b0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b01cb760 con 0x7fc8b0136e40 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0123cc0 con 0x7fc8b0137220 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0123570 con 0x7fc8b0135dc0 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8b615e640 10 monclient(hunting): authenticate will time out at 1609.618236s 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8aeffd640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0136e40 msgr2=0x7fc8b0134f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8aeffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0136e40 0x7fc8b0134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 0x7fc8b01cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 0x7fc8b01cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41020/0 (socket says 172.21.15.134:41020) 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 -- 172.21.15.134:0/1175280683 learned_addr learned my addr 172.21.15.134:0/1175280683 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request con 0x7fc8b0135dc0 auth_method 0 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8b0137220 0x7fc8b0135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): get_auth_request con 0x7fc8b0137220 auth_method 0 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_done global_id 5773 payload 824 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8b0137220 msgr2=0x7fc8b0135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8b0137220 0x7fc8b0135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0136e40 msgr2=0x7fc8b0134f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0136e40 0x7fc8b0134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.029+0000 7fc8af7fe640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40094d0 con 0x7fc8b0135dc0 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.031+0000 7fc8af7fe640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 msgr2=0x7fc8b01cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.031+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 0x7fc8b01cb220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9e0 tx=0x7fc8a000eea0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a0009670 con 0x7fc8b0135dc0 2024-09-16T15:14:33.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: dump: 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a000cbe0 con 0x7fc8b0135dc0 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 5 monclient: authenticate success, global_id 5773 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 10 monclient: _renew_subs 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b00a8a30 con 0x7fc8b0135dc0 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a0013d90 con 0x7fc8b0135dc0 2024-09-16T15:14:33.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc897fff640 10 monclient: dump: 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8adffb640 4 set_mon_vals no callback set 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 10 monclient: _renew_subs 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.032+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01363b0 con 0x7fc8b0135dc0 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8b615e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc8a000cd50 con 0x7fc8b0135dc0 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 0x7fc8b01221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0071b20 con 0x7fc8b01255c0 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 0x7fc8b01221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc897fff640 1 --2- 172.21.15.134:0/1175280683 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient: get_auth_request con 0x7fc8b01255c0 auth_method 0 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fc8a004a380 con 0x7fc8b0135dc0 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_done global_id 5782 payload 824 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8affff640 1 --2- 172.21.15.134:0/1175280683 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8b0061ae0 con 0x7fc8b01255c0 2024-09-16T15:14:33.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.033+0000 7fc8affff640 10 monclient: get_auth_request con 0x7fc89c0388b0 auth_method 0 2024-09-16T15:14:33.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 msgr2=0x7fc8b01221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 0x7fc8b01221f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4034000 tx=0x7fc8a40046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a40044a0 con 0x7fc8b01255c0 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 10 monclient: dump: 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.053 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc8affff640 1 --2- 172.21.15.134:0/1175280683 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0073780 tx=0x7fc8a800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc8a4030d40 con 0x7fc8b01255c0 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 msgr2=0x7fc8b01221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 0x7fc8b01221f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4034000 tx=0x7fc8a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 msgr2=0x7fc8b01221f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.036+0000 7fc897fff640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b01255c0 0x7fc8b01221f0 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4034000 tx=0x7fc8a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8b615e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 0x7fc8b002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8b615e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b007a680 con 0x7fc8b002b260 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 0x7fc8b002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient: get_auth_request con 0x7fc8b002b260 auth_method 0 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.042+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.043+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_done global_id 5791 payload 824 2024-09-16T15:14:33.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.043+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1175280683 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc8b009b490 con 0x7fc8b002b260 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc8aeffd640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 msgr2=0x7fc8b002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 0x7fc8b002b670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4030fd0 tx=0x7fc8a4009240 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a404a070 con 0x7fc8b002b260 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 10 monclient: dump: 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.056 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9329 (secure 0 0 0) 0x7fc8a4003e00 con 0x7fc8b002b260 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 msgr2=0x7fc8b002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 0x7fc8b002b670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4030fd0 tx=0x7fc8a4009240 comp rx=0 tx=0).stop 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 msgr2=0x7fc8b002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.045+0000 7fc897fff640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b002b260 0x7fc8b002b670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.058 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.059 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.060 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.061 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.062 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.063 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.064 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.070 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.071 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.072 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.073 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.074 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.075 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.076 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.077 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.078 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.079 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.080 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.081 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.081 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 msgr2=0x7fc89c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1175280683 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0073780 tx=0x7fc8a800b040 comp rx=0 tx=0).stop 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 10 monclient: shutdown 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 msgr2=0x7fc8b01cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 --2- 172.21.15.134:0/1175280683 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b0135dc0 0x7fc8b01cb220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9e0 tx=0x7fc8a000eea0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8affff640 1 -- 172.21.15.134:0/1175280683 reap_dead start 2024-09-16T15:14:33.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8affff640 1 -- 172.21.15.134:0/1175280683 reap_dead start 2024-09-16T15:14:33.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 shutdown_connections 2024-09-16T15:14:33.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 >> 172.21.15.134:0/1175280683 conn(0x7fc8b00a6110 msgr2=0x7fc8b009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 shutdown_connections 2024-09-16T15:14:33.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.046+0000 7fc8b615e640 1 -- 172.21.15.134:0/1175280683 wait complete. 2024-09-16T15:14:33.083 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.083 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-16T15:14:33.083 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:14:33.135 INFO:tasks.rados.rados.0.smithi162.stdout:3013: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:14:33.135 INFO:tasks.rados.rados.0.smithi162.stdout:3013: finishing copy_from to smithi16234619-26 2024-09-16T15:14:33.135 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 832 (ObjNum 812 snap 219 seq_num 812) dirty exists 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3004: done (4 left) 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3006: done (3 left) 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3012: done (2 left) 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3013: done (1 left) 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3014: done (0 left) 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3015: copy_from oid 22 from oid 11 current snap is 311 2024-09-16T15:14:33.136 INFO:tasks.rados.rados.0.smithi162.stdout:3016: delete oid 37 current snap is 311 2024-09-16T15:14:33.139 INFO:tasks.rados.rados.0.smithi162.stdout:3016: done (1 left) 2024-09-16T15:14:33.139 INFO:tasks.rados.rados.0.smithi162.stdout:3017: write oid 33 current snap is 311 2024-09-16T15:14:33.139 INFO:tasks.rados.rados.0.smithi162.stdout:3017: seq_num 1047 ranges {796929=726064,2184647=659366,3302344=294585} 2024-09-16T15:14:33.147 INFO:tasks.rados.rados.0.smithi162.stdout:3017: writing smithi16234619-33 from 796929 to 1522993 tid 1 2024-09-16T15:14:33.155 INFO:tasks.rados.rados.0.smithi162.stdout:3017: writing smithi16234619-33 from 2184647 to 2844013 tid 2 2024-09-16T15:14:33.160 INFO:tasks.rados.rados.0.smithi162.stdout:3017: writing smithi16234619-33 from 3302344 to 3596929 tid 3 2024-09-16T15:14:33.160 INFO:tasks.rados.rados.0.smithi162.stdout:3018: delete oid 7 current snap is 311 2024-09-16T15:14:33.163 INFO:tasks.rados.rados.0.smithi162.stdout:3018: done (2 left) 2024-09-16T15:14:33.163 INFO:tasks.rados.rados.0.smithi162.stdout:3019: rollback oid 18 current snap is 311 2024-09-16T15:14:33.163 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 285 2024-09-16T15:14:33.164 INFO:tasks.rados.rados.0.smithi162.stdout:3020: snap_remove snap 306 2024-09-16T15:14:33.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.186+0000 7efd3b2e9640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.186+0000 7efd3b2e9640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.186+0000 7efd3b2e9640 10 monclient: monmap: 2024-09-16T15:14:33.187 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.187230+0000 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.187230+0000 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.188 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.189 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.189 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.189 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.189 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.186+0000 7efd3b2e9640 1 Processor -- start 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 -- start start 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient: init 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient: _add_conns ranks=[4,5,3,0,6,2,1,7,9,10,8] 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 0x7efd340a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-c con 0x7efd340a31d0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 0x7efd340a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-h con 0x7efd340a1b00 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 0x7efd3413a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-g con 0x7efd341383a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3406ff80 con 0x7efd341383a0 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd340a35b0 con 0x7efd340a31d0 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd3406f930 con 0x7efd340a1b00 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd3b2e9640 10 monclient(hunting): authenticate will time out at 1609.776705s 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 0x7efd340a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 0x7efd340a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41052/0 (socket says 172.21.15.134:41052) 2024-09-16T15:14:33.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 0x7efd340a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 1 -- 172.21.15.134:0/1342917991 learned_addr learned my addr 172.21.15.134:0/1342917991 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 10 monclient(hunting): get_auth_request con 0x7efd340a31d0 auth_method 0 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39085640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39886640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 0x7efd3413a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): get_auth_request con 0x7efd340a1b00 auth_method 0 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39886640 10 monclient(hunting): get_auth_request con 0x7efd341383a0 auth_method 0 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39886640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39886640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd39886640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.187+0000 7efd38884640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 10 monclient(hunting): handle_auth_done global_id 45050 payload 274 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 -- 172.21.15.134:0/1342917991 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 msgr2=0x7efd340a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 0x7efd340a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 -- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 msgr2=0x7efd3413a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 0x7efd3413a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 monclient: found mon.noname-h 2024-09-16T15:14:33.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:14:33.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd38884640 1 -- 172.21.15.134:0/1342917991 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34073250 con 0x7efd340a1b00 2024-09-16T15:14:33.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.188+0000 7efd39886640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 0x7efd3413a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.190+0000 7efd38884640 1 -- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 msgr2=0x7efd340a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.190+0000 7efd38884640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 0x7efd340a1f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2400ba70 tx=0x7efd2402c660 comp rx=0 tx=0).ready entity=mon.5 client_cookie=53c889ffd69a546 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 1 -- 172.21.15.134:0/1342917991 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd24031b90 con 0x7efd340a1b00 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: dump: 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.202 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.203 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 1 -- 172.21.15.134:0/1342917991 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2402f050 con 0x7efd340a1b00 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 5 monclient: authenticate success, global_id 45050 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 1 -- 172.21.15.134:0/1342917991 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd24031d10 con 0x7efd340a1b00 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 4 set_mon_vals no callback set 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:33.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd2bfff640 10 monclient: dump: 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.205 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 10 monclient: shutdown 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 -- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 msgr2=0x7efd340a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 0x7efd340a1f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2400ba70 tx=0x7efd2402c660 comp rx=0 tx=0).stop 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 -- 172.21.15.134:0/1342917991 shutdown_connections 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd341383a0 0x7efd3413a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7efd340a1b00 0x7efd340a1f40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/1342917991 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd340a31d0 0x7efd340a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.191+0000 7efd3b2e9640 1 -- 172.21.15.134:0/1342917991 >> 172.21.15.134:0/1342917991 conn(0x7efd340a26b0 msgr2=0x7efd340a2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 1 -- 172.21.15.134:0/1342917991 shutdown_connections 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 1 -- 172.21.15.134:0/1342917991 wait complete. 2024-09-16T15:14:33.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 10 monclient: monmap: 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.207 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 1 Processor -- start 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 1 -- start start 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.192+0000 7efd3b2e9640 10 monclient: init 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient: _add_conns ranks=[6,2,4,7,3,1,0,5,8] 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd340a1b00 0x7efd3409dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-d con 0x7efd340a1b00 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd340a31d0 0x7efd3409e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-b con 0x7efd340a31d0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 0x7efd3409e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): picked mon.noname-c con 0x7efd341383a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd340a5880 con 0x7efd340a31d0 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd3411c200 con 0x7efd341383a0 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd340a5300 con 0x7efd340a1b00 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd3b2e9640 10 monclient(hunting): authenticate will time out at 1609.782435s 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd38884640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd340a31d0 msgr2=0x7efd3409e130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd38884640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd340a31d0 0x7efd3409e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd340a1b00 0x7efd3409dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 0x7efd3409e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 0x7efd3409e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41058/0 (socket says 172.21.15.134:41058) 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd340a1b00 0x7efd3409dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:45912/0 (socket says 172.21.15.134:45912) 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 learned_addr learned my addr 172.21.15.134:0/3109969815 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): get_auth_request con 0x7efd341383a0 auth_method 0 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): get_auth_request con 0x7efd340a1b00 auth_method 0 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient(hunting): handle_auth_done global_id 5809 payload 824 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd340a1b00 msgr2=0x7efd3409dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd340a1b00 0x7efd3409dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd340a31d0 msgr2=0x7efd3409e130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efd340a31d0 0x7efd3409e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.193+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009a70 con 0x7efd341383a0 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 msgr2=0x7efd3409e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd39886640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 0x7efd3409e670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00efb0 tx=0x7efd2c00c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd2c0148f0 con 0x7efd341383a0 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: dump: 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.212 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 5 monclient: authenticate success, global_id 5809 2024-09-16T15:14:33.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c00f040 con 0x7efd341383a0 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 10 monclient: _renew_subs 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2400b6a0 con 0x7efd341383a0 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd2c0133e0 con 0x7efd341383a0 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd29ffb640 10 monclient: dump: 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.214 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd2b7fe640 4 set_mon_vals no callback set 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 10 monclient: _renew_subs 2024-09-16T15:14:33.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.196+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3411cec0 con 0x7efd341383a0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7efd2c014480 con 0x7efd341383a0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd3b2e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 0x7efd34136950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd34067410 con 0x7efd3409ebb0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 0x7efd34136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient: get_auth_request con 0x7efd3409ebb0 auth_method 0 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd29ffb640 1 --2- 172.21.15.134:0/3109969815 >> v2:172.21.15.162:6800/980547263 conn(0x7efd18038880 0x7efd1803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (secure 0 0 0) 0x7efd2c04af30 con 0x7efd341383a0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 10 monclient(hunting): handle_auth_done global_id 94245 payload 824 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd39085640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd34061ae0 con 0x7efd3409ebb0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd38884640 1 --2- 172.21.15.134:0/3109969815 >> v2:172.21.15.162:6800/980547263 conn(0x7efd18038880 0x7efd1803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.197+0000 7efd38884640 10 monclient: get_auth_request con 0x7efd18038880 auth_method 0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.199+0000 7efd39085640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 msgr2=0x7efd34136950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.199+0000 7efd39085640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 0x7efd34136950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd30011db0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd300044a0 con 0x7efd3409ebb0 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 10 monclient: dump: 2024-09-16T15:14:33.217 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd38884640 1 --2- 172.21.15.134:0/3109969815 >> v2:172.21.15.162:6800/980547263 conn(0x7efd18038880 0x7efd1803ad40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd24033000 tx=0x7efd24031000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efd3000fe50 con 0x7efd3409ebb0 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 msgr2=0x7efd34136950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 0x7efd34136950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd30011db0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 msgr2=0x7efd34136950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.200+0000 7efd29ffb640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd3409ebb0 0x7efd34136950 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd30011db0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd3b2e9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 0x7efd341372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd3b2e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7efd3407a680 con 0x7efd34136e90 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 0x7efd341372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient: get_auth_request con 0x7efd34136e90 auth_method 0 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 10 monclient(hunting): handle_auth_done global_id 94254 payload 824 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.206+0000 7efd39085640 1 -- 172.21.15.134:0/3109969815 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efd3406e4f0 con 0x7efd34136e90 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd39085640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 msgr2=0x7efd341372a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd39085640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 0x7efd341372a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd300046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd30003e10 con 0x7efd34136e90 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 10 monclient: dump: 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.223 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9329 (secure 0 0 0) 0x7efd300273a0 con 0x7efd34136e90 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 msgr2=0x7efd341372a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 0x7efd341372a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd300046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 msgr2=0x7efd341372a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.209+0000 7efd29ffb640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7efd34136e90 0x7efd341372a0 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd300118f0 tx=0x7efd300046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.225 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.226 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.227 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.228 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.229 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.231 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.232 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.235 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.237 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.239 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.240 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.241 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.243 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.244 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.245 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.246 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 >> v2:172.21.15.162:6800/980547263 conn(0x7efd18038880 msgr2=0x7efd1803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/3109969815 >> v2:172.21.15.162:6800/980547263 conn(0x7efd18038880 0x7efd1803ad40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd24033000 tx=0x7efd24031000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 10 monclient: shutdown 2024-09-16T15:14:33.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 msgr2=0x7efd3409e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 --2- 172.21.15.134:0/3109969815 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7efd341383a0 0x7efd3409e670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00efb0 tx=0x7efd2c00c490 comp rx=0 tx=0).stop 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 reap_dead start 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd39886640 1 -- 172.21.15.134:0/3109969815 reap_dead start 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 shutdown_connections 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 >> 172.21.15.134:0/3109969815 conn(0x7efd340a26b0 msgr2=0x7efd340a42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 shutdown_connections 2024-09-16T15:14:33.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.210+0000 7efd3b2e9640 1 -- 172.21.15.134:0/3109969815 wait complete. 2024-09-16T15:14:33.248 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.248 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-16T15:14:33.249 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:14:33.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.345+0000 7f308e77d640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.345+0000 7f308e77d640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.345+0000 7f308e77d640 10 monclient: monmap: 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.346369+0000 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.346369+0000 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.348 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.348 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.348 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.348 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.348 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 Processor -- start 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 -- start start 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient: init 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient: _add_conns ranks=[8,7,2,5,0,9,6,10,1,4,3] 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-e con 0x7f3088136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:33.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 0x7f3088137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-i con 0x7f3088137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 0x7f3088136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-b con 0x7f3088135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308809abc0 con 0x7f3088135dc0 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f308809ba50 con 0x7f3088137220 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f30880a53c0 con 0x7f3088136e40 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308e77d640 10 monclient(hunting): authenticate will time out at 1609.935763s 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308cd1a640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 msgr2=0x7f3088136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f308cd1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 0x7f3088136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f30877fe640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 msgr2=0x7f3088137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52616/0 (socket says 172.21.15.134:52616) 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 1 -- 172.21.15.134:0/3918405392 learned_addr learned my addr 172.21.15.134:0/3918405392 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): get_auth_request con 0x7f3088136e40 auth_method 0 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f30877fe640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 0x7f3088137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.346+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 10 monclient(hunting): handle_auth_done global_id 84491 payload 274 2024-09-16T15:14:33.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 -- 172.21.15.134:0/3918405392 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 msgr2=0x7f3088137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 0x7f3088137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 -- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 msgr2=0x7f3088136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 0x7f3088136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 monclient: found mon.noname-e 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:33.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.347+0000 7f3087fff640 1 -- 172.21.15.134:0/3918405392 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088073250 con 0x7f3088136e40 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.349+0000 7f3087fff640 1 -- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 msgr2=0x7f308809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.349+0000 7f3087fff640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f30780098a0 tx=0x7f307802a0b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7170b5d977a499fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 1 -- 172.21.15.134:0/3918405392 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30780307b0 con 0x7f3088136e40 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: dump: 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.359 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.360 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 1 -- 172.21.15.134:0/3918405392 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078030930 con 0x7f3088136e40 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 1 -- 172.21.15.134:0/3918405392 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30780313a0 con 0x7f3088136e40 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 5 monclient: authenticate success, global_id 84491 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 4 set_mon_vals no callback set 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:33.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f3086ffd640 10 monclient: dump: 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.362 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 10 monclient: shutdown 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 -- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 msgr2=0x7f308809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f30780098a0 tx=0x7f307802a0b0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 -- 172.21.15.134:0/3918405392 shutdown_connections 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3088135dc0 0x7f3088136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088137220 0x7f3088137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 --2- 172.21.15.134:0/3918405392 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088136e40 0x7f308809c150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 -- 172.21.15.134:0/3918405392 >> 172.21.15.134:0/3918405392 conn(0x7f30880a6110 msgr2=0x7f30880a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 -- 172.21.15.134:0/3918405392 shutdown_connections 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.350+0000 7f308e77d640 1 -- 172.21.15.134:0/3918405392 wait complete. 2024-09-16T15:14:33.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.351+0000 7f308e77d640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.351+0000 7f308e77d640 10 monclient: monmap: 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.364 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.351+0000 7f308e77d640 1 Processor -- start 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.351+0000 7f308e77d640 1 -- start start 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.351+0000 7f308e77d640 10 monclient: init 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient: _add_conns ranks=[7,4,8,0,6,3,2,5,1] 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088135dc0 0x7f30881cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-i con 0x7f3088135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-c con 0x7f3088136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 0x7f3088135520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): picked mon.noname-e con 0x7f3088137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3088123710 con 0x7f3088136e40 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30881cb7f0 con 0x7f3088135dc0 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f30880a8a10 con 0x7f3088137220 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308e77d640 10 monclient(hunting): authenticate will time out at 1609.941253s 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 0x7f3088135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f3087fff640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088135dc0 msgr2=0x7f30881cb2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f3087fff640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088135dc0 0x7f30881cb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41070/0 (socket says 172.21.15.134:41070) 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 0x7f3088135520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52628/0 (socket says 172.21.15.134:52628) 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 -- 172.21.15.134:0/336859133 learned_addr learned my addr 172.21.15.134:0/336859133 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): get_auth_request con 0x7f3088136e40 auth_method 0 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): get_auth_request con 0x7f3088137220 auth_method 0 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient(hunting): handle_auth_done global_id 5818 payload 824 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 msgr2=0x7f3088135520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 0x7f3088135520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088135dc0 msgr2=0x7f30881cb2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3088135dc0 0x7f30881cb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f30877fe640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780094d0 con 0x7f3088136e40 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.352+0000 7f308cd1a640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088137220 0x7f3088135520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:14:33.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.354+0000 7f30877fe640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 msgr2=0x7f3088134fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.354+0000 7f30877fe640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f307c00b580 tx=0x7f307c00ba40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.354+0000 7f30877fe640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f307c00b580 tx=0x7f307c00ba40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f307c003dd0 mon_map magic: 0 v1 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f307c003dd0 con 0x7f3088136e40 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: dump: 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.369 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 5 monclient: authenticate success, global_id 5818 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c0054e0 con 0x7f3088136e40 2024-09-16T15:14:33.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 10 monclient: _renew_subs 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3088136910 con 0x7f3088136e40 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 10 monclient: _renew_subs 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f30867fc640 4 set_mon_vals no callback set 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881256f0 con 0x7f3088136e40 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f307c005650 con 0x7f3088136e40 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f3084ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.355+0000 7f308e77d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3084ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3084ff9640 10 monclient: dump: 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.371 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.372 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f307c005070 con 0x7f3088136e40 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f308e77d640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 0x7f308802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3088071b20 con 0x7f30881221a0 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 0x7f308802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient: get_auth_request con 0x7f30881221a0 auth_method 0 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3084ff9640 1 --2- 172.21.15.134:0/336859133 >> v2:172.21.15.162:6800/980547263 conn(0x7f3074038880 0x7f307403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 10 monclient(hunting): handle_auth_done global_id 84509 payload 824 2024-09-16T15:14:33.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f3087fff640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3088061ae0 con 0x7f30881221a0 2024-09-16T15:14:33.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f308cd1a640 1 --2- 172.21.15.134:0/336859133 >> v2:172.21.15.162:6800/980547263 conn(0x7f3074038880 0x7f307403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.356+0000 7f308cd1a640 10 monclient: get_auth_request con 0x7f3074038880 auth_method 0 2024-09-16T15:14:33.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3087fff640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 msgr2=0x7f308802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3087fff640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 0x7f308802b260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f30780046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f308cd1a640 1 --2- 172.21.15.134:0/336859133 >> v2:172.21.15.162:6800/980547263 conn(0x7f3074038880 0x7f307403ad40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3080003cb0 tx=0x7f308000d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30780313a0 con 0x7f30881221a0 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3084ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3084ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.358+0000 7f3084ff9640 10 monclient: dump: 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.377 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3078004250 con 0x7f30881221a0 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 msgr2=0x7f308802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 0x7f308802b260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f30780046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 msgr2=0x7f308802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.359+0000 7f3084ff9640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30881221a0 0x7f308802b260 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f30780046c0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f308e77d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f308e77d640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 0x7f308802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f308e77d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f308807a680 con 0x7f3088122580 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 0x7f308802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient: get_auth_request con 0x7f3088122580 auth_method 0 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 10 monclient(hunting): handle_auth_done global_id 84518 payload 824 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.365+0000 7f3087fff640 1 -- 172.21.15.134:0/336859133 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f308809b490 con 0x7f3088122580 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3087fff640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 msgr2=0x7f308802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3087fff640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 0x7f308802b7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30780097c0 tx=0x7f307803b000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30780044a0 con 0x7f3088122580 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3084ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3084ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.367+0000 7f3084ff9640 10 monclient: dump: 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.380 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.381 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9329 (secure 0 0 0) 0x7f3078048070 con 0x7f3088122580 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 msgr2=0x7f308802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 0x7f308802b7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30780097c0 tx=0x7f307803b000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 msgr2=0x7f308802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3088122580 0x7f308802b7a0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30780097c0 tx=0x7f307803b000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.382 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.383 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.384 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.385 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.386 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.387 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.389 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.390 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.391 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.394 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.396 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.399 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.400 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.401 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.402 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.403 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.404 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 >> v2:172.21.15.162:6800/980547263 conn(0x7f3074038880 msgr2=0x7f307403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 --2- 172.21.15.134:0/336859133 >> v2:172.21.15.162:6800/980547263 conn(0x7f3074038880 0x7f307403ad40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3080003cb0 tx=0x7f308000d040 comp rx=0 tx=0).stop 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 10 monclient: shutdown 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 msgr2=0x7f3088134fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 --2- 172.21.15.134:0/336859133 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3088136e40 0x7f3088134fe0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f307c00b580 tx=0x7f307c00ba40 comp rx=0 tx=0).stop 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308cd1a640 1 -- 172.21.15.134:0/336859133 reap_dead start 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f3084ff9640 1 -- 172.21.15.134:0/336859133 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (unknown 0 0 0) 0x7f307c04a4c0 con 0x7f3088136e40 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308cd1a640 1 -- 172.21.15.134:0/336859133 reap_dead start 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 shutdown_connections 2024-09-16T15:14:33.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 >> 172.21.15.134:0/336859133 conn(0x7f30880a6110 msgr2=0x7f308809f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 shutdown_connections 2024-09-16T15:14:33.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.368+0000 7f308e77d640 1 -- 172.21.15.134:0/336859133 wait complete. 2024-09-16T15:14:33.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-09-16T15:14:33.407 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.503+0000 7f8843b45640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.503+0000 7f8843b45640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.503+0000 7f8843b45640 10 monclient: monmap: 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.504683+0000 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.504683+0000 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.505 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 Processor -- start 2024-09-16T15:14:33.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 -- start start 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient: init 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient: _add_conns ranks=[2,6,1,8,9,10,7,0,4,3,5] 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 0x7f883c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-b con 0x7f883c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-d con 0x7f883c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f883c135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883c073250 con 0x7f883c135dc0 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c09abc0 con 0x7f883c136e40 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f883c09ba50 con 0x7f883c137220 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.504+0000 7f8843b45640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f8843b45640 10 monclient(hunting): authenticate will time out at 1610.094007s 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88418e1640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 msgr2=0x7f883c09c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88418e1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 0x7f883c09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:45936/0 (socket says 172.21.15.134:45936) 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 -- 172.21.15.134:0/5543454 learned_addr learned my addr 172.21.15.134:0/5543454 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): get_auth_request con 0x7f883c137220 auth_method 0 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient(hunting): handle_auth_done global_id 94263 payload 274 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 -- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 msgr2=0x7f883c09c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 --2- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 0x7f883c09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 -- 172.21.15.134:0/5543454 >> v1:172.21.15.162:6789/0 conn(0x7f883c135dc0 legacy=0x7f883c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 monclient: found mon.noname-d 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:14:33.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.505+0000 7f88410e0640 1 -- 172.21.15.134:0/5543454 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0a53c0 con 0x7f883c137220 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88410e0640 1 -- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 msgr2=0x7f883c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88410e0640 1 --2- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f882c0098d0 tx=0x7f882c02a110 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1092061603535702 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 1 -- 172.21.15.134:0/5543454 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f882c0307b0 con 0x7f883c137220 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: dump: 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.517 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 5 monclient: authenticate success, global_id 94263 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 1 -- 172.21.15.134:0/5543454 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c030930 con 0x7f883c137220 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 1 -- 172.21.15.134:0/5543454 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f882c0313a0 con 0x7f883c137220 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 4 set_mon_vals no callback set 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f88408df640 10 monclient: dump: 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.519 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 10 monclient: shutdown 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 -- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 msgr2=0x7f883c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 --2- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f882c0098d0 tx=0x7f882c02a110 comp rx=0 tx=0).stop 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 -- 172.21.15.134:0/5543454 shutdown_connections 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 --2- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f883c137220 0x7f883c137600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 --2- 172.21.15.134:0/5543454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f883c136e40 0x7f883c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.508+0000 7f8843b45640 1 -- 172.21.15.134:0/5543454 >> 172.21.15.134:0/5543454 conn(0x7f883c0a6110 msgr2=0x7f883c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 1 -- 172.21.15.134:0/5543454 shutdown_connections 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 1 -- 172.21.15.134:0/5543454 wait complete. 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 10 monclient: monmap: 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.521 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 1 Processor -- start 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 1 -- start start 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.509+0000 7f8843b45640 10 monclient: init 2024-09-16T15:14:33.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient: _add_conns ranks=[3,7,4,2,5,8,1,6,0] 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f883c135dc0 0x7f883c134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-g con 0x7f883c135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f883c136e40 0x7f883c135500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-i con 0x7f883c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 0x7f883c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): picked mon.noname-c con 0x7f883c137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c122100 con 0x7f883c135dc0 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f883c120fd0 con 0x7f883c137220 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1cc370 con 0x7f883c136e40 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f8843b45640 10 monclient(hunting): authenticate will time out at 1610.099490s 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 0x7f883c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 0x7f883c11fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41078/0 (socket says 172.21.15.134:41078) 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 learned_addr learned my addr 172.21.15.134:0/3318496716 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): get_auth_request con 0x7f883c137220 auth_method 0 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88410e0640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f883c136e40 msgr2=0x7f883c135500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88410e0640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f883c136e40 0x7f883c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88418e1640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f883c135dc0 0x7f883c134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88418e1640 10 monclient(hunting): get_auth_request con 0x7f883c135dc0 auth_method 0 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88418e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88418e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88418e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient(hunting): handle_auth_done global_id 5827 payload 824 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f883c136e40 msgr2=0x7f883c135500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f883c136e40 0x7f883c135500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f883c135dc0 msgr2=0x7f883c134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f883c135dc0 0x7f883c134fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.510+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c0094d0 con 0x7f883c137220 2024-09-16T15:14:33.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 msgr2=0x7f883c11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88420e2640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 0x7f883c11fc60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f883400c780 tx=0x7f883400cc40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f883400ce90 con 0x7f883c137220 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: dump: 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.526 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834004960 con 0x7f883c137220 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 5 monclient: authenticate success, global_id 5827 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 10 monclient: _renew_subs 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c1cb8f0 con 0x7f883c137220 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8834005030 con 0x7f883c137220 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f88327fc640 10 monclient: dump: 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.528 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8833fff640 4 set_mon_vals no callback set 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 10 monclient: _renew_subs 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.513+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c1229a0 con 0x7f883c137220 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f8843b45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f8843b45640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 0x7f883c02b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c071b20 con 0x7f883c02b260 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8834004b50 con 0x7f883c137220 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88327fc640 1 --2- 172.21.15.134:0/3318496716 >> v2:172.21.15.162:6800/980547263 conn(0x7f8820038880 0x7f882003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f883404a290 con 0x7f883c137220 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 0x7f883c02b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient: get_auth_request con 0x7f883c02b260 auth_method 0 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88410e0640 1 --2- 172.21.15.134:0/3318496716 >> v2:172.21.15.162:6800/980547263 conn(0x7f8820038880 0x7f882003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88410e0640 10 monclient: get_auth_request con 0x7f8820038880 auth_method 0 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 10 monclient(hunting): handle_auth_done global_id 45037 payload 824 2024-09-16T15:14:33.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.514+0000 7f88418e1640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f883c061ae0 con 0x7f883c02b260 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.516+0000 7f88410e0640 1 --2- 172.21.15.134:0/3318496716 >> v2:172.21.15.162:6800/980547263 conn(0x7f8820038880 0x7f882003ad40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f883c06b550 tx=0x7f882c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.516+0000 7f88418e1640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 msgr2=0x7f883c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.516+0000 7f88418e1640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 0x7f883c02b640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f883800d700 tx=0x7f883800dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8838003e20 con 0x7f883c02b260 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 10 monclient: dump: 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.533 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8838005c50 con 0x7f883c02b260 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 msgr2=0x7f883c02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 0x7f883c02b640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f883800d700 tx=0x7f883800dbc0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 msgr2=0x7f883c02b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.517+0000 7f88327fc640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02b260 0x7f883c02b640 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f883800d700 tx=0x7f883800dbc0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f8843b45640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f8843b45640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 0x7f883c02bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f8843b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c07a680 con 0x7f883c02bb80 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 0x7f883c02bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient: get_auth_request con 0x7f883c02bb80 auth_method 0 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.523+0000 7f88418e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.524+0000 7f88418e1640 10 monclient(hunting): handle_auth_done global_id 45046 payload 824 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.524+0000 7f88418e1640 1 -- 172.21.15.134:0/3318496716 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f883c09b490 con 0x7f883c02bb80 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88418e1640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 msgr2=0x7f883c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88418e1640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 0x7f883c02bf90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8838015000 tx=0x7f88380059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8838020070 con 0x7f883c02bb80 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 10 monclient: dump: 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.536 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.537 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9331 (secure 0 0 0) 0x7f8838005110 con 0x7f883c02bb80 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 msgr2=0x7f883c02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 0x7f883c02bf90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8838015000 tx=0x7f88380059d0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 msgr2=0x7f883c02bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.526+0000 7f88327fc640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f883c02bb80 0x7f883c02bf90 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8838015000 tx=0x7f88380059d0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.538 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.539 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.540 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.541 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.542 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.545 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.545 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.548 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.553 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.558 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.559 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.560 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.561 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.562 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.563 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 >> v2:172.21.15.162:6800/980547263 conn(0x7f8820038880 msgr2=0x7f882003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 --2- 172.21.15.134:0/3318496716 >> v2:172.21.15.162:6800/980547263 conn(0x7f8820038880 0x7f882003ad40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f883c06b550 tx=0x7f882c02e000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 10 monclient: shutdown 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 msgr2=0x7f883c11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 --2- 172.21.15.134:0/3318496716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f883c137220 0x7f883c11fc60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f883400c780 tx=0x7f883400cc40 comp rx=0 tx=0).stop 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 reap_dead start 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f88420e2640 1 -- 172.21.15.134:0/3318496716 reap_dead start 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 shutdown_connections 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 >> 172.21.15.134:0/3318496716 conn(0x7f883c0a6110 msgr2=0x7f883c09f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 shutdown_connections 2024-09-16T15:14:33.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.527+0000 7f8843b45640 1 -- 172.21.15.134:0/3318496716 wait complete. 2024-09-16T15:14:33.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.566 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-09-16T15:14:33.566 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:14:33.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.667+0000 7fdebb325640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.667+0000 7fdebb325640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.667+0000 7fdebb325640 10 monclient: monmap: 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.668535+0000 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.668535+0000 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.670 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 Processor -- start 2024-09-16T15:14:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 -- start start 2024-09-16T15:14:33.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient: init 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient: _add_conns ranks=[10,8,3,7,9,2,5,6,1,0,4] 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdeb4136e40 0x7fdeb409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-f con 0x7fdeb4136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-e con 0x7fdeb4137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb4135dc0 0x7fdeb4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-g con 0x7fdeb4135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb409abc0 con 0x7fdeb4136e40 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb409ba50 con 0x7fdeb4135dc0 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40a53c0 con 0x7fdeb4137220 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdebb325640 10 monclient(hunting): authenticate will time out at 1610.257899s 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.668+0000 7fdeb88c0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52668/0 (socket says 172.21.15.134:52668) 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 -- 172.21.15.134:0/183193008 learned_addr learned my addr 172.21.15.134:0/183193008 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): get_auth_request con 0x7fdeb4137220 auth_method 0 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb90c1640 1 --2- 172.21.15.134:0/183193008 >> v2:172.21.15.162:3300/0 conn(0x7fdeb4136e40 0x7fdeb409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb98c2640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb4135dc0 0x7fdeb4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb90c1640 10 monclient(hunting): get_auth_request con 0x7fdeb4136e40 auth_method 0 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb90c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb90c1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb90c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb98c2640 10 monclient(hunting): get_auth_request con 0x7fdeb4135dc0 auth_method 0 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb98c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb98c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb98c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient(hunting): handle_auth_done global_id 84527 payload 274 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 -- 172.21.15.134:0/183193008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb4135dc0 msgr2=0x7fdeb4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb4135dc0 0x7fdeb4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 -- 172.21.15.134:0/183193008 >> v2:172.21.15.162:3300/0 conn(0x7fdeb4136e40 msgr2=0x7fdeb409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:14:33.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 --2- 172.21.15.134:0/183193008 >> v2:172.21.15.162:3300/0 conn(0x7fdeb4136e40 0x7fdeb409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 monclient: found mon.noname-e 2024-09-16T15:14:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:14:33.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.669+0000 7fdeb88c0640 1 -- 172.21.15.134:0/183193008 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4073250 con 0x7fdeb4137220 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeb88c0640 1 -- 172.21.15.134:0/183193008 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 msgr2=0x7fdeb4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeb88c0640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdea40098a0 tx=0x7fdea402a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4d73786cfa01ec8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 1 -- 172.21.15.134:0/183193008 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdea402adf0 con 0x7fdeb4137220 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: dump: 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.683 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 5 monclient: authenticate success, global_id 84527 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 1 -- 172.21.15.134:0/183193008 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40307b0 con 0x7fdeb4137220 2024-09-16T15:14:33.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 1 -- 172.21.15.134:0/183193008 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdea4030d90 con 0x7fdeb4137220 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 4 set_mon_vals no callback set 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdeabfff640 10 monclient: dump: 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.685 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 10 monclient: shutdown 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 -- 172.21.15.134:0/183193008 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 msgr2=0x7fdeb4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdea40098a0 tx=0x7fdea402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 -- 172.21.15.134:0/183193008 shutdown_connections 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb4135dc0 0x7fdeb4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 --2- 172.21.15.134:0/183193008 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdeb4137220 0x7fdeb4137600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 --2- 172.21.15.134:0/183193008 >> v2:172.21.15.162:3300/0 conn(0x7fdeb4136e40 0x7fdeb409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 -- 172.21.15.134:0/183193008 >> 172.21.15.134:0/183193008 conn(0x7fdeb40a6110 msgr2=0x7fdeb40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 -- 172.21.15.134:0/183193008 shutdown_connections 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.672+0000 7fdebb325640 1 -- 172.21.15.134:0/183193008 wait complete. 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.673+0000 7fdebb325640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.673+0000 7fdebb325640 10 monclient: monmap: 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.687 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.673+0000 7fdebb325640 1 Processor -- start 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.673+0000 7fdebb325640 1 -- start start 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.673+0000 7fdebb325640 10 monclient: init 2024-09-16T15:14:33.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient: _add_conns ranks=[4,7,0,3,5,6,1,8,2] 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-c con 0x7fdeb4135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdeb4136e40 0x7fdeb4134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-i con 0x7fdeb4136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb4137220 0x7fdeb4135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): picked mon.noname-a con 0x7fdeb4137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb41cb760 con 0x7fdeb4137220 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4123cc0 con 0x7fdeb4135dc0 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4123570 con 0x7fdeb4136e40 2024-09-16T15:14:33.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdebb325640 10 monclient(hunting): authenticate will time out at 1610.263402s 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb98c2640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb4137220 msgr2=0x7fdeb4135490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb98c2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb4137220 0x7fdeb4135490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb88c0640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdeb4136e40 msgr2=0x7fdeb4134f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb88c0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdeb4136e40 0x7fdeb4134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41090/0 (socket says 172.21.15.134:41090) 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 -- 172.21.15.134:0/711108597 learned_addr learned my addr 172.21.15.134:0/711108597 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): get_auth_request con 0x7fdeb4135dc0 auth_method 0 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient(hunting): handle_auth_done global_id 5836 payload 824 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdeb4136e40 msgr2=0x7fdeb4134f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdeb4136e40 0x7fdeb4134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb4137220 msgr2=0x7fdeb4135490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdeb4137220 0x7fdeb4135490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.674+0000 7fdeb90c1640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40094d0 con 0x7fdeb4135dc0 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.676+0000 7fdeb90c1640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 msgr2=0x7fdeb41cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.676+0000 7fdeb90c1640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000d830 tx=0x7fdeb000dcf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdeb000bd40 con 0x7fdeb4135dc0 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: dump: 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.692 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0014430 con 0x7fdeb4135dc0 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 5 monclient: authenticate success, global_id 5836 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdeb0014a10 con 0x7fdeb4135dc0 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 10 monclient: _renew_subs 2024-09-16T15:14:33.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb40a8a30 con 0x7fdeb4135dc0 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdeab7fe640 4 set_mon_vals no callback set 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 10 monclient: dump: 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.694 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 10 monclient: _renew_subs 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb41363b0 con 0x7fdeb4135dc0 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.677+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdeb00145a0 con 0x7fdeb4135dc0 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdea9ffb640 1 --2- 172.21.15.134:0/711108597 >> v2:172.21.15.162:6800/980547263 conn(0x7fde980388b0 0x7fde9803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdebb325640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdebb325640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 0x7fdeb41221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (secure 0 0 0) 0x7fdeb004a140 con 0x7fdeb4135dc0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4065cf0 con 0x7fdeb41255c0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb88c0640 1 --2- 172.21.15.134:0/711108597 >> v2:172.21.15.162:6800/980547263 conn(0x7fde980388b0 0x7fde9803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 0x7fdeb41221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb88c0640 10 monclient: get_auth_request con 0x7fde980388b0 auth_method 0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient: get_auth_request con 0x7fdeb41255c0 auth_method 0 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_done global_id 84423 payload 824 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.678+0000 7fdeb98c2640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb4061ae0 con 0x7fdeb41255c0 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.680+0000 7fdeb98c2640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 msgr2=0x7fdeb41221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.680+0000 7fdeb88c0640 1 --2- 172.21.15.134:0/711108597 >> v2:172.21.15.162:6800/980547263 conn(0x7fde980388b0 0x7fde9803ad70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4137f00 tx=0x7fdea402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdeb98c2640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 0x7fdeb41221f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007c90 tx=0x7fdeac00e4b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdeac00ee40 con 0x7fdeb41255c0 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 10 monclient: dump: 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.697 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdeac003990 con 0x7fdeb41255c0 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 msgr2=0x7fdeb41221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 0x7fdeb41221f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdeac007c90 tx=0x7fdeac00e4b0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 msgr2=0x7fdeb41221f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.681+0000 7fdea9ffb640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb41255c0 0x7fdeb41221f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdebb325640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdebb325640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 0x7fdeb402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdebb325640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4071b20 con 0x7fdeb402b260 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 0x7fdeb402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient: get_auth_request con 0x7fdeb402b260 auth_method 0 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.687+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.688+0000 7fdeb98c2640 10 monclient(hunting): handle_auth_done global_id 84432 payload 824 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.688+0000 7fdeb98c2640 1 -- 172.21.15.134:0/711108597 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdeb407a680 con 0x7fdeb402b260 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdeb98c2640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 msgr2=0x7fdeb402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdeb98c2640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 0x7fdeb402b670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdeac016000 tx=0x7fdeac004ab0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdeac0044a0 con 0x7fdeb402b260 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdea9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdea9ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.690+0000 7fdea9ffb640 10 monclient: dump: 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.702 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9163 (secure 0 0 0) 0x7fdeac004600 con 0x7fdeb402b260 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 msgr2=0x7fdeb402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 0x7fdeb402b670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdeac016000 tx=0x7fdeac004ab0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 msgr2=0x7fdeb402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdea9ffb640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdeb402b260 0x7fdeb402b670 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdeac016000 tx=0x7fdeac004ab0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.703 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.704 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.705 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.706 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.707 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.708 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.710 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.715 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.718 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.719 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.721 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.724 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.725 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.725 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 >> v2:172.21.15.162:6800/980547263 conn(0x7fde980388b0 msgr2=0x7fde9803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 --2- 172.21.15.134:0/711108597 >> v2:172.21.15.162:6800/980547263 conn(0x7fde980388b0 0x7fde9803ad70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4137f00 tx=0x7fdea402e000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 10 monclient: shutdown 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 msgr2=0x7fdeb41cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdeb98c2640 1 -- 172.21.15.134:0/711108597 reap_dead start 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000d830 tx=0x7fdeb000dcf0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 shutdown_connections 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 --2- 172.21.15.134:0/711108597 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdeb4135dc0 0x7fdeb41cb220 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 >> 172.21.15.134:0/711108597 conn(0x7fdeb40a6110 msgr2=0x7fdeb409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 shutdown_connections 2024-09-16T15:14:33.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.691+0000 7fdebb325640 1 -- 172.21.15.134:0/711108597 wait complete. 2024-09-16T15:14:33.727 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-16T15:14:33.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:14:33.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.823+0000 7f597e280640 10 monclient: get_monmap_and_config 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.823+0000 7f597e280640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.823+0000 7f597e280640 10 monclient: monmap: 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:14:33.824532+0000 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:14:33.824532+0000 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.825 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:14:33.826 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 Processor -- start 2024-09-16T15:14:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 -- start start 2024-09-16T15:14:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient: init 2024-09-16T15:14:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient: _add_conns ranks=[2,3,0,6,5,1,10,9,7,4,8] 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 0x7f597809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): picked mon.noname-b con 0x7f5978136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): picked mon.noname-g con 0x7f5978137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5978135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5978073250 con 0x7f5978135dc0 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597809abc0 con 0x7f5978136e40 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597809ba50 con 0x7f5978137220 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597e280640 10 monclient(hunting): authenticate will time out at 1610.413862s 2024-09-16T15:14:33.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f5977fff640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 msgr2=0x7f597809c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f597c81d640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f5978135dc0 legacy=0x7f5978136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.824+0000 7f5977fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 0x7f597809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:46912/0 (socket says 172.21.15.134:46912) 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 -- 172.21.15.134:0/3015213880 learned_addr learned my addr 172.21.15.134:0/3015213880 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): get_auth_request con 0x7f5978137220 auth_method 0 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient(hunting): handle_auth_done global_id 84441 payload 274 2024-09-16T15:14:33.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 -- 172.21.15.134:0/3015213880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 msgr2=0x7f597809c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 --2- 172.21.15.134:0/3015213880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 0x7f597809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 -- 172.21.15.134:0/3015213880 >> v1:172.21.15.134:6789/0 conn(0x7f5978135dc0 legacy=0x7f5978136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 monclient: found mon.noname-g 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:14:33.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.825+0000 7f59777fe640 1 -- 172.21.15.134:0/3015213880 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780a53c0 con 0x7f5978137220 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f59777fe640 1 -- 172.21.15.134:0/3015213880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 msgr2=0x7f5978137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f59777fe640 1 --2- 172.21.15.134:0/3015213880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098d0 tx=0x7f596c02a110 comp rx=0 tx=0).ready entity=mon.3 client_cookie=f2cc2b01a909fe1e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 1 -- 172.21.15.134:0/3015213880 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f596c0307b0 con 0x7f5978137220 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: dump: 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.837 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 1 -- 172.21.15.134:0/3015213880 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c030930 con 0x7f5978137220 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 5 monclient: authenticate success, global_id 84441 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 10 monclient: get_monmap_and_config success 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 1 -- 172.21.15.134:0/3015213880 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f596c0313a0 con 0x7f5978137220 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 4 set_mon_vals no callback set 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f5976ffd640 10 monclient: dump: 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.839 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.840 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 10 monclient: shutdown 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 -- 172.21.15.134:0/3015213880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 msgr2=0x7f5978137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 --2- 172.21.15.134:0/3015213880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098d0 tx=0x7f596c02a110 comp rx=0 tx=0).stop 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 -- 172.21.15.134:0/3015213880 shutdown_connections 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 --2- 172.21.15.134:0/3015213880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5978137220 0x7f5978137600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 --2- 172.21.15.134:0/3015213880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5978136e40 0x7f597809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 -- 172.21.15.134:0/3015213880 >> 172.21.15.134:0/3015213880 conn(0x7f59780a6110 msgr2=0x7f59780a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 -- 172.21.15.134:0/3015213880 shutdown_connections 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.828+0000 7f597e280640 1 -- 172.21.15.134:0/3015213880 wait complete. 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.829+0000 7f597e280640 10 monclient: build_initial_monmap 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.829+0000 7f597e280640 10 monclient: monmap: 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:14:33.841 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:14:33.842 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.829+0000 7f597e280640 1 Processor -- start 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 -- start start 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient: init 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient: _reopen_session rank -1 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient: _add_conns ranks=[0,7,4,6,5,8,1,3,2] 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5978135dc0 0x7f59781cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): picked mon.noname-a con 0x7f5978135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5978136e40 0x7f5978134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): picked mon.noname-i con 0x7f5978136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 0x7f5978135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): picked mon.noname-c con 0x7f5978137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59781cb7f0 con 0x7f5978135dc0 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59780a8b90 con 0x7f5978137220 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5978136a30 con 0x7f5978136e40 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f5977fff640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5978135dc0 msgr2=0x7f59781cb2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): _renew_subs 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597e280640 10 monclient(hunting): authenticate will time out at 1610.419456s 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f5977fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5978135dc0 0x7f59781cb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 0x7f5978135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f59777fe640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5978136e40 msgr2=0x7f5978134f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f59777fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5978136e40 0x7f5978134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 0x7f5978135490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:41104/0 (socket says 172.21.15.134:41104) 2024-09-16T15:14:33.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 learned_addr learned my addr 172.21.15.134:0/3127007223 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): get_auth_request con 0x7f5978137220 auth_method 0 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient(hunting): handle_auth_done global_id 5845 payload 824 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5978136e40 msgr2=0x7f5978134f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f5978136e40 0x7f5978134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5978135dc0 msgr2=0x7f59781cb2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5978135dc0 0x7f59781cb2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient: _finish_hunting 0 2024-09-16T15:14:33.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 monclient: found mon.noname-c 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.830+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0094d0 con 0x7f5978137220 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 msgr2=0x7f5978135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597c81d640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 0x7f5978135490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f597000be90 tx=0x7f597000bf80 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f59700158e0 con 0x7f5978137220 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: dump: 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.846 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: _finish_auth 0 2024-09-16T15:14:33.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970015a60 con 0x7f5978137220 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 5 monclient: authenticate success, global_id 5845 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 10 monclient: _renew_subs 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5978125660 con 0x7f5978137220 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5970015bd0 con 0x7f5978137220 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5974ff9640 10 monclient: dump: 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.848 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.849 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 10 monclient: _renew_subs 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f5975ffb640 4 set_mon_vals no callback set 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.833+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978136500 con 0x7f5978137220 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f597e280640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f597e280640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 0x7f597802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5978125160 con 0x7f5978122170 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f5970012070 con 0x7f5978137220 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5974ff9640 1 --2- 172.21.15.134:0/3127007223 >> v2:172.21.15.162:6800/980547263 conn(0x7f5964038880 0x7f596403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(628..628 src has 1..628) v4 ==== 3959+0+0 (secure 0 0 0) 0x7f597004aa60 con 0x7f5978137220 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 0x7f597802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.850 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient: get_auth_request con 0x7f5978122170 auth_method 0 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f59777fe640 1 --2- 172.21.15.134:0/3127007223 >> v2:172.21.15.162:6800/980547263 conn(0x7f5964038880 0x7f596403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f59777fe640 10 monclient: get_auth_request con 0x7f5964038880 auth_method 0 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 10 monclient(hunting): handle_auth_done global_id 45059 payload 824 2024-09-16T15:14:33.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.834+0000 7f5977fff640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5978061ae0 con 0x7f5978122170 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5977fff640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 msgr2=0x7f597802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5977fff640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 0x7f597802b260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5968003980 tx=0x7f596800b830 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f59777fe640 1 --2- 172.21.15.134:0/3127007223 >> v2:172.21.15.162:6800/980547263 conn(0x7f5964038880 0x7f596403ad40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f596c034000 tx=0x7f596c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f59680054e0 con 0x7f5978122170 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 10 monclient: dump: 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.854 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5968005640 con 0x7f5978122170 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:14:33.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 msgr2=0x7f597802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 0x7f597802b260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5968003980 tx=0x7f596800b830 comp rx=0 tx=0).stop 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 msgr2=0x7f597802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.837+0000 7f5974ff9640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122170 0x7f597802b260 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5968003980 tx=0x7f596800b830 comp rx=0 tx=0).stop 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.843+0000 7f597e280640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.843+0000 7f597e280640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 0x7f597802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.843+0000 7f597e280640 10 monclient(hunting): start opening mon connection 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.843+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5978071b20 con 0x7f5978122550 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.843+0000 7f5977fff640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 0x7f597802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient: get_auth_request con 0x7f5978122550 auth_method 0 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:14:33.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 10 monclient(hunting): handle_auth_done global_id 45068 payload 824 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.844+0000 7f5977fff640 1 -- 172.21.15.134:0/3127007223 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f597807a680 con 0x7f5978122550 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5977fff640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 msgr2=0x7f597802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5977fff640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 0x7f597802b7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f59680037f0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f596801e070 con 0x7f5978122550 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5974ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5974ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.846+0000 7f5974ff9640 10 monclient: dump: 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:14:33.857 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:14:33.858 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9495 (secure 0 0 0) 0x7f5968022410 con 0x7f5978122550 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 msgr2=0x7f597802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 0x7f597802b7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f59680037f0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 msgr2=0x7f597802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f5974ff9640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f5978122550 0x7f597802b7a0 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5968000c00 tx=0x7f59680037f0 comp rx=0 tx=0).stop 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.859 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 114, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.860 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.861 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.862 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:14:33.863 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.864 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.865 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.866 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.867 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.868 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:14:33.869 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.870 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.871 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.872 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.873 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.874 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:14:33.875 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.876 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.877 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:14:33.878 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.879 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.880 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.881 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:14:33.882 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 >> v2:172.21.15.162:6800/980547263 conn(0x7f5964038880 msgr2=0x7f596403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 --2- 172.21.15.134:0/3127007223 >> v2:172.21.15.162:6800/980547263 conn(0x7f5964038880 0x7f596403ad40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f596c034000 tx=0x7f596c02e000 comp rx=0 tx=0).stop 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 10 monclient: shutdown 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 msgr2=0x7f5978135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 --2- 172.21.15.134:0/3127007223 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5978137220 0x7f5978135490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f597000be90 tx=0x7f597000bf80 comp rx=0 tx=0).stop 2024-09-16T15:14:33.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 reap_dead start 2024-09-16T15:14:33.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597c81d640 1 -- 172.21.15.134:0/3127007223 reap_dead start 2024-09-16T15:14:33.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 shutdown_connections 2024-09-16T15:14:33.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 >> 172.21.15.134:0/3127007223 conn(0x7f59780a6110 msgr2=0x7f597809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:14:33.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 shutdown_connections 2024-09-16T15:14:33.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:14:33.847+0000 7f597e280640 1 -- 172.21.15.134:0/3127007223 wait complete. 2024-09-16T15:14:33.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:14:33.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:14:34.019 INFO:tasks.rados.rados.0.smithi162.stdout:3019: finishing rollback tid 0 to smithi16234619-18 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3019: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3019: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1472 (ObjNum 983 snap 278 seq_num 983) dirty exists 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: finishing write tid 1 to smithi16234619-33 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3015: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: finishing write tid 2 to smithi16234619-33 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: finishing write tid 3 to smithi16234619-33 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: finishing write tid 4 to smithi16234619-33 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: finishing write tid 5 to smithi16234619-33 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: oid 33 updating version 0 to 838 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: oid 33 updating version 838 to 839 2024-09-16T15:14:34.020 INFO:tasks.rados.rados.0.smithi162.stdout:3017: oid 33 version 839 is already newer than 837 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3017: oid 33 version 839 is already newer than 836 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 839 (ObjNum 1047 snap 311 seq_num 1047) dirty exists 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3017: left oid 33 (ObjNum 1047 snap 311 seq_num 1047) 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3015: finishing copy_from to smithi16234619-22 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 1280 (ObjNum 1011 snap 293 seq_num 1011) dirty exists 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3015: done (3 left) 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3017: done (2 left) 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3019: done (1 left) 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3020: done (0 left) 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3021: write (excl) oid 36 current snap is 311 2024-09-16T15:14:34.021 INFO:tasks.rados.rados.0.smithi162.stdout:3021: seq_num 1049 ranges {430543=476562,1385258=663035,2840837=669165} 2024-09-16T15:14:34.025 INFO:tasks.rados.rados.0.smithi162.stdout:3021: writing smithi16234619-36 from 430543 to 907105 tid 1 2024-09-16T15:14:34.031 INFO:tasks.rados.rados.0.smithi162.stdout:3021: writing smithi16234619-36 from 1385258 to 2048293 tid 2 2024-09-16T15:14:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:3021: writing smithi16234619-36 from 2840837 to 3510002 tid 3 2024-09-16T15:14:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:3022: write oid 20 current snap is 311 2024-09-16T15:14:34.039 INFO:tasks.rados.rados.0.smithi162.stdout:3022: seq_num 1050 ranges {447350=668164,1560093=406693,2584613=480437} 2024-09-16T15:14:34.045 INFO:tasks.rados.rados.0.smithi162.stdout:3022: writing smithi16234619-20 from 447350 to 1115514 tid 1 2024-09-16T15:14:34.051 INFO:tasks.rados.rados.0.smithi162.stdout:3022: writing smithi16234619-20 from 1560093 to 1966786 tid 2 2024-09-16T15:14:34.058 INFO:tasks.rados.rados.0.smithi162.stdout:3022: writing smithi16234619-20 from 2584613 to 3065050 tid 3 2024-09-16T15:14:34.058 INFO:tasks.rados.rados.0.smithi162.stdout:3023: snap_create 2024-09-16T15:14:34.067 INFO:tasks.rados.rados.0.smithi162.stdout:3021: finishing write tid 1 to smithi16234619-36 2024-09-16T15:14:34.068 INFO:tasks.rados.rados.0.smithi162.stdout:3021: finishing write tid 2 to smithi16234619-36 2024-09-16T15:14:34.068 INFO:tasks.rados.rados.0.smithi162.stdout:3021: finishing write tid 3 to smithi16234619-36 2024-09-16T15:14:34.068 INFO:tasks.rados.rados.0.smithi162.stdout:3021: finishing write tid 4 to smithi16234619-36 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: finishing write tid 5 to smithi16234619-36 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: oid 36 updating version 0 to 1387 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: oid 36 updating version 1387 to 1388 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: oid 36 version 1388 is already newer than 1386 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: oid 36 version 1388 is already newer than 1385 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1388 (ObjNum 1049 snap 311 seq_num 1049) dirty exists 2024-09-16T15:14:34.069 INFO:tasks.rados.rados.0.smithi162.stdout:3021: left oid 36 (ObjNum 1049 snap 311 seq_num 1049) 2024-09-16T15:14:34.085 INFO:tasks.rados.rados.0.smithi162.stdout:3022: finishing write tid 1 to smithi16234619-20 2024-09-16T15:14:34.086 INFO:tasks.rados.rados.0.smithi162.stdout:3022: finishing write tid 2 to smithi16234619-20 2024-09-16T15:14:34.086 INFO:tasks.rados.rados.0.smithi162.stdout:3022: finishing write tid 3 to smithi16234619-20 2024-09-16T15:14:34.086 INFO:tasks.rados.rados.0.smithi162.stdout:3022: finishing write tid 4 to smithi16234619-20 2024-09-16T15:14:34.086 INFO:tasks.rados.rados.0.smithi162.stdout:3022: finishing write tid 5 to smithi16234619-20 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:3022: oid 20 updating version 0 to 842 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:3022: oid 20 version 842 is already newer than 840 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:3022: oid 20 updating version 842 to 843 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:3022: oid 20 version 843 is already newer than 841 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 843 (ObjNum 1050 snap 311 seq_num 1050) dirty exists 2024-09-16T15:14:34.087 INFO:tasks.rados.rados.0.smithi162.stdout:3022: left oid 20 (ObjNum 1050 snap 311 seq_num 1050) 2024-09-16T15:14:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:3021: done (2 left) 2024-09-16T15:14:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:3022: done (1 left) 2024-09-16T15:14:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:3023: done (0 left) 2024-09-16T15:14:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:3024: copy_from oid 20 from oid 19 current snap is 312 2024-09-16T15:14:35.026 INFO:tasks.rados.rados.0.smithi162.stdout:3025: snap_remove snap 310 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3024: finishing copy_from racing read to smithi16234619-20 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3024: finishing copy_from to smithi16234619-20 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1295 (ObjNum 1008 snap 290 seq_num 1008) dirty exists 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3024: done (1 left) 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3025: done (0 left) 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3026: write oid 28 current snap is 312 2024-09-16T15:14:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3026: seq_num 1051 ranges {464157=743413,1618575=666704,3012036=408062} 2024-09-16T15:14:36.040 INFO:tasks.rados.rados.0.smithi162.stdout:3026: writing smithi16234619-28 from 464157 to 1207570 tid 1 2024-09-16T15:14:36.045 INFO:tasks.rados.rados.0.smithi162.stdout:3026: writing smithi16234619-28 from 1618575 to 2285279 tid 2 2024-09-16T15:14:36.052 INFO:tasks.rados.rados.0.smithi162.stdout:3026: writing smithi16234619-28 from 3012036 to 3420098 tid 3 2024-09-16T15:14:36.052 INFO:tasks.rados.rados.0.smithi162.stdout:3027: read oid 11 snap -1 2024-09-16T15:14:36.052 INFO:tasks.rados.rados.0.smithi162.stdout:3027: expect (ObjNum 1011 snap 293 seq_num 1011) 2024-09-16T15:14:36.052 INFO:tasks.rados.rados.0.smithi162.stdout:3028: delete oid 16 current snap is 312 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:3028: done (2 left) 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:3029: rollback oid 42 current snap is 312 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 303 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:3030: rollback oid 30 current snap is 312 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 284 2024-09-16T15:14:36.055 INFO:tasks.rados.rados.0.smithi162.stdout:3031: snap_remove snap 305 2024-09-16T15:14:37.038 INFO:tasks.rados.rados.0.smithi162.stdout:3031: done (4 left) 2024-09-16T15:14:37.038 INFO:tasks.rados.rados.0.smithi162.stdout:3032: write (excl) oid 20 current snap is 312 2024-09-16T15:14:37.039 INFO:tasks.rados.rados.0.smithi162.stdout:3032: seq_num 1053 ranges {497771=493911,1619186=554020,2950529=430606} 2024-09-16T15:14:37.044 INFO:tasks.rados.rados.0.smithi162.stdout:3032: writing smithi16234619-20 from 497771 to 991682 tid 1 2024-09-16T15:14:37.052 INFO:tasks.rados.rados.0.smithi162.stdout:3032: writing smithi16234619-20 from 1619186 to 2173206 tid 2 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3032: writing smithi16234619-20 from 2950529 to 3381135 tid 3 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3033: rollback oid 38 current snap is 312 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 307 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3029: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3029: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1527 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:14:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3027: expect (ObjNum 1011 snap 293 seq_num 1011) 2024-09-16T15:14:37.236 INFO:tasks.rados.rados.0.smithi162.stdout:3026: finishing write tid 1 to smithi16234619-28 2024-09-16T15:14:37.236 INFO:tasks.rados.rados.0.smithi162.stdout:3026: finishing write tid 2 to smithi16234619-28 2024-09-16T15:14:37.236 INFO:tasks.rados.rados.0.smithi162.stdout:3026: finishing write tid 3 to smithi16234619-28 2024-09-16T15:14:37.236 INFO:tasks.rados.rados.0.smithi162.stdout:3026: finishing write tid 4 to smithi16234619-28 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: finishing write tid 5 to smithi16234619-28 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: oid 28 updating version 0 to 1172 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: oid 28 updating version 1172 to 1175 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: oid 28 version 1175 is already newer than 1174 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: oid 28 version 1175 is already newer than 1173 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1175 (ObjNum 1051 snap 312 seq_num 1051) dirty exists 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: left oid 28 (ObjNum 1051 snap 312 seq_num 1051) 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3030: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1167 (ObjNum 33 snap 0 seq_num 851383544) dirty dne 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3033: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3033: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3026: done (5 left) 2024-09-16T15:14:37.237 INFO:tasks.rados.rados.0.smithi162.stdout:3027: done (4 left) 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3029: done (3 left) 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3030: done (2 left) 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3034: snap_create 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3033: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1356 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: finishing write tid 1 to smithi16234619-20 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: finishing write tid 2 to smithi16234619-20 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: finishing write tid 3 to smithi16234619-20 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: finishing write tid 4 to smithi16234619-20 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: finishing write tid 5 to smithi16234619-20 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: oid 20 updating version 0 to 1299 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: oid 20 version 1299 is already newer than 1296 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: oid 20 version 1299 is already newer than 1298 2024-09-16T15:14:37.238 INFO:tasks.rados.rados.0.smithi162.stdout:3032: oid 20 version 1299 is already newer than 1297 2024-09-16T15:14:37.239 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1299 (ObjNum 1053 snap 312 seq_num 1053) dirty exists 2024-09-16T15:14:37.239 INFO:tasks.rados.rados.0.smithi162.stdout:3032: left oid 20 (ObjNum 1053 snap 312 seq_num 1053) 2024-09-16T15:14:38.535 INFO:tasks.rados.rados.0.smithi162.stdout:3032: done (2 left) 2024-09-16T15:14:38.535 INFO:tasks.rados.rados.0.smithi162.stdout:3033: done (1 left) 2024-09-16T15:14:38.535 INFO:tasks.rados.rados.0.smithi162.stdout:3034: done (0 left) 2024-09-16T15:14:38.535 INFO:tasks.rados.rados.0.smithi162.stdout:3035: write (excl) oid 35 current snap is 313 2024-09-16T15:14:38.535 INFO:tasks.rados.rados.0.smithi162.stdout:3035: seq_num 1054 ranges {514578=569160,1677668=414031,2577952=641878} 2024-09-16T15:14:38.540 INFO:tasks.rados.rados.0.smithi162.stdout:3035: writing smithi16234619-35 from 514578 to 1083738 tid 1 2024-09-16T15:14:38.546 INFO:tasks.rados.rados.0.smithi162.stdout:3035: writing smithi16234619-35 from 1677668 to 2091699 tid 2 2024-09-16T15:14:38.554 INFO:tasks.rados.rados.0.smithi162.stdout:3035: writing smithi16234619-35 from 2577952 to 3219830 tid 3 2024-09-16T15:14:38.554 INFO:tasks.rados.rados.0.smithi162.stdout:3036: read oid 50 snap -1 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3036: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3037: read oid 45 snap -1 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3037: expect (ObjNum 993 snap 281 seq_num 993) 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3038: read oid 43 snap -1 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3038: expect (ObjNum 1033 snap 305 seq_num 1033) 2024-09-16T15:14:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3039: snap_create 2024-09-16T15:14:38.561 INFO:tasks.rados.rados.0.smithi162.stdout:3038: expect (ObjNum 1033 snap 305 seq_num 1033) 2024-09-16T15:14:38.677 INFO:tasks.rados.rados.0.smithi162.stdout:3037: expect (ObjNum 993 snap 281 seq_num 993) 2024-09-16T15:14:38.821 INFO:tasks.rados.rados.0.smithi162.stdout:3036: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: finishing write tid 1 to smithi16234619-35 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: finishing write tid 2 to smithi16234619-35 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: finishing write tid 3 to smithi16234619-35 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: finishing write tid 4 to smithi16234619-35 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: finishing write tid 5 to smithi16234619-35 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: oid 35 updating version 0 to 1360 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: oid 35 version 1360 is already newer than 1357 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: oid 35 version 1360 is already newer than 1358 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: oid 35 version 1360 is already newer than 1359 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1360 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:14:38.987 INFO:tasks.rados.rados.0.smithi162.stdout:3035: left oid 35 (ObjNum 1054 snap 313 seq_num 1054) 2024-09-16T15:14:40.371 INFO:tasks.rados.rados.0.smithi162.stdout:3035: done (4 left) 2024-09-16T15:14:40.371 INFO:tasks.rados.rados.0.smithi162.stdout:3036: done (3 left) 2024-09-16T15:14:40.371 INFO:tasks.rados.rados.0.smithi162.stdout:3037: done (2 left) 2024-09-16T15:14:40.372 INFO:tasks.rados.rados.0.smithi162.stdout:3038: done (1 left) 2024-09-16T15:14:40.372 INFO:tasks.rados.rados.0.smithi162.stdout:3039: done (0 left) 2024-09-16T15:14:40.372 INFO:tasks.rados.rados.0.smithi162.stdout:3040: snap_create 2024-09-16T15:14:41.373 INFO:tasks.rados.rados.0.smithi162.stdout:3040: done (0 left) 2024-09-16T15:14:41.373 INFO:tasks.rados.rados.0.smithi162.stdout:3041: read oid 16 snap 286 2024-09-16T15:14:41.373 INFO:tasks.rados.rados.0.smithi162.stdout:3041: expect (ObjNum 978 snap 276 seq_num 978) 2024-09-16T15:14:41.374 INFO:tasks.rados.rados.0.smithi162.stdout:3042: write oid 42 current snap is 315 2024-09-16T15:14:41.374 INFO:tasks.rados.rados.0.smithi162.stdout:3042: seq_num 1055 ranges {531385=644409,1736150=557689,3005375=569503} 2024-09-16T15:14:41.380 INFO:tasks.rados.rados.0.smithi162.stdout:3042: writing smithi16234619-42 from 531385 to 1175794 tid 1 2024-09-16T15:14:41.387 INFO:tasks.rados.rados.0.smithi162.stdout:3042: writing smithi16234619-42 from 1736150 to 2293839 tid 2 2024-09-16T15:14:41.395 INFO:tasks.rados.rados.0.smithi162.stdout:3042: writing smithi16234619-42 from 3005375 to 3574878 tid 3 2024-09-16T15:14:41.396 INFO:tasks.rados.rados.0.smithi162.stdout:3043: write (excl) oid 46 current snap is 315 2024-09-16T15:14:41.396 INFO:tasks.rados.rados.0.smithi162.stdout:3043: seq_num 1056 ranges {548192=719658,1678279=417700,2516445=780775} 2024-09-16T15:14:41.403 INFO:tasks.rados.rados.0.smithi162.stdout:3043: writing smithi16234619-46 from 548192 to 1267850 tid 1 2024-09-16T15:14:41.408 INFO:tasks.rados.rados.0.smithi162.stdout:3043: writing smithi16234619-46 from 1678279 to 2095979 tid 2 2024-09-16T15:14:41.414 INFO:tasks.rados.rados.0.smithi162.stdout:3043: writing smithi16234619-46 from 2516445 to 3297220 tid 3 2024-09-16T15:14:41.415 INFO:tasks.rados.rados.0.smithi162.stdout:3044: snap_remove snap 282 2024-09-16T15:14:41.415 INFO:tasks.rados.rados.0.smithi162.stdout:3041: expect (ObjNum 978 snap 276 seq_num 978) 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: finishing write tid 1 to smithi16234619-42 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: finishing write tid 2 to smithi16234619-42 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: finishing write tid 3 to smithi16234619-42 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: finishing write tid 4 to smithi16234619-42 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: finishing write tid 5 to smithi16234619-42 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: oid 42 updating version 0 to 1539 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: oid 42 updating version 1539 to 1540 2024-09-16T15:14:41.500 INFO:tasks.rados.rados.0.smithi162.stdout:3042: oid 42 version 1540 is already newer than 1538 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:3042: oid 42 version 1540 is already newer than 1537 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1540 (ObjNum 1055 snap 315 seq_num 1055) dirty exists 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:3042: left oid 42 (ObjNum 1055 snap 315 seq_num 1055) 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:3043: finishing write tid 1 to smithi16234619-46 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:3043: finishing write tid 2 to smithi16234619-46 2024-09-16T15:14:41.501 INFO:tasks.rados.rados.0.smithi162.stdout:3043: finishing write tid 3 to smithi16234619-46 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3041: done (3 left) 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3042: done (2 left) 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3044: done (1 left) 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3045: read oid 3 snap -1 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3045: expect (ObjNum 1028 snap 299 seq_num 1028) 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3046: read oid 45 snap 309 2024-09-16T15:14:42.633 INFO:tasks.rados.rados.0.smithi162.stdout:3046: expect (ObjNum 834 snap 224 seq_num 834) 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3047: read oid 24 snap -1 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3047: expect (ObjNum 1038 snap 311 seq_num 1038) 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3048: rollback oid 40 current snap is 315 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 40 to 311 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: finishing write tid 4 to smithi16234619-46 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: finishing write tid 5 to smithi16234619-46 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: oid 46 updating version 0 to 1197 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: oid 46 updating version 1197 to 1200 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: oid 46 version 1200 is already newer than 1198 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: oid 46 version 1200 is already newer than 1199 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1200 (ObjNum 1056 snap 315 seq_num 1056) dirty exists 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: left oid 46 (ObjNum 1056 snap 315 seq_num 1056) 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3043: done (4 left) 2024-09-16T15:14:42.634 INFO:tasks.rados.rados.0.smithi162.stdout:3049: snap_create 2024-09-16T15:14:42.639 INFO:tasks.rados.rados.0.smithi162.stdout:3046: expect (ObjNum 834 snap 224 seq_num 834) 2024-09-16T15:14:42.772 INFO:tasks.rados.rados.0.smithi162.stdout:3048: finishing rollback tid 0 to smithi16234619-40 2024-09-16T15:14:42.772 INFO:tasks.rados.rados.0.smithi162.stdout:3045: expect (ObjNum 1028 snap 299 seq_num 1028) 2024-09-16T15:14:42.922 INFO:tasks.rados.rados.0.smithi162.stdout:3048: finishing rollback tid 1 to smithi16234619-40 2024-09-16T15:14:42.922 INFO:tasks.rados.rados.0.smithi162.stdout:3048: finishing rollback tid 2 to smithi16234619-40 2024-09-16T15:14:42.922 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1441 (ObjNum 1039 snap 311 seq_num 1039) dirty exists 2024-09-16T15:14:42.922 INFO:tasks.rados.rados.0.smithi162.stdout:3047: expect (ObjNum 1038 snap 311 seq_num 1038) 2024-09-16T15:14:43.118 INFO:tasks.rados.rados.0.smithi162.stdout:3045: done (4 left) 2024-09-16T15:14:43.118 INFO:tasks.rados.rados.0.smithi162.stdout:3046: done (3 left) 2024-09-16T15:14:43.118 INFO:tasks.rados.rados.0.smithi162.stdout:3047: done (2 left) 2024-09-16T15:14:43.119 INFO:tasks.rados.rados.0.smithi162.stdout:3048: done (1 left) 2024-09-16T15:14:43.119 INFO:tasks.rados.rados.0.smithi162.stdout:3049: done (0 left) 2024-09-16T15:14:43.119 INFO:tasks.rados.rados.0.smithi162.stdout:3050: snap_remove snap 278 2024-09-16T15:14:44.145 INFO:tasks.rados.rados.0.smithi162.stdout:3050: done (0 left) 2024-09-16T15:14:44.145 INFO:tasks.rados.rados.0.smithi162.stdout:3051: write oid 40 current snap is 316 2024-09-16T15:14:44.145 INFO:tasks.rados.rados.0.smithi162.stdout:3051: seq_num 1057 ranges {564999=511260,1853114=561358,3060221=592047} 2024-09-16T15:14:44.151 INFO:tasks.rados.rados.0.smithi162.stdout:3051: writing smithi16234619-40 from 564999 to 1076259 tid 1 2024-09-16T15:14:44.158 INFO:tasks.rados.rados.0.smithi162.stdout:3051: writing smithi16234619-40 from 1853114 to 2414472 tid 2 2024-09-16T15:14:44.165 INFO:tasks.rados.rados.0.smithi162.stdout:3051: writing smithi16234619-40 from 3060221 to 3652268 tid 3 2024-09-16T15:14:44.165 INFO:tasks.rados.rados.0.smithi162.stdout:3052: write oid 47 current snap is 316 2024-09-16T15:14:44.165 INFO:tasks.rados.rados.0.smithi162.stdout:3052: seq_num 1058 ranges {581806=586509,1911596=421369,3087644=403319} 2024-09-16T15:14:44.172 INFO:tasks.rados.rados.0.smithi162.stdout:3052: writing smithi16234619-47 from 581806 to 1168315 tid 1 2024-09-16T15:14:44.178 INFO:tasks.rados.rados.0.smithi162.stdout:3052: writing smithi16234619-47 from 1911596 to 2332965 tid 2 2024-09-16T15:14:44.185 INFO:tasks.rados.rados.0.smithi162.stdout:3052: writing smithi16234619-47 from 3087644 to 3490963 tid 3 2024-09-16T15:14:44.185 INFO:tasks.rados.rados.0.smithi162.stdout:3053: read oid 7 snap -1 2024-09-16T15:14:44.185 INFO:tasks.rados.rados.0.smithi162.stdout:3053: expect deleted 2024-09-16T15:14:44.185 INFO:tasks.rados.rados.0.smithi162.stdout:3054: write oid 5 current snap is 316 2024-09-16T15:14:44.185 INFO:tasks.rados.rados.0.smithi162.stdout:3054: seq_num 1059 ranges {598613=661758,1970078=565027,3115067=614591} 2024-09-16T15:14:44.191 INFO:tasks.rados.rados.0.smithi162.stdout:3054: writing smithi16234619-5 from 598613 to 1260371 tid 1 2024-09-16T15:14:44.199 INFO:tasks.rados.rados.0.smithi162.stdout:3054: writing smithi16234619-5 from 1970078 to 2535105 tid 2 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3054: writing smithi16234619-5 from 3115067 to 3729658 tid 3 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3055: read oid 13 snap -1 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3055: expect (ObjNum 1042 snap 311 seq_num 1042) 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3056: snap_create 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: finishing write tid 1 to smithi16234619-40 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: finishing write tid 2 to smithi16234619-40 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: finishing write tid 3 to smithi16234619-40 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: finishing write tid 4 to smithi16234619-40 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: finishing write tid 5 to smithi16234619-40 2024-09-16T15:14:44.206 INFO:tasks.rados.rados.0.smithi162.stdout:3051: oid 40 updating version 0 to 1444 2024-09-16T15:14:44.207 INFO:tasks.rados.rados.0.smithi162.stdout:3051: oid 40 version 1444 is already newer than 1443 2024-09-16T15:14:44.207 INFO:tasks.rados.rados.0.smithi162.stdout:3051: oid 40 updating version 1444 to 1445 2024-09-16T15:14:44.207 INFO:tasks.rados.rados.0.smithi162.stdout:3051: oid 40 version 1445 is already newer than 1442 2024-09-16T15:14:44.207 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1445 (ObjNum 1057 snap 316 seq_num 1057) dirty exists 2024-09-16T15:14:44.207 INFO:tasks.rados.rados.0.smithi162.stdout:3051: left oid 40 (ObjNum 1057 snap 316 seq_num 1057) 2024-09-16T15:14:44.212 INFO:tasks.rados.rados.0.smithi162.stdout:3052: finishing write tid 1 to smithi16234619-47 2024-09-16T15:14:44.212 INFO:tasks.rados.rados.0.smithi162.stdout:3052: finishing write tid 2 to smithi16234619-47 2024-09-16T15:14:44.213 INFO:tasks.rados.rados.0.smithi162.stdout:3052: finishing write tid 3 to smithi16234619-47 2024-09-16T15:14:44.213 INFO:tasks.rados.rados.0.smithi162.stdout:3052: finishing write tid 4 to smithi16234619-47 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: finishing write tid 5 to smithi16234619-47 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: oid 47 updating version 0 to 1142 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: oid 47 updating version 1142 to 1144 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: oid 47 version 1144 is already newer than 1141 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: oid 47 version 1144 is already newer than 1143 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1144 (ObjNum 1058 snap 316 seq_num 1058) dirty exists 2024-09-16T15:14:44.214 INFO:tasks.rados.rados.0.smithi162.stdout:3052: left oid 47 (ObjNum 1058 snap 316 seq_num 1058) 2024-09-16T15:14:44.241 INFO:tasks.rados.rados.0.smithi162.stdout:3054: finishing write tid 1 to smithi16234619-5 2024-09-16T15:14:44.246 INFO:tasks.rados.rados.0.smithi162.stdout:3054: finishing write tid 2 to smithi16234619-5 2024-09-16T15:14:44.246 INFO:tasks.rados.rados.0.smithi162.stdout:3054: finishing write tid 3 to smithi16234619-5 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: finishing write tid 4 to smithi16234619-5 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: finishing write tid 5 to smithi16234619-5 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: oid 5 updating version 0 to 1560 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: oid 5 updating version 1560 to 1561 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: oid 5 updating version 1561 to 1563 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: oid 5 version 1563 is already newer than 1562 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1563 (ObjNum 1059 snap 316 seq_num 1059) dirty exists 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3054: left oid 5 (ObjNum 1059 snap 316 seq_num 1059) 2024-09-16T15:14:44.247 INFO:tasks.rados.rados.0.smithi162.stdout:3055: expect (ObjNum 1042 snap 311 seq_num 1042) 2024-09-16T15:14:45.352 INFO:tasks.rados.rados.0.smithi162.stdout:3051: done (5 left) 2024-09-16T15:14:45.352 INFO:tasks.rados.rados.0.smithi162.stdout:3052: done (4 left) 2024-09-16T15:14:45.352 INFO:tasks.rados.rados.0.smithi162.stdout:3053: done (3 left) 2024-09-16T15:14:45.353 INFO:tasks.rados.rados.0.smithi162.stdout:3054: done (2 left) 2024-09-16T15:14:45.353 INFO:tasks.rados.rados.0.smithi162.stdout:3055: done (1 left) 2024-09-16T15:14:45.353 INFO:tasks.rados.rados.0.smithi162.stdout:3056: done (0 left) 2024-09-16T15:14:45.353 INFO:tasks.rados.rados.0.smithi162.stdout:3057: delete oid 18 current snap is 317 2024-09-16T15:14:45.355 INFO:tasks.rados.rados.0.smithi162.stdout:3057: done (0 left) 2024-09-16T15:14:45.355 INFO:tasks.rados.rados.0.smithi162.stdout:3058: snap_remove snap 314 2024-09-16T15:14:46.358 INFO:tasks.rados.rados.0.smithi162.stdout:3058: done (0 left) 2024-09-16T15:14:46.359 INFO:tasks.rados.rados.0.smithi162.stdout:3059: snap_remove snap 302 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3059: done (0 left) 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3060: read oid 21 snap -1 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3060: expect deleted 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3061: copy_from oid 3 from oid 17 current snap is 317 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3062: write oid 36 current snap is 317 2024-09-16T15:14:47.362 INFO:tasks.rados.rados.0.smithi162.stdout:3062: seq_num 1061 ranges {632227=412256,1570689=568696,2653560=753488} 2024-09-16T15:14:47.368 INFO:tasks.rados.rados.0.smithi162.stdout:3062: writing smithi16234619-36 from 632227 to 1044483 tid 1 2024-09-16T15:14:47.374 INFO:tasks.rados.rados.0.smithi162.stdout:3062: writing smithi16234619-36 from 1570689 to 2139385 tid 2 2024-09-16T15:14:47.381 INFO:tasks.rados.rados.0.smithi162.stdout:3062: writing smithi16234619-36 from 2653560 to 3407048 tid 3 2024-09-16T15:14:47.381 INFO:tasks.rados.rados.0.smithi162.stdout:3063: write (excl) oid 49 current snap is 317 2024-09-16T15:14:47.382 INFO:tasks.rados.rados.0.smithi162.stdout:3063: seq_num 1062 ranges {649034=487505,1629171=428707,2680983=564760,3798768=50266} 2024-09-16T15:14:47.388 INFO:tasks.rados.rados.0.smithi162.stdout:3063: writing smithi16234619-49 from 649034 to 1136539 tid 1 2024-09-16T15:14:47.393 INFO:tasks.rados.rados.0.smithi162.stdout:3063: writing smithi16234619-49 from 1629171 to 2057878 tid 2 2024-09-16T15:14:47.399 INFO:tasks.rados.rados.0.smithi162.stdout:3063: writing smithi16234619-49 from 2680983 to 3245743 tid 3 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3063: writing smithi16234619-49 from 3798768 to 3849034 tid 4 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3064: copy_from oid 50 from oid 33 current snap is 317 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3061: finishing copy_from to smithi16234619-3 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3061: got expected ENOENT (src dne) 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3061: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:14:47.403 INFO:tasks.rados.rados.0.smithi162.stdout:3060: done (4 left) 2024-09-16T15:14:47.404 INFO:tasks.rados.rados.0.smithi162.stdout:3061: done (3 left) 2024-09-16T15:14:47.404 INFO:tasks.rados.rados.0.smithi162.stdout:3065: snap_create 2024-09-16T15:14:47.428 INFO:tasks.rados.rados.0.smithi162.stdout:3062: finishing write tid 1 to smithi16234619-36 2024-09-16T15:14:47.428 INFO:tasks.rados.rados.0.smithi162.stdout:3062: finishing write tid 2 to smithi16234619-36 2024-09-16T15:14:47.429 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 1 to smithi16234619-49 2024-09-16T15:14:47.429 INFO:tasks.rados.rados.0.smithi162.stdout:3062: finishing write tid 3 to smithi16234619-36 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: finishing write tid 4 to smithi16234619-36 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: finishing write tid 5 to smithi16234619-36 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: oid 36 updating version 0 to 1403 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: oid 36 updating version 1403 to 1405 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: oid 36 version 1405 is already newer than 1404 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: oid 36 version 1405 is already newer than 1402 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1405 (ObjNum 1061 snap 317 seq_num 1061) dirty exists 2024-09-16T15:14:47.430 INFO:tasks.rados.rados.0.smithi162.stdout:3062: left oid 36 (ObjNum 1061 snap 317 seq_num 1061) 2024-09-16T15:14:47.438 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 2 to smithi16234619-49 2024-09-16T15:14:47.440 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 3 to smithi16234619-49 2024-09-16T15:14:47.440 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 4 to smithi16234619-49 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 5 to smithi16234619-49 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: finishing write tid 6 to smithi16234619-49 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: oid 49 updating version 0 to 1281 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: oid 49 updating version 1281 to 1282 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: oid 49 updating version 1282 to 1283 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: oid 49 updating version 1283 to 1284 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: oid 49 updating version 1284 to 1285 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 1285 (ObjNum 1062 snap 317 seq_num 1062) dirty exists 2024-09-16T15:14:47.443 INFO:tasks.rados.rados.0.smithi162.stdout:3063: left oid 49 (ObjNum 1062 snap 317 seq_num 1062) 2024-09-16T15:14:47.491 INFO:tasks.rados.rados.0.smithi162.stdout:3064: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:14:47.499 INFO:tasks.rados.rados.0.smithi162.stdout:3064: finishing copy_from to smithi16234619-50 2024-09-16T15:14:47.499 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 839 (ObjNum 1047 snap 311 seq_num 1047) dirty exists 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3062: done (3 left) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3063: done (2 left) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3064: done (1 left) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3065: done (0 left) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3066: read oid 8 snap -1 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3066: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3067: read oid 46 snap -1 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3067: expect (ObjNum 1056 snap 315 seq_num 1056) 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3068: copy_from oid 3 from oid 25 current snap is 318 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3069: write oid 23 current snap is 318 2024-09-16T15:14:47.894 INFO:tasks.rados.rados.0.smithi162.stdout:3069: seq_num 1063 ranges {665841=562754,1687653=572365,2708406=776032} 2024-09-16T15:14:47.902 INFO:tasks.rados.rados.0.smithi162.stdout:3069: writing smithi16234619-23 from 665841 to 1228595 tid 1 2024-09-16T15:14:47.909 INFO:tasks.rados.rados.0.smithi162.stdout:3069: writing smithi16234619-23 from 1687653 to 2260018 tid 2 2024-09-16T15:14:47.918 INFO:tasks.rados.rados.0.smithi162.stdout:3069: writing smithi16234619-23 from 2708406 to 3484438 tid 3 2024-09-16T15:14:47.918 INFO:tasks.rados.rados.0.smithi162.stdout:3070: snap_remove snap 308 2024-09-16T15:14:48.907 INFO:tasks.rados.rados.0.smithi162.stdout:3070: done (4 left) 2024-09-16T15:14:48.907 INFO:tasks.rados.rados.0.smithi162.stdout:3071: write (excl) oid 50 current snap is 318 2024-09-16T15:14:48.907 INFO:tasks.rados.rados.0.smithi162.stdout:3071: seq_num 1064 ranges {682648=638003,2029782=432376,3019476=587304} 2024-09-16T15:14:48.915 INFO:tasks.rados.rados.0.smithi162.stdout:3071: writing smithi16234619-50 from 682648 to 1320651 tid 1 2024-09-16T15:14:48.921 INFO:tasks.rados.rados.0.smithi162.stdout:3071: writing smithi16234619-50 from 2029782 to 2462158 tid 2 2024-09-16T15:14:48.928 INFO:tasks.rados.rados.0.smithi162.stdout:3071: writing smithi16234619-50 from 3019476 to 3606780 tid 3 2024-09-16T15:14:48.928 INFO:tasks.rados.rados.0.smithi162.stdout:3072: read oid 26 snap -1 2024-09-16T15:14:48.928 INFO:tasks.rados.rados.0.smithi162.stdout:3072: expect (ObjNum 812 snap 219 seq_num 812) 2024-09-16T15:14:48.929 INFO:tasks.rados.rados.0.smithi162.stdout:3073: rollback oid 42 current snap is 318 2024-09-16T15:14:48.929 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 284 2024-09-16T15:14:48.929 INFO:tasks.rados.rados.0.smithi162.stdout:3066: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:14:49.014 INFO:tasks.rados.rados.0.smithi162.stdout:3067: expect (ObjNum 1056 snap 315 seq_num 1056) 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3068: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3068: finishing copy_from to smithi16234619-3 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1242 (ObjNum 984 snap 278 seq_num 984) dirty exists 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3069: finishing write tid 1 to smithi16234619-23 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3069: finishing write tid 2 to smithi16234619-23 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3069: finishing write tid 3 to smithi16234619-23 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3069: finishing write tid 4 to smithi16234619-23 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3066: done (6 left) 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3067: done (5 left) 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3068: done (4 left) 2024-09-16T15:14:49.184 INFO:tasks.rados.rados.0.smithi162.stdout:3074: write oid 17 current snap is 318 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: finishing write tid 5 to smithi16234619-23 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: oid 23 updating version 0 to 609 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: oid 23 version 609 is already newer than 608 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: oid 23 updating version 609 to 611 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: oid 23 version 611 is already newer than 610 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 611 (ObjNum 1063 snap 318 seq_num 1063) dirty exists 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3069: left oid 23 (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3073: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:14:49.185 INFO:tasks.rados.rados.0.smithi162.stdout:3072: expect (ObjNum 812 snap 219 seq_num 812) 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3073: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1588 (ObjNum 145 snap 0 seq_num 865848992) dirty dne 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: finishing write tid 1 to smithi16234619-50 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: finishing write tid 2 to smithi16234619-50 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: finishing write tid 3 to smithi16234619-50 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: finishing write tid 4 to smithi16234619-50 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: finishing write tid 5 to smithi16234619-50 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: oid 50 updating version 0 to 1412 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: oid 50 updating version 1412 to 1414 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: oid 50 version 1414 is already newer than 1411 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:3071: oid 50 version 1414 is already newer than 1413 2024-09-16T15:14:49.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1414 (ObjNum 1064 snap 318 seq_num 1064) dirty exists 2024-09-16T15:14:49.301 INFO:tasks.rados.rados.0.smithi162.stdout:3071: left oid 50 (ObjNum 1064 snap 318 seq_num 1064) 2024-09-16T15:14:49.301 INFO:tasks.rados.rados.0.smithi162.stdout:3074: seq_num 1065 ranges {699455=429605,1804617=576034,3163252=514929} 2024-09-16T15:14:49.305 INFO:tasks.rados.rados.0.smithi162.stdout:3074: writing smithi16234619-17 from 699455 to 1129060 tid 1 2024-09-16T15:14:49.312 INFO:tasks.rados.rados.0.smithi162.stdout:3074: writing smithi16234619-17 from 1804617 to 2380651 tid 2 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3074: writing smithi16234619-17 from 3163252 to 3678181 tid 3 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3069: done (4 left) 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3071: done (3 left) 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3072: done (2 left) 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3073: done (1 left) 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3075: rollback oid 26 current snap is 318 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 273 2024-09-16T15:14:49.320 INFO:tasks.rados.rados.0.smithi162.stdout:3076: delete oid 22 current snap is 318 2024-09-16T15:14:49.323 INFO:tasks.rados.rados.0.smithi162.stdout:3076: done (2 left) 2024-09-16T15:14:49.323 INFO:tasks.rados.rados.0.smithi162.stdout:3077: write (excl) oid 41 current snap is 318 2024-09-16T15:14:49.323 INFO:tasks.rados.rados.0.smithi162.stdout:3077: seq_num 1067 ranges {733069=580103,1921581=579703,3218098=537473} 2024-09-16T15:14:49.330 INFO:tasks.rados.rados.0.smithi162.stdout:3077: writing smithi16234619-41 from 733069 to 1313172 tid 1 2024-09-16T15:14:49.337 INFO:tasks.rados.rados.0.smithi162.stdout:3077: writing smithi16234619-41 from 1921581 to 2501284 tid 2 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3077: writing smithi16234619-41 from 3218098 to 3755571 tid 3 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3078: read oid 7 snap -1 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3078: expect deleted 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3079: read oid 24 snap 296 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3079: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3080: write (excl) oid 47 current snap is 318 2024-09-16T15:14:49.344 INFO:tasks.rados.rados.0.smithi162.stdout:3074: finishing write tid 1 to smithi16234619-17 2024-09-16T15:14:49.345 INFO:tasks.rados.rados.0.smithi162.stdout:3074: finishing write tid 2 to smithi16234619-17 2024-09-16T15:14:49.345 INFO:tasks.rados.rados.0.smithi162.stdout:3080: seq_num 1068 ranges {749876=655352,1863710=723361,3012815=748745} 2024-09-16T15:14:49.351 INFO:tasks.rados.rados.0.smithi162.stdout:3080: writing smithi16234619-47 from 749876 to 1405228 tid 1 2024-09-16T15:14:49.359 INFO:tasks.rados.rados.0.smithi162.stdout:3080: writing smithi16234619-47 from 1863710 to 2587071 tid 2 2024-09-16T15:14:49.367 INFO:tasks.rados.rados.0.smithi162.stdout:3080: writing smithi16234619-47 from 3012815 to 3761560 tid 3 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3081: snap_create 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: finishing write tid 3 to smithi16234619-17 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: finishing write tid 4 to smithi16234619-17 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: finishing write tid 5 to smithi16234619-17 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: oid 17 updating version 0 to 1670 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: oid 17 version 1670 is already newer than 1668 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: oid 17 version 1670 is already newer than 1667 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: oid 17 version 1670 is already newer than 1669 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1670 (ObjNum 1065 snap 318 seq_num 1065) dirty exists 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3074: left oid 17 (ObjNum 1065 snap 318 seq_num 1065) 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3075: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:14:49.368 INFO:tasks.rados.rados.0.smithi162.stdout:3079: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:49.537 INFO:tasks.rados.rados.0.smithi162.stdout:3075: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:14:49.537 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1673 (ObjNum 865389584 snap 21987 seq_num 304) dirty dne 2024-09-16T15:14:49.537 INFO:tasks.rados.rados.0.smithi162.stdout:3077: finishing write tid 1 to smithi16234619-41 2024-09-16T15:14:49.537 INFO:tasks.rados.rados.0.smithi162.stdout:3077: finishing write tid 2 to smithi16234619-41 2024-09-16T15:14:49.537 INFO:tasks.rados.rados.0.smithi162.stdout:3077: finishing write tid 3 to smithi16234619-41 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: finishing write tid 4 to smithi16234619-41 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: finishing write tid 5 to smithi16234619-41 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: oid 41 updating version 0 to 1290 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: oid 41 version 1290 is already newer than 1289 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: oid 41 version 1290 is already newer than 1287 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: oid 41 version 1290 is already newer than 1288 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 1290 (ObjNum 1067 snap 318 seq_num 1067) dirty exists 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3077: left oid 41 (ObjNum 1067 snap 318 seq_num 1067) 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: finishing write tid 1 to smithi16234619-47 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: finishing write tid 2 to smithi16234619-47 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: finishing write tid 3 to smithi16234619-47 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: finishing write tid 4 to smithi16234619-47 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: finishing write tid 5 to smithi16234619-47 2024-09-16T15:14:49.538 INFO:tasks.rados.rados.0.smithi162.stdout:3080: oid 47 updating version 0 to 1146 2024-09-16T15:14:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:3080: oid 47 updating version 1146 to 1148 2024-09-16T15:14:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:3080: oid 47 version 1148 is already newer than 1147 2024-09-16T15:14:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:3080: oid 47 version 1148 is already newer than 1145 2024-09-16T15:14:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1148 (ObjNum 1068 snap 318 seq_num 1068) dirty exists 2024-09-16T15:14:49.539 INFO:tasks.rados.rados.0.smithi162.stdout:3080: left oid 47 (ObjNum 1068 snap 318 seq_num 1068) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3074: done (6 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3075: done (5 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3077: done (4 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3078: done (3 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3079: done (2 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3080: done (1 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3081: done (0 left) 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3082: copy_from oid 13 from oid 8 current snap is 319 2024-09-16T15:14:49.918 INFO:tasks.rados.rados.0.smithi162.stdout:3083: write (excl) oid 7 current snap is 319 2024-09-16T15:14:49.919 INFO:tasks.rados.rados.0.smithi162.stdout:3083: seq_num 1069 ranges {766683=730601,1922192=583372,3156591=676370} 2024-09-16T15:14:49.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: writing smithi16234619-7 from 766683 to 1497284 tid 1 2024-09-16T15:14:49.931 INFO:tasks.rados.rados.0.smithi162.stdout:3083: writing smithi16234619-7 from 1922192 to 2505564 tid 2 2024-09-16T15:14:49.938 INFO:tasks.rados.rados.0.smithi162.stdout:3083: writing smithi16234619-7 from 3156591 to 3832961 tid 3 2024-09-16T15:14:49.938 INFO:tasks.rados.rados.0.smithi162.stdout:3084: snap_remove snap 252 2024-09-16T15:14:50.925 INFO:tasks.rados.rados.0.smithi162.stdout:3084: done (2 left) 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3085: delete oid 26 current snap is 319 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3082: finishing copy_from racing read to smithi16234619-13 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3082: finishing copy_from to smithi16234619-13 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 922 (ObjNum 957 snap 268 seq_num 957) dirty exists 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: finishing write tid 1 to smithi16234619-7 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: finishing write tid 2 to smithi16234619-7 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: finishing write tid 3 to smithi16234619-7 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: finishing write tid 4 to smithi16234619-7 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: finishing write tid 5 to smithi16234619-7 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: oid 7 updating version 0 to 1477 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: oid 7 version 1477 is already newer than 1475 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: oid 7 version 1477 is already newer than 1476 2024-09-16T15:14:50.926 INFO:tasks.rados.rados.0.smithi162.stdout:3083: oid 7 version 1477 is already newer than 1474 2024-09-16T15:14:50.927 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1477 (ObjNum 1069 snap 319 seq_num 1069) dirty exists 2024-09-16T15:14:50.927 INFO:tasks.rados.rados.0.smithi162.stdout:3083: left oid 7 (ObjNum 1069 snap 319 seq_num 1069) 2024-09-16T15:14:50.933 INFO:tasks.rados.rados.0.smithi162.stdout:3082: done (2 left) 2024-09-16T15:14:50.933 INFO:tasks.rados.rados.0.smithi162.stdout:3083: done (1 left) 2024-09-16T15:14:50.933 INFO:tasks.rados.rados.0.smithi162.stdout:3085: done (0 left) 2024-09-16T15:14:50.933 INFO:tasks.rados.rados.0.smithi162.stdout:3086: snap_create 2024-09-16T15:14:51.927 INFO:tasks.rados.rados.0.smithi162.stdout:3086: done (0 left) 2024-09-16T15:14:51.928 INFO:tasks.rados.rados.0.smithi162.stdout:3087: write (excl) oid 46 current snap is 320 2024-09-16T15:14:51.928 INFO:tasks.rados.rados.0.smithi162.stdout:3087: seq_num 1071 ranges {400297=481099,1639156=361141} 2024-09-16T15:14:51.932 INFO:tasks.rados.rados.0.smithi162.stdout:3087: writing smithi16234619-46 from 400297 to 881396 tid 1 2024-09-16T15:14:51.939 INFO:tasks.rados.rados.0.smithi162.stdout:3087: writing smithi16234619-46 from 1639156 to 2000297 tid 2 2024-09-16T15:14:51.939 INFO:tasks.rados.rados.0.smithi162.stdout:3088: write oid 47 current snap is 320 2024-09-16T15:14:51.939 INFO:tasks.rados.rados.0.smithi162.stdout:3088: seq_num 1072 ranges {417104=672701,1813991=203113} 2024-09-16T15:14:51.945 INFO:tasks.rados.rados.0.smithi162.stdout:3088: writing smithi16234619-47 from 417104 to 1089805 tid 1 2024-09-16T15:14:51.950 INFO:tasks.rados.rados.0.smithi162.stdout:3088: writing smithi16234619-47 from 1813991 to 2017104 tid 2 2024-09-16T15:14:51.950 INFO:tasks.rados.rados.0.smithi162.stdout:3089: read oid 3 snap -1 2024-09-16T15:14:51.950 INFO:tasks.rados.rados.0.smithi162.stdout:3089: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:51.950 INFO:tasks.rados.rados.0.smithi162.stdout:3090: copy_from oid 34 from oid 20 current snap is 320 2024-09-16T15:14:51.950 INFO:tasks.rados.rados.0.smithi162.stdout:3091: snap_remove snap 311 2024-09-16T15:14:52.890 INFO:tasks.rados.rados.0.smithi162.stdout:3091: done (4 left) 2024-09-16T15:14:52.890 INFO:tasks.rados.rados.0.smithi162.stdout:3092: rollback oid 25 current snap is 320 2024-09-16T15:14:52.890 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 25 to 279 2024-09-16T15:14:52.890 INFO:tasks.rados.rados.0.smithi162.stdout:3087: finishing write tid 1 to smithi16234619-46 2024-09-16T15:14:52.890 INFO:tasks.rados.rados.0.smithi162.stdout:3089: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: finishing write tid 2 to smithi16234619-46 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: finishing write tid 3 to smithi16234619-46 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3089: done (4 left) 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3093: delete oid 11 current snap is 320 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: finishing write tid 4 to smithi16234619-46 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: oid 46 updating version 0 to 1201 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: oid 46 updating version 1201 to 1202 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: oid 46 updating version 1202 to 1203 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1203 (ObjNum 1071 snap 320 seq_num 1071) dirty exists 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3087: left oid 46 (ObjNum 1071 snap 320 seq_num 1071) 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3088: finishing write tid 1 to smithi16234619-47 2024-09-16T15:14:52.980 INFO:tasks.rados.rados.0.smithi162.stdout:3088: finishing write tid 2 to smithi16234619-47 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: finishing write tid 3 to smithi16234619-47 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: finishing write tid 4 to smithi16234619-47 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: oid 47 updating version 0 to 1149 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: oid 47 updating version 1149 to 1151 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: oid 47 version 1151 is already newer than 1150 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1151 (ObjNum 1072 snap 320 seq_num 1072) dirty exists 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3088: left oid 47 (ObjNum 1072 snap 320 seq_num 1072) 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3090: finishing copy_from racing read to smithi16234619-34 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3090: finishing copy_from to smithi16234619-34 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1299 (ObjNum 1053 snap 312 seq_num 1053) dirty exists 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3092: finishing rollback tid 0 to smithi16234619-25 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3092: finishing rollback tid 1 to smithi16234619-25 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:3092: finishing rollback tid 2 to smithi16234619-25 2024-09-16T15:14:52.981 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 1431 (ObjNum 984 snap 278 seq_num 984) dirty exists 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3087: done (4 left) 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3088: done (3 left) 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3090: done (2 left) 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3092: done (1 left) 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3093: done (0 left) 2024-09-16T15:14:52.982 INFO:tasks.rados.rados.0.smithi162.stdout:3094: read oid 49 snap -1 2024-09-16T15:14:52.983 INFO:tasks.rados.rados.0.smithi162.stdout:3094: expect (ObjNum 1062 snap 317 seq_num 1062) 2024-09-16T15:14:52.983 INFO:tasks.rados.rados.0.smithi162.stdout:3095: snap_create 2024-09-16T15:14:52.996 INFO:tasks.rados.rados.0.smithi162.stdout:3094: expect (ObjNum 1062 snap 317 seq_num 1062) 2024-09-16T15:14:53.896 INFO:tasks.rados.rados.0.smithi162.stdout:3094: done (1 left) 2024-09-16T15:14:53.896 INFO:tasks.rados.rados.0.smithi162.stdout:3095: done (0 left) 2024-09-16T15:14:53.897 INFO:tasks.rados.rados.0.smithi162.stdout:3096: read oid 46 snap -1 2024-09-16T15:14:53.897 INFO:tasks.rados.rados.0.smithi162.stdout:3096: expect (ObjNum 1071 snap 320 seq_num 1071) 2024-09-16T15:14:53.897 INFO:tasks.rados.rados.0.smithi162.stdout:3097: read oid 39 snap -1 2024-09-16T15:14:53.897 INFO:tasks.rados.rados.0.smithi162.stdout:3097: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:14:53.897 INFO:tasks.rados.rados.0.smithi162.stdout:3098: delete oid 25 current snap is 321 2024-09-16T15:14:53.900 INFO:tasks.rados.rados.0.smithi162.stdout:3097: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:14:54.100 INFO:tasks.rados.rados.0.smithi162.stdout:3096: expect (ObjNum 1071 snap 320 seq_num 1071) 2024-09-16T15:14:54.201 INFO:tasks.rados.rados.0.smithi162.stdout:3096: done (2 left) 2024-09-16T15:14:54.201 INFO:tasks.rados.rados.0.smithi162.stdout:3097: done (1 left) 2024-09-16T15:14:54.201 INFO:tasks.rados.rados.0.smithi162.stdout:3098: done (0 left) 2024-09-16T15:14:54.202 INFO:tasks.rados.rados.0.smithi162.stdout:3099: write oid 46 current snap is 321 2024-09-16T15:14:54.202 INFO:tasks.rados.rados.0.smithi162.stdout:3099: seq_num 1075 ranges {467525=498448,1473084=594379} 2024-09-16T15:14:54.206 INFO:tasks.rados.rados.0.smithi162.stdout:3099: writing smithi16234619-46 from 467525 to 965973 tid 1 2024-09-16T15:14:54.212 INFO:tasks.rados.rados.0.smithi162.stdout:3099: writing smithi16234619-46 from 1473084 to 2067463 tid 2 2024-09-16T15:14:54.212 INFO:tasks.rados.rados.0.smithi162.stdout:3100: write oid 43 current snap is 321 2024-09-16T15:14:54.212 INFO:tasks.rados.rados.0.smithi162.stdout:3100: seq_num 1076 ranges {484332=573697,1531566=552766} 2024-09-16T15:14:54.217 INFO:tasks.rados.rados.0.smithi162.stdout:3100: writing smithi16234619-43 from 484332 to 1058029 tid 1 2024-09-16T15:14:54.224 INFO:tasks.rados.rados.0.smithi162.stdout:3100: writing smithi16234619-43 from 1531566 to 2084332 tid 2 2024-09-16T15:14:54.224 INFO:tasks.rados.rados.0.smithi162.stdout:3101: snap_remove snap 301 2024-09-16T15:14:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:3101: done (2 left) 2024-09-16T15:14:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:3102: read oid 19 snap 320 2024-09-16T15:14:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:3102: expect (ObjNum 1008 snap 290 seq_num 1008) 2024-09-16T15:14:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:3103: write (excl) oid 10 current snap is 321 2024-09-16T15:14:55.267 INFO:tasks.rados.rados.0.smithi162.stdout:3103: seq_num 1077 ranges {501139=648946,1873695=227444} 2024-09-16T15:14:55.273 INFO:tasks.rados.rados.0.smithi162.stdout:3103: writing smithi16234619-10 from 501139 to 1150085 tid 1 2024-09-16T15:14:55.279 INFO:tasks.rados.rados.0.smithi162.stdout:3103: writing smithi16234619-10 from 1873695 to 2101139 tid 2 2024-09-16T15:14:55.279 INFO:tasks.rados.rados.0.smithi162.stdout:3104: write (excl) oid 44 current snap is 321 2024-09-16T15:14:55.279 INFO:tasks.rados.rados.0.smithi162.stdout:3099: finishing write tid 1 to smithi16234619-46 2024-09-16T15:14:55.279 INFO:tasks.rados.rados.0.smithi162.stdout:3104: seq_num 1078 ranges {517946=724195,1932177=185769} 2024-09-16T15:14:55.286 INFO:tasks.rados.rados.0.smithi162.stdout:3104: writing smithi16234619-44 from 517946 to 1242141 tid 1 2024-09-16T15:14:55.291 INFO:tasks.rados.rados.0.smithi162.stdout:3104: writing smithi16234619-44 from 1932177 to 2117946 tid 2 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3105: read oid 27 snap -1 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3105: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3106: read oid 48 snap -1 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3106: expect deleted 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3107: delete oid 30 current snap is 321 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: finishing write tid 2 to smithi16234619-46 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: finishing write tid 3 to smithi16234619-46 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: finishing write tid 4 to smithi16234619-46 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: oid 46 updating version 0 to 1204 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: oid 46 updating version 1204 to 1206 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:3099: oid 46 version 1206 is already newer than 1205 2024-09-16T15:14:55.292 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1206 (ObjNum 1075 snap 321 seq_num 1075) dirty exists 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3099: left oid 46 (ObjNum 1075 snap 321 seq_num 1075) 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: finishing write tid 1 to smithi16234619-43 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: finishing write tid 2 to smithi16234619-43 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: finishing write tid 3 to smithi16234619-43 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: finishing write tid 4 to smithi16234619-43 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: oid 43 updating version 0 to 1153 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: oid 43 version 1153 is already newer than 1152 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: oid 43 updating version 1153 to 1154 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1154 (ObjNum 1076 snap 321 seq_num 1076) dirty exists 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3100: left oid 43 (ObjNum 1076 snap 321 seq_num 1076) 2024-09-16T15:14:55.293 INFO:tasks.rados.rados.0.smithi162.stdout:3102: expect (ObjNum 1008 snap 290 seq_num 1008) 2024-09-16T15:14:55.394 INFO:tasks.rados.rados.0.smithi162.stdout:3105: expect (ObjNum 655 snap 169 seq_num 655) 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: finishing write tid 1 to smithi16234619-10 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: finishing write tid 2 to smithi16234619-10 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: finishing write tid 3 to smithi16234619-10 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: finishing write tid 4 to smithi16234619-10 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: oid 10 updating version 0 to 1694 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: oid 10 updating version 1694 to 1695 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:3103: oid 10 updating version 1695 to 1696 2024-09-16T15:14:55.562 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1696 (ObjNum 1077 snap 321 seq_num 1077) dirty exists 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3103: left oid 10 (ObjNum 1077 snap 321 seq_num 1077) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: finishing write tid 1 to smithi16234619-44 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3099: done (7 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3100: done (6 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3102: done (5 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3103: done (4 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3105: done (3 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3106: done (2 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3107: done (1 left) 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3108: copy_from oid 47 from oid 31 current snap is 321 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: finishing write tid 2 to smithi16234619-44 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: finishing write tid 3 to smithi16234619-44 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: finishing write tid 4 to smithi16234619-44 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: oid 44 updating version 0 to 1363 2024-09-16T15:14:55.563 INFO:tasks.rados.rados.0.smithi162.stdout:3104: oid 44 version 1363 is already newer than 1362 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3104: oid 44 version 1363 is already newer than 1361 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1363 (ObjNum 1078 snap 321 seq_num 1078) dirty exists 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3104: left oid 44 (ObjNum 1078 snap 321 seq_num 1078) 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3104: done (1 left) 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3109: copy_from oid 45 from oid 12 current snap is 321 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3110: delete oid 24 current snap is 321 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3108: finishing copy_from to smithi16234619-47 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3108: got expected ENOENT (src dne) 2024-09-16T15:14:55.564 INFO:tasks.rados.rados.0.smithi162.stdout:3108: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:14:55.565 INFO:tasks.rados.rados.0.smithi162.stdout:3108: done (2 left) 2024-09-16T15:14:55.565 INFO:tasks.rados.rados.0.smithi162.stdout:3110: done (1 left) 2024-09-16T15:14:55.565 INFO:tasks.rados.rados.0.smithi162.stdout:3111: snap_create 2024-09-16T15:14:55.592 INFO:tasks.rados.rados.0.smithi162.stdout:3109: finishing copy_from racing read to smithi16234619-45 2024-09-16T15:14:55.596 INFO:tasks.rados.rados.0.smithi162.stdout:3109: finishing copy_from to smithi16234619-45 2024-09-16T15:14:55.596 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 1428 (ObjNum 1021 snap 297 seq_num 1021) dirty exists 2024-09-16T15:14:56.277 INFO:tasks.rados.rados.0.smithi162.stdout:3109: done (1 left) 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3111: done (0 left) 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3112: read oid 24 snap -1 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3112: expect deleted 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3113: read oid 3 snap -1 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3113: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3114: rollback oid 30 current snap is 322 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 279 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3115: read oid 15 snap -1 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3115: expect deleted 2024-09-16T15:14:56.278 INFO:tasks.rados.rados.0.smithi162.stdout:3116: delete oid 26 current snap is 322 2024-09-16T15:14:56.280 INFO:tasks.rados.rados.0.smithi162.stdout:3114: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:14:56.280 INFO:tasks.rados.rados.0.smithi162.stdout:3114: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:14:56.280 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1177 (ObjNum 952 snap 268 seq_num 952) dirty exists 2024-09-16T15:14:56.280 INFO:tasks.rados.rados.0.smithi162.stdout:3113: expect (ObjNum 984 snap 278 seq_num 984) 2024-09-16T15:14:56.359 INFO:tasks.rados.rados.0.smithi162.stdout:3112: done (4 left) 2024-09-16T15:14:56.359 INFO:tasks.rados.rados.0.smithi162.stdout:3113: done (3 left) 2024-09-16T15:14:56.359 INFO:tasks.rados.rados.0.smithi162.stdout:3114: done (2 left) 2024-09-16T15:14:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:3115: done (1 left) 2024-09-16T15:14:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:3116: done (0 left) 2024-09-16T15:14:56.360 INFO:tasks.rados.rados.0.smithi162.stdout:3117: delete oid 15 current snap is 322 2024-09-16T15:14:56.361 INFO:tasks.rados.rados.0.smithi162.stdout:3117: done (0 left) 2024-09-16T15:14:56.361 INFO:tasks.rados.rados.0.smithi162.stdout:3118: snap_create 2024-09-16T15:14:57.282 INFO:tasks.rados.rados.0.smithi162.stdout:3118: done (0 left) 2024-09-16T15:14:57.283 INFO:tasks.rados.rados.0.smithi162.stdout:3119: read oid 15 snap -1 2024-09-16T15:14:57.283 INFO:tasks.rados.rados.0.smithi162.stdout:3119: expect deleted 2024-09-16T15:14:57.283 INFO:tasks.rados.rados.0.smithi162.stdout:3120: delete oid 8 current snap is 323 2024-09-16T15:14:57.285 INFO:tasks.rados.rados.0.smithi162.stdout:3119: done (1 left) 2024-09-16T15:14:57.286 INFO:tasks.rados.rados.0.smithi162.stdout:3120: done (0 left) 2024-09-16T15:14:57.286 INFO:tasks.rados.rados.0.smithi162.stdout:3121: snap_create 2024-09-16T15:14:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:3121: done (0 left) 2024-09-16T15:14:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:3122: rollback oid 18 current snap is 324 2024-09-16T15:14:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 18 to 284 2024-09-16T15:14:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:3123: write (excl) oid 19 current snap is 324 2024-09-16T15:14:57.940 INFO:tasks.rados.rados.0.smithi162.stdout:3123: seq_num 1084 ranges {618788=492042,1883069=335719} 2024-09-16T15:14:57.946 INFO:tasks.rados.rados.0.smithi162.stdout:3123: writing smithi16234619-19 from 618788 to 1110830 tid 1 2024-09-16T15:14:57.953 INFO:tasks.rados.rados.0.smithi162.stdout:3123: writing smithi16234619-19 from 1883069 to 2218788 tid 2 2024-09-16T15:14:57.953 INFO:tasks.rados.rados.0.smithi162.stdout:3124: delete oid 16 current snap is 324 2024-09-16T15:14:57.953 INFO:tasks.rados.rados.0.smithi162.stdout:3122: finishing rollback tid 1 to smithi16234619-18 2024-09-16T15:14:57.953 INFO:tasks.rados.rados.0.smithi162.stdout:3122: finishing rollback tid 2 to smithi16234619-18 2024-09-16T15:14:57.953 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1480 (ObjNum 983 snap 278 seq_num 983) dirty exists 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3122: done (2 left) 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3124: done (1 left) 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3125: read oid 13 snap -1 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3125: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3126: write (excl) oid 29 current snap is 324 2024-09-16T15:14:57.955 INFO:tasks.rados.rados.0.smithi162.stdout:3126: seq_num 1086 ranges {652402=642540,1883680=368722} 2024-09-16T15:14:57.962 INFO:tasks.rados.rados.0.smithi162.stdout:3126: writing smithi16234619-29 from 652402 to 1294942 tid 1 2024-09-16T15:14:57.968 INFO:tasks.rados.rados.0.smithi162.stdout:3126: writing smithi16234619-29 from 1883680 to 2252402 tid 2 2024-09-16T15:14:57.968 INFO:tasks.rados.rados.0.smithi162.stdout:3127: rollback oid 50 current snap is 324 2024-09-16T15:14:57.968 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 249 2024-09-16T15:14:57.968 INFO:tasks.rados.rados.0.smithi162.stdout:3125: expect (ObjNum 957 snap 268 seq_num 957) 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: finishing write tid 1 to smithi16234619-19 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: finishing write tid 2 to smithi16234619-19 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: finishing write tid 3 to smithi16234619-19 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: finishing write tid 4 to smithi16234619-19 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: oid 19 updating version 0 to 1444 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: oid 19 updating version 1444 to 1445 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: oid 19 version 1445 is already newer than 1443 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1445 (ObjNum 1084 snap 324 seq_num 1084) dirty exists 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: left oid 19 (ObjNum 1084 snap 324 seq_num 1084) 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3127: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3123: done (3 left) 2024-09-16T15:14:58.056 INFO:tasks.rados.rados.0.smithi162.stdout:3125: done (2 left) 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3128: snap_create 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3127: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3127: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1449 (ObjNum 811 snap 218 seq_num 811) dirty exists 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: finishing write tid 1 to smithi16234619-29 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: finishing write tid 2 to smithi16234619-29 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: finishing write tid 3 to smithi16234619-29 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: finishing write tid 4 to smithi16234619-29 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: oid 29 updating version 0 to 1613 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: oid 29 updating version 1613 to 1614 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: oid 29 updating version 1614 to 1615 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1615 (ObjNum 1086 snap 324 seq_num 1086) dirty exists 2024-09-16T15:14:58.057 INFO:tasks.rados.rados.0.smithi162.stdout:3126: left oid 29 (ObjNum 1086 snap 324 seq_num 1086) 2024-09-16T15:14:58.948 INFO:tasks.rados.rados.0.smithi162.stdout:3126: done (2 left) 2024-09-16T15:14:58.949 INFO:tasks.rados.rados.0.smithi162.stdout:3127: done (1 left) 2024-09-16T15:14:58.949 INFO:tasks.rados.rados.0.smithi162.stdout:3128: done (0 left) 2024-09-16T15:14:58.949 INFO:tasks.rados.rados.0.smithi162.stdout:3129: snap_remove snap 273 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3129: done (0 left) 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3130: copy_from oid 30 from oid 36 current snap is 325 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3131: read oid 7 snap -1 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3131: expect (ObjNum 1069 snap 319 seq_num 1069) 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3132: copy_from oid 1 from oid 17 current snap is 325 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3133: write (excl) oid 19 current snap is 325 2024-09-16T15:14:59.951 INFO:tasks.rados.rados.0.smithi162.stdout:3133: seq_num 1087 ranges {669209=717789,1942162=327047} 2024-09-16T15:14:59.959 INFO:tasks.rados.rados.0.smithi162.stdout:3133: writing smithi16234619-19 from 669209 to 1386998 tid 1 2024-09-16T15:14:59.965 INFO:tasks.rados.rados.0.smithi162.stdout:3133: writing smithi16234619-19 from 1942162 to 2269209 tid 2 2024-09-16T15:14:59.965 INFO:tasks.rados.rados.0.smithi162.stdout:3134: snap_create 2024-09-16T15:14:59.980 INFO:tasks.rados.rados.0.smithi162.stdout:3131: expect (ObjNum 1069 snap 319 seq_num 1069) 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3132: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: finishing write tid 1 to smithi16234619-19 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: finishing write tid 2 to smithi16234619-19 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: finishing write tid 3 to smithi16234619-19 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: finishing write tid 4 to smithi16234619-19 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: oid 19 updating version 0 to 1456 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: oid 19 version 1456 is already newer than 1455 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: oid 19 updating version 1456 to 1457 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1457 (ObjNum 1087 snap 325 seq_num 1087) dirty exists 2024-09-16T15:15:00.106 INFO:tasks.rados.rados.0.smithi162.stdout:3133: left oid 19 (ObjNum 1087 snap 325 seq_num 1087) 2024-09-16T15:15:00.107 INFO:tasks.rados.rados.0.smithi162.stdout:3132: finishing copy_from to smithi16234619-1 2024-09-16T15:15:00.107 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1670 (ObjNum 1065 snap 318 seq_num 1065) dirty exists 2024-09-16T15:15:00.107 INFO:tasks.rados.rados.0.smithi162.stdout:3130: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:15:00.107 INFO:tasks.rados.rados.0.smithi162.stdout:3130: finishing copy_from to smithi16234619-30 2024-09-16T15:15:00.107 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1405 (ObjNum 1061 snap 317 seq_num 1061) dirty exists 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3130: done (4 left) 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3131: done (3 left) 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3132: done (2 left) 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3133: done (1 left) 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3134: done (0 left) 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3135: copy_from oid 38 from oid 23 current snap is 326 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3136: write (excl) oid 26 current snap is 326 2024-09-16T15:15:00.958 INFO:tasks.rados.rados.0.smithi162.stdout:3136: seq_num 1088 ranges {686016=509391,1716997=569019} 2024-09-16T15:15:00.963 INFO:tasks.rados.rados.0.smithi162.stdout:3136: writing smithi16234619-26 from 686016 to 1195407 tid 1 2024-09-16T15:15:00.971 INFO:tasks.rados.rados.0.smithi162.stdout:3136: writing smithi16234619-26 from 1716997 to 2286016 tid 2 2024-09-16T15:15:00.971 INFO:tasks.rados.rados.0.smithi162.stdout:3137: write (excl) oid 15 current snap is 326 2024-09-16T15:15:00.971 INFO:tasks.rados.rados.0.smithi162.stdout:3137: seq_num 1089 ranges {702823=584640,2059126=243697} 2024-09-16T15:15:00.979 INFO:tasks.rados.rados.0.smithi162.stdout:3137: writing smithi16234619-15 from 702823 to 1287463 tid 1 2024-09-16T15:15:00.985 INFO:tasks.rados.rados.0.smithi162.stdout:3137: writing smithi16234619-15 from 2059126 to 2302823 tid 2 2024-09-16T15:15:00.985 INFO:tasks.rados.rados.0.smithi162.stdout:3138: delete oid 8 current snap is 326 2024-09-16T15:15:00.987 INFO:tasks.rados.rados.0.smithi162.stdout:3138: done (3 left) 2024-09-16T15:15:00.987 INFO:tasks.rados.rados.0.smithi162.stdout:3139: delete oid 17 current snap is 326 2024-09-16T15:15:01.003 INFO:tasks.rados.rados.0.smithi162.stdout:3136: finishing write tid 1 to smithi16234619-26 2024-09-16T15:15:01.006 INFO:tasks.rados.rados.0.smithi162.stdout:3137: finishing write tid 1 to smithi16234619-15 2024-09-16T15:15:01.006 INFO:tasks.rados.rados.0.smithi162.stdout:3137: finishing write tid 2 to smithi16234619-15 2024-09-16T15:15:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:3137: finishing write tid 3 to smithi16234619-15 2024-09-16T15:15:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:3137: finishing write tid 4 to smithi16234619-15 2024-09-16T15:15:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:3137: oid 15 updating version 0 to 1408 2024-09-16T15:15:01.007 INFO:tasks.rados.rados.0.smithi162.stdout:3137: oid 15 version 1408 is already newer than 1406 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3137: oid 15 version 1408 is already newer than 1407 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 1408 (ObjNum 1089 snap 326 seq_num 1089) dirty exists 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3137: left oid 15 (ObjNum 1089 snap 326 seq_num 1089) 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: finishing write tid 2 to smithi16234619-26 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: finishing write tid 3 to smithi16234619-26 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: finishing write tid 4 to smithi16234619-26 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: oid 26 updating version 0 to 1723 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: oid 26 updating version 1723 to 1724 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: oid 26 version 1724 is already newer than 1722 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1724 (ObjNum 1088 snap 326 seq_num 1088) dirty exists 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: left oid 26 (ObjNum 1088 snap 326 seq_num 1088) 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3136: done (3 left) 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3137: done (2 left) 2024-09-16T15:15:01.008 INFO:tasks.rados.rados.0.smithi162.stdout:3139: done (1 left) 2024-09-16T15:15:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:3140: write (excl) oid 9 current snap is 326 2024-09-16T15:15:01.009 INFO:tasks.rados.rados.0.smithi162.stdout:3140: seq_num 1092 ranges {753244=410387,1834572=518672} 2024-09-16T15:15:01.014 INFO:tasks.rados.rados.0.smithi162.stdout:3140: writing smithi16234619-9 from 753244 to 1163631 tid 1 2024-09-16T15:15:01.022 INFO:tasks.rados.rados.0.smithi162.stdout:3140: writing smithi16234619-9 from 1834572 to 2353244 tid 2 2024-09-16T15:15:01.022 INFO:tasks.rados.rados.0.smithi162.stdout:3141: snap_remove snap 298 2024-09-16T15:15:01.968 INFO:tasks.rados.rados.0.smithi162.stdout:3141: done (2 left) 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3142: write oid 29 current snap is 326 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3135: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3135: finishing copy_from to smithi16234619-38 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 611 (ObjNum 1063 snap 318 seq_num 1063) dirty exists 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: finishing write tid 1 to smithi16234619-9 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: finishing write tid 2 to smithi16234619-9 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: finishing write tid 3 to smithi16234619-9 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: finishing write tid 4 to smithi16234619-9 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: oid 9 updating version 0 to 1671 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: oid 9 updating version 1671 to 1673 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: oid 9 version 1673 is already newer than 1672 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1673 (ObjNum 1092 snap 326 seq_num 1092) dirty exists 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3140: left oid 9 (ObjNum 1092 snap 326 seq_num 1092) 2024-09-16T15:15:01.969 INFO:tasks.rados.rados.0.smithi162.stdout:3142: seq_num 1093 ranges {770051=485636,1776701=511047} 2024-09-16T15:15:01.975 INFO:tasks.rados.rados.0.smithi162.stdout:3142: writing smithi16234619-29 from 770051 to 1255687 tid 1 2024-09-16T15:15:01.981 INFO:tasks.rados.rados.0.smithi162.stdout:3142: writing smithi16234619-29 from 1776701 to 2287748 tid 2 2024-09-16T15:15:01.981 INFO:tasks.rados.rados.0.smithi162.stdout:3135: done (2 left) 2024-09-16T15:15:01.981 INFO:tasks.rados.rados.0.smithi162.stdout:3140: done (1 left) 2024-09-16T15:15:01.981 INFO:tasks.rados.rados.0.smithi162.stdout:3143: write oid 12 current snap is 326 2024-09-16T15:15:01.981 INFO:tasks.rados.rados.0.smithi162.stdout:3143: seq_num 1094 ranges {786858=560885,1835183=551675} 2024-09-16T15:15:01.989 INFO:tasks.rados.rados.0.smithi162.stdout:3143: writing smithi16234619-12 from 786858 to 1347743 tid 1 2024-09-16T15:15:01.994 INFO:tasks.rados.rados.0.smithi162.stdout:3143: writing smithi16234619-12 from 1835183 to 2386858 tid 2 2024-09-16T15:15:01.994 INFO:tasks.rados.rados.0.smithi162.stdout:3144: snap_remove snap 325 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: finishing write tid 1 to smithi16234619-29 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: finishing write tid 2 to smithi16234619-29 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: finishing write tid 3 to smithi16234619-29 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: finishing write tid 4 to smithi16234619-29 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: oid 29 updating version 0 to 1631 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: oid 29 updating version 1631 to 1632 2024-09-16T15:15:02.893 INFO:tasks.rados.rados.0.smithi162.stdout:3142: oid 29 updating version 1632 to 1633 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1633 (ObjNum 1093 snap 326 seq_num 1093) dirty exists 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3142: left oid 29 (ObjNum 1093 snap 326 seq_num 1093) 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: finishing write tid 1 to smithi16234619-12 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: finishing write tid 2 to smithi16234619-12 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: finishing write tid 3 to smithi16234619-12 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: finishing write tid 4 to smithi16234619-12 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: oid 12 updating version 0 to 1446 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: oid 12 updating version 1446 to 1447 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: oid 12 updating version 1447 to 1448 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1448 (ObjNum 1094 snap 326 seq_num 1094) dirty exists 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: left oid 12 (ObjNum 1094 snap 326 seq_num 1094) 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3142: done (2 left) 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3143: done (1 left) 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3144: done (0 left) 2024-09-16T15:15:02.894 INFO:tasks.rados.rados.0.smithi162.stdout:3145: snap_create 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3145: done (0 left) 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3146: rollback oid 47 current snap is 327 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 317 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3147: rollback oid 19 current snap is 327 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 326 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3148: read oid 48 snap -1 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3148: expect deleted 2024-09-16T15:15:03.895 INFO:tasks.rados.rados.0.smithi162.stdout:3149: snap_remove snap 313 2024-09-16T15:15:04.901 INFO:tasks.rados.rados.0.smithi162.stdout:3149: done (3 left) 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3150: rollback oid 50 current snap is 327 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 50 to 279 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3146: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3146: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3147: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3147: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3147: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1473 (ObjNum 1087 snap 325 seq_num 1087) dirty exists 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3146: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1157 (ObjNum 1058 snap 316 seq_num 1058) dirty exists 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3146: done (3 left) 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3147: done (2 left) 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3148: done (1 left) 2024-09-16T15:15:04.902 INFO:tasks.rados.rados.0.smithi162.stdout:3151: read oid 19 snap 326 2024-09-16T15:15:04.903 INFO:tasks.rados.rados.0.smithi162.stdout:3151: expect (ObjNum 1087 snap 325 seq_num 1087) 2024-09-16T15:15:04.903 INFO:tasks.rados.rados.0.smithi162.stdout:3152: snap_create 2024-09-16T15:15:04.918 INFO:tasks.rados.rados.0.smithi162.stdout:3150: finishing rollback tid 0 to smithi16234619-50 2024-09-16T15:15:04.918 INFO:tasks.rados.rados.0.smithi162.stdout:3150: finishing rollback tid 1 to smithi16234619-50 2024-09-16T15:15:04.919 INFO:tasks.rados.rados.0.smithi162.stdout:3150: finishing rollback tid 2 to smithi16234619-50 2024-09-16T15:15:04.919 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1477 (ObjNum 981 snap 276 seq_num 981) dirty exists 2024-09-16T15:15:04.922 INFO:tasks.rados.rados.0.smithi162.stdout:3151: expect (ObjNum 1087 snap 325 seq_num 1087) 2024-09-16T15:15:05.903 INFO:tasks.rados.rados.0.smithi162.stdout:3150: done (2 left) 2024-09-16T15:15:05.903 INFO:tasks.rados.rados.0.smithi162.stdout:3151: done (1 left) 2024-09-16T15:15:05.903 INFO:tasks.rados.rados.0.smithi162.stdout:3152: done (0 left) 2024-09-16T15:15:05.903 INFO:tasks.rados.rados.0.smithi162.stdout:3153: rollback oid 43 current snap is 328 2024-09-16T15:15:05.903 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 284 2024-09-16T15:15:05.904 INFO:tasks.rados.rados.0.smithi162.stdout:3154: snap_remove snap 279 2024-09-16T15:15:06.906 INFO:tasks.rados.rados.0.smithi162.stdout:3154: done (1 left) 2024-09-16T15:15:06.906 INFO:tasks.rados.rados.0.smithi162.stdout:3155: snap_remove snap 320 2024-09-16T15:15:07.904 INFO:tasks.rados.rados.0.smithi162.stdout:3155: done (1 left) 2024-09-16T15:15:07.904 INFO:tasks.rados.rados.0.smithi162.stdout:3156: write (excl) oid 44 current snap is 328 2024-09-16T15:15:07.904 INFO:tasks.rados.rados.0.smithi162.stdout:3156: seq_num 1095 ranges {403665=752487,1610018=514716} 2024-09-16T15:15:07.911 INFO:tasks.rados.rados.0.smithi162.stdout:3156: writing smithi16234619-44 from 403665 to 1156152 tid 1 2024-09-16T15:15:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:3156: writing smithi16234619-44 from 1610018 to 2124734 tid 2 2024-09-16T15:15:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:3157: read oid 47 snap -1 2024-09-16T15:15:07.917 INFO:tasks.rados.rados.0.smithi162.stdout:3157: expect (ObjNum 1058 snap 316 seq_num 1058) 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3158: read oid 46 snap -1 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3158: expect (ObjNum 1075 snap 321 seq_num 1075) 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3159: copy_from oid 49 from oid 50 current snap is 328 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3153: finishing rollback tid 0 to smithi16234619-43 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3153: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3153: finishing rollback tid 2 to smithi16234619-43 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1160 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3153: done (4 left) 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3160: write (excl) oid 32 current snap is 328 2024-09-16T15:15:07.918 INFO:tasks.rados.rados.0.smithi162.stdout:3160: seq_num 1096 ranges {420472=427736,1268500=658374,2347953=72519} 2024-09-16T15:15:07.922 INFO:tasks.rados.rados.0.smithi162.stdout:3160: writing smithi16234619-32 from 420472 to 848208 tid 1 2024-09-16T15:15:07.929 INFO:tasks.rados.rados.0.smithi162.stdout:3160: writing smithi16234619-32 from 1268500 to 1926874 tid 2 2024-09-16T15:15:07.932 INFO:tasks.rados.rados.0.smithi162.stdout:3160: writing smithi16234619-32 from 2347953 to 2420472 tid 3 2024-09-16T15:15:07.932 INFO:tasks.rados.rados.0.smithi162.stdout:3161: snap_remove snap 324 2024-09-16T15:15:08.920 INFO:tasks.rados.rados.0.smithi162.stdout:3161: done (5 left) 2024-09-16T15:15:08.920 INFO:tasks.rados.rados.0.smithi162.stdout:3162: read oid 29 snap 321 2024-09-16T15:15:08.920 INFO:tasks.rados.rados.0.smithi162.stdout:3162: expect deleted 2024-09-16T15:15:08.920 INFO:tasks.rados.rados.0.smithi162.stdout:3157: expect (ObjNum 1058 snap 316 seq_num 1058) 2024-09-16T15:15:09.088 INFO:tasks.rados.rados.0.smithi162.stdout:3158: expect (ObjNum 1075 snap 321 seq_num 1075) 2024-09-16T15:15:09.219 INFO:tasks.rados.rados.0.smithi162.stdout:3156: finishing write tid 1 to smithi16234619-44 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: finishing write tid 2 to smithi16234619-44 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: finishing write tid 3 to smithi16234619-44 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: finishing write tid 4 to smithi16234619-44 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: oid 44 updating version 0 to 1365 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: oid 44 version 1365 is already newer than 1364 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: oid 44 updating version 1365 to 1366 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1366 (ObjNum 1095 snap 328 seq_num 1095) dirty exists 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3156: left oid 44 (ObjNum 1095 snap 328 seq_num 1095) 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3160: finishing write tid 1 to smithi16234619-32 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3160: finishing write tid 2 to smithi16234619-32 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3160: finishing write tid 3 to smithi16234619-32 2024-09-16T15:15:09.220 INFO:tasks.rados.rados.0.smithi162.stdout:3160: finishing write tid 4 to smithi16234619-32 2024-09-16T15:15:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:3156: done (5 left) 2024-09-16T15:15:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:3157: done (4 left) 2024-09-16T15:15:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:3158: done (3 left) 2024-09-16T15:15:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:3163: write oid 2 current snap is 328 2024-09-16T15:15:09.221 INFO:tasks.rados.rados.0.smithi162.stdout:3163: seq_num 1097 ranges {437279=502985,1610629=518385} 2024-09-16T15:15:09.224 INFO:tasks.rados.rados.0.smithi162.stdout:3163: writing smithi16234619-2 from 437279 to 940264 tid 1 2024-09-16T15:15:09.231 INFO:tasks.rados.rados.0.smithi162.stdout:3163: writing smithi16234619-2 from 1610629 to 2129014 tid 2 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3164: rollback oid 16 current snap is 328 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 317 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: finishing write tid 5 to smithi16234619-32 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: oid 32 updating version 0 to 1452 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: oid 32 version 1452 is already newer than 1451 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: oid 32 version 1452 is already newer than 1450 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: oid 32 version 1452 is already newer than 1449 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1452 (ObjNum 1096 snap 328 seq_num 1096) dirty exists 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: left oid 32 (ObjNum 1096 snap 328 seq_num 1096) 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3160: done (4 left) 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3165: read oid 7 snap -1 2024-09-16T15:15:09.232 INFO:tasks.rados.rados.0.smithi162.stdout:3165: expect (ObjNum 1069 snap 319 seq_num 1069) 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3166: rollback oid 19 current snap is 328 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 19 to 307 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3159: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3159: finishing copy_from to smithi16234619-49 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 1477 (ObjNum 981 snap 276 seq_num 981) dirty exists 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3159: done (5 left) 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3162: done (4 left) 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3167: write oid 3 current snap is 328 2024-09-16T15:15:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3167: seq_num 1098 ranges {454086=578234,1669111=662043} 2024-09-16T15:15:09.236 INFO:tasks.rados.rados.0.smithi162.stdout:3167: writing smithi16234619-3 from 454086 to 1032320 tid 1 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3167: writing smithi16234619-3 from 1669111 to 2331154 tid 2 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3168: delete oid 48 current snap is 328 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3164: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1353 (ObjNum 145 snap 0 seq_num 865856608) dirty dne 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3166: finishing rollback tid 0 to smithi16234619-19 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3166: finishing rollback tid 1 to smithi16234619-19 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:3166: finishing rollback tid 2 to smithi16234619-19 2024-09-16T15:15:09.245 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1507 (ObjNum 1008 snap 290 seq_num 1008) dirty exists 2024-09-16T15:15:09.246 INFO:tasks.rados.rados.0.smithi162.stdout:3165: expect (ObjNum 1069 snap 319 seq_num 1069) 2024-09-16T15:15:09.372 INFO:tasks.rados.rados.0.smithi162.stdout:3163: finishing write tid 1 to smithi16234619-2 2024-09-16T15:15:09.372 INFO:tasks.rados.rados.0.smithi162.stdout:3163: finishing write tid 2 to smithi16234619-2 2024-09-16T15:15:09.372 INFO:tasks.rados.rados.0.smithi162.stdout:3163: finishing write tid 3 to smithi16234619-2 2024-09-16T15:15:09.372 INFO:tasks.rados.rados.0.smithi162.stdout:3163: finishing write tid 4 to smithi16234619-2 2024-09-16T15:15:09.372 INFO:tasks.rados.rados.0.smithi162.stdout:3163: oid 2 updating version 0 to 1776 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3163: oid 2 version 1776 is already newer than 1774 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3163: oid 2 version 1776 is already newer than 1775 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 1776 (ObjNum 1097 snap 328 seq_num 1097) dirty exists 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3163: left oid 2 (ObjNum 1097 snap 328 seq_num 1097) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3167: finishing write tid 1 to smithi16234619-3 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3163: done (5 left) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3164: done (4 left) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3165: done (3 left) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3166: done (2 left) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3168: done (1 left) 2024-09-16T15:15:09.373 INFO:tasks.rados.rados.0.smithi162.stdout:3169: snap_remove snap 317 2024-09-16T15:15:09.926 INFO:tasks.rados.rados.0.smithi162.stdout:3169: done (1 left) 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3170: read oid 30 snap -1 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3170: expect (ObjNum 1061 snap 317 seq_num 1061) 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3171: rollback oid 46 current snap is 328 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 276 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: finishing write tid 2 to smithi16234619-3 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: finishing write tid 3 to smithi16234619-3 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: finishing write tid 4 to smithi16234619-3 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: oid 3 updating version 0 to 1299 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: oid 3 version 1299 is already newer than 1297 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: oid 3 version 1299 is already newer than 1298 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1299 (ObjNum 1098 snap 328 seq_num 1098) dirty exists 2024-09-16T15:15:09.927 INFO:tasks.rados.rados.0.smithi162.stdout:3167: left oid 3 (ObjNum 1098 snap 328 seq_num 1098) 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3167: done (2 left) 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3172: read oid 10 snap -1 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3172: expect (ObjNum 1077 snap 321 seq_num 1077) 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3173: read oid 50 snap -1 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3173: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3174: read oid 22 snap 286 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3174: expect deleted 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3175: read oid 23 snap -1 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3175: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3176: write (excl) oid 26 current snap is 328 2024-09-16T15:15:09.928 INFO:tasks.rados.rados.0.smithi162.stdout:3176: seq_num 1100 ranges {487700=728732,1786075=665712} 2024-09-16T15:15:09.933 INFO:tasks.rados.rados.0.smithi162.stdout:3176: writing smithi16234619-26 from 487700 to 1216432 tid 1 2024-09-16T15:15:09.940 INFO:tasks.rados.rados.0.smithi162.stdout:3176: writing smithi16234619-26 from 1786075 to 2451787 tid 2 2024-09-16T15:15:09.940 INFO:tasks.rados.rados.0.smithi162.stdout:3177: write (excl) oid 43 current snap is 328 2024-09-16T15:15:09.941 INFO:tasks.rados.rados.0.smithi162.stdout:3177: seq_num 1101 ranges {504507=403981,1328204=409370,2252362=252145} 2024-09-16T15:15:09.945 INFO:tasks.rados.rados.0.smithi162.stdout:3177: writing smithi16234619-43 from 504507 to 908488 tid 1 2024-09-16T15:15:09.950 INFO:tasks.rados.rados.0.smithi162.stdout:3177: writing smithi16234619-43 from 1328204 to 1737574 tid 2 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3177: writing smithi16234619-43 from 2252362 to 2504507 tid 3 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3178: read oid 8 snap -1 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3178: expect deleted 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3179: snap_create 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3171: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3171: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3171: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1209 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:15:09.955 INFO:tasks.rados.rados.0.smithi162.stdout:3172: expect (ObjNum 1077 snap 321 seq_num 1077) 2024-09-16T15:15:10.072 INFO:tasks.rados.rados.0.smithi162.stdout:3173: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:15:10.248 INFO:tasks.rados.rados.0.smithi162.stdout:3170: expect (ObjNum 1061 snap 317 seq_num 1061) 2024-09-16T15:15:10.420 INFO:tasks.rados.rados.0.smithi162.stdout:3175: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:10.612 INFO:tasks.rados.rados.0.smithi162.stdout:3176: finishing write tid 1 to smithi16234619-26 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: finishing write tid 1 to smithi16234619-43 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: finishing write tid 2 to smithi16234619-43 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: finishing write tid 3 to smithi16234619-43 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: finishing write tid 4 to smithi16234619-43 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: finishing write tid 5 to smithi16234619-43 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: oid 43 updating version 0 to 1161 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: oid 43 updating version 1161 to 1162 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: oid 43 updating version 1162 to 1163 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: oid 43 updating version 1163 to 1164 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1164 (ObjNum 1101 snap 328 seq_num 1101) dirty exists 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3177: left oid 43 (ObjNum 1101 snap 328 seq_num 1101) 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3176: finishing write tid 2 to smithi16234619-26 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3176: finishing write tid 3 to smithi16234619-26 2024-09-16T15:15:10.613 INFO:tasks.rados.rados.0.smithi162.stdout:3176: finishing write tid 4 to smithi16234619-26 2024-09-16T15:15:10.614 INFO:tasks.rados.rados.0.smithi162.stdout:3176: oid 26 updating version 0 to 1783 2024-09-16T15:15:10.614 INFO:tasks.rados.rados.0.smithi162.stdout:3176: oid 26 updating version 1783 to 1784 2024-09-16T15:15:10.614 INFO:tasks.rados.rados.0.smithi162.stdout:3176: oid 26 version 1784 is already newer than 1782 2024-09-16T15:15:10.614 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1784 (ObjNum 1100 snap 328 seq_num 1100) dirty exists 2024-09-16T15:15:10.614 INFO:tasks.rados.rados.0.smithi162.stdout:3176: left oid 26 (ObjNum 1100 snap 328 seq_num 1100) 2024-09-16T15:15:10.932 INFO:tasks.rados.rados.0.smithi162.stdout:3170: done (9 left) 2024-09-16T15:15:10.932 INFO:tasks.rados.rados.0.smithi162.stdout:3171: done (8 left) 2024-09-16T15:15:10.932 INFO:tasks.rados.rados.0.smithi162.stdout:3172: done (7 left) 2024-09-16T15:15:10.932 INFO:tasks.rados.rados.0.smithi162.stdout:3173: done (6 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3174: done (5 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3175: done (4 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3176: done (3 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3177: done (2 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3178: done (1 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3179: done (0 left) 2024-09-16T15:15:10.933 INFO:tasks.rados.rados.0.smithi162.stdout:3180: delete oid 23 current snap is 329 2024-09-16T15:15:10.936 INFO:tasks.rados.rados.0.smithi162.stdout:3180: done (0 left) 2024-09-16T15:15:10.936 INFO:tasks.rados.rados.0.smithi162.stdout:3181: read oid 18 snap -1 2024-09-16T15:15:10.937 INFO:tasks.rados.rados.0.smithi162.stdout:3181: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:15:10.937 INFO:tasks.rados.rados.0.smithi162.stdout:3182: write oid 12 current snap is 329 2024-09-16T15:15:10.937 INFO:tasks.rados.rados.0.smithi162.stdout:3182: seq_num 1103 ranges {538121=670832,1961521=413039} 2024-09-16T15:15:10.943 INFO:tasks.rados.rados.0.smithi162.stdout:3182: writing smithi16234619-12 from 538121 to 1208953 tid 1 2024-09-16T15:15:10.949 INFO:tasks.rados.rados.0.smithi162.stdout:3182: writing smithi16234619-12 from 1961521 to 2374560 tid 2 2024-09-16T15:15:10.949 INFO:tasks.rados.rados.0.smithi162.stdout:3183: write oid 1 current snap is 329 2024-09-16T15:15:10.949 INFO:tasks.rados.rados.0.smithi162.stdout:3183: seq_num 1104 ranges {554928=746081,2020003=534925} 2024-09-16T15:15:10.956 INFO:tasks.rados.rados.0.smithi162.stdout:3183: writing smithi16234619-1 from 554928 to 1301009 tid 1 2024-09-16T15:15:10.963 INFO:tasks.rados.rados.0.smithi162.stdout:3183: writing smithi16234619-1 from 2020003 to 2554928 tid 2 2024-09-16T15:15:10.963 INFO:tasks.rados.rados.0.smithi162.stdout:3184: snap_create 2024-09-16T15:15:10.963 INFO:tasks.rados.rados.0.smithi162.stdout:3181: expect (ObjNum 983 snap 278 seq_num 983) 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: finishing write tid 1 to smithi16234619-12 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: finishing write tid 2 to smithi16234619-12 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: finishing write tid 3 to smithi16234619-12 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: finishing write tid 4 to smithi16234619-12 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: oid 12 updating version 0 to 1453 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: oid 12 updating version 1453 to 1454 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: oid 12 updating version 1454 to 1455 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1455 (ObjNum 1103 snap 329 seq_num 1103) dirty exists 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3182: left oid 12 (ObjNum 1103 snap 329 seq_num 1103) 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3183: finishing write tid 1 to smithi16234619-1 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3183: finishing write tid 2 to smithi16234619-1 2024-09-16T15:15:11.043 INFO:tasks.rados.rados.0.smithi162.stdout:3183: finishing write tid 3 to smithi16234619-1 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:3183: finishing write tid 4 to smithi16234619-1 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:3183: oid 1 updating version 0 to 1676 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:3183: oid 1 version 1676 is already newer than 1674 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:3183: oid 1 version 1676 is already newer than 1675 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1676 (ObjNum 1104 snap 329 seq_num 1104) dirty exists 2024-09-16T15:15:11.044 INFO:tasks.rados.rados.0.smithi162.stdout:3183: left oid 1 (ObjNum 1104 snap 329 seq_num 1104) 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3181: done (3 left) 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3182: done (2 left) 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3183: done (1 left) 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3184: done (0 left) 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3185: write (excl) oid 41 current snap is 330 2024-09-16T15:15:11.934 INFO:tasks.rados.rados.0.smithi162.stdout:3185: seq_num 1105 ranges {571735=421330,1562132=416708} 2024-09-16T15:15:11.939 INFO:tasks.rados.rados.0.smithi162.stdout:3185: writing smithi16234619-41 from 571735 to 993065 tid 1 2024-09-16T15:15:11.945 INFO:tasks.rados.rados.0.smithi162.stdout:3185: writing smithi16234619-41 from 1562132 to 1978840 tid 2 2024-09-16T15:15:11.945 INFO:tasks.rados.rados.0.smithi162.stdout:3186: rollback oid 26 current snap is 330 2024-09-16T15:15:11.945 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 307 2024-09-16T15:15:11.945 INFO:tasks.rados.rados.0.smithi162.stdout:3187: delete oid 43 current snap is 330 2024-09-16T15:15:11.949 INFO:tasks.rados.rados.0.smithi162.stdout:3187: done (2 left) 2024-09-16T15:15:11.949 INFO:tasks.rados.rados.0.smithi162.stdout:3188: snap_create 2024-09-16T15:15:11.953 INFO:tasks.rados.rados.0.smithi162.stdout:3186: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:15:11.953 INFO:tasks.rados.rados.0.smithi162.stdout:3186: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:15:11.954 INFO:tasks.rados.rados.0.smithi162.stdout:3186: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:15:11.954 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1794 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:15:11.969 INFO:tasks.rados.rados.0.smithi162.stdout:3185: finishing write tid 1 to smithi16234619-41 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: finishing write tid 2 to smithi16234619-41 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: finishing write tid 3 to smithi16234619-41 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: finishing write tid 4 to smithi16234619-41 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: oid 41 updating version 0 to 1480 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: oid 41 version 1480 is already newer than 1478 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: oid 41 version 1480 is already newer than 1479 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 1480 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:15:11.977 INFO:tasks.rados.rados.0.smithi162.stdout:3185: left oid 41 (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:15:12.896 INFO:tasks.rados.rados.0.smithi162.stdout:3185: done (2 left) 2024-09-16T15:15:12.896 INFO:tasks.rados.rados.0.smithi162.stdout:3186: done (1 left) 2024-09-16T15:15:12.896 INFO:tasks.rados.rados.0.smithi162.stdout:3188: done (0 left) 2024-09-16T15:15:12.896 INFO:tasks.rados.rados.0.smithi162.stdout:3189: snap_remove snap 296 2024-09-16T15:15:13.896 INFO:tasks.rados.rados.0.smithi162.stdout:3189: done (0 left) 2024-09-16T15:15:13.897 INFO:tasks.rados.rados.0.smithi162.stdout:3190: delete oid 50 current snap is 331 2024-09-16T15:15:13.985 INFO:tasks.rados.rados.0.smithi162.stdout:3190: done (0 left) 2024-09-16T15:15:13.986 INFO:tasks.rados.rados.0.smithi162.stdout:3191: rollback oid 38 current snap is 331 2024-09-16T15:15:13.986 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 330 2024-09-16T15:15:13.986 INFO:tasks.rados.rados.0.smithi162.stdout:3192: read oid 22 snap -1 2024-09-16T15:15:13.986 INFO:tasks.rados.rados.0.smithi162.stdout:3192: expect deleted 2024-09-16T15:15:13.986 INFO:tasks.rados.rados.0.smithi162.stdout:3193: snap_create 2024-09-16T15:15:13.990 INFO:tasks.rados.rados.0.smithi162.stdout:3191: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:15:13.990 INFO:tasks.rados.rados.0.smithi162.stdout:3191: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:15:13.992 INFO:tasks.rados.rados.0.smithi162.stdout:3191: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:15:13.992 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1369 (ObjNum 1063 snap 318 seq_num 1063) dirty exists 2024-09-16T15:15:14.902 INFO:tasks.rados.rados.0.smithi162.stdout:3191: done (2 left) 2024-09-16T15:15:14.903 INFO:tasks.rados.rados.0.smithi162.stdout:3192: done (1 left) 2024-09-16T15:15:14.903 INFO:tasks.rados.rados.0.smithi162.stdout:3193: done (0 left) 2024-09-16T15:15:14.903 INFO:tasks.rados.rados.0.smithi162.stdout:3194: read oid 37 snap -1 2024-09-16T15:15:14.903 INFO:tasks.rados.rados.0.smithi162.stdout:3194: expect deleted 2024-09-16T15:15:14.903 INFO:tasks.rados.rados.0.smithi162.stdout:3195: snap_remove snap 286 2024-09-16T15:15:16.593 INFO:tasks.rados.rados.0.smithi162.stdout:3195: done (1 left) 2024-09-16T15:15:16.593 INFO:tasks.rados.rados.0.smithi162.stdout:3196: snap_remove snap 315 2024-09-16T15:15:17.592 INFO:tasks.rados.rados.0.smithi162.stdout:3194: done (1 left) 2024-09-16T15:15:17.593 INFO:tasks.rados.rados.0.smithi162.stdout:3196: done (0 left) 2024-09-16T15:15:17.593 INFO:tasks.rados.rados.0.smithi162.stdout:3197: write (excl) oid 12 current snap is 332 2024-09-16T15:15:17.593 INFO:tasks.rados.rados.0.smithi162.stdout:3197: seq_num 1108 ranges {622156=647077,1737578=680388} 2024-09-16T15:15:17.600 INFO:tasks.rados.rados.0.smithi162.stdout:3197: writing smithi16234619-12 from 622156 to 1269233 tid 1 2024-09-16T15:15:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:3197: writing smithi16234619-12 from 1737578 to 2417966 tid 2 2024-09-16T15:15:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:3198: copy_from oid 33 from oid 10 current snap is 332 2024-09-16T15:15:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:3199: write (excl) oid 34 current snap is 332 2024-09-16T15:15:17.607 INFO:tasks.rados.rados.0.smithi162.stdout:3199: seq_num 1109 ranges {638963=722326,2079707=424046} 2024-09-16T15:15:17.615 INFO:tasks.rados.rados.0.smithi162.stdout:3199: writing smithi16234619-34 from 638963 to 1361289 tid 1 2024-09-16T15:15:17.623 INFO:tasks.rados.rados.0.smithi162.stdout:3199: writing smithi16234619-34 from 2079707 to 2503753 tid 2 2024-09-16T15:15:17.623 INFO:tasks.rados.rados.0.smithi162.stdout:3200: delete oid 7 current snap is 332 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: finishing write tid 1 to smithi16234619-12 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: finishing write tid 2 to smithi16234619-12 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: finishing write tid 3 to smithi16234619-12 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: finishing write tid 4 to smithi16234619-12 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: oid 12 updating version 0 to 1456 2024-09-16T15:15:17.646 INFO:tasks.rados.rados.0.smithi162.stdout:3197: oid 12 updating version 1456 to 1458 2024-09-16T15:15:17.647 INFO:tasks.rados.rados.0.smithi162.stdout:3197: oid 12 version 1458 is already newer than 1457 2024-09-16T15:15:17.647 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1458 (ObjNum 1108 snap 332 seq_num 1108) dirty exists 2024-09-16T15:15:17.647 INFO:tasks.rados.rados.0.smithi162.stdout:3197: left oid 12 (ObjNum 1108 snap 332 seq_num 1108) 2024-09-16T15:15:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:3197: done (3 left) 2024-09-16T15:15:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:3200: done (2 left) 2024-09-16T15:15:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:3201: read oid 7 snap -1 2024-09-16T15:15:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:3201: expect deleted 2024-09-16T15:15:17.648 INFO:tasks.rados.rados.0.smithi162.stdout:3202: read oid 26 snap -1 2024-09-16T15:15:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:3202: expect (ObjNum 995 snap 281 seq_num 995) 2024-09-16T15:15:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:3203: write (excl) oid 50 current snap is 332 2024-09-16T15:15:17.649 INFO:tasks.rados.rados.0.smithi162.stdout:3203: seq_num 1111 ranges {672577=589177,1913024=427715} 2024-09-16T15:15:17.655 INFO:tasks.rados.rados.0.smithi162.stdout:3203: writing smithi16234619-50 from 672577 to 1261754 tid 1 2024-09-16T15:15:17.661 INFO:tasks.rados.rados.0.smithi162.stdout:3203: writing smithi16234619-50 from 1913024 to 2340739 tid 2 2024-09-16T15:15:17.661 INFO:tasks.rados.rados.0.smithi162.stdout:3204: delete oid 3 current snap is 332 2024-09-16T15:15:17.661 INFO:tasks.rados.rados.0.smithi162.stdout:3202: expect (ObjNum 995 snap 281 seq_num 995) 2024-09-16T15:15:17.746 INFO:tasks.rados.rados.0.smithi162.stdout:3198: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: finishing write tid 1 to smithi16234619-34 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3198: finishing copy_from to smithi16234619-33 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1696 (ObjNum 1077 snap 321 seq_num 1077) dirty exists 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: finishing write tid 2 to smithi16234619-34 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: finishing write tid 3 to smithi16234619-34 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: finishing write tid 4 to smithi16234619-34 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: oid 34 updating version 0 to 1543 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: oid 34 version 1543 is already newer than 1541 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: oid 34 version 1543 is already newer than 1542 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1543 (ObjNum 1109 snap 332 seq_num 1109) dirty exists 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: left oid 34 (ObjNum 1109 snap 332 seq_num 1109) 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3203: finishing write tid 1 to smithi16234619-50 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3198: done (5 left) 2024-09-16T15:15:17.747 INFO:tasks.rados.rados.0.smithi162.stdout:3199: done (4 left) 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3201: done (3 left) 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3202: done (2 left) 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3204: done (1 left) 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3205: read oid 7 snap -1 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3205: expect deleted 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3206: snap_create 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: finishing write tid 2 to smithi16234619-50 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: finishing write tid 3 to smithi16234619-50 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: finishing write tid 4 to smithi16234619-50 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: oid 50 updating version 0 to 1545 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: oid 50 updating version 1545 to 1546 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: oid 50 version 1546 is already newer than 1544 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1546 (ObjNum 1111 snap 332 seq_num 1111) dirty exists 2024-09-16T15:15:17.748 INFO:tasks.rados.rados.0.smithi162.stdout:3203: left oid 50 (ObjNum 1111 snap 332 seq_num 1111) 2024-09-16T15:15:18.143 INFO:tasks.rados.rados.0.smithi162.stdout:3203: done (2 left) 2024-09-16T15:15:18.143 INFO:tasks.rados.rados.0.smithi162.stdout:3205: done (1 left) 2024-09-16T15:15:18.143 INFO:tasks.rados.rados.0.smithi162.stdout:3206: done (0 left) 2024-09-16T15:15:18.143 INFO:tasks.rados.rados.0.smithi162.stdout:3207: read oid 42 snap 287 2024-09-16T15:15:18.144 INFO:tasks.rados.rados.0.smithi162.stdout:3207: expect deleted 2024-09-16T15:15:18.144 INFO:tasks.rados.rados.0.smithi162.stdout:3208: delete oid 43 current snap is 333 2024-09-16T15:15:18.151 INFO:tasks.rados.rados.0.smithi162.stdout:3207: done (1 left) 2024-09-16T15:15:18.151 INFO:tasks.rados.rados.0.smithi162.stdout:3208: done (0 left) 2024-09-16T15:15:18.151 INFO:tasks.rados.rados.0.smithi162.stdout:3209: read oid 23 snap -1 2024-09-16T15:15:18.151 INFO:tasks.rados.rados.0.smithi162.stdout:3209: expect deleted 2024-09-16T15:15:18.152 INFO:tasks.rados.rados.0.smithi162.stdout:3210: write (excl) oid 50 current snap is 333 2024-09-16T15:15:18.152 INFO:tasks.rados.rados.0.smithi162.stdout:3210: seq_num 1114 ranges {722998=414924,1572117=575042} 2024-09-16T15:15:18.157 INFO:tasks.rados.rados.0.smithi162.stdout:3210: writing smithi16234619-50 from 722998 to 1137922 tid 1 2024-09-16T15:15:18.163 INFO:tasks.rados.rados.0.smithi162.stdout:3210: writing smithi16234619-50 from 1572117 to 2147159 tid 2 2024-09-16T15:15:18.163 INFO:tasks.rados.rados.0.smithi162.stdout:3211: snap_remove snap 284 2024-09-16T15:15:19.150 INFO:tasks.rados.rados.0.smithi162.stdout:3211: done (2 left) 2024-09-16T15:15:19.150 INFO:tasks.rados.rados.0.smithi162.stdout:3212: write (excl) oid 14 current snap is 333 2024-09-16T15:15:19.150 INFO:tasks.rados.rados.0.smithi162.stdout:3212: seq_num 1115 ranges {739805=490173,1630599=435053,2636284=103521} 2024-09-16T15:15:19.157 INFO:tasks.rados.rados.0.smithi162.stdout:3212: writing smithi16234619-14 from 739805 to 1229978 tid 1 2024-09-16T15:15:19.161 INFO:tasks.rados.rados.0.smithi162.stdout:3212: writing smithi16234619-14 from 1630599 to 2065652 tid 2 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3212: writing smithi16234619-14 from 2636284 to 2739805 tid 3 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3213: read oid 33 snap -1 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3213: expect (ObjNum 1077 snap 321 seq_num 1077) 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3214: copy_from oid 29 from oid 36 current snap is 333 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3210: finishing write tid 1 to smithi16234619-50 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3210: finishing write tid 2 to smithi16234619-50 2024-09-16T15:15:19.166 INFO:tasks.rados.rados.0.smithi162.stdout:3210: finishing write tid 3 to smithi16234619-50 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: finishing write tid 4 to smithi16234619-50 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: oid 50 updating version 0 to 1549 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: oid 50 version 1549 is already newer than 1548 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: oid 50 updating version 1549 to 1550 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1550 (ObjNum 1114 snap 333 seq_num 1114) dirty exists 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: left oid 50 (ObjNum 1114 snap 333 seq_num 1114) 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3209: done (4 left) 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3210: done (3 left) 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3215: write oid 32 current snap is 333 2024-09-16T15:15:19.167 INFO:tasks.rados.rados.0.smithi162.stdout:3215: seq_num 1116 ranges {756612=565422,2089081=578711} 2024-09-16T15:15:19.173 INFO:tasks.rados.rados.0.smithi162.stdout:3215: writing smithi16234619-32 from 756612 to 1322034 tid 1 2024-09-16T15:15:19.181 INFO:tasks.rados.rados.0.smithi162.stdout:3215: writing smithi16234619-32 from 2089081 to 2667792 tid 2 2024-09-16T15:15:19.181 INFO:tasks.rados.rados.0.smithi162.stdout:3216: read oid 37 snap -1 2024-09-16T15:15:19.181 INFO:tasks.rados.rados.0.smithi162.stdout:3216: expect deleted 2024-09-16T15:15:19.181 INFO:tasks.rados.rados.0.smithi162.stdout:3217: delete oid 31 current snap is 333 2024-09-16T15:15:19.182 INFO:tasks.rados.rados.0.smithi162.stdout:3213: expect (ObjNum 1077 snap 321 seq_num 1077) 2024-09-16T15:15:19.295 INFO:tasks.rados.rados.0.smithi162.stdout:3212: finishing write tid 1 to smithi16234619-14 2024-09-16T15:15:19.295 INFO:tasks.rados.rados.0.smithi162.stdout:3212: finishing write tid 2 to smithi16234619-14 2024-09-16T15:15:19.295 INFO:tasks.rados.rados.0.smithi162.stdout:3213: done (5 left) 2024-09-16T15:15:19.296 INFO:tasks.rados.rados.0.smithi162.stdout:3216: done (4 left) 2024-09-16T15:15:19.296 INFO:tasks.rados.rados.0.smithi162.stdout:3217: done (3 left) 2024-09-16T15:15:19.296 INFO:tasks.rados.rados.0.smithi162.stdout:3218: read oid 16 snap -1 2024-09-16T15:15:19.296 INFO:tasks.rados.rados.0.smithi162.stdout:3218: expect deleted 2024-09-16T15:15:19.296 INFO:tasks.rados.rados.0.smithi162.stdout:3219: snap_remove snap 321 2024-09-16T15:15:20.156 INFO:tasks.rados.rados.0.smithi162.stdout:3219: done (4 left) 2024-09-16T15:15:20.157 INFO:tasks.rados.rados.0.smithi162.stdout:3220: write (excl) oid 17 current snap is 333 2024-09-16T15:15:20.157 INFO:tasks.rados.rados.0.smithi162.stdout:3220: seq_num 1118 ranges {790226=432273,1806045=582380} 2024-09-16T15:15:20.163 INFO:tasks.rados.rados.0.smithi162.stdout:3220: writing smithi16234619-17 from 790226 to 1222499 tid 1 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3220: writing smithi16234619-17 from 1806045 to 2388425 tid 2 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3221: read oid 42 snap -1 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3221: expect deleted 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3222: copy_from oid 25 from oid 46 current snap is 333 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3212: finishing write tid 3 to smithi16234619-14 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3212: finishing write tid 4 to smithi16234619-14 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3212: finishing write tid 5 to smithi16234619-14 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3212: oid 14 updating version 0 to 1561 2024-09-16T15:15:20.171 INFO:tasks.rados.rados.0.smithi162.stdout:3212: oid 14 updating version 1561 to 1563 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3212: oid 14 version 1563 is already newer than 1562 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3212: oid 14 version 1563 is already newer than 1560 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1563 (ObjNum 1115 snap 333 seq_num 1115) dirty exists 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3212: left oid 14 (ObjNum 1115 snap 333 seq_num 1115) 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: finishing write tid 1 to smithi16234619-32 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: finishing write tid 2 to smithi16234619-32 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: finishing write tid 3 to smithi16234619-32 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: finishing write tid 4 to smithi16234619-32 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: oid 32 updating version 0 to 1459 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: oid 32 updating version 1459 to 1460 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: oid 32 updating version 1460 to 1461 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1461 (ObjNum 1116 snap 333 seq_num 1116) dirty exists 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3215: left oid 32 (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:15:20.172 INFO:tasks.rados.rados.0.smithi162.stdout:3214: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3214: finishing copy_from to smithi16234619-29 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1405 (ObjNum 1061 snap 317 seq_num 1061) dirty exists 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3212: done (6 left) 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3214: done (5 left) 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3215: done (4 left) 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3218: done (3 left) 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3223: copy_from oid 44 from oid 26 current snap is 333 2024-09-16T15:15:20.173 INFO:tasks.rados.rados.0.smithi162.stdout:3224: snap_create 2024-09-16T15:15:20.191 INFO:tasks.rados.rados.0.smithi162.stdout:3220: finishing write tid 1 to smithi16234619-17 2024-09-16T15:15:20.193 INFO:tasks.rados.rados.0.smithi162.stdout:3220: finishing write tid 2 to smithi16234619-17 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: finishing write tid 3 to smithi16234619-17 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: finishing write tid 4 to smithi16234619-17 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: oid 17 updating version 0 to 1846 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: oid 17 version 1846 is already newer than 1845 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: oid 17 updating version 1846 to 1847 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1847 (ObjNum 1118 snap 333 seq_num 1118) dirty exists 2024-09-16T15:15:20.194 INFO:tasks.rados.rados.0.smithi162.stdout:3220: left oid 17 (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:15:20.211 INFO:tasks.rados.rados.0.smithi162.stdout:3222: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:15:20.211 INFO:tasks.rados.rados.0.smithi162.stdout:3222: finishing copy_from to smithi16234619-25 2024-09-16T15:15:20.212 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 1209 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:15:20.226 INFO:tasks.rados.rados.0.smithi162.stdout:3223: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:15:20.228 INFO:tasks.rados.rados.0.smithi162.stdout:3223: finishing copy_from to smithi16234619-44 2024-09-16T15:15:20.228 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1794 (ObjNum 995 snap 281 seq_num 995) dirty exists 2024-09-16T15:15:21.171 INFO:tasks.rados.rados.0.smithi162.stdout:3220: done (4 left) 2024-09-16T15:15:21.171 INFO:tasks.rados.rados.0.smithi162.stdout:3221: done (3 left) 2024-09-16T15:15:21.171 INFO:tasks.rados.rados.0.smithi162.stdout:3222: done (2 left) 2024-09-16T15:15:21.171 INFO:tasks.rados.rados.0.smithi162.stdout:3223: done (1 left) 2024-09-16T15:15:21.171 INFO:tasks.rados.rados.0.smithi162.stdout:3224: done (0 left) 2024-09-16T15:15:21.172 INFO:tasks.rados.rados.0.smithi162.stdout:3225: write (excl) oid 36 current snap is 334 2024-09-16T15:15:21.172 INFO:tasks.rados.rados.0.smithi162.stdout:3225: seq_num 1119 ranges {407033=507522,1464527=442391,2345976=461057} 2024-09-16T15:15:21.176 INFO:tasks.rados.rados.0.smithi162.stdout:3225: writing smithi16234619-36 from 407033 to 914555 tid 1 2024-09-16T15:15:21.181 INFO:tasks.rados.rados.0.smithi162.stdout:3225: writing smithi16234619-36 from 1464527 to 1906918 tid 2 2024-09-16T15:15:21.186 INFO:tasks.rados.rados.0.smithi162.stdout:3225: writing smithi16234619-36 from 2345976 to 2807033 tid 3 2024-09-16T15:15:21.186 INFO:tasks.rados.rados.0.smithi162.stdout:3226: write oid 13 current snap is 334 2024-09-16T15:15:21.186 INFO:tasks.rados.rados.0.smithi162.stdout:3226: seq_num 1120 ranges {423840=582771,1523009=586049,2657046=166794} 2024-09-16T15:15:21.191 INFO:tasks.rados.rados.0.smithi162.stdout:3226: writing smithi16234619-13 from 423840 to 1006611 tid 1 2024-09-16T15:15:21.198 INFO:tasks.rados.rados.0.smithi162.stdout:3226: writing smithi16234619-13 from 1523009 to 2109058 tid 2 2024-09-16T15:15:21.202 INFO:tasks.rados.rados.0.smithi162.stdout:3226: writing smithi16234619-13 from 2657046 to 2823840 tid 3 2024-09-16T15:15:21.202 INFO:tasks.rados.rados.0.smithi162.stdout:3227: write oid 18 current snap is 334 2024-09-16T15:15:21.202 INFO:tasks.rados.rados.0.smithi162.stdout:3227: seq_num 1121 ranges {440647=658020,1581491=446060,2800822=39825} 2024-09-16T15:15:21.208 INFO:tasks.rados.rados.0.smithi162.stdout:3227: writing smithi16234619-18 from 440647 to 1098667 tid 1 2024-09-16T15:15:21.213 INFO:tasks.rados.rados.0.smithi162.stdout:3227: writing smithi16234619-18 from 1581491 to 2027551 tid 2 2024-09-16T15:15:21.218 INFO:tasks.rados.rados.0.smithi162.stdout:3227: writing smithi16234619-18 from 2800822 to 2840647 tid 3 2024-09-16T15:15:21.218 INFO:tasks.rados.rados.0.smithi162.stdout:3228: read oid 40 snap 333 2024-09-16T15:15:21.218 INFO:tasks.rados.rados.0.smithi162.stdout:3228: expect (ObjNum 1057 snap 316 seq_num 1057) 2024-09-16T15:15:21.218 INFO:tasks.rados.rados.0.smithi162.stdout:3229: snap_remove snap 307 2024-09-16T15:15:22.176 INFO:tasks.rados.rados.0.smithi162.stdout:3229: done (4 left) 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3230: read oid 22 snap -1 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3230: expect deleted 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3231: snap_create 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: finishing write tid 1 to smithi16234619-36 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: finishing write tid 2 to smithi16234619-36 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: finishing write tid 3 to smithi16234619-36 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: finishing write tid 4 to smithi16234619-36 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: finishing write tid 5 to smithi16234619-36 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: oid 36 updating version 0 to 1577 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: oid 36 version 1577 is already newer than 1576 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: oid 36 version 1577 is already newer than 1575 2024-09-16T15:15:22.177 INFO:tasks.rados.rados.0.smithi162.stdout:3225: oid 36 version 1577 is already newer than 1574 2024-09-16T15:15:22.178 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1577 (ObjNum 1119 snap 334 seq_num 1119) dirty exists 2024-09-16T15:15:22.178 INFO:tasks.rados.rados.0.smithi162.stdout:3225: left oid 36 (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:15:22.178 INFO:tasks.rados.rados.0.smithi162.stdout:3228: expect (ObjNum 1057 snap 316 seq_num 1057) 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: finishing write tid 1 to smithi16234619-13 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: finishing write tid 2 to smithi16234619-13 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: finishing write tid 3 to smithi16234619-13 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: finishing write tid 4 to smithi16234619-13 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: finishing write tid 5 to smithi16234619-13 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: oid 13 updating version 0 to 1704 2024-09-16T15:15:22.391 INFO:tasks.rados.rados.0.smithi162.stdout:3226: oid 13 updating version 1704 to 1706 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3226: oid 13 version 1706 is already newer than 1705 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3226: oid 13 version 1706 is already newer than 1703 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1706 (ObjNum 1120 snap 334 seq_num 1120) dirty exists 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3226: left oid 13 (ObjNum 1120 snap 334 seq_num 1120) 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: finishing write tid 1 to smithi16234619-18 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: finishing write tid 2 to smithi16234619-18 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: finishing write tid 3 to smithi16234619-18 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: finishing write tid 4 to smithi16234619-18 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: finishing write tid 5 to smithi16234619-18 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: oid 18 updating version 0 to 1485 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: oid 18 version 1485 is already newer than 1482 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: oid 18 version 1485 is already newer than 1484 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:3227: oid 18 version 1485 is already newer than 1483 2024-09-16T15:15:22.392 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1485 (ObjNum 1121 snap 334 seq_num 1121) dirty exists 2024-09-16T15:15:22.393 INFO:tasks.rados.rados.0.smithi162.stdout:3227: left oid 18 (ObjNum 1121 snap 334 seq_num 1121) 2024-09-16T15:15:22.897 INFO:tasks.rados.rados.0.smithi162.stdout:3225: done (5 left) 2024-09-16T15:15:22.897 INFO:tasks.rados.rados.0.smithi162.stdout:3226: done (4 left) 2024-09-16T15:15:22.897 INFO:tasks.rados.rados.0.smithi162.stdout:3227: done (3 left) 2024-09-16T15:15:22.898 INFO:tasks.rados.rados.0.smithi162.stdout:3228: done (2 left) 2024-09-16T15:15:22.898 INFO:tasks.rados.rados.0.smithi162.stdout:3230: done (1 left) 2024-09-16T15:15:22.898 INFO:tasks.rados.rados.0.smithi162.stdout:3231: done (0 left) 2024-09-16T15:15:22.898 INFO:tasks.rados.rados.0.smithi162.stdout:3232: snap_create 2024-09-16T15:15:23.908 INFO:tasks.rados.rados.0.smithi162.stdout:3232: done (0 left) 2024-09-16T15:15:23.908 INFO:tasks.rados.rados.0.smithi162.stdout:3233: read oid 2 snap -1 2024-09-16T15:15:23.908 INFO:tasks.rados.rados.0.smithi162.stdout:3233: expect (ObjNum 1097 snap 328 seq_num 1097) 2024-09-16T15:15:23.908 INFO:tasks.rados.rados.0.smithi162.stdout:3234: delete oid 47 current snap is 336 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3234: done (1 left) 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3235: copy_from oid 44 from oid 46 current snap is 336 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3236: read oid 38 snap -1 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3236: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3237: write (excl) oid 9 current snap is 336 2024-09-16T15:15:23.911 INFO:tasks.rados.rados.0.smithi162.stdout:3237: seq_num 1123 ranges {474261=408518,1582102=449729,2739315=134946} 2024-09-16T15:15:23.916 INFO:tasks.rados.rados.0.smithi162.stdout:3237: writing smithi16234619-9 from 474261 to 882779 tid 1 2024-09-16T15:15:23.923 INFO:tasks.rados.rados.0.smithi162.stdout:3237: writing smithi16234619-9 from 1582102 to 2031831 tid 2 2024-09-16T15:15:23.927 INFO:tasks.rados.rados.0.smithi162.stdout:3237: writing smithi16234619-9 from 2739315 to 2874261 tid 3 2024-09-16T15:15:23.927 INFO:tasks.rados.rados.0.smithi162.stdout:3238: write oid 21 current snap is 336 2024-09-16T15:15:23.928 INFO:tasks.rados.rados.0.smithi162.stdout:3238: seq_num 1124 ranges {491068=483767,1640584=593387,2650385=240683} 2024-09-16T15:15:23.933 INFO:tasks.rados.rados.0.smithi162.stdout:3238: writing smithi16234619-21 from 491068 to 974835 tid 1 2024-09-16T15:15:23.940 INFO:tasks.rados.rados.0.smithi162.stdout:3238: writing smithi16234619-21 from 1640584 to 2233971 tid 2 2024-09-16T15:15:23.944 INFO:tasks.rados.rados.0.smithi162.stdout:3238: writing smithi16234619-21 from 2650385 to 2891068 tid 3 2024-09-16T15:15:23.944 INFO:tasks.rados.rados.0.smithi162.stdout:3239: snap_create 2024-09-16T15:15:23.944 INFO:tasks.rados.rados.0.smithi162.stdout:3233: expect (ObjNum 1097 snap 328 seq_num 1097) 2024-09-16T15:15:24.120 INFO:tasks.rados.rados.0.smithi162.stdout:3236: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: finishing write tid 1 to smithi16234619-9 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: finishing write tid 2 to smithi16234619-9 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: finishing write tid 3 to smithi16234619-9 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: finishing write tid 4 to smithi16234619-9 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: finishing write tid 5 to smithi16234619-9 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: oid 9 updating version 0 to 1678 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: oid 9 updating version 1678 to 1679 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: oid 9 updating version 1679 to 1680 2024-09-16T15:15:24.312 INFO:tasks.rados.rados.0.smithi162.stdout:3237: oid 9 version 1680 is already newer than 1677 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1680 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3237: left oid 9 (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3235: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: finishing write tid 1 to smithi16234619-21 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: finishing write tid 2 to smithi16234619-21 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: finishing write tid 3 to smithi16234619-21 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: finishing write tid 4 to smithi16234619-21 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: finishing write tid 5 to smithi16234619-21 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: oid 21 updating version 0 to 1862 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: oid 21 updating version 1862 to 1863 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: oid 21 updating version 1863 to 1865 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: oid 21 version 1865 is already newer than 1864 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1865 (ObjNum 1124 snap 336 seq_num 1124) dirty exists 2024-09-16T15:15:24.313 INFO:tasks.rados.rados.0.smithi162.stdout:3238: left oid 21 (ObjNum 1124 snap 336 seq_num 1124) 2024-09-16T15:15:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:3235: finishing copy_from to smithi16234619-44 2024-09-16T15:15:24.314 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1209 (ObjNum 906 snap 250 seq_num 906) dirty exists 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3233: done (5 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3235: done (4 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3236: done (3 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3237: done (2 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3238: done (1 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3239: done (0 left) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3240: read oid 21 snap -1 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3240: expect (ObjNum 1124 snap 336 seq_num 1124) 2024-09-16T15:15:24.918 INFO:tasks.rados.rados.0.smithi162.stdout:3241: read oid 6 snap -1 2024-09-16T15:15:24.919 INFO:tasks.rados.rados.0.smithi162.stdout:3241: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:15:24.919 INFO:tasks.rados.rados.0.smithi162.stdout:3242: snap_create 2024-09-16T15:15:24.925 INFO:tasks.rados.rados.0.smithi162.stdout:3240: expect (ObjNum 1124 snap 336 seq_num 1124) 2024-09-16T15:15:25.017 INFO:tasks.rados.rados.0.smithi162.stdout:3241: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:15:25.916 INFO:tasks.rados.rados.0.smithi162.stdout:3240: done (2 left) 2024-09-16T15:15:25.917 INFO:tasks.rados.rados.0.smithi162.stdout:3241: done (1 left) 2024-09-16T15:15:25.917 INFO:tasks.rados.rados.0.smithi162.stdout:3242: done (0 left) 2024-09-16T15:15:25.917 INFO:tasks.rados.rados.0.smithi162.stdout:3243: copy_from oid 44 from oid 40 current snap is 338 2024-09-16T15:15:25.917 INFO:tasks.rados.rados.0.smithi162.stdout:3244: copy_from oid 10 from oid 39 current snap is 338 2024-09-16T15:15:25.917 INFO:tasks.rados.rados.0.smithi162.stdout:3245: snap_remove snap 303 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:3245: done (2 left) 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:3246: rollback oid 20 current snap is 338 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 333 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:3244: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:3244: finishing copy_from to smithi16234619-10 2024-09-16T15:15:26.936 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1446 (ObjNum 888 snap 246 seq_num 888) dirty exists 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3243: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3243: finishing copy_from to smithi16234619-44 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1445 (ObjNum 1057 snap 316 seq_num 1057) dirty exists 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3243: done (2 left) 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3244: done (1 left) 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3247: read oid 32 snap 285 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3247: expect deleted 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3248: write oid 23 current snap is 338 2024-09-16T15:15:26.937 INFO:tasks.rados.rados.0.smithi162.stdout:3248: seq_num 1125 ranges {507875=559016,1699066=737045} 2024-09-16T15:15:26.942 INFO:tasks.rados.rados.0.smithi162.stdout:3248: writing smithi16234619-23 from 507875 to 1066891 tid 1 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:3248: writing smithi16234619-23 from 1699066 to 2436111 tid 2 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:3249: copy_from oid 26 from oid 35 current snap is 338 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:3246: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:3246: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:3246: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:15:26.950 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1699 (ObjNum 1053 snap 312 seq_num 1053) dirty exists 2024-09-16T15:15:26.951 INFO:tasks.rados.rados.0.smithi162.stdout:3246: done (3 left) 2024-09-16T15:15:26.951 INFO:tasks.rados.rados.0.smithi162.stdout:3247: done (2 left) 2024-09-16T15:15:26.951 INFO:tasks.rados.rados.0.smithi162.stdout:3250: delete oid 7 current snap is 338 2024-09-16T15:15:26.957 INFO:tasks.rados.rados.0.smithi162.stdout:3250: done (2 left) 2024-09-16T15:15:26.957 INFO:tasks.rados.rados.0.smithi162.stdout:3251: read oid 30 snap -1 2024-09-16T15:15:26.958 INFO:tasks.rados.rados.0.smithi162.stdout:3251: expect (ObjNum 1061 snap 317 seq_num 1061) 2024-09-16T15:15:26.958 INFO:tasks.rados.rados.0.smithi162.stdout:3252: rollback oid 1 current snap is 338 2024-09-16T15:15:26.958 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 316 2024-09-16T15:15:26.958 INFO:tasks.rados.rados.0.smithi162.stdout:3253: write (excl) oid 3 current snap is 338 2024-09-16T15:15:26.958 INFO:tasks.rados.rados.0.smithi162.stdout:3253: seq_num 1127 ranges {541489=425867,1416030=740714,2732654=208835} 2024-09-16T15:15:26.963 INFO:tasks.rados.rados.0.smithi162.stdout:3253: writing smithi16234619-3 from 541489 to 967356 tid 1 2024-09-16T15:15:26.971 INFO:tasks.rados.rados.0.smithi162.stdout:3253: writing smithi16234619-3 from 1416030 to 2156744 tid 2 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3253: writing smithi16234619-3 from 2732654 to 2941489 tid 3 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3254: copy_from oid 38 from oid 9 current snap is 338 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3252: finishing rollback tid 0 to smithi16234619-1 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3252: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3252: finishing rollback tid 2 to smithi16234619-1 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1683 (ObjNum 1028 snap 299 seq_num 1028) dirty exists 2024-09-16T15:15:26.976 INFO:tasks.rados.rados.0.smithi162.stdout:3251: expect (ObjNum 1061 snap 317 seq_num 1061) 2024-09-16T15:15:27.163 INFO:tasks.rados.rados.0.smithi162.stdout:3248: finishing write tid 1 to smithi16234619-23 2024-09-16T15:15:27.163 INFO:tasks.rados.rados.0.smithi162.stdout:3248: finishing write tid 2 to smithi16234619-23 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: finishing write tid 3 to smithi16234619-23 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: finishing write tid 4 to smithi16234619-23 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: oid 23 updating version 0 to 613 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: oid 23 updating version 613 to 614 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: oid 23 updating version 614 to 615 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 615 (ObjNum 1125 snap 338 seq_num 1125) dirty exists 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3248: left oid 23 (ObjNum 1125 snap 338 seq_num 1125) 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3253: finishing write tid 1 to smithi16234619-3 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3253: finishing write tid 2 to smithi16234619-3 2024-09-16T15:15:27.164 INFO:tasks.rados.rados.0.smithi162.stdout:3253: finishing write tid 3 to smithi16234619-3 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3248: done (5 left) 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3251: done (4 left) 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3252: done (3 left) 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3255: write oid 12 current snap is 338 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: finishing write tid 4 to smithi16234619-3 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: finishing write tid 5 to smithi16234619-3 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: oid 3 updating version 0 to 1304 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: oid 3 version 1304 is already newer than 1302 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: oid 3 version 1304 is already newer than 1303 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:3253: oid 3 version 1304 is already newer than 1301 2024-09-16T15:15:27.165 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1304 (ObjNum 1127 snap 338 seq_num 1127) dirty exists 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:3253: left oid 3 (ObjNum 1127 snap 338 seq_num 1127) 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:3249: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:3249: finishing copy_from to smithi16234619-26 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1360 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:3254: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:15:27.166 INFO:tasks.rados.rados.0.smithi162.stdout:3255: seq_num 1128 ranges {558296=501116,1474512=600725,2760077=198219} 2024-09-16T15:15:27.169 INFO:tasks.rados.rados.0.smithi162.stdout:3255: writing smithi16234619-12 from 558296 to 1059412 tid 1 2024-09-16T15:15:27.174 INFO:tasks.rados.rados.0.smithi162.stdout:3255: writing smithi16234619-12 from 1474512 to 2075237 tid 2 2024-09-16T15:15:27.179 INFO:tasks.rados.rados.0.smithi162.stdout:3255: writing smithi16234619-12 from 2760077 to 2958296 tid 3 2024-09-16T15:15:27.179 INFO:tasks.rados.rados.0.smithi162.stdout:3249: done (3 left) 2024-09-16T15:15:27.179 INFO:tasks.rados.rados.0.smithi162.stdout:3253: done (2 left) 2024-09-16T15:15:27.179 INFO:tasks.rados.rados.0.smithi162.stdout:3256: delete oid 43 current snap is 338 2024-09-16T15:15:27.179 INFO:tasks.rados.rados.0.smithi162.stdout:3254: finishing copy_from to smithi16234619-38 2024-09-16T15:15:27.180 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1680 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2024-09-16T15:15:27.180 INFO:tasks.rados.rados.0.smithi162.stdout:3254: done (2 left) 2024-09-16T15:15:27.180 INFO:tasks.rados.rados.0.smithi162.stdout:3256: done (1 left) 2024-09-16T15:15:27.180 INFO:tasks.rados.rados.0.smithi162.stdout:3257: delete oid 11 current snap is 338 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3257: done (1 left) 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3258: read oid 4 snap -1 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3258: expect deleted 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3259: read oid 39 snap -1 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3259: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3260: write (excl) oid 37 current snap is 338 2024-09-16T15:15:27.182 INFO:tasks.rados.rados.0.smithi162.stdout:3260: seq_num 1131 ranges {608717=726863,1933605=748052} 2024-09-16T15:15:27.189 INFO:tasks.rados.rados.0.smithi162.stdout:3260: writing smithi16234619-37 from 608717 to 1335580 tid 1 2024-09-16T15:15:27.197 INFO:tasks.rados.rados.0.smithi162.stdout:3260: writing smithi16234619-37 from 1933605 to 2681657 tid 2 2024-09-16T15:15:27.197 INFO:tasks.rados.rados.0.smithi162.stdout:3261: copy_from oid 33 from oid 41 current snap is 338 2024-09-16T15:15:27.197 INFO:tasks.rados.rados.0.smithi162.stdout:3259: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: finishing write tid 1 to smithi16234619-12 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: finishing write tid 2 to smithi16234619-12 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: finishing write tid 3 to smithi16234619-12 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: finishing write tid 4 to smithi16234619-12 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: finishing write tid 5 to smithi16234619-12 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: oid 12 updating version 0 to 1463 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: oid 12 version 1463 is already newer than 1462 2024-09-16T15:15:27.406 INFO:tasks.rados.rados.0.smithi162.stdout:3255: oid 12 updating version 1463 to 1465 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3255: oid 12 version 1465 is already newer than 1464 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1465 (ObjNum 1128 snap 338 seq_num 1128) dirty exists 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3255: left oid 12 (ObjNum 1128 snap 338 seq_num 1128) 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3260: finishing write tid 1 to smithi16234619-37 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3255: done (4 left) 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3258: done (3 left) 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3259: done (2 left) 2024-09-16T15:15:27.407 INFO:tasks.rados.rados.0.smithi162.stdout:3262: snap_remove snap 335 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3262: done (2 left) 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3263: copy_from oid 21 from oid 6 current snap is 338 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3264: read oid 24 snap -1 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3264: expect deleted 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3265: snap_create 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3260: finishing write tid 2 to smithi16234619-37 2024-09-16T15:15:27.898 INFO:tasks.rados.rados.0.smithi162.stdout:3260: finishing write tid 3 to smithi16234619-37 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3260: finishing write tid 4 to smithi16234619-37 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3260: oid 37 updating version 0 to 1727 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3260: oid 37 version 1727 is already newer than 1726 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3260: oid 37 version 1727 is already newer than 1725 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1727 (ObjNum 1131 snap 338 seq_num 1131) dirty exists 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3260: left oid 37 (ObjNum 1131 snap 338 seq_num 1131) 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3261: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:3261: finishing copy_from to smithi16234619-33 2024-09-16T15:15:27.899 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1480 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:15:27.939 INFO:tasks.rados.rados.0.smithi162.stdout:3263: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:15:27.942 INFO:tasks.rados.rados.0.smithi162.stdout:3263: finishing copy_from to smithi16234619-21 2024-09-16T15:15:27.942 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1292 (ObjNum 1037 snap 311 seq_num 1037) dirty exists 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3260: done (4 left) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3261: done (3 left) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3263: done (2 left) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3264: done (1 left) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3265: done (0 left) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3266: copy_from oid 40 from oid 43 current snap is 339 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3267: read oid 16 snap -1 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3267: expect deleted 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3268: read oid 39 snap -1 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3268: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:3269: rollback oid 20 current snap is 339 2024-09-16T15:15:29.743 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 277 2024-09-16T15:15:29.744 INFO:tasks.rados.rados.0.smithi162.stdout:3270: delete oid 29 current snap is 339 2024-09-16T15:15:29.748 INFO:tasks.rados.rados.0.smithi162.stdout:3268: expect (ObjNum 888 snap 246 seq_num 888) 2024-09-16T15:15:29.948 INFO:tasks.rados.rados.0.smithi162.stdout:3269: finishing rollback tid 0 to smithi16234619-20 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3269: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3267: done (4 left) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3268: done (3 left) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3270: done (2 left) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3271: read oid 25 snap -1 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3271: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3272: read oid 32 snap -1 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3272: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3273: write oid 13 current snap is 339 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3266: finishing copy_from to smithi16234619-40 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3266: got expected ENOENT (src dne) 2024-09-16T15:15:29.949 INFO:tasks.rados.rados.0.smithi162.stdout:3266: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:15:29.950 INFO:tasks.rados.rados.0.smithi162.stdout:3269: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:15:29.950 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1702 (ObjNum 953 snap 268 seq_num 953) dirty exists 2024-09-16T15:15:29.950 INFO:tasks.rados.rados.0.smithi162.stdout:3273: seq_num 1133 ranges {642331=593714,1766922=751721} 2024-09-16T15:15:29.955 INFO:tasks.rados.rados.0.smithi162.stdout:3273: writing smithi16234619-13 from 642331 to 1236045 tid 1 2024-09-16T15:15:29.963 INFO:tasks.rados.rados.0.smithi162.stdout:3273: writing smithi16234619-13 from 1766922 to 2518643 tid 2 2024-09-16T15:15:29.963 INFO:tasks.rados.rados.0.smithi162.stdout:3266: done (4 left) 2024-09-16T15:15:29.963 INFO:tasks.rados.rados.0.smithi162.stdout:3269: done (3 left) 2024-09-16T15:15:29.963 INFO:tasks.rados.rados.0.smithi162.stdout:3274: snap_create 2024-09-16T15:15:29.963 INFO:tasks.rados.rados.0.smithi162.stdout:3271: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:15:30.069 INFO:tasks.rados.rados.0.smithi162.stdout:3272: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: finishing write tid 1 to smithi16234619-13 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: finishing write tid 2 to smithi16234619-13 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: finishing write tid 3 to smithi16234619-13 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: finishing write tid 4 to smithi16234619-13 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: oid 13 updating version 0 to 1731 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: oid 13 updating version 1731 to 1732 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: oid 13 updating version 1732 to 1733 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1733 (ObjNum 1133 snap 339 seq_num 1133) dirty exists 2024-09-16T15:15:30.189 INFO:tasks.rados.rados.0.smithi162.stdout:3273: left oid 13 (ObjNum 1133 snap 339 seq_num 1133) 2024-09-16T15:15:31.503 INFO:tasks.rados.rados.0.smithi162.stdout:3271: done (3 left) 2024-09-16T15:15:31.503 INFO:tasks.rados.rados.0.smithi162.stdout:3272: done (2 left) 2024-09-16T15:15:31.503 INFO:tasks.rados.rados.0.smithi162.stdout:3273: done (1 left) 2024-09-16T15:15:31.503 INFO:tasks.rados.rados.0.smithi162.stdout:3274: done (0 left) 2024-09-16T15:15:31.503 INFO:tasks.rados.rados.0.smithi162.stdout:3275: delete oid 15 current snap is 340 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3275: done (0 left) 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3276: read oid 33 snap -1 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3276: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3277: read oid 2 snap -1 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3277: expect (ObjNum 1097 snap 328 seq_num 1097) 2024-09-16T15:15:31.508 INFO:tasks.rados.rados.0.smithi162.stdout:3278: delete oid 27 current snap is 340 2024-09-16T15:15:31.512 INFO:tasks.rados.rados.0.smithi162.stdout:3277: expect (ObjNum 1097 snap 328 seq_num 1097) 2024-09-16T15:15:31.681 INFO:tasks.rados.rados.0.smithi162.stdout:3276: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:15:31.796 INFO:tasks.rados.rados.0.smithi162.stdout:3276: done (2 left) 2024-09-16T15:15:31.796 INFO:tasks.rados.rados.0.smithi162.stdout:3277: done (1 left) 2024-09-16T15:15:31.796 INFO:tasks.rados.rados.0.smithi162.stdout:3278: done (0 left) 2024-09-16T15:15:31.796 INFO:tasks.rados.rados.0.smithi162.stdout:3279: write (excl) oid 30 current snap is 340 2024-09-16T15:15:31.797 INFO:tasks.rados.rados.0.smithi162.stdout:3279: seq_num 1136 ranges {692752=419461,1826015=615401,2979461=113291} 2024-09-16T15:15:31.801 INFO:tasks.rados.rados.0.smithi162.stdout:3279: writing smithi16234619-30 from 692752 to 1112213 tid 1 2024-09-16T15:15:31.808 INFO:tasks.rados.rados.0.smithi162.stdout:3279: writing smithi16234619-30 from 1826015 to 2441416 tid 2 2024-09-16T15:15:31.812 INFO:tasks.rados.rados.0.smithi162.stdout:3279: writing smithi16234619-30 from 2979461 to 3092752 tid 3 2024-09-16T15:15:31.812 INFO:tasks.rados.rados.0.smithi162.stdout:3280: delete oid 6 current snap is 340 2024-09-16T15:15:31.816 INFO:tasks.rados.rados.0.smithi162.stdout:3280: done (1 left) 2024-09-16T15:15:31.816 INFO:tasks.rados.rados.0.smithi162.stdout:3281: rollback oid 8 current snap is 340 2024-09-16T15:15:31.816 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 338 2024-09-16T15:15:31.817 INFO:tasks.rados.rados.0.smithi162.stdout:3282: copy_from oid 23 from oid 41 current snap is 340 2024-09-16T15:15:31.817 INFO:tasks.rados.rados.0.smithi162.stdout:3283: read oid 36 snap -1 2024-09-16T15:15:31.817 INFO:tasks.rados.rados.0.smithi162.stdout:3283: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:15:31.817 INFO:tasks.rados.rados.0.smithi162.stdout:3284: write oid 20 current snap is 340 2024-09-16T15:15:31.817 INFO:tasks.rados.rados.0.smithi162.stdout:3284: seq_num 1138 ranges {726366=569959,1826626=502717,2801601=324765} 2024-09-16T15:15:31.822 INFO:tasks.rados.rados.0.smithi162.stdout:3284: writing smithi16234619-20 from 726366 to 1296325 tid 1 2024-09-16T15:15:31.828 INFO:tasks.rados.rados.0.smithi162.stdout:3284: writing smithi16234619-20 from 1826626 to 2329343 tid 2 2024-09-16T15:15:31.833 INFO:tasks.rados.rados.0.smithi162.stdout:3284: writing smithi16234619-20 from 2801601 to 3126366 tid 3 2024-09-16T15:15:31.833 INFO:tasks.rados.rados.0.smithi162.stdout:3285: write (excl) oid 4 current snap is 340 2024-09-16T15:15:31.833 INFO:tasks.rados.rados.0.smithi162.stdout:3285: seq_num 1139 ranges {743173=645208,1885108=762728} 2024-09-16T15:15:31.841 INFO:tasks.rados.rados.0.smithi162.stdout:3285: writing smithi16234619-4 from 743173 to 1388381 tid 1 2024-09-16T15:15:31.850 INFO:tasks.rados.rados.0.smithi162.stdout:3285: writing smithi16234619-4 from 1885108 to 2647836 tid 2 2024-09-16T15:15:31.850 INFO:tasks.rados.rados.0.smithi162.stdout:3286: write (excl) oid 12 current snap is 340 2024-09-16T15:15:31.850 INFO:tasks.rados.rados.0.smithi162.stdout:3286: seq_num 1140 ranges {759980=720457,1943590=506386,2856447=303533} 2024-09-16T15:15:31.858 INFO:tasks.rados.rados.0.smithi162.stdout:3286: writing smithi16234619-12 from 759980 to 1480437 tid 1 2024-09-16T15:15:31.863 INFO:tasks.rados.rados.0.smithi162.stdout:3286: writing smithi16234619-12 from 1943590 to 2449976 tid 2 2024-09-16T15:15:31.867 INFO:tasks.rados.rados.0.smithi162.stdout:3286: writing smithi16234619-12 from 2856447 to 3159980 tid 3 2024-09-16T15:15:31.868 INFO:tasks.rados.rados.0.smithi162.stdout:3287: snap_remove snap 277 2024-09-16T15:15:33.190 INFO:tasks.rados.rados.0.smithi162.stdout:3287: done (7 left) 2024-09-16T15:15:33.190 INFO:tasks.rados.rados.0.smithi162.stdout:3288: write (excl) oid 50 current snap is 340 2024-09-16T15:15:33.190 INFO:tasks.rados.rados.0.smithi162.stdout:3281: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:15:33.190 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1478 (ObjNum 866310496 snap 21987 seq_num 304) dirty dne 2024-09-16T15:15:33.190 INFO:tasks.rados.rados.0.smithi162.stdout:3288: seq_num 1141 ranges {776787=512059,1718425=766397,3000223=176564} 2024-09-16T15:15:33.197 INFO:tasks.rados.rados.0.smithi162.stdout:3288: writing smithi16234619-50 from 776787 to 1288846 tid 1 2024-09-16T15:15:33.204 INFO:tasks.rados.rados.0.smithi162.stdout:3288: writing smithi16234619-50 from 1718425 to 2484822 tid 2 2024-09-16T15:15:33.208 INFO:tasks.rados.rados.0.smithi162.stdout:3288: writing smithi16234619-50 from 3000223 to 3176787 tid 3 2024-09-16T15:15:33.208 INFO:tasks.rados.rados.0.smithi162.stdout:3281: done (7 left) 2024-09-16T15:15:33.208 INFO:tasks.rados.rados.0.smithi162.stdout:3289: rollback oid 26 current snap is 340 2024-09-16T15:15:33.208 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 338 2024-09-16T15:15:33.209 INFO:tasks.rados.rados.0.smithi162.stdout:3283: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: finishing write tid 1 to smithi16234619-30 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: finishing write tid 2 to smithi16234619-30 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: finishing write tid 3 to smithi16234619-30 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: finishing write tid 4 to smithi16234619-30 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: finishing write tid 5 to smithi16234619-30 2024-09-16T15:15:33.348 INFO:tasks.rados.rados.0.smithi162.stdout:3279: oid 30 updating version 0 to 1413 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3279: oid 30 version 1413 is already newer than 1410 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3279: oid 30 version 1413 is already newer than 1411 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3279: oid 30 version 1413 is already newer than 1412 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1413 (ObjNum 1136 snap 340 seq_num 1136) dirty exists 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3279: left oid 30 (ObjNum 1136 snap 340 seq_num 1136) 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3282: finishing copy_from racing read to smithi16234619-23 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3284: finishing write tid 1 to smithi16234619-20 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3284: finishing write tid 2 to smithi16234619-20 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3279: done (7 left) 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3283: done (6 left) 2024-09-16T15:15:33.349 INFO:tasks.rados.rados.0.smithi162.stdout:3290: snap_remove snap 322 2024-09-16T15:15:34.191 INFO:tasks.rados.rados.0.smithi162.stdout:3290: done (6 left) 2024-09-16T15:15:34.191 INFO:tasks.rados.rados.0.smithi162.stdout:3291: read oid 19 snap -1 2024-09-16T15:15:34.191 INFO:tasks.rados.rados.0.smithi162.stdout:3291: expect (ObjNum 1008 snap 290 seq_num 1008) 2024-09-16T15:15:34.191 INFO:tasks.rados.rados.0.smithi162.stdout:3292: snap_create 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: finishing write tid 3 to smithi16234619-20 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: finishing write tid 4 to smithi16234619-20 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: finishing write tid 5 to smithi16234619-20 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: oid 20 updating version 0 to 1706 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: oid 20 version 1706 is already newer than 1704 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: oid 20 version 1706 is already newer than 1703 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: oid 20 version 1706 is already newer than 1705 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1706 (ObjNum 1138 snap 340 seq_num 1138) dirty exists 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3284: left oid 20 (ObjNum 1138 snap 340 seq_num 1138) 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3282: finishing copy_from to smithi16234619-23 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 1480 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:15:34.192 INFO:tasks.rados.rados.0.smithi162.stdout:3285: finishing write tid 1 to smithi16234619-4 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: finishing write tid 2 to smithi16234619-4 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: finishing write tid 3 to smithi16234619-4 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: finishing write tid 4 to smithi16234619-4 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: oid 4 updating version 0 to 1685 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: oid 4 updating version 1685 to 1687 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:3285: oid 4 version 1687 is already newer than 1686 2024-09-16T15:15:34.193 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1687 (ObjNum 1139 snap 340 seq_num 1139) dirty exists 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3285: left oid 4 (ObjNum 1139 snap 340 seq_num 1139) 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: finishing write tid 1 to smithi16234619-12 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: finishing write tid 2 to smithi16234619-12 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: finishing write tid 3 to smithi16234619-12 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: finishing write tid 4 to smithi16234619-12 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: finishing write tid 5 to smithi16234619-12 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: oid 12 updating version 0 to 1468 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: oid 12 version 1468 is already newer than 1466 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: oid 12 updating version 1468 to 1469 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: oid 12 version 1469 is already newer than 1467 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1469 (ObjNum 1140 snap 340 seq_num 1140) dirty exists 2024-09-16T15:15:34.194 INFO:tasks.rados.rados.0.smithi162.stdout:3286: left oid 12 (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3289: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3289: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3289: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1905 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: finishing write tid 1 to smithi16234619-50 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: finishing write tid 2 to smithi16234619-50 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: finishing write tid 3 to smithi16234619-50 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: finishing write tid 4 to smithi16234619-50 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: finishing write tid 5 to smithi16234619-50 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: oid 50 updating version 0 to 1601 2024-09-16T15:15:34.195 INFO:tasks.rados.rados.0.smithi162.stdout:3288: oid 50 version 1601 is already newer than 1600 2024-09-16T15:15:34.196 INFO:tasks.rados.rados.0.smithi162.stdout:3288: oid 50 version 1601 is already newer than 1599 2024-09-16T15:15:34.196 INFO:tasks.rados.rados.0.smithi162.stdout:3288: oid 50 updating version 1601 to 1602 2024-09-16T15:15:34.196 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1602 (ObjNum 1141 snap 340 seq_num 1141) dirty exists 2024-09-16T15:15:34.196 INFO:tasks.rados.rados.0.smithi162.stdout:3288: left oid 50 (ObjNum 1141 snap 340 seq_num 1141) 2024-09-16T15:15:34.207 INFO:tasks.rados.rados.0.smithi162.stdout:3291: expect (ObjNum 1008 snap 290 seq_num 1008) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3282: done (7 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3284: done (6 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3285: done (5 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3286: done (4 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3288: done (3 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3289: done (2 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3291: done (1 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3292: done (0 left) 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3293: write oid 16 current snap is 341 2024-09-16T15:15:35.342 INFO:tasks.rados.rados.0.smithi162.stdout:3293: seq_num 1142 ranges {793594=587308,2060554=510055} 2024-09-16T15:15:35.349 INFO:tasks.rados.rados.0.smithi162.stdout:3293: writing smithi16234619-16 from 793594 to 1380902 tid 1 2024-09-16T15:15:35.357 INFO:tasks.rados.rados.0.smithi162.stdout:3293: writing smithi16234619-16 from 2060554 to 2570609 tid 2 2024-09-16T15:15:35.357 INFO:tasks.rados.rados.0.smithi162.stdout:3294: snap_remove snap 304 2024-09-16T15:15:36.352 INFO:tasks.rados.rados.0.smithi162.stdout:3294: done (1 left) 2024-09-16T15:15:36.352 INFO:tasks.rados.rados.0.smithi162.stdout:3295: copy_from oid 9 from oid 13 current snap is 341 2024-09-16T15:15:36.352 INFO:tasks.rados.rados.0.smithi162.stdout:3293: finishing write tid 1 to smithi16234619-16 2024-09-16T15:15:36.352 INFO:tasks.rados.rados.0.smithi162.stdout:3293: finishing write tid 2 to smithi16234619-16 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: finishing write tid 3 to smithi16234619-16 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: finishing write tid 4 to smithi16234619-16 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: oid 16 updating version 0 to 1796 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: oid 16 updating version 1796 to 1798 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: oid 16 version 1798 is already newer than 1797 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1798 (ObjNum 1142 snap 341 seq_num 1142) dirty exists 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: left oid 16 (ObjNum 1142 snap 341 seq_num 1142) 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3293: done (1 left) 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3296: write (excl) oid 8 current snap is 341 2024-09-16T15:15:36.353 INFO:tasks.rados.rados.0.smithi162.stdout:3296: seq_num 1143 ranges {410401=662557,1719036=770066,2938716=271685} 2024-09-16T15:15:36.358 INFO:tasks.rados.rados.0.smithi162.stdout:3296: writing smithi16234619-8 from 410401 to 1072958 tid 1 2024-09-16T15:15:36.366 INFO:tasks.rados.rados.0.smithi162.stdout:3296: writing smithi16234619-8 from 1719036 to 2489102 tid 2 2024-09-16T15:15:36.370 INFO:tasks.rados.rados.0.smithi162.stdout:3296: writing smithi16234619-8 from 2938716 to 3210401 tid 3 2024-09-16T15:15:36.371 INFO:tasks.rados.rados.0.smithi162.stdout:3297: snap_remove snap 337 2024-09-16T15:15:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:3297: done (2 left) 2024-09-16T15:15:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:3298: write oid 3 current snap is 341 2024-09-16T15:15:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:3295: finishing copy_from racing read to smithi16234619-9 2024-09-16T15:15:37.355 INFO:tasks.rados.rados.0.smithi162.stdout:3298: seq_num 1144 ranges {427208=737806,1777518=513724,2966139=261069} 2024-09-16T15:15:37.362 INFO:tasks.rados.rados.0.smithi162.stdout:3298: writing smithi16234619-3 from 427208 to 1165014 tid 1 2024-09-16T15:15:37.369 INFO:tasks.rados.rados.0.smithi162.stdout:3298: writing smithi16234619-3 from 1777518 to 2291242 tid 2 2024-09-16T15:15:37.374 INFO:tasks.rados.rados.0.smithi162.stdout:3298: writing smithi16234619-3 from 2966139 to 3227208 tid 3 2024-09-16T15:15:37.374 INFO:tasks.rados.rados.0.smithi162.stdout:3299: snap_remove snap 332 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: finishing write tid 1 to smithi16234619-8 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: finishing write tid 2 to smithi16234619-8 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: finishing write tid 3 to smithi16234619-8 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: finishing write tid 4 to smithi16234619-8 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: finishing write tid 5 to smithi16234619-8 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: oid 8 updating version 0 to 1487 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: oid 8 updating version 1487 to 1488 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: oid 8 updating version 1488 to 1489 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: oid 8 version 1489 is already newer than 1486 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1489 (ObjNum 1143 snap 341 seq_num 1143) dirty exists 2024-09-16T15:15:37.901 INFO:tasks.rados.rados.0.smithi162.stdout:3296: left oid 8 (ObjNum 1143 snap 341 seq_num 1143) 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3295: finishing copy_from to smithi16234619-9 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1733 (ObjNum 1133 snap 339 seq_num 1133) dirty exists 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: finishing write tid 1 to smithi16234619-3 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: finishing write tid 2 to smithi16234619-3 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: finishing write tid 3 to smithi16234619-3 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: finishing write tid 4 to smithi16234619-3 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: finishing write tid 5 to smithi16234619-3 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: oid 3 updating version 0 to 1308 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: oid 3 version 1308 is already newer than 1307 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: oid 3 version 1308 is already newer than 1305 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: oid 3 version 1308 is already newer than 1306 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1308 (ObjNum 1144 snap 341 seq_num 1144) dirty exists 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3298: left oid 3 (ObjNum 1144 snap 341 seq_num 1144) 2024-09-16T15:15:37.902 INFO:tasks.rados.rados.0.smithi162.stdout:3295: done (3 left) 2024-09-16T15:15:37.903 INFO:tasks.rados.rados.0.smithi162.stdout:3296: done (2 left) 2024-09-16T15:15:37.903 INFO:tasks.rados.rados.0.smithi162.stdout:3298: done (1 left) 2024-09-16T15:15:37.903 INFO:tasks.rados.rados.0.smithi162.stdout:3299: done (0 left) 2024-09-16T15:15:37.903 INFO:tasks.rados.rados.0.smithi162.stdout:3300: delete oid 18 current snap is 341 2024-09-16T15:15:37.907 INFO:tasks.rados.rados.0.smithi162.stdout:3300: done (0 left) 2024-09-16T15:15:37.907 INFO:tasks.rados.rados.0.smithi162.stdout:3301: copy_from oid 47 from oid 18 current snap is 341 2024-09-16T15:15:37.907 INFO:tasks.rados.rados.0.smithi162.stdout:3302: snap_create 2024-09-16T15:15:37.908 INFO:tasks.rados.rados.0.smithi162.stdout:3301: finishing copy_from to smithi16234619-47 2024-09-16T15:15:37.908 INFO:tasks.rados.rados.0.smithi162.stdout:3301: got expected ENOENT (src dne) 2024-09-16T15:15:37.908 INFO:tasks.rados.rados.0.smithi162.stdout:3301: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:15:39.099 INFO:tasks.rados.rados.0.smithi162.stdout:3301: done (1 left) 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3302: done (0 left) 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3303: read oid 16 snap -1 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3303: expect (ObjNum 1142 snap 341 seq_num 1142) 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3304: rollback oid 37 current snap is 342 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 341 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3305: copy_from oid 10 from oid 38 current snap is 342 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3306: read oid 22 snap 326 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3306: expect deleted 2024-09-16T15:15:39.100 INFO:tasks.rados.rados.0.smithi162.stdout:3307: snap_remove snap 312 2024-09-16T15:15:40.196 INFO:tasks.rados.rados.0.smithi162.stdout:3303: expect (ObjNum 1142 snap 341 seq_num 1142) 2024-09-16T15:15:40.306 INFO:tasks.rados.rados.0.smithi162.stdout:3304: finishing rollback tid 0 to smithi16234619-37 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3304: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3303: done (4 left) 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3306: done (3 left) 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3307: done (2 left) 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3308: write (excl) oid 29 current snap is 342 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3304: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1765 (ObjNum 1131 snap 338 seq_num 1131) dirty exists 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3305: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3305: finishing copy_from to smithi16234619-10 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1680 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2024-09-16T15:15:40.307 INFO:tasks.rados.rados.0.smithi162.stdout:3308: seq_num 1146 ranges {460822=488304,1378129=517393,2504632=405962} 2024-09-16T15:15:40.311 INFO:tasks.rados.rados.0.smithi162.stdout:3308: writing smithi16234619-29 from 460822 to 949126 tid 1 2024-09-16T15:15:40.316 INFO:tasks.rados.rados.0.smithi162.stdout:3308: writing smithi16234619-29 from 1378129 to 1895522 tid 2 2024-09-16T15:15:40.322 INFO:tasks.rados.rados.0.smithi162.stdout:3308: writing smithi16234619-29 from 2504632 to 2910594 tid 3 2024-09-16T15:15:40.322 INFO:tasks.rados.rados.0.smithi162.stdout:3304: done (2 left) 2024-09-16T15:15:40.322 INFO:tasks.rados.rados.0.smithi162.stdout:3305: done (1 left) 2024-09-16T15:15:40.322 INFO:tasks.rados.rados.0.smithi162.stdout:3309: delete oid 2 current snap is 342 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3309: done (1 left) 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3310: rollback oid 43 current snap is 342 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 285 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3311: read oid 2 snap -1 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3311: expect deleted 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3312: write (excl) oid 46 current snap is 342 2024-09-16T15:15:40.327 INFO:tasks.rados.rados.0.smithi162.stdout:3312: seq_num 1148 ranges {494436=755155,2011446=521062,3075831=218605} 2024-09-16T15:15:40.333 INFO:tasks.rados.rados.0.smithi162.stdout:3312: writing smithi16234619-46 from 494436 to 1249591 tid 1 2024-09-16T15:15:40.340 INFO:tasks.rados.rados.0.smithi162.stdout:3312: writing smithi16234619-46 from 2011446 to 2532508 tid 2 2024-09-16T15:15:40.345 INFO:tasks.rados.rados.0.smithi162.stdout:3312: writing smithi16234619-46 from 3075831 to 3294436 tid 3 2024-09-16T15:15:40.345 INFO:tasks.rados.rados.0.smithi162.stdout:3313: delete oid 8 current snap is 342 2024-09-16T15:15:40.345 INFO:tasks.rados.rados.0.smithi162.stdout:3310: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:15:40.345 INFO:tasks.rados.rados.0.smithi162.stdout:3310: finishing rollback tid 2 to smithi16234619-43 2024-09-16T15:15:40.345 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1168 (ObjNum 827 snap 224 seq_num 827) dirty exists 2024-09-16T15:15:40.346 INFO:tasks.rados.rados.0.smithi162.stdout:3308: finishing write tid 1 to smithi16234619-29 2024-09-16T15:15:40.349 INFO:tasks.rados.rados.0.smithi162.stdout:3310: done (4 left) 2024-09-16T15:15:40.349 INFO:tasks.rados.rados.0.smithi162.stdout:3311: done (3 left) 2024-09-16T15:15:40.349 INFO:tasks.rados.rados.0.smithi162.stdout:3313: done (2 left) 2024-09-16T15:15:40.349 INFO:tasks.rados.rados.0.smithi162.stdout:3314: snap_create 2024-09-16T15:15:40.351 INFO:tasks.rados.rados.0.smithi162.stdout:3308: finishing write tid 2 to smithi16234619-29 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: finishing write tid 3 to smithi16234619-29 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: finishing write tid 4 to smithi16234619-29 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: finishing write tid 5 to smithi16234619-29 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: oid 29 updating version 0 to 1772 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: oid 29 updating version 1772 to 1775 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: oid 29 version 1775 is already newer than 1774 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: oid 29 version 1775 is already newer than 1773 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1775 (ObjNum 1146 snap 342 seq_num 1146) dirty exists 2024-09-16T15:15:40.352 INFO:tasks.rados.rados.0.smithi162.stdout:3308: left oid 29 (ObjNum 1146 snap 342 seq_num 1146) 2024-09-16T15:15:40.375 INFO:tasks.rados.rados.0.smithi162.stdout:3312: finishing write tid 1 to smithi16234619-46 2024-09-16T15:15:40.382 INFO:tasks.rados.rados.0.smithi162.stdout:3312: finishing write tid 2 to smithi16234619-46 2024-09-16T15:15:40.382 INFO:tasks.rados.rados.0.smithi162.stdout:3312: finishing write tid 3 to smithi16234619-46 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: finishing write tid 4 to smithi16234619-46 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: finishing write tid 5 to smithi16234619-46 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: oid 46 updating version 0 to 1212 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: oid 46 version 1212 is already newer than 1211 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: oid 46 version 1212 is already newer than 1210 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: oid 46 updating version 1212 to 1213 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1213 (ObjNum 1148 snap 342 seq_num 1148) dirty exists 2024-09-16T15:15:40.383 INFO:tasks.rados.rados.0.smithi162.stdout:3312: left oid 46 (ObjNum 1148 snap 342 seq_num 1148) 2024-09-16T15:15:41.201 INFO:tasks.rados.rados.0.smithi162.stdout:3308: done (2 left) 2024-09-16T15:15:41.201 INFO:tasks.rados.rados.0.smithi162.stdout:3312: done (1 left) 2024-09-16T15:15:41.201 INFO:tasks.rados.rados.0.smithi162.stdout:3314: done (0 left) 2024-09-16T15:15:41.201 INFO:tasks.rados.rados.0.smithi162.stdout:3315: delete oid 37 current snap is 343 2024-09-16T15:15:41.205 INFO:tasks.rados.rados.0.smithi162.stdout:3315: done (0 left) 2024-09-16T15:15:41.205 INFO:tasks.rados.rados.0.smithi162.stdout:3316: read oid 35 snap 338 2024-09-16T15:15:41.205 INFO:tasks.rados.rados.0.smithi162.stdout:3316: expect (ObjNum 1054 snap 313 seq_num 1054) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3317: copy_from oid 48 from oid 14 current snap is 343 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3318: read oid 15 snap 334 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3318: expect (ObjNum 1089 snap 326 seq_num 1089) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3319: copy_from oid 10 from oid 31 current snap is 343 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3320: read oid 43 snap -1 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3320: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3321: read oid 49 snap 318 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3321: expect (ObjNum 1062 snap 317 seq_num 1062) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3322: read oid 44 snap -1 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3322: expect (ObjNum 1057 snap 316 seq_num 1057) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3323: copy_from oid 38 from oid 27 current snap is 343 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3324: read oid 25 snap -1 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3324: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:15:41.206 INFO:tasks.rados.rados.0.smithi162.stdout:3325: write (excl) oid 9 current snap is 343 2024-09-16T15:15:41.207 INFO:tasks.rados.rados.0.smithi162.stdout:3325: seq_num 1151 ranges {544857=580902,1670539=668389,3041747=303110} 2024-09-16T15:15:41.212 INFO:tasks.rados.rados.0.smithi162.stdout:3325: writing smithi16234619-9 from 544857 to 1125759 tid 1 2024-09-16T15:15:41.219 INFO:tasks.rados.rados.0.smithi162.stdout:3325: writing smithi16234619-9 from 1670539 to 2338928 tid 2 2024-09-16T15:15:41.224 INFO:tasks.rados.rados.0.smithi162.stdout:3325: writing smithi16234619-9 from 3041747 to 3344857 tid 3 2024-09-16T15:15:41.224 INFO:tasks.rados.rados.0.smithi162.stdout:3326: snap_remove snap 276 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3326: done (10 left) 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3327: read oid 18 snap -1 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3327: expect deleted 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3328: read oid 17 snap -1 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3328: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3329: read oid 45 snap 333 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3329: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3330: delete oid 3 current snap is 343 2024-09-16T15:15:42.207 INFO:tasks.rados.rados.0.smithi162.stdout:3319: finishing copy_from to smithi16234619-10 2024-09-16T15:15:42.208 INFO:tasks.rados.rados.0.smithi162.stdout:3319: got expected ENOENT (src dne) 2024-09-16T15:15:42.208 INFO:tasks.rados.rados.0.smithi162.stdout:3319: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:15:42.208 INFO:tasks.rados.rados.0.smithi162.stdout:3320: expect (ObjNum 827 snap 224 seq_num 827) 2024-09-16T15:15:42.338 INFO:tasks.rados.rados.0.smithi162.stdout:3316: expect (ObjNum 1054 snap 313 seq_num 1054) 2024-09-16T15:15:42.606 INFO:tasks.rados.rados.0.smithi162.stdout:3321: expect (ObjNum 1062 snap 317 seq_num 1062) 2024-09-16T15:15:42.832 INFO:tasks.rados.rados.0.smithi162.stdout:3322: expect (ObjNum 1057 snap 316 seq_num 1057) 2024-09-16T15:15:43.048 INFO:tasks.rados.rados.0.smithi162.stdout:3318: expect (ObjNum 1089 snap 326 seq_num 1089) 2024-09-16T15:15:43.124 INFO:tasks.rados.rados.0.smithi162.stdout:3324: expect (ObjNum 906 snap 250 seq_num 906) 2024-09-16T15:15:43.231 INFO:tasks.rados.rados.0.smithi162.stdout:3317: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:15:43.231 INFO:tasks.rados.rados.0.smithi162.stdout:3325: finishing write tid 1 to smithi16234619-9 2024-09-16T15:15:43.231 INFO:tasks.rados.rados.0.smithi162.stdout:3323: finishing copy_from to smithi16234619-38 2024-09-16T15:15:43.231 INFO:tasks.rados.rados.0.smithi162.stdout:3323: got expected ENOENT (src dne) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3323: finishing copy_from racing read to smithi16234619-38 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3317: finishing copy_from to smithi16234619-48 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1563 (ObjNum 1115 snap 333 seq_num 1115) dirty exists 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3325: finishing write tid 2 to smithi16234619-9 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3325: finishing write tid 3 to smithi16234619-9 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3325: finishing write tid 4 to smithi16234619-9 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3316: done (13 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3317: done (12 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3318: done (11 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3319: done (10 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3320: done (9 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3321: done (8 left) 2024-09-16T15:15:43.232 INFO:tasks.rados.rados.0.smithi162.stdout:3322: done (7 left) 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3323: done (6 left) 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3324: done (5 left) 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3330: done (4 left) 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3331: write (excl) oid 44 current snap is 343 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: finishing write tid 5 to smithi16234619-9 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: oid 9 updating version 0 to 1737 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: oid 9 version 1737 is already newer than 1735 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: oid 9 version 1737 is already newer than 1734 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: oid 9 version 1737 is already newer than 1736 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1737 (ObjNum 1151 snap 343 seq_num 1151) dirty exists 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3325: left oid 9 (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:15:43.233 INFO:tasks.rados.rados.0.smithi162.stdout:3331: seq_num 1153 ranges {578471=731400,1787503=672058,3096593=281878} 2024-09-16T15:15:43.237 INFO:tasks.rados.rados.0.smithi162.stdout:3331: writing smithi16234619-44 from 578471 to 1309871 tid 1 2024-09-16T15:15:43.244 INFO:tasks.rados.rados.0.smithi162.stdout:3331: writing smithi16234619-44 from 1787503 to 2459561 tid 2 2024-09-16T15:15:43.249 INFO:tasks.rados.rados.0.smithi162.stdout:3331: writing smithi16234619-44 from 3096593 to 3378471 tid 3 2024-09-16T15:15:43.249 INFO:tasks.rados.rados.0.smithi162.stdout:3325: done (4 left) 2024-09-16T15:15:43.249 INFO:tasks.rados.rados.0.smithi162.stdout:3332: snap_remove snap 330 2024-09-16T15:15:44.223 INFO:tasks.rados.rados.0.smithi162.stdout:3332: done (4 left) 2024-09-16T15:15:44.223 INFO:tasks.rados.rados.0.smithi162.stdout:3333: snap_create 2024-09-16T15:15:44.223 INFO:tasks.rados.rados.0.smithi162.stdout:3328: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:15:44.322 INFO:tasks.rados.rados.0.smithi162.stdout:3329: expect (ObjNum 1021 snap 297 seq_num 1021) 2024-09-16T15:15:44.427 INFO:tasks.rados.rados.0.smithi162.stdout:3331: finishing write tid 1 to smithi16234619-44 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: finishing write tid 2 to smithi16234619-44 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: finishing write tid 3 to smithi16234619-44 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: finishing write tid 4 to smithi16234619-44 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: finishing write tid 5 to smithi16234619-44 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: oid 44 updating version 0 to 1799 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: oid 44 updating version 1799 to 1802 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: oid 44 version 1802 is already newer than 1800 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: oid 44 version 1802 is already newer than 1801 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1802 (ObjNum 1153 snap 343 seq_num 1153) dirty exists 2024-09-16T15:15:44.428 INFO:tasks.rados.rados.0.smithi162.stdout:3331: left oid 44 (ObjNum 1153 snap 343 seq_num 1153) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3327: done (4 left) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3328: done (3 left) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3329: done (2 left) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3331: done (1 left) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3333: done (0 left) 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3334: copy_from oid 26 from oid 35 current snap is 344 2024-09-16T15:15:45.241 INFO:tasks.rados.rados.0.smithi162.stdout:3335: snap_create 2024-09-16T15:15:45.295 INFO:tasks.rados.rados.0.smithi162.stdout:3334: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:15:45.300 INFO:tasks.rados.rados.0.smithi162.stdout:3334: finishing copy_from to smithi16234619-26 2024-09-16T15:15:45.300 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1360 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:15:46.251 INFO:tasks.rados.rados.0.smithi162.stdout:3334: done (1 left) 2024-09-16T15:15:46.251 INFO:tasks.rados.rados.0.smithi162.stdout:3335: done (0 left) 2024-09-16T15:15:46.251 INFO:tasks.rados.rados.0.smithi162.stdout:3336: delete oid 45 current snap is 345 2024-09-16T15:15:46.254 INFO:tasks.rados.rados.0.smithi162.stdout:3336: done (0 left) 2024-09-16T15:15:46.254 INFO:tasks.rados.rados.0.smithi162.stdout:3337: read oid 9 snap 343 2024-09-16T15:15:46.254 INFO:tasks.rados.rados.0.smithi162.stdout:3337: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:15:46.254 INFO:tasks.rados.rados.0.smithi162.stdout:3338: snap_remove snap 249 2024-09-16T15:15:47.575 INFO:tasks.rados.rados.0.smithi162.stdout:3338: done (1 left) 2024-09-16T15:15:47.575 INFO:tasks.rados.rados.0.smithi162.stdout:3339: read oid 16 snap -1 2024-09-16T15:15:47.575 INFO:tasks.rados.rados.0.smithi162.stdout:3339: expect (ObjNum 1142 snap 341 seq_num 1142) 2024-09-16T15:15:47.575 INFO:tasks.rados.rados.0.smithi162.stdout:3340: write (excl) oid 7 current snap is 345 2024-09-16T15:15:47.575 INFO:tasks.rados.rados.0.smithi162.stdout:3340: seq_num 1155 ranges {612085=481898,1788114=675727,3035086=376999} 2024-09-16T15:15:47.580 INFO:tasks.rados.rados.0.smithi162.stdout:3340: writing smithi16234619-7 from 612085 to 1093983 tid 1 2024-09-16T15:15:47.589 INFO:tasks.rados.rados.0.smithi162.stdout:3340: writing smithi16234619-7 from 1788114 to 2463841 tid 2 2024-09-16T15:15:47.594 INFO:tasks.rados.rados.0.smithi162.stdout:3340: writing smithi16234619-7 from 3035086 to 3412085 tid 3 2024-09-16T15:15:47.594 INFO:tasks.rados.rados.0.smithi162.stdout:3341: copy_from oid 43 from oid 18 current snap is 345 2024-09-16T15:15:47.595 INFO:tasks.rados.rados.0.smithi162.stdout:3337: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:15:47.836 INFO:tasks.rados.rados.0.smithi162.stdout:3339: expect (ObjNum 1142 snap 341 seq_num 1142) 2024-09-16T15:15:47.953 INFO:tasks.rados.rados.0.smithi162.stdout:3340: finishing write tid 1 to smithi16234619-7 2024-09-16T15:15:47.953 INFO:tasks.rados.rados.0.smithi162.stdout:3340: finishing write tid 2 to smithi16234619-7 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3337: done (3 left) 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3339: done (2 left) 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3342: copy_from oid 9 from oid 24 current snap is 345 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: finishing write tid 3 to smithi16234619-7 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: finishing write tid 4 to smithi16234619-7 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: finishing write tid 5 to smithi16234619-7 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: oid 7 updating version 0 to 1495 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: oid 7 version 1495 is already newer than 1492 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: oid 7 version 1495 is already newer than 1494 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: oid 7 version 1495 is already newer than 1493 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1495 (ObjNum 1155 snap 345 seq_num 1155) dirty exists 2024-09-16T15:15:47.954 INFO:tasks.rados.rados.0.smithi162.stdout:3340: left oid 7 (ObjNum 1155 snap 345 seq_num 1155) 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3341: finishing copy_from to smithi16234619-43 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3341: got expected ENOENT (src dne) 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3341: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3340: done (2 left) 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3341: done (1 left) 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3343: read oid 38 snap 326 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3343: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:47.955 INFO:tasks.rados.rados.0.smithi162.stdout:3344: snap_remove snap 328 2024-09-16T15:15:48.926 INFO:tasks.rados.rados.0.smithi162.stdout:3344: done (2 left) 2024-09-16T15:15:48.926 INFO:tasks.rados.rados.0.smithi162.stdout:3345: snap_remove snap 341 2024-09-16T15:15:49.927 INFO:tasks.rados.rados.0.smithi162.stdout:3345: done (2 left) 2024-09-16T15:15:49.927 INFO:tasks.rados.rados.0.smithi162.stdout:3346: snap_remove snap 334 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3346: done (2 left) 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3347: read oid 23 snap -1 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3347: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3348: delete oid 31 current snap is 345 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3342: finishing copy_from to smithi16234619-9 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3342: got expected ENOENT (src dne) 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3342: finishing copy_from racing read to smithi16234619-9 2024-09-16T15:15:50.930 INFO:tasks.rados.rados.0.smithi162.stdout:3343: expect (ObjNum 1063 snap 318 seq_num 1063) 2024-09-16T15:15:51.126 INFO:tasks.rados.rados.0.smithi162.stdout:3347: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:15:51.241 INFO:tasks.rados.rados.0.smithi162.stdout:3342: done (3 left) 2024-09-16T15:15:51.241 INFO:tasks.rados.rados.0.smithi162.stdout:3343: done (2 left) 2024-09-16T15:15:51.241 INFO:tasks.rados.rados.0.smithi162.stdout:3347: done (1 left) 2024-09-16T15:15:51.241 INFO:tasks.rados.rados.0.smithi162.stdout:3348: done (0 left) 2024-09-16T15:15:51.241 INFO:tasks.rados.rados.0.smithi162.stdout:3349: snap_remove snap 309 2024-09-16T15:15:51.934 INFO:tasks.rados.rados.0.smithi162.stdout:3349: done (0 left) 2024-09-16T15:15:51.934 INFO:tasks.rados.rados.0.smithi162.stdout:3350: write oid 29 current snap is 345 2024-09-16T15:15:51.934 INFO:tasks.rados.rados.0.smithi162.stdout:3350: seq_num 1157 ranges {645699=748749,2021431=679396,3206285=239414} 2024-09-16T15:15:51.942 INFO:tasks.rados.rados.0.smithi162.stdout:3350: writing smithi16234619-29 from 645699 to 1394448 tid 1 2024-09-16T15:15:51.951 INFO:tasks.rados.rados.0.smithi162.stdout:3350: writing smithi16234619-29 from 2021431 to 2700827 tid 2 2024-09-16T15:15:51.956 INFO:tasks.rados.rados.0.smithi162.stdout:3350: writing smithi16234619-29 from 3206285 to 3445699 tid 3 2024-09-16T15:15:51.956 INFO:tasks.rados.rados.0.smithi162.stdout:3351: snap_remove snap 333 2024-09-16T15:15:52.908 INFO:tasks.rados.rados.0.smithi162.stdout:3351: done (1 left) 2024-09-16T15:15:52.908 INFO:tasks.rados.rados.0.smithi162.stdout:3352: write oid 47 current snap is 345 2024-09-16T15:15:52.908 INFO:tasks.rados.rados.0.smithi162.stdout:3352: seq_num 1158 ranges {662506=423998,1563560=539407,2717355=490285} 2024-09-16T15:15:52.915 INFO:tasks.rados.rados.0.smithi162.stdout:3352: writing smithi16234619-47 from 662506 to 1086504 tid 1 2024-09-16T15:15:52.922 INFO:tasks.rados.rados.0.smithi162.stdout:3352: writing smithi16234619-47 from 1563560 to 2102967 tid 2 2024-09-16T15:15:52.929 INFO:tasks.rados.rados.0.smithi162.stdout:3352: writing smithi16234619-47 from 2717355 to 3207640 tid 3 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3353: rollback oid 11 current snap is 345 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 343 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: finishing write tid 1 to smithi16234619-29 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: finishing write tid 2 to smithi16234619-29 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: finishing write tid 3 to smithi16234619-29 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: finishing write tid 4 to smithi16234619-29 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: finishing write tid 5 to smithi16234619-29 2024-09-16T15:15:52.930 INFO:tasks.rados.rados.0.smithi162.stdout:3350: oid 29 updating version 0 to 1812 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3350: oid 29 version 1812 is already newer than 1811 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3350: oid 29 version 1812 is already newer than 1810 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3350: oid 29 version 1812 is already newer than 1809 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1812 (ObjNum 1157 snap 345 seq_num 1157) dirty exists 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3350: left oid 29 (ObjNum 1157 snap 345 seq_num 1157) 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3350: done (2 left) 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3354: write oid 33 current snap is 345 2024-09-16T15:15:52.931 INFO:tasks.rados.rados.0.smithi162.stdout:3354: seq_num 1159 ranges {679313=499247,1622042=683065,2744778=701557} 2024-09-16T15:15:52.937 INFO:tasks.rados.rados.0.smithi162.stdout:3354: writing smithi16234619-33 from 679313 to 1178560 tid 1 2024-09-16T15:15:52.944 INFO:tasks.rados.rados.0.smithi162.stdout:3354: writing smithi16234619-33 from 1622042 to 2305107 tid 2 2024-09-16T15:15:52.951 INFO:tasks.rados.rados.0.smithi162.stdout:3354: writing smithi16234619-33 from 2744778 to 3446335 tid 3 2024-09-16T15:15:52.951 INFO:tasks.rados.rados.0.smithi162.stdout:3355: write (excl) oid 6 current snap is 345 2024-09-16T15:15:52.951 INFO:tasks.rados.rados.0.smithi162.stdout:3355: seq_num 1160 ranges {696120=574496,1680524=543076,2888554=512829} 2024-09-16T15:15:52.958 INFO:tasks.rados.rados.0.smithi162.stdout:3355: writing smithi16234619-6 from 696120 to 1270616 tid 1 2024-09-16T15:15:52.964 INFO:tasks.rados.rados.0.smithi162.stdout:3355: writing smithi16234619-6 from 1680524 to 2223600 tid 2 2024-09-16T15:15:52.972 INFO:tasks.rados.rados.0.smithi162.stdout:3355: writing smithi16234619-6 from 2888554 to 3401383 tid 3 2024-09-16T15:15:52.972 INFO:tasks.rados.rados.0.smithi162.stdout:3356: snap_create 2024-09-16T15:15:52.972 INFO:tasks.rados.rados.0.smithi162.stdout:3353: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:15:52.972 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1281 (ObjNum 129 snap 0 seq_num 865817296) dirty dne 2024-09-16T15:15:52.975 INFO:tasks.rados.rados.0.smithi162.stdout:3352: finishing write tid 1 to smithi16234619-47 2024-09-16T15:15:52.975 INFO:tasks.rados.rados.0.smithi162.stdout:3352: finishing write tid 2 to smithi16234619-47 2024-09-16T15:15:52.975 INFO:tasks.rados.rados.0.smithi162.stdout:3352: finishing write tid 3 to smithi16234619-47 2024-09-16T15:15:52.975 INFO:tasks.rados.rados.0.smithi162.stdout:3352: finishing write tid 4 to smithi16234619-47 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: finishing write tid 5 to smithi16234619-47 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: oid 47 updating version 0 to 1172 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: oid 47 version 1172 is already newer than 1169 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: oid 47 version 1172 is already newer than 1171 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: oid 47 version 1172 is already newer than 1170 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1172 (ObjNum 1158 snap 345 seq_num 1158) dirty exists 2024-09-16T15:15:52.976 INFO:tasks.rados.rados.0.smithi162.stdout:3352: left oid 47 (ObjNum 1158 snap 345 seq_num 1158) 2024-09-16T15:15:52.996 INFO:tasks.rados.rados.0.smithi162.stdout:3354: finishing write tid 1 to smithi16234619-33 2024-09-16T15:15:52.999 INFO:tasks.rados.rados.0.smithi162.stdout:3355: finishing write tid 1 to smithi16234619-6 2024-09-16T15:15:53.004 INFO:tasks.rados.rados.0.smithi162.stdout:3354: finishing write tid 2 to smithi16234619-33 2024-09-16T15:15:53.005 INFO:tasks.rados.rados.0.smithi162.stdout:3354: finishing write tid 3 to smithi16234619-33 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: finishing write tid 4 to smithi16234619-33 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: finishing write tid 5 to smithi16234619-33 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: oid 33 updating version 0 to 1710 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: oid 33 version 1710 is already newer than 1708 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: oid 33 version 1710 is already newer than 1709 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: oid 33 version 1710 is already newer than 1707 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1710 (ObjNum 1159 snap 345 seq_num 1159) dirty exists 2024-09-16T15:15:53.006 INFO:tasks.rados.rados.0.smithi162.stdout:3354: left oid 33 (ObjNum 1159 snap 345 seq_num 1159) 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: finishing write tid 2 to smithi16234619-6 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: finishing write tid 3 to smithi16234619-6 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: finishing write tid 4 to smithi16234619-6 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: finishing write tid 5 to smithi16234619-6 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: oid 6 updating version 0 to 1806 2024-09-16T15:15:53.011 INFO:tasks.rados.rados.0.smithi162.stdout:3355: oid 6 version 1806 is already newer than 1803 2024-09-16T15:15:53.012 INFO:tasks.rados.rados.0.smithi162.stdout:3355: oid 6 version 1806 is already newer than 1805 2024-09-16T15:15:53.012 INFO:tasks.rados.rados.0.smithi162.stdout:3355: oid 6 version 1806 is already newer than 1804 2024-09-16T15:15:53.012 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1806 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:15:53.012 INFO:tasks.rados.rados.0.smithi162.stdout:3355: left oid 6 (ObjNum 1160 snap 345 seq_num 1160) 2024-09-16T15:15:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3352: done (4 left) 2024-09-16T15:15:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3353: done (3 left) 2024-09-16T15:15:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3354: done (2 left) 2024-09-16T15:15:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3355: done (1 left) 2024-09-16T15:15:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3356: done (0 left) 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:3357: copy_from oid 40 from oid 32 current snap is 346 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:3358: rollback oid 8 current snap is 346 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 345 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:3359: rollback oid 11 current snap is 346 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 340 2024-09-16T15:15:53.939 INFO:tasks.rados.rados.0.smithi162.stdout:3360: snap_remove snap 344 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3360: done (3 left) 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3361: delete oid 43 current snap is 346 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3359: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1281 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3358: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1491 (ObjNum 433 snap 0 seq_num 2027921024) dirty dne 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3357: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:3357: finishing copy_from to smithi16234619-40 2024-09-16T15:15:54.949 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1461 (ObjNum 1116 snap 333 seq_num 1116) dirty exists 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3357: done (3 left) 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3358: done (2 left) 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3359: done (1 left) 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3361: done (0 left) 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3362: write (excl) oid 39 current snap is 346 2024-09-16T15:15:54.955 INFO:tasks.rados.rados.0.smithi162.stdout:3362: seq_num 1162 ranges {729734=724994,2197488=430392,3227047=302687} 2024-09-16T15:15:54.963 INFO:tasks.rados.rados.0.smithi162.stdout:3362: writing smithi16234619-39 from 729734 to 1454728 tid 1 2024-09-16T15:15:54.969 INFO:tasks.rados.rados.0.smithi162.stdout:3362: writing smithi16234619-39 from 2197488 to 2627880 tid 2 2024-09-16T15:15:54.974 INFO:tasks.rados.rados.0.smithi162.stdout:3362: writing smithi16234619-39 from 3227047 to 3529734 tid 3 2024-09-16T15:15:54.974 INFO:tasks.rados.rados.0.smithi162.stdout:3363: copy_from oid 7 from oid 23 current snap is 346 2024-09-16T15:15:54.975 INFO:tasks.rados.rados.0.smithi162.stdout:3364: read oid 24 snap -1 2024-09-16T15:15:54.975 INFO:tasks.rados.rados.0.smithi162.stdout:3364: expect deleted 2024-09-16T15:15:54.975 INFO:tasks.rados.rados.0.smithi162.stdout:3365: write oid 15 current snap is 346 2024-09-16T15:15:54.975 INFO:tasks.rados.rados.0.smithi162.stdout:3365: seq_num 1163 ranges {746541=400243,1739617=690403,3138117=408424} 2024-09-16T15:15:54.981 INFO:tasks.rados.rados.0.smithi162.stdout:3365: writing smithi16234619-15 from 746541 to 1146784 tid 1 2024-09-16T15:15:54.988 INFO:tasks.rados.rados.0.smithi162.stdout:3365: writing smithi16234619-15 from 1739617 to 2430020 tid 2 2024-09-16T15:15:54.994 INFO:tasks.rados.rados.0.smithi162.stdout:3365: writing smithi16234619-15 from 3138117 to 3546541 tid 3 2024-09-16T15:15:54.994 INFO:tasks.rados.rados.0.smithi162.stdout:3366: snap_remove snap 343 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3366: done (4 left) 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: finishing write tid 1 to smithi16234619-39 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: finishing write tid 2 to smithi16234619-39 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: finishing write tid 3 to smithi16234619-39 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: finishing write tid 4 to smithi16234619-39 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: finishing write tid 5 to smithi16234619-39 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: oid 39 updating version 0 to 1828 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: oid 39 updating version 1828 to 1830 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: oid 39 updating version 1830 to 1831 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:3362: oid 39 version 1831 is already newer than 1829 2024-09-16T15:15:55.952 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 1831 (ObjNum 1162 snap 346 seq_num 1162) dirty exists 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3362: left oid 39 (ObjNum 1162 snap 346 seq_num 1162) 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: finishing write tid 1 to smithi16234619-15 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: finishing write tid 2 to smithi16234619-15 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: finishing write tid 3 to smithi16234619-15 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: finishing write tid 4 to smithi16234619-15 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: finishing write tid 5 to smithi16234619-15 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: oid 15 updating version 0 to 1414 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: oid 15 updating version 1414 to 1416 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: oid 15 updating version 1416 to 1417 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: oid 15 version 1417 is already newer than 1415 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 1417 (ObjNum 1163 snap 346 seq_num 1163) dirty exists 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3365: left oid 15 (ObjNum 1163 snap 346 seq_num 1163) 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3363: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:15:55.953 INFO:tasks.rados.rados.0.smithi162.stdout:3363: finishing copy_from to smithi16234619-7 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1480 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3367: read oid 13 snap -1 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3367: expect (ObjNum 1133 snap 339 seq_num 1133) 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3362: done (4 left) 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3363: done (3 left) 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3364: done (2 left) 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3365: done (1 left) 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3368: copy_from oid 28 from oid 3 current snap is 346 2024-09-16T15:15:55.954 INFO:tasks.rados.rados.0.smithi162.stdout:3369: delete oid 8 current snap is 346 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:3369: done (2 left) 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:3370: rollback oid 32 current snap is 346 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 329 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:3371: rollback oid 42 current snap is 346 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 326 2024-09-16T15:15:55.955 INFO:tasks.rados.rados.0.smithi162.stdout:3372: read oid 40 snap -1 2024-09-16T15:15:55.956 INFO:tasks.rados.rados.0.smithi162.stdout:3372: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:15:55.956 INFO:tasks.rados.rados.0.smithi162.stdout:3373: rollback oid 2 current snap is 346 2024-09-16T15:15:55.956 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 338 2024-09-16T15:15:55.957 INFO:tasks.rados.rados.0.smithi162.stdout:3374: snap_create 2024-09-16T15:15:55.957 INFO:tasks.rados.rados.0.smithi162.stdout:3368: finishing copy_from to smithi16234619-28 2024-09-16T15:15:55.957 INFO:tasks.rados.rados.0.smithi162.stdout:3368: got expected ENOENT (src dne) 2024-09-16T15:15:55.957 INFO:tasks.rados.rados.0.smithi162.stdout:3368: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:15:55.959 INFO:tasks.rados.rados.0.smithi162.stdout:3367: expect (ObjNum 1133 snap 339 seq_num 1133) 2024-09-16T15:15:56.152 INFO:tasks.rados.rados.0.smithi162.stdout:3370: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:15:56.152 INFO:tasks.rados.rados.0.smithi162.stdout:3373: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:15:56.152 INFO:tasks.rados.rados.0.smithi162.stdout:3371: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:15:56.152 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1588 (ObjNum 305 snap 0 seq_num 1580415163) dirty dne 2024-09-16T15:15:56.152 INFO:tasks.rados.rados.0.smithi162.stdout:3373: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:15:56.153 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2037 (ObjNum 1097 snap 328 seq_num 1097) dirty exists 2024-09-16T15:15:56.153 INFO:tasks.rados.rados.0.smithi162.stdout:3370: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:15:56.153 INFO:tasks.rados.rados.0.smithi162.stdout:3370: finishing rollback tid 2 to smithi16234619-32 2024-09-16T15:15:56.153 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1472 (ObjNum 1096 snap 328 seq_num 1096) dirty exists 2024-09-16T15:15:56.153 INFO:tasks.rados.rados.0.smithi162.stdout:3372: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:15:56.968 INFO:tasks.rados.rados.0.smithi162.stdout:3367: done (6 left) 2024-09-16T15:15:56.968 INFO:tasks.rados.rados.0.smithi162.stdout:3368: done (5 left) 2024-09-16T15:15:56.968 INFO:tasks.rados.rados.0.smithi162.stdout:3370: done (4 left) 2024-09-16T15:15:56.968 INFO:tasks.rados.rados.0.smithi162.stdout:3371: done (3 left) 2024-09-16T15:15:56.969 INFO:tasks.rados.rados.0.smithi162.stdout:3372: done (2 left) 2024-09-16T15:15:56.969 INFO:tasks.rados.rados.0.smithi162.stdout:3373: done (1 left) 2024-09-16T15:15:56.969 INFO:tasks.rados.rados.0.smithi162.stdout:3374: done (0 left) 2024-09-16T15:15:56.969 INFO:tasks.rados.rados.0.smithi162.stdout:3375: snap_remove snap 331 2024-09-16T15:15:57.907 INFO:tasks.rados.rados.0.smithi162.stdout:3375: done (0 left) 2024-09-16T15:15:57.907 INFO:tasks.rados.rados.0.smithi162.stdout:3376: delete oid 28 current snap is 347 2024-09-16T15:15:57.915 INFO:tasks.rados.rados.0.smithi162.stdout:3376: done (0 left) 2024-09-16T15:15:57.915 INFO:tasks.rados.rados.0.smithi162.stdout:3377: rollback oid 44 current snap is 347 2024-09-16T15:15:57.915 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 338 2024-09-16T15:15:57.915 INFO:tasks.rados.rados.0.smithi162.stdout:3378: snap_create 2024-09-16T15:15:57.921 INFO:tasks.rados.rados.0.smithi162.stdout:3377: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:15:57.922 INFO:tasks.rados.rados.0.smithi162.stdout:3377: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:15:57.922 INFO:tasks.rados.rados.0.smithi162.stdout:3377: finishing rollback tid 2 to smithi16234619-44 2024-09-16T15:15:57.922 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1809 (ObjNum 1057 snap 316 seq_num 1057) dirty exists 2024-09-16T15:15:59.247 INFO:tasks.rados.rados.0.smithi162.stdout:3377: done (1 left) 2024-09-16T15:15:59.247 INFO:tasks.rados.rados.0.smithi162.stdout:3378: done (0 left) 2024-09-16T15:15:59.247 INFO:tasks.rados.rados.0.smithi162.stdout:3379: write oid 44 current snap is 348 2024-09-16T15:15:59.247 INFO:tasks.rados.rados.0.smithi162.stdout:3379: seq_num 1166 ranges {796962=742343,2031416=437730,2936739=413167} 2024-09-16T15:15:59.254 INFO:tasks.rados.rados.0.smithi162.stdout:3379: writing smithi16234619-44 from 796962 to 1539305 tid 1 2024-09-16T15:15:59.260 INFO:tasks.rados.rados.0.smithi162.stdout:3379: writing smithi16234619-44 from 2031416 to 2469146 tid 2 2024-09-16T15:15:59.266 INFO:tasks.rados.rados.0.smithi162.stdout:3379: writing smithi16234619-44 from 2936739 to 3349906 tid 3 2024-09-16T15:15:59.266 INFO:tasks.rados.rados.0.smithi162.stdout:3380: snap_remove snap 347 2024-09-16T15:16:00.251 INFO:tasks.rados.rados.0.smithi162.stdout:3380: done (1 left) 2024-09-16T15:16:00.251 INFO:tasks.rados.rados.0.smithi162.stdout:3381: rollback oid 16 current snap is 348 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 339 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: finishing write tid 1 to smithi16234619-44 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: finishing write tid 2 to smithi16234619-44 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: finishing write tid 3 to smithi16234619-44 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: finishing write tid 4 to smithi16234619-44 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: finishing write tid 5 to smithi16234619-44 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: oid 44 updating version 0 to 1812 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: oid 44 version 1812 is already newer than 1810 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: oid 44 updating version 1812 to 1813 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: oid 44 version 1813 is already newer than 1811 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1813 (ObjNum 1166 snap 348 seq_num 1166) dirty exists 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: left oid 44 (ObjNum 1166 snap 348 seq_num 1166) 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3379: done (1 left) 2024-09-16T15:16:00.252 INFO:tasks.rados.rados.0.smithi162.stdout:3382: snap_remove snap 329 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:3382: done (1 left) 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:3383: copy_from oid 33 from oid 36 current snap is 348 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:3381: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:3381: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1815 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:16:01.752 INFO:tasks.rados.rados.0.smithi162.stdout:3381: done (1 left) 2024-09-16T15:16:01.753 INFO:tasks.rados.rados.0.smithi162.stdout:3384: snap_remove snap 285 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:3383: finishing copy_from racing read to smithi16234619-33 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:3383: finishing copy_from to smithi16234619-33 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1577 (ObjNum 1119 snap 334 seq_num 1119) dirty exists 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:3383: done (1 left) 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:3384: done (0 left) 2024-09-16T15:16:03.009 INFO:tasks.rados.rados.0.smithi162.stdout:3385: rollback oid 10 current snap is 348 2024-09-16T15:16:03.010 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 10 to 336 2024-09-16T15:16:03.010 INFO:tasks.rados.rados.0.smithi162.stdout:3386: snap_create 2024-09-16T15:16:03.023 INFO:tasks.rados.rados.0.smithi162.stdout:3385: finishing rollback tid 0 to smithi16234619-10 2024-09-16T15:16:03.023 INFO:tasks.rados.rados.0.smithi162.stdout:3385: finishing rollback tid 1 to smithi16234619-10 2024-09-16T15:16:03.028 INFO:tasks.rados.rados.0.smithi162.stdout:3385: finishing rollback tid 2 to smithi16234619-10 2024-09-16T15:16:03.028 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 2057 (ObjNum 1077 snap 321 seq_num 1077) dirty exists 2024-09-16T15:16:03.886 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:16:03.886 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-16T15:16:03.886 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-16T15:16:03.886 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-16T15:16:03.890 INFO:tasks.ceph.mon.i:Started 2024-09-16T15:16:03.890 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:16:03.890 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-16T15:16:03.890 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:16:03.890 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:16:03.893 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:16:03.893 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-16T15:16:03.893 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-16T15:16:03.893 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:16:03.893 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:16:03.895 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:16:03.895 INFO:tasks.mon_thrash:Sending STOP to mon d 2024-09-16T15:16:03.895 INFO:tasks.ceph.mon.d:Sent signal 19 2024-09-16T15:16:03.896 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-16T15:16:18.897 INFO:tasks.mon_thrash:Sending CONT to mon d 2024-09-16T15:16:18.897 INFO:tasks.ceph.mon.d:Sent signal 18 2024-09-16T15:16:18.898 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:16:18.898 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.004+0000 7f139ab45640 10 monclient: get_monmap_and_config 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.004+0000 7f139ab45640 10 monclient: build_initial_monmap 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.004+0000 7f139ab45640 10 monclient: monmap: 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:19.005628+0000 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:19.005628+0000 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:19.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:19.007 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:19.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 1 Processor -- start 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 1 -- start start 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 10 monclient: init 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 10 monclient: _add_conns ranks=[8,10,2,1,4,9,5,0,3,6,7] 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-e con 0x7f1394136080 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.005+0000 7f139ab45640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-f con 0x7f13940a5700 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-b con 0x7f13940a44c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139409bb00 con 0x7f13940a5700 2024-09-16T15:16:19.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139409b4b0 con 0x7f13940a44c0 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f139409c6a0 con 0x7f1394136080 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f139ab45640 10 monclient(hunting): authenticate will time out at 1715.595035s 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:50014/0 (socket says 172.21.15.134:50014) 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:42928/0 (socket says 172.21.15.134:42928) 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 learned_addr learned my addr 172.21.15.134:0/3813378462 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 10 monclient(hunting): get_auth_request con 0x7f13940a44c0 auth_method 0 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 10 monclient(hunting): get_auth_request con 0x7f1394136080 auth_method 0 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 10 monclient(hunting): get_auth_request con 0x7f13940a5700 auth_method 0 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.006+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 10 monclient(hunting): get_auth_request con 0x7f13940a44c0 auth_method 0 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 10 monclient(hunting): get_auth_request con 0x7f1394136080 auth_method 0 2024-09-16T15:16:19.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 10 monclient(hunting): get_auth_request con 0x7f13940a5700 auth_method 0 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:19.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:19.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:19.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.207+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 10 monclient(hunting): get_auth_request con 0x7f13940a44c0 auth_method 0 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 10 monclient(hunting): get_auth_request con 0x7f1394136080 auth_method 0 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 10 monclient(hunting): get_auth_request con 0x7f13940a5700 auth_method 0 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:19.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:19.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:19.608+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:20.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): get_auth_request con 0x7f13940a44c0 auth_method 0 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 10 monclient(hunting): get_auth_request con 0x7f1394136080 auth_method 0 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:20.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13988e1640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 10 monclient(hunting): get_auth_request con 0x7f13940a5700 auth_method 0 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient(hunting): handle_auth_done global_id 114104 payload 274 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 msgr2=0x7f1394136460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 msgr2=0x7f13940a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:20.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient: _finish_hunting 0 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 monclient: found mon.noname-b 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394073250 con 0x7f13940a44c0 2024-09-16T15:16:20.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.409+0000 7f1393fff640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:16:20.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.412+0000 7f13990e2640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.412+0000 7f13990e2640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f138c009440 tx=0x7f138c02abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7d91289b94b547 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 1 -- 172.21.15.134:0/3813378462 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f138c003e00 con 0x7f13940a44c0 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: dump: 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:20.416 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:20.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 1 -- 172.21.15.134:0/3813378462 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c02c8e0 con 0x7f13940a44c0 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 1 -- 172.21.15.134:0/3813378462 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f138c004030 con 0x7f13940a44c0 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 5 monclient: authenticate success, global_id 114104 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 4 set_mon_vals no callback set 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f13937fe640 10 monclient: dump: 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.418 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 10 monclient: shutdown 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 -- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f13940a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f138c009440 tx=0x7f138c02abb0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 -- 172.21.15.134:0/3813378462 shutdown_connections 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f13940a4920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 --2- 172.21.15.134:0/3813378462 >> v2:172.21.15.162:3300/0 conn(0x7f13940a5700 0x7f13940a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 --2- 172.21.15.134:0/3813378462 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f1394136080 0x7f1394136460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.413+0000 7f139ab45640 1 -- 172.21.15.134:0/3813378462 >> 172.21.15.134:0/3813378462 conn(0x7f13940a99a0 msgr2=0x7f13940a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.414+0000 7f139ab45640 1 -- 172.21.15.134:0/3813378462 shutdown_connections 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.414+0000 7f139ab45640 1 -- 172.21.15.134:0/3813378462 wait complete. 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.414+0000 7f139ab45640 10 monclient: build_initial_monmap 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.414+0000 7f139ab45640 10 monclient: monmap: 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:20.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:20.421 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.415+0000 7f139ab45640 1 Processor -- start 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.415+0000 7f139ab45640 1 -- start start 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.415+0000 7f139ab45640 10 monclient: init 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient: _add_conns ranks=[2,8,4,3,6,7,5,0,1] 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-b con 0x7f13940a44c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:20.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-e con 0x7f13940a5700 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 0x7f1394135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): picked mon.noname-c con 0x7f1394136080 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394122630 con 0x7f13940a44c0 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1394137670 con 0x7f1394136080 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1394122070 con 0x7f13940a5700 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f139ab45640 10 monclient(hunting): authenticate will time out at 1717.005299s 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:50054/0 (socket says 172.21.15.134:50054) 2024-09-16T15:16:20.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 -- 172.21.15.134:0/3883734938 learned_addr learned my addr 172.21.15.134:0/3883734938 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): get_auth_request con 0x7f13940a44c0 auth_method 0 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 10 monclient(hunting): get_auth_request con 0x7f13940a5700 auth_method 0 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 0x7f1394135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): get_auth_request con 0x7f1394136080 auth_method 0 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:20.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 msgr2=0x7f1394134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 msgr2=0x7f1394134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f1393fff640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient(hunting): handle_auth_done global_id 114113 payload 824 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 msgr2=0x7f1394134f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 msgr2=0x7f1394135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:20.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 0x7f1394135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient: _finish_hunting 0 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 monclient: found mon.noname-b 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13988e1640 1 -- 172.21.15.134:0/3883734938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0097e0 con 0x7f13940a44c0 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.416+0000 7f13990e2640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 0x7f1394135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13988e1640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f139409f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13988e1640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1388010010 tx=0x7f1388013bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1388011ec0 con 0x7f13940a44c0 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: dump: 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:20.429 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:20.430 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1388016460 con 0x7f13940a44c0 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 5 monclient: authenticate success, global_id 114113 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1388016a40 con 0x7f13940a44c0 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 10 monclient: _renew_subs 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394122350 con 0x7f13940a44c0 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f13917fa640 10 monclient: dump: 2024-09-16T15:16:20.431 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:20.432 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f1392ffd640 4 set_mon_vals no callback set 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 10 monclient: _renew_subs 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.419+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139409fe20 con 0x7f13940a44c0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1388016c60 con 0x7f13940a44c0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f13917fa640 1 --2- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 0x7f137803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f139ab45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(714..714 src has 1..714) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f138804a8f0 con 0x7f13940a44c0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f139ab45640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f139ab45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.420+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394061ae0 con 0x7f13940a44c0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.421+0000 7f1393fff640 1 --2- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 0x7f137803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.421+0000 7f1393fff640 10 monclient: get_auth_request con 0x7f1378038880 auth_method 0 2024-09-16T15:16:20.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.423+0000 7f1393fff640 1 --2- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 0x7f137803ad40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f139409f980 tx=0x7f1380008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:20.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.425+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1394061ae0 con 0x7f13940a44c0 2024-09-16T15:16:20.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.425+0000 7f13917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:20.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.425+0000 7f13917fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:20.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.653+0000 7f139ab45640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:20.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.653+0000 7f139ab45640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:20.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.653+0000 7f139ab45640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:20.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.653+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1394122850 con 0x7f13940a44c0 2024-09-16T15:16:20.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.654+0000 7f13917fa640 1 -- 172.21.15.134:0/3883734938 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3190 (secure 0 0 0) 0x7f1394122850 con 0x7f13940a44c0 2024-09-16T15:16:20.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.654+0000 7f13917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:20.656 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.654+0000 7f13917fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:20.656 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":128,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 msgr2=0x7f137803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 0x7f137803ad40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f139409f980 tx=0x7f1380008040 comp rx=0 tx=0).stop 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 10 monclient: shutdown 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 msgr2=0x7f139409f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1388010010 tx=0x7f1388013bb0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 shutdown_connections 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> v2:172.21.15.162:6800/980547263 conn(0x7f1378038880 0x7f137803ad40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1394136080 0x7f1394135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f13940a5700 0x7f1394134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 --2- 172.21.15.134:0/3883734938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13940a44c0 0x7f139409f2b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:20.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 >> 172.21.15.134:0/3883734938 conn(0x7f13940a99a0 msgr2=0x7f13941bed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:20.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 shutdown_connections 2024-09-16T15:16:20.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:20.656+0000 7f139ab45640 1 -- 172.21.15.134:0/3883734938 wait complete. 2024-09-16T15:16:21.237 INFO:tasks.rados.rados.0.smithi162.stdout:3385: done (1 left) 2024-09-16T15:16:21.237 INFO:tasks.rados.rados.0.smithi162.stdout:3386: done (0 left) 2024-09-16T15:16:21.237 INFO:tasks.rados.rados.0.smithi162.stdout:3387: write oid 13 current snap is 349 2024-09-16T15:16:21.237 INFO:tasks.rados.rados.0.smithi162.stdout:3387: seq_num 1167 ranges {413769=417592,1573545=697741,2847809=624439} 2024-09-16T15:16:21.241 INFO:tasks.rados.rados.0.smithi162.stdout:3387: writing smithi16234619-13 from 413769 to 831361 tid 1 2024-09-16T15:16:21.249 INFO:tasks.rados.rados.0.smithi162.stdout:3387: writing smithi16234619-13 from 1573545 to 2271286 tid 2 2024-09-16T15:16:21.256 INFO:tasks.rados.rados.0.smithi162.stdout:3387: writing smithi16234619-13 from 2847809 to 3472248 tid 3 2024-09-16T15:16:21.256 INFO:tasks.rados.rados.0.smithi162.stdout:3388: write oid 4 current snap is 349 2024-09-16T15:16:21.256 INFO:tasks.rados.rados.0.smithi162.stdout:3388: seq_num 1168 ranges {430576=492841,1632027=441399,2475232=435711,3365397=265179} 2024-09-16T15:16:21.260 INFO:tasks.rados.rados.0.smithi162.stdout:3388: writing smithi16234619-4 from 430576 to 923417 tid 1 2024-09-16T15:16:21.267 INFO:tasks.rados.rados.0.smithi162.stdout:3388: writing smithi16234619-4 from 1632027 to 2073426 tid 2 2024-09-16T15:16:21.271 INFO:tasks.rados.rados.0.smithi162.stdout:3388: writing smithi16234619-4 from 2475232 to 2910943 tid 3 2024-09-16T15:16:21.275 INFO:tasks.rados.rados.0.smithi162.stdout:3388: writing smithi16234619-4 from 3365397 to 3630576 tid 4 2024-09-16T15:16:21.276 INFO:tasks.rados.rados.0.smithi162.stdout:3389: snap_create 2024-09-16T15:16:21.287 INFO:tasks.rados.rados.0.smithi162.stdout:3387: finishing write tid 1 to smithi16234619-13 2024-09-16T15:16:21.287 INFO:tasks.rados.rados.0.smithi162.stdout:3387: finishing write tid 2 to smithi16234619-13 2024-09-16T15:16:21.287 INFO:tasks.rados.rados.0.smithi162.stdout:3387: finishing write tid 3 to smithi16234619-13 2024-09-16T15:16:21.289 INFO:tasks.rados.rados.0.smithi162.stdout:3387: finishing write tid 4 to smithi16234619-13 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: finishing write tid 5 to smithi16234619-13 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: oid 13 updating version 0 to 1861 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: oid 13 version 1861 is already newer than 1860 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: oid 13 updating version 1861 to 1862 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: oid 13 updating version 1862 to 1863 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1863 (ObjNum 1167 snap 349 seq_num 1167) dirty exists 2024-09-16T15:16:21.290 INFO:tasks.rados.rados.0.smithi162.stdout:3387: left oid 13 (ObjNum 1167 snap 349 seq_num 1167) 2024-09-16T15:16:21.305 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 1 to smithi16234619-4 2024-09-16T15:16:21.305 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 2 to smithi16234619-4 2024-09-16T15:16:21.305 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 3 to smithi16234619-4 2024-09-16T15:16:21.305 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 4 to smithi16234619-4 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 5 to smithi16234619-4 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: finishing write tid 6 to smithi16234619-4 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: oid 4 updating version 0 to 1742 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: oid 4 version 1742 is already newer than 1741 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: oid 4 version 1742 is already newer than 1739 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: oid 4 version 1742 is already newer than 1740 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: oid 4 version 1742 is already newer than 1738 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1742 (ObjNum 1168 snap 349 seq_num 1168) dirty exists 2024-09-16T15:16:21.306 INFO:tasks.rados.rados.0.smithi162.stdout:3388: left oid 4 (ObjNum 1168 snap 349 seq_num 1168) 2024-09-16T15:16:23.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.774+0000 7f4ca7125640 10 monclient: get_monmap_and_config 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.774+0000 7f4ca7125640 10 monclient: build_initial_monmap 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.774+0000 7f4ca7125640 10 monclient: monmap: 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:23.776 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:23.775833+0000 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:23.775833+0000 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:23.777 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:23.778 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:23.778 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:23.778 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:23.778 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:23.778 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:23.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.775+0000 7f4ca7125640 1 Processor -- start 2024-09-16T15:16:23.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.775+0000 7f4ca7125640 1 -- start start 2024-09-16T15:16:23.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.775+0000 7f4ca7125640 10 monclient: init 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient: _add_conns ranks=[5,2,8,1,0,3,4,9,7,6,10] 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-h con 0x7f4ca0136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-b con 0x7f4ca009bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-e con 0x7f4ca0136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca009b600 con 0x7f4ca009bfe0 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a5740 con 0x7f4ca0136030 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a52c0 con 0x7f4ca0136c60 2024-09-16T15:16:23.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca7125640 10 monclient(hunting): authenticate will time out at 1720.365348s 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:42976/0 (socket says 172.21.15.134:42976) 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 learned_addr learned my addr 172.21.15.134:0/4117326625 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4ca0136c60 auth_method 0 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4ca009bfe0 auth_method 0 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4ca0136030 auth_method 0 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:23.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:23.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:23.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.776+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:23.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4ca0136c60 auth_method 0 2024-09-16T15:16:23.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4ca009bfe0 auth_method 0 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:23.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4ca0136030 auth_method 0 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:23.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:23.977+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4ca0136c60 auth_method 0 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:24.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4ca009bfe0 auth_method 0 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4ca0136030 auth_method 0 2024-09-16T15:16:24.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:24.382 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:24.378+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:25.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4ca0136c60 auth_method 0 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4ca009bfe0 auth_method 0 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:25.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4ca0136030 auth_method 0 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:25.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:25.179+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:26.352 INFO:tasks.rados.rados.0.smithi162.stdout:3387: done (2 left) 2024-09-16T15:16:26.352 INFO:tasks.rados.rados.0.smithi162.stdout:3388: done (1 left) 2024-09-16T15:16:26.353 INFO:tasks.rados.rados.0.smithi162.stdout:3389: done (0 left) 2024-09-16T15:16:26.353 INFO:tasks.rados.rados.0.smithi162.stdout:3390: snap_create 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): tick 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 monclient(hunting): continuing hunt 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 msgr2=0x7f4ca01370c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4ca0136c60 0x7f4ca01370c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 msgr2=0x7f4ca0136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:26.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136030 0x7f4ca0136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 msgr2=0x7f4ca009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca009bfe0 0x7f4ca009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient: _add_conns ranks=[4,10,3,0,9,7,1,6] 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4c88005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): picked mon.noname-c con 0x7f4c88003820 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> v2:172.21.15.162:3300/0 conn(0x7f4c88006210 0x7f4c880086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): picked mon.noname-f con 0x7f4c88006210 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c88008d00 0x7f4c8800b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): picked mon.noname-g con 0x7f4c88008d00 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca009b600 con 0x7f4c88006210 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a5740 con 0x7f4c88008d00 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 1 -- 172.21.15.134:0/4117326625 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a52c0 con 0x7f4c88003820 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c967fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4c88005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4c88003820 auth_method 0 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:26.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4117326625 >> v2:172.21.15.162:3300/0 conn(0x7f4c88006210 0x7f4c880086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c88008d00 0x7f4c8800b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.776+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4c88006210 auth_method 0 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4c97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4c88008d00 auth_method 0 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca56c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_done global_id 5863 payload 274 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c88008d00 msgr2=0x7f4c8800b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c88008d00 0x7f4c8800b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> v2:172.21.15.162:3300/0 conn(0x7f4c88006210 msgr2=0x7f4c880086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:26.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> v2:172.21.15.162:3300/0 conn(0x7f4c88006210 0x7f4c880086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 10 monclient: _finish_hunting 0 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 monclient: found mon.noname-c 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0073250 con 0x7f4c88003820 2024-09-16T15:16:26.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.777+0000 7f4ca56c2640 1 -- 172.21.15.134:0/4117326625 reap_dead start 2024-09-16T15:16:26.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 msgr2=0x7f4c88005c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4c88005c30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c02ad90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=80f2c24e7f9ca601 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 1 -- 172.21.15.134:0/4117326625 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9c02bd10 con 0x7f4c88003820 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: dump: 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:26.791 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 1 -- 172.21.15.134:0/4117326625 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9c02f070 con 0x7f4c88003820 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca7125640 5 monclient: authenticate success, global_id 5863 2024-09-16T15:16:26.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca7125640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca7125640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4ca7125640 4 set_mon_vals no callback set 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 1 -- 172.21.15.134:0/4117326625 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9c02be70 con 0x7f4c88003820 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.780+0000 7f4c977fe640 10 monclient: dump: 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:26.793 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 10 monclient: shutdown 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 -- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 msgr2=0x7f4c88005c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4c88005c30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c02ad90 comp rx=0 tx=0).stop 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 -- 172.21.15.134:0/4117326625 shutdown_connections 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4117326625 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4c88005c30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 -- 172.21.15.134:0/4117326625 >> 172.21.15.134:0/4117326625 conn(0x7f4ca00a9ea0 msgr2=0x7f4ca00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:26.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 -- 172.21.15.134:0/4117326625 shutdown_connections 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.781+0000 7f4ca7125640 1 -- 172.21.15.134:0/4117326625 wait complete. 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.782+0000 7f4ca7125640 10 monclient: build_initial_monmap 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.782+0000 7f4ca7125640 10 monclient: monmap: 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:26.795 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.782+0000 7f4ca7125640 1 Processor -- start 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.782+0000 7f4ca7125640 1 -- start start 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.782+0000 7f4ca7125640 10 monclient: init 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient: _add_conns ranks=[4,0,6,3,8,2,1,7,5] 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-c con 0x7f4c88003820 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca01354d0 0x7f4ca011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-a con 0x7f4ca01354d0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:26.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca011ffc0 0x7f4ca0120440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): picked mon.noname-d con 0x7f4ca011ffc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01cc4d0 con 0x7f4ca01354d0 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca009ff40 con 0x7f4c88003820 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01cba30 con 0x7f4ca011ffc0 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca7125640 10 monclient(hunting): authenticate will time out at 1723.372419s 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47802/0 (socket says 172.21.15.134:47802) 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4242630109 learned_addr learned my addr 172.21.15.134:0/4242630109 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request con 0x7f4c88003820 auth_method 0 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca01354d0 0x7f4ca011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca011ffc0 0x7f4ca0120440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): get_auth_request con 0x7f4ca01354d0 auth_method 0 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request con 0x7f4ca011ffc0 auth_method 0 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:26.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4c97fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca56c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient(hunting): handle_auth_done global_id 5872 payload 824 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca011ffc0 msgr2=0x7f4ca0120440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca011ffc0 0x7f4ca0120440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca01354d0 msgr2=0x7f4ca011fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca01354d0 0x7f4ca011fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 10 monclient: _finish_hunting 0 2024-09-16T15:16:26.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 1 monclient: found mon.noname-c 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.783+0000 7f4ca4ec1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.784+0000 7f4ca4ec1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.784+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4242630109 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c00e040 con 0x7f4c88003820 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca4ec1640 1 -- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 msgr2=0x7f4ca0134f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca4ec1640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9c035bd0 con 0x7f4c88003820 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: dump: 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 5 monclient: authenticate success, global_id 5872 2024-09-16T15:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9c035d30 con 0x7f4c88003820 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9c035ea0 con 0x7f4c88003820 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c957fa640 10 monclient: dump: 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4c96ffd640 4 set_mon_vals no callback set 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 10 monclient: _renew_subs 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca00a7af0 con 0x7f4c88003820 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 10 monclient: _renew_subs 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.786+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca01211c0 con 0x7f4c88003820 2024-09-16T15:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.787+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4c9c040460 con 0x7f4c88003820 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.787+0000 7f4ca7125640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.787+0000 7f4ca7125640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.787+0000 7f4ca7125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.787+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca00a7eb0 con 0x7f4c88003820 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.788+0000 7f4c957fa640 1 --2- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 0x7f4c8803be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.788+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(716..716 src has 1..716) v4 ==== 3963+0+0 (secure 0 0 0) 0x7f4c9c06b520 con 0x7f4c88003820 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.788+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 0x7f4c8803be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.788+0000 7f4c97fff640 10 monclient: get_auth_request con 0x7f4c88039980 auth_method 0 2024-09-16T15:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.791+0000 7f4c97fff640 1 --2- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 0x7f4c8803be40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00a29e0 tx=0x7f4c90008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.793+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4c9c03c310 con 0x7f4c88003820 2024-09-16T15:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.793+0000 7f4c957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:26.793+0000 7f4c957fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:27.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4ca7125640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:27.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4ca7125640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:27.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4ca7125640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:27.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4ca009c400 con 0x7f4c88003820 2024-09-16T15:16:27.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4c957fa640 1 -- 172.21.15.134:0/4242630109 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3190 (secure 0 0 0) 0x7f4c9c044080 con 0x7f4c88003820 2024-09-16T15:16:27.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4c957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:27.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.022+0000 7f4c957fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:27.024 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":132,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:27.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 msgr2=0x7f4c8803be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:27.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 0x7f4c8803be40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00a29e0 tx=0x7f4c90008040 comp rx=0 tx=0).stop 2024-09-16T15:16:27.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 10 monclient: shutdown 2024-09-16T15:16:27.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 msgr2=0x7f4ca0134f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.024+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 shutdown_connections 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c88039980 0x7f4c8803be40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca011ffc0 0x7f4ca0120440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ca01354d0 0x7f4ca011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 --2- 172.21.15.134:0/4242630109 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4c88003820 0x7f4ca0134f90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 >> 172.21.15.134:0/4242630109 conn(0x7f4ca00a9ea0 msgr2=0x7f4ca01bbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 shutdown_connections 2024-09-16T15:16:27.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:27.025+0000 7f4ca7125640 1 -- 172.21.15.134:0/4242630109 wait complete. 2024-09-16T15:16:29.500 INFO:tasks.rados.rados.0.smithi162.stdout:3390: done (0 left) 2024-09-16T15:16:29.500 INFO:tasks.rados.rados.0.smithi162.stdout:3391: write oid 14 current snap is 351 2024-09-16T15:16:29.500 INFO:tasks.rados.rados.0.smithi162.stdout:3391: seq_num 1169 ranges {447383=568090,1690509=701410,2902655=646983} 2024-09-16T15:16:29.505 INFO:tasks.rados.rados.0.smithi162.stdout:3391: writing smithi16234619-14 from 447383 to 1015473 tid 1 2024-09-16T15:16:29.513 INFO:tasks.rados.rados.0.smithi162.stdout:3391: writing smithi16234619-14 from 1690509 to 2391919 tid 2 2024-09-16T15:16:29.519 INFO:tasks.rados.rados.0.smithi162.stdout:3391: writing smithi16234619-14 from 2902655 to 3549638 tid 3 2024-09-16T15:16:29.519 INFO:tasks.rados.rados.0.smithi162.stdout:3392: snap_create 2024-09-16T15:16:29.553 INFO:tasks.rados.rados.0.smithi162.stdout:3391: finishing write tid 1 to smithi16234619-14 2024-09-16T15:16:29.553 INFO:tasks.rados.rados.0.smithi162.stdout:3391: finishing write tid 2 to smithi16234619-14 2024-09-16T15:16:29.554 INFO:tasks.rados.rados.0.smithi162.stdout:3391: finishing write tid 3 to smithi16234619-14 2024-09-16T15:16:29.554 INFO:tasks.rados.rados.0.smithi162.stdout:3391: finishing write tid 4 to smithi16234619-14 2024-09-16T15:16:29.554 INFO:tasks.rados.rados.0.smithi162.stdout:3391: finishing write tid 5 to smithi16234619-14 2024-09-16T15:16:29.554 INFO:tasks.rados.rados.0.smithi162.stdout:3391: oid 14 updating version 0 to 1712 2024-09-16T15:16:29.555 INFO:tasks.rados.rados.0.smithi162.stdout:3391: oid 14 updating version 1712 to 1713 2024-09-16T15:16:29.555 INFO:tasks.rados.rados.0.smithi162.stdout:3391: oid 14 version 1713 is already newer than 1711 2024-09-16T15:16:29.555 INFO:tasks.rados.rados.0.smithi162.stdout:3391: oid 14 version 1713 is already newer than 1710 2024-09-16T15:16:29.555 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1713 (ObjNum 1169 snap 351 seq_num 1169) dirty exists 2024-09-16T15:16:29.555 INFO:tasks.rados.rados.0.smithi162.stdout:3391: left oid 14 (ObjNum 1169 snap 351 seq_num 1169) 2024-09-16T15:16:30.038 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 10 monclient: get_monmap_and_config 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 10 monclient: monmap: 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:30.134025+0000 2024-09-16T15:16:30.134 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:30.134025+0000 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:30.135 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 1 Processor -- start 2024-09-16T15:16:30.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 1 -- start start 2024-09-16T15:16:30.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.133+0000 7f0403fce640 10 monclient: init 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient: _add_conns ranks=[1,2,4,10,6,8,9,7,0,3,5] 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f03fc136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-b con 0x7f03fc09bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 0x7f03fc13e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-c con 0x7f03fc13bf80 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc073250 con 0x7f03fc136030 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc09b600 con 0x7f03fc09bfe0 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc0a5740 con 0x7f03fc13bf80 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0403fce640 10 monclient(hunting): authenticate will time out at 1726.723384s 2024-09-16T15:16:30.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 0x7f03fc13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33800/0 (socket says 172.21.15.134:33800) 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 -- 172.21.15.134:0/536263669 learned_addr learned my addr 172.21.15.134:0/536263669 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): get_auth_request con 0x7f03fc09bfe0 auth_method 0 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): get_auth_request con 0x7f03fc13bf80 auth_method 0 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient(hunting): handle_auth_done global_id 114122 payload 274 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 -- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 msgr2=0x7f03fc13e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 0x7f03fc13e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 -- 172.21.15.134:0/536263669 >> v1:172.21.15.162:6789/0 conn(0x7f03fc136030 legacy=0x7f03fc136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 monclient: found mon.noname-b 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f0401569640 1 -- 172.21.15.134:0/536263669 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0a52c0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.134+0000 7f040256b640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 0x7f03fc13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0401569640 1 -- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 msgr2=0x7f03fc09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0401569640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03f00098a0 tx=0x7f03f002a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bebdaed6b34f5ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 1 -- 172.21.15.134:0/536263669 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03f002adf0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: dump: 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.146 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 1 -- 172.21.15.134:0/536263669 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f00307b0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0403fce640 5 monclient: authenticate success, global_id 114122 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0403fce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:30.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0403fce640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 1 -- 172.21.15.134:0/536263669 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03f0030d90 con 0x7f03fc09bfe0 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0403fce640 4 set_mon_vals no callback set 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.137+0000 7f0400d68640 10 monclient: dump: 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.148 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 10 monclient: shutdown 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 -- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 msgr2=0x7f03fc09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f03f00098a0 tx=0x7f03f002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 -- 172.21.15.134:0/536263669 shutdown_connections 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc13bf80 0x7f03fc13e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 --2- 172.21.15.134:0/536263669 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc09bfe0 0x7f03fc09c400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 -- 172.21.15.134:0/536263669 >> 172.21.15.134:0/536263669 conn(0x7f03fc0a9ea0 msgr2=0x7f03fc0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 -- 172.21.15.134:0/536263669 shutdown_connections 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.138+0000 7f0403fce640 1 -- 172.21.15.134:0/536263669 wait complete. 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient: monmap: 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.150 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 Processor -- start 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 -- start start 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient: init 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient: _add_conns ranks=[4,1,5,8,3,6,7,0,2] 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-c con 0x7f03fc09bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 0x7f03fc133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-f con 0x7f03fc136030 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:30.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f03fc13bf80 0x7f03fc134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): picked mon.noname-h con 0x7f03fc13bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1cb8d0 con 0x7f03fc136030 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc13b1b0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc09fd20 con 0x7f03fc13bf80 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0403fce640 10 monclient(hunting): authenticate will time out at 1726.728824s 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47814/0 (socket says 172.21.15.134:47814) 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 1 -- 172.21.15.134:0/1804278043 learned_addr learned my addr 172.21.15.134:0/1804278043 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): get_auth_request con 0x7f03fc09bfe0 auth_method 0 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.139+0000 7f0401d6a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 0x7f03fc133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 10 monclient(hunting): get_auth_request con 0x7f03fc136030 auth_method 0 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f040256b640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f03fc13bf80 0x7f03fc134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f040256b640 10 monclient(hunting): get_auth_request con 0x7f03fc13bf80 auth_method 0 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f040256b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f040256b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f040256b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 10 monclient(hunting): handle_auth_done global_id 5890 payload 824 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 -- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f03fc13bf80 msgr2=0x7f03fc134460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f03fc13bf80 0x7f03fc134460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 -- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 msgr2=0x7f03fc133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 0x7f03fc133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 monclient: found mon.noname-c 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401d6a640 1 -- 172.21.15.134:0/1804278043 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec009400 con 0x7f03fc09bfe0 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.140+0000 7f0401569640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 0x7f03fc133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0401d6a640 1 -- 172.21.15.134:0/1804278043 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 msgr2=0x7f03fc1cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0401d6a640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c7f0 tx=0x7f03f400ccb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03f4012420 con 0x7f03fc09bfe0 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: dump: 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 5 monclient: authenticate success, global_id 5890 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f40125a0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 10 monclient: _renew_subs 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f00094d0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f03f4012b80 con 0x7f03fc09bfe0 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:30.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fbfff640 4 set_mon_vals no callback set 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f03fa7fc640 10 monclient: dump: 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.157 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 10 monclient: _renew_subs 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.142+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc0a83f0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f03f4012ce0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f0403fce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f0403fce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f0403fce640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc09c400 con 0x7f03fc09bfe0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f03fa7fc640 1 --2- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 0x7f03e803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.143+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(717..717 src has 1..717) v4 ==== 3963+0+0 (secure 0 0 0) 0x7f03f4049fa0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.144+0000 7f0401569640 1 --2- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 0x7f03e803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.158 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.144+0000 7f0401569640 10 monclient: get_auth_request con 0x7f03e8038880 auth_method 0 2024-09-16T15:16:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.146+0000 7f0401569640 1 --2- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 0x7f03e803ad40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0a51e0 tx=0x7f03f002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.148+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f03f400aa60 con 0x7f03fc09bfe0 2024-09-16T15:16:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.148+0000 7f03fa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.148+0000 7f03fa7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.373+0000 7f0403fce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.373+0000 7f0403fce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.373+0000 7f0403fce640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.373+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f03fc0008d0 con 0x7f03fc09bfe0 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.374+0000 7f03fa7fc640 1 -- 172.21.15.134:0/1804278043 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f03f4016070 con 0x7f03fc09bfe0 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.374+0000 7f03fa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:30.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.374+0000 7f03fa7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:30.376 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":136,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 msgr2=0x7f03e803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 0x7f03e803ad40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0a51e0 tx=0x7f03f002e000 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 10 monclient: shutdown 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 msgr2=0x7f03fc1cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c7f0 tx=0x7f03f400ccb0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 shutdown_connections 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> v2:172.21.15.162:6800/980547263 conn(0x7f03e8038880 0x7f03e803ad40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f03fc13bf80 0x7f03fc134460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc136030 0x7f03fc133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 --2- 172.21.15.134:0/1804278043 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f03fc09bfe0 0x7f03fc1cb390 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 >> 172.21.15.134:0/1804278043 conn(0x7f03fc0a9ea0 msgr2=0x7f03fc134de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 shutdown_connections 2024-09-16T15:16:30.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.376+0000 7f0403fce640 1 -- 172.21.15.134:0/1804278043 wait complete. 2024-09-16T15:16:30.387 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:16:30.387 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:16:30.388 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:16:30.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.483+0000 7f85523e2640 10 monclient: get_monmap_and_config 2024-09-16T15:16:30.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.483+0000 7f85523e2640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.483+0000 7f85523e2640 10 monclient: monmap: 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:30.484428+0000 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:30.484428+0000 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.485 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:30.486 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 Processor -- start 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 -- start start 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient: init 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient: _add_conns ranks=[8,9,5,2,6,3,7,0,4,10,1] 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-e con 0x7f854c136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 0x7f854c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-a con 0x7f854c137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f854c135dc0 0x7f854c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-h con 0x7f854c135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854c09abc0 con 0x7f854c137220 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f854c09ba50 con 0x7f854c135dc0 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f854c0a53c0 con 0x7f854c136e40 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f85523e2640 10 monclient(hunting): authenticate will time out at 1727.073745s 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 0x7f854c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41682/0 (socket says 172.21.15.134:41682) 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 1 -- 172.21.15.134:0/875415947 learned_addr learned my addr 172.21.15.134:0/875415947 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): get_auth_request con 0x7f854c136e40 auth_method 0 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854b7fe640 10 monclient(hunting): get_auth_request con 0x7f854c137220 auth_method 0 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.484+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f855097f640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f854c135dc0 0x7f854c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f855097f640 10 monclient(hunting): get_auth_request con 0x7f854c135dc0 auth_method 0 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f855097f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f855097f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f855097f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 10 monclient(hunting): handle_auth_done global_id 84536 payload 274 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 -- 172.21.15.134:0/875415947 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f854c135dc0 msgr2=0x7f854c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f854c135dc0 0x7f854c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 -- 172.21.15.134:0/875415947 >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 msgr2=0x7f854c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 --2- 172.21.15.134:0/875415947 >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 0x7f854c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 monclient: found mon.noname-e 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854bfff640 1 -- 172.21.15.134:0/875415947 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c073250 con 0x7f854c136e40 2024-09-16T15:16:30.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.485+0000 7f854b7fe640 1 --2- 172.21.15.134:0/875415947 >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 0x7f854c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.487+0000 7f854bfff640 1 -- 172.21.15.134:0/875415947 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 msgr2=0x7f854c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.487+0000 7f854bfff640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85340098a0 tx=0x7f853402a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3b6d2d0e0b9df04d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 1 -- 172.21.15.134:0/875415947 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f853402adf0 con 0x7f854c136e40 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: dump: 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.495 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 5 monclient: authenticate success, global_id 84536 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 1 -- 172.21.15.134:0/875415947 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85340307b0 con 0x7f854c136e40 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 4 set_mon_vals no callback set 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 1 -- 172.21.15.134:0/875415947 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8534030d90 con 0x7f854c136e40 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f854affd640 10 monclient: dump: 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.497 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 10 monclient: shutdown 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 -- 172.21.15.134:0/875415947 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 msgr2=0x7f854c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85340098a0 tx=0x7f853402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 -- 172.21.15.134:0/875415947 shutdown_connections 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f854c135dc0 0x7f854c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 --2- 172.21.15.134:0/875415947 >> v2:172.21.15.134:3300/0 conn(0x7f854c137220 0x7f854c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 --2- 172.21.15.134:0/875415947 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f854c136e40 0x7f854c09c150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 -- 172.21.15.134:0/875415947 >> 172.21.15.134:0/875415947 conn(0x7f854c0a6110 msgr2=0x7f854c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 -- 172.21.15.134:0/875415947 shutdown_connections 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.488+0000 7f85523e2640 1 -- 172.21.15.134:0/875415947 wait complete. 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.489+0000 7f85523e2640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.489+0000 7f85523e2640 10 monclient: monmap: 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:30.499 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.489+0000 7f85523e2640 1 Processor -- start 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.489+0000 7f85523e2640 1 -- start start 2024-09-16T15:16:30.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.489+0000 7f85523e2640 10 monclient: init 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient: _add_conns ranks=[1,6,2,0,5,4,8,7,3] 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854c135dc0 0x7f854c134fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-f con 0x7f854c135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-d con 0x7f854c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f854c137220 0x7f854c121a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): picked mon.noname-b con 0x7f854c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c1cc260 con 0x7f854c135dc0 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854c1254e0 con 0x7f854c137220 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f854c136340 con 0x7f854c136e40 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f85523e2640 10 monclient(hunting): authenticate will time out at 1727.079385s 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f854c137220 0x7f854c121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33956/0 (socket says 172.21.15.134:33956) 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 -- 172.21.15.134:0/282505588 learned_addr learned my addr 172.21.15.134:0/282505588 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): get_auth_request con 0x7f854c137220 auth_method 0 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): get_auth_request con 0x7f854c136e40 auth_method 0 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854c135dc0 0x7f854c134fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): get_auth_request con 0x7f854c135dc0 auth_method 0 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient(hunting): handle_auth_done global_id 94281 payload 824 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f854c137220 msgr2=0x7f854c121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f854c137220 0x7f854c121a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854c135dc0 msgr2=0x7f854c134fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f854c135dc0 0x7f854c134fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 monclient: found mon.noname-d 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f854b7fe640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c009400 con 0x7f854c136e40 2024-09-16T15:16:30.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.490+0000 7f855097f640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f854c137220 0x7f854c121a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f854b7fe640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 msgr2=0x7f854c135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f854b7fe640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f853c00e990 tx=0x7f853c00ee50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f853c00cbf0 con 0x7f854c136e40 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: dump: 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.509 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 5 monclient: authenticate success, global_id 94281 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c00cd70 con 0x7f854c136e40 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f853c00cee0 con 0x7f854c136e40 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 10 monclient: _renew_subs 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340094d0 con 0x7f854c136e40 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f854a7fc640 4 set_mon_vals no callback set 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f8548ff9640 10 monclient: dump: 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.511 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 10 monclient: _renew_subs 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.493+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c136890 con 0x7f854c136e40 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f85523e2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f85523e2640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 0x7f854c121fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c071b20 con 0x7f854c1cb550 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f854b7fe640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f853c00e990 tx=0x7f853c00ee50 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f853c013470 mgrmap(e 5) v1 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f853c013470 con 0x7f854c136e40 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f8548ff9640 1 --2- 172.21.15.134:0/282505588 >> v2:172.21.15.162:6800/980547263 conn(0x7f852c038880 0x7f852c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f855097f640 1 --2- 172.21.15.134:0/282505588 >> v2:172.21.15.162:6800/980547263 conn(0x7f852c038880 0x7f852c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.494+0000 7f855097f640 10 monclient: get_auth_request con 0x7f852c038880 auth_method 0 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.497+0000 7f855097f640 1 --2- 172.21.15.134:0/282505588 >> v2:172.21.15.162:6800/980547263 conn(0x7f852c038880 0x7f852c03ad40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f85440077c0 tx=0x7f8544009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 0x7f854c121fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient: get_auth_request con 0x7f854c1cb550 auth_method 0 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 10 monclient(hunting): handle_auth_done global_id 114120 payload 824 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.500+0000 7f854bfff640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f854c061ae0 con 0x7f854c1cb550 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.502+0000 7f854bfff640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 msgr2=0x7f854c121fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.502+0000 7f854bfff640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 0x7f854c121fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8534034000 tx=0x7f85340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85340044f0 con 0x7f854c1cb550 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 10 monclient: dump: 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.514 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.515 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8534030b30 con 0x7f854c1cb550 2024-09-16T15:16:30.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 msgr2=0x7f854c121fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 0x7f854c121fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8534034000 tx=0x7f85340046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 msgr2=0x7f854c121fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.504+0000 7f8548ff9640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1cb550 0x7f854c121fa0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8534034000 tx=0x7f85340046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f85523e2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f85523e2640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 0x7f854c0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f85523e2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c07a680 con 0x7f854c1224e0 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f854bfff640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 0x7f854c0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f854bfff640 10 monclient: get_auth_request con 0x7f854c1224e0 auth_method 0 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f854bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f854bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.510+0000 7f854bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.511+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.511+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.511+0000 7f854bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.511+0000 7f854bfff640 10 monclient(hunting): handle_auth_done global_id 114129 payload 824 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.511+0000 7f854bfff640 1 -- 172.21.15.134:0/282505588 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f854c09b490 con 0x7f854c1224e0 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f854bfff640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 msgr2=0x7f854c0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f854bfff640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 0x7f854c0a1a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f853402a110 tx=0x7f8534008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8534030e40 con 0x7f854c1224e0 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 10 monclient: dump: 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.519 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9364 (secure 0 0 0) 0x7f8534049810 con 0x7f854c1224e0 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 msgr2=0x7f854c0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 0x7f854c0a1a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f853402a110 tx=0x7f8534008030 comp rx=0 tx=0).stop 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 msgr2=0x7f854c0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.513+0000 7f8548ff9640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c1224e0 0x7f854c0a1a20 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f853402a110 tx=0x7f8534008030 comp rx=0 tx=0).stop 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:30.521 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:30.522 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.523 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:30.524 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.525 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:30.526 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.527 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:30.528 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.529 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.530 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:30.531 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.532 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.533 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:30.534 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.535 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.536 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.537 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.538 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.539 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.540 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.541 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.542 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.542 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.542 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:30.542 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:30.543 INFO:tasks.rados.rados.0.smithi162.stdout:3391: done (1 left) 2024-09-16T15:16:30.543 INFO:tasks.rados.rados.0.smithi162.stdout:3392: done (0 left) 2024-09-16T15:16:30.543 INFO:tasks.rados.rados.0.smithi162.stdout:3393: delete oid 13 current snap is 352 2024-09-16T15:16:30.543 INFO:tasks.rados.rados.0.smithi162.stdout:3393: done (0 left) 2024-09-16T15:16:30.543 INFO:tasks.rados.rados.0.smithi162.stdout:3394: snap_remove snap 336 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 >> v2:172.21.15.162:6800/980547263 conn(0x7f852c038880 msgr2=0x7f852c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 --2- 172.21.15.134:0/282505588 >> v2:172.21.15.162:6800/980547263 conn(0x7f852c038880 0x7f852c03ad40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f85440077c0 tx=0x7f8544009820 comp rx=0 tx=0).stop 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 10 monclient: shutdown 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 msgr2=0x7f854c135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 --2- 172.21.15.134:0/282505588 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f854c136e40 0x7f854c135510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f853c00e990 tx=0x7f853c00ee50 comp rx=0 tx=0).stop 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f855097f640 1 -- 172.21.15.134:0/282505588 reap_dead start 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f8548ff9640 1 -- 172.21.15.134:0/282505588 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(717..717 src has 1..717) v4 ==== 3963+0+0 (unknown 0 0 0) 0x7f853c04a3d0 con 0x7f854c136e40 2024-09-16T15:16:30.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f855097f640 1 -- 172.21.15.134:0/282505588 reap_dead start 2024-09-16T15:16:30.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 shutdown_connections 2024-09-16T15:16:30.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 >> 172.21.15.134:0/282505588 conn(0x7f854c0a6110 msgr2=0x7f854c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 shutdown_connections 2024-09-16T15:16:30.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.514+0000 7f85523e2640 1 -- 172.21.15.134:0/282505588 wait complete. 2024-09-16T15:16:30.545 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:30.545 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:16:30.545 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 10 monclient: get_monmap_and_config 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 10 monclient: monmap: 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.643 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:30.643040+0000 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:30.643040+0000 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:30.644 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:30.645 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 1 Processor -- start 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 1 -- start start 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.642+0000 7fca0f111640 10 monclient: init 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient: _add_conns ranks=[10,9,0,4,1,7,6,2,8,5,3] 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca0809fb10 0x7fca0809c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-f con 0x7fca0809fb10 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fca080a06c0 0x7fca0809cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-a con 0x7fca080a06c0 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fca08069e50 addr v1:172.21.15.134:6789/0 2024-09-16T15:16:30.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca08073250 con 0x7fca08069e50 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca08135c90 con 0x7fca080a06c0 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080666e0 con 0x7fca0809fb10 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0f111640 10 monclient(hunting): authenticate will time out at 1727.232357s 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fca080a06c0 0x7fca0809cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fca080a06c0 0x7fca0809cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46060/0 (socket says 172.21.15.134:46060) 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca0d6ae640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fca08069e50 0x7fca0806a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45610/0 (socket says 172.21.15.134:45610) 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 -- 172.21.15.134:0/786785759 learned_addr learned my addr 172.21.15.134:0/786785759 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): get_auth_request con 0x7fca080a06c0 auth_method 0 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient(hunting): handle_auth_done global_id 114138 payload 274 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 -- 172.21.15.134:0/786785759 >> v2:172.21.15.162:3300/0 conn(0x7fca0809fb10 msgr2=0x7fca0809c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 --2- 172.21.15.134:0/786785759 >> v2:172.21.15.162:3300/0 conn(0x7fca0809fb10 0x7fca0809c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 -- 172.21.15.134:0/786785759 >> v1:172.21.15.134:6789/0 conn(0x7fca08069e50 legacy=0x7fca0806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 monclient: found mon.noname-a 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:30.649 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.643+0000 7fca07fff640 1 -- 172.21.15.134:0/786785759 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0809fef0 con 0x7fca080a06c0 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca07fff640 1 -- 172.21.15.134:0/786785759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca080a06c0 msgr2=0x7fca0809cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca07fff640 1 --2- 172.21.15.134:0/786785759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca080a06c0 0x7fca0809cdb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca0806be80 tx=0x7fc9f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dd5e1937281148 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 1 -- 172.21.15.134:0/786785759 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9f40307b0 con 0x7fca080a06c0 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: dump: 2024-09-16T15:16:30.655 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.656 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 1 -- 172.21.15.134:0/786785759 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f4030930 con 0x7fca080a06c0 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca0f111640 5 monclient: authenticate success, global_id 114138 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca0f111640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca0f111640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 1 -- 172.21.15.134:0/786785759 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9f40313a0 con 0x7fca080a06c0 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca0f111640 4 set_mon_vals no callback set 2024-09-16T15:16:30.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.646+0000 7fca077fe640 10 monclient: dump: 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.658 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 10 monclient: shutdown 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 -- 172.21.15.134:0/786785759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca080a06c0 msgr2=0x7fca0809cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 --2- 172.21.15.134:0/786785759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca080a06c0 0x7fca0809cdb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca0806be80 tx=0x7fc9f402a110 comp rx=0 tx=0).stop 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 -- 172.21.15.134:0/786785759 shutdown_connections 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 --2- 172.21.15.134:0/786785759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca080a06c0 0x7fca0809cdb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 --2- 172.21.15.134:0/786785759 >> v2:172.21.15.162:3300/0 conn(0x7fca0809fb10 0x7fca0809c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 -- 172.21.15.134:0/786785759 >> 172.21.15.134:0/786785759 conn(0x7fca080a6400 msgr2=0x7fca080a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 -- 172.21.15.134:0/786785759 shutdown_connections 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 1 -- 172.21.15.134:0/786785759 wait complete. 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.647+0000 7fca0f111640 10 monclient: monmap: 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.660 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 Processor -- start 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 -- start start 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient: init 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient: _add_conns ranks=[1,4,8,0,5,7,3,6,2] 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca08069e50 0x7fca081c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-f con 0x7fca08069e50 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fca0809fb10 0x7fca08131bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-c con 0x7fca0809fb10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): picked mon.noname-e con 0x7fca080a06c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:30.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca080718f0 con 0x7fca08069e50 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca080a20a0 con 0x7fca0809fb10 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fca0809cf10 con 0x7fca080a06c0 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0f111640 10 monclient(hunting): authenticate will time out at 1727.237840s 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca07fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fca0809fb10 0x7fca08131bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41692/0 (socket says 172.21.15.134:41692) 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 learned_addr learned my addr 172.21.15.134:0/369481961 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 10 monclient(hunting): get_auth_request con 0x7fca080a06c0 auth_method 0 2024-09-16T15:16:30.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca0d6ae640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.648+0000 7fca07fff640 10 monclient(hunting): get_auth_request con 0x7fca0809fb10 auth_method 0 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca07fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0cead640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca08069e50 0x7fca081c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0cead640 10 monclient(hunting): get_auth_request con 0x7fca08069e50 auth_method 0 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0cead640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0cead640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0cead640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient(hunting): handle_auth_done global_id 84545 payload 824 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fca0809fb10 msgr2=0x7fca08131bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fca0809fb10 0x7fca08131bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca08069e50 msgr2=0x7fca081c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca08069e50 0x7fca081c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 monclient: found mon.noname-e 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.649+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f40094d0 con 0x7fca080a06c0 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 msgr2=0x7fca081cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0d6ae640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c7b0 tx=0x7fca0000cc70 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca0000aab0 con 0x7fca080a06c0 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: dump: 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.665 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0000ac50 con 0x7fca080a06c0 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 5 monclient: authenticate success, global_id 84545 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 10 monclient: _renew_subs 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0809c9e0 con 0x7fca080a06c0 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca0000adc0 con 0x7fca080a06c0 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca057fa640 10 monclient: dump: 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.667 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 10 monclient: _renew_subs 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca06ffd640 4 set_mon_vals no callback set 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.651+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca08071f10 con 0x7fca080a06c0 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fca0000a640 con 0x7fca080a06c0 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0f111640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0f111640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 0x7fca081cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806b5d0 con 0x7fca08132110 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 0x7fca081cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient: get_auth_request con 0x7fca08132110 auth_method 0 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca057fa640 1 --2- 172.21.15.134:0/369481961 >> v2:172.21.15.162:6800/980547263 conn(0x7fc9ec038880 0x7fc9ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (secure 0 0 0) 0x7fca00049590 con 0x7fca080a06c0 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 10 monclient(hunting): handle_auth_done global_id 114140 payload 824 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.652+0000 7fca0cead640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca08061ae0 con 0x7fca08132110 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.653+0000 7fca07fff640 1 --2- 172.21.15.134:0/369481961 >> v2:172.21.15.162:6800/980547263 conn(0x7fc9ec038880 0x7fc9ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.653+0000 7fca07fff640 10 monclient: get_auth_request con 0x7fc9ec038880 auth_method 0 2024-09-16T15:16:30.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca0cead640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 msgr2=0x7fca081cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca0cead640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 0x7fca081cb9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc00d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9fc014d80 con 0x7fca08132110 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 10 monclient: dump: 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.673 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc9fc014910 con 0x7fca08132110 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 msgr2=0x7fca081cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 0x7fca081cb9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc00d9e0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 msgr2=0x7fca081cb9f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca057fa640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca08132110 0x7fca081cb9f0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc00d9e0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.655+0000 7fca07fff640 1 --2- 172.21.15.134:0/369481961 >> v2:172.21.15.162:6800/980547263 conn(0x7fc9ec038880 0x7fc9ec03ad40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fca080672e0 tx=0x7fc9f402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0f111640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0f111640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 0x7fca0809a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0f111640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0807a680 con 0x7fca081cbf30 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 0x7fca0809a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient: get_auth_request con 0x7fca081cbf30 auth_method 0 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 10 monclient(hunting): handle_auth_done global_id 114149 payload 824 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.661+0000 7fca0cead640 1 -- 172.21.15.134:0/369481961 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fca0809d350 con 0x7fca081cbf30 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0cead640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 msgr2=0x7fca0809a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0cead640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 0x7fca0809a850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9fc01fe30 con 0x7fca081cbf30 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 10 monclient: dump: 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.677 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fc9fc023070 con 0x7fca081cbf30 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 msgr2=0x7fca0809a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 0x7fca0809a850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 msgr2=0x7fca0809a850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca057fa640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca081cbf30 0x7fca0809a850 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00d520 tx=0x7fc9fc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:30.678 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:30.679 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.680 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.681 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:30.682 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.683 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.684 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.685 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.686 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:30.687 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.688 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.689 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.690 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:30.693 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.694 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.695 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:30.696 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.698 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:30.699 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.700 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.701 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:30.701 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 >> v2:172.21.15.162:6800/980547263 conn(0x7fc9ec038880 msgr2=0x7fc9ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 1 --2- 172.21.15.134:0/369481961 >> v2:172.21.15.162:6800/980547263 conn(0x7fc9ec038880 0x7fc9ec03ad40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fca080672e0 tx=0x7fc9f402e000 comp rx=0 tx=0).stop 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 10 monclient: shutdown 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 msgr2=0x7fca081cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0d6ae640 1 -- 172.21.15.134:0/369481961 reap_dead start 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.664+0000 7fca0f111640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c7b0 tx=0x7fca0000cc70 comp rx=0 tx=0).stop 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.665+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 shutdown_connections 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.665+0000 7fca0f111640 1 --2- 172.21.15.134:0/369481961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca080a06c0 0x7fca081cb4b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.665+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 >> 172.21.15.134:0/369481961 conn(0x7fca080a6400 msgr2=0x7fca081bcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.665+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 shutdown_connections 2024-09-16T15:16:30.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.665+0000 7fca0f111640 1 -- 172.21.15.134:0/369481961 wait complete. 2024-09-16T15:16:30.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:30.703 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:16:30.703 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.799+0000 7faee7a41640 10 monclient: get_monmap_and_config 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.799+0000 7faee7a41640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.800+0000 7faee7a41640 10 monclient: monmap: 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:30.800912+0000 2024-09-16T15:16:30.801 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:30.800912+0000 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.802 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:30.803 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:30.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.800+0000 7faee7a41640 1 Processor -- start 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.800+0000 7faee7a41640 1 -- start start 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.800+0000 7faee7a41640 10 monclient: init 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient: _add_conns ranks=[2,8,4,10,1,6,0,9,5,7,3] 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-b con 0x7faee0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-e con 0x7faee0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 0x7faee0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-c con 0x7faee0135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee009abc0 con 0x7faee0136e40 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee009ba50 con 0x7faee0135dc0 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faee00a53c0 con 0x7faee0137220 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee7a41640 10 monclient(hunting): authenticate will time out at 1727.390300s 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 0x7faee0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41696/0 (socket says 172.21.15.134:41696) 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33840/0 (socket says 172.21.15.134:33840) 2024-09-16T15:16:30.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 1 -- 172.21.15.134:0/715992248 learned_addr learned my addr 172.21.15.134:0/715992248 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): get_auth_request con 0x7faee0137220 auth_method 0 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): get_auth_request con 0x7faee0136e40 auth_method 0 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): get_auth_request con 0x7faee0135dc0 auth_method 0 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient(hunting): handle_auth_done global_id 114158 payload 274 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 -- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 msgr2=0x7faee0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 -- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 msgr2=0x7faee0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 0x7faee0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 monclient: found mon.noname-b 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee57dd640 1 -- 172.21.15.134:0/715992248 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0073250 con 0x7faee0136e40 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee5fde640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 0x7faee0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.801+0000 7faee4fdc640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee57dd640 1 -- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 msgr2=0x7faee009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee57dd640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faedc0097d0 tx=0x7faedc029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d69494cc0f3b235 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 1 -- 172.21.15.134:0/715992248 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faedc02ad10 con 0x7faee0136e40 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: dump: 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.817 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee7a41640 5 monclient: authenticate success, global_id 114158 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 1 -- 172.21.15.134:0/715992248 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc02ae90 con 0x7faee0136e40 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee7a41640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee7a41640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:30.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 1 -- 172.21.15.134:0/715992248 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faedc030c20 con 0x7faee0136e40 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faee7a41640 4 set_mon_vals no callback set 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.804+0000 7faed7fff640 10 monclient: dump: 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.819 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 10 monclient: shutdown 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 -- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 msgr2=0x7faee009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faedc0097d0 tx=0x7faedc029fb0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 -- 172.21.15.134:0/715992248 shutdown_connections 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0135dc0 0x7faee0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0137220 0x7faee0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 --2- 172.21.15.134:0/715992248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0136e40 0x7faee009c150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 -- 172.21.15.134:0/715992248 >> 172.21.15.134:0/715992248 conn(0x7faee00a6110 msgr2=0x7faee00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 -- 172.21.15.134:0/715992248 shutdown_connections 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 1 -- 172.21.15.134:0/715992248 wait complete. 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.805+0000 7faee7a41640 10 monclient: monmap: 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.821 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 Processor -- start 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 -- start start 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient: init 2024-09-16T15:16:30.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient: _add_conns ranks=[2,8,0,5,3,7,1,4,6] 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 0x7faee01cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-b con 0x7faee0135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 0x7faee0135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-e con 0x7faee0136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee0137220 0x7faee0135680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): picked mon.noname-a con 0x7faee0137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee01cc3d0 con 0x7faee0137220 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee01258e0 con 0x7faee0135dc0 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faee0125b30 con 0x7faee0136e40 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee7a41640 10 monclient(hunting): authenticate will time out at 1727.395732s 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 0x7faee01cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 0x7faee0135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 0x7faee01cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33854/0 (socket says 172.21.15.134:33854) 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 0x7faee0135140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41708/0 (socket says 172.21.15.134:41708) 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 1 -- 172.21.15.134:0/3566351059 learned_addr learned my addr 172.21.15.134:0/3566351059 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): get_auth_request con 0x7faee0135dc0 auth_method 0 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee0137220 0x7faee0135680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): get_auth_request con 0x7faee0136e40 auth_method 0 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): get_auth_request con 0x7faee0137220 auth_method 0 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee57dd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee4fdc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.806+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 10 monclient(hunting): handle_auth_done global_id 84563 payload 824 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 msgr2=0x7faee01cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 0x7faee01cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee0137220 msgr2=0x7faee0135680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee0137220 0x7faee0135680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 monclient: found mon.noname-e 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee4fdc640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8009a70 con 0x7faee0136e40 2024-09-16T15:16:30.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee57dd640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee0135dc0 0x7faee01cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.807+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee0137220 0x7faee0135680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee4fdc640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 msgr2=0x7faee0135140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee4fdc640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 0x7faee0135140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faed000e890 tx=0x7faed000ed50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faed0004030 con 0x7faee0136e40 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: dump: 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.827 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00041b0 con 0x7faee0136e40 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 5 monclient: authenticate success, global_id 84563 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 10 monclient: _renew_subs 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00094d0 con 0x7faee0136e40 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faed0004320 con 0x7faee0136e40 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed5ffb640 10 monclient: dump: 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.829 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faed77fe640 4 set_mon_vals no callback set 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 10 monclient: _renew_subs 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.809+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc009400 con 0x7faee0136e40 2024-09-16T15:16:30.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7faed000cb50 con 0x7faee0136e40 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faed5ffb640 1 --2- 172.21.15.134:0/3566351059 >> v2:172.21.15.162:6800/980547263 conn(0x7faec4038880 0x7faec403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (secure 0 0 0) 0x7faed004a1d0 con 0x7faee0136e40 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee7a41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee7a41640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 0x7faee002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee0071b20 con 0x7faee0122360 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 0x7faee002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient: get_auth_request con 0x7faee0122360 auth_method 0 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee57dd640 1 --2- 172.21.15.134:0/3566351059 >> v2:172.21.15.162:6800/980547263 conn(0x7faec4038880 0x7faec403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.810+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.811+0000 7faee57dd640 10 monclient: get_auth_request con 0x7faec4038880 auth_method 0 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.811+0000 7faee5fde640 10 monclient(hunting): handle_auth_done global_id 5917 payload 824 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.811+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faee0061ae0 con 0x7faee0122360 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 msgr2=0x7faee002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 0x7faee002b260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed800e8c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faed8015e80 con 0x7faee0122360 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 10 monclient: dump: 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.832 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faee57dd640 1 --2- 172.21.15.134:0/3566351059 >> v2:172.21.15.162:6800/980547263 conn(0x7faec4038880 0x7faec403ad40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faedc029ff0 tx=0x7faedc02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faed800f040 con 0x7faee0122360 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 msgr2=0x7faee002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 0x7faee002b260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed800e8c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 msgr2=0x7faee002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.813+0000 7faed5ffb640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122360 0x7faee002b260 secure :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed800e8c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee7a41640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee7a41640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 0x7faee002b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee7a41640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee0135c60 con 0x7faee0122740 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 0x7faee002b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient: get_auth_request con 0x7faee0122740 auth_method 0 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.819+0000 7faee5fde640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.820+0000 7faee5fde640 10 monclient(hunting): handle_auth_done global_id 5926 payload 824 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.820+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faee007a680 con 0x7faee0122740 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 msgr2=0x7faee002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faee5fde640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 0x7faee002b7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed8004a40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faed8024070 con 0x7faee0122740 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 10 monclient: dump: 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.837 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7faed802a420 con 0x7faee0122740 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:30.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 msgr2=0x7faee002b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 0x7faee002b7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed8004a40 comp rx=0 tx=0).stop 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 msgr2=0x7faee002b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.822+0000 7faed5ffb640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7faee0122740 0x7faee002b7a0 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faed8004950 tx=0x7faed8004a40 comp rx=0 tx=0).stop 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:30.839 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.840 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.842 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:30.843 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.844 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.845 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.846 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.847 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.848 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:30.849 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:30.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:30.855 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.858 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.859 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:30.860 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:30.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 >> v2:172.21.15.162:6800/980547263 conn(0x7faec4038880 msgr2=0x7faec403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 --2- 172.21.15.134:0/3566351059 >> v2:172.21.15.162:6800/980547263 conn(0x7faec4038880 0x7faec403ad40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faedc029ff0 tx=0x7faedc02e000 comp rx=0 tx=0).stop 2024-09-16T15:16:30.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 10 monclient: shutdown 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 msgr2=0x7faee0135140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 --2- 172.21.15.134:0/3566351059 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faee0136e40 0x7faee0135140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faed000e890 tx=0x7faed000ed50 comp rx=0 tx=0).stop 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 reap_dead start 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee5fde640 1 -- 172.21.15.134:0/3566351059 reap_dead start 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 shutdown_connections 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 >> 172.21.15.134:0/3566351059 conn(0x7faee00a6110 msgr2=0x7faee009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 shutdown_connections 2024-09-16T15:16:30.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.823+0000 7faee7a41640 1 -- 172.21.15.134:0/3566351059 wait complete. 2024-09-16T15:16:30.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:30.862 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:16:30.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 10 monclient: get_monmap_and_config 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 10 monclient: monmap: 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:30.961034+0000 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:30.961034+0000 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.962 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.963 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:30.963 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:30.963 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 1 Processor -- start 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 1 -- start start 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.960+0000 7f30b81c7640 10 monclient: init 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient: _add_conns ranks=[6,9,4,2,5,3,8,1,10,7,0] 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-d con 0x7f30b0134db0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:30.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-a con 0x7f30b0135190 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-c con 0x7f30b0131c10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b006f640 con 0x7f30b0135190 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00a3d60 con 0x7f30b0131c10 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00a3b60 con 0x7f30b0134db0 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b81c7640 10 monclient(hunting): authenticate will time out at 1727.550518s 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33972/0 (socket says 172.21.15.134:33972) 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46066/0 (socket says 172.21.15.134:46066) 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47846/0 (socket says 172.21.15.134:47846) 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 -- 172.21.15.134:0/1403471633 learned_addr learned my addr 172.21.15.134:0/1403471633 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): get_auth_request con 0x7f30b0134db0 auth_method 0 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): get_auth_request con 0x7f30b0135190 auth_method 0 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): get_auth_request con 0x7f30b0131c10 auth_method 0 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient(hunting): handle_auth_done global_id 94290 payload 274 2024-09-16T15:16:30.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 -- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 msgr2=0x7f30b0132090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 -- 172.21.15.134:0/1403471633 >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 msgr2=0x7f30b01355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 --2- 172.21.15.134:0/1403471633 >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 monclient: found mon.noname-d 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5f63640 1 -- 172.21.15.134:0/1403471633 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0073250 con 0x7f30b0134db0 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b5762640 1 --2- 172.21.15.134:0/1403471633 >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.961+0000 7f30b6764640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:30.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b5f63640 1 -- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 msgr2=0x7f30b0071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b5f63640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0098a0 tx=0x7f30ac02a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3035c6aa74356189 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 1 -- 172.21.15.134:0/1403471633 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac02adf0 con 0x7f30b0134db0 2024-09-16T15:16:30.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: dump: 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.977 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b81c7640 5 monclient: authenticate success, global_id 94290 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b81c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 1 -- 172.21.15.134:0/1403471633 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac0307b0 con 0x7f30b0134db0 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b81c7640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:30.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 1 -- 172.21.15.134:0/1403471633 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac030d90 con 0x7f30b0134db0 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b81c7640 4 set_mon_vals no callback set 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.964+0000 7f30b4f61640 10 monclient: dump: 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.979 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 10 monclient: shutdown 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 -- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 msgr2=0x7f30b0071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0098a0 tx=0x7f30ac02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 -- 172.21.15.134:0/1403471633 shutdown_connections 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0132090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 --2- 172.21.15.134:0/1403471633 >> v2:172.21.15.134:3300/0 conn(0x7f30b0135190 0x7f30b01355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 --2- 172.21.15.134:0/1403471633 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b0134db0 0x7f30b0071d20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 -- 172.21.15.134:0/1403471633 >> 172.21.15.134:0/1403471633 conn(0x7f30b00a7480 msgr2=0x7f30b00a7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 -- 172.21.15.134:0/1403471633 shutdown_connections 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.965+0000 7f30b81c7640 1 -- 172.21.15.134:0/1403471633 wait complete. 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient: build_initial_monmap 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient: monmap: 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:30.981 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 Processor -- start 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 -- start start 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient: init 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient: _add_conns ranks=[4,8,2,3,1,0,5,6,7] 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0130ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-c con 0x7f30b0131c10 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30b0134db0 0x7f30b0131430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-e con 0x7f30b0134db0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30b0135190 0x7f30b011dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): picked mon.noname-b con 0x7f30b0135190 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01ca250 con 0x7f30b0135190 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00a5760 con 0x7f30b0131c10 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f30b011fa90 con 0x7f30b0134db0 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.966+0000 7f30b81c7640 10 monclient(hunting): authenticate will time out at 1727.555960s 2024-09-16T15:16:30.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0130ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0130ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47854/0 (socket says 172.21.15.134:47854) 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 -- 172.21.15.134:0/2529174098 learned_addr learned my addr 172.21.15.134:0/2529174098 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): get_auth_request con 0x7f30b0131c10 auth_method 0 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30b0134db0 0x7f30b0131430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): get_auth_request con 0x7f30b0134db0 auth_method 0 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30b0135190 0x7f30b011dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): get_auth_request con 0x7f30b0135190 auth_method 0 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b6764640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient(hunting): handle_auth_done global_id 5944 payload 824 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30b0134db0 msgr2=0x7f30b0131430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f30b0134db0 0x7f30b0131430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30b0135190 msgr2=0x7f30b011dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30b0135190 0x7f30b011dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient: _finish_hunting 0 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 monclient: found mon.noname-c 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.967+0000 7f30b5f63640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009a70 con 0x7f30b0131c10 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b5f63640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 msgr2=0x7f30b0130ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b5f63640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0130ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0090b0 tx=0x7f30ac0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac030d90 con 0x7f30b0131c10 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: dump: 2024-09-16T15:16:30.986 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.987 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: _finish_auth 0 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac003610 con 0x7f30b0131c10 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30ac003bf0 con 0x7f30b0131c10 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 5 monclient: authenticate success, global_id 5944 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a67fc640 10 monclient: dump: 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.988 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30a7fff640 4 set_mon_vals no callback set 2024-09-16T15:16:30.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 10 monclient: _renew_subs 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a0009400 con 0x7f30b0131c10 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 10 monclient: _renew_subs 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.969+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac0094d0 con 0x7f30b0131c10 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f30ac039070 con 0x7f30b0131c10 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b81c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b81c7640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 0x7f30b0121560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00678a0 con 0x7f30b01c95a0 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 0x7f30b0121560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient: get_auth_request con 0x7f30b01c95a0 auth_method 0 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30a67fc640 1 --2- 172.21.15.134:0/2529174098 >> v2:172.21.15.162:6800/980547263 conn(0x7f3094038880 0x7f309403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (secure 0 0 0) 0x7f30ac06a890 con 0x7f30b0131c10 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.970+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.971+0000 7f30b5762640 10 monclient(hunting): handle_auth_done global_id 94299 payload 824 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.971+0000 7f30b5762640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30b0121040 con 0x7f30b01c95a0 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.971+0000 7f30b6764640 1 --2- 172.21.15.134:0/2529174098 >> v2:172.21.15.162:6800/980547263 conn(0x7f3094038880 0x7f309403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.971+0000 7f30b6764640 10 monclient: get_auth_request con 0x7f3094038880 auth_method 0 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30b5762640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 msgr2=0x7f30b0121560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30b5762640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 0x7f30b0121560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30b00a8120 tx=0x7f30a000f610 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30a0011e80 con 0x7f30b01c95a0 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:30.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 10 monclient: dump: 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.992 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f30a000fe70 con 0x7f30b01c95a0 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 msgr2=0x7f30b0121560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 0x7f30b0121560 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f30b00a8120 tx=0x7f30a000f610 comp rx=0 tx=0).stop 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 msgr2=0x7f30b0121560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30a67fc640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b01c95a0 0x7f30b0121560 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.973+0000 7f30b6764640 1 --2- 172.21.15.134:0/2529174098 >> v2:172.21.15.162:6800/980547263 conn(0x7f3094038880 0x7f309403ad40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30b011f400 tx=0x7f30a800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b81c7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b81c7640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 0x7f30b011e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b81c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f30b006a3f0 con 0x7f30b011e010 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 0x7f30b011e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient: get_auth_request con 0x7f30b011e010 auth_method 0 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:30.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.979+0000 7f30b5762640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.980+0000 7f30b5762640 10 monclient(hunting): handle_auth_done global_id 94308 payload 824 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.980+0000 7f30b5762640 1 -- 172.21.15.134:0/2529174098 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f30b0061ae0 con 0x7f30b011e010 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30b5762640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 msgr2=0x7f30b011e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30b5762640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 0x7f30b011e420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30b00a8120 tx=0x7f30a00046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f30a0014e80 con 0x7f30b011e010 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 10 monclient: dump: 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f30a0026070 con 0x7f30b011e010 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:30.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 msgr2=0x7f30b011e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 0x7f30b011e420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30b00a8120 tx=0x7f30a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 msgr2=0x7f30b011e420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.982+0000 7f30a67fc640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f30b011e010 0x7f30b011e420 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30b00a8120 tx=0x7f30a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:30.998 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:30.999 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.000 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.001 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.002 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.003 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.004 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.005 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.006 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.007 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.008 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.009 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.010 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.012 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.013 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.014 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.015 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.016 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.017 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.018 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.019 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.020 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 >> v2:172.21.15.162:6800/980547263 conn(0x7f3094038880 msgr2=0x7f309403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 --2- 172.21.15.134:0/2529174098 >> v2:172.21.15.162:6800/980547263 conn(0x7f3094038880 0x7f309403ad40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30b011f400 tx=0x7f30a800b040 comp rx=0 tx=0).stop 2024-09-16T15:16:31.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 10 monclient: shutdown 2024-09-16T15:16:31.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 msgr2=0x7f30b0130ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 --2- 172.21.15.134:0/2529174098 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f30b0131c10 0x7f30b0130ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0090b0 tx=0x7f30ac0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b6764640 1 -- 172.21.15.134:0/2529174098 reap_dead start 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b6764640 1 -- 172.21.15.134:0/2529174098 reap_dead start 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 shutdown_connections 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 >> 172.21.15.134:0/2529174098 conn(0x7f30b00a7480 msgr2=0x7f30b009d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 shutdown_connections 2024-09-16T15:16:31.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:30.983+0000 7f30b81c7640 1 -- 172.21.15.134:0/2529174098 wait complete. 2024-09-16T15:16:31.022 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.023 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:16:31.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.119+0000 7fdf87661640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.119+0000 7fdf87661640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.119+0000 7fdf87661640 10 monclient: monmap: 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.120566+0000 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.120566+0000 2024-09-16T15:16:31.121 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 Processor -- start 2024-09-16T15:16:31.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 -- start start 2024-09-16T15:16:31.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient: init 2024-09-16T15:16:31.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient: _add_conns ranks=[4,7,6,8,3,5,2,10,0,9,1] 2024-09-16T15:16:31.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-c con 0x7fdf80136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf80137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-i con 0x7fdf80137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-d con 0x7fdf80135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8009abc0 con 0x7fdf80136e40 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8009ba50 con 0x7fdf80135dc0 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800a53c0 con 0x7fdf80137220 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.120+0000 7fdf87661640 10 monclient(hunting): authenticate will time out at 1727.709929s 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33986/0 (socket says 172.21.15.134:33986) 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47856/0 (socket says 172.21.15.134:47856) 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 1 -- 172.21.15.134:0/1197762408 learned_addr learned my addr 172.21.15.134:0/1197762408 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): get_auth_request con 0x7fdf80135dc0 auth_method 0 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): get_auth_request con 0x7fdf80136e40 auth_method 0 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf84bfc640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf80137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf84bfc640 10 monclient(hunting): get_auth_request con 0x7fdf80137220 auth_method 0 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf84bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf84bfc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf84bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient(hunting): handle_auth_done global_id 5953 payload 274 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 -- 172.21.15.134:0/1197762408 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 msgr2=0x7fdf80137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf80137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 -- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 msgr2=0x7fdf80136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 monclient: found mon.noname-c 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf853fd640 1 -- 172.21.15.134:0/1197762408 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80073250 con 0x7fdf80136e40 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.121+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf853fd640 1 -- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 msgr2=0x7fdf8009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097d0 tx=0x7fdf7c029fb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2a21c75d76c9280b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 1 -- 172.21.15.134:0/1197762408 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7c02ad10 con 0x7fdf80136e40 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: dump: 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.130 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 5 monclient: authenticate success, global_id 5953 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 1 -- 172.21.15.134:0/1197762408 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c02ae90 con 0x7fdf80136e40 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 4 set_mon_vals no callback set 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 1 -- 172.21.15.134:0/1197762408 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7c030c20 con 0x7fdf80136e40 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf77fff640 10 monclient: dump: 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.132 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 10 monclient: shutdown 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 -- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 msgr2=0x7fdf8009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097d0 tx=0x7fdf7c029fb0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 -- 172.21.15.134:0/1197762408 shutdown_connections 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fdf80135dc0 0x7fdf80136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf80137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 --2- 172.21.15.134:0/1197762408 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdf80136e40 0x7fdf8009c150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 -- 172.21.15.134:0/1197762408 >> 172.21.15.134:0/1197762408 conn(0x7fdf800a6110 msgr2=0x7fdf800a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.124+0000 7fdf87661640 1 -- 172.21.15.134:0/1197762408 shutdown_connections 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 1 -- 172.21.15.134:0/1197762408 wait complete. 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 10 monclient: monmap: 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.134 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.135 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 1 Processor -- start 2024-09-16T15:16:31.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 1 -- start start 2024-09-16T15:16:31.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.125+0000 7fdf87661640 10 monclient: init 2024-09-16T15:16:31.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient: _add_conns ranks=[1,2,7,0,8,3,4,5,6] 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf80135dc0 0x7fdf8011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-f con 0x7fdf80135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-b con 0x7fdf80136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf801cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): picked mon.noname-i con 0x7fdf80137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80122940 con 0x7fdf80135dc0 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80120c20 con 0x7fdf80136e40 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80122280 con 0x7fdf80137220 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf87661640 10 monclient(hunting): authenticate will time out at 1727.715321s 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33880/0 (socket says 172.21.15.134:33880) 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 -- 172.21.15.134:0/1843904885 learned_addr learned my addr 172.21.15.134:0/1843904885 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): get_auth_request con 0x7fdf80136e40 auth_method 0 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf80135dc0 0x7fdf8011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): get_auth_request con 0x7fdf80135dc0 auth_method 0 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf801cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 10 monclient(hunting): get_auth_request con 0x7fdf80137220 auth_method 0 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient(hunting): handle_auth_done global_id 114185 payload 824 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 msgr2=0x7fdf801cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf801cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf80135dc0 msgr2=0x7fdf8011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf80135dc0 0x7fdf8011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 monclient: found mon.noname-b 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf84bfc640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009b30 con 0x7fdf80136e40 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.126+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdf80137220 0x7fdf801cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.128+0000 7fdf84bfc640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 msgr2=0x7fdf801350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.128+0000 7fdf84bfc640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d620 tx=0x7fdf7000dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7000bca0 con 0x7fdf80136e40 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: dump: 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.145 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7000be40 con 0x7fdf80136e40 2024-09-16T15:16:31.146 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 5 monclient: authenticate success, global_id 114185 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 10 monclient: _renew_subs 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c009400 con 0x7fdf80136e40 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf700148a0 con 0x7fdf80136e40 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf75ffb640 10 monclient: dump: 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.147 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 10 monclient: _renew_subs 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf777fe640 4 set_mon_vals no callback set 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.129+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80121fa0 con 0x7fdf80136e40 2024-09-16T15:16:31.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf84bfc640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d620 tx=0x7fdf7000dae0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fdf7004a490 osd_map(718..718 src has 1..718) v4 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf87661640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdf70014430 con 0x7fdf80136e40 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf87661640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 0x7fdf8002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80071b20 con 0x7fdf801355f0 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 0x7fdf8002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient: get_auth_request con 0x7fdf801355f0 auth_method 0 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf75ffb640 1 --2- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 0x7fdf6403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 10 monclient(hunting): handle_auth_done global_id 84581 payload 824 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf853fd640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf80061ae0 con 0x7fdf801355f0 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 0x7fdf6403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 10 monclient: get_auth_request con 0x7fdf64038880 auth_method 0 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 1 -- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 msgr2=0x7fdf6403ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 1 -- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 msgr2=0x7fdf6403ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 0x7fdf6403ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.130+0000 7fdf85bfe640 1 --2- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 0x7fdf6403ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.132+0000 7fdf853fd640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 msgr2=0x7fdf8002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.132+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 0x7fdf8002b260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c034000 tx=0x7fdf7c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7c003d30 con 0x7fdf801355f0 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 10 monclient: dump: 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.150 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdf7c003e90 con 0x7fdf801355f0 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 msgr2=0x7fdf8002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 0x7fdf8002b260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c034000 tx=0x7fdf7c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 msgr2=0x7fdf8002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.133+0000 7fdf75ffb640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf801355f0 0x7fdf8002b260 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c034000 tx=0x7fdf7c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf87661640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf87661640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 0x7fdf8002bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf87661640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8007a680 con 0x7fdf8002b7a0 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 0x7fdf8002bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient: get_auth_request con 0x7fdf8002b7a0 auth_method 0 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 10 monclient(hunting): handle_auth_done global_id 84590 payload 824 2024-09-16T15:16:31.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.139+0000 7fdf853fd640 1 -- 172.21.15.134:0/1843904885 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdf8009b490 con 0x7fdf8002b7a0 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.141+0000 7fdf853fd640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 msgr2=0x7fdf8002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.141+0000 7fdf853fd640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 0x7fdf8002bbb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097d0 tx=0x7fdf7c008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf7c04c300 con 0x7fdf8002b7a0 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 10 monclient: dump: 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.154 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fdf7c030af0 con 0x7fdf8002b7a0 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 msgr2=0x7fdf8002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 0x7fdf8002bbb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097d0 tx=0x7fdf7c008030 comp rx=0 tx=0).stop 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 msgr2=0x7fdf8002bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdf8002b7a0 0x7fdf8002bbb0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097d0 tx=0x7fdf7c008030 comp rx=0 tx=0).stop 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:31.156 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.157 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.158 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.159 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.160 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.161 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.162 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.163 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.164 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.165 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.166 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.167 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.168 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.169 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.170 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.171 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.172 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.173 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.174 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.175 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.176 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 msgr2=0x7fdf6403ad40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:31.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 --2- 172.21.15.134:0/1843904885 >> v2:172.21.15.162:6800/980547263 conn(0x7fdf64038880 0x7fdf6403ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 10 monclient: shutdown 2024-09-16T15:16:31.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 msgr2=0x7fdf801350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 --2- 172.21.15.134:0/1843904885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf80136e40 0x7fdf801350b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d620 tx=0x7fdf7000dae0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf85bfe640 1 -- 172.21.15.134:0/1843904885 reap_dead start 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf85bfe640 1 -- 172.21.15.134:0/1843904885 reap_dead start 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf75ffb640 1 -- 172.21.15.134:0/1843904885 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (unknown 0 0 0) 0x7fdf7004a490 con 0x7fdf80136e40 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 shutdown_connections 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 >> 172.21.15.134:0/1843904885 conn(0x7fdf800a6110 msgr2=0x7fdf8009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 shutdown_connections 2024-09-16T15:16:31.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.142+0000 7fdf87661640 1 -- 172.21.15.134:0/1843904885 wait complete. 2024-09-16T15:16:31.178 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.178 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:16:31.179 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:16:31.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.276+0000 7f24271a0640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.276+0000 7f24271a0640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.276+0000 7f24271a0640 10 monclient: monmap: 2024-09-16T15:16:31.277 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.277 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.277246+0000 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.277246+0000 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.278 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.279 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.279 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.279 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.279 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.276+0000 7f24271a0640 1 Processor -- start 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 -- start start 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient: init 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient: _add_conns ranks=[8,5,2,1,3,9,6,7,0,4,10] 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-e con 0x7f2420135de0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:31.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f24201361c0 0x7f24201365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-h con 0x7f24201361c0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 0x7f24200a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-b con 0x7f24200a3ce0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242009b620 con 0x7f24200a3ce0 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24200a5540 con 0x7f24201361c0 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f24200a4e80 con 0x7f2420135de0 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f24271a0640 10 monclient(hunting): authenticate will time out at 1727.866788s 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242699f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 0x7f24200a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41752/0 (socket says 172.21.15.134:41752) 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 1 -- 172.21.15.134:0/1433270559 learned_addr learned my addr 172.21.15.134:0/1433270559 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): get_auth_request con 0x7f2420135de0 auth_method 0 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242699f640 10 monclient(hunting): get_auth_request con 0x7f24200a3ce0 auth_method 0 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242699f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242699f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242699f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242599d640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f24201361c0 0x7f24201365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.277+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): get_auth_request con 0x7f24201361c0 auth_method 0 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 10 monclient(hunting): handle_auth_done global_id 84599 payload 274 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 -- 172.21.15.134:0/1433270559 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f24201361c0 msgr2=0x7f24201365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f24201361c0 0x7f24201365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 -- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 msgr2=0x7f24200a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 0x7f24200a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 monclient: found mon.noname-e 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242619e640 1 -- 172.21.15.134:0/1433270559 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420073250 con 0x7f2420135de0 2024-09-16T15:16:31.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.278+0000 7f242699f640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 0x7f24200a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.280+0000 7f242619e640 1 -- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 msgr2=0x7f242009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242619e640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098d0 tx=0x7f241c02a0a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c18e49ec42f8793f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 1 -- 172.21.15.134:0/1433270559 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f241c02adc0 con 0x7f2420135de0 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: dump: 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.293 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 1 -- 172.21.15.134:0/1433270559 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c0307b0 con 0x7f2420135de0 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 5 monclient: authenticate success, global_id 84599 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 1 -- 172.21.15.134:0/1433270559 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f241c030dd0 con 0x7f2420135de0 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 4 set_mon_vals no callback set 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f242519c640 10 monclient: dump: 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.295 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 10 monclient: shutdown 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 1 -- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 msgr2=0x7f242009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.281+0000 7f24271a0640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098d0 tx=0x7f241c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 -- 172.21.15.134:0/1433270559 shutdown_connections 2024-09-16T15:16:31.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24200a3ce0 0x7f24200a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f24201361c0 0x7f24201365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 --2- 172.21.15.134:0/1433270559 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f2420135de0 0x7f242009bf00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 -- 172.21.15.134:0/1433270559 >> 172.21.15.134:0/1433270559 conn(0x7f2420066070 msgr2=0x7f2420066480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 -- 172.21.15.134:0/1433270559 shutdown_connections 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 1 -- 172.21.15.134:0/1433270559 wait complete. 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.282+0000 7f24271a0640 10 monclient: monmap: 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.297 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 Processor -- start 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 -- start start 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient: init 2024-09-16T15:16:31.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient: _add_conns ranks=[7,4,8,5,6,3,0,1,2] 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f24200a3ce0 0x7f2420134c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-i con 0x7f24200a3ce0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 0x7f2420135180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-c con 0x7f2420135de0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 0x7f242011f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): picked mon.noname-e con 0x7f24201361c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24201cc240 con 0x7f2420135de0 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f2420121d60 con 0x7f24200a3ce0 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f2420122320 con 0x7f24201361c0 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f24271a0640 10 monclient(hunting): authenticate will time out at 1727.872573s 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 0x7f2420135180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 0x7f242011f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 0x7f2420135180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47860/0 (socket says 172.21.15.134:47860) 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 1 -- 172.21.15.134:0/4100041992 learned_addr learned my addr 172.21.15.134:0/4100041992 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): get_auth_request con 0x7f2420135de0 auth_method 0 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): get_auth_request con 0x7f24201361c0 auth_method 0 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242619e640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f24200a3ce0 0x7f2420134c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242619e640 10 monclient(hunting): get_auth_request con 0x7f24200a3ce0 auth_method 0 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242619e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242619e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242619e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient(hunting): handle_auth_done global_id 84608 payload 824 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f24200a3ce0 msgr2=0x7f2420134c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f24200a3ce0 0x7f2420134c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 msgr2=0x7f2420135180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 0x7f2420135180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 monclient: found mon.noname-e 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242699f640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009a70 con 0x7f24201361c0 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.283+0000 7f242599d640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f2420135de0 0x7f2420135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f242699f640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 msgr2=0x7f242011f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f242699f640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 0x7f242011f930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f241800efc0 tx=0x7f241800c490 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24180148f0 con 0x7f24201361c0 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: dump: 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.302 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 5 monclient: authenticate success, global_id 84608 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800f040 con 0x7f24201361c0 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24180133e0 con 0x7f24201361c0 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 10 monclient: _renew_subs 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0094d0 con 0x7f24201361c0 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f242499b640 4 set_mon_vals no callback set 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f2416ffd640 10 monclient: dump: 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.304 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.305 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 10 monclient: _renew_subs 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.286+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200a48d0 con 0x7f24201361c0 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f24271a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f24271a0640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 0x7f2420120250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420070750 con 0x7f242011fe70 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f2418014480 con 0x7f24201361c0 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 0x7f2420120250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient: get_auth_request con 0x7f242011fe70 auth_method 0 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f2416ffd640 1 --2- 172.21.15.134:0/4100041992 >> v2:172.21.15.162:6800/980547263 conn(0x7f2404038880 0x7f240403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (secure 0 0 0) 0x7f241804b0f0 con 0x7f24201361c0 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242599d640 1 --2- 172.21.15.134:0/4100041992 >> v2:172.21.15.162:6800/980547263 conn(0x7f2404038880 0x7f240403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242599d640 10 monclient: get_auth_request con 0x7f2404038880 auth_method 0 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 10 monclient(hunting): handle_auth_done global_id 45118 payload 824 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.287+0000 7f242619e640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2420073810 con 0x7f242011fe70 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f242619e640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 msgr2=0x7f2420120250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f242619e640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 0x7f2420120250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098a0 tx=0x7f241c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f242599d640 1 --2- 172.21.15.134:0/4100041992 >> v2:172.21.15.162:6800/980547263 conn(0x7f2404038880 0x7f240403ad40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2420074f70 tx=0x7f2410009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f241c02adc0 con 0x7f242011fe70 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 10 monclient: dump: 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.308 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f241c030b30 con 0x7f242011fe70 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 msgr2=0x7f2420120250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 0x7f2420120250 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098a0 tx=0x7f241c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 msgr2=0x7f2420120250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.290+0000 7f2416ffd640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242011fe70 0x7f2420120250 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f24271a0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f24271a0640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 0x7f242009da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f24271a0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420069fe0 con 0x7f242009d670 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f242619e640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 0x7f242009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f242619e640 10 monclient: get_auth_request con 0x7f242009d670 auth_method 0 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f242619e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f242619e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.296+0000 7f242619e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.297+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.297+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.297+0000 7f242619e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.297+0000 7f242619e640 10 monclient(hunting): handle_auth_done global_id 45127 payload 824 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.297+0000 7f242619e640 1 -- 172.21.15.134:0/4100041992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2420061ae0 con 0x7f242009d670 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f242619e640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 msgr2=0x7f242009da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f242619e640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 0x7f242009da80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f241c0492c0 con 0x7f242009d670 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f2416ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f2416ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.299+0000 7f2416ffd640 10 monclient: dump: 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.312 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f241c049a70 con 0x7f242009d670 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 msgr2=0x7f242009da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 0x7f242009da80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c008030 comp rx=0 tx=0).stop 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 msgr2=0x7f242009da80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f2416ffd640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f242009d670 0x7f242009da80 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c008030 comp rx=0 tx=0).stop 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:31.314 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.315 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.316 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.317 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.318 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.319 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.320 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.322 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.323 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.324 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.326 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.327 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.328 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.329 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.330 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.331 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.332 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.333 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.334 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.335 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 >> v2:172.21.15.162:6800/980547263 conn(0x7f2404038880 msgr2=0x7f240403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 --2- 172.21.15.134:0/4100041992 >> v2:172.21.15.162:6800/980547263 conn(0x7f2404038880 0x7f240403ad40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2420074f70 tx=0x7f2410009190 comp rx=0 tx=0).stop 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 10 monclient: shutdown 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 msgr2=0x7f242011f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f242699f640 1 -- 172.21.15.134:0/4100041992 reap_dead start 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 0x7f242011f930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f241800efc0 tx=0x7f241800c490 comp rx=0 tx=0).stop 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 shutdown_connections 2024-09-16T15:16:31.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 --2- 172.21.15.134:0/4100041992 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f24201361c0 0x7f242011f930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 >> 172.21.15.134:0/4100041992 conn(0x7f2420066070 msgr2=0x7f24201bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 shutdown_connections 2024-09-16T15:16:31.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.300+0000 7f24271a0640 1 -- 172.21.15.134:0/4100041992 wait complete. 2024-09-16T15:16:31.338 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:16:31.338 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:16:31.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.435+0000 7f77ad7b5640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.435+0000 7f77ad7b5640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.435+0000 7f77ad7b5640 10 monclient: monmap: 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.436605+0000 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.436605+0000 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.437 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.438 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 Processor -- start 2024-09-16T15:16:31.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 -- start start 2024-09-16T15:16:31.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient: init 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient: _add_conns ranks=[0,6,3,4,7,9,2,8,5,1,10] 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f77a8133f10 addr v1:172.21.15.134:6789/0 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 0x7f77a8139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-d con 0x7f77a8071bd0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a813a510 0x7f77a813c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-g con 0x7f77a813a510 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a8073250 con 0x7f77a8133f10 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80a3d50 con 0x7f77a813a510 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806f630 con 0x7f77a8071bd0 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77ad7b5640 10 monclient(hunting): authenticate will time out at 1728.025751s 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 0x7f77a8139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a6ffd640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f77a8133f10 0x7f77a81342f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45626/0 (socket says 172.21.15.134:45626) 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a6ffd640 1 -- 172.21.15.134:0/1582087093 learned_addr learned my addr 172.21.15.134:0/1582087093 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): get_auth_request con 0x7f77a8071bd0 auth_method 0 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.436+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 10 monclient(hunting): handle_auth_done global_id 94326 payload 274 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a77fe640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a813a510 0x7f77a813c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 1 -- 172.21.15.134:0/1582087093 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a813a510 msgr2=0x7f77a813c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a813a510 0x7f77a813c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 1 -- 172.21.15.134:0/1582087093 >> v1:172.21.15.134:6789/0 conn(0x7f77a8133f10 legacy=0x7f77a81342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 1 monclient: found mon.noname-d 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.437+0000 7f77a67fc640 1 -- 172.21.15.134:0/1582087093 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80a1e10 con 0x7f77a8071bd0 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.439+0000 7f77a67fc640 1 -- 172.21.15.134:0/1582087093 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 msgr2=0x7f77a8139fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.439+0000 7f77a67fc640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 0x7f77a8139fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f779c0097d0 tx=0x7f779c02a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=23840cf456cd4242 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 1 -- 172.21.15.134:0/1582087093 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c02ae00 con 0x7f77a8071bd0 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: dump: 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.449 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 5 monclient: authenticate success, global_id 94326 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 1 -- 172.21.15.134:0/1582087093 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c0307b0 con 0x7f77a8071bd0 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 4 set_mon_vals no callback set 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 1 -- 172.21.15.134:0/1582087093 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f779c030d90 con 0x7f77a8071bd0 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77a5ffb640 10 monclient: dump: 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 10 monclient: shutdown 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 -- 172.21.15.134:0/1582087093 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 msgr2=0x7f77a8139fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 0x7f77a8139fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f779c0097d0 tx=0x7f779c02a000 comp rx=0 tx=0).stop 2024-09-16T15:16:31.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 -- 172.21.15.134:0/1582087093 shutdown_connections 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a813a510 0x7f77a813c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/1582087093 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a8071bd0 0x7f77a8139fd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 -- 172.21.15.134:0/1582087093 >> 172.21.15.134:0/1582087093 conn(0x7f77a80a7410 msgr2=0x7f77a80a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 -- 172.21.15.134:0/1582087093 shutdown_connections 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.440+0000 7f77ad7b5640 1 -- 172.21.15.134:0/1582087093 wait complete. 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.441+0000 7f77ad7b5640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.441+0000 7f77ad7b5640 10 monclient: monmap: 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.453 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.441+0000 7f77ad7b5640 1 Processor -- start 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.441+0000 7f77ad7b5640 1 -- start start 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.441+0000 7f77ad7b5640 10 monclient: init 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient: _add_conns ranks=[3,8,6,5,0,4,7,1,2] 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a8071bd0 0x7f77a8135da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-g con 0x7f77a8071bd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f77a8133f10 0x7f77a81362e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-e con 0x7f77a8133f10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): picked mon.noname-d con 0x7f77a813a510 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8135830 con 0x7f77a8071bd0 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8139880 con 0x7f77a813a510 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8139460 con 0x7f77a8133f10 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77ad7b5640 10 monclient(hunting): authenticate will time out at 1728.031322s 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f77a8133f10 0x7f77a81362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f77a8133f10 0x7f77a81362e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41768/0 (socket says 172.21.15.134:41768) 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 1 -- 172.21.15.134:0/189310378 learned_addr learned my addr 172.21.15.134:0/189310378 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): get_auth_request con 0x7f77a8133f10 auth_method 0 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34008/0 (socket says 172.21.15.134:34008) 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): get_auth_request con 0x7f77a813a510 auth_method 0 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a8071bd0 0x7f77a8135da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient(hunting): handle_auth_done global_id 94335 payload 824 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f77a8133f10 msgr2=0x7f77a81362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f77a8133f10 0x7f77a81362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a8071bd0 msgr2=0x7f77a8135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a8071bd0 0x7f77a8135da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 monclient: found mon.noname-d 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a77fe640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c009400 con 0x7f77a813a510 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a6ffd640 10 monclient: get_auth_request con 0x7f77a8071bd0 auth_method 0 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.442+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a8071bd0 0x7f77a8135da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.444+0000 7f77a77fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 msgr2=0x7f77a8136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.444+0000 7f77a77fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77a000cc50 tx=0x7f77a000a5c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77a000ce00 con 0x7f77a813a510 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: dump: 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.458 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a000fa70 con 0x7f77a813a510 2024-09-16T15:16:31.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 5 monclient: authenticate success, global_id 94335 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 monclient: _renew_subs 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a8135370 con 0x7f77a813a510 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77a000fc20 con 0x7f77a813a510 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 10 monclient: dump: 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.460 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77a57fa640 4 set_mon_vals no callback set 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 monclient: _renew_subs 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a8137c60 con 0x7f77a813a510 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 0x7f77a81cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8073880 con 0x7f77a81cb730 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.445+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f77a000f600 con 0x7f77a813a510 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 0x7f77a81cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f778f7fe640 1 --2- 172.21.15.134:0/189310378 >> v2:172.21.15.162:6800/980547263 conn(0x7f7794038880 0x7f779403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient: get_auth_request con 0x7f77a81cb730 auth_method 0 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 3963+0+0 (secure 0 0 0) 0x7f77a004a110 con 0x7f77a813a510 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a67fc640 1 --2- 172.21.15.134:0/189310378 >> v2:172.21.15.162:6800/980547263 conn(0x7f7794038880 0x7f779403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a67fc640 10 monclient: get_auth_request con 0x7f7794038880 auth_method 0 2024-09-16T15:16:31.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_done global_id 84459 payload 824 2024-09-16T15:16:31.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.446+0000 7f77a6ffd640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77a8061ae0 con 0x7f77a81cb730 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f77a6ffd640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 msgr2=0x7f77a81cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 0x7f77a81cbb10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f779800d9b0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f77a67fc640 1 --2- 172.21.15.134:0/189310378 >> v2:172.21.15.162:6800/980547263 conn(0x7f7794038880 0x7f779403ad40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f779c000c00 tx=0x7f779c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7798005cf0 con 0x7f77a81cb730 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f778f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f778f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.448+0000 7f778f7fe640 10 monclient: dump: 2024-09-16T15:16:31.465 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.466 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7798005e50 con 0x7f77a81cb730 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 msgr2=0x7f77a81cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 0x7f77a81cbb10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f779800d9b0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 msgr2=0x7f77a81cbb10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.449+0000 7f778f7fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cb730 0x7f77a81cbb10 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f779800d9b0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77ad7b5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 0x7f77a806af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77ad7b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806a3f0 con 0x7f77a81cc050 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 0x7f77a806af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient: get_auth_request con 0x7f77a81cc050 auth_method 0 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 10 monclient(hunting): handle_auth_done global_id 84468 payload 824 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.455+0000 7f77a6ffd640 1 -- 172.21.15.134:0/189310378 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f77a807a680 con 0x7f77a81cc050 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f77a6ffd640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 msgr2=0x7f77a806af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f77a6ffd640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 0x7f77a806af60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f7798004940 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7798021070 con 0x7f77a81cc050 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 10 monclient: dump: 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.469 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f77980077e0 con 0x7f77a81cc050 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 msgr2=0x7f77a806af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 0x7f77a806af60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f7798004940 comp rx=0 tx=0).stop 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 msgr2=0x7f77a806af60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.458+0000 7f778f7fe640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77a81cc050 0x7f77a806af60 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f779800d4f0 tx=0x7f7798004940 comp rx=0 tx=0).stop 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:31.471 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.472 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.473 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.476 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.477 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.478 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.479 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.480 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.481 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.482 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.483 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.484 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.485 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.486 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.487 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.488 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.490 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.491 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 >> v2:172.21.15.162:6800/980547263 conn(0x7f7794038880 msgr2=0x7f779403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/189310378 >> v2:172.21.15.162:6800/980547263 conn(0x7f7794038880 0x7f779403ad40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f779c000c00 tx=0x7f779c02e000 comp rx=0 tx=0).stop 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 10 monclient: shutdown 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 msgr2=0x7f77a8136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77a77fe640 1 -- 172.21.15.134:0/189310378 reap_dead start 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77a000cc50 tx=0x7f77a000a5c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 shutdown_connections 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 --2- 172.21.15.134:0/189310378 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77a813a510 0x7f77a8136820 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 >> 172.21.15.134:0/189310378 conn(0x7f77a80a7410 msgr2=0x7f77a80a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 shutdown_connections 2024-09-16T15:16:31.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.459+0000 7f77ad7b5640 1 -- 172.21.15.134:0/189310378 wait complete. 2024-09-16T15:16:31.494 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.494 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:16:31.494 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:16:31.523 INFO:tasks.rados.rados.0.smithi162.stdout:3394: done (0 left) 2024-09-16T15:16:31.523 INFO:tasks.rados.rados.0.smithi162.stdout:3395: snap_create 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.597+0000 7f47553ae640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.597+0000 7f47553ae640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.598+0000 7f47553ae640 10 monclient: monmap: 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.598944+0000 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.598944+0000 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.599 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.600 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.598+0000 7f47553ae640 1 Processor -- start 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.598+0000 7f47553ae640 1 -- start start 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.598+0000 7f47553ae640 10 monclient: init 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient: _add_conns ranks=[10,5,3,7,8,1,0,9,6,4,2] 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-f con 0x7f4750136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 0x7f4750137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-h con 0x7f4750137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-g con 0x7f4750135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475009abc0 con 0x7f4750136e40 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475009ba50 con 0x7f4750135dc0 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47500a53c0 con 0x7f4750137220 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f47553ae640 10 monclient(hunting): authenticate will time out at 1728.188121s 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 0x7f4750137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:43084/0 (socket says 172.21.15.134:43084) 2024-09-16T15:16:31.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:39924/0 (socket says 172.21.15.134:39924) 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 0x7f4750137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:57472/0 (socket says 172.21.15.134:57472) 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 -- 172.21.15.134:0/1792234996 learned_addr learned my addr 172.21.15.134:0/1792234996 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): get_auth_request con 0x7f4750135dc0 auth_method 0 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): get_auth_request con 0x7f4750136e40 auth_method 0 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): get_auth_request con 0x7f4750137220 auth_method 0 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient(hunting): handle_auth_done global_id 84477 payload 274 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 -- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 msgr2=0x7f4750137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 --2- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 0x7f4750137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 -- 172.21.15.134:0/1792234996 >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 msgr2=0x7f475009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 --2- 172.21.15.134:0/1792234996 >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 monclient: found mon.noname-g 2024-09-16T15:16:31.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:16:31.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474f7fe640 1 -- 172.21.15.134:0/1792234996 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750073250 con 0x7f4750135dc0 2024-09-16T15:16:31.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.599+0000 7f474effd640 1 --2- 172.21.15.134:0/1792234996 >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.602+0000 7f474f7fe640 1 -- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 msgr2=0x7f4750136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.602+0000 7f474f7fe640 1 --2- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4748009e80 tx=0x7f474802a580 comp rx=0 tx=0).ready entity=mon.3 client_cookie=1c7811512b9214ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 1 -- 172.21.15.134:0/1792234996 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4748007a70 con 0x7f4750135dc0 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: dump: 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.613 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 1 -- 172.21.15.134:0/1792234996 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474802d050 con 0x7f4750135dc0 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 5 monclient: authenticate success, global_id 84477 2024-09-16T15:16:31.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 1 -- 172.21.15.134:0/1792234996 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4748007bf0 con 0x7f4750135dc0 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 4 set_mon_vals no callback set 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f474dffb640 10 monclient: dump: 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.615 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 10 monclient: shutdown 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 -- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 msgr2=0x7f4750136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 --2- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4748009e80 tx=0x7f474802a580 comp rx=0 tx=0).stop 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 -- 172.21.15.134:0/1792234996 shutdown_connections 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 --2- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750135dc0 0x7f4750136220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 --2- 172.21.15.134:0/1792234996 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750137220 0x7f4750137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 --2- 172.21.15.134:0/1792234996 >> v2:172.21.15.162:3300/0 conn(0x7f4750136e40 0x7f475009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 -- 172.21.15.134:0/1792234996 >> 172.21.15.134:0/1792234996 conn(0x7f47500a6110 msgr2=0x7f47500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 -- 172.21.15.134:0/1792234996 shutdown_connections 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.603+0000 7f47553ae640 1 -- 172.21.15.134:0/1792234996 wait complete. 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.604+0000 7f47553ae640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.604+0000 7f47553ae640 10 monclient: monmap: 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.617 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.604+0000 7f47553ae640 1 Processor -- start 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 -- start start 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient: init 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient: _add_conns ranks=[6,3,0,4,8,2,1,7,5] 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 0x7f475009f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-d con 0x7f4750135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:31.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750136e40 0x7f4750135000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-g con 0x7f4750136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750137220 0x7f475011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): picked mon.noname-a con 0x7f4750137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501225b0 con 0x7f4750137220 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47501363a0 con 0x7f4750136e40 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f47501221d0 con 0x7f4750135dc0 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f47553ae640 10 monclient(hunting): authenticate will time out at 1728.194499s 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 0x7f475009f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 0x7f475009f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34024/0 (socket says 172.21.15.134:34024) 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 -- 172.21.15.134:0/2722395638 learned_addr learned my addr 172.21.15.134:0/2722395638 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): get_auth_request con 0x7f4750135dc0 auth_method 0 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750137220 0x7f475011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): get_auth_request con 0x7f4750137220 auth_method 0 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750136e40 0x7f4750135000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): get_auth_request con 0x7f4750136e40 auth_method 0 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient(hunting): handle_auth_done global_id 94344 payload 824 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750136e40 msgr2=0x7f4750135000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4750136e40 0x7f4750135000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750137220 msgr2=0x7f475011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750137220 0x7f475011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 monclient: found mon.noname-d 2024-09-16T15:16:31.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474effd640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009400 con 0x7f4750135dc0 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.605+0000 7f474f7fe640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4750137220 0x7f475011fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f474effd640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 msgr2=0x7f475009f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f474effd640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 0x7f475009f320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4750122360 tx=0x7f474000ee40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47400096e0 con 0x7f4750135dc0 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: dump: 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.622 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000cc10 con 0x7f4750135dc0 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 5 monclient: authenticate success, global_id 94344 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4740013d50 con 0x7f4750135dc0 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47377fe640 10 monclient: dump: 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.624 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f474d7fa640 4 set_mon_vals no callback set 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 10 monclient: _renew_subs 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4748009b30 con 0x7f4750135dc0 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 10 monclient: _renew_subs 2024-09-16T15:16:31.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.608+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47501c0760 con 0x7f4750135dc0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f474000cd80 con 0x7f4750135dc0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47553ae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47553ae640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 0x7f4750120960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4750071b20 con 0x7f4750135540 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47377fe640 1 --2- 172.21.15.134:0/2722395638 >> v2:172.21.15.162:6800/980547263 conn(0x7f473c038880 0x7f473c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(719..719 src has 1..719) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f474004a920 con 0x7f4750135dc0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 0x7f4750120960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient: get_auth_request con 0x7f4750135540 auth_method 0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474f7fe640 1 --2- 172.21.15.134:0/2722395638 >> v2:172.21.15.162:6800/980547263 conn(0x7f473c038880 0x7f473c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474f7fe640 10 monclient: get_auth_request con 0x7f473c038880 auth_method 0 2024-09-16T15:16:31.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 10 monclient(hunting): handle_auth_done global_id 45113 payload 824 2024-09-16T15:16:31.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.609+0000 7f474e7fc640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4750122410 con 0x7f4750135540 2024-09-16T15:16:31.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f474e7fc640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 msgr2=0x7f4750120960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f474e7fc640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 0x7f4750120960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f4744011d20 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f474f7fe640 1 --2- 172.21.15.134:0/2722395638 >> v2:172.21.15.162:6800/980547263 conn(0x7f473c038880 0x7f473c03ad40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4748030000 tx=0x7f4748009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f474400fd00 con 0x7f4750135540 2024-09-16T15:16:31.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 10 monclient: dump: 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.630 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f474400fe60 con 0x7f4750135540 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 msgr2=0x7f4750120960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 0x7f4750120960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f4744011d20 comp rx=0 tx=0).stop 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 msgr2=0x7f4750120960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.612+0000 7f47377fe640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4750135540 0x7f4750120960 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f4744011d20 comp rx=0 tx=0).stop 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.618+0000 7f47553ae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.618+0000 7f47553ae640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 0x7f475002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.618+0000 7f47553ae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.618+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4750065cf0 con 0x7f475002b260 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.618+0000 7f474e7fc640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 0x7f475002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient: get_auth_request con 0x7f475002b260 auth_method 0 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 10 monclient(hunting): handle_auth_done global_id 45122 payload 824 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.619+0000 7f474e7fc640 1 -- 172.21.15.134:0/2722395638 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4750061ae0 con 0x7f475002b260 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f474e7fc640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 msgr2=0x7f475002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f474e7fc640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 0x7f475002b670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f47440046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f474402c070 con 0x7f475002b260 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f47377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f47377fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:31.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.621+0000 7f47377fe640 10 monclient: dump: 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.633 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f4744024040 con 0x7f475002b260 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 msgr2=0x7f475002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 0x7f475002b670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f47440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 msgr2=0x7f475002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47377fe640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f475002b260 0x7f475002b670 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4744011860 tx=0x7f47440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.634 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:31.635 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.636 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.637 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.638 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.639 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.640 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.641 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.642 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.643 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.644 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.645 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.646 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.647 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.648 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.649 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.650 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.654 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.656 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 >> v2:172.21.15.162:6800/980547263 conn(0x7f473c038880 msgr2=0x7f473c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 --2- 172.21.15.134:0/2722395638 >> v2:172.21.15.162:6800/980547263 conn(0x7f473c038880 0x7f473c03ad40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4748030000 tx=0x7f4748009000 comp rx=0 tx=0).stop 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 10 monclient: shutdown 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 msgr2=0x7f475009f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 --2- 172.21.15.134:0/2722395638 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4750135dc0 0x7f475009f320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4750122360 tx=0x7f474000ee40 comp rx=0 tx=0).stop 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f474f7fe640 1 -- 172.21.15.134:0/2722395638 reap_dead start 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f474f7fe640 1 -- 172.21.15.134:0/2722395638 reap_dead start 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 shutdown_connections 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 >> 172.21.15.134:0/2722395638 conn(0x7f47500a6110 msgr2=0x7f47500a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 shutdown_connections 2024-09-16T15:16:31.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.622+0000 7f47553ae640 1 -- 172.21.15.134:0/2722395638 wait complete. 2024-09-16T15:16:31.659 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.659 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:16:31.659 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 10 monclient: monmap: 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.756083+0000 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.756083+0000 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.758 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 1 Processor -- start 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 1 -- start start 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.755+0000 7f354a6f7640 10 monclient: init 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient: _add_conns ranks=[10,4,8,6,0,7,1,5,9,2,3] 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 0x7f354409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-f con 0x7f3544136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-c con 0x7f3544137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-e con 0x7f3544135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354409abc0 con 0x7f3544136e40 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f354409ba50 con 0x7f3544137220 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f35440a53c0 con 0x7f3544135dc0 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f354a6f7640 10 monclient(hunting): authenticate will time out at 1728.345390s 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41772/0 (socket says 172.21.15.134:41772) 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47870/0 (socket says 172.21.15.134:47870) 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 -- 172.21.15.134:0/683189535 learned_addr learned my addr 172.21.15.134:0/683189535 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): get_auth_request con 0x7f3544135dc0 auth_method 0 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): get_auth_request con 0x7f3544137220 auth_method 0 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 1 --2- 172.21.15.134:0/683189535 >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 0x7f354409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 10 monclient(hunting): get_auth_request con 0x7f3544136e40 auth_method 0 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient(hunting): handle_auth_done global_id 84626 payload 274 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 -- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 msgr2=0x7f3544137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 -- 172.21.15.134:0/683189535 >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 msgr2=0x7f354409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 --2- 172.21.15.134:0/683189535 >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 0x7f354409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 monclient: found mon.noname-e 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3548c94640 1 -- 172.21.15.134:0/683189535 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544073250 con 0x7f3544135dc0 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f3543fff640 1 --2- 172.21.15.134:0/683189535 >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 0x7f354409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.756+0000 7f35437fe640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3548c94640 1 -- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 msgr2=0x7f3544136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3548c94640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e80 tx=0x7f353c02a580 comp rx=0 tx=0).ready entity=mon.8 client_cookie=aafb972eab193b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 1 -- 172.21.15.134:0/683189535 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f353c007a70 con 0x7f3544135dc0 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: dump: 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.766 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 1 -- 172.21.15.134:0/683189535 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c02d050 con 0x7f3544135dc0 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f354a6f7640 5 monclient: authenticate success, global_id 84626 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f354a6f7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f354a6f7640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f354a6f7640 4 set_mon_vals no callback set 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 1 -- 172.21.15.134:0/683189535 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f353c007c30 con 0x7f3544135dc0 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.759+0000 7f3542ffd640 10 monclient: dump: 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.768 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 10 monclient: shutdown 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 -- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 msgr2=0x7f3544136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e80 tx=0x7f353c02a580 comp rx=0 tx=0).stop 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 -- 172.21.15.134:0/683189535 shutdown_connections 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3544135dc0 0x7f3544136220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 --2- 172.21.15.134:0/683189535 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544137220 0x7f3544137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 --2- 172.21.15.134:0/683189535 >> v2:172.21.15.162:3300/0 conn(0x7f3544136e40 0x7f354409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 -- 172.21.15.134:0/683189535 >> 172.21.15.134:0/683189535 conn(0x7f35440a6110 msgr2=0x7f35440a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 -- 172.21.15.134:0/683189535 shutdown_connections 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.760+0000 7f354a6f7640 1 -- 172.21.15.134:0/683189535 wait complete. 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.761+0000 7f354a6f7640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.761+0000 7f354a6f7640 10 monclient: monmap: 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.770 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.771 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.761+0000 7f354a6f7640 1 Processor -- start 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.761+0000 7f354a6f7640 1 -- start start 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.761+0000 7f354a6f7640 10 monclient: init 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient: _add_conns ranks=[3,4,6,0,1,5,2,8,7] 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3544135dc0 0x7f354409f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-g con 0x7f3544135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-c con 0x7f3544136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3544137220 0x7f3544135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): picked mon.noname-d con 0x7f3544137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354409fdf0 con 0x7f3544135dc0 2024-09-16T15:16:31.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3544122130 con 0x7f3544136e40 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35440a8ba0 con 0x7f3544137220 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f354a6f7640 10 monclient(hunting): authenticate will time out at 1728.351114s 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47886/0 (socket says 172.21.15.134:47886) 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 -- 172.21.15.134:0/435379301 learned_addr learned my addr 172.21.15.134:0/435379301 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): get_auth_request con 0x7f3544136e40 auth_method 0 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3544137220 0x7f3544135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): get_auth_request con 0x7f3544137220 auth_method 0 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3548c94640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3543fff640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3544135dc0 0x7f354409f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3543fff640 10 monclient(hunting): get_auth_request con 0x7f3544135dc0 auth_method 0 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f3543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient(hunting): handle_auth_done global_id 5980 payload 824 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3544137220 msgr2=0x7f3544135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3544137220 0x7f3544135270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3544135dc0 msgr2=0x7f354409f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3544135dc0 0x7f354409f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 monclient: found mon.noname-c 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.762+0000 7f35437fe640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3538009400 con 0x7f3544136e40 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f35437fe640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 msgr2=0x7f3544134d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f35437fe640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f353800e960 tx=0x7f353800ee20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f353800cc10 con 0x7f3544136e40 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: dump: 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.785 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353800cd90 con 0x7f3544136e40 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 5 monclient: authenticate success, global_id 5980 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 10 monclient: _renew_subs 2024-09-16T15:16:31.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c009b30 con 0x7f3544136e40 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3538013d50 con 0x7f3544136e40 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f3540ff9640 10 monclient: dump: 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 10 monclient: _renew_subs 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f35427fc640 4 set_mon_vals no callback set 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:31.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.764+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35441364c0 con 0x7f3544136e40 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.765+0000 7f354a6f7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.765+0000 7f354a6f7640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 0x7f3544120420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.765+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.765+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3544071b20 con 0x7f35440a0420 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.765+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3538013470 con 0x7f3544136e40 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3540ff9640 1 --2- 172.21.15.134:0/435379301 >> v2:172.21.15.162:6800/980547263 conn(0x7f35300388b0 0x7f353003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(719..719 src has 1..719) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f353804a400 con 0x7f3544136e40 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 0x7f3544120420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient: get_auth_request con 0x7f35440a0420 auth_method 0 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3548c94640 1 --2- 172.21.15.134:0/435379301 >> v2:172.21.15.162:6800/980547263 conn(0x7f35300388b0 0x7f353003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3548c94640 10 monclient: get_auth_request con 0x7f35300388b0 auth_method 0 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 10 monclient(hunting): handle_auth_done global_id 114136 payload 824 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.766+0000 7f3543fff640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3544061ae0 con 0x7f35440a0420 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3548c94640 1 --2- 172.21.15.134:0/435379301 >> v2:172.21.15.162:6800/980547263 conn(0x7f35300388b0 0x7f353003ad70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f353c033000 tx=0x7f353c009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3543fff640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 msgr2=0x7f3544120420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3543fff640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 0x7f3544120420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534010c50 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f353400ebe0 con 0x7f35440a0420 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3540ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:31.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.768+0000 7f3540ff9640 10 monclient: dump: 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.791 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f353400c050 con 0x7f35440a0420 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 msgr2=0x7f3544120420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 0x7f3544120420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534010c50 comp rx=0 tx=0).stop 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 msgr2=0x7f3544120420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.769+0000 7f3540ff9640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35440a0420 0x7f3544120420 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534010c50 comp rx=0 tx=0).stop 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f354a6f7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f354a6f7640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 0x7f354402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f354a6f7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f354407a680 con 0x7f354402b260 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 0x7f354402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient: get_auth_request con 0x7f354402b260 auth_method 0 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.775+0000 7f3543fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.776+0000 7f3543fff640 10 monclient(hunting): handle_auth_done global_id 114145 payload 824 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.776+0000 7f3543fff640 1 -- 172.21.15.134:0/435379301 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f354409b490 con 0x7f354402b260 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3543fff640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 msgr2=0x7f354402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3543fff640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 0x7f354402b670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534004730 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3534021ce0 con 0x7f354402b260 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3540ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3540ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.778+0000 7f3540ff9640 10 monclient: dump: 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.794 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f3534021870 con 0x7f354402b260 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 msgr2=0x7f354402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 0x7f354402b670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534004730 comp rx=0 tx=0).stop 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 msgr2=0x7f354402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:31.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3540ff9640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f354402b260 0x7f354402b670 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3534010790 tx=0x7f3534004730 comp rx=0 tx=0).stop 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.801 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.802 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.804 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:31.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:31.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:31.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:31.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:31.817 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:31.818 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 >> v2:172.21.15.162:6800/980547263 conn(0x7f35300388b0 msgr2=0x7f353003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 --2- 172.21.15.134:0/435379301 >> v2:172.21.15.162:6800/980547263 conn(0x7f35300388b0 0x7f353003ad70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f353c033000 tx=0x7f353c009000 comp rx=0 tx=0).stop 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 10 monclient: shutdown 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 msgr2=0x7f3544134d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f353800e960 tx=0x7f353800ee20 comp rx=0 tx=0).stop 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f3548c94640 1 -- 172.21.15.134:0/435379301 reap_dead start 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 shutdown_connections 2024-09-16T15:16:31.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 --2- 172.21.15.134:0/435379301 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3544136e40 0x7f3544134d30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 >> 172.21.15.134:0/435379301 conn(0x7f35440a6110 msgr2=0x7f35440a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 shutdown_connections 2024-09-16T15:16:31.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.779+0000 7f354a6f7640 1 -- 172.21.15.134:0/435379301 wait complete. 2024-09-16T15:16:31.820 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:31.820 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:16:31.820 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.915+0000 7f1df50c9640 10 monclient: get_monmap_and_config 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.915+0000 7f1df50c9640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.915+0000 7f1df50c9640 10 monclient: monmap: 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:31.916714+0000 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:31.916714+0000 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.917 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:31.918 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 Processor -- start 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 -- start start 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient: init 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient: _add_conns ranks=[2,9,5,0,3,4,10,7,6,8,1] 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-b con 0x7f1df0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1df009bfe0 0x7f1df009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-a con 0x7f1df009bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1df0136c60 0x7f1df01370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-h con 0x7f1df0136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df009b600 con 0x7f1df009bfe0 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00a5740 con 0x7f1df0136030 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00a52c0 con 0x7f1df0136c60 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.916+0000 7f1df50c9640 10 monclient(hunting): authenticate will time out at 1728.505906s 2024-09-16T15:16:31.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1df009bfe0 0x7f1df009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33906/0 (socket says 172.21.15.134:33906) 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 1 -- 172.21.15.134:0/2285337117 learned_addr learned my addr 172.21.15.134:0/2285337117 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): get_auth_request con 0x7f1df009bfe0 auth_method 0 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): get_auth_request con 0x7f1df0136030 auth_method 0 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1def59e640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1df0136c60 0x7f1df01370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1def59e640 10 monclient(hunting): get_auth_request con 0x7f1df0136c60 auth_method 0 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1def59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1def59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1def59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient(hunting): handle_auth_done global_id 114174 payload 274 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 -- 172.21.15.134:0/2285337117 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1df0136c60 msgr2=0x7f1df01370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1df0136c60 0x7f1df01370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 -- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 msgr2=0x7f1df0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:31.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 monclient: found mon.noname-a 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1de659c640 1 -- 172.21.15.134:0/2285337117 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0073250 con 0x7f1df009bfe0 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.917+0000 7f1deed9d640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1de659c640 1 -- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df009bfe0 msgr2=0x7f1df009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1de659c640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df009bfe0 0x7f1df009c400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80097a0 tx=0x7f1dd8029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3d93793848136 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 1 -- 172.21.15.134:0/2285337117 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dd802ad30 con 0x7f1df009bfe0 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: dump: 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.926 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.927 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 1 -- 172.21.15.134:0/2285337117 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd80307b0 con 0x7f1df009bfe0 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 5 monclient: authenticate success, global_id 114174 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 1 -- 172.21.15.134:0/2285337117 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dd8030d90 con 0x7f1df009bfe0 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 4 set_mon_vals no callback set 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:31.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1dee59c640 10 monclient: dump: 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.929 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 10 monclient: shutdown 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 -- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df009bfe0 msgr2=0x7f1df009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df009bfe0 0x7f1df009c400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80097a0 tx=0x7f1dd8029fd0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 -- 172.21.15.134:0/2285337117 shutdown_connections 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1df0136c60 0x7f1df01370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1df009bfe0 0x7f1df009c400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2285337117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 -- 172.21.15.134:0/2285337117 >> 172.21.15.134:0/2285337117 conn(0x7f1df00a9ea0 msgr2=0x7f1df00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 -- 172.21.15.134:0/2285337117 shutdown_connections 2024-09-16T15:16:31.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.920+0000 7f1df50c9640 1 -- 172.21.15.134:0/2285337117 wait complete. 2024-09-16T15:16:31.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.921+0000 7f1df50c9640 10 monclient: build_initial_monmap 2024-09-16T15:16:31.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.921+0000 7f1df50c9640 10 monclient: monmap: 2024-09-16T15:16:31.935 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:31.935 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.921+0000 7f1df50c9640 1 Processor -- start 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 -- start start 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient: init 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient: _add_conns ranks=[1,2,3,4,5,6,8,7,0] 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df009bfe0 0x7f1df01bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-f con 0x7f1df009bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-b con 0x7f1df0136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0136c60 0x7f1df01352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): picked mon.noname-g con 0x7f1df0136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00a3230 con 0x7f1df009bfe0 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00a2e90 con 0x7f1df0136030 2024-09-16T15:16:31.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0122450 con 0x7f1df0136c60 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1df50c9640 10 monclient(hunting): authenticate will time out at 1728.511522s 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33918/0 (socket says 172.21.15.134:33918) 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 -- 172.21.15.134:0/2847168387 learned_addr learned my addr 172.21.15.134:0/2847168387 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): get_auth_request con 0x7f1df0136030 auth_method 0 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df009bfe0 0x7f1df01bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): get_auth_request con 0x7f1df009bfe0 auth_method 0 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0136c60 0x7f1df01352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): get_auth_request con 0x7f1df0136c60 auth_method 0 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1deed9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:31.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1def59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient(hunting): handle_auth_done global_id 114212 payload 824 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 -- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0136c60 msgr2=0x7f1df01352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0136c60 0x7f1df01352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 -- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df009bfe0 msgr2=0x7f1df01bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df009bfe0 0x7f1df01bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 1 monclient: found mon.noname-b 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.922+0000 7f1de659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.923+0000 7f1de659c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.923+0000 7f1de659c640 1 -- 172.21.15.134:0/2847168387 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00094d0 con 0x7f1df0136030 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de659c640 1 -- 172.21.15.134:0/2847168387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 msgr2=0x7f1df0134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de659c640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8035040 tx=0x7f1dd80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dd8003610 con 0x7f1df0136030 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: dump: 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.941 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd80037b0 con 0x7f1df0136030 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 5 monclient: authenticate success, global_id 114212 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 10 monclient: _renew_subs 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd8009400 con 0x7f1df0136030 2024-09-16T15:16:31.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dd8003d90 con 0x7f1df0136030 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1de7fff640 10 monclient: dump: 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:31.943 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 10 monclient: _renew_subs 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0120d00 con 0x7f1df0136030 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.925+0000 7f1dedd9b640 4 set_mon_vals no callback set 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1dd8039040 con 0x7f1df0136030 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1df50c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1df50c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1df50c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:31.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df01cb790 con 0x7f1df0136030 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1de7fff640 1 --2- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 0x7f1dd003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(719..719 src has 1..719) v4 ==== 4007+0+0 (secure 0 0 0) 0x7f1dd806a5a0 con 0x7f1df0136030 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1deed9d640 1 --2- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 0x7f1dd003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.926+0000 7f1deed9d640 10 monclient: get_auth_request con 0x7f1dd0038880 auth_method 0 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.929+0000 7f1deed9d640 1 --2- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 0x7f1dd003ad40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0006ca0 tx=0x7f1de0009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.931+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1df01cb790 con 0x7f1df0136030 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.931+0000 7f1de7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:31.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:31.931+0000 7f1de7fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:32.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.156+0000 7f1df50c9640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:16:32.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.156+0000 7f1df50c9640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:16:32.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.156+0000 7f1df50c9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:32.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.156+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1df01cb790 con 0x7f1df0136030 2024-09-16T15:16:32.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.158+0000 7f1de7fff640 1 -- 172.21.15.134:0/2847168387 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1df01cb790 con 0x7f1df0136030 2024-09-16T15:16:32.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.158+0000 7f1de7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:16:32.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.158+0000 7f1de7fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:32.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=11,logm=55} crc {auth=2315664752,config=2586702516,health=2413639559,kv=174400417,logm=1554690751}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=2,health=11,kv=12,logm=55} crc {auth=2899595167,config=2586702516,health=3985290910,kv=1090497770,logm=2824853449}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=20,config=2,health=11,kv=12,logm=55} crc {auth=2315664752,config=2586702516,health=493768148,kv=1090497770,logm=464595406}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=17,config=2,health=9,kv=11,logm=61} crc {auth=153019087,config=2586702516,health=2736984785,kv=1388001194,logm=3122776453}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=20,config=2,health=11,kv=11,logm=56} crc {auth=2315664752,config=2586702516,health=493768148,kv=1388001194,logm=476073485}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=2315664752,config=2586702516,health=2413639559,kv=1090497770,logm=2007925566}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=19,config=2,health=10,kv=11,logm=58} crc {auth=2614334890,config=2586702516,health=3887956675,kv=1090497770,logm=887171690}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=11,logm=57} crc {auth=1954731892,config=2586702516,health=2413639559,kv=1388001194,logm=3704839164}) 2024-09-16T15:16:32.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.158+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=12,logm=56} crc {auth=47786651,config=2586702516,health=1028721309,kv=1090497770,logm=4090624164}) 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 msgr2=0x7f1dd003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 0x7f1dd003ad40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0006ca0 tx=0x7f1de0009240 comp rx=0 tx=0).stop 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 10 monclient: shutdown 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 msgr2=0x7f1df0134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8035040 tx=0x7f1dd80046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:32.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 shutdown_connections 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> v2:172.21.15.162:6800/980547263 conn(0x7f1dd0038880 0x7f1dd003ad40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1df0136c60 0x7f1df01352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1df0136030 0x7f1df0134d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 --2- 172.21.15.134:0/2847168387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1df009bfe0 0x7f1df01bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 >> 172.21.15.134:0/2847168387 conn(0x7f1df00a9ea0 msgr2=0x7f1df006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 shutdown_connections 2024-09-16T15:16:32.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:32.160+0000 7f1df50c9640 1 -- 172.21.15.134:0/2847168387 wait complete. 2024-09-16T15:16:32.172 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:16:32.520 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4131168320}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3237845163}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3005042873}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4073239825}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.521 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505830220}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2387937616}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1356031490}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1583819313}) 2024-09-16T15:16:32.522 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.518+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1106647986}) 2024-09-16T15:16:32.525 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.525 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.525 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=543875884}) 2024-09-16T15:16:32.525 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.525 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1276055365}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3474234078}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.522+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2537280036}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3248422645}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.526 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3187282081}) 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4047377863}) 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2495034393}) 2024-09-16T15:16:32.527 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.523+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2795269762}) 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3366524893}) 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=764787749}) 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=593379682}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=937328611}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3060102396}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3726738454}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2339873536}) 2024-09-16T15:16:32.536 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:32.536 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2806513369}) 2024-09-16T15:16:32.536 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:32.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2903114622}) 2024-09-16T15:16:32.536 INFO:tasks.rados.rados.0.smithi162.stdout:3395: done (0 left) 2024-09-16T15:16:32.536 INFO:tasks.rados.rados.0.smithi162.stdout:3396: write oid 32 current snap is 353 2024-09-16T15:16:32.536 INFO:tasks.rados.rados.0.smithi162.stdout:3396: seq_num 1171 ranges {480997=434941,1407473=705079,2557501=785880} 2024-09-16T15:16:32.538 INFO:tasks.rados.rados.0.smithi162.stdout:3396: writing smithi16234619-32 from 480997 to 915938 tid 1 2024-09-16T15:16:32.544 INFO:tasks.rados.rados.0.smithi162.stdout:3396: writing smithi16234619-32 from 1407473 to 2112552 tid 2 2024-09-16T15:16:32.551 INFO:tasks.rados.rados.0.smithi162.stdout:3396: writing smithi16234619-32 from 2557501 to 3343381 tid 3 2024-09-16T15:16:32.551 INFO:tasks.rados.rados.0.smithi162.stdout:3397: copy_from oid 8 from oid 16 current snap is 353 2024-09-16T15:16:32.551 INFO:tasks.rados.rados.0.smithi162.stdout:3398: snap_create 2024-09-16T15:16:32.552 INFO:tasks.rados.rados.0.smithi162.stdout:3397: finishing copy_from to smithi16234619-8 2024-09-16T15:16:32.552 INFO:tasks.rados.rados.0.smithi162.stdout:3397: got expected ENOENT (src dne) 2024-09-16T15:16:32.552 INFO:tasks.rados.rados.0.smithi162.stdout:3397: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:16:32.584 INFO:tasks.rados.rados.0.smithi162.stdout:3396: finishing write tid 1 to smithi16234619-32 2024-09-16T15:16:32.584 INFO:tasks.rados.rados.0.smithi162.stdout:3396: finishing write tid 2 to smithi16234619-32 2024-09-16T15:16:32.584 INFO:tasks.rados.rados.0.smithi162.stdout:3396: finishing write tid 3 to smithi16234619-32 2024-09-16T15:16:32.584 INFO:tasks.rados.rados.0.smithi162.stdout:3396: finishing write tid 4 to smithi16234619-32 2024-09-16T15:16:32.584 INFO:tasks.rados.rados.0.smithi162.stdout:3396: finishing write tid 5 to smithi16234619-32 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:3396: oid 32 updating version 0 to 1474 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:3396: oid 32 updating version 1474 to 1476 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:3396: oid 32 version 1476 is already newer than 1473 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:3396: oid 32 version 1476 is already newer than 1475 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 1476 (ObjNum 1171 snap 353 seq_num 1171) dirty exists 2024-09-16T15:16:32.585 INFO:tasks.rados.rados.0.smithi162.stdout:3396: left oid 32 (ObjNum 1171 snap 353 seq_num 1171) 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3023186861}) 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.022 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3297659774}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2417311771}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2153690813}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=877618864}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=616110606}) 2024-09-16T15:16:33.023 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1007335161}) 2024-09-16T15:16:33.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1468730831}) 2024-09-16T15:16:33.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.020+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1429622795}) 2024-09-16T15:16:33.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1596933999}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=183829966}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=163586843}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2032937208}) 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1149926958}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2932053106}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1995859756}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3889177251}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.024+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4055009915}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=998415028}) 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2185409355}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1643525340}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3981605145}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=979361348}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=137605183}) 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.043 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.025+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1647021059}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.026+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.026+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1436553084}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.026+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4211670809}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2362797974}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2764208827}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3312157850}) 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.044 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=852825719}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3087474243}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2310725342}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2655405572}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=677016386}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.027+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4280098415}) 2024-09-16T15:16:33.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3738164297}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3720178312}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=26266362}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1879893032}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2808297673}) 2024-09-16T15:16:33.046 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1643882577}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2106259144}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=140357818}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.028+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1064411348}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1828341157}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2225254542}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.029+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3624933811}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1480751174}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1093002247}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2169283027}) 2024-09-16T15:16:33.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2759397707}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3927699326}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.030+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195646037}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3903978168}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=255067092}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2694289468}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2731248717}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=315427451}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3748320932}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1220365520}) 2024-09-16T15:16:33.050 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3085700549}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.031+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4163994280}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3527818459}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4923408}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2502118377}) 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=97097307}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2281721398}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3546908482}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2861105569}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4188430441}) 2024-09-16T15:16:33.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.032+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3640256476}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1911184538}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=834759965}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2629118858}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2161304832}) 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.033+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=533527037}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=783798545}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2397260678}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2960909276}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.034+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3805398320}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3584349551}) 2024-09-16T15:16:33.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2683907694}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4011632717}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2399390931}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2190778088}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4115014830}) 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1968882325}) 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2490357061}) 2024-09-16T15:16:33.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.035+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3137621112}) 2024-09-16T15:16:33.173 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:16:33.173 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:33.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 10 monclient: get_monmap_and_config 2024-09-16T15:16:33.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 10 monclient: build_initial_monmap 2024-09-16T15:16:33.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 10 monclient: monmap: 2024-09-16T15:16:33.271 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:33.271 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:33.271129+0000 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:33.271129+0000 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:33.272 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:33.273 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:33.273 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:33.273 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 1 Processor -- start 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 1 -- start start 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.270+0000 7f5232c72640 10 monclient: init 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient: _add_conns ranks=[4,5,2,10,7,3,9,6,0,8,1] 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-c con 0x7f522c136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:33.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 0x7f522c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-h con 0x7f522c09bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-b con 0x7f522c136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c09b600 con 0x7f522c136c60 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0a5740 con 0x7f522c136030 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0a52c0 con 0x7f522c09bfe0 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5232c72640 10 monclient(hunting): authenticate will time out at 1729.860485s 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47890/0 (socket says 172.21.15.134:47890) 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33932/0 (socket says 172.21.15.134:33932) 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 -- 172.21.15.134:0/1354914250 learned_addr learned my addr 172.21.15.134:0/1354914250 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): get_auth_request con 0x7f522c136030 auth_method 0 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): get_auth_request con 0x7f522c136c60 auth_method 0 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:33.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient(hunting): handle_auth_done global_id 5989 payload 274 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f522bfff640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 0x7f522c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 -- 172.21.15.134:0/1354914250 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 msgr2=0x7f522c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 0x7f522c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 -- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 msgr2=0x7f522c1370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient: _finish_hunting 0 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 monclient: found mon.noname-c 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:33.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:33.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f5230a0e640 1 -- 172.21.15.134:0/1354914250 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c073250 con 0x7f522c136030 2024-09-16T15:16:33.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f522bfff640 10 monclient: get_auth_request con 0x7f522c09bfe0 auth_method 0 2024-09-16T15:16:33.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f523120f640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:33.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.271+0000 7f522bfff640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 0x7f522c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5230a0e640 1 -- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 msgr2=0x7f522c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5230a0e640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f52200098a0 tx=0x7f522002a080 comp rx=0 tx=0).ready entity=mon.4 client_cookie=87a899c07d6c49d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 1 -- 172.21.15.134:0/1354914250 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f522002ade0 con 0x7f522c136030 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: dump: 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:33.282 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 1 -- 172.21.15.134:0/1354914250 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52200307b0 con 0x7f522c136030 2024-09-16T15:16:33.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5232c72640 5 monclient: authenticate success, global_id 5989 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5232c72640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5232c72640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 1 -- 172.21.15.134:0/1354914250 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5220030d90 con 0x7f522c136030 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f5232c72640 4 set_mon_vals no callback set 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.274+0000 7f522b7fe640 10 monclient: dump: 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.284 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 10 monclient: shutdown 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:33.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 -- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 msgr2=0x7f522c136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f52200098a0 tx=0x7f522002a080 comp rx=0 tx=0).stop 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 -- 172.21.15.134:0/1354914250 shutdown_connections 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136c60 0x7f522c1370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f522c09bfe0 0x7f522c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 --2- 172.21.15.134:0/1354914250 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f522c136030 0x7f522c136410 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 -- 172.21.15.134:0/1354914250 >> 172.21.15.134:0/1354914250 conn(0x7f522c0a9ea0 msgr2=0x7f522c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 -- 172.21.15.134:0/1354914250 shutdown_connections 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 1 -- 172.21.15.134:0/1354914250 wait complete. 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.275+0000 7f5232c72640 10 monclient: build_initial_monmap 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.276+0000 7f5232c72640 10 monclient: monmap: 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:33.286 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.276+0000 7f5232c72640 1 Processor -- start 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.276+0000 7f5232c72640 1 -- start start 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.276+0000 7f5232c72640 10 monclient: init 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient: _add_conns ranks=[7,2,3,8,4,5,6,1,0] 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f522c09bfe0 0x7f522c11fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-i con 0x7f522c09bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-b con 0x7f522c136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c136c60 0x7f522c135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): picked mon.noname-g con 0x7f522c136c60 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c120b30 con 0x7f522c136030 2024-09-16T15:16:33.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0a7dd0 con 0x7f522c136c60 2024-09-16T15:16:33.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:33.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f522c121e10 con 0x7f522c09bfe0 2024-09-16T15:16:33.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:33.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5232c72640 10 monclient(hunting): authenticate will time out at 1729.866180s 2024-09-16T15:16:33.289 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3002749802}) 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1359843832}) 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.290 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2821462831}) 2024-09-16T15:16:33.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.291 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33944/0 (socket says 172.21.15.134:33944) 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 -- 172.21.15.134:0/276283538 learned_addr learned my addr 172.21.15.134:0/276283538 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): get_auth_request con 0x7f522c136030 auth_method 0 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:33.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:33.291 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.292 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2881218735}) 2024-09-16T15:16:33.292 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.292 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f522c09bfe0 0x7f522c11fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): get_auth_request con 0x7f522c09bfe0 auth_method 0 2024-09-16T15:16:33.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c136c60 0x7f522c135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): get_auth_request con 0x7f522c136c60 auth_method 0 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:33.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097054101}) 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2096652823}) 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=903940342}) 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.294 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4049072341}) 2024-09-16T15:16:33.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:33.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:33.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f5230a0e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:33.295 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3489038177}) 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f523120f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient(hunting): handle_auth_done global_id 114230 payload 824 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 -- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f522c09bfe0 msgr2=0x7f522c11fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f522c09bfe0 0x7f522c11fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 -- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c136c60 msgr2=0x7f522c135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c136c60 0x7f522c135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 monclient: found mon.noname-b 2024-09-16T15:16:33.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:33.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:33.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.277+0000 7f522bfff640 1 -- 172.21.15.134:0/276283538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009a70 con 0x7f522c136030 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1276238736}) 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1250751738}) 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.298 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=129631313}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3495837911}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1243726188}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2478743664}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=305479847}) 2024-09-16T15:16:33.299 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=68397714}) 2024-09-16T15:16:33.300 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=467974095}) 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.279+0000 7f522bfff640 1 -- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 msgr2=0x7f522c134cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.279+0000 7f522bfff640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f521800c7f0 tx=0x7f521800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f521800cea0 con 0x7f522c136030 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: dump: 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:33.302 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:33.303 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218004030 con 0x7f522c136030 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 5 monclient: authenticate success, global_id 114230 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 10 monclient: _renew_subs 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52200094d0 con 0x7f522c136030 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5218003610 con 0x7f522c136030 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f52297fa640 10 monclient: dump: 2024-09-16T15:16:33.304 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:33.305 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f522affd640 4 set_mon_vals no callback set 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 10 monclient: _renew_subs 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.280+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c1223d0 con 0x7f522c136030 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f522bfff640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f521800c7f0 tx=0x7f521800ccb0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f521804a050 osd_map(720..720 src has 1..720) v4 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f52180041e0 con 0x7f522c136030 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f52297fa640 1 --2- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 0x7f521003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5232c72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5232c72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5232c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c09c400 con 0x7f522c136030 2024-09-16T15:16:33.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5230a0e640 1 --2- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 0x7f521003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:33.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.281+0000 7f5230a0e640 10 monclient: get_auth_request con 0x7f5210038880 auth_method 0 2024-09-16T15:16:33.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.284+0000 7f5230a0e640 1 --2- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 0x7f521003ad40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5220009870 tx=0x7f522002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:33.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:33.286+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(720..720 src has 1..720) v4 ==== 3979+0+0 (secure 0 0 0) 0x7f521804a050 con 0x7f522c136030 2024-09-16T15:16:33.318 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2261921661}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=345081762}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3585681005}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1218949093}) 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1705013308}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292374550}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3428411783}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2393147445}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.304+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3049508604}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4255286770}) 2024-09-16T15:16:33.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2381203953}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2530532567}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3376305913}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=106423699}) 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=93369964}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2196857114}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4024383362}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.306+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3764292189}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=928040897}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3807098560}) 2024-09-16T15:16:33.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=612072555}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=602890213}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2259384735}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173415320}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2719813269}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1172848173}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.307+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1854808660}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=87334840}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=859289556}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3490951065}) 2024-09-16T15:16:33.324 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1085888982}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3206665326}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4105566464}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.308+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.309+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=218616000}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.309+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.309+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1028981059}) 2024-09-16T15:16:33.325 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.309+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=578550259}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2205583424}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3700862075}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2931552622}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=31264362}) 2024-09-16T15:16:33.326 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2029244127}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2505462287}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=192458366}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323993962}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.310+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=537643843}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.327 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1495289446}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4006882580}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2268336906}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=269054356}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=174929055}) 2024-09-16T15:16:33.328 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=978428746}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4173828428}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4240530848}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.311+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2062652966}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=993993766}) 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.329 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3506525428}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4210399747}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.312+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2849011744}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2983849059}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2922323252}) 2024-09-16T15:16:33.330 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.331 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.331 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2356548894}) 2024-09-16T15:16:33.331 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.331 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941078910}) 2024-09-16T15:16:33.331 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.313+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1310347051}) 2024-09-16T15:16:33.532 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.532 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=306041139}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2204357882}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3198601651}) 2024-09-16T15:16:33.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1897503273}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4003703083}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1262455995}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4231929757}) 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=118968079}) 2024-09-16T15:16:33.535 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=34691350}) 2024-09-16T15:16:33.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4158051175}) 2024-09-16T15:16:33.549 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1124292727}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3456033824}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=694685296}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2617521908}) 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.550 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1497336527}) 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3780941744}) 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=517204645}) 2024-09-16T15:16:33.551 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.547+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1138485936}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1289091931}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3079397211}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1068798973}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.558 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1143323467}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.555+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=304564391}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1776411789}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=802800022}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643460904}) 2024-09-16T15:16:33.559 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.556+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1840878094}) 2024-09-16T15:16:33.560 INFO:tasks.rados.rados.0.smithi162.stdout:3396: done (2 left) 2024-09-16T15:16:33.560 INFO:tasks.rados.rados.0.smithi162.stdout:3397: done (1 left) 2024-09-16T15:16:33.560 INFO:tasks.rados.rados.0.smithi162.stdout:3398: done (0 left) 2024-09-16T15:16:33.560 INFO:tasks.rados.rados.0.smithi162.stdout:3399: snap_create 2024-09-16T15:16:33.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3712245087}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2275720878}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2584229585}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2254052130}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2756211202}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231759092}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3023744056}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2503417900}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.921+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=299464663}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4196991661}) 2024-09-16T15:16:33.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3203156066}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1670138124}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2568798737}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1743357468}) 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2773125675}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=488630392}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3465015101}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.923+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4171085402}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2148503532}) 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=987051008}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=702048229}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=801750853}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=26914221}) 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1161903367}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3817696168}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3548586542}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.924+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2747755839}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4193280355}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1603927984}) 2024-09-16T15:16:33.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1222758487}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=837640514}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1688092806}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1064544891}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3402348146}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3643390020}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.925+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1054738966}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=868507381}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4230981070}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.926+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2327114706}) 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1238427883}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4119892673}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2743015655}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=702950170}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959174489}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.927+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=850646757}) 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4011041048}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1064451675}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3781877739}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2938098290}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2672042008}) 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2562757606}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=852602922}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701842655}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.928+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248307347}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=100137853}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2693307842}) 2024-09-16T15:16:33.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1129075013}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2672423269}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2567159018}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2735898665}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3470695103}) 2024-09-16T15:16:33.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885933501}) 2024-09-16T15:16:33.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.929+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=282724825}) 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=50323894}) 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3699785242}) 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:33.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1414821916}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=96749359}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1767775229}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2705222626}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=357613682}) 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.000 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3775499647}) 2024-09-16T15:16:34.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:33.996+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3849139726}) 2024-09-16T15:16:34.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2528340893}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2771956463}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2212556614}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2909777336}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=929391225}) 2024-09-16T15:16:34.004 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3752787350}) 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3418755413}) 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.001+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2559887280}) 2024-09-16T15:16:34.005 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.002+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2170696891}) 2024-09-16T15:16:34.005 INFO:tasks.rados.rados.0.smithi162.stdout:3399: done (0 left) 2024-09-16T15:16:34.006 INFO:tasks.rados.rados.0.smithi162.stdout:3400: delete oid 19 current snap is 355 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1341907512}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3727762672}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1019737812}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=574733611}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=669186959}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=814739788}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=825262323}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=477723552}) 2024-09-16T15:16:34.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.007+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=12986277}) 2024-09-16T15:16:34.017 INFO:tasks.rados.rados.0.smithi162.stdout:3400: done (0 left) 2024-09-16T15:16:34.017 INFO:tasks.rados.rados.0.smithi162.stdout:3401: write oid 13 current snap is 355 2024-09-16T15:16:34.018 INFO:tasks.rados.rados.0.smithi162.stdout:3401: seq_num 1173 ranges {514611=585439,1524437=592395,2895994=408424} 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=502366413}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3502029801}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2477788458}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=132439655}) 2024-09-16T15:16:34.024 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3398216058}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2060393506}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4108992125}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.021+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.022+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=262902329}) 2024-09-16T15:16:34.025 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.022+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3026828910}) 2024-09-16T15:16:34.026 INFO:tasks.rados.rados.0.smithi162.stdout:3401: writing smithi16234619-13 from 514611 to 1100050 tid 1 2024-09-16T15:16:34.029 INFO:tasks.rados.rados.0.smithi162.stdout:3401: writing smithi16234619-13 from 1524437 to 2116832 tid 2 2024-09-16T15:16:34.036 INFO:tasks.rados.rados.0.smithi162.stdout:3401: writing smithi16234619-13 from 2895994 to 3304418 tid 3 2024-09-16T15:16:34.036 INFO:tasks.rados.rados.0.smithi162.stdout:3402: snap_create 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: finishing write tid 1 to smithi16234619-13 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: finishing write tid 2 to smithi16234619-13 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: finishing write tid 3 to smithi16234619-13 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: finishing write tid 4 to smithi16234619-13 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: finishing write tid 5 to smithi16234619-13 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: oid 13 updating version 0 to 1873 2024-09-16T15:16:34.065 INFO:tasks.rados.rados.0.smithi162.stdout:3401: oid 13 updating version 1873 to 1875 2024-09-16T15:16:34.066 INFO:tasks.rados.rados.0.smithi162.stdout:3401: oid 13 version 1875 is already newer than 1874 2024-09-16T15:16:34.066 INFO:tasks.rados.rados.0.smithi162.stdout:3401: oid 13 version 1875 is already newer than 1872 2024-09-16T15:16:34.066 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 1875 (ObjNum 1173 snap 355 seq_num 1173) dirty exists 2024-09-16T15:16:34.066 INFO:tasks.rados.rados.0.smithi162.stdout:3401: left oid 13 (ObjNum 1173 snap 355 seq_num 1173) 2024-09-16T15:16:34.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.109+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5218004490 con 0x7f522c136030 2024-09-16T15:16:34.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.109+0000 7f52297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.109+0000 7f52297fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f5232c72640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f5232c72640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f5232c72640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f522c072070 con 0x7f522c136030 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f52297fa640 1 -- 172.21.15.134:0/276283538 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f521803c070 con 0x7f522c136030 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f52297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.333+0000 7f52297fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:34.335 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":136,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 msgr2=0x7f521003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 0x7f521003ad40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5220009870 tx=0x7f522002e000 comp rx=0 tx=0).stop 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 10 monclient: shutdown 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 msgr2=0x7f522c134cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.335+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f521800c7f0 tx=0x7f521800ccb0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 shutdown_connections 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> v2:172.21.15.162:6800/980547263 conn(0x7f5210038880 0x7f521003ad40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f522c136c60 0x7f522c135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f522c136030 0x7f522c134cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 --2- 172.21.15.134:0/276283538 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f522c09bfe0 0x7f522c11fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 >> 172.21.15.134:0/276283538 conn(0x7f522c0a9ea0 msgr2=0x7f522c0a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 shutdown_connections 2024-09-16T15:16:34.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.336+0000 7f5232c72640 1 -- 172.21.15.134:0/276283538 wait complete. 2024-09-16T15:16:34.347 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:16:34.348 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:16:34.348 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:16:34.348 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.444+0000 7fd122e61640 10 monclient: get_monmap_and_config 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.444+0000 7fd122e61640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.444+0000 7fd122e61640 10 monclient: monmap: 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:34.445678+0000 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:34.445678+0000 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.446 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:34.447 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 Processor -- start 2024-09-16T15:16:34.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 -- start start 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient: init 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient: _add_conns ranks=[8,5,4,9,0,1,10,7,3,2,6] 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-e con 0x7fd11c136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd11c137220 0x7fd11c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-h con 0x7fd11c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-c con 0x7fd11c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c09abc0 con 0x7fd11c135dc0 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c09ba50 con 0x7fd11c137220 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c0a53c0 con 0x7fd11c136e40 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.445+0000 7fd122e61640 10 monclient(hunting): authenticate will time out at 1731.034986s 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47904/0 (socket says 172.21.15.134:47904) 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41776/0 (socket says 172.21.15.134:41776) 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 1 -- 172.21.15.134:0/285006945 learned_addr learned my addr 172.21.15.134:0/285006945 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): get_auth_request con 0x7fd11c135dc0 auth_method 0 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): get_auth_request con 0x7fd11c136e40 auth_method 0 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd11bfff640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd11c137220 0x7fd11c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd11bfff640 10 monclient(hunting): get_auth_request con 0x7fd11c137220 auth_method 0 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd11bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd11bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd11bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient(hunting): handle_auth_done global_id 84635 payload 274 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 -- 172.21.15.134:0/285006945 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd11c137220 msgr2=0x7fd11c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd11c137220 0x7fd11c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 -- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 msgr2=0x7fd11c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 monclient: found mon.noname-e 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd120bfd640 1 -- 172.21.15.134:0/285006945 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c073250 con 0x7fd11c136e40 2024-09-16T15:16:34.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.446+0000 7fd1213fe640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd120bfd640 1 -- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 msgr2=0x7fd11c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd120bfd640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1100097d0 tx=0x7fd110029fb0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=16ae9423644555ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 1 -- 172.21.15.134:0/285006945 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd11002ad10 con 0x7fd11c136e40 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: dump: 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.461 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 5 monclient: authenticate success, global_id 84635 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 1 -- 172.21.15.134:0/285006945 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11002ae90 con 0x7fd11c136e40 2024-09-16T15:16:34.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 1 -- 172.21.15.134:0/285006945 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd110030c20 con 0x7fd11c136e40 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 4 set_mon_vals no callback set 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd11b7fe640 10 monclient: dump: 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.463 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 10 monclient: shutdown 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 -- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 msgr2=0x7fd11c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1100097d0 tx=0x7fd110029fb0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 -- 172.21.15.134:0/285006945 shutdown_connections 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c135dc0 0x7fd11c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd11c137220 0x7fd11c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 --2- 172.21.15.134:0/285006945 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd11c136e40 0x7fd11c09c150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.449+0000 7fd122e61640 1 -- 172.21.15.134:0/285006945 >> 172.21.15.134:0/285006945 conn(0x7fd11c0a6110 msgr2=0x7fd11c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 1 -- 172.21.15.134:0/285006945 shutdown_connections 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 1 -- 172.21.15.134:0/285006945 wait complete. 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 10 monclient: monmap: 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.465 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 1 Processor -- start 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 1 -- start start 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.450+0000 7fd122e61640 10 monclient: init 2024-09-16T15:16:34.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient: _add_conns ranks=[0,4,7,3,6,1,2,5,8] 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c135dc0 0x7fd11c1c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-a con 0x7fd11c135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 0x7fd11c135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-c con 0x7fd11c136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd11c137220 0x7fd11c1cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): picked mon.noname-i con 0x7fd11c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c1363b0 con 0x7fd11c135dc0 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c1233c0 con 0x7fd11c136e40 2024-09-16T15:16:34.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c124660 con 0x7fd11c137220 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd122e61640 10 monclient(hunting): authenticate will time out at 1731.040539s 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 0x7fd11c135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c135dc0 0x7fd11c1c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 0x7fd11c135470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47916/0 (socket says 172.21.15.134:47916) 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 -- 172.21.15.134:0/4280657483 learned_addr learned my addr 172.21.15.134:0/4280657483 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): get_auth_request con 0x7fd11c136e40 auth_method 0 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): get_auth_request con 0x7fd11c135dc0 auth_method 0 2024-09-16T15:16:34.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd1213fe640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd11c137220 0x7fd11c1cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient(hunting): handle_auth_done global_id 6007 payload 824 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd11c137220 msgr2=0x7fd11c1cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd11c137220 0x7fd11c1cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c135dc0 msgr2=0x7fd11c1c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c135dc0 0x7fd11c1c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 monclient: found mon.noname-c 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd11bfff640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd114009b30 con 0x7fd11c136e40 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd1213fe640 10 monclient: get_auth_request con 0x7fd11c137220 auth_method 0 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.451+0000 7fd1213fe640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd11c137220 0x7fd11c1cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd11bfff640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 msgr2=0x7fd11c135470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd11bfff640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 0x7fd11c135470 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd10800d5f0 tx=0x7fd10800dab0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd108012c70 con 0x7fd11c136e40 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: dump: 2024-09-16T15:16:34.470 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.471 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 5 monclient: authenticate success, global_id 6007 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108012e10 con 0x7fd11c136e40 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 10 monclient: _renew_subs 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd110009400 con 0x7fd11c136e40 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1080148a0 con 0x7fd11c136e40 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd11affd640 4 set_mon_vals no callback set 2024-09-16T15:16:34.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd1197fa640 10 monclient: dump: 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.473 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 10 monclient: _renew_subs 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.454+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c125e60 con 0x7fd11c136e40 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd122e61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd108014430 con 0x7fd11c136e40 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd122e61640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 0x7fd11c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c071b20 con 0x7fd11c1359b0 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 0x7fd11c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient: get_auth_request con 0x7fd11c1359b0 auth_method 0 2024-09-16T15:16:34.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd1197fa640 1 --2- 172.21.15.134:0/4280657483 >> v2:172.21.15.162:6800/980547263 conn(0x7fd100038880 0x7fd10003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(722..722 src has 1..722) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fd10804a6d0 con 0x7fd11c136e40 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 10 monclient(hunting): handle_auth_done global_id 114192 payload 824 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd120bfd640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd11c061ae0 con 0x7fd11c1359b0 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd1213fe640 1 --2- 172.21.15.134:0/4280657483 >> v2:172.21.15.162:6800/980547263 conn(0x7fd100038880 0x7fd10003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.455+0000 7fd1213fe640 10 monclient: get_auth_request con 0x7fd100038880 auth_method 0 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd120bfd640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 msgr2=0x7fd11c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd120bfd640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 0x7fd11c02b260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd110034000 tx=0x7fd1100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd110003d30 con 0x7fd11c1359b0 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd1197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd1197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.457+0000 7fd1197fa640 10 monclient: dump: 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.476 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd110003e90 con 0x7fd11c1359b0 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 msgr2=0x7fd11c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 0x7fd11c02b260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd110034000 tx=0x7fd1100046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 msgr2=0x7fd11c02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1197fa640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c1359b0 0x7fd11c02b260 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd110034000 tx=0x7fd1100046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.458+0000 7fd1213fe640 1 --2- 172.21.15.134:0/4280657483 >> v2:172.21.15.162:6800/980547263 conn(0x7fd100038880 0x7fd10003ad40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd114009fd0 tx=0x7fd11400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd122e61640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd122e61640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 0x7fd11c02bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd122e61640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c07a680 con 0x7fd11c02b7a0 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 0x7fd11c02bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient: get_auth_request con 0x7fd11c02b7a0 auth_method 0 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 10 monclient(hunting): handle_auth_done global_id 114201 payload 824 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.464+0000 7fd120bfd640 1 -- 172.21.15.134:0/4280657483 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd11c09b490 con 0x7fd11c02b7a0 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd120bfd640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 msgr2=0x7fd11c02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd120bfd640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 0x7fd11c02bbb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1100097d0 tx=0x7fd110008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd11004c2f0 con 0x7fd11c02b7a0 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd1197fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd1197fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.466+0000 7fd1197fa640 10 monclient: dump: 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.480 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9364 (secure 0 0 0) 0x7fd110030af0 con 0x7fd11c02b7a0 2024-09-16T15:16:34.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 msgr2=0x7fd11c02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 0x7fd11c02bbb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1100097d0 tx=0x7fd110008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 msgr2=0x7fd11c02bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1197fa640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd11c02b7a0 0x7fd11c02bbb0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1100097d0 tx=0x7fd110008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:34.482 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.483 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:34.484 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:34.485 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.486 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:34.487 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.488 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:34.489 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.490 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.491 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.492 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.493 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:34.494 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.495 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.496 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.497 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.498 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.499 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.500 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.501 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.502 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.503 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.503 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.503 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.503 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:34.503 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:34.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 >> v2:172.21.15.162:6800/980547263 conn(0x7fd100038880 msgr2=0x7fd10003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 --2- 172.21.15.134:0/4280657483 >> v2:172.21.15.162:6800/980547263 conn(0x7fd100038880 0x7fd10003ad40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd114009fd0 tx=0x7fd11400b040 comp rx=0 tx=0).stop 2024-09-16T15:16:34.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 10 monclient: shutdown 2024-09-16T15:16:34.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 msgr2=0x7fd11c135470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 --2- 172.21.15.134:0/4280657483 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd11c136e40 0x7fd11c135470 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd10800d5f0 tx=0x7fd10800dab0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1213fe640 1 -- 172.21.15.134:0/4280657483 reap_dead start 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd1213fe640 1 -- 172.21.15.134:0/4280657483 reap_dead start 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 shutdown_connections 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 >> 172.21.15.134:0/4280657483 conn(0x7fd11c0a6110 msgr2=0x7fd11c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 shutdown_connections 2024-09-16T15:16:34.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.467+0000 7fd122e61640 1 -- 172.21.15.134:0/4280657483 wait complete. 2024-09-16T15:16:34.505 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:34.505 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:16:34.506 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:16:34.537 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.537 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.537 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1729922853}) 2024-09-16T15:16:34.537 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4170608432}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=695364532}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2626553572}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2805805171}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2167623029}) 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.538 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.539 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4199122785}) 2024-09-16T15:16:34.539 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.539 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3933179384}) 2024-09-16T15:16:34.539 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3262244312}) 2024-09-16T15:16:34.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.555 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4247671570}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2886041025}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=588755172}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2264851379}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2620794931}) 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.556 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2764013875}) 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=434391415}) 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162394774}) 2024-09-16T15:16:34.557 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.553+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3470148771}) 2024-09-16T15:16:34.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.608+0000 7fbe58027640 10 monclient: get_monmap_and_config 2024-09-16T15:16:34.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.608+0000 7fbe58027640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.608+0000 7fbe58027640 10 monclient: monmap: 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:34.609362+0000 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:34.609362+0000 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.610 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.611 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.611 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.611 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:34.611 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:34.611 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 Processor -- start 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 -- start start 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient: init 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient: _add_conns ranks=[2,3,1,0,4,8,7,5,10,6,9] 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-b con 0x7fbe50136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe50137220 0x7fbe50137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-g con 0x7fbe50137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:34.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbe50135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe50073250 con 0x7fbe50135dc0 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5009abc0 con 0x7fbe50136e40 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5009ba50 con 0x7fbe50137220 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe58027640 10 monclient(hunting): authenticate will time out at 1731.198808s 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33952/0 (socket says 172.21.15.134:33952) 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 1 -- 172.21.15.134:0/1046006673 learned_addr learned my addr 172.21.15.134:0/1046006673 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request con 0x7fbe50136e40 auth_method 0 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.609+0000 7fbe55dc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe555c2640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe50137220 0x7fbe50137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe555c2640 10 monclient(hunting): get_auth_request con 0x7fbe50137220 auth_method 0 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe555c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe555c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe555c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_done global_id 114239 payload 274 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 1 -- 172.21.15.134:0/1046006673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe50137220 msgr2=0x7fbe50137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe50137220 0x7fbe50137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 1 -- 172.21.15.134:0/1046006673 >> v1:172.21.15.162:6789/0 conn(0x7fbe50135dc0 legacy=0x7fbe50136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 1 monclient: found mon.noname-b 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:34.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.610+0000 7fbe55dc3640 1 -- 172.21.15.134:0/1046006673 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500a53c0 con 0x7fbe50136e40 2024-09-16T15:16:34.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.612+0000 7fbe55dc3640 1 -- 172.21.15.134:0/1046006673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 msgr2=0x7fbe5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.612+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe440098a0 tx=0x7fbe4402a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c604e835c044adc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 1 -- 172.21.15.134:0/1046006673 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4402adf0 con 0x7fbe50136e40 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: dump: 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.622 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 1 -- 172.21.15.134:0/1046006673 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440307b0 con 0x7fbe50136e40 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 5 monclient: authenticate success, global_id 114239 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:34.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 1 -- 172.21.15.134:0/1046006673 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe44030d90 con 0x7fbe50136e40 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe54dc1640 10 monclient: dump: 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.624 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 4 set_mon_vals no callback set 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 10 monclient: shutdown 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 -- 172.21.15.134:0/1046006673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 msgr2=0x7fbe5009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe440098a0 tx=0x7fbe4402a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 -- 172.21.15.134:0/1046006673 shutdown_connections 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe50137220 0x7fbe50137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 --2- 172.21.15.134:0/1046006673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50136e40 0x7fbe5009c150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 -- 172.21.15.134:0/1046006673 >> 172.21.15.134:0/1046006673 conn(0x7fbe500a6110 msgr2=0x7fbe500a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 -- 172.21.15.134:0/1046006673 shutdown_connections 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.613+0000 7fbe58027640 1 -- 172.21.15.134:0/1046006673 wait complete. 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.614+0000 7fbe58027640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.614+0000 7fbe58027640 10 monclient: monmap: 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.626 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.614+0000 7fbe58027640 1 Processor -- start 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.614+0000 7fbe58027640 1 -- start start 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.614+0000 7fbe58027640 10 monclient: init 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient: _add_conns ranks=[6,4,8,5,3,7,0,2,1] 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe50135dc0 0x7fbe501349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-d con 0x7fbe50135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:34.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 0x7fbe50134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-c con 0x7fbe50136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 0x7fbe5011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): picked mon.noname-e con 0x7fbe50137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500a8ae0 con 0x7fbe50136e40 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50136930 con 0x7fbe50135dc0 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501363e0 con 0x7fbe50137220 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe58027640 10 monclient(hunting): authenticate will time out at 1731.204224s 2024-09-16T15:16:34.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe50135dc0 0x7fbe501349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 0x7fbe50134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 0x7fbe5011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 0x7fbe50134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47924/0 (socket says 172.21.15.134:47924) 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe50135dc0 0x7fbe501349f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34038/0 (socket says 172.21.15.134:34038) 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 1 -- 172.21.15.134:0/3514760358 learned_addr learned my addr 172.21.15.134:0/3514760358 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): get_auth_request con 0x7fbe50136e40 auth_method 0 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): get_auth_request con 0x7fbe50137220 auth_method 0 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request con 0x7fbe50135dc0 auth_method 0 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient(hunting): handle_auth_done global_id 84644 payload 824 2024-09-16T15:16:34.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe50135dc0 msgr2=0x7fbe501349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbe50135dc0 0x7fbe501349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 msgr2=0x7fbe50134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 0x7fbe50134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 monclient: found mon.noname-e 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe565c4640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440094d0 con 0x7fbe50137220 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.615+0000 7fbe555c2640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbe50136e40 0x7fbe50134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.617+0000 7fbe565c4640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 msgr2=0x7fbe5011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.617+0000 7fbe565c4640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 0x7fbe5011fc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c7e0 tx=0x7fbe4800cca0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe4800ce90 con 0x7fbe50137220 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: dump: 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.632 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48012420 con 0x7fbe50137220 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 5 monclient: authenticate success, global_id 84644 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 10 monclient: _renew_subs 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe48012a00 con 0x7fbe50137220 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:34.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe50122410 con 0x7fbe50137220 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 10 monclient: dump: 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.634 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe43fff640 4 set_mon_vals no callback set 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 10 monclient: _renew_subs 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe50120ef0 con 0x7fbe50137220 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.618+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbe48012590 con 0x7fbe50137220 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe58027640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe58027640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 0x7fbe5002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50073670 con 0x7fbe50135470 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe427fc640 1 --2- 172.21.15.134:0/3514760358 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe34038880 0x7fbe3403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(722..722 src has 1..722) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fbe4804a140 con 0x7fbe50137220 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 0x7fbe5002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient: get_auth_request con 0x7fbe50135470 auth_method 0 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe555c2640 1 --2- 172.21.15.134:0/3514760358 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe34038880 0x7fbe3403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe555c2640 10 monclient: get_auth_request con 0x7fbe34038880 auth_method 0 2024-09-16T15:16:34.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_done global_id 114248 payload 824 2024-09-16T15:16:34.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.619+0000 7fbe55dc3640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe50061ae0 con 0x7fbe50135470 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe55dc3640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 msgr2=0x7fbe5002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 0x7fbe5002b260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe5006b550 tx=0x7fbe440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe44004490 con 0x7fbe50135470 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:34.639 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe427fc640 10 monclient: dump: 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.640 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.621+0000 7fbe555c2640 1 --2- 172.21.15.134:0/3514760358 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe34038880 0x7fbe3403ad40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe500a2fa0 tx=0x7fbe4c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbe44030d40 con 0x7fbe50135470 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 msgr2=0x7fbe5002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 0x7fbe5002b260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe5006b550 tx=0x7fbe440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 msgr2=0x7fbe5002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.622+0000 7fbe427fc640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe50135470 0x7fbe5002b260 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe5006b550 tx=0x7fbe440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe58027640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe58027640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 0x7fbe5002bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe58027640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50071b20 con 0x7fbe5002b7a0 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 0x7fbe5002bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient: get_auth_request con 0x7fbe5002b7a0 auth_method 0 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 10 monclient(hunting): handle_auth_done global_id 114257 payload 824 2024-09-16T15:16:34.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.628+0000 7fbe55dc3640 1 -- 172.21.15.134:0/3514760358 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbe5007a680 con 0x7fbe5002b7a0 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe55dc3640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 msgr2=0x7fbe5002bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe55dc3640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 0x7fbe5002bb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe440097c0 tx=0x7fbe44008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe44003d80 con 0x7fbe5002b7a0 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe427fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe427fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.630+0000 7fbe427fc640 10 monclient: dump: 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.643 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fbe44003910 con 0x7fbe5002b7a0 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:34.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 msgr2=0x7fbe5002bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 0x7fbe5002bb80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe440097c0 tx=0x7fbe44008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 msgr2=0x7fbe5002bb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe427fc640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe5002b7a0 0x7fbe5002bb80 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe440097c0 tx=0x7fbe44008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:34.645 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.646 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.647 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:34.648 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:34.649 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:34.650 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:34.651 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.652 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.653 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.654 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.655 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:34.657 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.658 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.660 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.662 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.663 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.664 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.665 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.666 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:34.666 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe34038880 msgr2=0x7fbe3403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 --2- 172.21.15.134:0/3514760358 >> v2:172.21.15.162:6800/980547263 conn(0x7fbe34038880 0x7fbe3403ad40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe500a2fa0 tx=0x7fbe4c008040 comp rx=0 tx=0).stop 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 10 monclient: shutdown 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 msgr2=0x7fbe5011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 0x7fbe5011fc60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c7e0 tx=0x7fbe4800cca0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe565c4640 1 -- 172.21.15.134:0/3514760358 reap_dead start 2024-09-16T15:16:34.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 shutdown_connections 2024-09-16T15:16:34.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 --2- 172.21.15.134:0/3514760358 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbe50137220 0x7fbe5011fc60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 >> 172.21.15.134:0/3514760358 conn(0x7fbe500a6110 msgr2=0x7fbe5009f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 shutdown_connections 2024-09-16T15:16:34.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.631+0000 7fbe58027640 1 -- 172.21.15.134:0/3514760358 wait complete. 2024-09-16T15:16:34.668 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:34.668 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:16:34.668 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3742482154}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3832422989}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3788888775}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=790937512}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3999817318}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2769618738}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2810738545}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3060013268}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.713+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1178497022}) 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.736 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2448612449}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1071805175}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1289209826}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3431838795}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1704769384}) 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.737 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2956602000}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4256505365}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3859106219}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.715+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1401721649}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3328732965}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=857963598}) 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.738 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3210812948}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2839502834}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=363783391}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4082915929}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2534471216}) 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.739 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4222555571}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.716+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1569553094}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3611439148}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2954987293}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1388298881}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=112858475}) 2024-09-16T15:16:34.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1298848232}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.717+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1818354517}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2130208088}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3888568717}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.718+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2390003879}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=629327183}) 2024-09-16T15:16:34.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=698848778}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3144143169}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1441122338}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2125828395}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1861001352}) 2024-09-16T15:16:34.742 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=495668299}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3622979881}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.719+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2189179328}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3844775442}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=91426367}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=727891976}) 2024-09-16T15:16:34.743 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2394593648}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3078733487}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=30712053}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2314770894}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=533979942}) 2024-09-16T15:16:34.744 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.720+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=341711261}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4181084715}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4066187110}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.721+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2256418147}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1788517327}) 2024-09-16T15:16:34.745 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=990224152}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362312198}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4279044863}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3330301973}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.722+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=230715146}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2847456805}) 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.746 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1238982635}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2887160178}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=144866800}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1399148677}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.747 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2228895352}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3307319925}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4021114456}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.723+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=327508040}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=57938916}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=86684479}) 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=791110481}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1400933342}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390974254}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1900217190}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1693172803}) 2024-09-16T15:16:34.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1133095637}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.724+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1894484299}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4209955143}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=972428510}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.725+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=814791607}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3550322106}) 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.750 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2620597053}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3337967257}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2716700127}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=607197097}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.726+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423661367}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3967364933}) 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.751 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1155443829}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4250930001}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1918250473}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1105138847}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2580362248}) 2024-09-16T15:16:34.752 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=438530400}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=904330218}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.727+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1902147369}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3318687708}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1458605375}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.753 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2939784619}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2094944559}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2087232849}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3850125447}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2315631837}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323667873}) 2024-09-16T15:16:34.754 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.728+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1200663427}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1628023678}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=958967913}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.729+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1293830135}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1169508948}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2714980701}) 2024-09-16T15:16:34.755 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2568385650}) 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2777637811}) 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2247085177}) 2024-09-16T15:16:34.756 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.730+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3782089585}) 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.808+0000 7f7c796a3640 10 monclient: get_monmap_and_config 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.808+0000 7f7c796a3640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.809+0000 7f7c796a3640 10 monclient: monmap: 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:34.809907+0000 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:34.809907+0000 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.810 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:34.811 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.809+0000 7f7c796a3640 1 Processor -- start 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.809+0000 7f7c796a3640 1 -- start start 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.809+0000 7f7c796a3640 10 monclient: init 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient: _add_conns ranks=[9,8,1,6,0,7,4,3,2,5,10] 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c74136e40 0x7f7c7409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-a con 0x7f7c74136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 0x7f7c74137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-e con 0x7f7c74137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f7c74135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c74073250 con 0x7f7c74135dc0 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7409abc0 con 0x7f7c74136e40 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7409ba50 con 0x7f7c74137220 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c796a3640 10 monclient(hunting): authenticate will time out at 1731.399124s 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c74136e40 0x7f7c7409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 0x7f7c74137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c74136e40 0x7f7c7409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46120/0 (socket says 172.21.15.134:46120) 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1824447081 learned_addr learned my addr 172.21.15.134:0/1824447081 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request con 0x7f7c74136e40 auth_method 0 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): get_auth_request con 0x7f7c74137220 auth_method 0 2024-09-16T15:16:34.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_done global_id 114210 payload 274 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 msgr2=0x7f7c74137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 0x7f7c74137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1824447081 >> v1:172.21.15.162:6789/0 conn(0x7f7c74135dc0 legacy=0x7f7c74136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 monclient: found mon.noname-a 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1824447081 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740a53c0 con 0x7f7c74136e40 2024-09-16T15:16:34.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.810+0000 7f7c727fc640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 0x7f7c74137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c74136e40 msgr2=0x7f7c7409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c74136e40 0x7f7c7409c150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009870 tx=0x7f7c6402a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5581b80818aca89e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 1 -- 172.21.15.134:0/1824447081 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c6402ae00 con 0x7f7c74136e40 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: dump: 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.824 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 1 -- 172.21.15.134:0/1824447081 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640307b0 con 0x7f7c74136e40 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.825 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 1 -- 172.21.15.134:0/1824447081 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c64030d90 con 0x7f7c74136e40 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 5 monclient: authenticate success, global_id 114210 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 4 set_mon_vals no callback set 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c71ffb640 10 monclient: dump: 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.826 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 10 monclient: shutdown 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 1 -- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c74136e40 msgr2=0x7f7c7409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.813+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c74136e40 0x7f7c7409c150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009870 tx=0x7f7c6402a0e0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.827 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 -- 172.21.15.134:0/1824447081 shutdown_connections 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7c74137220 0x7f7c74137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1824447081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c74136e40 0x7f7c7409c150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 -- 172.21.15.134:0/1824447081 >> 172.21.15.134:0/1824447081 conn(0x7f7c740a6110 msgr2=0x7f7c740a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 -- 172.21.15.134:0/1824447081 shutdown_connections 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 1 -- 172.21.15.134:0/1824447081 wait complete. 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 10 monclient: build_initial_monmap 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.814+0000 7f7c796a3640 10 monclient: monmap: 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:34.828 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 Processor -- start 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 -- start start 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient: init 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient: _add_conns ranks=[1,6,2,4,8,5,7,3,0] 2024-09-16T15:16:34.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c74135dc0 0x7f7c74134df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-f con 0x7f7c74135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 0x7f7c74135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-d con 0x7f7c74136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c74137220 0x7f7c741cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): picked mon.noname-b con 0x7f7c74137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741224d0 con 0x7f7c74135dc0 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74120c10 con 0x7f7c74137220 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741363e0 con 0x7f7c74136e40 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c796a3640 10 monclient(hunting): authenticate will time out at 1731.404685s 2024-09-16T15:16:34.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 0x7f7c74135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 0x7f7c74135330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34040/0 (socket says 172.21.15.134:34040) 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 1 -- 172.21.15.134:0/1907649194 learned_addr learned my addr 172.21.15.134:0/1907649194 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): get_auth_request con 0x7f7c74136e40 auth_method 0 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c74137220 0x7f7c741cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): get_auth_request con 0x7f7c74137220 auth_method 0 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c74135dc0 0x7f7c74134df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request con 0x7f7c74135dc0 auth_method 0 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.815+0000 7f7c72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 10 monclient(hunting): handle_auth_done global_id 94371 payload 824 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c74137220 msgr2=0x7f7c741cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c74137220 0x7f7c741cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c74135dc0 msgr2=0x7f7c74134df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c74135dc0 0x7f7c74134df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 10 monclient: _finish_hunting 0 2024-09-16T15:16:34.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 monclient: found mon.noname-d 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.816+0000 7f7c727fc640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68009400 con 0x7f7c74136e40 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c727fc640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 msgr2=0x7f7c74135330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c727fc640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 0x7f7c74135330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e960 tx=0x7f7c6800ee20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c6800cc10 con 0x7f7c74136e40 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: dump: 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.833 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6800cd90 con 0x7f7c74136e40 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 5 monclient: authenticate success, global_id 94371 2024-09-16T15:16:34.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 10 monclient: _renew_subs 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c640094d0 con 0x7f7c74136e40 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c68013d50 con 0x7f7c74136e40 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c5b7fe640 10 monclient: dump: 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.835 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 10 monclient: _renew_subs 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c717fa640 4 set_mon_vals no callback set 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.818+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741221f0 con 0x7f7c74136e40 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c796a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7c68013470 con 0x7f7c74136e40 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 0x7f7c741cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74071b20 con 0x7f7c741cb900 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 0x7f7c741cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient: get_auth_request con 0x7f7c741cb900 auth_method 0 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c5b7fe640 1 --2- 172.21.15.134:0/1907649194 >> v2:172.21.15.162:6800/980547263 conn(0x7f7c600388b0 0x7f7c6003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(722..722 src has 1..722) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f7c6804a530 con 0x7f7c74136e40 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_done global_id 6025 payload 824 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c74061ae0 con 0x7f7c741cb900 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c737fe640 1 --2- 172.21.15.134:0/1907649194 >> v2:172.21.15.162:6800/980547263 conn(0x7f7c600388b0 0x7f7c6003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.837 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.819+0000 7f7c737fe640 10 monclient: get_auth_request con 0x7f7c600388b0 auth_method 0 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.821+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 msgr2=0x7f7c741cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.821+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 0x7f7c741cbce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c7406b4b0 tx=0x7f7c640046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c64030d40 con 0x7f7c741cb900 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 10 monclient: dump: 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.838 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7c64003610 con 0x7f7c741cb900 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 msgr2=0x7f7c741cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 0x7f7c741cbce0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7c7406b4b0 tx=0x7f7c640046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 msgr2=0x7f7c741cbce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c5b7fe640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741cb900 0x7f7c741cbce0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:34.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.822+0000 7f7c737fe640 1 --2- 172.21.15.134:0/1907649194 >> v2:172.21.15.162:6800/980547263 conn(0x7f7c600388b0 0x7f7c6003ad70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7c740a2f90 tx=0x7f7c6c003d10 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c796a3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 0x7f7c741be480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c796a3640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7407a680 con 0x7f7c741be070 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 0x7f7c741be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:34.841 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient: get_auth_request con 0x7f7c741be070 auth_method 0 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 10 monclient(hunting): handle_auth_done global_id 6034 payload 824 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.828+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1907649194 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7c7409b490 con 0x7f7c741be070 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.830+0000 7f7c72ffd640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 msgr2=0x7f7c741be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c72ffd640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 0x7f7c741be480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097c0 tx=0x7f7c64008030 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7c64030ea0 con 0x7f7c741be070 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 10 monclient: dump: 2024-09-16T15:16:34.842 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:34.843 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f7c64048240 con 0x7f7c741be070 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 msgr2=0x7f7c741be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 0x7f7c741be480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097c0 tx=0x7f7c64008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 msgr2=0x7f7c741be480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c5b7fe640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f7c741be070 0x7f7c741be480 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097c0 tx=0x7f7c64008030 comp rx=0 tx=0).stop 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:34.844 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.845 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.846 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:34.847 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:34.848 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:34.849 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.850 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.851 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.852 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.853 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.854 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.855 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.856 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.857 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.858 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.859 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.860 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.861 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:34.862 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:34.863 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:34.864 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:34.865 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 >> v2:172.21.15.162:6800/980547263 conn(0x7f7c600388b0 msgr2=0x7f7c6003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1907649194 >> v2:172.21.15.162:6800/980547263 conn(0x7f7c600388b0 0x7f7c6003ad70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7c740a2f90 tx=0x7f7c6c003d10 comp rx=0 tx=0).stop 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 10 monclient: shutdown 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 msgr2=0x7f7c74135330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:34.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 --2- 172.21.15.134:0/1907649194 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7c74136e40 0x7f7c74135330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e960 tx=0x7f7c6800ee20 comp rx=0 tx=0).stop 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c737fe640 1 -- 172.21.15.134:0/1907649194 reap_dead start 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c737fe640 1 -- 172.21.15.134:0/1907649194 reap_dead start 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 shutdown_connections 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 >> 172.21.15.134:0/1907649194 conn(0x7f7c740a6110 msgr2=0x7f7c7406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 shutdown_connections 2024-09-16T15:16:34.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:34.831+0000 7f7c796a3640 1 -- 172.21.15.134:0/1907649194 wait complete. 2024-09-16T15:16:34.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:34.867 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:16:34.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:16:34.954 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2070248273}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2249790189}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1085766595}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3316718031}) 2024-09-16T15:16:34.955 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=312871914}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=418319622}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1922670203}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1817061534}) 2024-09-16T15:16:34.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:34.952+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1794965840}) 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.003+0000 7fc9a4648640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.003+0000 7fc9a4648640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.003+0000 7fc9a4648640 10 monclient: monmap: 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.004472+0000 2024-09-16T15:16:35.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.004472+0000 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 Processor -- start 2024-09-16T15:16:35.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 -- start start 2024-09-16T15:16:35.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient: init 2024-09-16T15:16:35.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient: _add_conns ranks=[9,3,4,0,10,1,8,5,6,7,2] 2024-09-16T15:16:35.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 0x7fc99c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-a con 0x7fc99c136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:35.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc99c137220 0x7fc99c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-g con 0x7fc99c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-c con 0x7fc99c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c09abc0 con 0x7fc99c136e40 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c09ba50 con 0x7fc99c137220 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0a53c0 con 0x7fc99c135dc0 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a4648640 10 monclient(hunting): authenticate will time out at 1731.593878s 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a2be5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.004+0000 7fc9a23e4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 0x7fc99c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47946/0 (socket says 172.21.15.134:47946) 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 -- 172.21.15.134:0/4173669751 learned_addr learned my addr 172.21.15.134:0/4173669751 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): get_auth_request con 0x7fc99c135dc0 auth_method 0 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): get_auth_request con 0x7fc99c136e40 auth_method 0 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc99c137220 0x7fc99c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request con 0x7fc99c137220 auth_method 0 2024-09-16T15:16:35.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_done global_id 6043 payload 274 2024-09-16T15:16:35.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 -- 172.21.15.134:0/4173669751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc99c137220 msgr2=0x7fc99c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc99c137220 0x7fc99c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 -- 172.21.15.134:0/4173669751 >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 msgr2=0x7fc99c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/4173669751 >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 0x7fc99c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 monclient: found mon.noname-c 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a2be5640 1 -- 172.21.15.134:0/4173669751 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c073250 con 0x7fc99c135dc0 2024-09-16T15:16:35.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.005+0000 7fc9a23e4640 1 --2- 172.21.15.134:0/4173669751 >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 0x7fc99c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1844873497}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2746000743}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3296895997}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=186676146}) 2024-09-16T15:16:35.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2239561934}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4268120394}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=310052162}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=938597898}) 2024-09-16T15:16:35.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.006+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3984795709}) 2024-09-16T15:16:35.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.008+0000 7fc9a2be5640 1 -- 172.21.15.134:0/4173669751 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 msgr2=0x7fc99c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.008+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc994009e80 tx=0x7fc99402a580 comp rx=0 tx=0).ready entity=mon.4 client_cookie=a8f1fcf42a66b504 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3417209259}) 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1844697313}) 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.017 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4145353880}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.009+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2957693263}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=241930965}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=312103739}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2419836694}) 2024-09-16T15:16:35.018 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413355243}) 2024-09-16T15:16:35.019 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1541840463}) 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 1 -- 172.21.15.134:0/4173669751 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc994007a70 con 0x7fc99c135dc0 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: dump: 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.024 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 1 -- 172.21.15.134:0/4173669751 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99402d050 con 0x7fc99c135dc0 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a4648640 5 monclient: authenticate success, global_id 6043 2024-09-16T15:16:35.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a4648640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a4648640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 1 -- 172.21.15.134:0/4173669751 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc994007bf0 con 0x7fc99c135dc0 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a4648640 4 set_mon_vals no callback set 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.010+0000 7fc9a13e2640 10 monclient: dump: 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.026 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 10 monclient: shutdown 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 -- 172.21.15.134:0/4173669751 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 msgr2=0x7fc99c136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc994009e80 tx=0x7fc99402a580 comp rx=0 tx=0).stop 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 -- 172.21.15.134:0/4173669751 shutdown_connections 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c136220 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 --2- 172.21.15.134:0/4173669751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc99c137220 0x7fc99c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 --2- 172.21.15.134:0/4173669751 >> v2:172.21.15.134:3300/0 conn(0x7fc99c136e40 0x7fc99c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 -- 172.21.15.134:0/4173669751 >> 172.21.15.134:0/4173669751 conn(0x7fc99c0a6110 msgr2=0x7fc99c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 -- 172.21.15.134:0/4173669751 shutdown_connections 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.011+0000 7fc9a4648640 1 -- 172.21.15.134:0/4173669751 wait complete. 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.012+0000 7fc9a4648640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.012+0000 7fc9a4648640 10 monclient: monmap: 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.028 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.012+0000 7fc9a4648640 1 Processor -- start 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.012+0000 7fc9a4648640 1 -- start start 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.012+0000 7fc9a4648640 10 monclient: init 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient: _add_conns ranks=[4,6,2,1,3,8,0,7,5] 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c1344e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-c con 0x7fc99c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c136e40 0x7fc99c134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-d con 0x7fc99c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:35.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc99c137220 0x7fc99c134f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): picked mon.noname-b con 0x7fc99c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c136880 con 0x7fc99c137220 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c136330 con 0x7fc99c135dc0 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c121e20 con 0x7fc99c136e40 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a4648640 10 monclient(hunting): authenticate will time out at 1731.602158s 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c1344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c1344e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47952/0 (socket says 172.21.15.134:47952) 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c136e40 0x7fc99c134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 -- 172.21.15.134:0/1922376959 learned_addr learned my addr 172.21.15.134:0/1922376959 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): get_auth_request con 0x7fc99c135dc0 auth_method 0 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request con 0x7fc99c136e40 auth_method 0 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc99c137220 0x7fc99c134f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): get_auth_request con 0x7fc99c137220 auth_method 0 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient(hunting): handle_auth_done global_id 6052 payload 824 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c136e40 msgr2=0x7fc99c134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c136e40 0x7fc99c134a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc99c137220 msgr2=0x7fc99c134f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc99c137220 0x7fc99c134f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.032 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 monclient: found mon.noname-c 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a23e4640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998009400 con 0x7fc99c135dc0 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.013+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc99c137220 0x7fc99c134f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.015+0000 7fc9a23e4640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 msgr2=0x7fc99c1344e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.015+0000 7fc9a23e4640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c1344e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e980 tx=0x7fc99800ee40 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9980096e0 con 0x7fc99c135dc0 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: dump: 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.033 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 5 monclient: authenticate success, global_id 6052 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 monclient: _renew_subs 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99800cc10 con 0x7fc99c135dc0 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994009b30 con 0x7fc99c135dc0 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc998013d50 con 0x7fc99c135dc0 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a0be1640 4 set_mon_vals no callback set 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 10 monclient: dump: 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.035 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 monclient: _renew_subs 2024-09-16T15:16:35.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c0a32c0 con 0x7fc99c135dc0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc99800cd80 con 0x7fc99c135dc0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 0x7fc99c061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.016+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c071b20 con 0x7fc99c0610b0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 0x7fc99c061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient: get_auth_request con 0x7fc99c0610b0 auth_method 0 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc992ffd640 1 --2- 172.21.15.134:0/1922376959 >> v2:172.21.15.162:6800/980547263 conn(0x7fc980038880 0x7fc98003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(722..722 src has 1..722) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fc99804a500 con 0x7fc99c135dc0 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_done global_id 94389 payload 824 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a1be3640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc99c061ae0 con 0x7fc99c0610b0 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/1922376959 >> v2:172.21.15.162:6800/980547263 conn(0x7fc980038880 0x7fc98003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.017+0000 7fc9a2be5640 10 monclient: get_auth_request con 0x7fc980038880 auth_method 0 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.019+0000 7fc9a1be3640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 msgr2=0x7fc99c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.019+0000 7fc9a1be3640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 0x7fc99c061490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc98c011860 tx=0x7fc98c011d20 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.020+0000 7fc9a2be5640 1 --2- 172.21.15.134:0/1922376959 >> v2:172.21.15.162:6800/980547263 conn(0x7fc980038880 0x7fc98003ad40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc99c137f00 tx=0x7fc994009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.039 INFO:tasks.rados.rados.0.smithi162.stdout:3401: done (1 left) 2024-09-16T15:16:35.039 INFO:tasks.rados.rados.0.smithi162.stdout:3402: done (0 left) 2024-09-16T15:16:35.039 INFO:tasks.rados.rados.0.smithi162.stdout:3403: snap_create 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=46848398}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1013968240}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1278044704}) 2024-09-16T15:16:35.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2743579532}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2201415545}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1093351108}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2605871548}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612323246}) 2024-09-16T15:16:35.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.023+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2313681336}) 2024-09-16T15:16:35.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc98c00fd00 con 0x7fc99c0610b0 2024-09-16T15:16:35.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 10 monclient: dump: 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.045 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc98c00fe60 con 0x7fc99c0610b0 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 msgr2=0x7fc99c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 0x7fc99c061490 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc98c011860 tx=0x7fc98c011d20 comp rx=0 tx=0).stop 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 msgr2=0x7fc99c061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.025+0000 7fc992ffd640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c0610b0 0x7fc99c061490 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a4648640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a4648640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 0x7fc99c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a4648640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c120de0 con 0x7fc99c02b260 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 0x7fc99c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient: get_auth_request con 0x7fc99c02b260 auth_method 0 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 10 monclient(hunting): handle_auth_done global_id 94398 payload 824 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.032+0000 7fc9a1be3640 1 -- 172.21.15.134:0/1922376959 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc99c07a680 con 0x7fc99c02b260 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.034+0000 7fc9a1be3640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 msgr2=0x7fc99c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.034+0000 7fc9a1be3640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 0x7fc99c02b670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc98c011860 tx=0x7fc98c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc98c02c070 con 0x7fc99c02b260 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 10 monclient: dump: 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7fc98c024040 con 0x7fc99c02b260 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 msgr2=0x7fc99c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 0x7fc99c02b670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc98c011860 tx=0x7fc98c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 msgr2=0x7fc99c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc992ffd640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fc99c02b260 0x7fc99c02b670 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc98c011860 tx=0x7fc98c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.049 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.050 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.051 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.052 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.053 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.055 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.056 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.057 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.058 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.059 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.060 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.061 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.062 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.063 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.064 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.065 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.067 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.068 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.069 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.070 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 >> v2:172.21.15.162:6800/980547263 conn(0x7fc980038880 msgr2=0x7fc98003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 --2- 172.21.15.134:0/1922376959 >> v2:172.21.15.162:6800/980547263 conn(0x7fc980038880 0x7fc98003ad40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc99c137f00 tx=0x7fc994009000 comp rx=0 tx=0).stop 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 10 monclient: shutdown 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 msgr2=0x7fc99c1344e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 --2- 172.21.15.134:0/1922376959 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc99c135dc0 0x7fc99c1344e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e980 tx=0x7fc99800ee40 comp rx=0 tx=0).stop 2024-09-16T15:16:35.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a2be5640 1 -- 172.21.15.134:0/1922376959 reap_dead start 2024-09-16T15:16:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a2be5640 1 -- 172.21.15.134:0/1922376959 reap_dead start 2024-09-16T15:16:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 shutdown_connections 2024-09-16T15:16:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 >> 172.21.15.134:0/1922376959 conn(0x7fc99c0a6110 msgr2=0x7fc99c09f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 shutdown_connections 2024-09-16T15:16:35.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.035+0000 7fc9a4648640 1 -- 172.21.15.134:0/1922376959 wait complete. 2024-09-16T15:16:35.072 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:16:35.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.170+0000 7f6cc2333640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.170+0000 7f6cc2333640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.170+0000 7f6cc2333640 10 monclient: monmap: 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.171033+0000 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.171033+0000 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.171 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.172 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.170+0000 7f6cc2333640 1 Processor -- start 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 -- start start 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient: init 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient: _add_conns ranks=[9,8,5,4,2,0,6,3,1,10,7] 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cbc133f10 0x7f6cbc1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-a con 0x7f6cbc133f10 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 0x7f6cbc134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-e con 0x7f6cbc134830 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 0x7f6cbc1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.175 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-h con 0x7f6cbc071bd0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0a3d50 con 0x7f6cbc133f10 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc06f630 con 0x7f6cbc071bd0 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0a1e10 con 0x7f6cbc134830 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc2333640 10 monclient(hunting): authenticate will time out at 1731.760581s 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cbc133f10 0x7f6cbc1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 0x7f6cbc134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cbc133f10 0x7f6cbc1342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46142/0 (socket says 172.21.15.134:46142) 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2767440607 learned_addr learned my addr 172.21.15.134:0/2767440607 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): get_auth_request con 0x7f6cbc133f10 auth_method 0 2024-09-16T15:16:35.176 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request con 0x7f6cbc134830 auth_method 0 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_done global_id 114228 payload 274 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 0x7f6cbc1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 msgr2=0x7f6cbc134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 0x7f6cbc134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2767440607 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 msgr2=0x7f6cbc1351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 0x7f6cbc1351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 monclient: found mon.noname-a 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2767440607 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc073250 con 0x7f6cbc133f10 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc08d0640 10 monclient: get_auth_request con 0x7f6cbc071bd0 auth_method 0 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 0x7f6cbc134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.171+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 0x7f6cbc1351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc133f10 msgr2=0x7f6cbc1342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc133f10 0x7f6cbc1342f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847aa6a88ebeb9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 1 -- 172.21.15.134:0/2767440607 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cac02ae00 con 0x7f6cbc133f10 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: dump: 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.185 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cc2333640 5 monclient: authenticate success, global_id 114228 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cc2333640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 1 -- 172.21.15.134:0/2767440607 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac0307b0 con 0x7f6cbc133f10 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cc2333640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cc2333640 4 set_mon_vals no callback set 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 1 -- 172.21.15.134:0/2767440607 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cac030dd0 con 0x7f6cbc133f10 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.174+0000 7f6cbb7fe640 10 monclient: dump: 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.187 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 10 monclient: shutdown 2024-09-16T15:16:35.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 -- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc133f10 msgr2=0x7f6cbc1342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc133f10 0x7f6cbc1342f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009870 tx=0x7f6cac02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 -- 172.21.15.134:0/2767440607 shutdown_connections 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6cbc071bd0 0x7f6cbc1351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc134830 0x7f6cbc134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2767440607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc133f10 0x7f6cbc1342f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 -- 172.21.15.134:0/2767440607 >> 172.21.15.134:0/2767440607 conn(0x7f6cbc0a7410 msgr2=0x7f6cbc0a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 -- 172.21.15.134:0/2767440607 shutdown_connections 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.175+0000 7f6cc2333640 1 -- 172.21.15.134:0/2767440607 wait complete. 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 10 monclient: monmap: 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.189 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 1 Processor -- start 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 1 -- start start 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 10 monclient: init 2024-09-16T15:16:35.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 10 monclient: _add_conns ranks=[2,3,1,7,0,5,6,4,8] 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.176+0000 7f6cc2333640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 0x7f6cbc09d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-b con 0x7f6cbc071bd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cbc133f10 0x7f6cbc121e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-g con 0x7f6cbc133f10 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cbc134830 0x7f6cbc1223b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): picked mon.noname-f con 0x7f6cbc134830 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc123ba0 con 0x7f6cbc134830 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc123980 con 0x7f6cbc071bd0 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0a6dd0 con 0x7f6cbc133f10 2024-09-16T15:16:35.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc2333640 10 monclient(hunting): authenticate will time out at 1731.766071s 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 0x7f6cbc09d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 0x7f6cbc09d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:33988/0 (socket says 172.21.15.134:33988) 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2616958091 learned_addr learned my addr 172.21.15.134:0/2616958091 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): get_auth_request con 0x7f6cbc071bd0 auth_method 0 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cbc133f10 0x7f6cbc121e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cbc134830 0x7f6cbc1223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request con 0x7f6cbc133f10 auth_method 0 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient(hunting): handle_auth_done global_id 114284 payload 824 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cbc133f10 msgr2=0x7f6cbc121e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cbc133f10 0x7f6cbc121e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cbc134830 msgr2=0x7f6cbc1223b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cbc134830 0x7f6cbc1223b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 monclient: found mon.noname-b 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009400 con 0x7f6cbc071bd0 2024-09-16T15:16:35.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc08d0640 10 monclient: get_auth_request con 0x7f6cbc134830 auth_method 0 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.177+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cbc134830 0x7f6cbc1223b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cbbfff640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 msgr2=0x7f6cbc09d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cbbfff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 0x7f6cbc09d8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0090b0 tx=0x7f6cac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cac030d40 con 0x7f6cbc071bd0 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: dump: 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.194 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac003a80 con 0x7f6cbc071bd0 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 5 monclient: authenticate success, global_id 114284 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 10 monclient: _renew_subs 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac0094d0 con 0x7f6cbc071bd0 2024-09-16T15:16:35.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6cac003bf0 con 0x7f6cbc071bd0 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cb97fa640 10 monclient: dump: 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.196 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cbaffd640 4 set_mon_vals no callback set 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 10 monclient: _renew_subs 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.179+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc124ea0 con 0x7f6cbc071bd0 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cc2333640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6cac039070 con 0x7f6cbc071bd0 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 0x7f6cbc06b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc073880 con 0x7f6cbc1228f0 2024-09-16T15:16:35.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 0x7f6cbc06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient: get_auth_request con 0x7f6cbc1228f0 auth_method 0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.180+0000 7f6cb97fa640 1 --2- 172.21.15.134:0/2616958091 >> v2:172.21.15.162:6800/980547263 conn(0x7f6c9c038880 0x7f6c9c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.181+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f6cac06ab20 con 0x7f6cbc071bd0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.181+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_done global_id 84671 payload 824 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.181+0000 7f6cb3fff640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cbc0a5410 con 0x7f6cbc1228f0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.181+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2616958091 >> v2:172.21.15.162:6800/980547263 conn(0x7f6c9c038880 0x7f6c9c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.181+0000 7f6cc08d0640 10 monclient: get_auth_request con 0x7f6c9c038880 auth_method 0 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb3fff640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 msgr2=0x7f6cbc06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.198 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 0x7f6cbc06b380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca4011a60 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ca400fe20 con 0x7f6cbc1228f0 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 10 monclient: dump: 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.199 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6ca400f9b0 con 0x7f6cbc1228f0 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 msgr2=0x7f6cbc06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 0x7f6cbc06b380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca4011a60 comp rx=0 tx=0).stop 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 msgr2=0x7f6cbc06b380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cb97fa640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc1228f0 0x7f6cbc06b380 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca4011a60 comp rx=0 tx=0).stop 2024-09-16T15:16:35.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.183+0000 7f6cc08d0640 1 --2- 172.21.15.134:0/2616958091 >> v2:172.21.15.162:6800/980547263 conn(0x7f6c9c038880 0x7f6c9c03ad40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc132760 tx=0x7f6cb4007490 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cc2333640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 0x7f6cbc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cc2333640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc06a3f0 con 0x7f6cbc06b8c0 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 0x7f6cbc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cb3fff640 10 monclient: get_auth_request con 0x7f6cbc06b8c0 auth_method 0 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cb3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cb3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.189+0000 7f6cb3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.190+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.190+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.190+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.190+0000 7f6cb3fff640 10 monclient(hunting): handle_auth_done global_id 84680 payload 824 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.190+0000 7f6cb3fff640 1 -- 172.21.15.134:0/2616958091 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6cbc061ae0 con 0x7f6cbc06b8c0 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb3fff640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 msgr2=0x7f6cbc06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb3fff640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 0x7f6cbc06bcf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca40046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ca40066c0 con 0x7f6cbc06b8c0 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 10 monclient: dump: 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f6ca4026070 con 0x7f6cbc06b8c0 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 msgr2=0x7f6cbc06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 0x7f6cbc06bcf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca40046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 msgr2=0x7f6cbc06bcf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.192+0000 7f6cb97fa640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6cbc06b8c0 0x7f6cbc06bcf0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06ceb0 tx=0x7f6ca40046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.205 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.206 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.207 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.208 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.209 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.210 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.211 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.212 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.213 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.214 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.215 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.216 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.217 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.218 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.219 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.221 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.222 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.223 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.224 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.225 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 >> v2:172.21.15.162:6800/980547263 conn(0x7f6c9c038880 msgr2=0x7f6c9c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2616958091 >> v2:172.21.15.162:6800/980547263 conn(0x7f6c9c038880 0x7f6c9c03ad40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc132760 tx=0x7f6cb4007490 comp rx=0 tx=0).stop 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 10 monclient: shutdown 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 msgr2=0x7f6cbc09d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 --2- 172.21.15.134:0/2616958091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc071bd0 0x7f6cbc09d8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0090b0 tx=0x7f6cac0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc08d0640 1 -- 172.21.15.134:0/2616958091 reap_dead start 2024-09-16T15:16:35.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc08d0640 1 -- 172.21.15.134:0/2616958091 reap_dead start 2024-09-16T15:16:35.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 shutdown_connections 2024-09-16T15:16:35.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 >> 172.21.15.134:0/2616958091 conn(0x7f6cbc0a7410 msgr2=0x7f6cbc1333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 shutdown_connections 2024-09-16T15:16:35.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.193+0000 7f6cc2333640 1 -- 172.21.15.134:0/2616958091 wait complete. 2024-09-16T15:16:35.227 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.227 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:16:35.227 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:16:35.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4239352717}) 2024-09-16T15:16:35.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.279 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=876439304}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3246339179}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2399823032}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2056802713}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.280 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929649892}) 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1354654732}) 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2281946626}) 2024-09-16T15:16:35.281 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.277+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=88337493}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=466314560}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3723636421}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2437114537}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3596895182}) 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.283 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2054681536}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3139587323}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3006186274}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4083564865}) 2024-09-16T15:16:35.284 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.279+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1627741018}) 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2970867694}) 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=630802788}) 2024-09-16T15:16:35.319 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2104779375}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1980262642}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1453721378}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=191372399}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.320 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=926540779}) 2024-09-16T15:16:35.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.316+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3261637570}) 2024-09-16T15:16:35.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.317+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1025077784}) 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.326+0000 7fe576357640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.326+0000 7fe576357640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.326+0000 7fe576357640 10 monclient: monmap: 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.327210+0000 2024-09-16T15:16:35.327 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.327210+0000 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.328 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.326+0000 7fe576357640 1 Processor -- start 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 -- start start 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient: init 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient: _add_conns ranks=[7,5,0,6,3,10,1,2,4,9,8] 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 0x7fe570065ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): picked mon.noname-i con 0x7fe570069e50 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 0x7fe570066910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): picked mon.noname-h con 0x7fe570066530 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe57013b5a0 addr v1:172.21.15.134:6789/0 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe570073250 con 0x7fe57013b5a0 2024-09-16T15:16:35.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe57009e4a0 con 0x7fe570066530 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe570066e50 con 0x7fe570069e50 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe576357640 10 monclient(hunting): authenticate will time out at 1731.916497s 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe5748f4640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe57013b5a0 0x7fe57013d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45642/0 (socket says 172.21.15.134:45642) 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe5748f4640 1 -- 172.21.15.134:0/1955976338 learned_addr learned my addr 172.21.15.134:0/1955976338 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 0x7fe570066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): get_auth_request con 0x7fe570066530 auth_method 0 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 0x7fe570065ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): get_auth_request con 0x7fe570069e50 auth_method 0 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.327+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_done global_id 45149 payload 274 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 1 -- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 msgr2=0x7fe570065ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 0x7fe570065ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 1 -- 172.21.15.134:0/1955976338 >> v1:172.21.15.134:6789/0 conn(0x7fe57013b5a0 legacy=0x7fe57013d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 1 monclient: found mon.noname-h 2024-09-16T15:16:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:16:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56f7fe640 1 -- 172.21.15.134:0/1955976338 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57006a230 con 0x7fe570066530 2024-09-16T15:16:35.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.328+0000 7fe56ffff640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 0x7fe570065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.330+0000 7fe56f7fe640 1 -- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 msgr2=0x7fe570066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.330+0000 7fe56f7fe640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 0x7fe570066910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097d0 tx=0x7fe56402a110 comp rx=0 tx=0).ready entity=mon.5 client_cookie=a6a2fefe139c678a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 1 -- 172.21.15.134:0/1955976338 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe56402ae30 con 0x7fe570066530 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: dump: 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.340 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 1 -- 172.21.15.134:0/1955976338 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640307b0 con 0x7fe570066530 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 1 -- 172.21.15.134:0/1955976338 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe564030d90 con 0x7fe570066530 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 5 monclient: authenticate success, global_id 45149 2024-09-16T15:16:35.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 4 set_mon_vals no callback set 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe56effd640 10 monclient: dump: 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.342 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 10 monclient: shutdown 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 -- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 msgr2=0x7fe570066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 0x7fe570066910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097d0 tx=0x7fe56402a110 comp rx=0 tx=0).stop 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 -- 172.21.15.134:0/1955976338 shutdown_connections 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe570066530 0x7fe570066910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 --2- 172.21.15.134:0/1955976338 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570069e50 0x7fe570065ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 -- 172.21.15.134:0/1955976338 >> 172.21.15.134:0/1955976338 conn(0x7fe5700a6400 msgr2=0x7fe5700a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 -- 172.21.15.134:0/1955976338 shutdown_connections 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.331+0000 7fe576357640 1 -- 172.21.15.134:0/1955976338 wait complete. 2024-09-16T15:16:35.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.332+0000 7fe576357640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.332+0000 7fe576357640 10 monclient: monmap: 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.344 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.332+0000 7fe576357640 1 Processor -- start 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.332+0000 7fe576357640 1 -- start start 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.332+0000 7fe576357640 10 monclient: init 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient: _add_conns ranks=[7,4,0,5,1,3,6,2,8] 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570066530 0x7fe570138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): picked mon.noname-i con 0x7fe570066530 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe570069e50 0x7fe5700903b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): picked mon.noname-c con 0x7fe570069e50 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 0x7fe5700908f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): picked mon.noname-a con 0x7fe57013b5a0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700997e0 con 0x7fe57013b5a0 2024-09-16T15:16:35.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe570091a90 con 0x7fe570069e50 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700a0f00 con 0x7fe570066530 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe576357640 10 monclient(hunting): authenticate will time out at 1731.922271s 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 0x7fe5700908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe570069e50 0x7fe5700903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 0x7fe5700908f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46156/0 (socket says 172.21.15.134:46156) 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 learned_addr learned my addr 172.21.15.134:0/3273405012 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): get_auth_request con 0x7fe57013b5a0 auth_method 0 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): get_auth_request con 0x7fe570069e50 auth_method 0 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56ffff640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570066530 0x7fe570138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56ffff640 10 monclient(hunting): get_auth_request con 0x7fe570066530 auth_method 0 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe56ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient(hunting): handle_auth_done global_id 114237 payload 824 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570066530 msgr2=0x7fe570138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe570066530 0x7fe570138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe570069e50 msgr2=0x7fe5700903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe570069e50 0x7fe5700903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 monclient: found mon.noname-a 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.333+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009400 con 0x7fe57013b5a0 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.335+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 msgr2=0x7fe5700908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.335+0000 7fe5748f4640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 0x7fe5700908f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe56800caf0 tx=0x7fe56800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.335+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe56800acb0 con 0x7fe57013b5a0 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: dump: 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.349 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 5 monclient: authenticate success, global_id 114237 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56800ae50 con 0x7fe57013b5a0 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 monclient: _renew_subs 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe564009400 con 0x7fe57013b5a0 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe568012ce0 con 0x7fe57013b5a0 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 10 monclient: dump: 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.351 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 monclient: _renew_subs 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe570093b80 con 0x7fe57013b5a0 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56e7fc640 4 set_mon_vals no callback set 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe568012400 con 0x7fe57013b5a0 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 0x7fe570090e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.336+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57006b5d0 con 0x7fe5700935c0 2024-09-16T15:16:35.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56cff9640 1 --2- 172.21.15.134:0/3273405012 >> v2:172.21.15.162:6800/980547263 conn(0x7fe55c038880 0x7fe55c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 0x7fe570090e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7fe56804a530 con 0x7fe57013b5a0 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient: get_auth_request con 0x7fe5700935c0 auth_method 0 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56f7fe640 1 --2- 172.21.15.134:0/3273405012 >> v2:172.21.15.162:6800/980547263 conn(0x7fe55c038880 0x7fe55c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56f7fe640 10 monclient: get_auth_request con 0x7fe55c038880 auth_method 0 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 10 monclient(hunting): handle_auth_done global_id 45172 payload 824 2024-09-16T15:16:35.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.337+0000 7fe56ffff640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe570061ae0 con 0x7fe5700935c0 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.339+0000 7fe56f7fe640 1 --2- 172.21.15.134:0/3273405012 >> v2:172.21.15.162:6800/980547263 conn(0x7fe55c038880 0x7fe55c03ad40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe564034000 tx=0x7fe56402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.339+0000 7fe56ffff640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 msgr2=0x7fe570090e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.339+0000 7fe56ffff640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 0x7fe570090e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe560011830 tx=0x7fe560011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe5600044a0 con 0x7fe5700935c0 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 10 monclient: dump: 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.356 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe56000fc50 con 0x7fe5700935c0 2024-09-16T15:16:35.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 msgr2=0x7fe570090e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 0x7fe570090e30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe560011830 tx=0x7fe560011cf0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 msgr2=0x7fe570090e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.340+0000 7fe56cff9640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700935c0 0x7fe570090e30 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe560011830 tx=0x7fe560011cf0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe576357640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe576357640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 0x7fe570091370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe576357640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57007a680 con 0x7fe570139290 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 0x7fe570091370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient: get_auth_request con 0x7fe570139290 auth_method 0 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.346+0000 7fe56ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.347+0000 7fe56ffff640 10 monclient(hunting): handle_auth_done global_id 45181 payload 824 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.347+0000 7fe56ffff640 1 -- 172.21.15.134:0/3273405012 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe57009e1c0 con 0x7fe570139290 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56ffff640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 msgr2=0x7fe570091370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56ffff640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 0x7fe570091370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe560008010 tx=0x7fe5600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe560024070 con 0x7fe570139290 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56cff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56cff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.349+0000 7fe56cff9640 10 monclient: dump: 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7fe560006250 con 0x7fe570139290 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 msgr2=0x7fe570091370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 0x7fe570091370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe560008010 tx=0x7fe5600046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 msgr2=0x7fe570091370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe56cff9640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570139290 0x7fe570091370 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.361 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.362 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.363 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.364 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.365 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.366 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.367 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.368 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.369 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.370 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.371 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.373 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.374 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.375 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.376 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.377 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.378 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.379 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.380 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:35.381 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.382 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 >> v2:172.21.15.162:6800/980547263 conn(0x7fe55c038880 msgr2=0x7fe55c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 --2- 172.21.15.134:0/3273405012 >> v2:172.21.15.162:6800/980547263 conn(0x7fe55c038880 0x7fe55c03ad40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe564034000 tx=0x7fe56402e000 comp rx=0 tx=0).stop 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 10 monclient: shutdown 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 msgr2=0x7fe5700908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 --2- 172.21.15.134:0/3273405012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe57013b5a0 0x7fe5700908f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe56800caf0 tx=0x7fe56800a060 comp rx=0 tx=0).stop 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 reap_dead start 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe5748f4640 1 -- 172.21.15.134:0/3273405012 reap_dead start 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 shutdown_connections 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 >> 172.21.15.134:0/3273405012 conn(0x7fe5700a6400 msgr2=0x7fe5701397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 shutdown_connections 2024-09-16T15:16:35.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.350+0000 7fe576357640 1 -- 172.21.15.134:0/3273405012 wait complete. 2024-09-16T15:16:35.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:16:35.385 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.480+0000 7fb1e7508640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.480+0000 7fb1e7508640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.480+0000 7fb1e7508640 10 monclient: monmap: 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.481682+0000 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.481682+0000 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.482 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 Processor -- start 2024-09-16T15:16:35.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 -- start start 2024-09-16T15:16:35.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient: init 2024-09-16T15:16:35.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient: _add_conns ranks=[9,3,10,0,6,5,4,7,1,8,2] 2024-09-16T15:16:35.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1e0136e40 0x7fb1e009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-a con 0x7fb1e0136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0137220 0x7fb1e0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-g con 0x7fb1e0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb1e0135dc0 0x7fb1e0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-f con 0x7fb1e0135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e009abc0 con 0x7fb1e0136e40 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e009ba50 con 0x7fb1e0135dc0 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00a53c0 con 0x7fb1e0137220 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.481+0000 7fb1e7508640 10 monclient(hunting): authenticate will time out at 1732.070960s 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1e0136e40 0x7fb1e009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1e0136e40 0x7fb1e009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46172/0 (socket says 172.21.15.134:46172) 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 -- 172.21.15.134:0/56318727 learned_addr learned my addr 172.21.15.134:0/56318727 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): get_auth_request con 0x7fb1e0136e40 auth_method 0 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0137220 0x7fb1e0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 1 --2- 172.21.15.134:0/56318727 >> v2:172.21.15.162:3300/0 conn(0x7fb1e0135dc0 0x7fb1e0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request con 0x7fb1e0137220 auth_method 0 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): get_auth_request con 0x7fb1e0135dc0 auth_method 0 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_done global_id 114246 payload 274 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 -- 172.21.15.134:0/56318727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0137220 msgr2=0x7fb1e0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0137220 0x7fb1e0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 -- 172.21.15.134:0/56318727 >> v2:172.21.15.162:3300/0 conn(0x7fb1e0135dc0 msgr2=0x7fb1e0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/56318727 >> v2:172.21.15.162:3300/0 conn(0x7fb1e0135dc0 0x7fb1e0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 monclient: found mon.noname-a 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.482+0000 7fb1e52a4640 1 -- 172.21.15.134:0/56318727 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0073250 con 0x7fb1e0136e40 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e52a4640 1 -- 172.21.15.134:0/56318727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e0136e40 msgr2=0x7fb1e009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e0136e40 0x7fb1e009c150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb1e006b4b0 tx=0x7fb1dc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1375bb3eded9b2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 1 -- 172.21.15.134:0/56318727 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1dc02ae10 con 0x7fb1e0136e40 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: dump: 2024-09-16T15:16:35.491 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.492 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 5 monclient: authenticate success, global_id 114246 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 1 -- 172.21.15.134:0/56318727 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0307b0 con 0x7fb1e0136e40 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 4 set_mon_vals no callback set 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 1 -- 172.21.15.134:0/56318727 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1dc030d90 con 0x7fb1e0136e40 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1d7fff640 10 monclient: dump: 2024-09-16T15:16:35.493 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.494 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 10 monclient: shutdown 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 -- 172.21.15.134:0/56318727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e0136e40 msgr2=0x7fb1e009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e0136e40 0x7fb1e009c150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb1e006b4b0 tx=0x7fb1dc02a0b0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 -- 172.21.15.134:0/56318727 shutdown_connections 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 --2- 172.21.15.134:0/56318727 >> v2:172.21.15.162:3300/0 conn(0x7fb1e0135dc0 0x7fb1e0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0137220 0x7fb1e0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 --2- 172.21.15.134:0/56318727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e0136e40 0x7fb1e009c150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.485+0000 7fb1e7508640 1 -- 172.21.15.134:0/56318727 >> 172.21.15.134:0/56318727 conn(0x7fb1e00a6110 msgr2=0x7fb1e00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 1 -- 172.21.15.134:0/56318727 shutdown_connections 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 1 -- 172.21.15.134:0/56318727 wait complete. 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 10 monclient: monmap: 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.495 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.496 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 1 Processor -- start 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 1 -- start start 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.486+0000 7fb1e7508640 10 monclient: init 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient: _add_conns ranks=[2,8,5,0,7,6,1,3,4] 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-b con 0x7fb1e0135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 0x7fb1e0135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-e con 0x7fb1e0136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb1e0137220 0x7fb1e0135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): picked mon.noname-h con 0x7fb1e0137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e01cb7f0 con 0x7fb1e0135dc0 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00a8a30 con 0x7fb1e0137220 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0136380 con 0x7fb1e0136e40 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e7508640 10 monclient(hunting): authenticate will time out at 1732.076454s 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 0x7fb1e0135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:34002/0 (socket says 172.21.15.134:34002) 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 0x7fb1e0135010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41820/0 (socket says 172.21.15.134:41820) 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 -- 172.21.15.134:0/3872890154 learned_addr learned my addr 172.21.15.134:0/3872890154 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): get_auth_request con 0x7fb1e0135dc0 auth_method 0 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request con 0x7fb1e0136e40 auth_method 0 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb1e0137220 0x7fb1e0135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): get_auth_request con 0x7fb1e0137220 auth_method 0 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e5aa5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient(hunting): handle_auth_done global_id 114293 payload 824 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 msgr2=0x7fb1e0135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 0x7fb1e0135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb1e0137220 msgr2=0x7fb1e0135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb1e0137220 0x7fb1e0135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 monclient: found mon.noname-b 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e52a4640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0094d0 con 0x7fb1e0135dc0 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.487+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb1e0136e40 0x7fb1e0135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e52a4640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 msgr2=0x7fb1e01cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e52a4640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc000c00 tx=0x7fb1dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1dc003610 con 0x7fb1e0135dc0 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: dump: 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.505 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 5 monclient: authenticate success, global_id 114293 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0037b0 con 0x7fb1e0135dc0 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 10 monclient: _renew_subs 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e0123630 con 0x7fb1e0135dc0 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1dc003d90 con 0x7fb1e0135dc0 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d5ffb640 10 monclient: dump: 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.507 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 10 monclient: _renew_subs 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1d77fe640 4 set_mon_vals no callback set 2024-09-16T15:16:35.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.490+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e0125720 con 0x7fb1e0135dc0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb1dc039070 con 0x7fb1e0135dc0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e7508640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e7508640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 0x7fb1e0121e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e0071b20 con 0x7fb1e0121a50 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1d5ffb640 1 --2- 172.21.15.134:0/3872890154 >> v2:172.21.15.162:6800/980547263 conn(0x7fb1c40388b0 0x7fb1c403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7fb1dc06a9b0 con 0x7fb1e0135dc0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 0x7fb1e0121e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient: get_auth_request con 0x7fb1e0121a50 auth_method 0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e5aa5640 1 --2- 172.21.15.134:0/3872890154 >> v2:172.21.15.162:6800/980547263 conn(0x7fb1c40388b0 0x7fb1c403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e5aa5640 10 monclient: get_auth_request con 0x7fb1c40388b0 auth_method 0 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_done global_id 84558 payload 824 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.491+0000 7fb1e4aa3640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1e0061ae0 con 0x7fb1e0121a50 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1e4aa3640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 msgr2=0x7fb1e0121e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 0x7fb1e0121e30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0011bd0 tx=0x7fb1d000f610 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1e5aa5640 1 --2- 172.21.15.134:0/3872890154 >> v2:172.21.15.162:6800/980547263 conn(0x7fb1c40388b0 0x7fb1c403ad70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1e01231b0 tx=0x7fb1d800b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1d0004240 con 0x7fb1e0121a50 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 10 monclient: dump: 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.510 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb1d00043a0 con 0x7fb1e0121a50 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 msgr2=0x7fb1e0121e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 0x7fb1e0121e30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0011bd0 tx=0x7fb1d000f610 comp rx=0 tx=0).stop 2024-09-16T15:16:35.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 msgr2=0x7fb1e0121e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.494+0000 7fb1d5ffb640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0121a50 0x7fb1e0121e30 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0011bd0 tx=0x7fb1d000f610 comp rx=0 tx=0).stop 2024-09-16T15:16:35.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.500+0000 7fb1e7508640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.500+0000 7fb1e7508640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 0x7fb1e00a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.500+0000 7fb1e7508640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.500+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e007a680 con 0x7fb1e0122370 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 0x7fb1e00a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient: get_auth_request con 0x7fb1e0122370 auth_method 0 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 10 monclient(hunting): handle_auth_done global_id 84567 payload 824 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.501+0000 7fb1e4aa3640 1 -- 172.21.15.134:0/3872890154 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb1e009b490 con 0x7fb1e0122370 2024-09-16T15:16:35.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.503+0000 7fb1e4aa3640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 msgr2=0x7fb1e00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.503+0000 7fb1e4aa3640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 0x7fb1e00a1a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001b000 tx=0x7fb1d00046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb1d001ece0 con 0x7fb1e0122370 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 10 monclient: dump: 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.515 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fb1d001e870 con 0x7fb1e0122370 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 msgr2=0x7fb1e00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 0x7fb1e00a1a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001b000 tx=0x7fb1d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 msgr2=0x7fb1e00a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1d5ffb640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb1e0122370 0x7fb1e00a1a20 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001b000 tx=0x7fb1d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.517 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.518 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.518 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.518 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:16:35.596 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.596 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.596 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.596 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.598 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.599 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.600 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.601 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.602 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.603 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.604 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.605 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.606 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.607 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.608 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.609 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.610 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.611 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.612 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.613 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.614 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.615 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.616 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.617 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1474731075}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2067340776}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=460325712}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=85646459}) 2024-09-16T15:16:35.618 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 >> v2:172.21.15.162:6800/980547263 conn(0x7fb1c40388b0 msgr2=0x7fb1c403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 --2- 172.21.15.134:0/3872890154 >> v2:172.21.15.162:6800/980547263 conn(0x7fb1c40388b0 0x7fb1c403ad70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1e01231b0 tx=0x7fb1d800b040 comp rx=0 tx=0).stop 2024-09-16T15:16:35.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 10 monclient: shutdown 2024-09-16T15:16:35.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 msgr2=0x7fb1e01cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.619 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2461157673}) 2024-09-16T15:16:35.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc000c00 tx=0x7fb1dc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e5aa5640 1 -- 172.21.15.134:0/3872890154 reap_dead start 2024-09-16T15:16:35.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2215192280}) 2024-09-16T15:16:35.620 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 shutdown_connections 2024-09-16T15:16:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 --2- 172.21.15.134:0/3872890154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e0135dc0 0x7fb1e01cb2b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 >> 172.21.15.134:0/3872890154 conn(0x7fb1e00a6110 msgr2=0x7fb1e006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.621 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4061434191}) 2024-09-16T15:16:35.622 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.622 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3993446482}) 2024-09-16T15:16:35.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 shutdown_connections 2024-09-16T15:16:35.622 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1457522897}) 2024-09-16T15:16:35.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.504+0000 7fb1e7508640 1 -- 172.21.15.134:0/3872890154 wait complete. 2024-09-16T15:16:35.623 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.623 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:16:35.624 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:16:35.641 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=163749917}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1026513391}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=592582364}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=234053475}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.642 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1152935132}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3642179096}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1793125484}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3752069020}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.529+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367600180}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2870919895}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.643 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3989294233}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3083409602}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4266009078}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=975760861}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1208411538}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.644 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1053669489}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=328102216}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.531+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3680201936}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1306630771}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2490553591}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1575701183}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.645 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1754852565}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2732451672}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=483769920}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.532+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3624876349}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.533+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.533+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3681447001}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.533+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=83394123}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.646 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3389698739}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=503035988}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4046372635}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2191918177}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1475161256}) 2024-09-16T15:16:35.647 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=62307286}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1771662902}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121205876}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.534+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1449913276}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=962077509}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1069842985}) 2024-09-16T15:16:35.648 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1114510432}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1963512290}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.535+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3981578622}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=171038098}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1614491729}) 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.649 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327583582}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.536+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2487118047}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=980790897}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825626129}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3221274677}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3050096184}) 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.650 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=655040343}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=413657493}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=300331246}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=395843458}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.537+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2479259480}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1230866182}) 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.651 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4142713829}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1831978038}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=432695896}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2114858408}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=228095659}) 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.652 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3282134253}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=333830653}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.539+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=509620983}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=249652251}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2111942227}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=678387874}) 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.653 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2241405238}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2424992308}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2231121244}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3256717381}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3213659028}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.540+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=66688302}) 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3603278599}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2158873026}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2787494012}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3473076121}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=740958147}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1153170041}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3608501671}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3592944927}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.545+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1254421191}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=293619483}) 2024-09-16T15:16:35.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4154805321}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1637777449}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=8825667}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2685327832}) 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3404421100}) 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=199829262}) 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2982168532}) 2024-09-16T15:16:35.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:16:35.548+0000 7fb6ce00e640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2351977606}) 2024-09-16T15:16:35.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.754+0000 7f11d7896640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.754+0000 7f11d7896640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.754+0000 7f11d7896640 10 monclient: monmap: 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.755496+0000 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.755496+0000 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.756 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.757 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 Processor -- start 2024-09-16T15:16:35.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 -- start start 2024-09-16T15:16:35.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient: init 2024-09-16T15:16:35.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient: _add_conns ranks=[9,6,3,5,0,4,7,2,1,10,8] 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f11d0136e40 0x7f11d009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-a con 0x7f11d0136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-d con 0x7f11d0137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-g con 0x7f11d0135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d009abc0 con 0x7f11d0136e40 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d009ba50 con 0x7f11d0135dc0 2024-09-16T15:16:35.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00a53c0 con 0x7f11d0137220 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d7896640 10 monclient(hunting): authenticate will time out at 1732.344797s 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f11d0136e40 0x7f11d009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f11d0136e40 0x7f11d009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46174/0 (socket says 172.21.15.134:46174) 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34072/0 (socket says 172.21.15.134:34072) 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 1 -- 172.21.15.134:0/1162834041 learned_addr learned my addr 172.21.15.134:0/1162834041 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 10 monclient(hunting): get_auth_request con 0x7f11d0136e40 auth_method 0 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d5632640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 10 monclient(hunting): get_auth_request con 0x7f11d0137220 auth_method 0 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.755+0000 7f11d4e31640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5e33640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5e33640 10 monclient(hunting): get_auth_request con 0x7f11d0135dc0 auth_method 0 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5e33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5e33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5e33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient(hunting): handle_auth_done global_id 114255 payload 274 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 -- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 msgr2=0x7f11d0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 -- 172.21.15.134:0/1162834041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 msgr2=0x7f11d0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 monclient: found mon.noname-a 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d5632640 1 -- 172.21.15.134:0/1162834041 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d0073250 con 0x7f11d0136e40 2024-09-16T15:16:35.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.756+0000 7f11d4e31640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.758+0000 7f11d5632640 1 -- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11d0136e40 msgr2=0x7f11d009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.758+0000 7f11d5632640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11d0136e40 0x7f11d009c150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009870 tx=0x7f11cc02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926321ef8b07bf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 1 -- 172.21.15.134:0/1162834041 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11cc02ae00 con 0x7f11d0136e40 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: dump: 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.770 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 1 -- 172.21.15.134:0/1162834041 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc0307b0 con 0x7f11d0136e40 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 5 monclient: authenticate success, global_id 114255 2024-09-16T15:16:35.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 1 -- 172.21.15.134:0/1162834041 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11cc030d90 con 0x7f11d0136e40 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 4 set_mon_vals no callback set 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11c7fff640 10 monclient: dump: 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.772 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 10 monclient: shutdown 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 -- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11d0136e40 msgr2=0x7f11d009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11d0136e40 0x7f11d009c150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009870 tx=0x7f11cc02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 -- 172.21.15.134:0/1162834041 shutdown_connections 2024-09-16T15:16:35.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f11d0137220 0x7f11d0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 --2- 172.21.15.134:0/1162834041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11d0136e40 0x7f11d009c150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 -- 172.21.15.134:0/1162834041 >> 172.21.15.134:0/1162834041 conn(0x7f11d00a6110 msgr2=0x7f11d00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 -- 172.21.15.134:0/1162834041 shutdown_connections 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.759+0000 7f11d7896640 1 -- 172.21.15.134:0/1162834041 wait complete. 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.760+0000 7f11d7896640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.760+0000 7f11d7896640 10 monclient: monmap: 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.774 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.760+0000 7f11d7896640 1 Processor -- start 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.760+0000 7f11d7896640 1 -- start start 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.760+0000 7f11d7896640 10 monclient: init 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient: _add_conns ranks=[3,5,1,8,0,2,6,4,7] 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-g con 0x7f11d0135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 0x7f11d0134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-h con 0x7f11d0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d0137220 0x7f11d0121820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): picked mon.noname-f con 0x7f11d0137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00a8a30 con 0x7f11d0137220 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01368e0 con 0x7f11d0135dc0 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01363b0 con 0x7f11d0136e40 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d7896640 10 monclient(hunting): authenticate will time out at 1732.350213s 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 0x7f11d0134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d0137220 0x7f11d0121820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 0x7f11d0134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:57578/0 (socket says 172.21.15.134:57578) 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 -- 172.21.15.134:0/2023662995 learned_addr learned my addr 172.21.15.134:0/2023662995 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): get_auth_request con 0x7f11d0136e40 auth_method 0 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): get_auth_request con 0x7f11d0137220 auth_method 0 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): get_auth_request con 0x7f11d0135dc0 auth_method 0 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient(hunting): handle_auth_done global_id 45167 payload 824 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 msgr2=0x7f11d0134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d0135dc0 0x7f11d0134a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d0137220 msgr2=0x7f11d0121820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d0137220 0x7f11d0121820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 monclient: found mon.noname-h 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d4e31640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009400 con 0x7f11d0136e40 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.761+0000 7f11d5e33640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d0137220 0x7f11d0121820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d4e31640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 msgr2=0x7f11d0134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d4e31640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 0x7f11d0134fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11c000e960 tx=0x7f11c000ee20 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11c000cc10 con 0x7f11d0136e40 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: dump: 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 5 monclient: authenticate success, global_id 45167 2024-09-16T15:16:35.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c000cd90 con 0x7f11d0136e40 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 10 monclient: _renew_subs 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11c0013d50 con 0x7f11d0136e40 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc0094d0 con 0x7f11d0136e40 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c5ffb640 10 monclient: dump: 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.781 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11c77fe640 4 set_mon_vals no callback set 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 10 monclient: _renew_subs 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.764+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0125210 con 0x7f11d0136e40 2024-09-16T15:16:35.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f11c0013470 con 0x7f11d0136e40 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d7896640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d7896640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 0x7f11d0061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0071b20 con 0x7f11d00610b0 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11c5ffb640 1 --2- 172.21.15.134:0/2023662995 >> v2:172.21.15.162:6800/980547263 conn(0x7f11b4038880 0x7f11b403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 0x7f11d0061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f11c004a3b0 con 0x7f11d0136e40 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient: get_auth_request con 0x7f11d00610b0 auth_method 0 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5e33640 1 --2- 172.21.15.134:0/2023662995 >> v2:172.21.15.162:6800/980547263 conn(0x7f11b4038880 0x7f11b403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5e33640 10 monclient: get_auth_request con 0x7f11b4038880 auth_method 0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 10 monclient(hunting): handle_auth_done global_id 45176 payload 824 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.765+0000 7f11d5632640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11d0061ae0 con 0x7f11d00610b0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11d5632640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 msgr2=0x7f11d0061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11d5632640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 0x7f11d0061490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11d0137f00 tx=0x7f11cc0046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11d5e33640 1 --2- 172.21.15.134:0/2023662995 >> v2:172.21.15.162:6800/980547263 conn(0x7f11b4038880 0x7f11b403ad40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f11d0123000 tx=0x7f11c8009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11cc030d40 con 0x7f11d00610b0 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 10 monclient: dump: 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.784 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f11cc003610 con 0x7f11d00610b0 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 msgr2=0x7f11d0061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 0x7f11d0061490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11d0137f00 tx=0x7f11cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 msgr2=0x7f11d0061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.768+0000 7f11c5ffb640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d00610b0 0x7f11d0061490 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f11d0137f00 tx=0x7f11cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d7896640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d7896640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 0x7f11d002b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d7896640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11d007a680 con 0x7f11d002b260 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 0x7f11d002b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient: get_auth_request con 0x7f11d002b260 auth_method 0 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.774+0000 7f11d5632640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.775+0000 7f11d5632640 10 monclient(hunting): handle_auth_done global_id 45185 payload 824 2024-09-16T15:16:35.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.775+0000 7f11d5632640 1 -- 172.21.15.134:0/2023662995 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f11d009b490 con 0x7f11d002b260 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11d5632640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 msgr2=0x7f11d002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11d5632640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 0x7f11d002b670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc008030 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f11cc030ea0 con 0x7f11d002b260 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11c5ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11c5ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.777+0000 7f11c5ffb640 10 monclient: dump: 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f11cc003a00 con 0x7f11d002b260 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 msgr2=0x7f11d002b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 0x7f11d002b670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc008030 comp rx=0 tx=0).stop 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 msgr2=0x7f11d002b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11c5ffb640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d002b260 0x7f11d002b670 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc008030 comp rx=0 tx=0).stop 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.791 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.792 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.793 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.794 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.795 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.797 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.800 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.803 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.806 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.809 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.811 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 >> v2:172.21.15.162:6800/980547263 conn(0x7f11b4038880 msgr2=0x7f11b403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 --2- 172.21.15.134:0/2023662995 >> v2:172.21.15.162:6800/980547263 conn(0x7f11b4038880 0x7f11b403ad40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f11d0123000 tx=0x7f11c8009820 comp rx=0 tx=0).stop 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 10 monclient: shutdown 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 msgr2=0x7f11d0134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 --2- 172.21.15.134:0/2023662995 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f11d0136e40 0x7f11d0134fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11c000e960 tx=0x7f11c000ee20 comp rx=0 tx=0).stop 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d5e33640 1 -- 172.21.15.134:0/2023662995 reap_dead start 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d5e33640 1 -- 172.21.15.134:0/2023662995 reap_dead start 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 shutdown_connections 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 >> 172.21.15.134:0/2023662995 conn(0x7f11d00a6110 msgr2=0x7f11d009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 shutdown_connections 2024-09-16T15:16:35.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.778+0000 7f11d7896640 1 -- 172.21.15.134:0/2023662995 wait complete. 2024-09-16T15:16:35.813 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.814 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:16:35.814 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 10 monclient: get_monmap_and_config 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 10 monclient: monmap: 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:35.912105+0000 2024-09-16T15:16:35.912 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:35.912105+0000 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:35.913 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 1 Processor -- start 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 1 -- start start 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.911+0000 7f7541096640 10 monclient: init 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient: _add_conns ranks=[7,1,3,9,6,10,2,0,4,5,8] 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 0x7f753c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): picked mon.noname-i con 0x7f753c136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f753c137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f753c13c370 0x7f753c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): picked mon.noname-g con 0x7f753c13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c073250 con 0x7f753c137220 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a56e0 con 0x7f753c13c370 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f753c09abc0 con 0x7f753c136e40 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f7541096640 10 monclient(hunting): authenticate will time out at 1732.501251s 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753a59c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f753c137220 0x7f753c137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:58236/0 (socket says 172.21.15.134:58236) 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753a59c640 1 -- 172.21.15.134:0/3066867098 learned_addr learned my addr 172.21.15.134:0/3066867098 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 0x7f753c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f753c13c370 0x7f753c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): get_auth_request con 0x7f753c136e40 auth_method 0 2024-09-16T15:16:35.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): get_auth_request con 0x7f753c13c370 auth_method 0 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient(hunting): handle_auth_done global_id 114181 payload 274 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 1 -- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f753c13c370 msgr2=0x7f753c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f753c13c370 0x7f753c13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 1 -- 172.21.15.134:0/3066867098 >> v1:172.21.15.162:6789/0 conn(0x7f753c137220 legacy=0x7f753c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.912+0000 7f753ad9d640 1 monclient: found mon.noname-i 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.913+0000 7f753ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.913+0000 7f753ad9d640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:16:35.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.913+0000 7f753ad9d640 1 -- 172.21.15.134:0/3066867098 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c09ba50 con 0x7f753c136e40 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.915+0000 7f753ad9d640 1 -- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 msgr2=0x7f753c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.915+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 0x7f753c09c150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f75240097d0 tx=0x7f752402a000 comp rx=0 tx=0).ready entity=mon.7 client_cookie=dedd7c76c2ba547e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 1 -- 172.21.15.134:0/3066867098 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f752402ad20 con 0x7f753c136e40 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: dump: 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.925 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 1 -- 172.21.15.134:0/3066867098 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75240307b0 con 0x7f753c136e40 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 5 monclient: authenticate success, global_id 114181 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 1 -- 172.21.15.134:0/3066867098 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f752402aea0 con 0x7f753c136e40 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 4 set_mon_vals no callback set 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7539d9b640 10 monclient: dump: 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.927 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 10 monclient: shutdown 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 -- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 msgr2=0x7f753c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 0x7f753c09c150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f75240097d0 tx=0x7f752402a000 comp rx=0 tx=0).stop 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 -- 172.21.15.134:0/3066867098 shutdown_connections 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f753c13c370 0x7f753c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 --2- 172.21.15.134:0/3066867098 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c136e40 0x7f753c09c150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 -- 172.21.15.134:0/3066867098 >> 172.21.15.134:0/3066867098 conn(0x7f753c0a6110 msgr2=0x7f753c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 -- 172.21.15.134:0/3066867098 shutdown_connections 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.916+0000 7f7541096640 1 -- 172.21.15.134:0/3066867098 wait complete. 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.917+0000 7f7541096640 10 monclient: build_initial_monmap 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.917+0000 7f7541096640 10 monclient: monmap: 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:35.929 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.917+0000 7f7541096640 1 Processor -- start 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.917+0000 7f7541096640 1 -- start start 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.917+0000 7f7541096640 10 monclient: init 2024-09-16T15:16:35.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient: _add_conns ranks=[4,7,1,2,8,5,3,6,0] 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): picked mon.noname-c con 0x7f753c136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c137220 0x7f753c135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): picked mon.noname-i con 0x7f753c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f753c13c370 0x7f753c136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): picked mon.noname-f con 0x7f753c13c370 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c13b610 con 0x7f753c13c370 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a7be0 con 0x7f753c136e40 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1225c0 con 0x7f753c137220 2024-09-16T15:16:35.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f7541096640 10 monclient(hunting): authenticate will time out at 1732.507339s 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:47966/0 (socket says 172.21.15.134:47966) 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 -- 172.21.15.134:0/3631857118 learned_addr learned my addr 172.21.15.134:0/3631857118 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): get_auth_request con 0x7f753c136e40 auth_method 0 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c137220 0x7f753c135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f753c13c370 0x7f753c136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): get_auth_request con 0x7f753c137220 auth_method 0 2024-09-16T15:16:35.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): get_auth_request con 0x7f753c13c370 auth_method 0 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient(hunting): handle_auth_done global_id 6070 payload 824 2024-09-16T15:16:35.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c137220 msgr2=0x7f753c135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c137220 0x7f753c135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f753c13c370 msgr2=0x7f753c136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f753c13c370 0x7f753c136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 monclient: found mon.noname-c 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.918+0000 7f753ad9d640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009400 con 0x7f753c136e40 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f753ad9d640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 msgr2=0x7f753c0a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7524000c00 tx=0x7f75240046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7524004460 con 0x7f753c136e40 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: dump: 2024-09-16T15:16:35.934 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7524003a80 con 0x7f753c136e40 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 5 monclient: authenticate success, global_id 6070 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 10 monclient: _renew_subs 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7524003bf0 con 0x7f753c136e40 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c0a76d0 con 0x7f753c136e40 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f75337fe640 10 monclient: dump: 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.936 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f753959a640 4 set_mon_vals no callback set 2024-09-16T15:16:35.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 10 monclient: _renew_subs 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.921+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c0a32c0 con 0x7f753c136e40 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f7541096640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753ad9d640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7524000c00 tx=0x7f75240046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f752406a790 osd_map(723..723 src has 1..723) v4 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f7541096640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 0x7f753c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f753c071b20 con 0x7f753c121d20 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7524039040 con 0x7f753c136e40 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f75337fe640 1 --2- 172.21.15.134:0/3631857118 >> v2:172.21.15.162:6800/980547263 conn(0x7f751c038880 0x7f751c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 0x7f753c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient: get_auth_request con 0x7f753c121d20 auth_method 0 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753b59e640 1 --2- 172.21.15.134:0/3631857118 >> v2:172.21.15.162:6800/980547263 conn(0x7f751c038880 0x7f751c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753b59e640 10 monclient: get_auth_request con 0x7f751c038880 auth_method 0 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 10 monclient(hunting): handle_auth_done global_id 114199 payload 824 2024-09-16T15:16:35.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.922+0000 7f753a59c640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f753c061ae0 con 0x7f753c121d20 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.924+0000 7f753a59c640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 msgr2=0x7f753c11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.924+0000 7f753a59c640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 0x7f753c11fc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c00ef00 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f753b59e640 1 --2- 172.21.15.134:0/3631857118 >> v2:172.21.15.162:6800/980547263 conn(0x7f751c038880 0x7f751c03ad40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f753c06b7d0 tx=0x7f753400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f752c00cc50 con 0x7f753c121d20 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 10 monclient: dump: 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.941 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f752c00cdb0 con 0x7f753c121d20 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 msgr2=0x7f753c11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 0x7f753c11fc60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c00ef00 comp rx=0 tx=0).stop 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 msgr2=0x7f753c11fc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.925+0000 7f75337fe640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c121d20 0x7f753c11fc60 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c00ef00 comp rx=0 tx=0).stop 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f7541096640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f7541096640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 0x7f753c1205b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f7541096640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f753c07a680 con 0x7f753c1201a0 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 0x7f753c1205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient: get_auth_request con 0x7f753c1201a0 auth_method 0 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:35.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.931+0000 7f753a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.932+0000 7f753a59c640 10 monclient(hunting): handle_auth_done global_id 114208 payload 824 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.932+0000 7f753a59c640 1 -- 172.21.15.134:0/3631857118 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f753c0a53c0 con 0x7f753c1201a0 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f753a59c640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 msgr2=0x7f753c1205b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f753a59c640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 0x7f753c1205b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f752c018040 tx=0x7f752c004940 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f752c01d300 con 0x7f753c1201a0 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f75337fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f75337fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.934+0000 7f75337fe640 10 monclient: dump: 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:35.944 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f752c021070 con 0x7f753c1201a0 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:35.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 msgr2=0x7f753c1205b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 0x7f753c1205b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f752c018040 tx=0x7f752c004940 comp rx=0 tx=0).stop 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 msgr2=0x7f753c1205b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f753c1201a0 0x7f753c1205b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 136, 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:16:35.946 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.947 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.948 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:35.949 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:35.950 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.952 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:35.953 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.955 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.956 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:35.957 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:35.958 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.959 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:35.960 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.961 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:35.963 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.965 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:35.966 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:35.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 >> v2:172.21.15.162:6800/980547263 conn(0x7f751c038880 msgr2=0x7f751c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 --2- 172.21.15.134:0/3631857118 >> v2:172.21.15.162:6800/980547263 conn(0x7f751c038880 0x7f751c03ad40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f753c06b7d0 tx=0x7f753400b040 comp rx=0 tx=0).stop 2024-09-16T15:16:35.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 10 monclient: shutdown 2024-09-16T15:16:35.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 msgr2=0x7f753c0a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 --2- 172.21.15.134:0/3631857118 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f753c136e40 0x7f753c0a2720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7524000c00 tx=0x7f75240046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f753b59e640 1 -- 172.21.15.134:0/3631857118 reap_dead start 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f75337fe640 1 -- 172.21.15.134:0/3631857118 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (unknown 0 0 0) 0x7f752406a790 con 0x7f753c136e40 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f753b59e640 1 -- 172.21.15.134:0/3631857118 reap_dead start 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 shutdown_connections 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 >> 172.21.15.134:0/3631857118 conn(0x7f753c0a6110 msgr2=0x7f753c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 shutdown_connections 2024-09-16T15:16:35.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:35.935+0000 7f7541096640 1 -- 172.21.15.134:0/3631857118 wait complete. 2024-09-16T15:16:35.968 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:16:35.969 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'a', 'c'] 2024-09-16T15:16:35.969 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:16:35.969 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-16T15:16:35.969 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:16:35.969 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-16T15:16:35.969 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:16:35.987 INFO:tasks.ceph.mon.f:Stopped 2024-09-16T15:16:35.987 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-16T15:16:35.987 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:16:35.988 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:16:35.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:16:35.999 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:16:35.999 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-16T15:16:35.999 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-16T15:16:35.999 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T15:16:35.999 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:16:36.056 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T15:16:36.056 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-16T15:16:36.056 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.190+0000 7fa4ef6e9640 10 monclient: get_monmap_and_config 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.190+0000 7fa4ef6e9640 10 monclient: build_initial_monmap 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.190+0000 7fa4ef6e9640 10 monclient: monmap: 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:36.191616+0000 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:36.191616+0000 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:36.192 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:36.193 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 Processor -- start 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 -- start start 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient: init 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient: _add_conns ranks=[4,8,9,7,6,1,0,3,2,5,10] 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 0x7fa4e8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-c con 0x7fa4e8136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-e con 0x7fa4e809bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 0x7fa4e81370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-a con 0x7fa4e8136c60 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:36.200 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e809b600 con 0x7fa4e8136c60 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80a5740 con 0x7fa4e8136030 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80a52c0 con 0x7fa4e809bfe0 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ef6e9640 10 monclient(hunting): authenticate will time out at 1732.780915s 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ed485640 1 -- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 msgr2=0x7fa4e8136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4ed485640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 0x7fa4e8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4edc86640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 msgr2=0x7fa4e81370e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.191+0000 7fa4edc86640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 0x7fa4e81370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:36.201 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:43996/0 (socket says 172.21.15.134:43996) 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 -- 172.21.15.134:0/4264050260 learned_addr learned my addr 172.21.15.134:0/4264050260 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): get_auth_request con 0x7fa4e809bfe0 auth_method 0 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient(hunting): handle_auth_done global_id 84698 payload 274 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 -- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 msgr2=0x7fa4e8136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 --2- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 0x7fa4e8136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 -- 172.21.15.134:0/4264050260 >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 msgr2=0x7fa4e81370e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 --2- 172.21.15.134:0/4264050260 >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 0x7fa4e81370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient: _finish_hunting 0 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 monclient: found mon.noname-e 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.192+0000 7fa4ecc84640 1 -- 172.21.15.134:0/4264050260 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8073250 con 0x7fa4e809bfe0 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ecc84640 1 -- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 msgr2=0x7fa4e809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ecc84640 1 --2- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009870 tx=0x7fa4d802a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=7f48ec8f9c23dc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 1 -- 172.21.15.134:0/4264050260 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d80307b0 con 0x7fa4e809bfe0 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: dump: 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:36.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:36.204 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: _finish_auth 0 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 1 -- 172.21.15.134:0/4264050260 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8030930 con 0x7fa4e809bfe0 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 5 monclient: authenticate success, global_id 84698 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 1 -- 172.21.15.134:0/4264050260 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4d80313a0 con 0x7fa4e809bfe0 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 4 set_mon_vals no callback set 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:36.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4dffff640 10 monclient: dump: 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:36.206 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 10 monclient: shutdown 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 msgr2=0x7fa4e809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009870 tx=0x7fa4d802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/4264050260 shutdown_connections 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/4264050260 >> v2:172.21.15.134:3300/0 conn(0x7fa4e8136c60 0x7fa4e81370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa4e809bfe0 0x7fa4e809c400 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/4264050260 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e8136030 0x7fa4e8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/4264050260 >> 172.21.15.134:0/4264050260 conn(0x7fa4e80a9ea0 msgr2=0x7fa4e80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:36.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/4264050260 shutdown_connections 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.195+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/4264050260 wait complete. 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.196+0000 7fa4ef6e9640 10 monclient: build_initial_monmap 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.196+0000 7fa4ef6e9640 10 monclient: monmap: 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:36.208 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.196+0000 7fa4ef6e9640 1 Processor -- start 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.196+0000 7fa4ef6e9640 1 -- start start 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.196+0000 7fa4ef6e9640 10 monclient: init 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient: _add_conns ranks=[4,5,2,6,1,7,8,0,3] 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 0x7fa4e81bef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-c con 0x7fa4e809bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 0x7fa4e8135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-h con 0x7fa4e8136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): picked mon.noname-b con 0x7fa4e8136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e811ff30 con 0x7fa4e8136c60 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80a7a20 con 0x7fa4e809bfe0 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80a33e0 con 0x7fa4e8136030 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ef6e9640 10 monclient(hunting): authenticate will time out at 1732.786236s 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ed485640 1 -- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 msgr2=0x7fa4e81bef40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ed485640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 0x7fa4e81bef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36076/0 (socket says 172.21.15.134:36076) 2024-09-16T15:16:36.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 -- 172.21.15.134:0/636297767 learned_addr learned my addr 172.21.15.134:0/636297767 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): get_auth_request con 0x7fa4e8136c60 auth_method 0 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ecc84640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 0x7fa4e8135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient(hunting): handle_auth_done global_id 114302 payload 824 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 -- 172.21.15.134:0/636297767 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 msgr2=0x7fa4e8135120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 0x7fa4e8135120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 -- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 msgr2=0x7fa4e81bef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 0x7fa4e81bef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient: _finish_hunting 0 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 monclient: found mon.noname-b 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4edc86640 1 -- 172.21.15.134:0/636297767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80094d0 con 0x7fa4e8136c60 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ecc84640 10 monclient: get_auth_request con 0x7fa4e8136030 auth_method 0 2024-09-16T15:16:36.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.197+0000 7fa4ecc84640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 0x7fa4e8135120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.199+0000 7fa4edc86640 1 -- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 msgr2=0x7fa4e81cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.199+0000 7fa4edc86640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000be60 tx=0x7fa4e000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4e0015850 con 0x7fa4e8136c60 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:36.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: dump: 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:36.215 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00159d0 con 0x7fa4e8136c60 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 5 monclient: authenticate success, global_id 114302 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 10 monclient: _renew_subs 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80a7d50 con 0x7fa4e8136c60 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa4e00060d0 con 0x7fa4e8136c60 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:36.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 10 monclient: dump: 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:36.217 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4df7fe640 4 set_mon_vals no callback set 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 10 monclient: _renew_subs 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e8122800 con 0x7fa4e8136c60 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.200+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa4e0012020 con 0x7fa4e8136c60 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ddffb640 1 --2- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 0x7fa4cc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7fa4e004a7a0 con 0x7fa4e8136c60 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ef6e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ef6e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ef6e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e80679b0 con 0x7fa4e8136c60 2024-09-16T15:16:36.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ed485640 1 --2- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 0x7fa4cc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:36.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.201+0000 7fa4ed485640 10 monclient: get_auth_request con 0x7fa4cc0388b0 auth_method 0 2024-09-16T15:16:36.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.203+0000 7fa4ed485640 1 --2- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 0x7fa4cc03ad70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4006fd0 tx=0x7fa4e4006ce0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:36.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.205+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa4e00078e0 con 0x7fa4e8136c60 2024-09-16T15:16:36.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.205+0000 7fa4ddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:36.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.205+0000 7fa4ddffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:36.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.428+0000 7fa4ef6e9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:36.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.428+0000 7fa4ef6e9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:36.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.428+0000 7fa4ef6e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:36.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.428+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa4e8121350 con 0x7fa4e8136c60 2024-09-16T15:16:36.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.429+0000 7fa4ddffb640 1 -- 172.21.15.134:0/636297767 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fa4e003c030 con 0x7fa4e8136c60 2024-09-16T15:16:36.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.429+0000 7fa4ddffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:36.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.429+0000 7fa4ddffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:36.431 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":136,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 msgr2=0x7fa4cc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 0x7fa4cc03ad70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4006fd0 tx=0x7fa4e4006ce0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 10 monclient: shutdown 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 msgr2=0x7fa4e81cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000be60 tx=0x7fa4e000bf50 comp rx=0 tx=0).stop 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 shutdown_connections 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> v2:172.21.15.162:6800/980547263 conn(0x7fa4cc0388b0 0x7fa4cc03ad70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4e8136c60 0x7fa4e81cb5a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa4e8136030 0x7fa4e8135120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 --2- 172.21.15.134:0/636297767 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fa4e809bfe0 0x7fa4e81bef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:36.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 >> 172.21.15.134:0/636297767 conn(0x7fa4e80a9ea0 msgr2=0x7fa4e80a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:36.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 shutdown_connections 2024-09-16T15:16:36.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:36.431+0000 7fa4ef6e9640 1 -- 172.21.15.134:0/636297767 wait complete. 2024-09-16T15:16:39.443 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:39.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.549+0000 7f3f482e9640 10 monclient: get_monmap_and_config 2024-09-16T15:16:39.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.549+0000 7f3f482e9640 10 monclient: build_initial_monmap 2024-09-16T15:16:39.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.549+0000 7f3f482e9640 10 monclient: monmap: 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:39.550733+0000 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:39.550733+0000 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:39.552 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.550+0000 7f3f482e9640 1 Processor -- start 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.550+0000 7f3f482e9640 1 -- start start 2024-09-16T15:16:39.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.550+0000 7f3f482e9640 10 monclient: init 2024-09-16T15:16:39.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:39.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient: _add_conns ranks=[9,10,7,1,0,4,8,2,5,3,6] 2024-09-16T15:16:39.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 0x7f3f40136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-a con 0x7f3f40136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:39.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 0x7f3f4009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-f con 0x7f3f4009bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-i con 0x7f3f40136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4009b600 con 0x7f3f40136030 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400a5740 con 0x7f3f4009bfe0 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400a52c0 con 0x7f3f40136c60 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f482e9640 10 monclient(hunting): authenticate will time out at 1736.140249s 2024-09-16T15:16:39.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46085640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 msgr2=0x7f3f40136410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46085640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 0x7f3f40136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f45884640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 msgr2=0x7f3f4009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f45884640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 0x7f3f4009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:41466/0 (socket says 172.21.15.134:41466) 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 -- 172.21.15.134:0/3640852061 learned_addr learned my addr 172.21.15.134:0/3640852061 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): get_auth_request con 0x7f3f40136c60 auth_method 0 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:39.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient(hunting): handle_auth_done global_id 114217 payload 274 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 -- 172.21.15.134:0/3640852061 >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 msgr2=0x7f3f4009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 --2- 172.21.15.134:0/3640852061 >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 0x7f3f4009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 -- 172.21.15.134:0/3640852061 >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 msgr2=0x7f3f40136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 --2- 172.21.15.134:0/3640852061 >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 0x7f3f40136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient: _finish_hunting 0 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 monclient: found mon.noname-i 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:16:39.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.551+0000 7f3f46886640 1 -- 172.21.15.134:0/3640852061 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40073250 con 0x7f3f40136c60 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.554+0000 7f3f46886640 1 -- 172.21.15.134:0/3640852061 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 msgr2=0x7f3f401370f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.554+0000 7f3f46886640 1 --2- 172.21.15.134:0/3640852061 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f400a5950 tx=0x7f3f3802a5b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=cafa6a8750e940b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 1 -- 172.21.15.134:0/3640852061 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f38007a70 con 0x7f3f40136c60 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: dump: 2024-09-16T15:16:39.564 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:39.565 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: _finish_auth 0 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: _check_auth_tickets 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 1 -- 172.21.15.134:0/3640852061 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3802d050 con 0x7f3f40136c60 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 5 monclient: authenticate success, global_id 114217 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 1 -- 172.21.15.134:0/3640852061 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f38007bf0 con 0x7f3f40136c60 2024-09-16T15:16:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f45083640 10 monclient: dump: 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:39.567 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 4 set_mon_vals no callback set 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 10 monclient: shutdown 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 -- 172.21.15.134:0/3640852061 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 msgr2=0x7f3f401370f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 --2- 172.21.15.134:0/3640852061 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f400a5950 tx=0x7f3f3802a5b0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 -- 172.21.15.134:0/3640852061 shutdown_connections 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 --2- 172.21.15.134:0/3640852061 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f40136c60 0x7f3f401370f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 --2- 172.21.15.134:0/3640852061 >> v2:172.21.15.162:3300/0 conn(0x7f3f4009bfe0 0x7f3f4009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 --2- 172.21.15.134:0/3640852061 >> v2:172.21.15.134:3300/0 conn(0x7f3f40136030 0x7f3f40136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.555+0000 7f3f482e9640 1 -- 172.21.15.134:0/3640852061 >> 172.21.15.134:0/3640852061 conn(0x7f3f400a9ea0 msgr2=0x7f3f400a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.556+0000 7f3f482e9640 1 -- 172.21.15.134:0/3640852061 shutdown_connections 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.556+0000 7f3f482e9640 1 -- 172.21.15.134:0/3640852061 wait complete. 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.556+0000 7f3f482e9640 10 monclient: build_initial_monmap 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.556+0000 7f3f482e9640 10 monclient: monmap: 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:39.569 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.556+0000 7f3f482e9640 1 Processor -- start 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 -- start start 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient: init 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient: _add_conns ranks=[7,2,4,8,5,3,1,0,6] 2024-09-16T15:16:39.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f4009bfe0 0x7f3f401bda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-i con 0x7f3f4009bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-b con 0x7f3f40136030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 0x7f3f401351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): picked mon.noname-c con 0x7f3f40136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40122620 con 0x7f3f40136030 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40122340 con 0x7f3f40136c60 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40121d80 con 0x7f3f4009bfe0 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f482e9640 10 monclient(hunting): authenticate will time out at 1736.146549s 2024-09-16T15:16:39.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46886640 1 -- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 msgr2=0x7f3f401351a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46886640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 0x7f3f401351a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36088/0 (socket says 172.21.15.134:36088) 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 1 -- 172.21.15.134:0/221694865 learned_addr learned my addr 172.21.15.134:0/221694865 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): get_auth_request con 0x7f3f40136030 auth_method 0 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f4009bfe0 0x7f3f401bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): get_auth_request con 0x7f3f4009bfe0 auth_method 0 2024-09-16T15:16:39.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f46085640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 10 monclient(hunting): handle_auth_done global_id 114311 payload 824 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 1 -- 172.21.15.134:0/221694865 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f4009bfe0 msgr2=0x7f3f401bda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.557+0000 7f3f45884640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f4009bfe0 0x7f3f401bda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 1 -- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 msgr2=0x7f3f401351a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 0x7f3f401351a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 10 monclient: _finish_hunting 0 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 1 monclient: found mon.noname-b 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:39.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.558+0000 7f3f45884640 1 -- 172.21.15.134:0/221694865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38009b30 con 0x7f3f40136030 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f45884640 1 -- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 msgr2=0x7f3f40134c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f45884640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000b550 tx=0x7f3f3000b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f30009320 con 0x7f3f40136030 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: dump: 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.574 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30003dd0 con 0x7f3f40136030 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3f30010400 con 0x7f3f40136030 2024-09-16T15:16:39.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 5 monclient: authenticate success, global_id 114311 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f44882640 4 set_mon_vals no callback set 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f36ffd640 10 monclient: dump: 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:39.576 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 10 monclient: _renew_subs 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f40122060 con 0x7f3f40136030 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 10 monclient: _renew_subs 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.560+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f40120120 con 0x7f3f40136030 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3f30005070 con 0x7f3f40136030 2024-09-16T15:16:39.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f482e9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f482e9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f482e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4009c3e0 con 0x7f3f40136030 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f36ffd640 1 --2- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 0x7f3f2403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f3f30049c30 con 0x7f3f40136030 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f46085640 1 --2- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 0x7f3f2403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.561+0000 7f3f46085640 10 monclient: get_auth_request con 0x7f3f24038880 auth_method 0 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.564+0000 7f3f46085640 1 --2- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 0x7f3f2403ad40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c003980 tx=0x7f3f3c009110 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:39.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.566+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3f300057e0 con 0x7f3f40136030 2024-09-16T15:16:39.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.566+0000 7f3f36ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:39.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.566+0000 7f3f36ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:39.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.788+0000 7f3f482e9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:39.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.788+0000 7f3f482e9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:39.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.788+0000 7f3f482e9640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:39.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.788+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3f40072070 con 0x7f3f40136030 2024-09-16T15:16:39.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.788+0000 7f3f36ffd640 1 -- 172.21.15.134:0/221694865 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3197 (secure 0 0 0) 0x7f3f3000fb50 con 0x7f3f40136030 2024-09-16T15:16:39.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.789+0000 7f3f36ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:39.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.789+0000 7f3f36ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:39.791 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":136,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:39.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 msgr2=0x7f3f2403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:39.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 0x7f3f2403ad40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c003980 tx=0x7f3f3c009110 comp rx=0 tx=0).stop 2024-09-16T15:16:39.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 10 monclient: shutdown 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 msgr2=0x7f3f40134c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000b550 tx=0x7f3f3000b950 comp rx=0 tx=0).stop 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 shutdown_connections 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> v2:172.21.15.162:6800/980547263 conn(0x7f3f24038880 0x7f3f2403ad40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f3f40136c60 0x7f3f401351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f40136030 0x7f3f40134c60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 --2- 172.21.15.134:0/221694865 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f3f4009bfe0 0x7f3f401bda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.790+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 >> 172.21.15.134:0/221694865 conn(0x7f3f400a9ea0 msgr2=0x7f3f4006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.791+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 shutdown_connections 2024-09-16T15:16:39.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:39.791+0000 7f3f482e9640 1 -- 172.21.15.134:0/221694865 wait complete. 2024-09-16T15:16:42.803 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:42.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 10 monclient: get_monmap_and_config 2024-09-16T15:16:42.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 10 monclient: build_initial_monmap 2024-09-16T15:16:42.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 10 monclient: monmap: 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:42.911170+0000 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:42.911170+0000 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:42.912 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:42.913 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:42.913 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:42.913 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:42.913 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:42.913 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 1 Processor -- start 2024-09-16T15:16:42.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 1 -- start start 2024-09-16T15:16:42.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.910+0000 7ff0d4b95640 10 monclient: init 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient: _add_conns ranks=[8,9,0,5,2,1,3,7,6,10,4] 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-e con 0x7ff0d0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 0x7ff0d009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-a con 0x7ff0d009bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff0d0136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d0073250 con 0x7ff0d0136c60 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d009b600 con 0x7ff0d009bfe0 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:42.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00a5740 con 0x7ff0d0136030 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0d4b95640 10 monclient(hunting): authenticate will time out at 1739.500611s 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0cdd9b640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 msgr2=0x7ff0d009c400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ced9d640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7ff0d0136c60 legacy=0x7ff0d0137110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0cdd9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 0x7ff0d009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:44012/0 (socket says 172.21.15.134:44012) 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 -- 172.21.15.134:0/2061658635 learned_addr learned my addr 172.21.15.134:0/2061658635 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): get_auth_request con 0x7ff0d0136030 auth_method 0 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:42.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient(hunting): handle_auth_done global_id 84707 payload 274 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 -- 172.21.15.134:0/2061658635 >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 msgr2=0x7ff0d009c400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 --2- 172.21.15.134:0/2061658635 >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 0x7ff0d009c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 -- 172.21.15.134:0/2061658635 >> v1:172.21.15.134:6789/0 conn(0x7ff0d0136c60 legacy=0x7ff0d0137110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 monclient: found mon.noname-e 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.911+0000 7ff0ce59c640 1 -- 172.21.15.134:0/2061658635 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d00a52c0 con 0x7ff0d0136030 2024-09-16T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.914+0000 7ff0ce59c640 1 -- 172.21.15.134:0/2061658635 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 msgr2=0x7ff0d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.914+0000 7ff0ce59c640 1 --2- 172.21.15.134:0/2061658635 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009870 tx=0x7ff0c002a110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=4db38ecc57fb42ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:42.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 1 -- 172.21.15.134:0/2061658635 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c00307b0 con 0x7ff0d0136030 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: dump: 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:42.924 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: _finish_auth 0 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: _check_auth_tickets 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 1 -- 172.21.15.134:0/2061658635 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0030930 con 0x7ff0d0136030 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 5 monclient: authenticate success, global_id 84707 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 1 -- 172.21.15.134:0/2061658635 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c00313a0 con 0x7ff0d0136030 2024-09-16T15:16:42.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 4 set_mon_vals no callback set 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0cd59a640 10 monclient: dump: 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:42.926 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 10 monclient: shutdown 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 -- 172.21.15.134:0/2061658635 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 msgr2=0x7ff0d0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/2061658635 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009870 tx=0x7ff0c002a110 comp rx=0 tx=0).stop 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 -- 172.21.15.134:0/2061658635 shutdown_connections 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/2061658635 >> v2:172.21.15.134:3300/0 conn(0x7ff0d009bfe0 0x7ff0d009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/2061658635 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136030 0x7ff0d0136410 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 -- 172.21.15.134:0/2061658635 >> 172.21.15.134:0/2061658635 conn(0x7ff0d00a9ea0 msgr2=0x7ff0d00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:42.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 -- 172.21.15.134:0/2061658635 shutdown_connections 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.915+0000 7ff0d4b95640 1 -- 172.21.15.134:0/2061658635 wait complete. 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.916+0000 7ff0d4b95640 10 monclient: build_initial_monmap 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.916+0000 7ff0d4b95640 10 monclient: monmap: 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:42.928 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.916+0000 7ff0d4b95640 1 Processor -- start 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.916+0000 7ff0d4b95640 1 -- start start 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.916+0000 7ff0d4b95640 10 monclient: init 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient: _add_conns ranks=[5,7,8,0,4,1,6,3,2] 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff0d009bfe0 0x7ff0d00a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-h con 0x7ff0d009bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff0d0136030 0x7ff0d0134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-i con 0x7ff0d0136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): picked mon.noname-e con 0x7ff0d0136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0121ce0 con 0x7ff0d009bfe0 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00a7ac0 con 0x7ff0d0136030 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00a7e80 con 0x7ff0d0136c60 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0d4b95640 10 monclient(hunting): authenticate will time out at 1739.506428s 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:44018/0 (socket says 172.21.15.134:44018) 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 -- 172.21.15.134:0/3144743213 learned_addr learned my addr 172.21.15.134:0/3144743213 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): get_auth_request con 0x7ff0d0136c60 auth_method 0 2024-09-16T15:16:42.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0cdd9b640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff0d0136030 0x7ff0d0134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0cdd9b640 10 monclient(hunting): get_auth_request con 0x7ff0d0136030 auth_method 0 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0cdd9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ce59c640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff0d009bfe0 0x7ff0d00a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0cdd9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0cdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:42.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ce59c640 10 monclient(hunting): get_auth_request con 0x7ff0d009bfe0 auth_method 0 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ce59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ce59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ce59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient(hunting): handle_auth_done global_id 84716 payload 824 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 -- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff0d0136030 msgr2=0x7ff0d0134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff0d0136030 0x7ff0d0134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 -- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff0d009bfe0 msgr2=0x7ff0d00a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff0d009bfe0 0x7ff0d00a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 monclient: found mon.noname-e 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.917+0000 7ff0ced9d640 1 -- 172.21.15.134:0/3144743213 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00094d0 con 0x7ff0d0136c60 2024-09-16T15:16:42.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0ced9d640 1 -- 172.21.15.134:0/3144743213 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 msgr2=0x7ff0d0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0ced9d640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800be60 tx=0x7ff0c800bf50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c80158e0 con 0x7ff0d0136c60 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: dump: 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:42.933 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8015a60 con 0x7ff0d0136c60 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 5 monclient: authenticate success, global_id 84716 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 10 monclient: _renew_subs 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:42.934 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d009ff70 con 0x7ff0d0136c60 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff0c8015c10 con 0x7ff0d0136c60 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c6ffd640 10 monclient: dump: 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:42.935 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 10 monclient: _renew_subs 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0c7fff640 4 set_mon_vals no callback set 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.920+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d0120d80 con 0x7ff0d0136c60 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff0c8012070 con 0x7ff0d0136c60 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0d4b95640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0d4b95640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0d4b95640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:42.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0121060 con 0x7ff0d0136c60 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0c6ffd640 1 --2- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 0x7ff0b003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(723..723 src has 1..723) v4 ==== 3899+0+0 (secure 0 0 0) 0x7ff0c804a420 con 0x7ff0d0136c60 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0ce59c640 1 --2- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 0x7ff0b003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.921+0000 7ff0ce59c640 10 monclient: get_auth_request con 0x7ff0b0038880 auth_method 0 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.924+0000 7ff0ce59c640 1 --2- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 0x7ff0b003ad40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0d006ccb0 tx=0x7ff0c002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.924+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:16:42.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.926+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff0c80072f0 con 0x7ff0d0136c60 2024-09-16T15:16:42.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.926+0000 7ff0c6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:42.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:42.926+0000 7ff0c6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.150+0000 7ff0d4b95640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.150+0000 7ff0d4b95640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.150+0000 7ff0d4b95640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.150+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff0d00679b0 con 0x7ff0d0136c60 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.151+0000 7ff0c6ffd640 1 -- 172.21.15.134:0/3144743213 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3197 (secure 0 0 0) 0x7ff0c8012b10 con 0x7ff0d0136c60 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.151+0000 7ff0c6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:43.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.151+0000 7ff0c6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:43.153 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":136,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 msgr2=0x7ff0b003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 0x7ff0b003ad40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0d006ccb0 tx=0x7ff0c002e000 comp rx=0 tx=0).stop 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 10 monclient: shutdown 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 msgr2=0x7ff0d0135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:43.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800be60 tx=0x7ff0c800bf50 comp rx=0 tx=0).stop 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.152+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 shutdown_connections 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> v2:172.21.15.162:6800/980547263 conn(0x7ff0b0038880 0x7ff0b003ad40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff0d0136c60 0x7ff0d0135100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ff0d0136030 0x7ff0d0134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 --2- 172.21.15.134:0/3144743213 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff0d009bfe0 0x7ff0d00a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 >> 172.21.15.134:0/3144743213 conn(0x7ff0d00a9ea0 msgr2=0x7ff0d01bbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 shutdown_connections 2024-09-16T15:16:43.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:43.153+0000 7ff0d4b95640 1 -- 172.21.15.134:0/3144743213 wait complete. 2024-09-16T15:16:46.165 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.272+0000 7f4b005ce640 10 monclient: get_monmap_and_config 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.272+0000 7f4b005ce640 10 monclient: build_initial_monmap 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.272+0000 7f4b005ce640 10 monclient: monmap: 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:46.273698+0000 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:46.273698+0000 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:46.274 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:46.275 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.273+0000 7f4b005ce640 1 Processor -- start 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.273+0000 7f4b005ce640 1 -- start start 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.273+0000 7f4b005ce640 10 monclient: init 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient: _add_conns ranks=[5,9,6,2,1,8,4,10,3,7,0] 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:46.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-h con 0x7f4af8137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-a con 0x7f4af809c130 addr v2:172.21.15.134:3300/0 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-d con 0x7f4af809c510 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af80a5840 con 0x7f4af809c130 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8137b40 con 0x7f4af8137220 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80a53c0 con 0x7f4af809c510 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4b005ce640 10 monclient(hunting): authenticate will time out at 1742.863210s 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afdb69640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 msgr2=0x7f4af80a3f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afdb69640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:46.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33594/0 (socket says 172.21.15.134:33594) 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 learned_addr learned my addr 172.21.15.134:0/3512856121 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4af809c510 auth_method 0 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4af8137220 auth_method 0 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:46.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:46.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 msgr2=0x7f4af80a3f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4af809c510 auth_method 0 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4af8137220 auth_method 0 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:46.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 msgr2=0x7f4af80a3f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:46.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:46.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4af809c510 auth_method 0 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4af8137220 auth_method 0 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:46.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:46.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:46.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 msgr2=0x7f4af80a3f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4af809c510 auth_method 0 2024-09-16T15:16:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4af8137220 auth_method 0 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:47.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:47.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:47.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:47.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:47.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:49.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): tick 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 monclient(hunting): continuing hunt 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 msgr2=0x7f4af80a4450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4af809c510 0x7f4af80a4450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 msgr2=0x7f4af8137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4af8137220 0x7f4af8137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 msgr2=0x7f4af80a3f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> v2:172.21.15.134:3300/0 conn(0x7f4af809c130 0x7f4af80a3f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient: _add_conns ranks=[8,4,7,3,1,2,0,10] 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): picked mon.noname-e con 0x7f4adc003810 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): picked mon.noname-c con 0x7f4adc006220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): picked mon.noname-i con 0x7f4adc008da0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:49.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80a5840 con 0x7f4adc006220 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8137b40 con 0x7f4adc008da0 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 1 -- 172.21.15.134:0/3512856121 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80a53c0 con 0x7f4adc003810 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4aeffff640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 msgr2=0x7f4adc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4adc003810 auth_method 0 2024-09-16T15:16:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4adc008da0 auth_method 0 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:49.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:16:49.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.274+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.275+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 msgr2=0x7f4adc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4adc003810 auth_method 0 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.475+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:49.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4adc008da0 auth_method 0 2024-09-16T15:16:49.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:49.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:49.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.476+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:16:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 msgr2=0x7f4adc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:49.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4adc003810 auth_method 0 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.876+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request con 0x7f4adc008da0 auth_method 0 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:16:49.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:49.877+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:16:50.638 INFO:tasks.rados.rados.0.smithi162.stdout:3403: done (0 left) 2024-09-16T15:16:50.638 INFO:tasks.rados.rados.0.smithi162.stdout:3404: snap_remove snap 318 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afdb69640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 msgr2=0x7f4adc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3302/0 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afdb69640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4adc003810 auth_method 0 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:50.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient(hunting): handle_auth_done global_id 84725 payload 274 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 msgr2=0x7f4adc00b230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4adc008da0 0x7f4adc00b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 msgr2=0x7f4adc0086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4adc006220 0x7f4adc0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient: _finish_hunting 0 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 monclient: found mon.noname-e 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afeb6b640 1 -- 172.21.15.134:0/3512856121 reap_dead start 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.677+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8073250 con 0x7f4adc003810 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.680+0000 7f4afe36a640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.680+0000 7f4afe36a640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009f00 tx=0x7f4af402c220 comp rx=0 tx=0).ready entity=mon.8 client_cookie=e1a6bead8dc9a040 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.680+0000 7f4afd368640 1 -- 172.21.15.134:0/3512856121 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4af4009660 con 0x7f4adc003810 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.680+0000 7f4afd368640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: dump: 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.683 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: _finish_auth 0 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: _check_auth_tickets 2024-09-16T15:16:50.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 5 monclient: authenticate success, global_id 84725 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 1 -- 172.21.15.134:0/3512856121 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4030070 con 0x7f4adc003810 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 4 set_mon_vals no callback set 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 1 -- 172.21.15.134:0/3512856121 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4af4036ce0 con 0x7f4adc003810 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4afd368640 10 monclient: dump: 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 10 monclient: shutdown 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 1 -- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4adc005c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009f00 tx=0x7f4af402c220 comp rx=0 tx=0).stop 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 1 -- 172.21.15.134:0/3512856121 shutdown_connections 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 1 --2- 172.21.15.134:0/3512856121 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4adc005c20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.681+0000 7f4b005ce640 1 -- 172.21.15.134:0/3512856121 >> 172.21.15.134:0/3512856121 conn(0x7f4af80a61f0 msgr2=0x7f4af80a6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.682+0000 7f4b005ce640 1 -- 172.21.15.134:0/3512856121 shutdown_connections 2024-09-16T15:16:50.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.682+0000 7f4b005ce640 1 -- 172.21.15.134:0/3512856121 wait complete. 2024-09-16T15:16:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.682+0000 7f4b005ce640 10 monclient: build_initial_monmap 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.682+0000 7f4b005ce640 10 monclient: monmap: 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 Processor -- start 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 -- start start 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient: init 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient: _add_conns ranks=[8,1,7,4,3,0,6,5,2] 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-e con 0x7f4adc003810 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 0x7f4af8121ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-f con 0x7f4af8121c10 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4af8122530 0x7f4af81d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): picked mon.noname-i con 0x7f4af8122530 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8124ce0 con 0x7f4af8121c10 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81256c0 con 0x7f4af8122530 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81252a0 con 0x7f4adc003810 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4b005ce640 10 monclient(hunting): authenticate will time out at 1747.272959s 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.683+0000 7f4afe36a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47510/0 (socket says 172.21.15.134:47510) 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 -- 172.21.15.134:0/887548843 learned_addr learned my addr 172.21.15.134:0/887548843 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): get_auth_request con 0x7f4adc003810 auth_method 0 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afdb69640 1 -- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 msgr2=0x7f4af8121ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afdb69640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 0x7f4af8121ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afeb6b640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4af8122530 0x7f4af81d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient(hunting): handle_auth_done global_id 84734 payload 824 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 -- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4af8122530 msgr2=0x7f4af81d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4af8122530 0x7f4af81d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 -- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 msgr2=0x7f4af8121ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 0x7f4af8121ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient: _finish_hunting 0 2024-09-16T15:16:50.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 monclient: found mon.noname-e 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.684+0000 7f4afe36a640 1 -- 172.21.15.134:0/887548843 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4009b60 con 0x7f4adc003810 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4afe36a640 1 -- 172.21.15.134:0/887548843 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4af81216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4afe36a640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009e50 tx=0x7f4af4004ab0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4af4036e00 con 0x7f4adc003810 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: dump: 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:50.695 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af400f040 con 0x7f4adc003810 2024-09-16T15:16:50.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4af4041cd0 con 0x7f4adc003810 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4b005ce640 5 monclient: authenticate success, global_id 84734 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4b005ce640 10 monclient: _renew_subs 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4b005ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af80a8400 con 0x7f4adc003810 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4afcb67640 4 set_mon_vals no callback set 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.686+0000 7f4aeeffd640 10 monclient: dump: 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:50.697 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.687+0000 7f4b005ce640 10 monclient: _renew_subs 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.687+0000 7f4b005ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.687+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af8124a00 con 0x7f4adc003810 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4af4041860 con 0x7f4adc003810 2024-09-16T15:16:50.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4aeeffd640 1 --2- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 0x7f4adc03bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4b005ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f4af406ca30 con 0x7f4adc003810 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4b005ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4b005ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af8074e70 con 0x7f4adc003810 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4afdb69640 1 --2- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 0x7f4adc03bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.688+0000 7f4afdb69640 10 monclient: get_auth_request con 0x7f4adc0399e0 auth_method 0 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.691+0000 7f4afdb69640 1 --2- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 0x7f4adc03bea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8003fd0 tx=0x7f4ae8007070 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.693+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4af4041b10 con 0x7f4adc003810 2024-09-16T15:16:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.693+0000 7f4aeeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.693+0000 7f4aeeffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.921+0000 7f4b005ce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.921+0000 7f4b005ce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.921+0000 7f4b005ce640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.921+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4af8137600 con 0x7f4adc003810 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.922+0000 7f4aeeffd640 1 -- 172.21.15.134:0/887548843 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3178 (secure 0 0 0) 0x7f4af4048360 con 0x7f4adc003810 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.922+0000 7f4aeeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.922+0000 7f4aeeffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:50.923 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":138,"quorum":[2,3,5,6,7,8],"quorum_names":["b","g","h","d","i","e"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:16:50.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 msgr2=0x7f4adc03bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:50.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 0x7f4adc03bea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8003fd0 tx=0x7f4ae8007070 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 10 monclient: shutdown 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 msgr2=0x7f4af81216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.923+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009e50 tx=0x7f4af4004ab0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 shutdown_connections 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> v2:172.21.15.162:6800/980547263 conn(0x7f4adc0399e0 0x7f4adc03bea0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4af8122530 0x7f4af81d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4af8121c10 0x7f4af8121ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 --2- 172.21.15.134:0/887548843 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4adc003810 0x7f4af81216d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 >> 172.21.15.134:0/887548843 conn(0x7f4af80a61f0 msgr2=0x7f4af8135220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 shutdown_connections 2024-09-16T15:16:50.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:50.924+0000 7f4b005ce640 1 -- 172.21.15.134:0/887548843 wait complete. 2024-09-16T15:16:50.936 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-16T15:16:50.936 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-16T15:16:50.936 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.032+0000 7f3a1093a640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.032+0000 7f3a1093a640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.032+0000 7f3a1093a640 10 monclient: monmap: 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.033290+0000 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.033290+0000 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.034 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.032+0000 7f3a1093a640 1 Processor -- start 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.032+0000 7f3a1093a640 1 -- start start 2024-09-16T15:16:51.035 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient: init 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient: _add_conns ranks=[0,5,8,7,10,1,4,3,9,6,2] 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3a0c136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 0x7f3a0c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-h con 0x7f3a0c137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-e con 0x7f3a0c13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0c073250 con 0x7f3a0c136e40 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0a56e0 con 0x7f3a0c137220 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c09abc0 con 0x7f3a0c13c370 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0a59c640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f3a0c136e40 legacy=0x7f3a0c09c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a1093a640 10 monclient(hunting): authenticate will time out at 1747.622717s 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47516/0 (socket says 172.21.15.134:47516) 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/931900855 learned_addr learned my addr 172.21.15.134:0/931900855 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): get_auth_request con 0x7f3a0c13c370 auth_method 0 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.043 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.033+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 0x7f3a0c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a09d9b640 10 monclient(hunting): get_auth_request con 0x7f3a0c137220 auth_method 0 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a09d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a09d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a09d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_done global_id 84743 payload 274 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/931900855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 msgr2=0x7f3a0c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 0x7f3a0c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/931900855 >> v1:172.21.15.134:6789/0 conn(0x7f3a0c136e40 legacy=0x7f3a0c09c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 1 monclient: found mon.noname-e 2024-09-16T15:16:51.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/931900855 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c09ba50 con 0x7f3a0c13c370 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.034+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 0x7f3a0c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.036+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/931900855 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 msgr2=0x7f3a0c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.036+0000 7f3a0ad9d640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a0402a5b0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=82d5cc7fb974ad1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.036+0000 7f3a0959a640 1 -- 172.21.15.134:0/931900855 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a04007a70 con 0x7f3a0c13c370 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.036+0000 7f3a0959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: dump: 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 1 -- 172.21.15.134:0/931900855 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0402d050 con 0x7f3a0c13c370 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 5 monclient: authenticate success, global_id 84743 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 1 -- 172.21.15.134:0/931900855 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a04007bf0 con 0x7f3a0c13c370 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 4 set_mon_vals no callback set 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a0959a640 10 monclient: dump: 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.047 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.048 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 10 monclient: shutdown 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 -- 172.21.15.134:0/931900855 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 msgr2=0x7f3a0c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a0402a5b0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 -- 172.21.15.134:0/931900855 shutdown_connections 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3a0c13c370 0x7f3a0c13e760 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 --2- 172.21.15.134:0/931900855 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c137220 0x7f3a0c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 -- 172.21.15.134:0/931900855 >> 172.21.15.134:0/931900855 conn(0x7f3a0c0a6110 msgr2=0x7f3a0c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 -- 172.21.15.134:0/931900855 shutdown_connections 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.037+0000 7f3a1093a640 1 -- 172.21.15.134:0/931900855 wait complete. 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.038+0000 7f3a1093a640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.038+0000 7f3a1093a640 10 monclient: monmap: 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.049 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.038+0000 7f3a1093a640 1 Processor -- start 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.038+0000 7f3a1093a640 1 -- start start 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.038+0000 7f3a1093a640 10 monclient: init 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient: _add_conns ranks=[6,0,5,3,4,8,1,2,7] 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-d con 0x7f3a0c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a0c137220 0x7f3a0c135f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-a con 0x7f3a0c137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c13c370 0x7f3a0c136450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): picked mon.noname-h con 0x7f3a0c13c370 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c121130 con 0x7f3a0c137220 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c121360 con 0x7f3a0c13c370 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0a3a30 con 0x7f3a0c136e40 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a1093a640 10 monclient(hunting): authenticate will time out at 1747.628212s 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a09d9b640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a0c137220 msgr2=0x7f3a0c135f10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a09d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a0c137220 0x7f3a0c135f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33626/0 (socket says 172.21.15.134:33626) 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 -- 172.21.15.134:0/2662984353 learned_addr learned my addr 172.21.15.134:0/2662984353 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): get_auth_request con 0x7f3a0c136e40 auth_method 0 2024-09-16T15:16:51.052 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c13c370 0x7f3a0c136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): get_auth_request con 0x7f3a0c13c370 auth_method 0 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.053 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient(hunting): handle_auth_done global_id 94416 payload 824 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c13c370 msgr2=0x7f3a0c136450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3a0c13c370 0x7f3a0c136450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a0c137220 msgr2=0x7f3a0c135f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a0c137220 0x7f3a0c135f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 monclient: found mon.noname-d 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.039+0000 7f3a0a59c640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04009b30 con 0x7f3a0c136e40 2024-09-16T15:16:51.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.041+0000 7f3a0a59c640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 msgr2=0x7f3a0c0a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.041+0000 7f3a0a59c640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39f400b550 tx=0x7f39f400ba10 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.041+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:16:51.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.043+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f39f40092c0 con 0x7f3a0c136e40 2024-09-16T15:16:51.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.043+0000 7f3a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.043+0000 7f3a02ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: dump: 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.060 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f4003dd0 con 0x7f3a0c136e40 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 5 monclient: authenticate success, global_id 94416 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 10 monclient: _renew_subs 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f39f4010480 con 0x7f3a0c136e40 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0a7960 con 0x7f3a0c136e40 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 10 monclient: dump: 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a08d99640 4 set_mon_vals no callback set 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 10 monclient: _renew_subs 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c1225e0 con 0x7f3a0c136e40 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f39f4005070 con 0x7f3a0c136e40 2024-09-16T15:16:51.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.044+0000 7f3a1093a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a1093a640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 0x7f3a0c120040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c122770 con 0x7f3a0c11fc60 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 0x7f3a0c120040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient: get_auth_request con 0x7f3a0c11fc60 auth_method 0 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a02ffd640 1 --2- 172.21.15.134:0/2662984353 >> v2:172.21.15.162:6800/980547263 conn(0x7f39ec038880 0x7f39ec03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 3899+0+0 (secure 0 0 0) 0x7f39f4049e30 con 0x7f3a0c136e40 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_done global_id 114320 payload 824 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a09d9b640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a0c122350 con 0x7f3a0c11fc60 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a0ad9d640 1 --2- 172.21.15.134:0/2662984353 >> v2:172.21.15.162:6800/980547263 conn(0x7f39ec038880 0x7f39ec03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.045+0000 7f3a0ad9d640 10 monclient: get_auth_request con 0x7f39ec038880 auth_method 0 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a09d9b640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 msgr2=0x7f3a0c120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 0x7f3a0c120040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009100 tx=0x7f39fc00eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f39fc0044a0 con 0x7f3a0c11fc60 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 10 monclient: dump: 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.065 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f39fc004030 con 0x7f3a0c11fc60 2024-09-16T15:16:51.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 msgr2=0x7f3a0c120040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 0x7f3a0c120040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009100 tx=0x7f39fc00eba0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 msgr2=0x7f3a0c120040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.047+0000 7f3a02ffd640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c11fc60 0x7f3a0c120040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.048+0000 7f3a0ad9d640 1 --2- 172.21.15.134:0/2662984353 >> v2:172.21.15.162:6800/980547263 conn(0x7f39ec038880 0x7f39ec03ad40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a04009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.053+0000 7f3a1093a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.053+0000 7f3a1093a640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 0x7f3a0c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.053+0000 7f3a1093a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.053+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c071b20 con 0x7f3a0c120580 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 0x7f3a0c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient: get_auth_request con 0x7f3a0c120580 auth_method 0 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 10 monclient(hunting): handle_auth_done global_id 114329 payload 824 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.054+0000 7f3a09d9b640 1 -- 172.21.15.134:0/2662984353 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3a0c061ae0 con 0x7f3a0c120580 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a09d9b640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 msgr2=0x7f3a0c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a09d9b640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 0x7f3a0c09d4f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009100 tx=0x7f39fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f39fc029070 con 0x7f3a0c120580 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 10 monclient: dump: 2024-09-16T15:16:51.068 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.069 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9331 (secure 0 0 0) 0x7f39fc021040 con 0x7f3a0c120580 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 msgr2=0x7f3a0c09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 0x7f3a0c09d4f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009100 tx=0x7f39fc004730 comp rx=0 tx=0).stop 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 msgr2=0x7f3a0c09d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.056+0000 7f3a02ffd640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a0c120580 0x7f3a0c09d4f0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009100 tx=0x7f39fc004730 comp rx=0 tx=0).stop 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:16:51.070 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.071 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.072 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.073 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.074 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.075 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.076 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.077 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.078 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.079 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.080 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.081 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.082 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.083 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.084 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.085 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.086 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.087 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.089 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.090 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.091 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.092 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 >> v2:172.21.15.162:6800/980547263 conn(0x7f39ec038880 msgr2=0x7f39ec03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 --2- 172.21.15.134:0/2662984353 >> v2:172.21.15.162:6800/980547263 conn(0x7f39ec038880 0x7f39ec03ad40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e80 tx=0x7f3a04009000 comp rx=0 tx=0).stop 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 10 monclient: shutdown 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 msgr2=0x7f3a0c0a2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39f400b550 tx=0x7f39f400ba10 comp rx=0 tx=0).stop 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a0ad9d640 1 -- 172.21.15.134:0/2662984353 reap_dead start 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 shutdown_connections 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 --2- 172.21.15.134:0/2662984353 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3a0c136e40 0x7f3a0c0a2930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 >> 172.21.15.134:0/2662984353 conn(0x7f3a0c0a6110 msgr2=0x7f3a0c13b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 shutdown_connections 2024-09-16T15:16:51.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.057+0000 7f3a1093a640 1 -- 172.21.15.134:0/2662984353 wait complete. 2024-09-16T15:16:51.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.095 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-16T15:16:51.095 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:16:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.190+0000 7ff17cbe2640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.190+0000 7ff17cbe2640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.191+0000 7ff17cbe2640 10 monclient: monmap: 2024-09-16T15:16:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.192 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.191894+0000 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.191894+0000 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.193 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.191+0000 7ff17cbe2640 1 Processor -- start 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.191+0000 7ff17cbe2640 1 -- start start 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.191+0000 7ff17cbe2640 10 monclient: init 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient: _add_conns ranks=[8,10,3,9,0,1,6,7,2,5,4] 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-e con 0x7ff178136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 0x7ff178137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-f con 0x7ff178137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-g con 0x7ff178135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17809abc0 con 0x7ff178137220 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17809ba50 con 0x7ff178135dc0 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780a53c0 con 0x7ff178136e40 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17cbe2640 10 monclient(hunting): authenticate will time out at 1747.781306s 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47520/0 (socket says 172.21.15.134:47520) 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 -- 172.21.15.134:0/2798354384 learned_addr learned my addr 172.21.15.134:0/2798354384 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): get_auth_request con 0x7ff178136e40 auth_method 0 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff175d9b640 1 -- 172.21.15.134:0/2798354384 >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 msgr2=0x7ff178137600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff175d9b640 1 --2- 172.21.15.134:0/2798354384 >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 0x7ff178137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient(hunting): handle_auth_done global_id 84752 payload 274 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 -- 172.21.15.134:0/2798354384 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 msgr2=0x7ff178136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 --2- 172.21.15.134:0/2798354384 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 -- 172.21.15.134:0/2798354384 >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 msgr2=0x7ff178137600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 --2- 172.21.15.134:0/2798354384 >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 0x7ff178137600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 monclient: found mon.noname-e 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.192+0000 7ff17659c640 1 -- 172.21.15.134:0/2798354384 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff178073250 con 0x7ff178136e40 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17659c640 1 -- 172.21.15.134:0/2798354384 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 msgr2=0x7ff17809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17659c640 1 --2- 172.21.15.134:0/2798354384 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff1680098a0 tx=0x7ff16802a0e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=2955b267cd11361d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 1 -- 172.21.15.134:0/2798354384 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1680307b0 con 0x7ff178136e40 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: dump: 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.213 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 1 -- 172.21.15.134:0/2798354384 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168030930 con 0x7ff178136e40 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 1 -- 172.21.15.134:0/2798354384 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1680313a0 con 0x7ff178136e40 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17cbe2640 5 monclient: authenticate success, global_id 84752 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17cbe2640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17cbe2640 4 set_mon_vals no callback set 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.195+0000 7ff17559a640 10 monclient: dump: 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.215 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.216 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 10 monclient: shutdown 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 -- 172.21.15.134:0/2798354384 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 msgr2=0x7ff17809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/2798354384 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff1680098a0 tx=0x7ff16802a0e0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 -- 172.21.15.134:0/2798354384 shutdown_connections 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/2798354384 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/2798354384 >> v2:172.21.15.162:3300/0 conn(0x7ff178137220 0x7ff178137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/2798354384 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17809c150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 -- 172.21.15.134:0/2798354384 >> 172.21.15.134:0/2798354384 conn(0x7ff1780a6110 msgr2=0x7ff1780a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 -- 172.21.15.134:0/2798354384 shutdown_connections 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.196+0000 7ff17cbe2640 1 -- 172.21.15.134:0/2798354384 wait complete. 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.208+0000 7ff17cbe2640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.208+0000 7ff17cbe2640 10 monclient: monmap: 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.218 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.219 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.208+0000 7ff17cbe2640 1 Processor -- start 2024-09-16T15:16:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.208+0000 7ff17cbe2640 1 -- start start 2024-09-16T15:16:51.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.208+0000 7ff17cbe2640 10 monclient: init 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient: _add_conns ranks=[3,8,2,4,1,0,7,6,5] 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178132f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-g con 0x7ff178135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17808b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-e con 0x7ff178136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 0x7ff17808b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): picked mon.noname-b con 0x7ff178137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff178133440 con 0x7ff178137220 2024-09-16T15:16:51.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff178136390 con 0x7ff178135dc0 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff178080b80 con 0x7ff178136e40 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17cbe2640 10 monclient(hunting): authenticate will time out at 1747.798502s 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17808b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 0x7ff17808b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17808b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47536/0 (socket says 172.21.15.134:47536) 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): get_auth_request con 0x7ff178137220 auth_method 0 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 1 -- 172.21.15.134:0/1393644903 learned_addr learned my addr 172.21.15.134:0/1393644903 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): get_auth_request con 0x7ff178136e40 auth_method 0 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17659c640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178132f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17659c640 10 monclient(hunting): get_auth_request con 0x7ff178135dc0 auth_method 0 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff17659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient(hunting): handle_auth_done global_id 114338 payload 824 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 msgr2=0x7ff17808b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17808b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 msgr2=0x7ff178132f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff178135dc0 0x7ff178132f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 monclient: found mon.noname-b 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1680094d0 con 0x7ff178137220 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.209+0000 7ff175d9b640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff178136e40 0x7ff17808b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.212+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 msgr2=0x7ff17808b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.212+0000 7ff176d9d640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 0x7ff17808b680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff17000be20 tx=0x7ff17000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.212+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff17000ce80 con 0x7ff178137220 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: dump: 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.228 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 5 monclient: authenticate success, global_id 114338 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 10 monclient: _renew_subs 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1700127b0 con 0x7ff178137220 2024-09-16T15:16:51.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1780a8a10 con 0x7ff178137220 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff170012d90 con 0x7ff178137220 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff174d99640 4 set_mon_vals no callback set 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff166ffd640 10 monclient: dump: 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.230 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 10 monclient: _renew_subs 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.214+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1780808a0 con 0x7ff178137220 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17cbe2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 0x7ff178130140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff170012920 con 0x7ff178137220 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff178071b20 con 0x7ff17812fd60 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 0x7ff178130140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient: get_auth_request con 0x7ff17812fd60 auth_method 0 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff166ffd640 1 --2- 172.21.15.134:0/1393644903 >> v2:172.21.15.162:6800/980547263 conn(0x7ff1580388b0 0x7ff15803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 3899+0+0 (secure 0 0 0) 0x7ff170049ea0 con 0x7ff178137220 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 10 monclient(hunting): handle_auth_done global_id 94425 payload 824 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff17659c640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff178061ae0 con 0x7ff17812fd60 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff175d9b640 1 --2- 172.21.15.134:0/1393644903 >> v2:172.21.15.162:6800/980547263 conn(0x7ff1580388b0 0x7ff15803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.215+0000 7ff175d9b640 10 monclient: get_auth_request con 0x7ff1580388b0 auth_method 0 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff17659c640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 msgr2=0x7ff178130140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff17659c640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 0x7ff178130140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff168034000 tx=0x7ff1680046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1680307d0 con 0x7ff17812fd60 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff166ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.217+0000 7ff166ffd640 10 monclient: dump: 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff175d9b640 1 --2- 172.21.15.134:0/1393644903 >> v2:172.21.15.162:6800/980547263 conn(0x7ff1580388b0 0x7ff15803ad70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009980 tx=0x7ff16c003b90 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff168004460 con 0x7ff17812fd60 2024-09-16T15:16:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 msgr2=0x7ff178130140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 0x7ff178130140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff168034000 tx=0x7ff1680046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 msgr2=0x7ff178130140 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.218+0000 7ff166ffd640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff17812fd60 0x7ff178130140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17cbe2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 0x7ff17807cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17cbe2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff17807a680 con 0x7ff178130680 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 0x7ff17807cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient: get_auth_request con 0x7ff178130680 auth_method 0 2024-09-16T15:16:51.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 10 monclient(hunting): handle_auth_done global_id 94434 payload 824 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.224+0000 7ff17659c640 1 -- 172.21.15.134:0/1393644903 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff17809b490 con 0x7ff178130680 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.226+0000 7ff17659c640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 msgr2=0x7ff17807cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.226+0000 7ff17659c640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 0x7ff17807cbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff168000c00 tx=0x7ff168009240 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.226+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff168048cd0 con 0x7ff178130680 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.226+0000 7ff166ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 10 monclient: dump: 2024-09-16T15:16:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9329 (secure 0 0 0) 0x7ff168048e30 con 0x7ff178130680 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 msgr2=0x7ff17807cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 0x7ff17807cbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff168000c00 tx=0x7ff168009240 comp rx=0 tx=0).stop 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 msgr2=0x7ff17807cbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff166ffd640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff178130680 0x7ff17807cbb0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff168000c00 tx=0x7ff168009240 comp rx=0 tx=0).stop 2024-09-16T15:16:51.240 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.241 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.243 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.244 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.245 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.246 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.247 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.249 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.250 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.251 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.252 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.253 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.254 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.255 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.256 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.260 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.262 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:51.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.264 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.265 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.265 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 >> v2:172.21.15.162:6800/980547263 conn(0x7ff1580388b0 msgr2=0x7ff15803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/1393644903 >> v2:172.21.15.162:6800/980547263 conn(0x7ff1580388b0 0x7ff15803ad70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009980 tx=0x7ff16c003b90 comp rx=0 tx=0).stop 2024-09-16T15:16:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 10 monclient: shutdown 2024-09-16T15:16:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 msgr2=0x7ff17808b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 --2- 172.21.15.134:0/1393644903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff178137220 0x7ff17808b680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff17000be20 tx=0x7ff17000bf10 comp rx=0 tx=0).stop 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 reap_dead start 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff176d9d640 1 -- 172.21.15.134:0/1393644903 reap_dead start 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 shutdown_connections 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 >> 172.21.15.134:0/1393644903 conn(0x7ff1780a6110 msgr2=0x7ff1780a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 shutdown_connections 2024-09-16T15:16:51.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.227+0000 7ff17cbe2640 1 -- 172.21.15.134:0/1393644903 wait complete. 2024-09-16T15:16:51.267 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-16T15:16:51.268 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:16:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 10 monclient: monmap: 2024-09-16T15:16:51.365 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.365087+0000 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.365087+0000 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.366 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.367 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.367 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 1 Processor -- start 2024-09-16T15:16:51.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 1 -- start start 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.364+0000 7fc858be2640 10 monclient: init 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient: _add_conns ranks=[2,5,10,1,9,4,3,6,8,0,7] 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-b con 0x7fc854136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854137220 0x7fc854137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-h con 0x7fc854137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 0x7fc854136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-f con 0x7fc854135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85409abc0 con 0x7fc854135dc0 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85409ba50 con 0x7fc854136e40 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540a53c0 con 0x7fc854137220 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc858be2640 10 monclient(hunting): authenticate will time out at 1747.954408s 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:35662/0 (socket says 172.21.15.134:35662) 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 -- 172.21.15.134:0/1633070204 learned_addr learned my addr 172.21.15.134:0/1633070204 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc852d9d640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 msgr2=0x7fc854136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): get_auth_request con 0x7fc854136e40 auth_method 0 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc852d9d640 1 --2- 172.21.15.134:0/1633070204 >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 0x7fc854136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc851d9b640 1 -- 172.21.15.134:0/1633070204 DispatchQueue::entry inject delay of 0.002000 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc84bfff640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854137220 0x7fc854137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc84bfff640 10 monclient(hunting): get_auth_request con 0x7fc854137220 auth_method 0 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc84bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc84bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc84bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient(hunting): handle_auth_done global_id 114347 payload 274 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 -- 172.21.15.134:0/1633070204 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854137220 msgr2=0x7fc854137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854137220 0x7fc854137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 -- 172.21.15.134:0/1633070204 >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 msgr2=0x7fc854136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 --2- 172.21.15.134:0/1633070204 >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 0x7fc854136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 monclient: found mon.noname-b 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.365+0000 7fc85259c640 1 -- 172.21.15.134:0/1633070204 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854073250 con 0x7fc854136e40 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc85259c640 1 -- 172.21.15.134:0/1633070204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 msgr2=0x7fc85409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc85259c640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8440098a0 tx=0x7fc84402a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a230694ea7e44fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 1 -- 172.21.15.134:0/1633070204 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84402ae00 con 0x7fc854136e40 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: dump: 2024-09-16T15:16:51.375 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.376 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 1 -- 172.21.15.134:0/1633070204 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440307b0 con 0x7fc854136e40 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 1 -- 172.21.15.134:0/1633070204 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc844030d90 con 0x7fc854136e40 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc858be2640 5 monclient: authenticate success, global_id 114347 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc858be2640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc858be2640 4 set_mon_vals no callback set 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:16:51.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.368+0000 7fc851d9b640 10 monclient: dump: 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.378 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 10 monclient: shutdown 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 -- 172.21.15.134:0/1633070204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 msgr2=0x7fc85409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8440098a0 tx=0x7fc84402a0e0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 -- 172.21.15.134:0/1633070204 shutdown_connections 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 --2- 172.21.15.134:0/1633070204 >> v2:172.21.15.162:3300/0 conn(0x7fc854135dc0 0x7fc854136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854137220 0x7fc854137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 --2- 172.21.15.134:0/1633070204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc854136e40 0x7fc85409c150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 -- 172.21.15.134:0/1633070204 >> 172.21.15.134:0/1633070204 conn(0x7fc8540a6110 msgr2=0x7fc8540a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 -- 172.21.15.134:0/1633070204 shutdown_connections 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.369+0000 7fc858be2640 1 -- 172.21.15.134:0/1633070204 wait complete. 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.370+0000 7fc858be2640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.370+0000 7fc858be2640 10 monclient: monmap: 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.380 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.381 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.370+0000 7fc858be2640 1 Processor -- start 2024-09-16T15:16:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.370+0000 7fc858be2640 1 -- start start 2024-09-16T15:16:51.381 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.370+0000 7fc858be2640 10 monclient: init 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient: _add_conns ranks=[1,5,3,8,2,4,0,7,6] 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc854135dc0 0x7fc8541cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-f con 0x7fc854135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 0x7fc8541351c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-h con 0x7fc854136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 0x7fc854135700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): picked mon.noname-g con 0x7fc854137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854124010 con 0x7fc854135dc0 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc854135c40 con 0x7fc854137220 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540a8c90 con 0x7fc854136e40 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc858be2640 10 monclient(hunting): authenticate will time out at 1747.960231s 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc85259c640 1 -- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc854135dc0 msgr2=0x7fc8541cb6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc85259c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc854135dc0 0x7fc8541cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 0x7fc8541351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 0x7fc854135700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 0x7fc8541351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:32926/0 (socket says 172.21.15.134:32926) 2024-09-16T15:16:51.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 1 -- 172.21.15.134:0/4055746640 learned_addr learned my addr 172.21.15.134:0/4055746640 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): get_auth_request con 0x7fc854136e40 auth_method 0 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): get_auth_request con 0x7fc854137220 auth_method 0 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient(hunting): handle_auth_done global_id 84612 payload 824 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 msgr2=0x7fc8541351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 0x7fc8541351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc854135dc0 msgr2=0x7fc8541cb6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc854135dc0 0x7fc8541cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 monclient: found mon.noname-g 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc852d9d640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440094d0 con 0x7fc854137220 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.371+0000 7fc84bfff640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc854136e40 0x7fc8541351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc852d9d640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 msgr2=0x7fc854135700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc852d9d640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 0x7fc854135700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00bdc0 tx=0x7fc84c009eb0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84c006960 con 0x7fc854137220 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: dump: 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.393 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 5 monclient: authenticate success, global_id 84612 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 10 monclient: _renew_subs 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:16:51.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8541253a0 con 0x7fc854137220 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00f070 con 0x7fc854137220 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84c006b20 con 0x7fc854137220 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc84affd640 10 monclient: dump: 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.395 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 10 monclient: _renew_subs 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541cc3a0 con 0x7fc854137220 2024-09-16T15:16:51.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.374+0000 7fc85159a640 4 set_mon_vals no callback set 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc858be2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc858be2640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 0x7fc85402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541238a0 con 0x7fc854122450 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 0x7fc85402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient: get_auth_request con 0x7fc854122450 auth_method 0 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc84c0064f0 con 0x7fc854137220 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc84affd640 1 --2- 172.21.15.134:0/4055746640 >> v2:172.21.15.162:6800/980547263 conn(0x7fc834038880 0x7fc83403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 10 monclient(hunting): handle_auth_done global_id 84770 payload 824 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 3899+0+0 (secure 0 0 0) 0x7fc84c04abb0 con 0x7fc854137220 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc85259c640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc854061ae0 con 0x7fc854122450 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc84bfff640 1 --2- 172.21.15.134:0/4055746640 >> v2:172.21.15.162:6800/980547263 conn(0x7fc834038880 0x7fc83403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.375+0000 7fc84bfff640 10 monclient: get_auth_request con 0x7fc834038880 auth_method 0 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.377+0000 7fc85259c640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 msgr2=0x7fc85402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.377+0000 7fc85259c640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 0x7fc85402b260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc844034000 tx=0x7fc8440046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.377+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc844004250 con 0x7fc854122450 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 10 monclient: dump: 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.398 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.399 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc844030e80 con 0x7fc854122450 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84bfff640 1 --2- 172.21.15.134:0/4055746640 >> v2:172.21.15.162:6800/980547263 conn(0x7fc834038880 0x7fc83403ad40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc85404e230 tx=0x7fc83c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 msgr2=0x7fc85402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 0x7fc85402b260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc844034000 tx=0x7fc8440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 msgr2=0x7fc85402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.378+0000 7fc84affd640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122450 0x7fc85402b260 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc844034000 tx=0x7fc8440046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc858be2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc858be2640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 0x7fc85402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc858be2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc854071b20 con 0x7fc854122830 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 0x7fc85402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient: get_auth_request con 0x7fc854122830 auth_method 0 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 10 monclient(hunting): handle_auth_done global_id 84779 payload 824 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.384+0000 7fc85259c640 1 -- 172.21.15.134:0/4055746640 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc85407a680 con 0x7fc854122830 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.386+0000 7fc85259c640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 msgr2=0x7fc85402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.386+0000 7fc85259c640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 0x7fc85402b7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc844000c00 tx=0x7fc844009240 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc84404a070 con 0x7fc854122830 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 10 monclient: dump: 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.402 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9163 (secure 0 0 0) 0x7fc844045870 con 0x7fc854122830 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 msgr2=0x7fc85402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 0x7fc85402b7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc844000c00 tx=0x7fc844009240 comp rx=0 tx=0).stop 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 msgr2=0x7fc85402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc84affd640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc854122830 0x7fc85402b7a0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc844000c00 tx=0x7fc844009240 comp rx=0 tx=0).stop 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.403 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.404 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.405 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.406 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.407 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.408 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.409 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.410 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.412 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.413 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.413 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.414 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.414 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.415 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.416 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.420 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.422 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.424 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.426 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.427 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 >> v2:172.21.15.162:6800/980547263 conn(0x7fc834038880 msgr2=0x7fc83403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 --2- 172.21.15.134:0/4055746640 >> v2:172.21.15.162:6800/980547263 conn(0x7fc834038880 0x7fc83403ad40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc85404e230 tx=0x7fc83c008040 comp rx=0 tx=0).stop 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 10 monclient: shutdown 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 msgr2=0x7fc854135700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 0x7fc854135700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00bdc0 tx=0x7fc84c009eb0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc852d9d640 1 -- 172.21.15.134:0/4055746640 reap_dead start 2024-09-16T15:16:51.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 shutdown_connections 2024-09-16T15:16:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 --2- 172.21.15.134:0/4055746640 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc854137220 0x7fc854135700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 >> 172.21.15.134:0/4055746640 conn(0x7fc8540a6110 msgr2=0x7fc85409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 shutdown_connections 2024-09-16T15:16:51.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.387+0000 7fc858be2640 1 -- 172.21.15.134:0/4055746640 wait complete. 2024-09-16T15:16:51.430 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.430 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-09-16T15:16:51.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.526+0000 7fed2f0b1640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.526+0000 7fed2f0b1640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.526+0000 7fed2f0b1640 10 monclient: monmap: 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.527660+0000 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.527660+0000 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.528 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.529 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 Processor -- start 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 -- start start 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient: init 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient: _add_conns ranks=[8,10,6,9,2,0,4,5,1,7,3] 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-e con 0x7fed28136eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 0x7fed28137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-f con 0x7fed28137290 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-d con 0x7fed28133d10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed280a5560 con 0x7fed28137290 2024-09-16T15:16:51.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fed2809b240 con 0x7fed28133d10 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fed2809b520 con 0x7fed28136eb0 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.527+0000 7fed2f0b1640 10 monclient(hunting): authenticate will time out at 1748.116976s 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed27fff640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 msgr2=0x7fed28137690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed27fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 0x7fed28137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47576/0 (socket says 172.21.15.134:47576) 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 -- 172.21.15.134:0/2593169634 learned_addr learned my addr 172.21.15.134:0/2593169634 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33646/0 (socket says 172.21.15.134:33646) 2024-09-16T15:16:51.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): get_auth_request con 0x7fed28136eb0 auth_method 0 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): get_auth_request con 0x7fed28133d10 auth_method 0 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_done global_id 84788 payload 274 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 -- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 msgr2=0x7fed28134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 -- 172.21.15.134:0/2593169634 >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 msgr2=0x7fed28137690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/2593169634 >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 0x7fed28137690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 monclient: found mon.noname-e 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2ce4d640 1 -- 172.21.15.134:0/2593169634 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28073250 con 0x7fed28136eb0 2024-09-16T15:16:51.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.528+0000 7fed2d64e640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:51.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2ce4d640 1 -- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 msgr2=0x7fed2809bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009870 tx=0x7fed1c02a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=b64606aa05a9b34c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 1 -- 172.21.15.134:0/2593169634 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed1c02ae30 con 0x7fed28136eb0 2024-09-16T15:16:51.543 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: dump: 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.544 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 5 monclient: authenticate success, global_id 84788 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 1 -- 172.21.15.134:0/2593169634 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c0307b0 con 0x7fed28136eb0 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.545 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 4 set_mon_vals no callback set 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 1 -- 172.21.15.134:0/2593169634 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed1c030d90 con 0x7fed28136eb0 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed277fe640 10 monclient: dump: 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.546 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 10 monclient: shutdown 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 -- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 msgr2=0x7fed2809bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009870 tx=0x7fed1c02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 -- 172.21.15.134:0/2593169634 shutdown_connections 2024-09-16T15:16:51.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed28134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/2593169634 >> v2:172.21.15.162:3300/0 conn(0x7fed28137290 0x7fed28137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/2593169634 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed2809bf00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 -- 172.21.15.134:0/2593169634 >> 172.21.15.134:0/2593169634 conn(0x7fed280a99a0 msgr2=0x7fed280a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.531+0000 7fed2f0b1640 1 -- 172.21.15.134:0/2593169634 shutdown_connections 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 1 -- 172.21.15.134:0/2593169634 wait complete. 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 10 monclient: monmap: 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.548 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 1 Processor -- start 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 1 -- start start 2024-09-16T15:16:51.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.532+0000 7fed2f0b1640 10 monclient: init 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient: _add_conns ranks=[6,8,1,7,5,0,4,2,3] 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed281cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-d con 0x7fed28133d10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed28132fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-e con 0x7fed28136eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed28137290 0x7fed281334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): picked mon.noname-f con 0x7fed28137290 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed28121750 con 0x7fed28137290 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fed281cb9e0 con 0x7fed28133d10 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fed28136430 con 0x7fed28136eb0 2024-09-16T15:16:51.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2f0b1640 10 monclient(hunting): authenticate will time out at 1748.122564s 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed281cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed281cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33662/0 (socket says 172.21.15.134:33662) 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 -- 172.21.15.134:0/1155626300 learned_addr learned my addr 172.21.15.134:0/1155626300 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): get_auth_request con 0x7fed28133d10 auth_method 0 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed28132fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): get_auth_request con 0x7fed28136eb0 auth_method 0 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2d64e640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed28137290 msgr2=0x7fed281334f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2d64e640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed28137290 0x7fed281334f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient(hunting): handle_auth_done global_id 94452 payload 824 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 msgr2=0x7fed28132fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fed28136eb0 0x7fed28132fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed28137290 msgr2=0x7fed281334f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed28137290 0x7fed281334f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 monclient: found mon.noname-d 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.533+0000 7fed2ce4d640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20009a70 con 0x7fed28133d10 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2ce4d640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 msgr2=0x7fed281cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2ce4d640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed281cb4a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed1c030de0 con 0x7fed28133d10 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: dump: 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.553 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c003a80 con 0x7fed28133d10 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 5 monclient: authenticate success, global_id 94452 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 10 monclient: _renew_subs 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed1c003bf0 con 0x7fed28133d10 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c0094d0 con 0x7fed28133d10 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed257fa640 10 monclient: dump: 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.555 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed26ffd640 4 set_mon_vals no callback set 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 10 monclient: _renew_subs 2024-09-16T15:16:51.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.536+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed28121b20 con 0x7fed28133d10 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2f0b1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fed1c039070 con 0x7fed28133d10 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 0x7fed2811fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed28071b20 con 0x7fed28123620 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed257fa640 1 --2- 172.21.15.134:0/1155626300 >> v2:172.21.15.162:6800/980547263 conn(0x7fed0c038880 0x7fed0c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 3899+0+0 (secure 0 0 0) 0x7fed1c06aa00 con 0x7fed28133d10 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 0x7fed2811fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient: get_auth_request con 0x7fed28123620 auth_method 0 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2d64e640 1 --2- 172.21.15.134:0/1155626300 >> v2:172.21.15.162:6800/980547263 conn(0x7fed0c038880 0x7fed0c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed2d64e640 10 monclient: get_auth_request con 0x7fed0c038880 auth_method 0 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 10 monclient(hunting): handle_auth_done global_id 84621 payload 824 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.537+0000 7fed27fff640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed28123100 con 0x7fed28123620 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed2d64e640 1 --2- 172.21.15.134:0/1155626300 >> v2:172.21.15.162:6800/980547263 conn(0x7fed0c038880 0x7fed0c03ad40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed20009fd0 tx=0x7fed2000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed27fff640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 msgr2=0x7fed2811fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed27fff640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 0x7fed2811fbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed14009ac0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed14012d60 con 0x7fed28123620 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 10 monclient: dump: 2024-09-16T15:16:51.558 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.559 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fed140128f0 con 0x7fed28123620 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 msgr2=0x7fed2811fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 0x7fed2811fbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed14009ac0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 msgr2=0x7fed2811fbd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.540+0000 7fed257fa640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28123620 0x7fed2811fbd0 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed14009ac0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.546+0000 7fed2f0b1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.546+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 0x7fed28120520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.546+0000 7fed2f0b1640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.546+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed280a5cf0 con 0x7fed28120110 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 0x7fed28120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient: get_auth_request con 0x7fed28120110 auth_method 0 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 10 monclient(hunting): handle_auth_done global_id 84630 payload 824 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.547+0000 7fed27fff640 1 -- 172.21.15.134:0/1155626300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fed28061ae0 con 0x7fed28120110 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.549+0000 7fed27fff640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 msgr2=0x7fed28120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.549+0000 7fed27fff640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 0x7fed28120520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed140059c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed1401be90 con 0x7fed28120110 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 10 monclient: dump: 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.563 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9163 (secure 0 0 0) 0x7fed14005c40 con 0x7fed28120110 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 msgr2=0x7fed28120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 0x7fed28120520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed140059c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 msgr2=0x7fed28120520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.550+0000 7fed257fa640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed28120110 0x7fed28120520 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed280a0540 tx=0x7fed140059c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.565 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.566 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.567 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.568 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.569 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.571 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.576 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.579 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.580 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.581 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.582 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.583 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.584 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.585 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 >> v2:172.21.15.162:6800/980547263 conn(0x7fed0c038880 msgr2=0x7fed0c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/1155626300 >> v2:172.21.15.162:6800/980547263 conn(0x7fed0c038880 0x7fed0c03ad40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed20009fd0 tx=0x7fed2000b040 comp rx=0 tx=0).stop 2024-09-16T15:16:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 10 monclient: shutdown 2024-09-16T15:16:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 msgr2=0x7fed281cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 --2- 172.21.15.134:0/1155626300 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fed28133d10 0x7fed281cb4a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0097c0 tx=0x7fed1c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2d64e640 1 -- 172.21.15.134:0/1155626300 reap_dead start 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2d64e640 1 -- 172.21.15.134:0/1155626300 reap_dead start 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 shutdown_connections 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 >> 172.21.15.134:0/1155626300 conn(0x7fed280a99a0 msgr2=0x7fed2806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 shutdown_connections 2024-09-16T15:16:51.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.551+0000 7fed2f0b1640 1 -- 172.21.15.134:0/1155626300 wait complete. 2024-09-16T15:16:51.588 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.588 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-16T15:16:51.589 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:16:51.640 INFO:tasks.rados.rados.0.smithi162.stdout:3404: done (0 left) 2024-09-16T15:16:51.640 INFO:tasks.rados.rados.0.smithi162.stdout:3405: snap_remove snap 353 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.697+0000 7f36bf1bc640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.697+0000 7f36bf1bc640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.697+0000 7f36bf1bc640 10 monclient: monmap: 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.698596+0000 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.698596+0000 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.699 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.700 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 Processor -- start 2024-09-16T15:16:51.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 -- start start 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient: init 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient: _add_conns ranks=[10,8,7,0,6,4,2,9,3,5,1] 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 0x7f36b809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-f con 0x7f36b8136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-e con 0x7f36b8137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f36b8135dc0 0x7f36b8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-i con 0x7f36b8135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:51.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b809abc0 con 0x7f36b8136e40 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36b809ba50 con 0x7f36b8135dc0 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80a53c0 con 0x7f36b8137220 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36bf1bc640 10 monclient(hunting): authenticate will time out at 1748.287881s 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.698+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:47598/0 (socket says 172.21.15.134:47598) 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bcf58640 1 -- >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 msgr2=0x7f36b809c150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3300/0 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 -- 172.21.15.134:0/3943728325 learned_addr learned my addr 172.21.15.134:0/3943728325 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): get_auth_request con 0x7f36b8137220 auth_method 0 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bcf58640 1 --2- 172.21.15.134:0/3943728325 >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 0x7f36b809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bd759640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f36b8135dc0 0x7f36b8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bd759640 10 monclient(hunting): get_auth_request con 0x7f36b8135dc0 auth_method 0 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bd759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bd759640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36bd759640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient(hunting): handle_auth_done global_id 84806 payload 274 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 -- 172.21.15.134:0/3943728325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f36b8135dc0 msgr2=0x7f36b8136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f36b8135dc0 0x7f36b8136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 -- 172.21.15.134:0/3943728325 >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 msgr2=0x7f36b809c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 --2- 172.21.15.134:0/3943728325 >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 0x7f36b809c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 monclient: found mon.noname-e 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:16:51.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.699+0000 7f36b7fff640 1 -- 172.21.15.134:0/3943728325 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8073250 con 0x7f36b8137220 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b7fff640 1 -- 172.21.15.134:0/3943728325 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 msgr2=0x7f36b8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b7fff640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36a40098a0 tx=0x7f36a402a0e0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d36565fc7e8319d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 1 -- 172.21.15.134:0/3943728325 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36a402ae00 con 0x7f36b8137220 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:16:51.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: dump: 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.714 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 1 -- 172.21.15.134:0/3943728325 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a40307b0 con 0x7f36b8137220 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 5 monclient: authenticate success, global_id 84806 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 4 set_mon_vals no callback set 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 1 -- 172.21.15.134:0/3943728325 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36a4030d90 con 0x7f36b8137220 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36b77fe640 10 monclient: dump: 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.716 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 10 monclient: shutdown 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 -- 172.21.15.134:0/3943728325 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 msgr2=0x7f36b8137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f36a40098a0 tx=0x7f36a402a0e0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 -- 172.21.15.134:0/3943728325 shutdown_connections 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f36b8135dc0 0x7f36b8136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/3943728325 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f36b8137220 0x7f36b8137600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/3943728325 >> v2:172.21.15.162:3300/0 conn(0x7f36b8136e40 0x7f36b809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 -- 172.21.15.134:0/3943728325 >> 172.21.15.134:0/3943728325 conn(0x7f36b80a6110 msgr2=0x7f36b80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 -- 172.21.15.134:0/3943728325 shutdown_connections 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.702+0000 7f36bf1bc640 1 -- 172.21.15.134:0/3943728325 wait complete. 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.703+0000 7f36bf1bc640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.703+0000 7f36bf1bc640 10 monclient: monmap: 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.718 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.703+0000 7f36bf1bc640 1 Processor -- start 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.703+0000 7f36bf1bc640 1 -- start start 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.703+0000 7f36bf1bc640 10 monclient: init 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient: _add_conns ranks=[6,5,3,1,0,2,4,7,8] 2024-09-16T15:16:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 0x7f36b81cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-d con 0x7f36b8135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b8136e40 0x7f36b8134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-h con 0x7f36b8136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36b8137220 0x7f36b8135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): picked mon.noname-g con 0x7f36b8137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81cb760 con 0x7f36b8137220 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8123cc0 con 0x7f36b8136e40 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8123570 con 0x7f36b8135dc0 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bf1bc640 10 monclient(hunting): authenticate will time out at 1748.293400s 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 0x7f36b81cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 0x7f36b81cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33674/0 (socket says 172.21.15.134:33674) 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 -- 172.21.15.134:0/1326646746 learned_addr learned my addr 172.21.15.134:0/1326646746 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): get_auth_request con 0x7f36b8135dc0 auth_method 0 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b8136e40 0x7f36b8134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36b8137220 0x7f36b8135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): get_auth_request con 0x7f36b8136e40 auth_method 0 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): get_auth_request con 0x7f36b8137220 auth_method 0 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bd759640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient(hunting): handle_auth_done global_id 94461 payload 824 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b8136e40 msgr2=0x7f36b8134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b8136e40 0x7f36b8134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36b8137220 msgr2=0x7f36b8135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36b8137220 0x7f36b8135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 monclient: found mon.noname-d 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.704+0000 7f36bcf58640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40094d0 con 0x7f36b8135dc0 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bcf58640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 msgr2=0x7f36b81cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bcf58640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 0x7f36b81cb220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b580 tx=0x7f36ac00ba40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ac003dd0 con 0x7f36b8135dc0 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: dump: 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.724 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 5 monclient: authenticate success, global_id 94461 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac0054e0 con 0x7f36b8135dc0 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 10 monclient: _renew_subs 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b80a8a30 con 0x7f36b8135dc0 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36ac005650 con 0x7f36b8135dc0 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b6ffd640 4 set_mon_vals no callback set 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36b57fa640 10 monclient: dump: 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.725 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 10 monclient: _renew_subs 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.707+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81363b0 con 0x7f36b8135dc0 2024-09-16T15:16:51.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bf1bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f36ac005070 con 0x7f36b8135dc0 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 0x7f36b81221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8071b20 con 0x7f36b81255c0 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b57fa640 1 --2- 172.21.15.134:0/1326646746 >> v2:172.21.15.162:6800/980547263 conn(0x7f369c0388b0 0x7f369c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 0x7f36b81221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient: get_auth_request con 0x7f36b81255c0 auth_method 0 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(725..725 src has 1..725) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f36ac049fe0 con 0x7f36b8135dc0 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bd759640 1 --2- 172.21.15.134:0/1326646746 >> v2:172.21.15.162:6800/980547263 conn(0x7f369c0388b0 0x7f369c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36bd759640 10 monclient: get_auth_request con 0x7f369c0388b0 auth_method 0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 10 monclient(hunting): handle_auth_done global_id 45248 payload 824 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.708+0000 7f36b7fff640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36b8061ae0 con 0x7f36b81255c0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.710+0000 7f36b7fff640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 msgr2=0x7f36b81221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.710+0000 7f36b7fff640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 0x7f36b81221f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a40046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.710+0000 7f36bd759640 1 --2- 172.21.15.134:0/1326646746 >> v2:172.21.15.162:6800/980547263 conn(0x7f369c0388b0 0x7f369c03ad70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36b8073780 tx=0x7f36b0009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36a4030d40 con 0x7f36b81255c0 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 10 monclient: dump: 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.728 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.729 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f36a4003610 con 0x7f36b81255c0 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 msgr2=0x7f36b81221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 0x7f36b81221f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 msgr2=0x7f36b81221f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.730 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.711+0000 7f36b57fa640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b81255c0 0x7f36b81221f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36bf1bc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 0x7f36b802b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36bf1bc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36b807a680 con 0x7f36b802b260 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 0x7f36b802b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient: get_auth_request con 0x7f36b802b260 auth_method 0 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 10 monclient(hunting): handle_auth_done global_id 45257 payload 824 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.717+0000 7f36b7fff640 1 -- 172.21.15.134:0/1326646746 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f36b809b490 con 0x7f36b802b260 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b7fff640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 msgr2=0x7f36b802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b7fff640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 0x7f36b802b670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f36a40097c0 tx=0x7f36a4008030 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f36a404d070 con 0x7f36b802b260 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 10 monclient: dump: 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.733 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.734 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9495 (secure 0 0 0) 0x7f36a403dbb0 con 0x7f36b802b260 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 msgr2=0x7f36b802b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 0x7f36b802b670 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f36a40097c0 tx=0x7f36a4008030 comp rx=0 tx=0).stop 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 msgr2=0x7f36b802b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.720+0000 7f36b57fa640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f36b802b260 0x7f36b802b670 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f36a40097c0 tx=0x7f36a4008030 comp rx=0 tx=0).stop 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.735 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.736 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.737 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.738 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.739 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.741 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.742 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.743 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.744 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.745 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.746 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.747 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.748 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.749 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.750 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.751 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.752 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.753 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.754 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:51.755 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.756 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.757 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.757 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 >> v2:172.21.15.162:6800/980547263 conn(0x7f369c0388b0 msgr2=0x7f369c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/1326646746 >> v2:172.21.15.162:6800/980547263 conn(0x7f369c0388b0 0x7f369c03ad70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36b8073780 tx=0x7f36b0009820 comp rx=0 tx=0).stop 2024-09-16T15:16:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 10 monclient: shutdown 2024-09-16T15:16:51.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 msgr2=0x7f36b81cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 --2- 172.21.15.134:0/1326646746 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f36b8135dc0 0x7f36b81cb220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00b580 tx=0x7f36ac00ba40 comp rx=0 tx=0).stop 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bd759640 1 -- 172.21.15.134:0/1326646746 reap_dead start 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bd759640 1 -- 172.21.15.134:0/1326646746 reap_dead start 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 shutdown_connections 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 >> 172.21.15.134:0/1326646746 conn(0x7f36b80a6110 msgr2=0x7f36b809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 shutdown_connections 2024-09-16T15:16:51.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.721+0000 7f36bf1bc640 1 -- 172.21.15.134:0/1326646746 wait complete. 2024-09-16T15:16:51.759 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.759 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-16T15:16:51.760 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.856+0000 7f56e35ed640 10 monclient: get_monmap_and_config 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.856+0000 7f56e35ed640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.856+0000 7f56e35ed640 10 monclient: monmap: 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:16:51.857487+0000 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:16:51.857487+0000 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:16:51.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 Processor -- start 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 -- start start 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient: init 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient: _add_conns ranks=[6,7,5,8,2,10,0,1,3,4,9] 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-d con 0x7f56dc136eb0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc137290 0x7f56dc137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-i con 0x7f56dc137290 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:16:51.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f56dc133d10 0x7f56dc134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-h con 0x7f56dc133d10 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0a5560 con 0x7f56dc133d10 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc09b240 con 0x7f56dc136eb0 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc09b520 con 0x7f56dc137290 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e35ed640 10 monclient(hunting): authenticate will time out at 1748.446823s 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33676/0 (socket says 172.21.15.134:33676) 2024-09-16T15:16:51.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 1 -- 172.21.15.134:0/2086594056 learned_addr learned my addr 172.21.15.134:0/2086594056 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 10 monclient(hunting): get_auth_request con 0x7f56dc136eb0 auth_method 0 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.857+0000 7f56e1389640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e0b88640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc137290 0x7f56dc137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1b8a640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f56dc133d10 0x7f56dc134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e0b88640 10 monclient(hunting): get_auth_request con 0x7f56dc137290 auth_method 0 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e0b88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e0b88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e0b88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1b8a640 10 monclient(hunting): get_auth_request con 0x7f56dc133d10 auth_method 0 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1b8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1b8a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1b8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient(hunting): handle_auth_done global_id 94470 payload 274 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 -- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc137290 msgr2=0x7f56dc137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc137290 0x7f56dc137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 -- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f56dc133d10 msgr2=0x7f56dc134190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f56dc133d10 0x7f56dc134190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 monclient: found mon.noname-d 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.858+0000 7f56e1389640 1 -- 172.21.15.134:0/2086594056 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc073250 con 0x7f56dc136eb0 2024-09-16T15:16:51.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.860+0000 7f56e1389640 1 -- 172.21.15.134:0/2086594056 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 msgr2=0x7f56dc09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.860+0000 7f56e1389640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f56d80098a0 tx=0x7f56d802a080 comp rx=0 tx=0).ready entity=mon.6 client_cookie=844e4da3d05f6f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 1 -- 172.21.15.134:0/2086594056 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56d802ade0 con 0x7f56dc136eb0 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: dump: 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.873 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 1 -- 172.21.15.134:0/2086594056 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80307b0 con 0x7f56dc136eb0 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 5 monclient: authenticate success, global_id 94470 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:16:51.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 10 monclient: get_monmap_and_config success 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 1 -- 172.21.15.134:0/2086594056 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56d8030dd0 con 0x7f56dc136eb0 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 4 set_mon_vals no callback set 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56d3fff640 10 monclient: dump: 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.875 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 10 monclient: shutdown 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 -- 172.21.15.134:0/2086594056 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 msgr2=0x7f56dc09bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f56d80098a0 tx=0x7f56d802a080 comp rx=0 tx=0).stop 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 -- 172.21.15.134:0/2086594056 shutdown_connections 2024-09-16T15:16:51.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f56dc133d10 0x7f56dc134190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc137290 0x7f56dc137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 --2- 172.21.15.134:0/2086594056 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc136eb0 0x7f56dc09bf00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 -- 172.21.15.134:0/2086594056 >> 172.21.15.134:0/2086594056 conn(0x7f56dc0a99a0 msgr2=0x7f56dc0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 -- 172.21.15.134:0/2086594056 shutdown_connections 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.861+0000 7f56e35ed640 1 -- 172.21.15.134:0/2086594056 wait complete. 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.862+0000 7f56e35ed640 10 monclient: build_initial_monmap 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.862+0000 7f56e35ed640 10 monclient: monmap: 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:16:51.877 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.862+0000 7f56e35ed640 1 Processor -- start 2024-09-16T15:16:51.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.862+0000 7f56e35ed640 1 -- start start 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.862+0000 7f56e35ed640 10 monclient: init 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient: _reopen_session rank -1 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient: _add_conns ranks=[6,8,3,1,2,7,4,5,0] 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 0x7f56dc1cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-d con 0x7f56dc133d10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f56dc136eb0 0x7f56dc1cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-e con 0x7f56dc136eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56dc137290 0x7f56dc0a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): picked mon.noname-g con 0x7f56dc137290 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0a7710 con 0x7f56dc137290 2024-09-16T15:16:51.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc120650 con 0x7f56dc133d10 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0a2410 con 0x7f56dc136eb0 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): _renew_subs 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e35ed640 10 monclient(hunting): authenticate will time out at 1748.452109s 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 0x7f56dc1cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 0x7f56dc1cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:33686/0 (socket says 172.21.15.134:33686) 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 -- 172.21.15.134:0/3141576509 learned_addr learned my addr 172.21.15.134:0/3141576509 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): get_auth_request con 0x7f56dc133d10 auth_method 0 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f56dc136eb0 0x7f56dc1cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): get_auth_request con 0x7f56dc136eb0 auth_method 0 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56dc137290 0x7f56dc0a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): get_auth_request con 0x7f56dc137290 auth_method 0 2024-09-16T15:16:51.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1b8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient(hunting): handle_auth_done global_id 94479 payload 824 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f56dc136eb0 msgr2=0x7f56dc1cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f56dc136eb0 0x7f56dc1cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56dc137290 msgr2=0x7f56dc0a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56dc137290 0x7f56dc0a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient: _finish_hunting 0 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 monclient: found mon.noname-d 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:16:51.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e1389640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80094d0 con 0x7f56dc133d10 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.863+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f56dc136eb0 0x7f56dc1cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e1389640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 msgr2=0x7f56dc1cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e1389640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 0x7f56dc1cb5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56d80090b0 tx=0x7f56d80046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56d80044b0 con 0x7f56dc133d10 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: dump: 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.883 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: _finish_auth 0 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d8030d40 con 0x7f56dc133d10 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e35ed640 5 monclient: authenticate success, global_id 94479 2024-09-16T15:16:51.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e35ed640 10 monclient: _renew_subs 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e35ed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc136530 con 0x7f56dc133d10 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.865+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56d8032440 con 0x7f56dc133d10 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56d1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56d1ffb640 10 monclient: dump: 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.885 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56d37fe640 4 set_mon_vals no callback set 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 10 monclient: _renew_subs 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc120370 con 0x7f56dc133d10 2024-09-16T15:16:51.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f56d8039040 con 0x7f56dc133d10 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 0x7f56dc0a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.866+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc04e300 con 0x7f56dc1cc210 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56d1ffb640 1 --2- 172.21.15.134:0/3141576509 >> v2:172.21.15.162:6800/980547263 conn(0x7f56c00388b0 0x7f56c003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(725..725 src has 1..725) v4 ==== 3943+0+0 (secure 0 0 0) 0x7f56d806a8b0 con 0x7f56dc133d10 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 0x7f56dc0a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient: get_auth_request con 0x7f56dc1cc210 auth_method 0 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e1b8a640 1 --2- 172.21.15.134:0/3141576509 >> v2:172.21.15.162:6800/980547263 conn(0x7f56c00388b0 0x7f56c003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e1b8a640 10 monclient: get_auth_request con 0x7f56c00388b0 auth_method 0 2024-09-16T15:16:51.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 10 monclient(hunting): handle_auth_done global_id 114262 payload 824 2024-09-16T15:16:51.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.867+0000 7f56e0b88640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56dc061ae0 con 0x7f56dc1cc210 2024-09-16T15:16:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56e1b8a640 1 --2- 172.21.15.134:0/3141576509 >> v2:172.21.15.162:6800/980547263 conn(0x7f56c00388b0 0x7f56c003ad70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0a2010 tx=0x7f56d4009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56e0b88640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 msgr2=0x7f56dc0a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 0x7f56dc0a0ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f56cc010b60 tx=0x7f56cc00e5a0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56cc010e60 con 0x7f56dc1cc210 2024-09-16T15:16:51.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56d1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.869+0000 7f56d1ffb640 10 monclient: dump: 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.891 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f56cc01d450 con 0x7f56dc1cc210 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 msgr2=0x7f56dc0a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 0x7f56dc0a0ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f56cc010b60 tx=0x7f56cc00e5a0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 msgr2=0x7f56dc0a0ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.870+0000 7f56d1ffb640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc1cc210 0x7f56dc0a0ec0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f56cc010b60 tx=0x7f56cc00e5a0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e35ed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:16:51.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e35ed640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 0x7f56dc134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e35ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc073880 con 0x7f56dc0a1400 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 0x7f56dc134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient: get_auth_request con 0x7f56dc0a1400 auth_method 0 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 10 monclient(hunting): handle_auth_done global_id 114271 payload 824 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.876+0000 7f56e0b88640 1 -- 172.21.15.134:0/3141576509 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56dc0a5cf0 con 0x7f56dc0a1400 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56e0b88640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 msgr2=0x7f56dc134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:16:51.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56e0b88640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 0x7f56dc134190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56cc01a000 tx=0x7f56cc0046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f56cc02b070 con 0x7f56dc0a1400 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 10 monclient: dump: 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:16:51.894 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9163 (secure 0 0 0) 0x7f56cc023040 con 0x7f56dc0a1400 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 msgr2=0x7f56dc134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 0x7f56dc134190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56cc01a000 tx=0x7f56cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 msgr2=0x7f56dc134190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:16:51.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.879+0000 7f56d1ffb640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f56dc0a1400 0x7f56dc134190 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56cc01a000 tx=0x7f56cc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 138, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:16:51.896 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.897 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.898 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:16:51.899 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:16:51.900 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:16:51.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:16:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:16:51.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:16:51.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.910 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:16:51.913 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.915 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:16:51.916 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:16:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 >> v2:172.21.15.162:6800/980547263 conn(0x7f56c00388b0 msgr2=0x7f56c003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 --2- 172.21.15.134:0/3141576509 >> v2:172.21.15.162:6800/980547263 conn(0x7f56c00388b0 0x7f56c003ad70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0a2010 tx=0x7f56d4009820 comp rx=0 tx=0).stop 2024-09-16T15:16:51.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 10 monclient: shutdown 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 msgr2=0x7f56dc1cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 --2- 172.21.15.134:0/3141576509 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f56dc133d10 0x7f56dc1cb5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56d80090b0 tx=0x7f56d80046c0 comp rx=0 tx=0).stop 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e1b8a640 1 -- 172.21.15.134:0/3141576509 reap_dead start 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e1b8a640 1 -- 172.21.15.134:0/3141576509 reap_dead start 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 shutdown_connections 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 >> 172.21.15.134:0/3141576509 conn(0x7f56dc0a99a0 msgr2=0x7f56dc06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 shutdown_connections 2024-09-16T15:16:51.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:16:51.880+0000 7f56e35ed640 1 -- 172.21.15.134:0/3141576509 wait complete. 2024-09-16T15:16:51.918 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-16T15:16:51.918 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:16:52.656 INFO:tasks.rados.rados.0.smithi162.stdout:3405: done (0 left) 2024-09-16T15:16:52.656 INFO:tasks.rados.rados.0.smithi162.stdout:3406: delete oid 4 current snap is 357 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:3406: done (0 left) 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:3407: rollback oid 44 current snap is 357 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 44 to 287 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:3408: read oid 47 snap -1 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:3408: expect (ObjNum 1158 snap 345 seq_num 1158) 2024-09-16T15:16:52.662 INFO:tasks.rados.rados.0.smithi162.stdout:3409: snap_remove snap 354 2024-09-16T15:16:53.663 INFO:tasks.rados.rados.0.smithi162.stdout:3409: done (2 left) 2024-09-16T15:16:53.663 INFO:tasks.rados.rados.0.smithi162.stdout:3410: copy_from oid 48 from oid 30 current snap is 357 2024-09-16T15:16:53.663 INFO:tasks.rados.rados.0.smithi162.stdout:3407: finishing rollback tid 0 to smithi16234619-44 2024-09-16T15:16:53.663 INFO:tasks.rados.rados.0.smithi162.stdout:3408: expect (ObjNum 1158 snap 345 seq_num 1158) 2024-09-16T15:16:53.778 INFO:tasks.rados.rados.0.smithi162.stdout:3407: finishing rollback tid 1 to smithi16234619-44 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1817 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3410: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3410: finishing copy_from to smithi16234619-48 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1413 (ObjNum 1136 snap 340 seq_num 1136) dirty exists 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3407: done (2 left) 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3408: done (1 left) 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3410: done (0 left) 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3411: read oid 36 snap -1 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3411: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3412: read oid 41 snap -1 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3412: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3413: write oid 50 current snap is 357 2024-09-16T15:16:53.779 INFO:tasks.rados.rados.0.smithi162.stdout:3413: seq_num 1175 ranges {548225=735937,1925048=596064,2950840=547321} 2024-09-16T15:16:53.785 INFO:tasks.rados.rados.0.smithi162.stdout:3413: writing smithi16234619-50 from 548225 to 1284162 tid 1 2024-09-16T15:16:53.792 INFO:tasks.rados.rados.0.smithi162.stdout:3413: writing smithi16234619-50 from 1925048 to 2521112 tid 2 2024-09-16T15:16:53.797 INFO:tasks.rados.rados.0.smithi162.stdout:3413: writing smithi16234619-50 from 2950840 to 3498161 tid 3 2024-09-16T15:16:53.797 INFO:tasks.rados.rados.0.smithi162.stdout:3414: rollback oid 29 current snap is 357 2024-09-16T15:16:53.797 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 352 2024-09-16T15:16:53.797 INFO:tasks.rados.rados.0.smithi162.stdout:3411: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:16:53.938 INFO:tasks.rados.rados.0.smithi162.stdout:3412: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3414: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3414: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3414: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1883 (ObjNum 1157 snap 345 seq_num 1157) dirty exists 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3411: done (3 left) 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3412: done (2 left) 2024-09-16T15:16:54.058 INFO:tasks.rados.rados.0.smithi162.stdout:3414: done (1 left) 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3415: copy_from oid 5 from oid 39 current snap is 357 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: finishing write tid 1 to smithi16234619-50 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: finishing write tid 2 to smithi16234619-50 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: finishing write tid 3 to smithi16234619-50 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: finishing write tid 4 to smithi16234619-50 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: finishing write tid 5 to smithi16234619-50 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: oid 50 updating version 0 to 1736 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: oid 50 version 1736 is already newer than 1735 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: oid 50 updating version 1736 to 1738 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: oid 50 version 1738 is already newer than 1737 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1738 (ObjNum 1175 snap 357 seq_num 1175) dirty exists 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: left oid 50 (ObjNum 1175 snap 357 seq_num 1175) 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3413: done (1 left) 2024-09-16T15:16:54.059 INFO:tasks.rados.rados.0.smithi162.stdout:3416: snap_remove snap 327 2024-09-16T15:16:55.848 INFO:tasks.rados.rados.0.smithi162.stdout:3416: done (1 left) 2024-09-16T15:16:55.848 INFO:tasks.rados.rados.0.smithi162.stdout:3417: snap_create 2024-09-16T15:16:55.848 INFO:tasks.rados.rados.0.smithi162.stdout:3415: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:16:55.848 INFO:tasks.rados.rados.0.smithi162.stdout:3415: finishing copy_from to smithi16234619-5 2024-09-16T15:16:55.848 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1831 (ObjNum 1162 snap 346 seq_num 1162) dirty exists 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3415: done (1 left) 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3417: done (0 left) 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3418: read oid 33 snap -1 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3418: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3419: rollback oid 7 current snap is 358 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 345 2024-09-16T15:16:56.869 INFO:tasks.rados.rados.0.smithi162.stdout:3420: snap_create 2024-09-16T15:16:56.873 INFO:tasks.rados.rados.0.smithi162.stdout:3419: finishing rollback tid 0 to smithi16234619-7 2024-09-16T15:16:56.873 INFO:tasks.rados.rados.0.smithi162.stdout:3419: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:16:56.876 INFO:tasks.rados.rados.0.smithi162.stdout:3419: finishing rollback tid 2 to smithi16234619-7 2024-09-16T15:16:56.876 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1498 (ObjNum 1155 snap 345 seq_num 1155) dirty exists 2024-09-16T15:16:56.877 INFO:tasks.rados.rados.0.smithi162.stdout:3418: expect (ObjNum 1119 snap 334 seq_num 1119) 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:3418: done (2 left) 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:3419: done (1 left) 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:3420: done (0 left) 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:3421: rollback oid 35 current snap is 359 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 340 2024-09-16T15:16:57.886 INFO:tasks.rados.rados.0.smithi162.stdout:3422: delete oid 11 current snap is 359 2024-09-16T15:16:57.889 INFO:tasks.rados.rados.0.smithi162.stdout:3422: done (1 left) 2024-09-16T15:16:57.889 INFO:tasks.rados.rados.0.smithi162.stdout:3423: snap_remove snap 348 2024-09-16T15:16:58.893 INFO:tasks.rados.rados.0.smithi162.stdout:3423: done (1 left) 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3424: read oid 43 snap -1 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3424: expect deleted 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3425: copy_from oid 5 from oid 41 current snap is 359 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3421: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3421: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3421: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1820 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:16:58.894 INFO:tasks.rados.rados.0.smithi162.stdout:3421: done (2 left) 2024-09-16T15:16:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:3426: copy_from oid 47 from oid 24 current snap is 359 2024-09-16T15:16:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:3427: read oid 17 snap -1 2024-09-16T15:16:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:3427: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:16:58.895 INFO:tasks.rados.rados.0.smithi162.stdout:3428: snap_remove snap 300 2024-09-16T15:16:59.623 INFO:tasks.rados.rados.0.smithi162.stdout:3428: done (4 left) 2024-09-16T15:16:59.624 INFO:tasks.rados.rados.0.smithi162.stdout:3429: copy_from oid 8 from oid 49 current snap is 359 2024-09-16T15:16:59.624 INFO:tasks.rados.rados.0.smithi162.stdout:3427: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:16:59.725 INFO:tasks.rados.rados.0.smithi162.stdout:3426: finishing copy_from to smithi16234619-47 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3426: got expected ENOENT (src dne) 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3426: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3425: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3425: finishing copy_from to smithi16234619-5 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1480 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3424: done (4 left) 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3425: done (3 left) 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3426: done (2 left) 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3427: done (1 left) 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3430: rollback oid 27 current snap is 359 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 27 to 338 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3431: write (excl) oid 41 current snap is 359 2024-09-16T15:16:59.726 INFO:tasks.rados.rados.0.smithi162.stdout:3431: seq_num 1177 ranges {581839=486435,1642012=599733,3005686=569865} 2024-09-16T15:16:59.731 INFO:tasks.rados.rados.0.smithi162.stdout:3431: writing smithi16234619-41 from 581839 to 1068274 tid 1 2024-09-16T15:16:59.737 INFO:tasks.rados.rados.0.smithi162.stdout:3431: writing smithi16234619-41 from 1642012 to 2241745 tid 2 2024-09-16T15:16:59.744 INFO:tasks.rados.rados.0.smithi162.stdout:3431: writing smithi16234619-41 from 3005686 to 3575551 tid 3 2024-09-16T15:16:59.744 INFO:tasks.rados.rados.0.smithi162.stdout:3432: write (excl) oid 38 current snap is 359 2024-09-16T15:16:59.744 INFO:tasks.rados.rados.0.smithi162.stdout:3432: seq_num 1178 ranges {598646=561684,1700494=459744,2633109=781137} 2024-09-16T15:16:59.750 INFO:tasks.rados.rados.0.smithi162.stdout:3432: writing smithi16234619-38 from 598646 to 1160330 tid 1 2024-09-16T15:16:59.755 INFO:tasks.rados.rados.0.smithi162.stdout:3432: writing smithi16234619-38 from 1700494 to 2160238 tid 2 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:3432: writing smithi16234619-38 from 2633109 to 3414246 tid 3 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:3433: write oid 26 current snap is 359 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:3430: finishing rollback tid 1 to smithi16234619-27 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:3430: finishing rollback tid 2 to smithi16234619-27 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 1745 (ObjNum 655 snap 169 seq_num 655) dirty exists 2024-09-16T15:16:59.763 INFO:tasks.rados.rados.0.smithi162.stdout:3433: seq_num 1179 ranges {615453=753286,2158976=603402,3460532=354921} 2024-09-16T15:16:59.770 INFO:tasks.rados.rados.0.smithi162.stdout:3433: writing smithi16234619-26 from 615453 to 1368739 tid 1 2024-09-16T15:16:59.778 INFO:tasks.rados.rados.0.smithi162.stdout:3433: writing smithi16234619-26 from 2158976 to 2762378 tid 2 2024-09-16T15:16:59.784 INFO:tasks.rados.rados.0.smithi162.stdout:3433: writing smithi16234619-26 from 3460532 to 3815453 tid 3 2024-09-16T15:16:59.784 INFO:tasks.rados.rados.0.smithi162.stdout:3430: done (4 left) 2024-09-16T15:16:59.784 INFO:tasks.rados.rados.0.smithi162.stdout:3434: write (excl) oid 19 current snap is 359 2024-09-16T15:16:59.784 INFO:tasks.rados.rados.0.smithi162.stdout:3434: seq_num 1180 ranges {632260=428535,1817458=463413,2687955=520034,3709324=122936} 2024-09-16T15:16:59.789 INFO:tasks.rados.rados.0.smithi162.stdout:3434: writing smithi16234619-19 from 632260 to 1060795 tid 1 2024-09-16T15:16:59.797 INFO:tasks.rados.rados.0.smithi162.stdout:3434: writing smithi16234619-19 from 1817458 to 2280871 tid 2 2024-09-16T15:16:59.803 INFO:tasks.rados.rados.0.smithi162.stdout:3434: writing smithi16234619-19 from 2687955 to 3207989 tid 3 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3434: writing smithi16234619-19 from 3709324 to 3832260 tid 4 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3435: delete oid 25 current snap is 359 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3429: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: finishing write tid 1 to smithi16234619-41 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: finishing write tid 2 to smithi16234619-41 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: finishing write tid 3 to smithi16234619-41 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: finishing write tid 4 to smithi16234619-41 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: finishing write tid 5 to smithi16234619-41 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: oid 41 updating version 0 to 1483 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: oid 41 version 1483 is already newer than 1482 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: oid 41 version 1483 is already newer than 1481 2024-09-16T15:16:59.808 INFO:tasks.rados.rados.0.smithi162.stdout:3431: oid 41 updating version 1483 to 1484 2024-09-16T15:16:59.809 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 1484 (ObjNum 1177 snap 359 seq_num 1177) dirty exists 2024-09-16T15:16:59.809 INFO:tasks.rados.rados.0.smithi162.stdout:3431: left oid 41 (ObjNum 1177 snap 359 seq_num 1177) 2024-09-16T15:16:59.809 INFO:tasks.rados.rados.0.smithi162.stdout:3429: finishing copy_from to smithi16234619-8 2024-09-16T15:16:59.809 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1477 (ObjNum 981 snap 276 seq_num 981) dirty exists 2024-09-16T15:16:59.811 INFO:tasks.rados.rados.0.smithi162.stdout:3432: finishing write tid 1 to smithi16234619-38 2024-09-16T15:16:59.812 INFO:tasks.rados.rados.0.smithi162.stdout:3432: finishing write tid 2 to smithi16234619-38 2024-09-16T15:16:59.812 INFO:tasks.rados.rados.0.smithi162.stdout:3432: finishing write tid 3 to smithi16234619-38 2024-09-16T15:16:59.814 INFO:tasks.rados.rados.0.smithi162.stdout:3432: finishing write tid 4 to smithi16234619-38 2024-09-16T15:16:59.814 INFO:tasks.rados.rados.0.smithi162.stdout:3432: finishing write tid 5 to smithi16234619-38 2024-09-16T15:16:59.814 INFO:tasks.rados.rados.0.smithi162.stdout:3432: oid 38 updating version 0 to 1823 2024-09-16T15:16:59.814 INFO:tasks.rados.rados.0.smithi162.stdout:3432: oid 38 version 1823 is already newer than 1822 2024-09-16T15:16:59.815 INFO:tasks.rados.rados.0.smithi162.stdout:3432: oid 38 updating version 1823 to 1824 2024-09-16T15:16:59.815 INFO:tasks.rados.rados.0.smithi162.stdout:3432: oid 38 version 1824 is already newer than 1821 2024-09-16T15:16:59.815 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1824 (ObjNum 1178 snap 359 seq_num 1178) dirty exists 2024-09-16T15:16:59.815 INFO:tasks.rados.rados.0.smithi162.stdout:3432: left oid 38 (ObjNum 1178 snap 359 seq_num 1178) 2024-09-16T15:16:59.824 INFO:tasks.rados.rados.0.smithi162.stdout:3433: finishing write tid 1 to smithi16234619-26 2024-09-16T15:16:59.824 INFO:tasks.rados.rados.0.smithi162.stdout:3433: finishing write tid 2 to smithi16234619-26 2024-09-16T15:16:59.824 INFO:tasks.rados.rados.0.smithi162.stdout:3433: finishing write tid 3 to smithi16234619-26 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: finishing write tid 4 to smithi16234619-26 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: finishing write tid 5 to smithi16234619-26 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: oid 26 updating version 0 to 2114 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: oid 26 updating version 2114 to 2115 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: oid 26 updating version 2115 to 2116 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: oid 26 version 2116 is already newer than 2113 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2116 (ObjNum 1179 snap 359 seq_num 1179) dirty exists 2024-09-16T15:16:59.825 INFO:tasks.rados.rados.0.smithi162.stdout:3433: left oid 26 (ObjNum 1179 snap 359 seq_num 1179) 2024-09-16T15:16:59.827 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 1 to smithi16234619-19 2024-09-16T15:16:59.827 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 2 to smithi16234619-19 2024-09-16T15:16:59.828 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 3 to smithi16234619-19 2024-09-16T15:16:59.830 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 4 to smithi16234619-19 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 5 to smithi16234619-19 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: finishing write tid 6 to smithi16234619-19 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: oid 19 updating version 0 to 1764 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: oid 19 updating version 1764 to 1767 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: oid 19 version 1767 is already newer than 1765 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: oid 19 version 1767 is already newer than 1766 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: oid 19 version 1767 is already newer than 1763 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 1767 (ObjNum 1180 snap 359 seq_num 1180) dirty exists 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3434: left oid 19 (ObjNum 1180 snap 359 seq_num 1180) 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3429: done (5 left) 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3431: done (4 left) 2024-09-16T15:16:59.831 INFO:tasks.rados.rados.0.smithi162.stdout:3432: done (3 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3433: done (2 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3434: done (1 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3435: done (0 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3436: read oid 44 snap -1 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3436: expect deleted 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3437: delete oid 22 current snap is 359 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3436: done (1 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3437: done (0 left) 2024-09-16T15:16:59.832 INFO:tasks.rados.rados.0.smithi162.stdout:3438: snap_remove snap 349 2024-09-16T15:17:00.634 INFO:tasks.rados.rados.0.smithi162.stdout:3438: done (0 left) 2024-09-16T15:17:00.634 INFO:tasks.rados.rados.0.smithi162.stdout:3439: write oid 12 current snap is 359 2024-09-16T15:17:00.634 INFO:tasks.rados.rados.0.smithi162.stdout:3439: seq_num 1183 ranges {682681=654282,1876551=610740,3053871=470203} 2024-09-16T15:17:00.641 INFO:tasks.rados.rados.0.smithi162.stdout:3439: writing smithi16234619-12 from 682681 to 1336963 tid 1 2024-09-16T15:17:00.648 INFO:tasks.rados.rados.0.smithi162.stdout:3439: writing smithi16234619-12 from 1876551 to 2487291 tid 2 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:3439: writing smithi16234619-12 from 3053871 to 3524074 tid 3 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:3440: read oid 6 snap -1 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:3440: expect (ObjNum 1160 snap 345 seq_num 1160) 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:3441: rollback oid 2 current snap is 359 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 350 2024-09-16T15:17:00.654 INFO:tasks.rados.rados.0.smithi162.stdout:3442: snap_create 2024-09-16T15:17:00.661 INFO:tasks.rados.rados.0.smithi162.stdout:3441: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:17:00.661 INFO:tasks.rados.rados.0.smithi162.stdout:3441: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:17:00.661 INFO:tasks.rados.rados.0.smithi162.stdout:3440: expect (ObjNum 1160 snap 345 seq_num 1160) 2024-09-16T15:17:00.772 INFO:tasks.rados.rados.0.smithi162.stdout:3441: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2122 (ObjNum 1097 snap 328 seq_num 1097) dirty exists 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: finishing write tid 1 to smithi16234619-12 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: finishing write tid 2 to smithi16234619-12 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: finishing write tid 3 to smithi16234619-12 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: finishing write tid 4 to smithi16234619-12 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: finishing write tid 5 to smithi16234619-12 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: oid 12 updating version 0 to 1478 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: oid 12 version 1478 is already newer than 1477 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: oid 12 updating version 1478 to 1479 2024-09-16T15:17:00.773 INFO:tasks.rados.rados.0.smithi162.stdout:3439: oid 12 updating version 1479 to 1480 2024-09-16T15:17:00.774 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1480 (ObjNum 1183 snap 359 seq_num 1183) dirty exists 2024-09-16T15:17:00.774 INFO:tasks.rados.rados.0.smithi162.stdout:3439: left oid 12 (ObjNum 1183 snap 359 seq_num 1183) 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3439: done (3 left) 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3440: done (2 left) 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3441: done (1 left) 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3442: done (0 left) 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3443: rollback oid 48 current snap is 360 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 48 to 338 2024-09-16T15:17:01.637 INFO:tasks.rados.rados.0.smithi162.stdout:3444: delete oid 8 current snap is 360 2024-09-16T15:17:01.640 INFO:tasks.rados.rados.0.smithi162.stdout:3444: done (1 left) 2024-09-16T15:17:01.640 INFO:tasks.rados.rados.0.smithi162.stdout:3445: write oid 34 current snap is 360 2024-09-16T15:17:01.640 INFO:tasks.rados.rados.0.smithi162.stdout:3445: seq_num 1185 ranges {716295=404780,1593515=614409,2708717=492747,3640519=275776} 2024-09-16T15:17:01.646 INFO:tasks.rados.rados.0.smithi162.stdout:3445: writing smithi16234619-34 from 716295 to 1121075 tid 1 2024-09-16T15:17:01.652 INFO:tasks.rados.rados.0.smithi162.stdout:3445: writing smithi16234619-34 from 1593515 to 2207924 tid 2 2024-09-16T15:17:01.657 INFO:tasks.rados.rados.0.smithi162.stdout:3445: writing smithi16234619-34 from 2708717 to 3201464 tid 3 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3445: writing smithi16234619-34 from 3640519 to 3916295 tid 4 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3446: read oid 35 snap 323 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3446: expect (ObjNum 1054 snap 313 seq_num 1054) 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3447: read oid 12 snap 350 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3447: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:01.662 INFO:tasks.rados.rados.0.smithi162.stdout:3448: snap_remove snap 326 2024-09-16T15:17:03.595 INFO:tasks.rados.rados.0.smithi162.stdout:3448: done (4 left) 2024-09-16T15:17:03.595 INFO:tasks.rados.rados.0.smithi162.stdout:3449: write oid 30 current snap is 360 2024-09-16T15:17:03.595 INFO:tasks.rados.rados.0.smithi162.stdout:3449: seq_num 1186 ranges {733102=596382,1768350=758067,3136140=704019} 2024-09-16T15:17:03.602 INFO:tasks.rados.rados.0.smithi162.stdout:3449: writing smithi16234619-30 from 733102 to 1329484 tid 1 2024-09-16T15:17:03.609 INFO:tasks.rados.rados.0.smithi162.stdout:3449: writing smithi16234619-30 from 1768350 to 2526417 tid 2 2024-09-16T15:17:03.617 INFO:tasks.rados.rados.0.smithi162.stdout:3449: writing smithi16234619-30 from 3136140 to 3840159 tid 3 2024-09-16T15:17:03.617 INFO:tasks.rados.rados.0.smithi162.stdout:3450: rollback oid 27 current snap is 360 2024-09-16T15:17:03.617 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 27 to 350 2024-09-16T15:17:03.617 INFO:tasks.rados.rados.0.smithi162.stdout:3443: finishing rollback tid 0 to smithi16234619-48 2024-09-16T15:17:03.617 INFO:tasks.rados.rados.0.smithi162.stdout:3443: finishing rollback tid 1 to smithi16234619-48 2024-09-16T15:17:03.618 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1747 (ObjNum 129 snap 0 seq_num 864459328) dirty dne 2024-09-16T15:17:03.618 INFO:tasks.rados.rados.0.smithi162.stdout:3446: expect (ObjNum 1054 snap 313 seq_num 1054) 2024-09-16T15:17:03.887 INFO:tasks.rados.rados.0.smithi162.stdout:3447: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 1 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 2 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 3 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 4 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 5 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: finishing write tid 6 to smithi16234619-34 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: oid 34 updating version 0 to 1783 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: oid 34 version 1783 is already newer than 1779 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: oid 34 version 1783 is already newer than 1781 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: oid 34 version 1783 is already newer than 1780 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: oid 34 version 1783 is already newer than 1782 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1783 (ObjNum 1185 snap 360 seq_num 1185) dirty exists 2024-09-16T15:17:04.143 INFO:tasks.rados.rados.0.smithi162.stdout:3445: left oid 34 (ObjNum 1185 snap 360 seq_num 1185) 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3443: done (5 left) 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3445: done (4 left) 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3446: done (3 left) 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3447: done (2 left) 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3451: copy_from oid 26 from oid 3 current snap is 360 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3450: finishing rollback tid 0 to smithi16234619-27 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3450: finishing rollback tid 1 to smithi16234619-27 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 27 v 1749 (ObjNum 33 snap 0 seq_num 864820800) dirty dne 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: finishing write tid 1 to smithi16234619-30 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: finishing write tid 2 to smithi16234619-30 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: finishing write tid 3 to smithi16234619-30 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: finishing write tid 4 to smithi16234619-30 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: finishing write tid 5 to smithi16234619-30 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: oid 30 updating version 0 to 1419 2024-09-16T15:17:04.144 INFO:tasks.rados.rados.0.smithi162.stdout:3449: oid 30 updating version 1419 to 1420 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3449: oid 30 updating version 1420 to 1421 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3449: oid 30 updating version 1421 to 1422 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1422 (ObjNum 1186 snap 360 seq_num 1186) dirty exists 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3449: left oid 30 (ObjNum 1186 snap 360 seq_num 1186) 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3449: done (2 left) 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3450: done (1 left) 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3452: read oid 42 snap -1 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3452: expect deleted 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3453: snap_create 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3451: finishing copy_from to smithi16234619-26 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3451: got expected ENOENT (src dne) 2024-09-16T15:17:04.145 INFO:tasks.rados.rados.0.smithi162.stdout:3451: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:17:05.024 INFO:tasks.rados.rados.0.smithi162.stdout:3451: done (2 left) 2024-09-16T15:17:05.024 INFO:tasks.rados.rados.0.smithi162.stdout:3452: done (1 left) 2024-09-16T15:17:05.024 INFO:tasks.rados.rados.0.smithi162.stdout:3453: done (0 left) 2024-09-16T15:17:05.024 INFO:tasks.rados.rados.0.smithi162.stdout:3454: copy_from oid 28 from oid 6 current snap is 361 2024-09-16T15:17:05.024 INFO:tasks.rados.rados.0.smithi162.stdout:3455: snap_remove snap 356 2024-09-16T15:17:06.031 INFO:tasks.rados.rados.0.smithi162.stdout:3455: done (1 left) 2024-09-16T15:17:06.031 INFO:tasks.rados.rados.0.smithi162.stdout:3456: delete oid 34 current snap is 361 2024-09-16T15:17:06.031 INFO:tasks.rados.rados.0.smithi162.stdout:3454: finishing copy_from racing read to smithi16234619-28 2024-09-16T15:17:06.031 INFO:tasks.rados.rados.0.smithi162.stdout:3454: finishing copy_from to smithi16234619-28 2024-09-16T15:17:06.031 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1806 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:17:06.043 INFO:tasks.rados.rados.0.smithi162.stdout:3454: done (1 left) 2024-09-16T15:17:06.043 INFO:tasks.rados.rados.0.smithi162.stdout:3456: done (0 left) 2024-09-16T15:17:06.043 INFO:tasks.rados.rados.0.smithi162.stdout:3457: rollback oid 17 current snap is 361 2024-09-16T15:17:06.043 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 17 to 360 2024-09-16T15:17:06.043 INFO:tasks.rados.rados.0.smithi162.stdout:3458: snap_create 2024-09-16T15:17:06.048 INFO:tasks.rados.rados.0.smithi162.stdout:3457: finishing rollback tid 0 to smithi16234619-17 2024-09-16T15:17:06.051 INFO:tasks.rados.rados.0.smithi162.stdout:3457: finishing rollback tid 1 to smithi16234619-17 2024-09-16T15:17:06.052 INFO:tasks.rados.rados.0.smithi162.stdout:3457: finishing rollback tid 2 to smithi16234619-17 2024-09-16T15:17:06.052 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 2138 (ObjNum 1118 snap 333 seq_num 1118) dirty exists 2024-09-16T15:17:07.046 INFO:tasks.rados.rados.0.smithi162.stdout:3457: done (1 left) 2024-09-16T15:17:07.046 INFO:tasks.rados.rados.0.smithi162.stdout:3458: done (0 left) 2024-09-16T15:17:07.046 INFO:tasks.rados.rados.0.smithi162.stdout:3459: copy_from oid 5 from oid 33 current snap is 362 2024-09-16T15:17:07.046 INFO:tasks.rados.rados.0.smithi162.stdout:3460: snap_create 2024-09-16T15:17:07.090 INFO:tasks.rados.rados.0.smithi162.stdout:3459: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:17:07.093 INFO:tasks.rados.rados.0.smithi162.stdout:3459: finishing copy_from to smithi16234619-5 2024-09-16T15:17:07.093 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 1577 (ObjNum 1119 snap 334 seq_num 1119) dirty exists 2024-09-16T15:17:08.044 INFO:tasks.rados.rados.0.smithi162.stdout:3459: done (1 left) 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3460: done (0 left) 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3461: rollback oid 12 current snap is 363 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 12 to 352 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3462: copy_from oid 17 from oid 8 current snap is 363 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3463: rollback oid 35 current snap is 363 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 35 to 339 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3464: write oid 14 current snap is 363 2024-09-16T15:17:08.045 INFO:tasks.rados.rados.0.smithi162.stdout:3464: seq_num 1188 ranges {766716=746880,2168961=761736,3474633=442916} 2024-09-16T15:17:08.053 INFO:tasks.rados.rados.0.smithi162.stdout:3464: writing smithi16234619-14 from 766716 to 1513596 tid 1 2024-09-16T15:17:08.061 INFO:tasks.rados.rados.0.smithi162.stdout:3464: writing smithi16234619-14 from 2168961 to 2930697 tid 2 2024-09-16T15:17:08.066 INFO:tasks.rados.rados.0.smithi162.stdout:3464: writing smithi16234619-14 from 3474633 to 3917549 tid 3 2024-09-16T15:17:08.066 INFO:tasks.rados.rados.0.smithi162.stdout:3465: snap_remove snap 362 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3465: done (4 left) 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3466: read oid 38 snap -1 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3466: expect (ObjNum 1178 snap 359 seq_num 1178) 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3467: read oid 50 snap -1 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3467: expect (ObjNum 1175 snap 357 seq_num 1175) 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3468: copy_from oid 3 from oid 6 current snap is 363 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:3469: rollback oid 4 current snap is 363 2024-09-16T15:17:09.013 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 4 to 319 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3462: finishing copy_from to smithi16234619-17 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3462: got expected ENOENT (src dne) 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3462: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3461: finishing rollback tid 0 to smithi16234619-12 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3461: finishing rollback tid 1 to smithi16234619-12 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3463: finishing rollback tid 0 to smithi16234619-35 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3463: finishing rollback tid 1 to smithi16234619-35 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3463: finishing rollback tid 2 to smithi16234619-35 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1827 (ObjNum 1054 snap 313 seq_num 1054) dirty exists 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3461: finishing rollback tid 2 to smithi16234619-12 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 1483 (ObjNum 1140 snap 340 seq_num 1140) dirty exists 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3464: finishing write tid 1 to smithi16234619-14 2024-09-16T15:17:09.014 INFO:tasks.rados.rados.0.smithi162.stdout:3464: finishing write tid 2 to smithi16234619-14 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: finishing write tid 3 to smithi16234619-14 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: finishing write tid 4 to smithi16234619-14 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: finishing write tid 5 to smithi16234619-14 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: oid 14 updating version 0 to 1805 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: oid 14 version 1805 is already newer than 1804 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: oid 14 updating version 1805 to 1806 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: oid 14 version 1806 is already newer than 1803 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1806 (ObjNum 1188 snap 363 seq_num 1188) dirty exists 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: left oid 14 (ObjNum 1188 snap 363 seq_num 1188) 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3461: done (7 left) 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3462: done (6 left) 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3463: done (5 left) 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3464: done (4 left) 2024-09-16T15:17:09.015 INFO:tasks.rados.rados.0.smithi162.stdout:3470: snap_create 2024-09-16T15:17:09.016 INFO:tasks.rados.rados.0.smithi162.stdout:3469: finishing rollback tid 1 to smithi16234619-4 2024-09-16T15:17:09.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1743 (ObjNum 33 snap 0 seq_num 4) dirty dne 2024-09-16T15:17:09.026 INFO:tasks.rados.rados.0.smithi162.stdout:3466: expect (ObjNum 1178 snap 359 seq_num 1178) 2024-09-16T15:17:09.233 INFO:tasks.rados.rados.0.smithi162.stdout:3467: expect (ObjNum 1175 snap 357 seq_num 1175) 2024-09-16T15:17:09.488 INFO:tasks.rados.rados.0.smithi162.stdout:3468: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:17:09.488 INFO:tasks.rados.rados.0.smithi162.stdout:3468: finishing copy_from to smithi16234619-3 2024-09-16T15:17:09.488 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1806 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3466: done (4 left) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3467: done (3 left) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3468: done (2 left) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3469: done (1 left) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3470: done (0 left) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3471: read oid 30 snap -1 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3471: expect (ObjNum 1186 snap 360 seq_num 1186) 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3472: write (excl) oid 42 current snap is 364 2024-09-16T15:17:10.290 INFO:tasks.rados.rados.0.smithi162.stdout:3472: seq_num 1189 ranges {783523=422129,1827443=621747,3218409=654188} 2024-09-16T15:17:10.296 INFO:tasks.rados.rados.0.smithi162.stdout:3472: writing smithi16234619-42 from 783523 to 1205652 tid 1 2024-09-16T15:17:10.303 INFO:tasks.rados.rados.0.smithi162.stdout:3472: writing smithi16234619-42 from 1827443 to 2449190 tid 2 2024-09-16T15:17:10.312 INFO:tasks.rados.rados.0.smithi162.stdout:3472: writing smithi16234619-42 from 3218409 to 3872597 tid 3 2024-09-16T15:17:10.312 INFO:tasks.rados.rados.0.smithi162.stdout:3473: snap_remove snap 363 2024-09-16T15:17:11.304 INFO:tasks.rados.rados.0.smithi162.stdout:3473: done (2 left) 2024-09-16T15:17:11.304 INFO:tasks.rados.rados.0.smithi162.stdout:3474: copy_from oid 2 from oid 34 current snap is 364 2024-09-16T15:17:11.304 INFO:tasks.rados.rados.0.smithi162.stdout:3471: expect (ObjNum 1186 snap 360 seq_num 1186) 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: finishing write tid 1 to smithi16234619-42 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: finishing write tid 2 to smithi16234619-42 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: finishing write tid 3 to smithi16234619-42 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: finishing write tid 4 to smithi16234619-42 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: finishing write tid 5 to smithi16234619-42 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: oid 42 updating version 0 to 1936 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: oid 42 updating version 1936 to 1937 2024-09-16T15:17:11.542 INFO:tasks.rados.rados.0.smithi162.stdout:3472: oid 42 version 1937 is already newer than 1934 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:3472: oid 42 version 1937 is already newer than 1935 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1937 (ObjNum 1189 snap 364 seq_num 1189) dirty exists 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:3472: left oid 42 (ObjNum 1189 snap 364 seq_num 1189) 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:3471: done (2 left) 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:3472: done (1 left) 2024-09-16T15:17:11.543 INFO:tasks.rados.rados.0.smithi162.stdout:3475: snap_remove snap 319 2024-09-16T15:17:12.315 INFO:tasks.rados.rados.0.smithi162.stdout:3475: done (1 left) 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3476: read oid 15 snap 323 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3476: expect deleted 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3477: write oid 26 current snap is 364 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3474: finishing copy_from to smithi16234619-2 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3474: got expected ENOENT (src dne) 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3474: finishing copy_from racing read to smithi16234619-2 2024-09-16T15:17:12.316 INFO:tasks.rados.rados.0.smithi162.stdout:3477: seq_num 1190 ranges {400330=497378,1485925=514405} 2024-09-16T15:17:12.320 INFO:tasks.rados.rados.0.smithi162.stdout:3477: writing smithi16234619-26 from 400330 to 897708 tid 1 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3477: writing smithi16234619-26 from 1485925 to 2000330 tid 2 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3474: done (2 left) 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3478: copy_from oid 35 from oid 4 current snap is 364 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3476: done (2 left) 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3479: write (excl) oid 7 current snap is 364 2024-09-16T15:17:12.327 INFO:tasks.rados.rados.0.smithi162.stdout:3479: seq_num 1191 ranges {417137=572627,1428054=589083} 2024-09-16T15:17:12.332 INFO:tasks.rados.rados.0.smithi162.stdout:3479: writing smithi16234619-7 from 417137 to 989764 tid 1 2024-09-16T15:17:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:3479: writing smithi16234619-7 from 1428054 to 2017137 tid 2 2024-09-16T15:17:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:3480: snap_create 2024-09-16T15:17:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:3478: finishing copy_from to smithi16234619-35 2024-09-16T15:17:12.338 INFO:tasks.rados.rados.0.smithi162.stdout:3478: got expected ENOENT (src dne) 2024-09-16T15:17:12.339 INFO:tasks.rados.rados.0.smithi162.stdout:3478: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:17:12.350 INFO:tasks.rados.rados.0.smithi162.stdout:3477: finishing write tid 1 to smithi16234619-26 2024-09-16T15:17:12.351 INFO:tasks.rados.rados.0.smithi162.stdout:3477: finishing write tid 2 to smithi16234619-26 2024-09-16T15:17:12.352 INFO:tasks.rados.rados.0.smithi162.stdout:3477: finishing write tid 3 to smithi16234619-26 2024-09-16T15:17:12.352 INFO:tasks.rados.rados.0.smithi162.stdout:3477: finishing write tid 4 to smithi16234619-26 2024-09-16T15:17:12.352 INFO:tasks.rados.rados.0.smithi162.stdout:3477: oid 26 updating version 0 to 2150 2024-09-16T15:17:12.352 INFO:tasks.rados.rados.0.smithi162.stdout:3477: oid 26 updating version 2150 to 2152 2024-09-16T15:17:12.353 INFO:tasks.rados.rados.0.smithi162.stdout:3477: oid 26 version 2152 is already newer than 2151 2024-09-16T15:17:12.353 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2152 (ObjNum 1190 snap 364 seq_num 1190) dirty exists 2024-09-16T15:17:12.353 INFO:tasks.rados.rados.0.smithi162.stdout:3477: left oid 26 (ObjNum 1190 snap 364 seq_num 1190) 2024-09-16T15:17:12.363 INFO:tasks.rados.rados.0.smithi162.stdout:3479: finishing write tid 1 to smithi16234619-7 2024-09-16T15:17:12.363 INFO:tasks.rados.rados.0.smithi162.stdout:3479: finishing write tid 2 to smithi16234619-7 2024-09-16T15:17:12.365 INFO:tasks.rados.rados.0.smithi162.stdout:3479: finishing write tid 3 to smithi16234619-7 2024-09-16T15:17:12.365 INFO:tasks.rados.rados.0.smithi162.stdout:3479: finishing write tid 4 to smithi16234619-7 2024-09-16T15:17:12.365 INFO:tasks.rados.rados.0.smithi162.stdout:3479: oid 7 updating version 0 to 1501 2024-09-16T15:17:12.365 INFO:tasks.rados.rados.0.smithi162.stdout:3479: oid 7 version 1501 is already newer than 1500 2024-09-16T15:17:12.365 INFO:tasks.rados.rados.0.smithi162.stdout:3479: oid 7 updating version 1501 to 1502 2024-09-16T15:17:12.366 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1502 (ObjNum 1191 snap 364 seq_num 1191) dirty exists 2024-09-16T15:17:12.366 INFO:tasks.rados.rados.0.smithi162.stdout:3479: left oid 7 (ObjNum 1191 snap 364 seq_num 1191) 2024-09-16T15:17:13.315 INFO:tasks.rados.rados.0.smithi162.stdout:3477: done (3 left) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3478: done (2 left) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3479: done (1 left) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3480: done (0 left) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3481: rollback oid 38 current snap is 365 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 38 to 355 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3482: copy_from oid 42 from oid 9 current snap is 365 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3483: read oid 14 snap -1 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3483: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3484: read oid 12 snap -1 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3484: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:13.316 INFO:tasks.rados.rados.0.smithi162.stdout:3485: delete oid 49 current snap is 365 2024-09-16T15:17:13.321 INFO:tasks.rados.rados.0.smithi162.stdout:3485: done (4 left) 2024-09-16T15:17:13.321 INFO:tasks.rados.rados.0.smithi162.stdout:3486: write (excl) oid 1 current snap is 365 2024-09-16T15:17:13.321 INFO:tasks.rados.rados.0.smithi162.stdout:3486: seq_num 1193 ranges {450751=723125,1945018=105733} 2024-09-16T15:17:13.328 INFO:tasks.rados.rados.0.smithi162.stdout:3486: writing smithi16234619-1 from 450751 to 1173876 tid 1 2024-09-16T15:17:13.333 INFO:tasks.rados.rados.0.smithi162.stdout:3486: writing smithi16234619-1 from 1945018 to 2050751 tid 2 2024-09-16T15:17:13.333 INFO:tasks.rados.rados.0.smithi162.stdout:3487: rollback oid 37 current snap is 365 2024-09-16T15:17:13.333 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 338 2024-09-16T15:17:13.333 INFO:tasks.rados.rados.0.smithi162.stdout:3481: finishing rollback tid 0 to smithi16234619-38 2024-09-16T15:17:13.333 INFO:tasks.rados.rados.0.smithi162.stdout:3484: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:13.590 INFO:tasks.rados.rados.0.smithi162.stdout:3481: finishing rollback tid 1 to smithi16234619-38 2024-09-16T15:17:13.590 INFO:tasks.rados.rados.0.smithi162.stdout:3481: finishing rollback tid 2 to smithi16234619-38 2024-09-16T15:17:13.591 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 38 v 1830 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2024-09-16T15:17:13.591 INFO:tasks.rados.rados.0.smithi162.stdout:3483: expect (ObjNum 1188 snap 363 seq_num 1188) 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: finishing write tid 1 to smithi16234619-1 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: finishing write tid 2 to smithi16234619-1 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: finishing write tid 3 to smithi16234619-1 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: finishing write tid 4 to smithi16234619-1 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: oid 1 updating version 0 to 1751 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: oid 1 version 1751 is already newer than 1750 2024-09-16T15:17:13.810 INFO:tasks.rados.rados.0.smithi162.stdout:3486: oid 1 updating version 1751 to 1752 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1752 (ObjNum 1193 snap 365 seq_num 1193) dirty exists 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3486: left oid 1 (ObjNum 1193 snap 365 seq_num 1193) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3482: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3482: finishing copy_from to smithi16234619-42 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 1737 (ObjNum 1151 snap 343 seq_num 1151) dirty exists 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3487: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3487: finishing rollback tid 2 to smithi16234619-37 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1947 (ObjNum 1131 snap 338 seq_num 1131) dirty exists 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3481: done (5 left) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3482: done (4 left) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3483: done (3 left) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3484: done (2 left) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3486: done (1 left) 2024-09-16T15:17:13.811 INFO:tasks.rados.rados.0.smithi162.stdout:3487: done (0 left) 2024-09-16T15:17:13.812 INFO:tasks.rados.rados.0.smithi162.stdout:3488: write (excl) oid 11 current snap is 365 2024-09-16T15:17:13.812 INFO:tasks.rados.rados.0.smithi162.stdout:3488: seq_num 1194 ranges {467558=514727,1719853=347705} 2024-09-16T15:17:13.815 INFO:tasks.rados.rados.0.smithi162.stdout:3488: writing smithi16234619-11 from 467558 to 982285 tid 1 2024-09-16T15:17:13.821 INFO:tasks.rados.rados.0.smithi162.stdout:3488: writing smithi16234619-11 from 1719853 to 2067558 tid 2 2024-09-16T15:17:13.821 INFO:tasks.rados.rados.0.smithi162.stdout:3489: read oid 12 snap 355 2024-09-16T15:17:13.821 INFO:tasks.rados.rados.0.smithi162.stdout:3489: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:13.821 INFO:tasks.rados.rados.0.smithi162.stdout:3490: write oid 48 current snap is 365 2024-09-16T15:17:13.821 INFO:tasks.rados.rados.0.smithi162.stdout:3490: seq_num 1195 ranges {484365=589976,1661982=422383} 2024-09-16T15:17:13.827 INFO:tasks.rados.rados.0.smithi162.stdout:3490: writing smithi16234619-48 from 484365 to 1074341 tid 1 2024-09-16T15:17:13.833 INFO:tasks.rados.rados.0.smithi162.stdout:3490: writing smithi16234619-48 from 1661982 to 2084365 tid 2 2024-09-16T15:17:13.833 INFO:tasks.rados.rados.0.smithi162.stdout:3491: read oid 24 snap 342 2024-09-16T15:17:13.833 INFO:tasks.rados.rados.0.smithi162.stdout:3491: expect deleted 2024-09-16T15:17:13.833 INFO:tasks.rados.rados.0.smithi162.stdout:3492: snap_remove snap 364 2024-09-16T15:17:14.008 INFO:tasks.rados.rados.0.smithi162.stdout:3492: done (4 left) 2024-09-16T15:17:14.009 INFO:tasks.rados.rados.0.smithi162.stdout:3493: write oid 15 current snap is 365 2024-09-16T15:17:14.009 INFO:tasks.rados.rados.0.smithi162.stdout:3493: seq_num 1196 ranges {501172=665225,1720464=380708} 2024-09-16T15:17:14.016 INFO:tasks.rados.rados.0.smithi162.stdout:3493: writing smithi16234619-15 from 501172 to 1166397 tid 1 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3493: writing smithi16234619-15 from 1720464 to 2101172 tid 2 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3494: read oid 41 snap 357 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3494: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3495: read oid 20 snap -1 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3495: expect (ObjNum 1138 snap 340 seq_num 1138) 2024-09-16T15:17:14.023 INFO:tasks.rados.rados.0.smithi162.stdout:3496: read oid 45 snap -1 2024-09-16T15:17:14.024 INFO:tasks.rados.rados.0.smithi162.stdout:3496: expect deleted 2024-09-16T15:17:14.024 INFO:tasks.rados.rados.0.smithi162.stdout:3497: rollback oid 40 current snap is 365 2024-09-16T15:17:14.024 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 40 to 352 2024-09-16T15:17:14.024 INFO:tasks.rados.rados.0.smithi162.stdout:3489: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: finishing write tid 1 to smithi16234619-11 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: finishing write tid 2 to smithi16234619-11 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: finishing write tid 3 to smithi16234619-11 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: finishing write tid 4 to smithi16234619-11 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: oid 11 updating version 0 to 1282 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: oid 11 updating version 1282 to 1283 2024-09-16T15:17:14.278 INFO:tasks.rados.rados.0.smithi162.stdout:3488: oid 11 updating version 1283 to 1284 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1284 (ObjNum 1194 snap 365 seq_num 1194) dirty exists 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3488: left oid 11 (ObjNum 1194 snap 365 seq_num 1194) 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3490: finishing write tid 1 to smithi16234619-48 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3488: done (8 left) 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3489: done (7 left) 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3491: done (6 left) 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3498: read oid 17 snap 340 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3498: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:17:14.279 INFO:tasks.rados.rados.0.smithi162.stdout:3499: snap_remove snap 350 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3499: done (7 left) 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3500: write (excl) oid 22 current snap is 365 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3490: finishing write tid 2 to smithi16234619-48 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3490: finishing write tid 3 to smithi16234619-48 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3490: finishing write tid 4 to smithi16234619-48 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3490: oid 48 updating version 0 to 1753 2024-09-16T15:17:15.017 INFO:tasks.rados.rados.0.smithi162.stdout:3490: oid 48 updating version 1753 to 1754 2024-09-16T15:17:15.018 INFO:tasks.rados.rados.0.smithi162.stdout:3490: oid 48 updating version 1754 to 1755 2024-09-16T15:17:15.018 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1755 (ObjNum 1195 snap 365 seq_num 1195) dirty exists 2024-09-16T15:17:15.018 INFO:tasks.rados.rados.0.smithi162.stdout:3490: left oid 48 (ObjNum 1195 snap 365 seq_num 1195) 2024-09-16T15:17:15.018 INFO:tasks.rados.rados.0.smithi162.stdout:3500: seq_num 1197 ranges {517979=740474,1778946=339033} 2024-09-16T15:17:15.024 INFO:tasks.rados.rados.0.smithi162.stdout:3500: writing smithi16234619-22 from 517979 to 1258453 tid 1 2024-09-16T15:17:15.029 INFO:tasks.rados.rados.0.smithi162.stdout:3500: writing smithi16234619-22 from 1778946 to 2117979 tid 2 2024-09-16T15:17:15.029 INFO:tasks.rados.rados.0.smithi162.stdout:3490: done (7 left) 2024-09-16T15:17:15.029 INFO:tasks.rados.rados.0.smithi162.stdout:3501: write oid 2 current snap is 365 2024-09-16T15:17:15.029 INFO:tasks.rados.rados.0.smithi162.stdout:3495: expect (ObjNum 1138 snap 340 seq_num 1138) 2024-09-16T15:17:15.160 INFO:tasks.rados.rados.0.smithi162.stdout:3497: finishing rollback tid 0 to smithi16234619-40 2024-09-16T15:17:15.160 INFO:tasks.rados.rados.0.smithi162.stdout:3494: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:17:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:3497: finishing rollback tid 1 to smithi16234619-40 2024-09-16T15:17:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:3497: finishing rollback tid 2 to smithi16234619-40 2024-09-16T15:17:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1486 (ObjNum 1116 snap 333 seq_num 1116) dirty exists 2024-09-16T15:17:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:3493: finishing write tid 1 to smithi16234619-15 2024-09-16T15:17:15.275 INFO:tasks.rados.rados.0.smithi162.stdout:3501: seq_num 1198 ranges {534786=415723,1437428=697358} 2024-09-16T15:17:15.280 INFO:tasks.rados.rados.0.smithi162.stdout:3501: writing smithi16234619-2 from 534786 to 950509 tid 1 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3501: writing smithi16234619-2 from 1437428 to 2134786 tid 2 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3494: done (7 left) 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3495: done (6 left) 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3496: done (5 left) 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3497: done (4 left) 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3502: write oid 33 current snap is 365 2024-09-16T15:17:15.287 INFO:tasks.rados.rados.0.smithi162.stdout:3502: seq_num 1199 ranges {551593=490972,1779557=372036} 2024-09-16T15:17:15.292 INFO:tasks.rados.rados.0.smithi162.stdout:3502: writing smithi16234619-33 from 551593 to 1042565 tid 1 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3502: writing smithi16234619-33 from 1779557 to 2151593 tid 2 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3503: snap_create 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: finishing write tid 2 to smithi16234619-15 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: finishing write tid 3 to smithi16234619-15 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: finishing write tid 4 to smithi16234619-15 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: oid 15 updating version 0 to 1809 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: oid 15 version 1809 is already newer than 1808 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: oid 15 version 1809 is already newer than 1807 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 1809 (ObjNum 1196 snap 365 seq_num 1196) dirty exists 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3493: left oid 15 (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:15.299 INFO:tasks.rados.rados.0.smithi162.stdout:3498: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:17:15.396 INFO:tasks.rados.rados.0.smithi162.stdout:3500: finishing write tid 1 to smithi16234619-22 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: finishing write tid 2 to smithi16234619-22 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: finishing write tid 3 to smithi16234619-22 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: finishing write tid 4 to smithi16234619-22 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: oid 22 updating version 0 to 1487 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: oid 22 version 1487 is already newer than 1486 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: oid 22 updating version 1487 to 1488 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 1488 (ObjNum 1197 snap 365 seq_num 1197) dirty exists 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3500: left oid 22 (ObjNum 1197 snap 365 seq_num 1197) 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3501: finishing write tid 1 to smithi16234619-2 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3501: finishing write tid 2 to smithi16234619-2 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3501: finishing write tid 3 to smithi16234619-2 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3501: finishing write tid 4 to smithi16234619-2 2024-09-16T15:17:15.397 INFO:tasks.rados.rados.0.smithi162.stdout:3501: oid 2 updating version 0 to 2162 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3501: oid 2 version 2162 is already newer than 2161 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3501: oid 2 version 2162 is already newer than 2160 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2162 (ObjNum 1198 snap 365 seq_num 1198) dirty exists 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3501: left oid 2 (ObjNum 1198 snap 365 seq_num 1198) 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: finishing write tid 1 to smithi16234619-33 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: finishing write tid 2 to smithi16234619-33 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: finishing write tid 3 to smithi16234619-33 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: finishing write tid 4 to smithi16234619-33 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: oid 33 updating version 0 to 1712 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: oid 33 version 1712 is already newer than 1711 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: oid 33 updating version 1712 to 1713 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1713 (ObjNum 1199 snap 365 seq_num 1199) dirty exists 2024-09-16T15:17:15.398 INFO:tasks.rados.rados.0.smithi162.stdout:3502: left oid 33 (ObjNum 1199 snap 365 seq_num 1199) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3493: done (5 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3498: done (4 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3500: done (3 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3501: done (2 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3502: done (1 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3503: done (0 left) 2024-09-16T15:17:16.128 INFO:tasks.rados.rados.0.smithi162.stdout:3504: snap_remove snap 340 2024-09-16T15:17:17.552 INFO:tasks.rados.rados.0.smithi162.stdout:3504: done (0 left) 2024-09-16T15:17:17.552 INFO:tasks.rados.rados.0.smithi162.stdout:3505: copy_from oid 21 from oid 34 current snap is 366 2024-09-16T15:17:17.553 INFO:tasks.rados.rados.0.smithi162.stdout:3506: copy_from oid 5 from oid 25 current snap is 366 2024-09-16T15:17:17.553 INFO:tasks.rados.rados.0.smithi162.stdout:3507: write (excl) oid 37 current snap is 366 2024-09-16T15:17:17.553 INFO:tasks.rados.rados.0.smithi162.stdout:3507: seq_num 1200 ranges {568400=566221,1838039=330361} 2024-09-16T15:17:17.558 INFO:tasks.rados.rados.0.smithi162.stdout:3507: writing smithi16234619-37 from 568400 to 1134621 tid 1 2024-09-16T15:17:17.565 INFO:tasks.rados.rados.0.smithi162.stdout:3507: writing smithi16234619-37 from 1838039 to 2168400 tid 2 2024-09-16T15:17:17.565 INFO:tasks.rados.rados.0.smithi162.stdout:3508: read oid 29 snap -1 2024-09-16T15:17:17.565 INFO:tasks.rados.rados.0.smithi162.stdout:3508: expect (ObjNum 1157 snap 345 seq_num 1157) 2024-09-16T15:17:17.565 INFO:tasks.rados.rados.0.smithi162.stdout:3509: snap_create 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3506: finishing copy_from to smithi16234619-5 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3506: got expected ENOENT (src dne) 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3505: finishing copy_from to smithi16234619-21 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3505: got expected ENOENT (src dne) 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3506: finishing copy_from racing read to smithi16234619-5 2024-09-16T15:17:17.581 INFO:tasks.rados.rados.0.smithi162.stdout:3505: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:17:17.588 INFO:tasks.rados.rados.0.smithi162.stdout:3507: finishing write tid 1 to smithi16234619-37 2024-09-16T15:17:17.594 INFO:tasks.rados.rados.0.smithi162.stdout:3507: finishing write tid 2 to smithi16234619-37 2024-09-16T15:17:17.594 INFO:tasks.rados.rados.0.smithi162.stdout:3507: finishing write tid 3 to smithi16234619-37 2024-09-16T15:17:17.594 INFO:tasks.rados.rados.0.smithi162.stdout:3507: finishing write tid 4 to smithi16234619-37 2024-09-16T15:17:17.594 INFO:tasks.rados.rados.0.smithi162.stdout:3507: oid 37 updating version 0 to 1957 2024-09-16T15:17:17.595 INFO:tasks.rados.rados.0.smithi162.stdout:3507: oid 37 updating version 1957 to 1962 2024-09-16T15:17:17.595 INFO:tasks.rados.rados.0.smithi162.stdout:3507: oid 37 updating version 1962 to 1963 2024-09-16T15:17:17.595 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 1963 (ObjNum 1200 snap 366 seq_num 1200) dirty exists 2024-09-16T15:17:17.595 INFO:tasks.rados.rados.0.smithi162.stdout:3507: left oid 37 (ObjNum 1200 snap 366 seq_num 1200) 2024-09-16T15:17:17.596 INFO:tasks.rados.rados.0.smithi162.stdout:3508: expect (ObjNum 1157 snap 345 seq_num 1157) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3505: done (4 left) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3506: done (3 left) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3507: done (2 left) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3508: done (1 left) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3509: done (0 left) 2024-09-16T15:17:18.562 INFO:tasks.rados.rados.0.smithi162.stdout:3510: snap_create 2024-09-16T15:17:19.010 INFO:tasks.rados.rados.0.smithi162.stdout:3510: done (0 left) 2024-09-16T15:17:19.010 INFO:tasks.rados.rados.0.smithi162.stdout:3511: snap_remove snap 342 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3511: done (0 left) 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3512: copy_from oid 47 from oid 7 current snap is 368 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3513: copy_from oid 26 from oid 4 current snap is 368 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3514: read oid 30 snap 351 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3514: expect (ObjNum 1136 snap 340 seq_num 1136) 2024-09-16T15:17:20.020 INFO:tasks.rados.rados.0.smithi162.stdout:3515: snap_remove snap 357 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3515: done (3 left) 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3516: copy_from oid 10 from oid 33 current snap is 368 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3513: finishing copy_from to smithi16234619-26 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3513: got expected ENOENT (src dne) 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3513: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:17:21.036 INFO:tasks.rados.rados.0.smithi162.stdout:3514: expect (ObjNum 1136 snap 340 seq_num 1136) 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3512: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3512: finishing copy_from to smithi16234619-47 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1502 (ObjNum 1191 snap 364 seq_num 1191) dirty exists 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3516: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3516: finishing copy_from to smithi16234619-10 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1713 (ObjNum 1199 snap 365 seq_num 1199) dirty exists 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3512: done (3 left) 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3513: done (2 left) 2024-09-16T15:17:21.200 INFO:tasks.rados.rados.0.smithi162.stdout:3514: done (1 left) 2024-09-16T15:17:21.201 INFO:tasks.rados.rados.0.smithi162.stdout:3516: done (0 left) 2024-09-16T15:17:21.201 INFO:tasks.rados.rados.0.smithi162.stdout:3517: snap_create 2024-09-16T15:17:22.035 INFO:tasks.rados.rados.0.smithi162.stdout:3517: done (0 left) 2024-09-16T15:17:22.036 INFO:tasks.rados.rados.0.smithi162.stdout:3518: write oid 34 current snap is 369 2024-09-16T15:17:22.036 INFO:tasks.rados.rados.0.smithi162.stdout:3518: seq_num 1201 ranges {585207=641470,1896521=288686} 2024-09-16T15:17:22.042 INFO:tasks.rados.rados.0.smithi162.stdout:3518: writing smithi16234619-34 from 585207 to 1226677 tid 1 2024-09-16T15:17:22.047 INFO:tasks.rados.rados.0.smithi162.stdout:3518: writing smithi16234619-34 from 1896521 to 2185207 tid 2 2024-09-16T15:17:22.047 INFO:tasks.rados.rados.0.smithi162.stdout:3519: snap_create 2024-09-16T15:17:22.071 INFO:tasks.rados.rados.0.smithi162.stdout:3518: finishing write tid 1 to smithi16234619-34 2024-09-16T15:17:22.072 INFO:tasks.rados.rados.0.smithi162.stdout:3518: finishing write tid 2 to smithi16234619-34 2024-09-16T15:17:22.072 INFO:tasks.rados.rados.0.smithi162.stdout:3518: finishing write tid 3 to smithi16234619-34 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:3518: finishing write tid 4 to smithi16234619-34 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:3518: oid 34 updating version 0 to 1855 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:3518: oid 34 updating version 1855 to 1857 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:3518: oid 34 version 1857 is already newer than 1856 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1857 (ObjNum 1201 snap 369 seq_num 1201) dirty exists 2024-09-16T15:17:22.073 INFO:tasks.rados.rados.0.smithi162.stdout:3518: left oid 34 (ObjNum 1201 snap 369 seq_num 1201) 2024-09-16T15:17:23.036 INFO:tasks.rados.rados.0.smithi162.stdout:3518: done (1 left) 2024-09-16T15:17:23.036 INFO:tasks.rados.rados.0.smithi162.stdout:3519: done (0 left) 2024-09-16T15:17:23.036 INFO:tasks.rados.rados.0.smithi162.stdout:3520: read oid 45 snap -1 2024-09-16T15:17:23.036 INFO:tasks.rados.rados.0.smithi162.stdout:3520: expect deleted 2024-09-16T15:17:23.036 INFO:tasks.rados.rados.0.smithi162.stdout:3521: copy_from oid 42 from oid 4 current snap is 370 2024-09-16T15:17:23.037 INFO:tasks.rados.rados.0.smithi162.stdout:3522: write oid 46 current snap is 370 2024-09-16T15:17:23.037 INFO:tasks.rados.rados.0.smithi162.stdout:3522: seq_num 1202 ranges {602014=433072,1671356=530658} 2024-09-16T15:17:23.038 INFO:tasks.rados.rados.0.smithi162.stdout:3522: writing smithi16234619-46 from 602014 to 1035086 tid 1 2024-09-16T15:17:23.045 INFO:tasks.rados.rados.0.smithi162.stdout:3522: writing smithi16234619-46 from 1671356 to 2202014 tid 2 2024-09-16T15:17:23.045 INFO:tasks.rados.rados.0.smithi162.stdout:3523: snap_create 2024-09-16T15:17:23.045 INFO:tasks.rados.rados.0.smithi162.stdout:3521: finishing copy_from to smithi16234619-42 2024-09-16T15:17:23.045 INFO:tasks.rados.rados.0.smithi162.stdout:3521: got expected ENOENT (src dne) 2024-09-16T15:17:23.045 INFO:tasks.rados.rados.0.smithi162.stdout:3521: finishing copy_from racing read to smithi16234619-42 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: finishing write tid 1 to smithi16234619-46 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: finishing write tid 2 to smithi16234619-46 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: finishing write tid 3 to smithi16234619-46 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: finishing write tid 4 to smithi16234619-46 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: oid 46 updating version 0 to 1216 2024-09-16T15:17:23.067 INFO:tasks.rados.rados.0.smithi162.stdout:3522: oid 46 version 1216 is already newer than 1214 2024-09-16T15:17:23.068 INFO:tasks.rados.rados.0.smithi162.stdout:3522: oid 46 version 1216 is already newer than 1215 2024-09-16T15:17:23.068 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1216 (ObjNum 1202 snap 370 seq_num 1202) dirty exists 2024-09-16T15:17:23.068 INFO:tasks.rados.rados.0.smithi162.stdout:3522: left oid 46 (ObjNum 1202 snap 370 seq_num 1202) 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3520: done (3 left) 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3521: done (2 left) 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3522: done (1 left) 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3523: done (0 left) 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3524: copy_from oid 21 from oid 6 current snap is 371 2024-09-16T15:17:24.018 INFO:tasks.rados.rados.0.smithi162.stdout:3525: read oid 40 snap -1 2024-09-16T15:17:24.019 INFO:tasks.rados.rados.0.smithi162.stdout:3525: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:17:24.019 INFO:tasks.rados.rados.0.smithi162.stdout:3526: delete oid 4 current snap is 371 2024-09-16T15:17:24.027 INFO:tasks.rados.rados.0.smithi162.stdout:3525: expect (ObjNum 1116 snap 333 seq_num 1116) 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3524: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3524: finishing copy_from to smithi16234619-21 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1806 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3524: done (2 left) 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3525: done (1 left) 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3526: done (0 left) 2024-09-16T15:17:24.150 INFO:tasks.rados.rados.0.smithi162.stdout:3527: read oid 38 snap -1 2024-09-16T15:17:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3527: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:17:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3528: snap_remove snap 352 2024-09-16T15:17:25.029 INFO:tasks.rados.rados.0.smithi162.stdout:3528: done (1 left) 2024-09-16T15:17:25.029 INFO:tasks.rados.rados.0.smithi162.stdout:3529: write oid 11 current snap is 371 2024-09-16T15:17:25.029 INFO:tasks.rados.rados.0.smithi162.stdout:3529: seq_num 1204 ranges {635628=583570,1671967=563661} 2024-09-16T15:17:25.036 INFO:tasks.rados.rados.0.smithi162.stdout:3529: writing smithi16234619-11 from 635628 to 1219198 tid 1 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3529: writing smithi16234619-11 from 1671967 to 2235628 tid 2 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3530: read oid 15 snap 360 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3530: expect (ObjNum 1163 snap 346 seq_num 1163) 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3531: read oid 47 snap -1 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3531: expect (ObjNum 1191 snap 364 seq_num 1191) 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3532: delete oid 16 current snap is 371 2024-09-16T15:17:25.043 INFO:tasks.rados.rados.0.smithi162.stdout:3527: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:17:25.170 INFO:tasks.rados.rados.0.smithi162.stdout:3531: expect (ObjNum 1191 snap 364 seq_num 1191) 2024-09-16T15:17:25.247 INFO:tasks.rados.rados.0.smithi162.stdout:3530: expect (ObjNum 1163 snap 346 seq_num 1163) 2024-09-16T15:17:25.361 INFO:tasks.rados.rados.0.smithi162.stdout:3529: finishing write tid 1 to smithi16234619-11 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: finishing write tid 2 to smithi16234619-11 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: finishing write tid 3 to smithi16234619-11 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: finishing write tid 4 to smithi16234619-11 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: oid 11 updating version 0 to 1285 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: oid 11 updating version 1285 to 1286 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: oid 11 updating version 1286 to 1287 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1287 (ObjNum 1204 snap 371 seq_num 1204) dirty exists 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: left oid 11 (ObjNum 1204 snap 371 seq_num 1204) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3527: done (4 left) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3529: done (3 left) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3530: done (2 left) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3531: done (1 left) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3532: done (0 left) 2024-09-16T15:17:25.362 INFO:tasks.rados.rados.0.smithi162.stdout:3533: rollback oid 29 current snap is 371 2024-09-16T15:17:25.363 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 358 2024-09-16T15:17:25.363 INFO:tasks.rados.rados.0.smithi162.stdout:3534: write oid 26 current snap is 371 2024-09-16T15:17:25.363 INFO:tasks.rados.rados.0.smithi162.stdout:3534: seq_num 1206 ranges {669242=734068,2188931=80311} 2024-09-16T15:17:25.369 INFO:tasks.rados.rados.0.smithi162.stdout:3534: writing smithi16234619-26 from 669242 to 1403310 tid 1 2024-09-16T15:17:25.374 INFO:tasks.rados.rados.0.smithi162.stdout:3534: writing smithi16234619-26 from 2188931 to 2269242 tid 2 2024-09-16T15:17:25.374 INFO:tasks.rados.rados.0.smithi162.stdout:3535: snap_remove snap 367 2024-09-16T15:17:26.037 INFO:tasks.rados.rados.0.smithi162.stdout:3535: done (2 left) 2024-09-16T15:17:26.037 INFO:tasks.rados.rados.0.smithi162.stdout:3536: write oid 46 current snap is 371 2024-09-16T15:17:26.037 INFO:tasks.rados.rados.0.smithi162.stdout:3533: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:17:26.037 INFO:tasks.rados.rados.0.smithi162.stdout:3536: seq_num 1207 ranges {686049=409317,1847413=438636} 2024-09-16T15:17:26.043 INFO:tasks.rados.rados.0.smithi162.stdout:3536: writing smithi16234619-46 from 686049 to 1095366 tid 1 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3536: writing smithi16234619-46 from 1847413 to 2286049 tid 2 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3537: read oid 27 snap -1 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3537: expect deleted 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3538: read oid 13 snap -1 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3538: expect (ObjNum 1173 snap 355 seq_num 1173) 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3539: rollback oid 43 current snap is 371 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 369 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3533: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:3533: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:17:26.051 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 1987 (ObjNum 1157 snap 345 seq_num 1157) dirty exists 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: finishing write tid 1 to smithi16234619-26 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: finishing write tid 2 to smithi16234619-26 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3533: done (5 left) 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3540: delete oid 25 current snap is 371 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: finishing write tid 3 to smithi16234619-26 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: finishing write tid 4 to smithi16234619-26 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: oid 26 updating version 0 to 2204 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: oid 26 version 2204 is already newer than 2202 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: oid 26 version 2204 is already newer than 2203 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2204 (ObjNum 1206 snap 371 seq_num 1206) dirty exists 2024-09-16T15:17:26.052 INFO:tasks.rados.rados.0.smithi162.stdout:3534: left oid 26 (ObjNum 1206 snap 371 seq_num 1206) 2024-09-16T15:17:26.053 INFO:tasks.rados.rados.0.smithi162.stdout:3539: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:17:26.053 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1173 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3534: done (5 left) 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3537: done (4 left) 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3539: done (3 left) 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3540: done (2 left) 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3541: write (excl) oid 10 current snap is 371 2024-09-16T15:17:26.055 INFO:tasks.rados.rados.0.smithi162.stdout:3541: seq_num 1209 ranges {719663=676168,1964377=355286} 2024-09-16T15:17:26.062 INFO:tasks.rados.rados.0.smithi162.stdout:3541: writing smithi16234619-10 from 719663 to 1395831 tid 1 2024-09-16T15:17:26.068 INFO:tasks.rados.rados.0.smithi162.stdout:3541: writing smithi16234619-10 from 1964377 to 2319663 tid 2 2024-09-16T15:17:26.068 INFO:tasks.rados.rados.0.smithi162.stdout:3542: snap_remove snap 369 2024-09-16T15:17:27.049 INFO:tasks.rados.rados.0.smithi162.stdout:3542: done (3 left) 2024-09-16T15:17:27.050 INFO:tasks.rados.rados.0.smithi162.stdout:3543: delete oid 30 current snap is 371 2024-09-16T15:17:27.050 INFO:tasks.rados.rados.0.smithi162.stdout:3538: expect (ObjNum 1173 snap 355 seq_num 1173) 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: finishing write tid 1 to smithi16234619-46 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: finishing write tid 2 to smithi16234619-46 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: finishing write tid 3 to smithi16234619-46 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: finishing write tid 4 to smithi16234619-46 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: oid 46 updating version 0 to 1217 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: oid 46 updating version 1217 to 1219 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: oid 46 version 1219 is already newer than 1218 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1219 (ObjNum 1207 snap 371 seq_num 1207) dirty exists 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3536: left oid 46 (ObjNum 1207 snap 371 seq_num 1207) 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3541: finishing write tid 1 to smithi16234619-10 2024-09-16T15:17:27.172 INFO:tasks.rados.rados.0.smithi162.stdout:3541: finishing write tid 2 to smithi16234619-10 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3536: done (3 left) 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3538: done (2 left) 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3543: done (1 left) 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3544: snap_create 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: finishing write tid 3 to smithi16234619-10 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: finishing write tid 4 to smithi16234619-10 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: oid 10 updating version 0 to 2212 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: oid 10 updating version 2212 to 2214 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: oid 10 version 2214 is already newer than 2213 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 2214 (ObjNum 1209 snap 371 seq_num 1209) dirty exists 2024-09-16T15:17:27.173 INFO:tasks.rados.rados.0.smithi162.stdout:3541: left oid 10 (ObjNum 1209 snap 371 seq_num 1209) 2024-09-16T15:17:28.047 INFO:tasks.rados.rados.0.smithi162.stdout:3541: done (1 left) 2024-09-16T15:17:28.047 INFO:tasks.rados.rados.0.smithi162.stdout:3544: done (0 left) 2024-09-16T15:17:28.047 INFO:tasks.rados.rados.0.smithi162.stdout:3545: snap_create 2024-09-16T15:17:29.018 INFO:tasks.rados.rados.0.smithi162.stdout:3545: done (0 left) 2024-09-16T15:17:29.019 INFO:tasks.rados.rados.0.smithi162.stdout:3546: write (excl) oid 36 current snap is 373 2024-09-16T15:17:29.019 INFO:tasks.rados.rados.0.smithi162.stdout:3546: seq_num 1211 ranges {753277=426666,1681341=545753} 2024-09-16T15:17:29.025 INFO:tasks.rados.rados.0.smithi162.stdout:3546: writing smithi16234619-36 from 753277 to 1179943 tid 1 2024-09-16T15:17:29.032 INFO:tasks.rados.rados.0.smithi162.stdout:3546: writing smithi16234619-36 from 1681341 to 2227094 tid 2 2024-09-16T15:17:29.032 INFO:tasks.rados.rados.0.smithi162.stdout:3547: snap_remove snap 351 2024-09-16T15:17:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:3547: done (1 left) 2024-09-16T15:17:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:3548: write oid 9 current snap is 373 2024-09-16T15:17:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:3546: finishing write tid 1 to smithi16234619-36 2024-09-16T15:17:30.068 INFO:tasks.rados.rados.0.smithi162.stdout:3548: seq_num 1212 ranges {770084=501915,2023470=346614} 2024-09-16T15:17:30.075 INFO:tasks.rados.rados.0.smithi162.stdout:3548: writing smithi16234619-9 from 770084 to 1271999 tid 1 2024-09-16T15:17:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:3548: writing smithi16234619-9 from 2023470 to 2370084 tid 2 2024-09-16T15:17:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:3549: delete oid 14 current snap is 373 2024-09-16T15:17:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:3546: finishing write tid 2 to smithi16234619-36 2024-09-16T15:17:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:3546: finishing write tid 3 to smithi16234619-36 2024-09-16T15:17:30.082 INFO:tasks.rados.rados.0.smithi162.stdout:3546: finishing write tid 4 to smithi16234619-36 2024-09-16T15:17:30.083 INFO:tasks.rados.rados.0.smithi162.stdout:3546: oid 36 updating version 0 to 1872 2024-09-16T15:17:30.083 INFO:tasks.rados.rados.0.smithi162.stdout:3546: oid 36 version 1872 is already newer than 1870 2024-09-16T15:17:30.083 INFO:tasks.rados.rados.0.smithi162.stdout:3546: oid 36 version 1872 is already newer than 1871 2024-09-16T15:17:30.083 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1872 (ObjNum 1211 snap 373 seq_num 1211) dirty exists 2024-09-16T15:17:30.083 INFO:tasks.rados.rados.0.smithi162.stdout:3546: left oid 36 (ObjNum 1211 snap 373 seq_num 1211) 2024-09-16T15:17:30.092 INFO:tasks.rados.rados.0.smithi162.stdout:3546: done (2 left) 2024-09-16T15:17:30.092 INFO:tasks.rados.rados.0.smithi162.stdout:3549: done (1 left) 2024-09-16T15:17:30.092 INFO:tasks.rados.rados.0.smithi162.stdout:3550: snap_create 2024-09-16T15:17:30.100 INFO:tasks.rados.rados.0.smithi162.stdout:3548: finishing write tid 1 to smithi16234619-9 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: finishing write tid 2 to smithi16234619-9 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: finishing write tid 3 to smithi16234619-9 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: finishing write tid 4 to smithi16234619-9 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: oid 9 updating version 0 to 1758 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: oid 9 version 1758 is already newer than 1757 2024-09-16T15:17:30.103 INFO:tasks.rados.rados.0.smithi162.stdout:3548: oid 9 version 1758 is already newer than 1756 2024-09-16T15:17:30.104 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1758 (ObjNum 1212 snap 373 seq_num 1212) dirty exists 2024-09-16T15:17:30.104 INFO:tasks.rados.rados.0.smithi162.stdout:3548: left oid 9 (ObjNum 1212 snap 373 seq_num 1212) 2024-09-16T15:17:31.078 INFO:tasks.rados.rados.0.smithi162.stdout:3548: done (1 left) 2024-09-16T15:17:31.078 INFO:tasks.rados.rados.0.smithi162.stdout:3550: done (0 left) 2024-09-16T15:17:31.078 INFO:tasks.rados.rados.0.smithi162.stdout:3551: write (excl) oid 8 current snap is 374 2024-09-16T15:17:31.078 INFO:tasks.rados.rados.0.smithi162.stdout:3551: seq_num 1214 ranges {403698=652413,1740434=663264} 2024-09-16T15:17:31.084 INFO:tasks.rados.rados.0.smithi162.stdout:3551: writing smithi16234619-8 from 403698 to 1056111 tid 1 2024-09-16T15:17:31.091 INFO:tasks.rados.rados.0.smithi162.stdout:3551: writing smithi16234619-8 from 1740434 to 2403698 tid 2 2024-09-16T15:17:31.091 INFO:tasks.rados.rados.0.smithi162.stdout:3552: write (excl) oid 30 current snap is 374 2024-09-16T15:17:31.091 INFO:tasks.rados.rados.0.smithi162.stdout:3552: seq_num 1215 ranges {420505=727662,1798916=553091} 2024-09-16T15:17:31.097 INFO:tasks.rados.rados.0.smithi162.stdout:3552: writing smithi16234619-30 from 420505 to 1148167 tid 1 2024-09-16T15:17:31.103 INFO:tasks.rados.rados.0.smithi162.stdout:3552: writing smithi16234619-30 from 1798916 to 2352007 tid 2 2024-09-16T15:17:31.103 INFO:tasks.rados.rados.0.smithi162.stdout:3553: rollback oid 2 current snap is 374 2024-09-16T15:17:31.104 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 316 2024-09-16T15:17:31.104 INFO:tasks.rados.rados.0.smithi162.stdout:3554: write (excl) oid 11 current snap is 374 2024-09-16T15:17:31.104 INFO:tasks.rados.rados.0.smithi162.stdout:3554: seq_num 1216 ranges {437312=402911,1341045=696749} 2024-09-16T15:17:31.108 INFO:tasks.rados.rados.0.smithi162.stdout:3554: writing smithi16234619-11 from 437312 to 840223 tid 1 2024-09-16T15:17:31.116 INFO:tasks.rados.rados.0.smithi162.stdout:3554: writing smithi16234619-11 from 1341045 to 2037794 tid 2 2024-09-16T15:17:31.116 INFO:tasks.rados.rados.0.smithi162.stdout:3555: write oid 6 current snap is 374 2024-09-16T15:17:31.117 INFO:tasks.rados.rados.0.smithi162.stdout:3555: seq_num 1217 ranges {454119=594513,1515880=556760} 2024-09-16T15:17:31.123 INFO:tasks.rados.rados.0.smithi162.stdout:3555: writing smithi16234619-6 from 454119 to 1048632 tid 1 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:3555: writing smithi16234619-6 from 1515880 to 2072640 tid 2 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:3556: snap_create 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:3553: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:3553: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:3553: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:17:31.129 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2234 (ObjNum 1011 snap 293 seq_num 1011) dirty exists 2024-09-16T15:17:31.136 INFO:tasks.rados.rados.0.smithi162.stdout:3551: finishing write tid 1 to smithi16234619-8 2024-09-16T15:17:31.136 INFO:tasks.rados.rados.0.smithi162.stdout:3551: finishing write tid 2 to smithi16234619-8 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:3551: finishing write tid 3 to smithi16234619-8 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:3551: finishing write tid 4 to smithi16234619-8 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:3551: oid 8 updating version 0 to 1505 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:3551: oid 8 version 1505 is already newer than 1504 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:3551: oid 8 version 1505 is already newer than 1503 2024-09-16T15:17:31.137 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1505 (ObjNum 1214 snap 374 seq_num 1214) dirty exists 2024-09-16T15:17:31.138 INFO:tasks.rados.rados.0.smithi162.stdout:3551: left oid 8 (ObjNum 1214 snap 374 seq_num 1214) 2024-09-16T15:17:31.142 INFO:tasks.rados.rados.0.smithi162.stdout:3552: finishing write tid 1 to smithi16234619-30 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: finishing write tid 2 to smithi16234619-30 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: finishing write tid 3 to smithi16234619-30 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: finishing write tid 4 to smithi16234619-30 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: oid 30 updating version 0 to 1813 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: oid 30 version 1813 is already newer than 1811 2024-09-16T15:17:31.143 INFO:tasks.rados.rados.0.smithi162.stdout:3552: oid 30 version 1813 is already newer than 1812 2024-09-16T15:17:31.144 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1813 (ObjNum 1215 snap 374 seq_num 1215) dirty exists 2024-09-16T15:17:31.144 INFO:tasks.rados.rados.0.smithi162.stdout:3552: left oid 30 (ObjNum 1215 snap 374 seq_num 1215) 2024-09-16T15:17:31.146 INFO:tasks.rados.rados.0.smithi162.stdout:3554: finishing write tid 1 to smithi16234619-11 2024-09-16T15:17:31.150 INFO:tasks.rados.rados.0.smithi162.stdout:3554: finishing write tid 2 to smithi16234619-11 2024-09-16T15:17:31.150 INFO:tasks.rados.rados.0.smithi162.stdout:3554: finishing write tid 3 to smithi16234619-11 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:3554: finishing write tid 4 to smithi16234619-11 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:3554: oid 11 updating version 0 to 1289 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:3554: oid 11 updating version 1289 to 1290 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:3554: oid 11 version 1290 is already newer than 1288 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1290 (ObjNum 1216 snap 374 seq_num 1216) dirty exists 2024-09-16T15:17:31.151 INFO:tasks.rados.rados.0.smithi162.stdout:3554: left oid 11 (ObjNum 1216 snap 374 seq_num 1216) 2024-09-16T15:17:31.161 INFO:tasks.rados.rados.0.smithi162.stdout:3555: finishing write tid 1 to smithi16234619-6 2024-09-16T15:17:31.161 INFO:tasks.rados.rados.0.smithi162.stdout:3555: finishing write tid 2 to smithi16234619-6 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: finishing write tid 3 to smithi16234619-6 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: finishing write tid 4 to smithi16234619-6 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: oid 6 updating version 0 to 1831 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: oid 6 updating version 1831 to 1832 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: oid 6 updating version 1832 to 1833 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1833 (ObjNum 1217 snap 374 seq_num 1217) dirty exists 2024-09-16T15:17:31.162 INFO:tasks.rados.rados.0.smithi162.stdout:3555: left oid 6 (ObjNum 1217 snap 374 seq_num 1217) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3551: done (5 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3552: done (4 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3553: done (3 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3554: done (2 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3555: done (1 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3556: done (0 left) 2024-09-16T15:17:32.083 INFO:tasks.rados.rados.0.smithi162.stdout:3557: snap_remove snap 345 2024-09-16T15:17:33.294 INFO:tasks.rados.rados.0.smithi162.stdout:3557: done (0 left) 2024-09-16T15:17:33.294 INFO:tasks.rados.rados.0.smithi162.stdout:3558: read oid 49 snap 359 2024-09-16T15:17:33.294 INFO:tasks.rados.rados.0.smithi162.stdout:3558: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:17:33.294 INFO:tasks.rados.rados.0.smithi162.stdout:3559: write oid 29 current snap is 375 2024-09-16T15:17:33.294 INFO:tasks.rados.rados.0.smithi162.stdout:3559: seq_num 1218 ranges {470926=669762,1574362=700418} 2024-09-16T15:17:33.300 INFO:tasks.rados.rados.0.smithi162.stdout:3559: writing smithi16234619-29 from 470926 to 1140688 tid 1 2024-09-16T15:17:33.307 INFO:tasks.rados.rados.0.smithi162.stdout:3559: writing smithi16234619-29 from 1574362 to 2274780 tid 2 2024-09-16T15:17:33.307 INFO:tasks.rados.rados.0.smithi162.stdout:3560: snap_remove snap 323 2024-09-16T15:17:34.019 INFO:tasks.rados.rados.0.smithi162.stdout:3560: done (2 left) 2024-09-16T15:17:34.019 INFO:tasks.rados.rados.0.smithi162.stdout:3561: snap_remove snap 346 2024-09-16T15:17:35.032 INFO:tasks.rados.rados.0.smithi162.stdout:3561: done (2 left) 2024-09-16T15:17:35.032 INFO:tasks.rados.rados.0.smithi162.stdout:3562: write oid 30 current snap is 375 2024-09-16T15:17:35.032 INFO:tasks.rados.rados.0.smithi162.stdout:3562: seq_num 1219 ranges {487733=745011,1632844=560429} 2024-09-16T15:17:35.039 INFO:tasks.rados.rados.0.smithi162.stdout:3562: writing smithi16234619-30 from 487733 to 1232744 tid 1 2024-09-16T15:17:35.045 INFO:tasks.rados.rados.0.smithi162.stdout:3562: writing smithi16234619-30 from 1632844 to 2193273 tid 2 2024-09-16T15:17:35.045 INFO:tasks.rados.rados.0.smithi162.stdout:3563: write oid 41 current snap is 375 2024-09-16T15:17:35.045 INFO:tasks.rados.rados.0.smithi162.stdout:3563: seq_num 1220 ranges {504540=420260,1574973=704087} 2024-09-16T15:17:35.050 INFO:tasks.rados.rados.0.smithi162.stdout:3563: writing smithi16234619-41 from 504540 to 924800 tid 1 2024-09-16T15:17:35.058 INFO:tasks.rados.rados.0.smithi162.stdout:3563: writing smithi16234619-41 from 1574973 to 2279060 tid 2 2024-09-16T15:17:35.058 INFO:tasks.rados.rados.0.smithi162.stdout:3564: read oid 24 snap -1 2024-09-16T15:17:35.058 INFO:tasks.rados.rados.0.smithi162.stdout:3564: expect deleted 2024-09-16T15:17:35.058 INFO:tasks.rados.rados.0.smithi162.stdout:3565: snap_create 2024-09-16T15:17:35.058 INFO:tasks.rados.rados.0.smithi162.stdout:3558: expect (ObjNum 981 snap 276 seq_num 981) 2024-09-16T15:17:35.235 INFO:tasks.rados.rados.0.smithi162.stdout:3559: finishing write tid 1 to smithi16234619-29 2024-09-16T15:17:35.235 INFO:tasks.rados.rados.0.smithi162.stdout:3559: finishing write tid 2 to smithi16234619-29 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: finishing write tid 3 to smithi16234619-29 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: finishing write tid 4 to smithi16234619-29 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: oid 29 updating version 0 to 2007 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: oid 29 updating version 2007 to 2009 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: oid 29 version 2009 is already newer than 2008 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 2009 (ObjNum 1218 snap 375 seq_num 1218) dirty exists 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3559: left oid 29 (ObjNum 1218 snap 375 seq_num 1218) 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: finishing write tid 1 to smithi16234619-41 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: finishing write tid 2 to smithi16234619-41 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: finishing write tid 3 to smithi16234619-41 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: finishing write tid 4 to smithi16234619-41 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: oid 41 updating version 0 to 1491 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: oid 41 version 1491 is already newer than 1490 2024-09-16T15:17:35.236 INFO:tasks.rados.rados.0.smithi162.stdout:3563: oid 41 version 1491 is already newer than 1489 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 1491 (ObjNum 1220 snap 375 seq_num 1220) dirty exists 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3563: left oid 41 (ObjNum 1220 snap 375 seq_num 1220) 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: finishing write tid 1 to smithi16234619-30 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: finishing write tid 2 to smithi16234619-30 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: finishing write tid 3 to smithi16234619-30 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: finishing write tid 4 to smithi16234619-30 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: oid 30 updating version 0 to 1816 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: oid 30 version 1816 is already newer than 1815 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: oid 30 version 1816 is already newer than 1814 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1816 (ObjNum 1219 snap 375 seq_num 1219) dirty exists 2024-09-16T15:17:35.237 INFO:tasks.rados.rados.0.smithi162.stdout:3562: left oid 30 (ObjNum 1219 snap 375 seq_num 1219) 2024-09-16T15:17:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3558: done (5 left) 2024-09-16T15:17:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3559: done (4 left) 2024-09-16T15:17:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3562: done (3 left) 2024-09-16T15:17:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3563: done (2 left) 2024-09-16T15:17:36.033 INFO:tasks.rados.rados.0.smithi162.stdout:3564: done (1 left) 2024-09-16T15:17:36.034 INFO:tasks.rados.rados.0.smithi162.stdout:3565: done (0 left) 2024-09-16T15:17:36.034 INFO:tasks.rados.rados.0.smithi162.stdout:3566: delete oid 28 current snap is 376 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:3566: done (0 left) 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:3567: read oid 34 snap -1 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:3567: expect (ObjNum 1201 snap 369 seq_num 1201) 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:3568: rollback oid 23 current snap is 376 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 365 2024-09-16T15:17:36.038 INFO:tasks.rados.rados.0.smithi162.stdout:3569: snap_remove snap 372 2024-09-16T15:17:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:3569: done (2 left) 2024-09-16T15:17:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:3570: copy_from oid 47 from oid 49 current snap is 376 2024-09-16T15:17:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:3568: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:17:37.041 INFO:tasks.rados.rados.0.smithi162.stdout:3567: expect (ObjNum 1201 snap 369 seq_num 1201) 2024-09-16T15:17:37.113 INFO:tasks.rados.rados.0.smithi162.stdout:3568: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:17:37.113 INFO:tasks.rados.rados.0.smithi162.stdout:3568: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:17:37.113 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 1483 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3570: finishing copy_from to smithi16234619-47 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3570: got expected ENOENT (src dne) 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3570: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3567: done (2 left) 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3568: done (1 left) 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3570: done (0 left) 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3571: rollback oid 34 current snap is 376 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 34 to 358 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3572: read oid 14 snap 287 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3572: expect (ObjNum 959 snap 269 seq_num 959) 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3573: copy_from oid 21 from oid 15 current snap is 376 2024-09-16T15:17:37.114 INFO:tasks.rados.rados.0.smithi162.stdout:3574: snap_remove snap 368 2024-09-16T15:17:38.499 INFO:tasks.rados.rados.0.smithi162.stdout:3571: finishing rollback tid 0 to smithi16234619-34 2024-09-16T15:17:38.499 INFO:tasks.rados.rados.0.smithi162.stdout:3571: finishing rollback tid 1 to smithi16234619-34 2024-09-16T15:17:38.500 INFO:tasks.rados.rados.0.smithi162.stdout:3571: finishing rollback tid 2 to smithi16234619-34 2024-09-16T15:17:38.500 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1928 (ObjNum 1109 snap 332 seq_num 1109) dirty exists 2024-09-16T15:17:38.500 INFO:tasks.rados.rados.0.smithi162.stdout:3572: expect (ObjNum 959 snap 269 seq_num 959) 2024-09-16T15:17:38.616 INFO:tasks.rados.rados.0.smithi162.stdout:3573: finishing copy_from racing read to smithi16234619-21 2024-09-16T15:17:38.616 INFO:tasks.rados.rados.0.smithi162.stdout:3573: finishing copy_from to smithi16234619-21 2024-09-16T15:17:38.616 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 1809 (ObjNum 1196 snap 365 seq_num 1196) dirty exists 2024-09-16T15:17:38.616 INFO:tasks.rados.rados.0.smithi162.stdout:3571: done (3 left) 2024-09-16T15:17:38.617 INFO:tasks.rados.rados.0.smithi162.stdout:3572: done (2 left) 2024-09-16T15:17:38.617 INFO:tasks.rados.rados.0.smithi162.stdout:3573: done (1 left) 2024-09-16T15:17:38.617 INFO:tasks.rados.rados.0.smithi162.stdout:3574: done (0 left) 2024-09-16T15:17:38.617 INFO:tasks.rados.rados.0.smithi162.stdout:3575: write oid 40 current snap is 376 2024-09-16T15:17:38.617 INFO:tasks.rados.rados.0.smithi162.stdout:3575: seq_num 1222 ranges {538154=570758,1691937=707756} 2024-09-16T15:17:38.622 INFO:tasks.rados.rados.0.smithi162.stdout:3575: writing smithi16234619-40 from 538154 to 1108912 tid 1 2024-09-16T15:17:38.629 INFO:tasks.rados.rados.0.smithi162.stdout:3575: writing smithi16234619-40 from 1691937 to 2399693 tid 2 2024-09-16T15:17:38.629 INFO:tasks.rados.rados.0.smithi162.stdout:3576: snap_remove snap 338 2024-09-16T15:17:39.015 INFO:tasks.rados.rados.0.smithi162.stdout:3576: done (1 left) 2024-09-16T15:17:39.015 INFO:tasks.rados.rados.0.smithi162.stdout:3577: delete oid 33 current snap is 376 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: finishing write tid 1 to smithi16234619-40 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: finishing write tid 2 to smithi16234619-40 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: finishing write tid 3 to smithi16234619-40 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: finishing write tid 4 to smithi16234619-40 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: oid 40 updating version 0 to 1488 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: oid 40 updating version 1488 to 1489 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: oid 40 version 1489 is already newer than 1487 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1489 (ObjNum 1222 snap 376 seq_num 1222) dirty exists 2024-09-16T15:17:39.016 INFO:tasks.rados.rados.0.smithi162.stdout:3575: left oid 40 (ObjNum 1222 snap 376 seq_num 1222) 2024-09-16T15:17:39.023 INFO:tasks.rados.rados.0.smithi162.stdout:3575: done (1 left) 2024-09-16T15:17:39.023 INFO:tasks.rados.rados.0.smithi162.stdout:3577: done (0 left) 2024-09-16T15:17:39.023 INFO:tasks.rados.rados.0.smithi162.stdout:3578: write (excl) oid 23 current snap is 376 2024-09-16T15:17:39.023 INFO:tasks.rados.rados.0.smithi162.stdout:3578: seq_num 1224 ranges {571768=437609,1808901=711425} 2024-09-16T15:17:39.029 INFO:tasks.rados.rados.0.smithi162.stdout:3578: writing smithi16234619-23 from 571768 to 1009377 tid 1 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:3578: writing smithi16234619-23 from 1808901 to 2520326 tid 2 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:3579: rollback oid 43 current snap is 376 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 375 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:3580: read oid 39 snap -1 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:3580: expect (ObjNum 1162 snap 346 seq_num 1162) 2024-09-16T15:17:39.040 INFO:tasks.rados.rados.0.smithi162.stdout:3581: delete oid 25 current snap is 376 2024-09-16T15:17:39.053 INFO:tasks.rados.rados.0.smithi162.stdout:3579: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:17:39.053 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 1173 (ObjNum 129 snap 0 seq_num 1838677774) dirty dne 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3579: done (3 left) 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3581: done (2 left) 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3582: read oid 37 snap -1 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3582: expect (ObjNum 1200 snap 366 seq_num 1200) 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3583: read oid 14 snap -1 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3583: expect deleted 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3584: rollback oid 26 current snap is 376 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 26 to 374 2024-09-16T15:17:39.059 INFO:tasks.rados.rados.0.smithi162.stdout:3585: copy_from oid 47 from oid 50 current snap is 376 2024-09-16T15:17:39.060 INFO:tasks.rados.rados.0.smithi162.stdout:3586: read oid 8 snap -1 2024-09-16T15:17:39.060 INFO:tasks.rados.rados.0.smithi162.stdout:3586: expect (ObjNum 1214 snap 374 seq_num 1214) 2024-09-16T15:17:39.060 INFO:tasks.rados.rados.0.smithi162.stdout:3587: snap_create 2024-09-16T15:17:39.060 INFO:tasks.rados.rados.0.smithi162.stdout:3580: expect (ObjNum 1162 snap 346 seq_num 1162) 2024-09-16T15:17:39.267 INFO:tasks.rados.rados.0.smithi162.stdout:3578: finishing write tid 1 to smithi16234619-23 2024-09-16T15:17:39.268 INFO:tasks.rados.rados.0.smithi162.stdout:3584: finishing rollback tid 0 to smithi16234619-26 2024-09-16T15:17:39.268 INFO:tasks.rados.rados.0.smithi162.stdout:3578: finishing write tid 2 to smithi16234619-23 2024-09-16T15:17:39.268 INFO:tasks.rados.rados.0.smithi162.stdout:3582: expect (ObjNum 1200 snap 366 seq_num 1200) 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3584: finishing rollback tid 1 to smithi16234619-26 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3584: finishing rollback tid 2 to smithi16234619-26 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2288 (ObjNum 1206 snap 371 seq_num 1206) dirty exists 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: finishing write tid 3 to smithi16234619-23 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: finishing write tid 4 to smithi16234619-23 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: oid 23 updating version 0 to 1485 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: oid 23 version 1485 is already newer than 1484 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: oid 23 updating version 1485 to 1486 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 1486 (ObjNum 1224 snap 376 seq_num 1224) dirty exists 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3578: left oid 23 (ObjNum 1224 snap 376 seq_num 1224) 2024-09-16T15:17:39.356 INFO:tasks.rados.rados.0.smithi162.stdout:3586: expect (ObjNum 1214 snap 374 seq_num 1214) 2024-09-16T15:17:39.429 INFO:tasks.rados.rados.0.smithi162.stdout:3585: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:17:39.429 INFO:tasks.rados.rados.0.smithi162.stdout:3585: finishing copy_from to smithi16234619-47 2024-09-16T15:17:39.429 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 1738 (ObjNum 1175 snap 357 seq_num 1175) dirty exists 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3578: done (7 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3580: done (6 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3582: done (5 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3583: done (4 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3584: done (3 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3585: done (2 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3586: done (1 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3587: done (0 left) 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3588: read oid 21 snap -1 2024-09-16T15:17:40.043 INFO:tasks.rados.rados.0.smithi162.stdout:3588: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:40.044 INFO:tasks.rados.rados.0.smithi162.stdout:3589: snap_remove snap 287 2024-09-16T15:17:41.046 INFO:tasks.rados.rados.0.smithi162.stdout:3589: done (1 left) 2024-09-16T15:17:41.047 INFO:tasks.rados.rados.0.smithi162.stdout:3590: snap_remove snap 359 2024-09-16T15:17:42.054 INFO:tasks.rados.rados.0.smithi162.stdout:3590: done (1 left) 2024-09-16T15:17:42.055 INFO:tasks.rados.rados.0.smithi162.stdout:3591: copy_from oid 18 from oid 49 current snap is 377 2024-09-16T15:17:42.055 INFO:tasks.rados.rados.0.smithi162.stdout:3588: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:42.125 INFO:tasks.rados.rados.0.smithi162.stdout:3591: finishing copy_from to smithi16234619-18 2024-09-16T15:17:42.125 INFO:tasks.rados.rados.0.smithi162.stdout:3591: got expected ENOENT (src dne) 2024-09-16T15:17:42.126 INFO:tasks.rados.rados.0.smithi162.stdout:3591: finishing copy_from racing read to smithi16234619-18 2024-09-16T15:17:42.126 INFO:tasks.rados.rados.0.smithi162.stdout:3588: done (1 left) 2024-09-16T15:17:42.126 INFO:tasks.rados.rados.0.smithi162.stdout:3591: done (0 left) 2024-09-16T15:17:42.126 INFO:tasks.rados.rados.0.smithi162.stdout:3592: delete oid 19 current snap is 377 2024-09-16T15:17:42.129 INFO:tasks.rados.rados.0.smithi162.stdout:3592: done (0 left) 2024-09-16T15:17:42.129 INFO:tasks.rados.rados.0.smithi162.stdout:3593: snap_create 2024-09-16T15:17:43.073 INFO:tasks.rados.rados.0.smithi162.stdout:3593: done (0 left) 2024-09-16T15:17:43.073 INFO:tasks.rados.rados.0.smithi162.stdout:3594: copy_from oid 47 from oid 43 current snap is 378 2024-09-16T15:17:43.073 INFO:tasks.rados.rados.0.smithi162.stdout:3595: snap_create 2024-09-16T15:17:43.074 INFO:tasks.rados.rados.0.smithi162.stdout:3594: finishing copy_from to smithi16234619-47 2024-09-16T15:17:43.074 INFO:tasks.rados.rados.0.smithi162.stdout:3594: got expected ENOENT (src dne) 2024-09-16T15:17:43.074 INFO:tasks.rados.rados.0.smithi162.stdout:3594: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:17:44.022 INFO:tasks.rados.rados.0.smithi162.stdout:3594: done (1 left) 2024-09-16T15:17:44.022 INFO:tasks.rados.rados.0.smithi162.stdout:3595: done (0 left) 2024-09-16T15:17:44.022 INFO:tasks.rados.rados.0.smithi162.stdout:3596: snap_create 2024-09-16T15:17:45.031 INFO:tasks.rados.rados.0.smithi162.stdout:3596: done (0 left) 2024-09-16T15:17:45.031 INFO:tasks.rados.rados.0.smithi162.stdout:3597: snap_remove snap 373 2024-09-16T15:17:46.036 INFO:tasks.rados.rados.0.smithi162.stdout:3597: done (0 left) 2024-09-16T15:17:46.036 INFO:tasks.rados.rados.0.smithi162.stdout:3598: snap_create 2024-09-16T15:17:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:3598: done (0 left) 2024-09-16T15:17:47.434 INFO:tasks.rados.rados.0.smithi162.stdout:3599: snap_create 2024-09-16T15:17:48.460 INFO:tasks.rados.rados.0.smithi162.stdout:3599: done (0 left) 2024-09-16T15:17:48.460 INFO:tasks.rados.rados.0.smithi162.stdout:3600: read oid 17 snap -1 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3600: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3601: read oid 39 snap -1 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3601: expect (ObjNum 1162 snap 346 seq_num 1162) 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3602: copy_from oid 35 from oid 45 current snap is 382 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3603: rollback oid 49 current snap is 382 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 375 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3604: write oid 34 current snap is 382 2024-09-16T15:17:48.461 INFO:tasks.rados.rados.0.smithi162.stdout:3604: seq_num 1227 ranges {622189=663356,1984347=458752} 2024-09-16T15:17:48.467 INFO:tasks.rados.rados.0.smithi162.stdout:3604: writing smithi16234619-34 from 622189 to 1285545 tid 1 2024-09-16T15:17:48.474 INFO:tasks.rados.rados.0.smithi162.stdout:3604: writing smithi16234619-34 from 1984347 to 2443099 tid 2 2024-09-16T15:17:48.474 INFO:tasks.rados.rados.0.smithi162.stdout:3605: snap_create 2024-09-16T15:17:48.475 INFO:tasks.rados.rados.0.smithi162.stdout:3603: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:17:48.475 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 1485 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:17:48.475 INFO:tasks.rados.rados.0.smithi162.stdout:3600: expect (ObjNum 1118 snap 333 seq_num 1118) 2024-09-16T15:17:48.566 INFO:tasks.rados.rados.0.smithi162.stdout:3601: expect (ObjNum 1162 snap 346 seq_num 1162) 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3602: finishing copy_from to smithi16234619-35 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3602: got expected ENOENT (src dne) 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3602: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: finishing write tid 1 to smithi16234619-34 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: finishing write tid 2 to smithi16234619-34 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: finishing write tid 3 to smithi16234619-34 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: finishing write tid 4 to smithi16234619-34 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: oid 34 updating version 0 to 1968 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: oid 34 version 1968 is already newer than 1967 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: oid 34 updating version 1968 to 1969 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1969 (ObjNum 1227 snap 382 seq_num 1227) dirty exists 2024-09-16T15:17:48.769 INFO:tasks.rados.rados.0.smithi162.stdout:3604: left oid 34 (ObjNum 1227 snap 382 seq_num 1227) 2024-09-16T15:17:49.022 INFO:tasks.rados.rados.0.smithi162.stdout:3600: done (5 left) 2024-09-16T15:17:49.022 INFO:tasks.rados.rados.0.smithi162.stdout:3601: done (4 left) 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3602: done (3 left) 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3603: done (2 left) 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3604: done (1 left) 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3605: done (0 left) 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3606: write oid 10 current snap is 383 2024-09-16T15:17:49.023 INFO:tasks.rados.rados.0.smithi162.stdout:3606: seq_num 1228 ranges {638996=738605,1926476=712520} 2024-09-16T15:17:49.030 INFO:tasks.rados.rados.0.smithi162.stdout:3606: writing smithi16234619-10 from 638996 to 1377601 tid 1 2024-09-16T15:17:49.037 INFO:tasks.rados.rados.0.smithi162.stdout:3606: writing smithi16234619-10 from 1926476 to 2638996 tid 2 2024-09-16T15:17:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:3607: read oid 14 snap -1 2024-09-16T15:17:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:3607: expect deleted 2024-09-16T15:17:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:3608: copy_from oid 16 from oid 4 current snap is 383 2024-09-16T15:17:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:3609: write (excl) oid 1 current snap is 383 2024-09-16T15:17:49.038 INFO:tasks.rados.rados.0.smithi162.stdout:3609: seq_num 1229 ranges {655803=413854,1584958=462421} 2024-09-16T15:17:49.043 INFO:tasks.rados.rados.0.smithi162.stdout:3609: writing smithi16234619-1 from 655803 to 1069657 tid 1 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3609: writing smithi16234619-1 from 1584958 to 2047379 tid 2 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3610: read oid 15 snap -1 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3610: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3611: copy_from oid 32 from oid 25 current snap is 383 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3608: finishing copy_from to smithi16234619-16 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3608: got expected ENOENT (src dne) 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3608: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3607: done (5 left) 2024-09-16T15:17:49.051 INFO:tasks.rados.rados.0.smithi162.stdout:3608: done (4 left) 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3612: read oid 21 snap -1 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3612: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3613: rollback oid 7 current snap is 383 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 360 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3614: copy_from oid 3 from oid 18 current snap is 383 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3615: read oid 19 snap -1 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3615: expect deleted 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3616: write oid 42 current snap is 383 2024-09-16T15:17:49.052 INFO:tasks.rados.rados.0.smithi162.stdout:3616: seq_num 1230 ranges {672610=489103,1643440=722432} 2024-09-16T15:17:49.058 INFO:tasks.rados.rados.0.smithi162.stdout:3616: writing smithi16234619-42 from 672610 to 1161713 tid 1 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3616: writing smithi16234619-42 from 1643440 to 2365872 tid 2 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3617: copy_from oid 17 from oid 22 current snap is 383 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3611: finishing copy_from to smithi16234619-32 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3611: got expected ENOENT (src dne) 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3611: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3614: finishing copy_from to smithi16234619-3 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3614: got expected ENOENT (src dne) 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3614: finishing copy_from racing read to smithi16234619-3 2024-09-16T15:17:49.067 INFO:tasks.rados.rados.0.smithi162.stdout:3610: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:49.139 INFO:tasks.rados.rados.0.smithi162.stdout:3613: finishing rollback tid 0 to smithi16234619-7 2024-09-16T15:17:49.139 INFO:tasks.rados.rados.0.smithi162.stdout:3613: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:17:49.139 INFO:tasks.rados.rados.0.smithi162.stdout:3613: finishing rollback tid 2 to smithi16234619-7 2024-09-16T15:17:49.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1508 (ObjNum 1155 snap 345 seq_num 1155) dirty exists 2024-09-16T15:17:49.139 INFO:tasks.rados.rados.0.smithi162.stdout:3606: finishing write tid 1 to smithi16234619-10 2024-09-16T15:17:49.140 INFO:tasks.rados.rados.0.smithi162.stdout:3606: finishing write tid 2 to smithi16234619-10 2024-09-16T15:17:49.140 INFO:tasks.rados.rados.0.smithi162.stdout:3612: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: finishing write tid 3 to smithi16234619-10 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: finishing write tid 4 to smithi16234619-10 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: oid 10 updating version 0 to 2320 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: oid 10 version 2320 is already newer than 2319 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: oid 10 version 2320 is already newer than 2318 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 2320 (ObjNum 1228 snap 383 seq_num 1228) dirty exists 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3606: left oid 10 (ObjNum 1228 snap 383 seq_num 1228) 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3609: finishing write tid 1 to smithi16234619-1 2024-09-16T15:17:49.206 INFO:tasks.rados.rados.0.smithi162.stdout:3609: finishing write tid 2 to smithi16234619-1 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3609: finishing write tid 3 to smithi16234619-1 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3606: done (9 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3610: done (8 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3611: done (7 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3612: done (6 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3613: done (5 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3614: done (4 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3615: done (3 left) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3618: read oid 38 snap -1 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3618: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3619: rollback oid 9 current snap is 383 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 9 to 361 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3609: finishing write tid 4 to smithi16234619-1 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3609: oid 1 updating version 0 to 1759 2024-09-16T15:17:49.207 INFO:tasks.rados.rados.0.smithi162.stdout:3609: oid 1 updating version 1759 to 1761 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3609: oid 1 version 1761 is already newer than 1760 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1761 (ObjNum 1229 snap 383 seq_num 1229) dirty exists 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3609: left oid 1 (ObjNum 1229 snap 383 seq_num 1229) 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: finishing write tid 1 to smithi16234619-42 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: finishing write tid 2 to smithi16234619-42 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: finishing write tid 3 to smithi16234619-42 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: finishing write tid 4 to smithi16234619-42 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: oid 42 updating version 0 to 2055 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: oid 42 version 2055 is already newer than 2053 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: oid 42 version 2055 is already newer than 2054 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 2055 (ObjNum 1230 snap 383 seq_num 1230) dirty exists 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3616: left oid 42 (ObjNum 1230 snap 383 seq_num 1230) 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3617: finishing copy_from racing read to smithi16234619-17 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:3617: finishing copy_from to smithi16234619-17 2024-09-16T15:17:49.208 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 17 v 1488 (ObjNum 1197 snap 365 seq_num 1197) dirty exists 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3609: done (4 left) 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3616: done (3 left) 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3617: done (2 left) 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3620: delete oid 11 current snap is 383 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3620: done (2 left) 2024-09-16T15:17:49.209 INFO:tasks.rados.rados.0.smithi162.stdout:3621: copy_from oid 27 from oid 45 current snap is 383 2024-09-16T15:17:49.210 INFO:tasks.rados.rados.0.smithi162.stdout:3622: write (excl) oid 44 current snap is 383 2024-09-16T15:17:49.210 INFO:tasks.rados.rados.0.smithi162.stdout:3622: seq_num 1232 ranges {706224=755954,2160404=545820} 2024-09-16T15:17:49.217 INFO:tasks.rados.rados.0.smithi162.stdout:3622: writing smithi16234619-44 from 706224 to 1462178 tid 1 2024-09-16T15:17:49.224 INFO:tasks.rados.rados.0.smithi162.stdout:3622: writing smithi16234619-44 from 2160404 to 2706224 tid 2 2024-09-16T15:17:49.224 INFO:tasks.rados.rados.0.smithi162.stdout:3623: read oid 10 snap -1 2024-09-16T15:17:49.224 INFO:tasks.rados.rados.0.smithi162.stdout:3623: expect (ObjNum 1228 snap 383 seq_num 1228) 2024-09-16T15:17:49.224 INFO:tasks.rados.rados.0.smithi162.stdout:3624: snap_remove snap 376 2024-09-16T15:17:50.031 INFO:tasks.rados.rados.0.smithi162.stdout:3624: done (5 left) 2024-09-16T15:17:50.031 INFO:tasks.rados.rados.0.smithi162.stdout:3625: read oid 47 snap -1 2024-09-16T15:17:50.031 INFO:tasks.rados.rados.0.smithi162.stdout:3625: expect (ObjNum 1175 snap 357 seq_num 1175) 2024-09-16T15:17:50.031 INFO:tasks.rados.rados.0.smithi162.stdout:3626: write oid 34 current snap is 383 2024-09-16T15:17:50.031 INFO:tasks.rados.rados.0.smithi162.stdout:3618: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3619: finishing rollback tid 0 to smithi16234619-9 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3619: finishing rollback tid 1 to smithi16234619-9 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3619: finishing rollback tid 2 to smithi16234619-9 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 1764 (ObjNum 1151 snap 343 seq_num 1151) dirty exists 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3621: finishing copy_from to smithi16234619-27 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3621: got expected ENOENT (src dne) 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3621: finishing copy_from racing read to smithi16234619-27 2024-09-16T15:17:50.158 INFO:tasks.rados.rados.0.smithi162.stdout:3623: expect (ObjNum 1228 snap 383 seq_num 1228) 2024-09-16T15:17:50.331 INFO:tasks.rados.rados.0.smithi162.stdout:3622: finishing write tid 1 to smithi16234619-44 2024-09-16T15:17:50.331 INFO:tasks.rados.rados.0.smithi162.stdout:3622: finishing write tid 2 to smithi16234619-44 2024-09-16T15:17:50.331 INFO:tasks.rados.rados.0.smithi162.stdout:3622: finishing write tid 3 to smithi16234619-44 2024-09-16T15:17:50.331 INFO:tasks.rados.rados.0.smithi162.stdout:3622: finishing write tid 4 to smithi16234619-44 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:3622: oid 44 updating version 0 to 1834 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:3622: oid 44 updating version 1834 to 1835 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:3622: oid 44 updating version 1835 to 1836 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 44 v 1836 (ObjNum 1232 snap 383 seq_num 1232) dirty exists 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:3622: left oid 44 (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:17:50.332 INFO:tasks.rados.rados.0.smithi162.stdout:3625: expect (ObjNum 1175 snap 357 seq_num 1175) 2024-09-16T15:17:50.582 INFO:tasks.rados.rados.0.smithi162.stdout:3626: seq_num 1233 ranges {723031=431203,1818886=469759} 2024-09-16T15:17:50.588 INFO:tasks.rados.rados.0.smithi162.stdout:3626: writing smithi16234619-34 from 723031 to 1154234 tid 1 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3626: writing smithi16234619-34 from 1818886 to 2288645 tid 2 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3618: done (6 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3619: done (5 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3621: done (4 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3622: done (3 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3623: done (2 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3625: done (1 left) 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3627: write oid 39 current snap is 383 2024-09-16T15:17:50.595 INFO:tasks.rados.rados.0.smithi162.stdout:3627: seq_num 1234 ranges {739838=506452,1877368=613417} 2024-09-16T15:17:50.601 INFO:tasks.rados.rados.0.smithi162.stdout:3627: writing smithi16234619-39 from 739838 to 1246290 tid 1 2024-09-16T15:17:50.608 INFO:tasks.rados.rados.0.smithi162.stdout:3627: writing smithi16234619-39 from 1877368 to 2490785 tid 2 2024-09-16T15:17:50.608 INFO:tasks.rados.rados.0.smithi162.stdout:3628: write (excl) oid 21 current snap is 383 2024-09-16T15:17:50.608 INFO:tasks.rados.rados.0.smithi162.stdout:3628: seq_num 1235 ranges {756645=581701,1935850=473428} 2024-09-16T15:17:50.615 INFO:tasks.rados.rados.0.smithi162.stdout:3628: writing smithi16234619-21 from 756645 to 1338346 tid 1 2024-09-16T15:17:50.621 INFO:tasks.rados.rados.0.smithi162.stdout:3628: writing smithi16234619-21 from 1935850 to 2409278 tid 2 2024-09-16T15:17:50.621 INFO:tasks.rados.rados.0.smithi162.stdout:3629: delete oid 37 current snap is 383 2024-09-16T15:17:50.623 INFO:tasks.rados.rados.0.smithi162.stdout:3626: finishing write tid 1 to smithi16234619-34 2024-09-16T15:17:50.623 INFO:tasks.rados.rados.0.smithi162.stdout:3626: finishing write tid 2 to smithi16234619-34 2024-09-16T15:17:50.623 INFO:tasks.rados.rados.0.smithi162.stdout:3626: finishing write tid 3 to smithi16234619-34 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:3626: finishing write tid 4 to smithi16234619-34 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:3626: oid 34 updating version 0 to 1977 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:3626: oid 34 version 1977 is already newer than 1976 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:3626: oid 34 version 1977 is already newer than 1975 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 1977 (ObjNum 1233 snap 383 seq_num 1233) dirty exists 2024-09-16T15:17:50.624 INFO:tasks.rados.rados.0.smithi162.stdout:3626: left oid 34 (ObjNum 1233 snap 383 seq_num 1233) 2024-09-16T15:17:50.632 INFO:tasks.rados.rados.0.smithi162.stdout:3627: finishing write tid 1 to smithi16234619-39 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: finishing write tid 2 to smithi16234619-39 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: finishing write tid 3 to smithi16234619-39 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: finishing write tid 4 to smithi16234619-39 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: oid 39 updating version 0 to 2060 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: oid 39 version 2060 is already newer than 2059 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: oid 39 updating version 2060 to 2061 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 2061 (ObjNum 1234 snap 383 seq_num 1234) dirty exists 2024-09-16T15:17:50.634 INFO:tasks.rados.rados.0.smithi162.stdout:3627: left oid 39 (ObjNum 1234 snap 383 seq_num 1234) 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3626: done (3 left) 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3627: done (2 left) 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3629: done (1 left) 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3630: copy_from oid 8 from oid 32 current snap is 383 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3631: write oid 36 current snap is 383 2024-09-16T15:17:50.637 INFO:tasks.rados.rados.0.smithi162.stdout:3631: seq_num 1237 ranges {790259=732199,1936461=477097} 2024-09-16T15:17:50.645 INFO:tasks.rados.rados.0.smithi162.stdout:3631: writing smithi16234619-36 from 790259 to 1522458 tid 1 2024-09-16T15:17:50.650 INFO:tasks.rados.rados.0.smithi162.stdout:3631: writing smithi16234619-36 from 1936461 to 2413558 tid 2 2024-09-16T15:17:50.650 INFO:tasks.rados.rados.0.smithi162.stdout:3632: delete oid 22 current snap is 383 2024-09-16T15:17:50.650 INFO:tasks.rados.rados.0.smithi162.stdout:3628: finishing write tid 1 to smithi16234619-21 2024-09-16T15:17:50.650 INFO:tasks.rados.rados.0.smithi162.stdout:3628: finishing write tid 2 to smithi16234619-21 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: finishing write tid 3 to smithi16234619-21 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: finishing write tid 4 to smithi16234619-21 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: oid 21 updating version 0 to 2328 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: oid 21 updating version 2328 to 2329 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: oid 21 updating version 2329 to 2330 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 2330 (ObjNum 1235 snap 383 seq_num 1235) dirty exists 2024-09-16T15:17:50.651 INFO:tasks.rados.rados.0.smithi162.stdout:3628: left oid 21 (ObjNum 1235 snap 383 seq_num 1235) 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3628: done (3 left) 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3632: done (2 left) 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3633: read oid 12 snap -1 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3633: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3634: copy_from oid 49 from oid 5 current snap is 383 2024-09-16T15:17:50.654 INFO:tasks.rados.rados.0.smithi162.stdout:3635: read oid 20 snap 378 2024-09-16T15:17:50.655 INFO:tasks.rados.rados.0.smithi162.stdout:3635: expect (ObjNum 1138 snap 340 seq_num 1138) 2024-09-16T15:17:50.655 INFO:tasks.rados.rados.0.smithi162.stdout:3636: delete oid 48 current snap is 383 2024-09-16T15:17:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:3636: done (5 left) 2024-09-16T15:17:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:3637: write (excl) oid 40 current snap is 383 2024-09-16T15:17:50.658 INFO:tasks.rados.rados.0.smithi162.stdout:3637: seq_num 1240 ranges {440680=674299,1711907=624424} 2024-09-16T15:17:50.663 INFO:tasks.rados.rados.0.smithi162.stdout:3637: writing smithi16234619-40 from 440680 to 1114979 tid 1 2024-09-16T15:17:50.670 INFO:tasks.rados.rados.0.smithi162.stdout:3637: writing smithi16234619-40 from 1711907 to 2336331 tid 2 2024-09-16T15:17:50.670 INFO:tasks.rados.rados.0.smithi162.stdout:3638: snap_remove snap 379 2024-09-16T15:17:51.039 INFO:tasks.rados.rados.0.smithi162.stdout:3638: done (6 left) 2024-09-16T15:17:51.039 INFO:tasks.rados.rados.0.smithi162.stdout:3639: rollback oid 3 current snap is 383 2024-09-16T15:17:51.039 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 371 2024-09-16T15:17:51.039 INFO:tasks.rados.rados.0.smithi162.stdout:3635: expect (ObjNum 1138 snap 340 seq_num 1138) 2024-09-16T15:17:51.169 INFO:tasks.rados.rados.0.smithi162.stdout:3633: expect (ObjNum 1140 snap 340 seq_num 1140) 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: finishing write tid 1 to smithi16234619-36 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: finishing write tid 2 to smithi16234619-36 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3633: done (6 left) 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3635: done (5 left) 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3640: read oid 18 snap -1 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3640: expect deleted 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3641: copy_from oid 48 from oid 16 current snap is 383 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: finishing write tid 3 to smithi16234619-36 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: finishing write tid 4 to smithi16234619-36 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: oid 36 updating version 0 to 1979 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: oid 36 updating version 1979 to 1981 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:3631: oid 36 version 1981 is already newer than 1980 2024-09-16T15:17:51.423 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 1981 (ObjNum 1237 snap 383 seq_num 1237) dirty exists 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3631: left oid 36 (ObjNum 1237 snap 383 seq_num 1237) 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: finishing write tid 1 to smithi16234619-40 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: finishing write tid 2 to smithi16234619-40 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3630: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: finishing write tid 3 to smithi16234619-40 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: finishing write tid 4 to smithi16234619-40 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: oid 40 updating version 0 to 1492 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: oid 40 version 1492 is already newer than 1491 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: oid 40 version 1492 is already newer than 1490 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1492 (ObjNum 1240 snap 383 seq_num 1240) dirty exists 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3637: left oid 40 (ObjNum 1240 snap 383 seq_num 1240) 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3630: finishing copy_from to smithi16234619-8 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1476 (ObjNum 1171 snap 353 seq_num 1171) dirty exists 2024-09-16T15:17:51.424 INFO:tasks.rados.rados.0.smithi162.stdout:3634: finishing copy_from racing read to smithi16234619-49 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3634: finishing copy_from to smithi16234619-49 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 1577 (ObjNum 1119 snap 334 seq_num 1119) dirty exists 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3639: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3639: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3639: finishing rollback tid 2 to smithi16234619-3 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1809 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3630: done (6 left) 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3631: done (5 left) 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3634: done (4 left) 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3637: done (3 left) 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3639: done (2 left) 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3642: copy_from oid 26 from oid 50 current snap is 383 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3643: snap_create 2024-09-16T15:17:51.425 INFO:tasks.rados.rados.0.smithi162.stdout:3641: finishing copy_from to smithi16234619-48 2024-09-16T15:17:51.426 INFO:tasks.rados.rados.0.smithi162.stdout:3641: got expected ENOENT (src dne) 2024-09-16T15:17:51.426 INFO:tasks.rados.rados.0.smithi162.stdout:3641: finishing copy_from racing read to smithi16234619-48 2024-09-16T15:17:51.475 INFO:tasks.rados.rados.0.smithi162.stdout:3642: finishing copy_from racing read to smithi16234619-26 2024-09-16T15:17:51.480 INFO:tasks.rados.rados.0.smithi162.stdout:3642: finishing copy_from to smithi16234619-26 2024-09-16T15:17:51.480 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 1738 (ObjNum 1175 snap 357 seq_num 1175) dirty exists 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3640: done (3 left) 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3641: done (2 left) 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3642: done (1 left) 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3643: done (0 left) 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3644: write oid 50 current snap is 384 2024-09-16T15:17:52.037 INFO:tasks.rados.rados.0.smithi162.stdout:3644: seq_num 1241 ranges {457487=749548,1770389=484435,2746287=111200} 2024-09-16T15:17:52.043 INFO:tasks.rados.rados.0.smithi162.stdout:3644: writing smithi16234619-50 from 457487 to 1207035 tid 1 2024-09-16T15:17:52.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: writing smithi16234619-50 from 1770389 to 2254824 tid 2 2024-09-16T15:17:52.052 INFO:tasks.rados.rados.0.smithi162.stdout:3644: writing smithi16234619-50 from 2746287 to 2857487 tid 3 2024-09-16T15:17:52.052 INFO:tasks.rados.rados.0.smithi162.stdout:3645: copy_from oid 12 from oid 10 current snap is 384 2024-09-16T15:17:52.052 INFO:tasks.rados.rados.0.smithi162.stdout:3646: snap_remove snap 371 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: finishing write tid 1 to smithi16234619-50 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3646: done (2 left) 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3647: rollback oid 28 current snap is 384 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 377 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: finishing write tid 2 to smithi16234619-50 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: finishing write tid 3 to smithi16234619-50 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: finishing write tid 4 to smithi16234619-50 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: finishing write tid 5 to smithi16234619-50 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: oid 50 updating version 0 to 1989 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: oid 50 version 1989 is already newer than 1988 2024-09-16T15:17:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:3644: oid 50 updating version 1989 to 1990 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3644: oid 50 version 1990 is already newer than 1987 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1990 (ObjNum 1241 snap 384 seq_num 1241) dirty exists 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3644: left oid 50 (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3645: finishing copy_from racing read to smithi16234619-12 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3645: finishing copy_from to smithi16234619-12 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 2320 (ObjNum 1228 snap 383 seq_num 1228) dirty exists 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3644: done (2 left) 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3645: done (1 left) 2024-09-16T15:17:53.050 INFO:tasks.rados.rados.0.smithi162.stdout:3648: snap_remove snap 375 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3648: done (1 left) 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3649: rollback oid 29 current snap is 384 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 29 to 360 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3647: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1817 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3647: done (1 left) 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3650: read oid 6 snap -1 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3650: expect (ObjNum 1217 snap 374 seq_num 1217) 2024-09-16T15:17:54.024 INFO:tasks.rados.rados.0.smithi162.stdout:3651: rollback oid 5 current snap is 384 2024-09-16T15:17:54.025 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 5 to 361 2024-09-16T15:17:54.025 INFO:tasks.rados.rados.0.smithi162.stdout:3652: snap_remove snap 358 2024-09-16T15:17:55.302 INFO:tasks.rados.rados.0.smithi162.stdout:3652: done (3 left) 2024-09-16T15:17:55.302 INFO:tasks.rados.rados.0.smithi162.stdout:3653: delete oid 28 current snap is 384 2024-09-16T15:17:55.302 INFO:tasks.rados.rados.0.smithi162.stdout:3650: expect (ObjNum 1217 snap 374 seq_num 1217) 2024-09-16T15:17:55.413 INFO:tasks.rados.rados.0.smithi162.stdout:3649: finishing rollback tid 0 to smithi16234619-29 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3649: finishing rollback tid 1 to smithi16234619-29 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3649: finishing rollback tid 2 to smithi16234619-29 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 2081 (ObjNum 1157 snap 345 seq_num 1157) dirty exists 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3651: finishing rollback tid 0 to smithi16234619-5 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3651: finishing rollback tid 1 to smithi16234619-5 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3649: done (3 left) 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3650: done (2 left) 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3653: done (1 left) 2024-09-16T15:17:55.414 INFO:tasks.rados.rados.0.smithi162.stdout:3654: snap_remove snap 339 2024-09-16T15:17:56.309 INFO:tasks.rados.rados.0.smithi162.stdout:3654: done (1 left) 2024-09-16T15:17:56.309 INFO:tasks.rados.rados.0.smithi162.stdout:3655: snap_create 2024-09-16T15:17:56.309 INFO:tasks.rados.rados.0.smithi162.stdout:3651: finishing rollback tid 2 to smithi16234619-5 2024-09-16T15:17:56.310 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 2085 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3651: done (1 left) 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3655: done (0 left) 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3656: read oid 18 snap -1 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3656: expect deleted 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3657: read oid 11 snap -1 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3657: expect deleted 2024-09-16T15:17:57.312 INFO:tasks.rados.rados.0.smithi162.stdout:3658: snap_remove snap 384 2024-09-16T15:17:58.316 INFO:tasks.rados.rados.0.smithi162.stdout:3658: done (2 left) 2024-09-16T15:17:58.316 INFO:tasks.rados.rados.0.smithi162.stdout:3659: snap_create 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:3656: done (2 left) 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:3657: done (1 left) 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:3659: done (0 left) 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:3660: rollback oid 31 current snap is 386 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 31 to 383 2024-09-16T15:17:59.022 INFO:tasks.rados.rados.0.smithi162.stdout:3661: snap_remove snap 378 2024-09-16T15:18:00.039 INFO:tasks.rados.rados.0.smithi162.stdout:3661: done (1 left) 2024-09-16T15:18:00.039 INFO:tasks.rados.rados.0.smithi162.stdout:3662: read oid 36 snap -1 2024-09-16T15:18:00.039 INFO:tasks.rados.rados.0.smithi162.stdout:3662: expect (ObjNum 1237 snap 383 seq_num 1237) 2024-09-16T15:18:00.039 INFO:tasks.rados.rados.0.smithi162.stdout:3663: snap_remove snap 366 2024-09-16T15:18:01.045 INFO:tasks.rados.rados.0.smithi162.stdout:3663: done (2 left) 2024-09-16T15:18:01.046 INFO:tasks.rados.rados.0.smithi162.stdout:3664: copy_from oid 14 from oid 37 current snap is 386 2024-09-16T15:18:01.046 INFO:tasks.rados.rados.0.smithi162.stdout:3660: finishing rollback tid 1 to smithi16234619-31 2024-09-16T15:18:01.046 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 0 (ObjNum 49 snap 0 seq_num 851383280) dirty dne 2024-09-16T15:18:01.046 INFO:tasks.rados.rados.0.smithi162.stdout:3662: expect (ObjNum 1237 snap 383 seq_num 1237) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3664: finishing copy_from to smithi16234619-14 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3664: got expected ENOENT (src dne) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3664: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3660: done (2 left) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3662: done (1 left) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3664: done (0 left) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3665: read oid 15 snap -1 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3665: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3666: write oid 26 current snap is 386 2024-09-16T15:18:01.193 INFO:tasks.rados.rados.0.smithi162.stdout:3666: seq_num 1243 ranges {491101=500046,1487353=488104,2401133=489968} 2024-09-16T15:18:01.198 INFO:tasks.rados.rados.0.smithi162.stdout:3666: writing smithi16234619-26 from 491101 to 991147 tid 1 2024-09-16T15:18:01.204 INFO:tasks.rados.rados.0.smithi162.stdout:3666: writing smithi16234619-26 from 1487353 to 1975457 tid 2 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:3666: writing smithi16234619-26 from 2401133 to 2891101 tid 3 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:3667: read oid 45 snap -1 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:3667: expect deleted 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:3668: rollback oid 21 current snap is 386 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 21 to 355 2024-09-16T15:18:01.209 INFO:tasks.rados.rados.0.smithi162.stdout:3665: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:01.277 INFO:tasks.rados.rados.0.smithi162.stdout:3666: finishing write tid 1 to smithi16234619-26 2024-09-16T15:18:01.277 INFO:tasks.rados.rados.0.smithi162.stdout:3666: finishing write tid 2 to smithi16234619-26 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: finishing write tid 3 to smithi16234619-26 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: finishing write tid 4 to smithi16234619-26 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: finishing write tid 5 to smithi16234619-26 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: oid 26 updating version 0 to 2399 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: oid 26 updating version 2399 to 2401 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: oid 26 version 2401 is already newer than 2400 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: oid 26 version 2401 is already newer than 2398 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2401 (ObjNum 1243 snap 386 seq_num 1243) dirty exists 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: left oid 26 (ObjNum 1243 snap 386 seq_num 1243) 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3668: finishing rollback tid 0 to smithi16234619-21 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3665: done (3 left) 2024-09-16T15:18:01.278 INFO:tasks.rados.rados.0.smithi162.stdout:3666: done (2 left) 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3667: done (1 left) 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3669: copy_from oid 1 from oid 50 current snap is 386 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3668: finishing rollback tid 1 to smithi16234619-21 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3668: finishing rollback tid 2 to smithi16234619-21 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 21 v 2405 (ObjNum 1037 snap 311 seq_num 1037) dirty exists 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3668: done (1 left) 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3670: write (excl) oid 19 current snap is 386 2024-09-16T15:18:01.279 INFO:tasks.rados.rados.0.smithi162.stdout:3670: seq_num 1244 ranges {507908=575295,1829482=631762} 2024-09-16T15:18:01.283 INFO:tasks.rados.rados.0.smithi162.stdout:3670: writing smithi16234619-19 from 507908 to 1083203 tid 1 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3670: writing smithi16234619-19 from 1829482 to 2461244 tid 2 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3671: rollback oid 32 current snap is 386 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 385 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3672: copy_from oid 43 from oid 21 current snap is 386 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3673: read oid 44 snap -1 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3673: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3674: write (excl) oid 28 current snap is 386 2024-09-16T15:18:01.291 INFO:tasks.rados.rados.0.smithi162.stdout:3674: seq_num 1245 ranges {524715=650544,1887964=491773} 2024-09-16T15:18:01.297 INFO:tasks.rados.rados.0.smithi162.stdout:3674: writing smithi16234619-28 from 524715 to 1175259 tid 1 2024-09-16T15:18:01.304 INFO:tasks.rados.rados.0.smithi162.stdout:3674: writing smithi16234619-28 from 1887964 to 2379737 tid 2 2024-09-16T15:18:01.304 INFO:tasks.rados.rados.0.smithi162.stdout:3675: read oid 41 snap -1 2024-09-16T15:18:01.304 INFO:tasks.rados.rados.0.smithi162.stdout:3675: expect (ObjNum 1220 snap 375 seq_num 1220) 2024-09-16T15:18:01.305 INFO:tasks.rados.rados.0.smithi162.stdout:3676: rollback oid 6 current snap is 386 2024-09-16T15:18:01.305 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 374 2024-09-16T15:18:01.305 INFO:tasks.rados.rados.0.smithi162.stdout:3673: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:01.406 INFO:tasks.rados.rados.0.smithi162.stdout:3671: finishing rollback tid 0 to smithi16234619-32 2024-09-16T15:18:01.406 INFO:tasks.rados.rados.0.smithi162.stdout:3671: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:18:01.406 INFO:tasks.rados.rados.0.smithi162.stdout:3671: finishing rollback tid 2 to smithi16234619-32 2024-09-16T15:18:01.406 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 2323 (ObjNum 1171 snap 353 seq_num 1171) dirty exists 2024-09-16T15:18:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:3676: finishing rollback tid 0 to smithi16234619-6 2024-09-16T15:18:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:3676: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:18:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:3676: finishing rollback tid 2 to smithi16234619-6 2024-09-16T15:18:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1839 (ObjNum 1217 snap 374 seq_num 1217) dirty exists 2024-09-16T15:18:01.407 INFO:tasks.rados.rados.0.smithi162.stdout:3675: expect (ObjNum 1220 snap 375 seq_num 1220) 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: finishing write tid 1 to smithi16234619-19 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: finishing write tid 2 to smithi16234619-19 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: finishing write tid 3 to smithi16234619-19 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: finishing write tid 4 to smithi16234619-19 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: oid 19 updating version 0 to 2047 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: oid 19 version 2047 is already newer than 2045 2024-09-16T15:18:01.559 INFO:tasks.rados.rados.0.smithi162.stdout:3670: oid 19 version 2047 is already newer than 2046 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 2047 (ObjNum 1244 snap 386 seq_num 1244) dirty exists 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3670: left oid 19 (ObjNum 1244 snap 386 seq_num 1244) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3669: finishing copy_from racing read to smithi16234619-1 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3670: done (7 left) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3671: done (6 left) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3673: done (5 left) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3675: done (4 left) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3676: done (3 left) 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3677: copy_from oid 39 from oid 11 current snap is 386 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: finishing write tid 1 to smithi16234619-28 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: finishing write tid 2 to smithi16234619-28 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: finishing write tid 3 to smithi16234619-28 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: finishing write tid 4 to smithi16234619-28 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: oid 28 updating version 0 to 1818 2024-09-16T15:18:01.560 INFO:tasks.rados.rados.0.smithi162.stdout:3674: oid 28 updating version 1818 to 1820 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3674: oid 28 version 1820 is already newer than 1819 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1820 (ObjNum 1245 snap 386 seq_num 1245) dirty exists 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3674: left oid 28 (ObjNum 1245 snap 386 seq_num 1245) 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3669: finishing copy_from to smithi16234619-1 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 1990 (ObjNum 1241 snap 384 seq_num 1241) dirty exists 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3672: finishing copy_from racing read to smithi16234619-43 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3672: finishing copy_from to smithi16234619-43 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 2405 (ObjNum 1037 snap 311 seq_num 1037) dirty exists 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3669: done (3 left) 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3672: done (2 left) 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3674: done (1 left) 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3678: write (excl) oid 14 current snap is 386 2024-09-16T15:18:01.561 INFO:tasks.rados.rados.0.smithi162.stdout:3678: seq_num 1246 ranges {541522=725793,1946446=635431} 2024-09-16T15:18:01.565 INFO:tasks.rados.rados.0.smithi162.stdout:3678: writing smithi16234619-14 from 541522 to 1267315 tid 1 2024-09-16T15:18:01.572 INFO:tasks.rados.rados.0.smithi162.stdout:3678: writing smithi16234619-14 from 1946446 to 2581877 tid 2 2024-09-16T15:18:01.572 INFO:tasks.rados.rados.0.smithi162.stdout:3679: snap_remove snap 355 2024-09-16T15:18:02.046 INFO:tasks.rados.rados.0.smithi162.stdout:3679: done (2 left) 2024-09-16T15:18:02.046 INFO:tasks.rados.rados.0.smithi162.stdout:3680: read oid 50 snap -1 2024-09-16T15:18:02.046 INFO:tasks.rados.rados.0.smithi162.stdout:3680: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:02.046 INFO:tasks.rados.rados.0.smithi162.stdout:3677: finishing copy_from to smithi16234619-39 2024-09-16T15:18:02.046 INFO:tasks.rados.rados.0.smithi162.stdout:3677: got expected ENOENT (src dne) 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3677: finishing copy_from racing read to smithi16234619-39 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: finishing write tid 1 to smithi16234619-14 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: finishing write tid 2 to smithi16234619-14 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: finishing write tid 3 to smithi16234619-14 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: finishing write tid 4 to smithi16234619-14 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: oid 14 updating version 0 to 2050 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: oid 14 version 2050 is already newer than 2049 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: oid 14 version 2050 is already newer than 2048 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 2050 (ObjNum 1246 snap 386 seq_num 1246) dirty exists 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: left oid 14 (ObjNum 1246 snap 386 seq_num 1246) 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3681: read oid 7 snap -1 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3681: expect (ObjNum 1155 snap 345 seq_num 1155) 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3677: done (3 left) 2024-09-16T15:18:02.047 INFO:tasks.rados.rados.0.smithi162.stdout:3678: done (2 left) 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3682: read oid 37 snap -1 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3682: expect deleted 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3683: read oid 44 snap -1 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3683: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3684: rollback oid 24 current snap is 386 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 382 2024-09-16T15:18:02.048 INFO:tasks.rados.rados.0.smithi162.stdout:3685: snap_remove snap 383 2024-09-16T15:18:03.051 INFO:tasks.rados.rados.0.smithi162.stdout:3685: done (5 left) 2024-09-16T15:18:03.052 INFO:tasks.rados.rados.0.smithi162.stdout:3686: copy_from oid 25 from oid 29 current snap is 386 2024-09-16T15:18:03.052 INFO:tasks.rados.rados.0.smithi162.stdout:3683: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:03.139 INFO:tasks.rados.rados.0.smithi162.stdout:3684: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:18:03.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1708 (ObjNum 129 snap 0 seq_num 861978368) dirty dne 2024-09-16T15:18:03.139 INFO:tasks.rados.rados.0.smithi162.stdout:3680: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:03.376 INFO:tasks.rados.rados.0.smithi162.stdout:3681: expect (ObjNum 1155 snap 345 seq_num 1155) 2024-09-16T15:18:03.484 INFO:tasks.rados.rados.0.smithi162.stdout:3686: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3686: finishing copy_from to smithi16234619-25 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 2081 (ObjNum 1157 snap 345 seq_num 1157) dirty exists 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3680: done (5 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3681: done (4 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3682: done (3 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3683: done (2 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3684: done (1 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3686: done (0 left) 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3687: write oid 4 current snap is 386 2024-09-16T15:18:03.485 INFO:tasks.rados.rados.0.smithi162.stdout:3687: seq_num 1247 ranges {558329=517395,1721281=779089} 2024-09-16T15:18:03.490 INFO:tasks.rados.rados.0.smithi162.stdout:3687: writing smithi16234619-4 from 558329 to 1075724 tid 1 2024-09-16T15:18:03.499 INFO:tasks.rados.rados.0.smithi162.stdout:3687: writing smithi16234619-4 from 1721281 to 2500370 tid 2 2024-09-16T15:18:03.499 INFO:tasks.rados.rados.0.smithi162.stdout:3688: write (excl) oid 33 current snap is 386 2024-09-16T15:18:03.499 INFO:tasks.rados.rados.0.smithi162.stdout:3688: seq_num 1248 ranges {575136=592644,1663410=639100,2705542=269594} 2024-09-16T15:18:03.506 INFO:tasks.rados.rados.0.smithi162.stdout:3688: writing smithi16234619-33 from 575136 to 1167780 tid 1 2024-09-16T15:18:03.514 INFO:tasks.rados.rados.0.smithi162.stdout:3688: writing smithi16234619-33 from 1663410 to 2302510 tid 2 2024-09-16T15:18:03.519 INFO:tasks.rados.rados.0.smithi162.stdout:3688: writing smithi16234619-33 from 2705542 to 2975136 tid 3 2024-09-16T15:18:03.519 INFO:tasks.rados.rados.0.smithi162.stdout:3689: write (excl) oid 46 current snap is 386 2024-09-16T15:18:03.519 INFO:tasks.rados.rados.0.smithi162.stdout:3689: seq_num 1249 ranges {591943=667893,1721892=782758} 2024-09-16T15:18:03.527 INFO:tasks.rados.rados.0.smithi162.stdout:3689: writing smithi16234619-46 from 591943 to 1259836 tid 1 2024-09-16T15:18:03.534 INFO:tasks.rados.rados.0.smithi162.stdout:3689: writing smithi16234619-46 from 1721892 to 2504650 tid 2 2024-09-16T15:18:03.535 INFO:tasks.rados.rados.0.smithi162.stdout:3690: write oid 49 current snap is 386 2024-09-16T15:18:03.535 INFO:tasks.rados.rados.0.smithi162.stdout:3687: finishing write tid 1 to smithi16234619-4 2024-09-16T15:18:03.535 INFO:tasks.rados.rados.0.smithi162.stdout:3690: seq_num 1250 ranges {608750=743142,1780374=642769} 2024-09-16T15:18:03.542 INFO:tasks.rados.rados.0.smithi162.stdout:3690: writing smithi16234619-49 from 608750 to 1351892 tid 1 2024-09-16T15:18:03.548 INFO:tasks.rados.rados.0.smithi162.stdout:3690: writing smithi16234619-49 from 1780374 to 2423143 tid 2 2024-09-16T15:18:03.548 INFO:tasks.rados.rados.0.smithi162.stdout:3691: snap_remove snap 381 2024-09-16T15:18:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:3691: done (4 left) 2024-09-16T15:18:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:3692: write oid 48 current snap is 386 2024-09-16T15:18:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:3687: finishing write tid 2 to smithi16234619-4 2024-09-16T15:18:04.020 INFO:tasks.rados.rados.0.smithi162.stdout:3692: seq_num 1251 ranges {625557=418391,1838856=786427} 2024-09-16T15:18:04.026 INFO:tasks.rados.rados.0.smithi162.stdout:3692: writing smithi16234619-48 from 625557 to 1043948 tid 1 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3692: writing smithi16234619-48 from 1838856 to 2625283 tid 2 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3693: rollback oid 6 current snap is 386 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 6 to 380 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: finishing write tid 3 to smithi16234619-4 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: finishing write tid 4 to smithi16234619-4 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: oid 4 updating version 0 to 1993 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: oid 4 version 1993 is already newer than 1991 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: oid 4 version 1993 is already newer than 1992 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 1993 (ObjNum 1247 snap 386 seq_num 1247) dirty exists 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3687: left oid 4 (ObjNum 1247 snap 386 seq_num 1247) 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3688: finishing write tid 1 to smithi16234619-33 2024-09-16T15:18:04.037 INFO:tasks.rados.rados.0.smithi162.stdout:3688: finishing write tid 2 to smithi16234619-33 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: finishing write tid 3 to smithi16234619-33 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: finishing write tid 4 to smithi16234619-33 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: finishing write tid 5 to smithi16234619-33 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: oid 33 updating version 0 to 1716 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: oid 33 version 1716 is already newer than 1715 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: oid 33 updating version 1716 to 1717 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: oid 33 updating version 1717 to 1718 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1718 (ObjNum 1248 snap 386 seq_num 1248) dirty exists 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3688: left oid 33 (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: finishing write tid 1 to smithi16234619-46 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: finishing write tid 2 to smithi16234619-46 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: finishing write tid 3 to smithi16234619-46 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: finishing write tid 4 to smithi16234619-46 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: oid 46 updating version 0 to 1222 2024-09-16T15:18:04.038 INFO:tasks.rados.rados.0.smithi162.stdout:3689: oid 46 version 1222 is already newer than 1220 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3689: oid 46 version 1222 is already newer than 1221 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1222 (ObjNum 1249 snap 386 seq_num 1249) dirty exists 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3689: left oid 46 (ObjNum 1249 snap 386 seq_num 1249) 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: finishing write tid 1 to smithi16234619-49 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: finishing write tid 2 to smithi16234619-49 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: finishing write tid 3 to smithi16234619-49 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: finishing write tid 4 to smithi16234619-49 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: oid 49 updating version 0 to 1579 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: oid 49 updating version 1579 to 1580 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: oid 49 version 1580 is already newer than 1578 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 1580 (ObjNum 1250 snap 386 seq_num 1250) dirty exists 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3690: left oid 49 (ObjNum 1250 snap 386 seq_num 1250) 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3687: done (5 left) 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3688: done (4 left) 2024-09-16T15:18:04.039 INFO:tasks.rados.rados.0.smithi162.stdout:3689: done (3 left) 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3690: done (2 left) 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3694: read oid 49 snap -1 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3694: expect (ObjNum 1250 snap 386 seq_num 1250) 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3695: copy_from oid 22 from oid 32 current snap is 386 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3696: write oid 37 current snap is 386 2024-09-16T15:18:04.040 INFO:tasks.rados.rados.0.smithi162.stdout:3696: seq_num 1252 ranges {642364=493640,1897338=646438} 2024-09-16T15:18:04.044 INFO:tasks.rados.rados.0.smithi162.stdout:3696: writing smithi16234619-37 from 642364 to 1136004 tid 1 2024-09-16T15:18:04.053 INFO:tasks.rados.rados.0.smithi162.stdout:3696: writing smithi16234619-37 from 1897338 to 2543776 tid 2 2024-09-16T15:18:04.053 INFO:tasks.rados.rados.0.smithi162.stdout:3697: snap_create 2024-09-16T15:18:04.053 INFO:tasks.rados.rados.0.smithi162.stdout:3693: finishing rollback tid 0 to smithi16234619-6 2024-09-16T15:18:04.054 INFO:tasks.rados.rados.0.smithi162.stdout:3693: finishing rollback tid 1 to smithi16234619-6 2024-09-16T15:18:04.054 INFO:tasks.rados.rados.0.smithi162.stdout:3693: finishing rollback tid 2 to smithi16234619-6 2024-09-16T15:18:04.054 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1842 (ObjNum 1217 snap 374 seq_num 1217) dirty exists 2024-09-16T15:18:04.058 INFO:tasks.rados.rados.0.smithi162.stdout:3694: expect (ObjNum 1250 snap 386 seq_num 1250) 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: finishing write tid 1 to smithi16234619-48 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: finishing write tid 2 to smithi16234619-48 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: finishing write tid 3 to smithi16234619-48 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: finishing write tid 4 to smithi16234619-48 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: oid 48 updating version 0 to 1994 2024-09-16T15:18:04.212 INFO:tasks.rados.rados.0.smithi162.stdout:3692: oid 48 updating version 1994 to 1996 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3692: oid 48 version 1996 is already newer than 1995 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 48 v 1996 (ObjNum 1251 snap 386 seq_num 1251) dirty exists 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3692: left oid 48 (ObjNum 1251 snap 386 seq_num 1251) 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: finishing write tid 1 to smithi16234619-37 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: finishing write tid 2 to smithi16234619-37 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: finishing write tid 3 to smithi16234619-37 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: finishing write tid 4 to smithi16234619-37 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: oid 37 updating version 0 to 2148 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: oid 37 version 2148 is already newer than 2146 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: oid 37 version 2148 is already newer than 2147 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 2148 (ObjNum 1252 snap 386 seq_num 1252) dirty exists 2024-09-16T15:18:04.213 INFO:tasks.rados.rados.0.smithi162.stdout:3696: left oid 37 (ObjNum 1252 snap 386 seq_num 1252) 2024-09-16T15:18:04.214 INFO:tasks.rados.rados.0.smithi162.stdout:3695: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:18:04.214 INFO:tasks.rados.rados.0.smithi162.stdout:3695: finishing copy_from to smithi16234619-22 2024-09-16T15:18:04.214 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 2323 (ObjNum 1171 snap 353 seq_num 1171) dirty exists 2024-09-16T15:18:05.393 INFO:tasks.rados.rados.0.smithi162.stdout:3692: done (5 left) 2024-09-16T15:18:05.393 INFO:tasks.rados.rados.0.smithi162.stdout:3693: done (4 left) 2024-09-16T15:18:05.393 INFO:tasks.rados.rados.0.smithi162.stdout:3694: done (3 left) 2024-09-16T15:18:05.393 INFO:tasks.rados.rados.0.smithi162.stdout:3695: done (2 left) 2024-09-16T15:18:05.394 INFO:tasks.rados.rados.0.smithi162.stdout:3696: done (1 left) 2024-09-16T15:18:05.394 INFO:tasks.rados.rados.0.smithi162.stdout:3697: done (0 left) 2024-09-16T15:18:05.394 INFO:tasks.rados.rados.0.smithi162.stdout:3698: write oid 12 current snap is 387 2024-09-16T15:18:05.394 INFO:tasks.rados.rados.0.smithi162.stdout:3698: seq_num 1253 ranges {659171=568889,1839467=790096} 2024-09-16T15:18:05.399 INFO:tasks.rados.rados.0.smithi162.stdout:3698: writing smithi16234619-12 from 659171 to 1228060 tid 1 2024-09-16T15:18:05.407 INFO:tasks.rados.rados.0.smithi162.stdout:3698: writing smithi16234619-12 from 1839467 to 2629563 tid 2 2024-09-16T15:18:05.407 INFO:tasks.rados.rados.0.smithi162.stdout:3699: rollback oid 7 current snap is 387 2024-09-16T15:18:05.407 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 7 to 374 2024-09-16T15:18:05.407 INFO:tasks.rados.rados.0.smithi162.stdout:3700: snap_remove snap 377 2024-09-16T15:18:06.070 INFO:tasks.rados.rados.0.smithi162.stdout:3700: done (2 left) 2024-09-16T15:18:06.070 INFO:tasks.rados.rados.0.smithi162.stdout:3701: write oid 25 current snap is 387 2024-09-16T15:18:06.070 INFO:tasks.rados.rados.0.smithi162.stdout:3699: finishing rollback tid 0 to smithi16234619-7 2024-09-16T15:18:06.071 INFO:tasks.rados.rados.0.smithi162.stdout:3701: seq_num 1254 ranges {675978=644138,1897949=650107} 2024-09-16T15:18:06.077 INFO:tasks.rados.rados.0.smithi162.stdout:3701: writing smithi16234619-25 from 675978 to 1320116 tid 1 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3701: writing smithi16234619-25 from 1897949 to 2548056 tid 2 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3702: snap_create 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: finishing write tid 1 to smithi16234619-12 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: finishing write tid 2 to smithi16234619-12 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3699: finishing rollback tid 1 to smithi16234619-7 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3699: finishing rollback tid 2 to smithi16234619-7 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1511 (ObjNum 1191 snap 364 seq_num 1191) dirty exists 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: finishing write tid 3 to smithi16234619-12 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: finishing write tid 4 to smithi16234619-12 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: oid 12 updating version 0 to 2325 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: oid 12 updating version 2325 to 2326 2024-09-16T15:18:06.085 INFO:tasks.rados.rados.0.smithi162.stdout:3698: oid 12 version 2326 is already newer than 2324 2024-09-16T15:18:06.086 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 12 v 2326 (ObjNum 1253 snap 387 seq_num 1253) dirty exists 2024-09-16T15:18:06.086 INFO:tasks.rados.rados.0.smithi162.stdout:3698: left oid 12 (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:18:06.113 INFO:tasks.rados.rados.0.smithi162.stdout:3701: finishing write tid 1 to smithi16234619-25 2024-09-16T15:18:06.113 INFO:tasks.rados.rados.0.smithi162.stdout:3701: finishing write tid 2 to smithi16234619-25 2024-09-16T15:18:06.113 INFO:tasks.rados.rados.0.smithi162.stdout:3701: finishing write tid 3 to smithi16234619-25 2024-09-16T15:18:06.113 INFO:tasks.rados.rados.0.smithi162.stdout:3701: finishing write tid 4 to smithi16234619-25 2024-09-16T15:18:06.114 INFO:tasks.rados.rados.0.smithi162.stdout:3701: oid 25 updating version 0 to 2084 2024-09-16T15:18:06.114 INFO:tasks.rados.rados.0.smithi162.stdout:3701: oid 25 version 2084 is already newer than 2083 2024-09-16T15:18:06.114 INFO:tasks.rados.rados.0.smithi162.stdout:3701: oid 25 version 2084 is already newer than 2082 2024-09-16T15:18:06.114 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 2084 (ObjNum 1254 snap 387 seq_num 1254) dirty exists 2024-09-16T15:18:06.114 INFO:tasks.rados.rados.0.smithi162.stdout:3701: left oid 25 (ObjNum 1254 snap 387 seq_num 1254) 2024-09-16T15:18:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:3698: done (3 left) 2024-09-16T15:18:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:3699: done (2 left) 2024-09-16T15:18:07.080 INFO:tasks.rados.rados.0.smithi162.stdout:3701: done (1 left) 2024-09-16T15:18:07.081 INFO:tasks.rados.rados.0.smithi162.stdout:3702: done (0 left) 2024-09-16T15:18:07.081 INFO:tasks.rados.rados.0.smithi162.stdout:3703: snap_remove snap 361 2024-09-16T15:18:08.083 INFO:tasks.rados.rados.0.smithi162.stdout:3703: done (0 left) 2024-09-16T15:18:08.083 INFO:tasks.rados.rados.0.smithi162.stdout:3704: delete oid 7 current snap is 388 2024-09-16T15:18:08.090 INFO:tasks.rados.rados.0.smithi162.stdout:3704: done (0 left) 2024-09-16T15:18:08.090 INFO:tasks.rados.rados.0.smithi162.stdout:3705: snap_create 2024-09-16T15:18:09.027 INFO:tasks.rados.rados.0.smithi162.stdout:3705: done (0 left) 2024-09-16T15:18:09.027 INFO:tasks.rados.rados.0.smithi162.stdout:3706: write (excl) oid 18 current snap is 389 2024-09-16T15:18:09.027 INFO:tasks.rados.rados.0.smithi162.stdout:3706: seq_num 1256 ranges {709592=510989,1731266=653776,3041279=68313} 2024-09-16T15:18:09.034 INFO:tasks.rados.rados.0.smithi162.stdout:3706: writing smithi16234619-18 from 709592 to 1220581 tid 1 2024-09-16T15:18:09.041 INFO:tasks.rados.rados.0.smithi162.stdout:3706: writing smithi16234619-18 from 1731266 to 2385042 tid 2 2024-09-16T15:18:09.045 INFO:tasks.rados.rados.0.smithi162.stdout:3706: writing smithi16234619-18 from 3041279 to 3109592 tid 3 2024-09-16T15:18:09.045 INFO:tasks.rados.rados.0.smithi162.stdout:3707: snap_create 2024-09-16T15:18:09.076 INFO:tasks.rados.rados.0.smithi162.stdout:3706: finishing write tid 1 to smithi16234619-18 2024-09-16T15:18:09.076 INFO:tasks.rados.rados.0.smithi162.stdout:3706: finishing write tid 2 to smithi16234619-18 2024-09-16T15:18:09.077 INFO:tasks.rados.rados.0.smithi162.stdout:3706: finishing write tid 3 to smithi16234619-18 2024-09-16T15:18:09.077 INFO:tasks.rados.rados.0.smithi162.stdout:3706: finishing write tid 4 to smithi16234619-18 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: finishing write tid 5 to smithi16234619-18 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: oid 18 updating version 0 to 1516 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: oid 18 version 1516 is already newer than 1515 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: oid 18 version 1516 is already newer than 1514 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: oid 18 version 1516 is already newer than 1513 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1516 (ObjNum 1256 snap 389 seq_num 1256) dirty exists 2024-09-16T15:18:09.078 INFO:tasks.rados.rados.0.smithi162.stdout:3706: left oid 18 (ObjNum 1256 snap 389 seq_num 1256) 2024-09-16T15:18:10.035 INFO:tasks.rados.rados.0.smithi162.stdout:3706: done (1 left) 2024-09-16T15:18:10.035 INFO:tasks.rados.rados.0.smithi162.stdout:3707: done (0 left) 2024-09-16T15:18:10.036 INFO:tasks.rados.rados.0.smithi162.stdout:3708: delete oid 10 current snap is 390 2024-09-16T15:18:10.039 INFO:tasks.rados.rados.0.smithi162.stdout:3708: done (0 left) 2024-09-16T15:18:10.039 INFO:tasks.rados.rados.0.smithi162.stdout:3709: delete oid 37 current snap is 390 2024-09-16T15:18:10.042 INFO:tasks.rados.rados.0.smithi162.stdout:3709: done (0 left) 2024-09-16T15:18:10.042 INFO:tasks.rados.rados.0.smithi162.stdout:3710: snap_create 2024-09-16T15:18:11.570 INFO:tasks.rados.rados.0.smithi162.stdout:3710: done (0 left) 2024-09-16T15:18:11.570 INFO:tasks.rados.rados.0.smithi162.stdout:3711: write oid 28 current snap is 391 2024-09-16T15:18:11.570 INFO:tasks.rados.rados.0.smithi162.stdout:3711: seq_num 1259 ranges {760013=736736,2190359=401103} 2024-09-16T15:18:11.578 INFO:tasks.rados.rados.0.smithi162.stdout:3711: writing smithi16234619-28 from 760013 to 1496749 tid 1 2024-09-16T15:18:11.584 INFO:tasks.rados.rados.0.smithi162.stdout:3711: writing smithi16234619-28 from 2190359 to 2591462 tid 2 2024-09-16T15:18:11.584 INFO:tasks.rados.rados.0.smithi162.stdout:3712: rollback oid 33 current snap is 391 2024-09-16T15:18:11.584 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 33 to 389 2024-09-16T15:18:11.584 INFO:tasks.rados.rados.0.smithi162.stdout:3713: delete oid 8 current snap is 391 2024-09-16T15:18:11.587 INFO:tasks.rados.rados.0.smithi162.stdout:3713: done (2 left) 2024-09-16T15:18:11.588 INFO:tasks.rados.rados.0.smithi162.stdout:3714: snap_create 2024-09-16T15:18:11.589 INFO:tasks.rados.rados.0.smithi162.stdout:3712: finishing rollback tid 0 to smithi16234619-33 2024-09-16T15:18:11.589 INFO:tasks.rados.rados.0.smithi162.stdout:3712: finishing rollback tid 1 to smithi16234619-33 2024-09-16T15:18:11.589 INFO:tasks.rados.rados.0.smithi162.stdout:3712: finishing rollback tid 2 to smithi16234619-33 2024-09-16T15:18:11.589 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1721 (ObjNum 1248 snap 386 seq_num 1248) dirty exists 2024-09-16T15:18:11.611 INFO:tasks.rados.rados.0.smithi162.stdout:3711: finishing write tid 1 to smithi16234619-28 2024-09-16T15:18:11.611 INFO:tasks.rados.rados.0.smithi162.stdout:3711: finishing write tid 2 to smithi16234619-28 2024-09-16T15:18:11.613 INFO:tasks.rados.rados.0.smithi162.stdout:3711: finishing write tid 3 to smithi16234619-28 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:3711: finishing write tid 4 to smithi16234619-28 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:3711: oid 28 updating version 0 to 1823 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:3711: oid 28 version 1823 is already newer than 1822 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:3711: oid 28 version 1823 is already newer than 1821 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1823 (ObjNum 1259 snap 391 seq_num 1259) dirty exists 2024-09-16T15:18:11.614 INFO:tasks.rados.rados.0.smithi162.stdout:3711: left oid 28 (ObjNum 1259 snap 391 seq_num 1259) 2024-09-16T15:18:12.648 INFO:tasks.rados.rados.0.smithi162.stdout:3711: done (2 left) 2024-09-16T15:18:12.648 INFO:tasks.rados.rados.0.smithi162.stdout:3712: done (1 left) 2024-09-16T15:18:12.648 INFO:tasks.rados.rados.0.smithi162.stdout:3714: done (0 left) 2024-09-16T15:18:12.648 INFO:tasks.rados.rados.0.smithi162.stdout:3715: write (excl) oid 46 current snap is 392 2024-09-16T15:18:12.648 INFO:tasks.rados.rados.0.smithi162.stdout:3715: seq_num 1261 ranges {793627=487234,1790970=404772,2829335=364292} 2024-09-16T15:18:12.654 INFO:tasks.rados.rados.0.smithi162.stdout:3715: writing smithi16234619-46 from 793627 to 1280861 tid 1 2024-09-16T15:18:12.659 INFO:tasks.rados.rados.0.smithi162.stdout:3715: writing smithi16234619-46 from 1790970 to 2195742 tid 2 2024-09-16T15:18:12.665 INFO:tasks.rados.rados.0.smithi162.stdout:3715: writing smithi16234619-46 from 2829335 to 3193627 tid 3 2024-09-16T15:18:12.665 INFO:tasks.rados.rados.0.smithi162.stdout:3716: rollback oid 43 current snap is 392 2024-09-16T15:18:12.665 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 43 to 370 2024-09-16T15:18:12.665 INFO:tasks.rados.rados.0.smithi162.stdout:3717: write oid 9 current snap is 392 2024-09-16T15:18:12.665 INFO:tasks.rados.rados.0.smithi162.stdout:3717: seq_num 1262 ranges {410434=678836,1565805=548430,2573111=637323} 2024-09-16T15:18:12.671 INFO:tasks.rados.rados.0.smithi162.stdout:3717: writing smithi16234619-9 from 410434 to 1089270 tid 1 2024-09-16T15:18:12.676 INFO:tasks.rados.rados.0.smithi162.stdout:3717: writing smithi16234619-9 from 1565805 to 2114235 tid 2 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3717: writing smithi16234619-9 from 2573111 to 3210434 tid 3 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3718: read oid 50 snap -1 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3718: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3719: read oid 21 snap 360 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3719: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:3720: rollback oid 3 current snap is 392 2024-09-16T15:18:12.683 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 3 to 391 2024-09-16T15:18:12.684 INFO:tasks.rados.rados.0.smithi162.stdout:3716: finishing rollback tid 0 to smithi16234619-43 2024-09-16T15:18:12.684 INFO:tasks.rados.rados.0.smithi162.stdout:3716: finishing rollback tid 1 to smithi16234619-43 2024-09-16T15:18:12.684 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 2407 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:18:12.684 INFO:tasks.rados.rados.0.smithi162.stdout:3716: done (5 left) 2024-09-16T15:18:12.684 INFO:tasks.rados.rados.0.smithi162.stdout:3721: delete oid 20 current snap is 392 2024-09-16T15:18:12.688 INFO:tasks.rados.rados.0.smithi162.stdout:3720: finishing rollback tid 0 to smithi16234619-3 2024-09-16T15:18:12.688 INFO:tasks.rados.rados.0.smithi162.stdout:3721: done (5 left) 2024-09-16T15:18:12.688 INFO:tasks.rados.rados.0.smithi162.stdout:3722: delete oid 19 current snap is 392 2024-09-16T15:18:12.688 INFO:tasks.rados.rados.0.smithi162.stdout:3719: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:12.800 INFO:tasks.rados.rados.0.smithi162.stdout:3719: done (5 left) 2024-09-16T15:18:12.800 INFO:tasks.rados.rados.0.smithi162.stdout:3722: done (4 left) 2024-09-16T15:18:12.801 INFO:tasks.rados.rados.0.smithi162.stdout:3723: read oid 44 snap -1 2024-09-16T15:18:12.801 INFO:tasks.rados.rados.0.smithi162.stdout:3723: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:12.801 INFO:tasks.rados.rados.0.smithi162.stdout:3724: snap_remove snap 385 2024-09-16T15:18:13.576 INFO:tasks.rados.rados.0.smithi162.stdout:3724: done (5 left) 2024-09-16T15:18:13.576 INFO:tasks.rados.rados.0.smithi162.stdout:3725: write (excl) oid 45 current snap is 392 2024-09-16T15:18:13.577 INFO:tasks.rados.rados.0.smithi162.stdout:3725: seq_num 1265 ranges {460855=504583,1624898=412110,2539027=572508} 2024-09-16T15:18:13.582 INFO:tasks.rados.rados.0.smithi162.stdout:3725: writing smithi16234619-45 from 460855 to 965438 tid 1 2024-09-16T15:18:13.587 INFO:tasks.rados.rados.0.smithi162.stdout:3725: writing smithi16234619-45 from 1624898 to 2037008 tid 2 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:3725: writing smithi16234619-45 from 2539027 to 3111535 tid 3 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:3726: snap_create 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:3720: finishing rollback tid 1 to smithi16234619-3 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:3720: finishing rollback tid 2 to smithi16234619-3 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1812 (ObjNum 1160 snap 345 seq_num 1160) dirty exists 2024-09-16T15:18:13.594 INFO:tasks.rados.rados.0.smithi162.stdout:3718: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: finishing write tid 1 to smithi16234619-46 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: finishing write tid 2 to smithi16234619-46 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: finishing write tid 3 to smithi16234619-46 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: finishing write tid 4 to smithi16234619-46 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: finishing write tid 5 to smithi16234619-46 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: oid 46 updating version 0 to 1223 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: oid 46 updating version 1223 to 1225 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: oid 46 updating version 1225 to 1226 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:3715: oid 46 version 1226 is already newer than 1224 2024-09-16T15:18:13.831 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1226 (ObjNum 1261 snap 392 seq_num 1261) dirty exists 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3715: left oid 46 (ObjNum 1261 snap 392 seq_num 1261) 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: finishing write tid 1 to smithi16234619-9 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: finishing write tid 2 to smithi16234619-9 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: finishing write tid 3 to smithi16234619-9 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: finishing write tid 4 to smithi16234619-9 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: finishing write tid 5 to smithi16234619-9 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: oid 9 updating version 0 to 1997 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: oid 9 updating version 1997 to 2000 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: oid 9 version 2000 is already newer than 1998 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: oid 9 version 2000 is already newer than 1999 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 2000 (ObjNum 1262 snap 392 seq_num 1262) dirty exists 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3717: left oid 9 (ObjNum 1262 snap 392 seq_num 1262) 2024-09-16T15:18:13.832 INFO:tasks.rados.rados.0.smithi162.stdout:3723: expect (ObjNum 1232 snap 383 seq_num 1232) 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: finishing write tid 1 to smithi16234619-45 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: finishing write tid 2 to smithi16234619-45 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: finishing write tid 3 to smithi16234619-45 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: finishing write tid 4 to smithi16234619-45 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: finishing write tid 5 to smithi16234619-45 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: oid 45 updating version 0 to 2455 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: oid 45 updating version 2455 to 2457 2024-09-16T15:18:13.915 INFO:tasks.rados.rados.0.smithi162.stdout:3725: oid 45 updating version 2457 to 2458 2024-09-16T15:18:13.916 INFO:tasks.rados.rados.0.smithi162.stdout:3725: oid 45 version 2458 is already newer than 2456 2024-09-16T15:18:13.916 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 2458 (ObjNum 1265 snap 392 seq_num 1265) dirty exists 2024-09-16T15:18:13.916 INFO:tasks.rados.rados.0.smithi162.stdout:3725: left oid 45 (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3715: done (6 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3717: done (5 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3718: done (4 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3720: done (3 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3723: done (2 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3725: done (1 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3726: done (0 left) 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3727: write oid 9 current snap is 393 2024-09-16T15:18:14.443 INFO:tasks.rados.rados.0.smithi162.stdout:3727: seq_num 1266 ranges {477662=579832,1683380=555768,2966450=311212} 2024-09-16T15:18:14.448 INFO:tasks.rados.rados.0.smithi162.stdout:3727: writing smithi16234619-9 from 477662 to 1057494 tid 1 2024-09-16T15:18:14.456 INFO:tasks.rados.rados.0.smithi162.stdout:3727: writing smithi16234619-9 from 1683380 to 2239148 tid 2 2024-09-16T15:18:14.462 INFO:tasks.rados.rados.0.smithi162.stdout:3727: writing smithi16234619-9 from 2966450 to 3277662 tid 3 2024-09-16T15:18:14.463 INFO:tasks.rados.rados.0.smithi162.stdout:3728: snap_create 2024-09-16T15:18:14.488 INFO:tasks.rados.rados.0.smithi162.stdout:3727: finishing write tid 1 to smithi16234619-9 2024-09-16T15:18:14.489 INFO:tasks.rados.rados.0.smithi162.stdout:3727: finishing write tid 2 to smithi16234619-9 2024-09-16T15:18:14.489 INFO:tasks.rados.rados.0.smithi162.stdout:3727: finishing write tid 3 to smithi16234619-9 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: finishing write tid 4 to smithi16234619-9 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: finishing write tid 5 to smithi16234619-9 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: oid 9 updating version 0 to 2003 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: oid 9 updating version 2003 to 2004 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: oid 9 version 2004 is already newer than 2002 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: oid 9 version 2004 is already newer than 2001 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 2004 (ObjNum 1266 snap 393 seq_num 1266) dirty exists 2024-09-16T15:18:14.490 INFO:tasks.rados.rados.0.smithi162.stdout:3727: left oid 9 (ObjNum 1266 snap 393 seq_num 1266) 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:3727: done (1 left) 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:3728: done (0 left) 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:3729: rollback oid 8 current snap is 394 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 316 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:3730: copy_from oid 7 from oid 18 current snap is 394 2024-09-16T15:18:15.740 INFO:tasks.rados.rados.0.smithi162.stdout:3731: snap_remove snap 391 2024-09-16T15:18:16.655 INFO:tasks.rados.rados.0.smithi162.stdout:3731: done (2 left) 2024-09-16T15:18:16.656 INFO:tasks.rados.rados.0.smithi162.stdout:3732: read oid 32 snap -1 2024-09-16T15:18:16.656 INFO:tasks.rados.rados.0.smithi162.stdout:3732: expect (ObjNum 1171 snap 353 seq_num 1171) 2024-09-16T15:18:16.656 INFO:tasks.rados.rados.0.smithi162.stdout:3733: snap_remove snap 370 2024-09-16T15:18:17.657 INFO:tasks.rados.rados.0.smithi162.stdout:3733: done (3 left) 2024-09-16T15:18:17.657 INFO:tasks.rados.rados.0.smithi162.stdout:3734: rollback oid 20 current snap is 394 2024-09-16T15:18:17.657 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 392 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:3729: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:3729: finishing rollback tid 2 to smithi16234619-8 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1519 (ObjNum 957 snap 268 seq_num 957) dirty exists 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:3730: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:3730: finishing copy_from to smithi16234619-7 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1516 (ObjNum 1256 snap 389 seq_num 1256) dirty exists 2024-09-16T15:18:17.658 INFO:tasks.rados.rados.0.smithi162.stdout:3732: expect (ObjNum 1171 snap 353 seq_num 1171) 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3734: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1722 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3729: done (3 left) 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3730: done (2 left) 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3732: done (1 left) 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3734: done (0 left) 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3735: read oid 16 snap 360 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3735: expect deleted 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3736: write (excl) oid 40 current snap is 394 2024-09-16T15:18:17.826 INFO:tasks.rados.rados.0.smithi162.stdout:3736: seq_num 1267 ranges {494469=655081,1741862=415779,2710226=584243} 2024-09-16T15:18:17.831 INFO:tasks.rados.rados.0.smithi162.stdout:3736: writing smithi16234619-40 from 494469 to 1149550 tid 1 2024-09-16T15:18:17.837 INFO:tasks.rados.rados.0.smithi162.stdout:3736: writing smithi16234619-40 from 1741862 to 2157641 tid 2 2024-09-16T15:18:17.843 INFO:tasks.rados.rados.0.smithi162.stdout:3736: writing smithi16234619-40 from 2710226 to 3294469 tid 3 2024-09-16T15:18:17.843 INFO:tasks.rados.rados.0.smithi162.stdout:3737: snap_create 2024-09-16T15:18:17.880 INFO:tasks.rados.rados.0.smithi162.stdout:3736: finishing write tid 1 to smithi16234619-40 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: finishing write tid 2 to smithi16234619-40 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: finishing write tid 3 to smithi16234619-40 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: finishing write tid 4 to smithi16234619-40 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: finishing write tid 5 to smithi16234619-40 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: oid 40 updating version 0 to 2329 2024-09-16T15:18:17.882 INFO:tasks.rados.rados.0.smithi162.stdout:3736: oid 40 version 2329 is already newer than 2327 2024-09-16T15:18:17.883 INFO:tasks.rados.rados.0.smithi162.stdout:3736: oid 40 version 2329 is already newer than 2328 2024-09-16T15:18:17.883 INFO:tasks.rados.rados.0.smithi162.stdout:3736: oid 40 updating version 2329 to 2330 2024-09-16T15:18:17.883 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 2330 (ObjNum 1267 snap 394 seq_num 1267) dirty exists 2024-09-16T15:18:17.883 INFO:tasks.rados.rados.0.smithi162.stdout:3736: left oid 40 (ObjNum 1267 snap 394 seq_num 1267) 2024-09-16T15:18:18.670 INFO:tasks.rados.rados.0.smithi162.stdout:3735: done (2 left) 2024-09-16T15:18:18.670 INFO:tasks.rados.rados.0.smithi162.stdout:3736: done (1 left) 2024-09-16T15:18:18.670 INFO:tasks.rados.rados.0.smithi162.stdout:3737: done (0 left) 2024-09-16T15:18:18.671 INFO:tasks.rados.rados.0.smithi162.stdout:3738: delete oid 44 current snap is 395 2024-09-16T15:18:18.674 INFO:tasks.rados.rados.0.smithi162.stdout:3738: done (0 left) 2024-09-16T15:18:18.674 INFO:tasks.rados.rados.0.smithi162.stdout:3739: snap_create 2024-09-16T15:18:19.028 INFO:tasks.rados.rados.0.smithi162.stdout:3739: done (0 left) 2024-09-16T15:18:19.028 INFO:tasks.rados.rados.0.smithi162.stdout:3740: write oid 39 current snap is 396 2024-09-16T15:18:19.028 INFO:tasks.rados.rados.0.smithi162.stdout:3740: seq_num 1269 ranges {528083=405579,1342473=419448,2248719=733949} 2024-09-16T15:18:19.033 INFO:tasks.rados.rados.0.smithi162.stdout:3740: writing smithi16234619-39 from 528083 to 933662 tid 1 2024-09-16T15:18:19.038 INFO:tasks.rados.rados.0.smithi162.stdout:3740: writing smithi16234619-39 from 1342473 to 1761921 tid 2 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3740: writing smithi16234619-39 from 2248719 to 2982668 tid 3 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3741: read oid 45 snap 390 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3741: expect deleted 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3742: rollback oid 47 current snap is 396 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 392 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3743: read oid 8 snap 365 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3743: expect deleted 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3744: rollback oid 24 current snap is 396 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 374 2024-09-16T15:18:19.047 INFO:tasks.rados.rados.0.smithi162.stdout:3745: copy_from oid 10 from oid 49 current snap is 396 2024-09-16T15:18:19.048 INFO:tasks.rados.rados.0.smithi162.stdout:3746: write oid 46 current snap is 396 2024-09-16T15:18:19.048 INFO:tasks.rados.rados.0.smithi162.stdout:3746: seq_num 1270 ranges {544890=597181,1917308=563106,3076142=268748} 2024-09-16T15:18:19.054 INFO:tasks.rados.rados.0.smithi162.stdout:3746: writing smithi16234619-46 from 544890 to 1142071 tid 1 2024-09-16T15:18:19.062 INFO:tasks.rados.rados.0.smithi162.stdout:3746: writing smithi16234619-46 from 1917308 to 2480414 tid 2 2024-09-16T15:18:19.067 INFO:tasks.rados.rados.0.smithi162.stdout:3746: writing smithi16234619-46 from 3076142 to 3344890 tid 3 2024-09-16T15:18:19.067 INFO:tasks.rados.rados.0.smithi162.stdout:3747: write oid 41 current snap is 396 2024-09-16T15:18:19.067 INFO:tasks.rados.rados.0.smithi162.stdout:3742: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:18:19.068 INFO:tasks.rados.rados.0.smithi162.stdout:3742: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:18:19.068 INFO:tasks.rados.rados.0.smithi162.stdout:3742: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:18:19.068 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 2410 (ObjNum 1175 snap 357 seq_num 1175) dirty exists 2024-09-16T15:18:19.068 INFO:tasks.rados.rados.0.smithi162.stdout:3740: finishing write tid 1 to smithi16234619-39 2024-09-16T15:18:19.068 INFO:tasks.rados.rados.0.smithi162.stdout:3747: seq_num 1271 ranges {561697=672430,1975790=706764,3103565=258132} 2024-09-16T15:18:19.074 INFO:tasks.rados.rados.0.smithi162.stdout:3747: writing smithi16234619-41 from 561697 to 1234127 tid 1 2024-09-16T15:18:19.082 INFO:tasks.rados.rados.0.smithi162.stdout:3747: writing smithi16234619-41 from 1975790 to 2682554 tid 2 2024-09-16T15:18:19.086 INFO:tasks.rados.rados.0.smithi162.stdout:3747: writing smithi16234619-41 from 3103565 to 3361697 tid 3 2024-09-16T15:18:19.086 INFO:tasks.rados.rados.0.smithi162.stdout:3741: done (7 left) 2024-09-16T15:18:19.086 INFO:tasks.rados.rados.0.smithi162.stdout:3742: done (6 left) 2024-09-16T15:18:19.086 INFO:tasks.rados.rados.0.smithi162.stdout:3743: done (5 left) 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3748: copy_from oid 4 from oid 32 current snap is 396 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: finishing write tid 2 to smithi16234619-39 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: finishing write tid 3 to smithi16234619-39 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: finishing write tid 4 to smithi16234619-39 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: finishing write tid 5 to smithi16234619-39 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: oid 39 updating version 0 to 2179 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: oid 39 updating version 2179 to 2180 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: oid 39 version 2180 is already newer than 2178 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: oid 39 updating version 2180 to 2181 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 39 v 2181 (ObjNum 1269 snap 396 seq_num 1269) dirty exists 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: left oid 39 (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3740: done (5 left) 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3749: read oid 30 snap 394 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3749: expect (ObjNum 1219 snap 375 seq_num 1219) 2024-09-16T15:18:19.087 INFO:tasks.rados.rados.0.smithi162.stdout:3750: snap_remove snap 389 2024-09-16T15:18:20.036 INFO:tasks.rados.rados.0.smithi162.stdout:3750: done (6 left) 2024-09-16T15:18:20.036 INFO:tasks.rados.rados.0.smithi162.stdout:3751: write (excl) oid 18 current snap is 396 2024-09-16T15:18:20.037 INFO:tasks.rados.rados.0.smithi162.stdout:3751: seq_num 1272 ranges {578504=747679,2034272=566775,3130988=247516} 2024-09-16T15:18:20.044 INFO:tasks.rados.rados.0.smithi162.stdout:3751: writing smithi16234619-18 from 578504 to 1326183 tid 1 2024-09-16T15:18:20.051 INFO:tasks.rados.rados.0.smithi162.stdout:3751: writing smithi16234619-18 from 2034272 to 2601047 tid 2 2024-09-16T15:18:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:3751: writing smithi16234619-18 from 3130988 to 3378504 tid 3 2024-09-16T15:18:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:3752: rollback oid 23 current snap is 396 2024-09-16T15:18:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 23 to 360 2024-09-16T15:18:20.055 INFO:tasks.rados.rados.0.smithi162.stdout:3749: expect (ObjNum 1219 snap 375 seq_num 1219) 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3746: finishing write tid 1 to smithi16234619-46 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3745: finishing copy_from racing read to smithi16234619-10 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3746: finishing write tid 2 to smithi16234619-46 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3746: finishing write tid 3 to smithi16234619-46 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3746: finishing write tid 4 to smithi16234619-46 2024-09-16T15:18:20.168 INFO:tasks.rados.rados.0.smithi162.stdout:3746: finishing write tid 5 to smithi16234619-46 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3746: oid 46 updating version 0 to 1229 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3746: oid 46 updating version 1229 to 1230 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3746: oid 46 version 1230 is already newer than 1228 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3746: oid 46 version 1230 is already newer than 1227 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1230 (ObjNum 1270 snap 396 seq_num 1270) dirty exists 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3746: left oid 46 (ObjNum 1270 snap 396 seq_num 1270) 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3744: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 1708 (ObjNum 65 snap 0 seq_num 1839808179) dirty dne 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3745: finishing copy_from to smithi16234619-10 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 1580 (ObjNum 1250 snap 386 seq_num 1250) dirty exists 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3747: finishing write tid 1 to smithi16234619-41 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3747: finishing write tid 2 to smithi16234619-41 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3747: finishing write tid 3 to smithi16234619-41 2024-09-16T15:18:20.169 INFO:tasks.rados.rados.0.smithi162.stdout:3744: done (7 left) 2024-09-16T15:18:20.170 INFO:tasks.rados.rados.0.smithi162.stdout:3745: done (6 left) 2024-09-16T15:18:20.170 INFO:tasks.rados.rados.0.smithi162.stdout:3746: done (5 left) 2024-09-16T15:18:20.170 INFO:tasks.rados.rados.0.smithi162.stdout:3749: done (4 left) 2024-09-16T15:18:20.170 INFO:tasks.rados.rados.0.smithi162.stdout:3753: snap_remove snap 392 2024-09-16T15:18:21.599 INFO:tasks.rados.rados.0.smithi162.stdout:3753: done (4 left) 2024-09-16T15:18:21.599 INFO:tasks.rados.rados.0.smithi162.stdout:3754: rollback oid 46 current snap is 396 2024-09-16T15:18:21.599 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 380 2024-09-16T15:18:21.599 INFO:tasks.rados.rados.0.smithi162.stdout:3755: read oid 39 snap -1 2024-09-16T15:18:21.599 INFO:tasks.rados.rados.0.smithi162.stdout:3755: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3756: copy_from oid 16 from oid 33 current snap is 396 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: finishing write tid 4 to smithi16234619-41 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: finishing write tid 5 to smithi16234619-41 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: oid 41 updating version 0 to 2325 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: oid 41 updating version 2325 to 2326 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: oid 41 updating version 2326 to 2327 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: oid 41 version 2327 is already newer than 2324 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 2327 (ObjNum 1271 snap 396 seq_num 1271) dirty exists 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3747: left oid 41 (ObjNum 1271 snap 396 seq_num 1271) 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3748: finishing copy_from racing read to smithi16234619-4 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3748: finishing copy_from to smithi16234619-4 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 2323 (ObjNum 1171 snap 353 seq_num 1171) dirty exists 2024-09-16T15:18:21.600 INFO:tasks.rados.rados.0.smithi162.stdout:3752: finishing rollback tid 0 to smithi16234619-23 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3752: finishing rollback tid 1 to smithi16234619-23 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3752: finishing rollback tid 2 to smithi16234619-23 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 1489 (ObjNum 1105 snap 330 seq_num 1105) dirty exists 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: finishing write tid 1 to smithi16234619-18 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: finishing write tid 2 to smithi16234619-18 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: finishing write tid 3 to smithi16234619-18 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: finishing write tid 4 to smithi16234619-18 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: finishing write tid 5 to smithi16234619-18 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: oid 18 updating version 0 to 1522 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: oid 18 updating version 1522 to 1523 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: oid 18 version 1523 is already newer than 1520 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:3751: oid 18 version 1523 is already newer than 1521 2024-09-16T15:18:21.601 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 1523 (ObjNum 1272 snap 396 seq_num 1272) dirty exists 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3751: left oid 18 (ObjNum 1272 snap 396 seq_num 1272) 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3747: done (6 left) 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3748: done (5 left) 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3751: done (4 left) 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3752: done (3 left) 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3757: write oid 42 current snap is 396 2024-09-16T15:18:21.602 INFO:tasks.rados.rados.0.smithi162.stdout:3757: seq_num 1273 ranges {595311=422928,1576401=710433,3042058=353253} 2024-09-16T15:18:21.604 INFO:tasks.rados.rados.0.smithi162.stdout:3757: writing smithi16234619-42 from 595311 to 1018239 tid 1 2024-09-16T15:18:21.612 INFO:tasks.rados.rados.0.smithi162.stdout:3757: writing smithi16234619-42 from 1576401 to 2286834 tid 2 2024-09-16T15:18:21.618 INFO:tasks.rados.rados.0.smithi162.stdout:3757: writing smithi16234619-42 from 3042058 to 3395311 tid 3 2024-09-16T15:18:21.618 INFO:tasks.rados.rados.0.smithi162.stdout:3758: delete oid 5 current snap is 396 2024-09-16T15:18:21.618 INFO:tasks.rados.rados.0.smithi162.stdout:3754: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:18:21.618 INFO:tasks.rados.rados.0.smithi162.stdout:3754: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:18:21.619 INFO:tasks.rados.rados.0.smithi162.stdout:3754: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:18:21.619 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1233 (ObjNum 1207 snap 371 seq_num 1207) dirty exists 2024-09-16T15:18:21.619 INFO:tasks.rados.rados.0.smithi162.stdout:3755: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3756: finishing copy_from racing read to smithi16234619-16 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: finishing write tid 1 to smithi16234619-42 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: finishing write tid 2 to smithi16234619-42 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: finishing write tid 3 to smithi16234619-42 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3756: finishing copy_from to smithi16234619-16 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1721 (ObjNum 1248 snap 386 seq_num 1248) dirty exists 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: finishing write tid 4 to smithi16234619-42 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: finishing write tid 5 to smithi16234619-42 2024-09-16T15:18:21.839 INFO:tasks.rados.rados.0.smithi162.stdout:3757: oid 42 updating version 0 to 2192 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3757: oid 42 version 2192 is already newer than 2190 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3757: oid 42 version 2192 is already newer than 2189 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3757: oid 42 version 2192 is already newer than 2191 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 2192 (ObjNum 1273 snap 396 seq_num 1273) dirty exists 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3757: left oid 42 (ObjNum 1273 snap 396 seq_num 1273) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3754: done (4 left) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3755: done (3 left) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3756: done (2 left) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3757: done (1 left) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3758: done (0 left) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3759: read oid 7 snap -1 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3759: expect (ObjNum 1256 snap 389 seq_num 1256) 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3760: write oid 28 current snap is 396 2024-09-16T15:18:21.840 INFO:tasks.rados.rados.0.smithi162.stdout:3760: seq_num 1275 ranges {628925=573426,1693365=714102,3096904=332021} 2024-09-16T15:18:21.845 INFO:tasks.rados.rados.0.smithi162.stdout:3760: writing smithi16234619-28 from 628925 to 1202351 tid 1 2024-09-16T15:18:21.851 INFO:tasks.rados.rados.0.smithi162.stdout:3760: writing smithi16234619-28 from 1693365 to 2407467 tid 2 2024-09-16T15:18:21.857 INFO:tasks.rados.rados.0.smithi162.stdout:3760: writing smithi16234619-28 from 3096904 to 3428925 tid 3 2024-09-16T15:18:21.857 INFO:tasks.rados.rados.0.smithi162.stdout:3761: rollback oid 11 current snap is 396 2024-09-16T15:18:21.857 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 11 to 390 2024-09-16T15:18:21.857 INFO:tasks.rados.rados.0.smithi162.stdout:3759: expect (ObjNum 1256 snap 389 seq_num 1256) 2024-09-16T15:18:21.920 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-16T15:18:21.920 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-16T15:18:21.920 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-16T15:18:21.920 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-16T15:18:21.923 INFO:tasks.ceph.mon.f:Started 2024-09-16T15:18:21.923 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-16T15:18:21.923 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-16T15:18:21.923 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:18:21.923 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:18:21.925 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:18:21.925 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-16T15:18:21.925 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-16T15:18:21.926 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T15:18:21.926 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T15:18:21.928 INFO:tasks.ceph.mon.c:Started 2024-09-16T15:18:21.928 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:18:21.928 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3761: finishing rollback tid 1 to smithi16234619-11 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1291 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3760: finishing write tid 1 to smithi16234619-28 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3760: finishing write tid 2 to smithi16234619-28 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3759: done (2 left) 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3761: done (1 left) 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3762: read oid 21 snap -1 2024-09-16T15:18:21.962 INFO:tasks.rados.rados.0.smithi162.stdout:3762: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3763: read oid 42 snap 388 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3763: expect (ObjNum 1230 snap 383 seq_num 1230) 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3764: read oid 45 snap -1 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3764: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3765: copy_from oid 31 from oid 17 current snap is 396 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: finishing write tid 3 to smithi16234619-28 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: finishing write tid 4 to smithi16234619-28 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: finishing write tid 5 to smithi16234619-28 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: oid 28 updating version 0 to 1827 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: oid 28 version 1827 is already newer than 1825 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: oid 28 version 1827 is already newer than 1824 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: oid 28 version 1827 is already newer than 1826 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1827 (ObjNum 1275 snap 396 seq_num 1275) dirty exists 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: left oid 28 (ObjNum 1275 snap 396 seq_num 1275) 2024-09-16T15:18:21.963 INFO:tasks.rados.rados.0.smithi162.stdout:3760: done (4 left) 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3766: read oid 50 snap -1 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3766: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3767: rollback oid 8 current snap is 396 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 8 to 382 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3768: read oid 23 snap -1 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3768: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3769: rollback oid 37 current snap is 396 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 37 to 395 2024-09-16T15:18:21.964 INFO:tasks.rados.rados.0.smithi162.stdout:3770: snap_remove snap 316 2024-09-16T15:18:22.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 10 monclient: get_monmap_and_config 2024-09-16T15:18:22.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 10 monclient: build_initial_monmap 2024-09-16T15:18:22.044 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 10 monclient: monmap: 2024-09-16T15:18:22.044 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:22.044 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:22.044140+0000 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:22.044140+0000 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:22.045 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:22.046 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 1 Processor -- start 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 1 -- start start 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.042+0000 7fbcc7e96640 10 monclient: init 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient: _add_conns ranks=[7,10,1,8,5,6,2,9,0,3,4] 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc009cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-i con 0x7fbcc0133a10 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 0x7fbcc01341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-f con 0x7fbcc0133df0 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:22.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbcc0099070 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc0073250 con 0x7fbcc0099070 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc00a3ca0 con 0x7fbcc0133df0 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc009bcf0 con 0x7fbcc0133a10 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc7e96640 10 monclient(hunting): authenticate will time out at 1838.633472s 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 0x7fbcc01341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc009cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 0x7fbcc01341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:53280/0 (socket says 172.21.15.134:53280) 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 1 -- 172.21.15.134:0/2246707095 learned_addr learned my addr 172.21.15.134:0/2246707095 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 10 monclient(hunting): get_auth_request con 0x7fbcc0133df0 auth_method 0 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:22.049 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5431640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): get_auth_request con 0x7fbcc0133a10 auth_method 0 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_done global_id 114280 payload 274 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 -- 172.21.15.134:0/2246707095 >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 msgr2=0x7fbcc01341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 --2- 172.21.15.134:0/2246707095 >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 0x7fbcc01341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 -- 172.21.15.134:0/2246707095 >> v1:172.21.15.162:6789/0 conn(0x7fbcc0099070 legacy=0x7fbcc0099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient: _finish_hunting 0 2024-09-16T15:18:22.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 monclient: found mon.noname-i 2024-09-16T15:18:22.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:22.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:18:22.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.043+0000 7fbcc5c32640 1 -- 172.21.15.134:0/2246707095 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc009bfd0 con 0x7fbcc0133a10 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.046+0000 7fbcc5c32640 1 -- 172.21.15.134:0/2246707095 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 msgr2=0x7fbcc009cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.046+0000 7fbcc5c32640 1 --2- 172.21.15.134:0/2246707095 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc009cbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ba40 tx=0x7fbcbc02c6d0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=ec300b73dbc36ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 1 -- 172.21.15.134:0/2246707095 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcbc031b90 con 0x7fbcc0133a10 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: dump: 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:22.057 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:22.058 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: _finish_auth 0 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: _check_auth_tickets 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 1 -- 172.21.15.134:0/2246707095 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc02f050 con 0x7fbcc0133a10 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 1 -- 172.21.15.134:0/2246707095 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcbc031d50 con 0x7fbcc0133a10 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc4c30640 10 monclient: dump: 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:22.059 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 5 monclient: authenticate success, global_id 114280 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 4 set_mon_vals no callback set 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 10 monclient: shutdown 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 -- 172.21.15.134:0/2246707095 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 msgr2=0x7fbcc009cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/2246707095 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc009cbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ba40 tx=0x7fbcbc02c6d0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 -- 172.21.15.134:0/2246707095 shutdown_connections 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/2246707095 >> v2:172.21.15.162:3300/0 conn(0x7fbcc0133df0 0x7fbcc01341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/2246707095 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc009cbb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 -- 172.21.15.134:0/2246707095 >> 172.21.15.134:0/2246707095 conn(0x7fbcc00a2e20 msgr2=0x7fbcc00a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 -- 172.21.15.134:0/2246707095 shutdown_connections 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.047+0000 7fbcc7e96640 1 -- 172.21.15.134:0/2246707095 wait complete. 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.048+0000 7fbcc7e96640 10 monclient: build_initial_monmap 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.048+0000 7fbcc7e96640 10 monclient: monmap: 2024-09-16T15:18:22.061 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.048+0000 7fbcc7e96640 1 Processor -- start 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.048+0000 7fbcc7e96640 1 -- start start 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.048+0000 7fbcc7e96640 10 monclient: init 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient: _add_conns ranks=[6,7,8,2,0,3,5,4,1] 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-d con 0x7fbcc0099070 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc0130d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-i con 0x7fbcc0133a10 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 0x7fbcc006e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): picked mon.noname-e con 0x7fbcc0133df0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0065de0 con 0x7fbcc0099070 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:22.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:57726/0 (socket says 172.21.15.134:57726) 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00702a0 con 0x7fbcc0133a10 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 -- 172.21.15.134:0/1758206512 learned_addr learned my addr 172.21.15.134:0/1758206512 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01c81b0 con 0x7fbcc0133df0 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc7e96640 10 monclient(hunting): authenticate will time out at 1838.639518s 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): get_auth_request con 0x7fbcc0099070 auth_method 0 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 0x7fbcc006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc0130d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): get_auth_request con 0x7fbcc0133df0 auth_method 0 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): get_auth_request con 0x7fbcc0133a10 auth_method 0 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:22.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5431640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient(hunting): handle_auth_done global_id 94488 payload 824 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 -- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 msgr2=0x7fbcc006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 0x7fbcc006e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 -- 172.21.15.134:0/1758206512 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 msgr2=0x7fbcc0130d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc0130d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient: _finish_hunting 0 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 monclient: found mon.noname-d 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc5c32640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc00b6a0 con 0x7fbcc0099070 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.049+0000 7fbcc6433640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 0x7fbcc006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc5c32640 1 -- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 msgr2=0x7fbcc0130810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc5c32640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02c710 tx=0x7fbcbc0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:22.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcbc031e90 con 0x7fbcc0099070 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: dump: 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:22.067 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 5 monclient: authenticate success, global_id 94488 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 10 monclient: _renew_subs 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc0099c00 con 0x7fbcc0099070 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 10 monclient: _renew_subs 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc00a23e0 con 0x7fbcc0099070 2024-09-16T15:18:22.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc003820 con 0x7fbcc0099070 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbcbc003e00 con 0x7fbcc0099070 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb67fc640 10 monclient: dump: 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:22.069 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.052+0000 7fbcb7fff640 4 set_mon_vals no callback set 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc7e96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc7e96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc7e96640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8005040 con 0x7fbcc0099070 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbcbc003990 con 0x7fbcc0099070 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcb67fc640 1 --2- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 0x7fbca403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(814..814 src has 1..814) v4 ==== 3975+0+0 (secure 0 0 0) 0x7fbcbc06d920 con 0x7fbcc0099070 2024-09-16T15:18:22.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc5431640 1 --2- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 0x7fbca403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:22.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.053+0000 7fbcc5431640 10 monclient: get_auth_request con 0x7fbca4038880 auth_method 0 2024-09-16T15:18:22.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.056+0000 7fbcc5431640 1 --2- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 0x7fbca403ad40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0072210 tx=0x7fbcb0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:22.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.058+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fbcbc0477d0 con 0x7fbcc0099070 2024-09-16T15:18:22.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.058+0000 7fbcb67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:22.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.058+0000 7fbcb67fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:22.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcc7e96640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:22.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcc7e96640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:22.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcc7e96640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:22.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbca8005870 con 0x7fbcc0099070 2024-09-16T15:18:22.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcb67fc640 1 -- 172.21.15.134:0/1758206512 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3179 (secure 0 0 0) 0x7fbcbc04c070 con 0x7fbcc0099070 2024-09-16T15:18:22.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcb67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:22.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.286+0000 7fbcb67fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:22.289 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":138,"quorum":[2,3,5,6,7,8],"quorum_names":["b","g","h","d","i","e"],"quorum_leader_name":"b","quorum_age":91,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 msgr2=0x7fbca403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 0x7fbca403ad40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0072210 tx=0x7fbcb0008040 comp rx=0 tx=0).stop 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 10 monclient: shutdown 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 msgr2=0x7fbcc0130810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02c710 tx=0x7fbcbc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 shutdown_connections 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> v2:172.21.15.162:6800/980547263 conn(0x7fbca4038880 0x7fbca403ad40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbcc0133df0 0x7fbcc006e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbcc0133a10 0x7fbcc0130d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 --2- 172.21.15.134:0/1758206512 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbcc0099070 0x7fbcc0130810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 >> 172.21.15.134:0/1758206512 conn(0x7fbcc00a2e20 msgr2=0x7fbcc0069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 shutdown_connections 2024-09-16T15:18:22.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:22.288+0000 7fbcc7e96640 1 -- 172.21.15.134:0/1758206512 wait complete. 2024-09-16T15:18:22.661 INFO:tasks.rados.rados.0.smithi162.stdout:3770: done (8 left) 2024-09-16T15:18:22.661 INFO:tasks.rados.rados.0.smithi162.stdout:3771: write (excl) oid 25 current snap is 396 2024-09-16T15:18:22.662 INFO:tasks.rados.rados.0.smithi162.stdout:3771: seq_num 1276 ranges {645732=648675,1751847=574113,3124327=321405} 2024-09-16T15:18:22.667 INFO:tasks.rados.rados.0.smithi162.stdout:3771: writing smithi16234619-25 from 645732 to 1294407 tid 1 2024-09-16T15:18:22.674 INFO:tasks.rados.rados.0.smithi162.stdout:3771: writing smithi16234619-25 from 1751847 to 2325960 tid 2 2024-09-16T15:18:22.680 INFO:tasks.rados.rados.0.smithi162.stdout:3771: writing smithi16234619-25 from 3124327 to 3445732 tid 3 2024-09-16T15:18:22.681 INFO:tasks.rados.rados.0.smithi162.stdout:3772: delete oid 22 current snap is 396 2024-09-16T15:18:22.681 INFO:tasks.rados.rados.0.smithi162.stdout:3762: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:22.790 INFO:tasks.rados.rados.0.smithi162.stdout:3767: finishing rollback tid 0 to smithi16234619-8 2024-09-16T15:18:22.790 INFO:tasks.rados.rados.0.smithi162.stdout:3767: finishing rollback tid 1 to smithi16234619-8 2024-09-16T15:18:22.790 INFO:tasks.rados.rados.0.smithi162.stdout:3767: finishing rollback tid 2 to smithi16234619-8 2024-09-16T15:18:22.790 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1526 (ObjNum 1214 snap 374 seq_num 1214) dirty exists 2024-09-16T15:18:22.790 INFO:tasks.rados.rados.0.smithi162.stdout:3764: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:18:22.893 INFO:tasks.rados.rados.0.smithi162.stdout:3763: expect (ObjNum 1230 snap 383 seq_num 1230) 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3769: finishing rollback tid 1 to smithi16234619-37 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 2166 (ObjNum 65 snap 0 seq_num 1840903575) dirty dne 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3765: finishing copy_from racing read to smithi16234619-31 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3765: finishing copy_from to smithi16234619-31 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 1488 (ObjNum 1197 snap 365 seq_num 1197) dirty exists 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3762: done (9 left) 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3763: done (8 left) 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3764: done (7 left) 2024-09-16T15:18:23.121 INFO:tasks.rados.rados.0.smithi162.stdout:3765: done (6 left) 2024-09-16T15:18:23.122 INFO:tasks.rados.rados.0.smithi162.stdout:3767: done (5 left) 2024-09-16T15:18:23.122 INFO:tasks.rados.rados.0.smithi162.stdout:3769: done (4 left) 2024-09-16T15:18:23.122 INFO:tasks.rados.rados.0.smithi162.stdout:3772: done (3 left) 2024-09-16T15:18:23.122 INFO:tasks.rados.rados.0.smithi162.stdout:3773: write oid 8 current snap is 396 2024-09-16T15:18:23.122 INFO:tasks.rados.rados.0.smithi162.stdout:3773: seq_num 1278 ranges {679346=515526,1868811=577782,3179173=300173} 2024-09-16T15:18:23.126 INFO:tasks.rados.rados.0.smithi162.stdout:3773: writing smithi16234619-8 from 679346 to 1194872 tid 1 2024-09-16T15:18:23.133 INFO:tasks.rados.rados.0.smithi162.stdout:3773: writing smithi16234619-8 from 1868811 to 2446593 tid 2 2024-09-16T15:18:23.139 INFO:tasks.rados.rados.0.smithi162.stdout:3773: writing smithi16234619-8 from 3179173 to 3479346 tid 3 2024-09-16T15:18:23.139 INFO:tasks.rados.rados.0.smithi162.stdout:3774: rollback oid 1 current snap is 396 2024-09-16T15:18:23.139 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 1 to 382 2024-09-16T15:18:23.139 INFO:tasks.rados.rados.0.smithi162.stdout:3766: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:18:23.374 INFO:tasks.rados.rados.0.smithi162.stdout:3768: expect (ObjNum 1105 snap 330 seq_num 1105) 2024-09-16T15:18:23.491 INFO:tasks.rados.rados.0.smithi162.stdout:3771: finishing write tid 1 to smithi16234619-25 2024-09-16T15:18:23.491 INFO:tasks.rados.rados.0.smithi162.stdout:3771: finishing write tid 2 to smithi16234619-25 2024-09-16T15:18:23.491 INFO:tasks.rados.rados.0.smithi162.stdout:3771: finishing write tid 3 to smithi16234619-25 2024-09-16T15:18:23.491 INFO:tasks.rados.rados.0.smithi162.stdout:3771: finishing write tid 4 to smithi16234619-25 2024-09-16T15:18:23.491 INFO:tasks.rados.rados.0.smithi162.stdout:3771: finishing write tid 5 to smithi16234619-25 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3771: oid 25 updating version 0 to 2110 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3771: oid 25 updating version 2110 to 2112 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3771: oid 25 version 2112 is already newer than 2111 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3771: oid 25 version 2112 is already newer than 2109 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 2112 (ObjNum 1276 snap 396 seq_num 1276) dirty exists 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3771: left oid 25 (ObjNum 1276 snap 396 seq_num 1276) 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3774: finishing rollback tid 0 to smithi16234619-1 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3774: finishing rollback tid 1 to smithi16234619-1 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3774: finishing rollback tid 2 to smithi16234619-1 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 2326 (ObjNum 1193 snap 365 seq_num 1193) dirty exists 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3773: finishing write tid 1 to smithi16234619-8 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3773: finishing write tid 2 to smithi16234619-8 2024-09-16T15:18:23.492 INFO:tasks.rados.rados.0.smithi162.stdout:3766: done (4 left) 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3768: done (3 left) 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3771: done (2 left) 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3774: done (1 left) 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3775: rollback oid 42 current snap is 396 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 374 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: finishing write tid 3 to smithi16234619-8 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: finishing write tid 4 to smithi16234619-8 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: finishing write tid 5 to smithi16234619-8 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: oid 8 updating version 0 to 1527 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: oid 8 updating version 1527 to 1528 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: oid 8 updating version 1528 to 1530 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:3773: oid 8 version 1530 is already newer than 1529 2024-09-16T15:18:23.493 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 1530 (ObjNum 1278 snap 396 seq_num 1278) dirty exists 2024-09-16T15:18:23.494 INFO:tasks.rados.rados.0.smithi162.stdout:3773: left oid 8 (ObjNum 1278 snap 396 seq_num 1278) 2024-09-16T15:18:23.494 INFO:tasks.rados.rados.0.smithi162.stdout:3773: done (1 left) 2024-09-16T15:18:23.494 INFO:tasks.rados.rados.0.smithi162.stdout:3776: copy_from oid 47 from oid 5 current snap is 396 2024-09-16T15:18:23.494 INFO:tasks.rados.rados.0.smithi162.stdout:3777: write oid 1 current snap is 396 2024-09-16T15:18:23.494 INFO:tasks.rados.rados.0.smithi162.stdout:3777: seq_num 1279 ranges {696153=590775,1927293=721440,3206596=289557} 2024-09-16T15:18:23.497 INFO:tasks.rados.rados.0.smithi162.stdout:3777: writing smithi16234619-1 from 696153 to 1286928 tid 1 2024-09-16T15:18:23.505 INFO:tasks.rados.rados.0.smithi162.stdout:3777: writing smithi16234619-1 from 1927293 to 2648733 tid 2 2024-09-16T15:18:23.509 INFO:tasks.rados.rados.0.smithi162.stdout:3777: writing smithi16234619-1 from 3206596 to 3496153 tid 3 2024-09-16T15:18:23.510 INFO:tasks.rados.rados.0.smithi162.stdout:3778: write (excl) oid 13 current snap is 396 2024-09-16T15:18:23.510 INFO:tasks.rados.rados.0.smithi162.stdout:3778: seq_num 1280 ranges {712960=666024,1985775=581451,3234019=278941} 2024-09-16T15:18:23.517 INFO:tasks.rados.rados.0.smithi162.stdout:3778: writing smithi16234619-13 from 712960 to 1378984 tid 1 2024-09-16T15:18:23.524 INFO:tasks.rados.rados.0.smithi162.stdout:3778: writing smithi16234619-13 from 1985775 to 2567226 tid 2 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3778: writing smithi16234619-13 from 3234019 to 3512960 tid 3 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3779: rollback oid 2 current snap is 396 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 2 to 386 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3776: finishing copy_from to smithi16234619-47 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3776: got expected ENOENT (src dne) 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3775: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3775: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3776: finishing copy_from racing read to smithi16234619-47 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3775: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 2206 (ObjNum 1151 snap 343 seq_num 1151) dirty exists 2024-09-16T15:18:23.530 INFO:tasks.rados.rados.0.smithi162.stdout:3775: done (4 left) 2024-09-16T15:18:23.531 INFO:tasks.rados.rados.0.smithi162.stdout:3776: done (3 left) 2024-09-16T15:18:23.531 INFO:tasks.rados.rados.0.smithi162.stdout:3780: snap_remove snap 360 2024-09-16T15:18:23.665 INFO:tasks.rados.rados.0.smithi162.stdout:3780: done (3 left) 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3781: read oid 16 snap 382 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3781: expect deleted 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3782: read oid 37 snap -1 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3782: expect deleted 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3783: rollback oid 28 current snap is 396 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 28 to 393 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3779: finishing rollback tid 0 to smithi16234619-2 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3779: finishing rollback tid 1 to smithi16234619-2 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3779: finishing rollback tid 2 to smithi16234619-2 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2486 (ObjNum 1011 snap 293 seq_num 1011) dirty exists 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3777: finishing write tid 1 to smithi16234619-1 2024-09-16T15:18:23.669 INFO:tasks.rados.rados.0.smithi162.stdout:3777: finishing write tid 2 to smithi16234619-1 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: finishing write tid 3 to smithi16234619-1 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: finishing write tid 4 to smithi16234619-1 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: finishing write tid 5 to smithi16234619-1 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: oid 1 updating version 0 to 2329 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: oid 1 version 2329 is already newer than 2328 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: oid 1 updating version 2329 to 2330 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: oid 1 version 2330 is already newer than 2327 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 2330 (ObjNum 1279 snap 396 seq_num 1279) dirty exists 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3777: left oid 1 (ObjNum 1279 snap 396 seq_num 1279) 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3778: finishing write tid 1 to smithi16234619-13 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3778: finishing write tid 2 to smithi16234619-13 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3778: finishing write tid 3 to smithi16234619-13 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3778: finishing write tid 4 to smithi16234619-13 2024-09-16T15:18:23.670 INFO:tasks.rados.rados.0.smithi162.stdout:3778: finishing write tid 5 to smithi16234619-13 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: oid 13 updating version 0 to 2211 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: oid 13 version 2211 is already newer than 2210 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: oid 13 version 2211 is already newer than 2209 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: oid 13 version 2211 is already newer than 2208 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 2211 (ObjNum 1280 snap 396 seq_num 1280) dirty exists 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: left oid 13 (ObjNum 1280 snap 396 seq_num 1280) 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3777: done (5 left) 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3778: done (4 left) 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3779: done (3 left) 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3784: copy_from oid 7 from oid 33 current snap is 396 2024-09-16T15:18:23.671 INFO:tasks.rados.rados.0.smithi162.stdout:3785: snap_create 2024-09-16T15:18:23.673 INFO:tasks.rados.rados.0.smithi162.stdout:3783: finishing rollback tid 0 to smithi16234619-28 2024-09-16T15:18:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:3783: finishing rollback tid 1 to smithi16234619-28 2024-09-16T15:18:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:3783: finishing rollback tid 2 to smithi16234619-28 2024-09-16T15:18:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1830 (ObjNum 1259 snap 391 seq_num 1259) dirty exists 2024-09-16T15:18:23.722 INFO:tasks.rados.rados.0.smithi162.stdout:3784: finishing copy_from racing read to smithi16234619-7 2024-09-16T15:18:23.722 INFO:tasks.rados.rados.0.smithi162.stdout:3784: finishing copy_from to smithi16234619-7 2024-09-16T15:18:23.722 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 1721 (ObjNum 1248 snap 386 seq_num 1248) dirty exists 2024-09-16T15:18:24.029 INFO:tasks.rados.rados.0.smithi162.stdout:3781: done (4 left) 2024-09-16T15:18:24.029 INFO:tasks.rados.rados.0.smithi162.stdout:3782: done (3 left) 2024-09-16T15:18:24.029 INFO:tasks.rados.rados.0.smithi162.stdout:3783: done (2 left) 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3784: done (1 left) 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3785: done (0 left) 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3786: read oid 33 snap -1 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3786: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3787: write oid 2 current snap is 397 2024-09-16T15:18:24.030 INFO:tasks.rados.rados.0.smithi162.stdout:3787: seq_num 1281 ranges {729767=741273,1927904=725109,3145089=384678} 2024-09-16T15:18:24.037 INFO:tasks.rados.rados.0.smithi162.stdout:3787: writing smithi16234619-2 from 729767 to 1471040 tid 1 2024-09-16T15:18:24.044 INFO:tasks.rados.rados.0.smithi162.stdout:3787: writing smithi16234619-2 from 1927904 to 2653013 tid 2 2024-09-16T15:18:24.049 INFO:tasks.rados.rados.0.smithi162.stdout:3787: writing smithi16234619-2 from 3145089 to 3529767 tid 3 2024-09-16T15:18:24.049 INFO:tasks.rados.rados.0.smithi162.stdout:3788: read oid 32 snap -1 2024-09-16T15:18:24.049 INFO:tasks.rados.rados.0.smithi162.stdout:3788: expect (ObjNum 1171 snap 353 seq_num 1171) 2024-09-16T15:18:24.049 INFO:tasks.rados.rados.0.smithi162.stdout:3789: copy_from oid 8 from oid 22 current snap is 397 2024-09-16T15:18:24.050 INFO:tasks.rados.rados.0.smithi162.stdout:3786: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:18:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3789: finishing copy_from to smithi16234619-8 2024-09-16T15:18:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3789: got expected ENOENT (src dne) 2024-09-16T15:18:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3789: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:18:24.151 INFO:tasks.rados.rados.0.smithi162.stdout:3788: expect (ObjNum 1171 snap 353 seq_num 1171) 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: finishing write tid 1 to smithi16234619-2 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: finishing write tid 2 to smithi16234619-2 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: finishing write tid 3 to smithi16234619-2 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: finishing write tid 4 to smithi16234619-2 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: finishing write tid 5 to smithi16234619-2 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: oid 2 updating version 0 to 2499 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: oid 2 updating version 2499 to 2501 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: oid 2 version 2501 is already newer than 2500 2024-09-16T15:18:24.315 INFO:tasks.rados.rados.0.smithi162.stdout:3787: oid 2 version 2501 is already newer than 2498 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2501 (ObjNum 1281 snap 397 seq_num 1281) dirty exists 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3787: left oid 2 (ObjNum 1281 snap 397 seq_num 1281) 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3786: done (3 left) 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3787: done (2 left) 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3788: done (1 left) 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3789: done (0 left) 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3790: write oid 50 current snap is 397 2024-09-16T15:18:24.316 INFO:tasks.rados.rados.0.smithi162.stdout:3790: seq_num 1282 ranges {746574=416522,1586386=468767,2772512=629544} 2024-09-16T15:18:24.320 INFO:tasks.rados.rados.0.smithi162.stdout:3790: writing smithi16234619-50 from 746574 to 1163096 tid 1 2024-09-16T15:18:24.325 INFO:tasks.rados.rados.0.smithi162.stdout:3790: writing smithi16234619-50 from 1586386 to 2055153 tid 2 2024-09-16T15:18:24.332 INFO:tasks.rados.rados.0.smithi162.stdout:3790: writing smithi16234619-50 from 2772512 to 3402056 tid 3 2024-09-16T15:18:24.332 INFO:tasks.rados.rados.0.smithi162.stdout:3791: delete oid 32 current snap is 397 2024-09-16T15:18:24.336 INFO:tasks.rados.rados.0.smithi162.stdout:3791: done (1 left) 2024-09-16T15:18:24.336 INFO:tasks.rados.rados.0.smithi162.stdout:3792: delete oid 13 current snap is 397 2024-09-16T15:18:24.341 INFO:tasks.rados.rados.0.smithi162.stdout:3792: done (1 left) 2024-09-16T15:18:24.341 INFO:tasks.rados.rados.0.smithi162.stdout:3793: write oid 30 current snap is 397 2024-09-16T15:18:24.341 INFO:tasks.rados.rados.0.smithi162.stdout:3793: seq_num 1285 ranges {796995=758622,2161832=732447} 2024-09-16T15:18:24.349 INFO:tasks.rados.rados.0.smithi162.stdout:3793: writing smithi16234619-30 from 796995 to 1555617 tid 1 2024-09-16T15:18:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:3793: writing smithi16234619-30 from 2161832 to 2894279 tid 2 2024-09-16T15:18:24.356 INFO:tasks.rados.rados.0.smithi162.stdout:3794: delete oid 2 current snap is 397 2024-09-16T15:18:24.360 INFO:tasks.rados.rados.0.smithi162.stdout:3794: done (2 left) 2024-09-16T15:18:24.360 INFO:tasks.rados.rados.0.smithi162.stdout:3795: copy_from oid 8 from oid 25 current snap is 397 2024-09-16T15:18:24.360 INFO:tasks.rados.rados.0.smithi162.stdout:3796: delete oid 41 current snap is 397 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3790: finishing write tid 1 to smithi16234619-50 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3790: finishing write tid 2 to smithi16234619-50 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3796: done (3 left) 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3797: copy_from oid 29 from oid 39 current snap is 397 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3798: delete oid 36 current snap is 397 2024-09-16T15:18:24.364 INFO:tasks.rados.rados.0.smithi162.stdout:3790: finishing write tid 3 to smithi16234619-50 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: finishing write tid 4 to smithi16234619-50 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: finishing write tid 5 to smithi16234619-50 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: oid 50 updating version 0 to 2132 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: oid 50 updating version 2132 to 2133 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: oid 50 version 2133 is already newer than 2131 2024-09-16T15:18:24.369 INFO:tasks.rados.rados.0.smithi162.stdout:3790: oid 50 version 2133 is already newer than 2130 2024-09-16T15:18:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 2133 (ObjNum 1282 snap 397 seq_num 1282) dirty exists 2024-09-16T15:18:24.370 INFO:tasks.rados.rados.0.smithi162.stdout:3790: left oid 50 (ObjNum 1282 snap 397 seq_num 1282) 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3790: done (4 left) 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3798: done (3 left) 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3799: read oid 32 snap -1 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3799: expect deleted 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3800: write oid 34 current snap is 397 2024-09-16T15:18:24.377 INFO:tasks.rados.rados.0.smithi162.stdout:3800: seq_num 1289 ranges {464223=659618,1879407=739785,3248120=416103} 2024-09-16T15:18:24.383 INFO:tasks.rados.rados.0.smithi162.stdout:3800: writing smithi16234619-34 from 464223 to 1123841 tid 1 2024-09-16T15:18:24.391 INFO:tasks.rados.rados.0.smithi162.stdout:3800: writing smithi16234619-34 from 1879407 to 2619192 tid 2 2024-09-16T15:18:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:3800: writing smithi16234619-34 from 3248120 to 3664223 tid 3 2024-09-16T15:18:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:3801: write oid 35 current snap is 397 2024-09-16T15:18:24.397 INFO:tasks.rados.rados.0.smithi162.stdout:3801: seq_num 1290 ranges {481030=734867,1937889=483443,2875543=552426} 2024-09-16T15:18:24.404 INFO:tasks.rados.rados.0.smithi162.stdout:3801: writing smithi16234619-35 from 481030 to 1215897 tid 1 2024-09-16T15:18:24.411 INFO:tasks.rados.rados.0.smithi162.stdout:3801: writing smithi16234619-35 from 1937889 to 2421332 tid 2 2024-09-16T15:18:24.418 INFO:tasks.rados.rados.0.smithi162.stdout:3801: writing smithi16234619-35 from 2875543 to 3427969 tid 3 2024-09-16T15:18:24.418 INFO:tasks.rados.rados.0.smithi162.stdout:3802: read oid 28 snap -1 2024-09-16T15:18:24.418 INFO:tasks.rados.rados.0.smithi162.stdout:3802: expect (ObjNum 1259 snap 391 seq_num 1259) 2024-09-16T15:18:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3803: write oid 1 current snap is 397 2024-09-16T15:18:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3803: seq_num 1291 ranges {497837=410116,1596371=743454,2902966=763698} 2024-09-16T15:18:24.424 INFO:tasks.rados.rados.0.smithi162.stdout:3803: writing smithi16234619-1 from 497837 to 907953 tid 1 2024-09-16T15:18:24.432 INFO:tasks.rados.rados.0.smithi162.stdout:3803: writing smithi16234619-1 from 1596371 to 2339825 tid 2 2024-09-16T15:18:24.440 INFO:tasks.rados.rados.0.smithi162.stdout:3803: writing smithi16234619-1 from 2902966 to 3666664 tid 3 2024-09-16T15:18:24.440 INFO:tasks.rados.rados.0.smithi162.stdout:3804: write oid 3 current snap is 397 2024-09-16T15:18:24.440 INFO:tasks.rados.rados.0.smithi162.stdout:3804: seq_num 1292 ranges {514644=485365,1654853=487112,2930389=574970} 2024-09-16T15:18:24.446 INFO:tasks.rados.rados.0.smithi162.stdout:3804: writing smithi16234619-3 from 514644 to 1000009 tid 1 2024-09-16T15:18:24.453 INFO:tasks.rados.rados.0.smithi162.stdout:3804: writing smithi16234619-3 from 1654853 to 2141965 tid 2 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3804: writing smithi16234619-3 from 2930389 to 3505359 tid 3 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3805: snap_create 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: finishing write tid 1 to smithi16234619-30 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: finishing write tid 2 to smithi16234619-30 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: finishing write tid 3 to smithi16234619-30 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: finishing write tid 4 to smithi16234619-30 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: oid 30 updating version 0 to 1832 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: oid 30 version 1832 is already newer than 1831 2024-09-16T15:18:24.462 INFO:tasks.rados.rados.0.smithi162.stdout:3793: oid 30 updating version 1832 to 1833 2024-09-16T15:18:24.463 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 1833 (ObjNum 1285 snap 397 seq_num 1285) dirty exists 2024-09-16T15:18:24.463 INFO:tasks.rados.rados.0.smithi162.stdout:3793: left oid 30 (ObjNum 1285 snap 397 seq_num 1285) 2024-09-16T15:18:24.463 INFO:tasks.rados.rados.0.smithi162.stdout:3797: finishing copy_from racing read to smithi16234619-29 2024-09-16T15:18:24.463 INFO:tasks.rados.rados.0.smithi162.stdout:3802: expect (ObjNum 1259 snap 391 seq_num 1259) 2024-09-16T15:18:24.609 INFO:tasks.rados.rados.0.smithi162.stdout:3800: finishing write tid 1 to smithi16234619-34 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3800: finishing write tid 2 to smithi16234619-34 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3795: finishing copy_from racing read to smithi16234619-8 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3795: finishing copy_from to smithi16234619-8 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 2112 (ObjNum 1276 snap 396 seq_num 1276) dirty exists 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: finishing write tid 1 to smithi16234619-35 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: finishing write tid 2 to smithi16234619-35 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: finishing write tid 3 to smithi16234619-35 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: finishing write tid 4 to smithi16234619-35 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: finishing write tid 5 to smithi16234619-35 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: oid 35 updating version 0 to 1844 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: oid 35 updating version 1844 to 1847 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: oid 35 version 1847 is already newer than 1846 2024-09-16T15:18:24.610 INFO:tasks.rados.rados.0.smithi162.stdout:3801: oid 35 version 1847 is already newer than 1845 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1847 (ObjNum 1290 snap 397 seq_num 1290) dirty exists 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3801: left oid 35 (ObjNum 1290 snap 397 seq_num 1290) 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3797: finishing copy_from to smithi16234619-29 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 2181 (ObjNum 1269 snap 396 seq_num 1269) dirty exists 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: finishing write tid 3 to smithi16234619-34 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: finishing write tid 4 to smithi16234619-34 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: finishing write tid 5 to smithi16234619-34 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: oid 34 updating version 0 to 2140 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: oid 34 version 2140 is already newer than 2139 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: oid 34 version 2140 is already newer than 2137 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: oid 34 version 2140 is already newer than 2138 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 2140 (ObjNum 1289 snap 397 seq_num 1289) dirty exists 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3800: left oid 34 (ObjNum 1289 snap 397 seq_num 1289) 2024-09-16T15:18:24.611 INFO:tasks.rados.rados.0.smithi162.stdout:3803: finishing write tid 1 to smithi16234619-1 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: finishing write tid 2 to smithi16234619-1 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: finishing write tid 3 to smithi16234619-1 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: finishing write tid 4 to smithi16234619-1 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: finishing write tid 5 to smithi16234619-1 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: oid 1 updating version 0 to 2334 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: oid 1 version 2334 is already newer than 2332 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: oid 1 version 2334 is already newer than 2333 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: oid 1 version 2334 is already newer than 2331 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 2334 (ObjNum 1291 snap 397 seq_num 1291) dirty exists 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3803: left oid 1 (ObjNum 1291 snap 397 seq_num 1291) 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3804: finishing write tid 1 to smithi16234619-3 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3804: finishing write tid 2 to smithi16234619-3 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3804: finishing write tid 3 to smithi16234619-3 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3804: finishing write tid 4 to smithi16234619-3 2024-09-16T15:18:24.612 INFO:tasks.rados.rados.0.smithi162.stdout:3804: finishing write tid 5 to smithi16234619-3 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:3804: oid 3 updating version 0 to 1814 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:3804: oid 3 version 1814 is already newer than 1813 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:3804: oid 3 updating version 1814 to 1815 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:3804: oid 3 updating version 1815 to 1816 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 3 v 1816 (ObjNum 1292 snap 397 seq_num 1292) dirty exists 2024-09-16T15:18:24.613 INFO:tasks.rados.rados.0.smithi162.stdout:3804: left oid 3 (ObjNum 1292 snap 397 seq_num 1292) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3793: done (9 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3795: done (8 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3797: done (7 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3799: done (6 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3800: done (5 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3801: done (4 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3802: done (3 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3803: done (2 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3804: done (1 left) 2024-09-16T15:18:25.048 INFO:tasks.rados.rados.0.smithi162.stdout:3805: done (0 left) 2024-09-16T15:18:25.049 INFO:tasks.rados.rados.0.smithi162.stdout:3806: snap_remove snap 374 2024-09-16T15:18:25.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.401+0000 7fb0f26e5640 10 monclient: get_monmap_and_config 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.401+0000 7fb0f26e5640 10 monclient: build_initial_monmap 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.401+0000 7fb0f26e5640 10 monclient: monmap: 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:25.402810+0000 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:25.402810+0000 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.403 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:25.404 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.402+0000 7fb0f26e5640 1 Processor -- start 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.402+0000 7fb0f26e5640 1 -- start start 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.402+0000 7fb0f26e5640 10 monclient: init 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient: _add_conns ranks=[10,5,7,2,0,9,4,3,8,6,1] 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-f con 0x7fb0ec136030 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-h con 0x7fb0ec09bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:25.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-i con 0x7fb0ec136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec09b600 con 0x7fb0ec136030 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a5740 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a52c0 con 0x7fb0ec136c60 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f26e5640 10 monclient(hunting): authenticate will time out at 1841.992472s 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:50434/0 (socket says 172.21.15.134:50434) 2024-09-16T15:18:25.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 1 -- 172.21.15.134:0/1846686130 learned_addr learned my addr 172.21.15.134:0/1846686130 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): get_auth_request con 0x7fb0ec136c60 auth_method 0 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): get_auth_request con 0x7fb0ec09bfe0 auth_method 0 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request con 0x7fb0ec136030 auth_method 0 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:25.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 1 -- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 msgr2=0x7fb0ec136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 1 -- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 msgr2=0x7fb0ec136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 1 --2- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.403+0000 7fb0ebfff640 1 --2- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_done global_id 45275 payload 274 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 -- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 msgr2=0x7fb0ec1370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 -- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 msgr2=0x7fb0ec136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:25.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 monclient: found mon.noname-h 2024-09-16T15:18:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0e3fff640 1 -- 172.21.15.134:0/1846686130 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec073250 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.404+0000 7fb0f0c82640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.406+0000 7fb0e3fff640 1 -- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 msgr2=0x7fb0ec09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.406+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec09c400 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d4029fd0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=e726a1ab15e8fbab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 1 -- 172.21.15.134:0/1846686130 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0d402ad30 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: dump: 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:25.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 1 -- 172.21.15.134:0/1846686130 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d40307b0 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 1 -- 172.21.15.134:0/1846686130 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0d4030d90 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0f26e5640 5 monclient: authenticate success, global_id 45275 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0f26e5640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0f26e5640 4 set_mon_vals no callback set 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.407+0000 7fb0eb7fe640 10 monclient: dump: 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.418 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 10 monclient: shutdown 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 -- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 msgr2=0x7fb0ec09c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:25.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec09c400 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d4029fd0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 -- 172.21.15.134:0/1846686130 shutdown_connections 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136c60 0x7fb0ec1370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/1846686130 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec09c400 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/1846686130 >> v2:172.21.15.162:3300/0 conn(0x7fb0ec136030 0x7fb0ec136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 -- 172.21.15.134:0/1846686130 >> 172.21.15.134:0/1846686130 conn(0x7fb0ec0a9ea0 msgr2=0x7fb0ec0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 -- 172.21.15.134:0/1846686130 shutdown_connections 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 1 -- 172.21.15.134:0/1846686130 wait complete. 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 10 monclient: build_initial_monmap 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.408+0000 7fb0f26e5640 10 monclient: monmap: 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:25.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 Processor -- start 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 -- start start 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient: init 2024-09-16T15:18:25.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient: _add_conns ranks=[5,7,8,0,2,6,1,3,4] 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-h con 0x7fb0ec09bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 0x7fb0ec1351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-i con 0x7fb0ec136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb0ec136c60 0x7fb0ec11fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): picked mon.noname-e con 0x7fb0ec136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec1221a0 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a80f0 con 0x7fb0ec136030 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a7c50 con 0x7fb0ec136c60 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f26e5640 10 monclient(hunting): authenticate will time out at 1841.998842s 2024-09-16T15:18:25.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:59220/0 (socket says 172.21.15.134:59220) 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 1 -- 172.21.15.134:0/3641478993 learned_addr learned my addr 172.21.15.134:0/3641478993 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request con 0x7fb0ec09bfe0 auth_method 0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 0x7fb0ec1351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0f0c82640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb0ec136c60 0x7fb0ec11fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0e3fff640 10 monclient(hunting): get_auth_request con 0x7fb0ec136030 auth_method 0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0e3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0e3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.409+0000 7fb0e3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): get_auth_request con 0x7fb0ec136c60 auth_method 0 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:25.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0e3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0f0c82640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient(hunting): handle_auth_done global_id 45284 payload 824 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 -- 172.21.15.134:0/3641478993 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb0ec136c60 msgr2=0x7fb0ec11fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb0ec136c60 0x7fb0ec11fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 -- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 msgr2=0x7fb0ec1351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 0x7fb0ec1351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 monclient: found mon.noname-h 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0ebfff640 1 -- 172.21.15.134:0/3641478993 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009b30 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.410+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 0x7fb0ec1351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0ebfff640 1 -- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 msgr2=0x7fb0ec134ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0ebfff640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00d7d0 tx=0x7fb0dc00dc90 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0dc00bd40 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: dump: 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:25.425 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: _finish_auth 0 2024-09-16T15:18:25.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 5 monclient: authenticate success, global_id 45284 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 10 monclient: _renew_subs 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc014430 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d4009400 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0dc014a10 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0e97fa640 10 monclient: dump: 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:25.427 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0eaffd640 4 set_mon_vals no callback set 2024-09-16T15:18:25.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 10 monclient: _renew_subs 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.412+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec121dc0 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb0dc0145a0 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0f26e5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0f26e5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0f26e5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec09c400 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0e97fa640 1 --2- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 0x7fb0cc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(818..818 src has 1..818) v4 ==== 3947+0+0 (secure 0 0 0) 0x7fb0dc04a780 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 0x7fb0cc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:25.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.413+0000 7fb0e3fff640 10 monclient: get_auth_request con 0x7fb0cc0388b0 auth_method 0 2024-09-16T15:18:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.416+0000 7fb0e3fff640 1 --2- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 0x7fb0cc03ad70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0a51e0 tx=0x7fb0d402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.418+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb0dc014b70 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.418+0000 7fb0e97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:25.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.418+0000 7fb0e97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:25.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0f26e5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:25.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0f26e5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:25.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0f26e5640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:25.641 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb0ec072070 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0e97fa640 1 -- 172.21.15.134:0/3641478993 <== mon.5 v2:172.21.15.162:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3179 (secure 0 0 0) 0x7fb0dc01d3c0 con 0x7fb0ec09bfe0 2024-09-16T15:18:25.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0e97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:25.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.640+0000 7fb0e97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:25.642 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":138,"quorum":[2,3,5,6,7,8],"quorum_names":["b","g","h","d","i","e"],"quorum_leader_name":"b","quorum_age":95,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:25.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 msgr2=0x7fb0cc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:25.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 0x7fb0cc03ad70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0a51e0 tx=0x7fb0d402e000 comp rx=0 tx=0).stop 2024-09-16T15:18:25.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 10 monclient: shutdown 2024-09-16T15:18:25.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:25.644 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 msgr2=0x7fb0ec134ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00d7d0 tx=0x7fb0dc00dc90 comp rx=0 tx=0).stop 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 shutdown_connections 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> v2:172.21.15.162:6800/980547263 conn(0x7fb0cc0388b0 0x7fb0cc03ad70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb0ec136c60 0x7fb0ec11fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.642+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb0ec136030 0x7fb0ec1351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.643+0000 7fb0f26e5640 1 --2- 172.21.15.134:0/3641478993 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb0ec09bfe0 0x7fb0ec134ca0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.643+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 >> 172.21.15.134:0/3641478993 conn(0x7fb0ec0a9ea0 msgr2=0x7fb0ec0a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.643+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 shutdown_connections 2024-09-16T15:18:25.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:25.643+0000 7fb0f26e5640 1 -- 172.21.15.134:0/3641478993 wait complete. 2024-09-16T15:18:26.049 INFO:tasks.rados.rados.0.smithi162.stdout:3806: done (0 left) 2024-09-16T15:18:26.049 INFO:tasks.rados.rados.0.smithi162.stdout:3807: write (excl) oid 50 current snap is 398 2024-09-16T15:18:26.049 INFO:tasks.rados.rados.0.smithi162.stdout:3807: seq_num 1293 ranges {531451=676967,1713335=747123,2957812=502595} 2024-09-16T15:18:26.057 INFO:tasks.rados.rados.0.smithi162.stdout:3807: writing smithi16234619-50 from 531451 to 1208418 tid 1 2024-09-16T15:18:26.063 INFO:tasks.rados.rados.0.smithi162.stdout:3807: writing smithi16234619-50 from 1713335 to 2460458 tid 2 2024-09-16T15:18:26.069 INFO:tasks.rados.rados.0.smithi162.stdout:3807: writing smithi16234619-50 from 2957812 to 3460407 tid 3 2024-09-16T15:18:26.069 INFO:tasks.rados.rados.0.smithi162.stdout:3808: snap_create 2024-09-16T15:18:26.108 INFO:tasks.rados.rados.0.smithi162.stdout:3807: finishing write tid 1 to smithi16234619-50 2024-09-16T15:18:26.108 INFO:tasks.rados.rados.0.smithi162.stdout:3807: finishing write tid 2 to smithi16234619-50 2024-09-16T15:18:26.108 INFO:tasks.rados.rados.0.smithi162.stdout:3807: finishing write tid 3 to smithi16234619-50 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: finishing write tid 4 to smithi16234619-50 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: finishing write tid 5 to smithi16234619-50 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: oid 50 updating version 0 to 2146 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: oid 50 updating version 2146 to 2148 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: oid 50 updating version 2148 to 2149 2024-09-16T15:18:26.110 INFO:tasks.rados.rados.0.smithi162.stdout:3807: oid 50 version 2149 is already newer than 2147 2024-09-16T15:18:26.111 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 2149 (ObjNum 1293 snap 398 seq_num 1293) dirty exists 2024-09-16T15:18:26.111 INFO:tasks.rados.rados.0.smithi162.stdout:3807: left oid 50 (ObjNum 1293 snap 398 seq_num 1293) 2024-09-16T15:18:27.665 INFO:tasks.rados.rados.0.smithi162.stdout:3807: done (1 left) 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3808: done (0 left) 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3809: read oid 21 snap -1 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3809: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3810: read oid 16 snap -1 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3810: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3811: write oid 47 current snap is 399 2024-09-16T15:18:27.666 INFO:tasks.rados.rados.0.smithi162.stdout:3811: seq_num 1294 ranges {548258=752216,1771817=490781,2985235=713867} 2024-09-16T15:18:27.672 INFO:tasks.rados.rados.0.smithi162.stdout:3811: writing smithi16234619-47 from 548258 to 1300474 tid 1 2024-09-16T15:18:27.678 INFO:tasks.rados.rados.0.smithi162.stdout:3811: writing smithi16234619-47 from 1771817 to 2262598 tid 2 2024-09-16T15:18:27.688 INFO:tasks.rados.rados.0.smithi162.stdout:3811: writing smithi16234619-47 from 2985235 to 3699102 tid 3 2024-09-16T15:18:27.688 INFO:tasks.rados.rados.0.smithi162.stdout:3812: delete oid 11 current snap is 399 2024-09-16T15:18:27.688 INFO:tasks.rados.rados.0.smithi162.stdout:3809: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:18:27.799 INFO:tasks.rados.rados.0.smithi162.stdout:3810: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: finishing write tid 1 to smithi16234619-47 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: finishing write tid 2 to smithi16234619-47 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: finishing write tid 3 to smithi16234619-47 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: finishing write tid 4 to smithi16234619-47 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: finishing write tid 5 to smithi16234619-47 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: oid 47 updating version 0 to 2412 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: oid 47 updating version 2412 to 2413 2024-09-16T15:18:27.892 INFO:tasks.rados.rados.0.smithi162.stdout:3811: oid 47 version 2413 is already newer than 2411 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3811: oid 47 updating version 2413 to 2414 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 2414 (ObjNum 1294 snap 399 seq_num 1294) dirty exists 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3811: left oid 47 (ObjNum 1294 snap 399 seq_num 1294) 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3809: done (3 left) 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3810: done (2 left) 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3811: done (1 left) 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3812: done (0 left) 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3813: write oid 28 current snap is 399 2024-09-16T15:18:27.893 INFO:tasks.rados.rados.0.smithi162.stdout:3813: seq_num 1296 ranges {581872=502714,1488781=494450,2640081=736411} 2024-09-16T15:18:27.897 INFO:tasks.rados.rados.0.smithi162.stdout:3813: writing smithi16234619-28 from 581872 to 1084586 tid 1 2024-09-16T15:18:27.902 INFO:tasks.rados.rados.0.smithi162.stdout:3813: writing smithi16234619-28 from 1488781 to 1983231 tid 2 2024-09-16T15:18:27.910 INFO:tasks.rados.rados.0.smithi162.stdout:3813: writing smithi16234619-28 from 2640081 to 3376492 tid 3 2024-09-16T15:18:27.910 INFO:tasks.rados.rados.0.smithi162.stdout:3814: rollback oid 16 current snap is 399 2024-09-16T15:18:27.910 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 16 to 382 2024-09-16T15:18:27.910 INFO:tasks.rados.rados.0.smithi162.stdout:3815: snap_create 2024-09-16T15:18:27.913 INFO:tasks.rados.rados.0.smithi162.stdout:3814: finishing rollback tid 0 to smithi16234619-16 2024-09-16T15:18:27.913 INFO:tasks.rados.rados.0.smithi162.stdout:3814: finishing rollback tid 1 to smithi16234619-16 2024-09-16T15:18:27.913 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1849 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:18:27.943 INFO:tasks.rados.rados.0.smithi162.stdout:3813: finishing write tid 1 to smithi16234619-28 2024-09-16T15:18:27.943 INFO:tasks.rados.rados.0.smithi162.stdout:3813: finishing write tid 2 to smithi16234619-28 2024-09-16T15:18:27.943 INFO:tasks.rados.rados.0.smithi162.stdout:3813: finishing write tid 3 to smithi16234619-28 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: finishing write tid 4 to smithi16234619-28 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: finishing write tid 5 to smithi16234619-28 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: oid 28 updating version 0 to 1837 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: oid 28 version 1837 is already newer than 1835 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: oid 28 version 1837 is already newer than 1836 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: oid 28 version 1837 is already newer than 1834 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 28 v 1837 (ObjNum 1296 snap 399 seq_num 1296) dirty exists 2024-09-16T15:18:27.945 INFO:tasks.rados.rados.0.smithi162.stdout:3813: left oid 28 (ObjNum 1296 snap 399 seq_num 1296) 2024-09-16T15:18:28.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.752+0000 7fb6e6733640 10 monclient: get_monmap_and_config 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.752+0000 7fb6e6733640 10 monclient: build_initial_monmap 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.752+0000 7fb6e6733640 10 monclient: monmap: 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:28.753639+0000 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:28.753639+0000 2024-09-16T15:18:28.754 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:28.755 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:28.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 Processor -- start 2024-09-16T15:18:28.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 -- start start 2024-09-16T15:18:28.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient: init 2024-09-16T15:18:28.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:28.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient: _add_conns ranks=[5,2,8,7,10,1,4,3,6,0,9] 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-h con 0x7fb6e0136080 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-b con 0x7fb6e00a5700 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-e con 0x7fb6e00a44c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6e00a5700 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6e0136080 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009c6a0 con 0x7fb6e00a44c0 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.753+0000 7fb6e6733640 10 monclient(hunting): authenticate will time out at 1845.342979s 2024-09-16T15:18:28.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:44334/0 (socket says 172.21.15.134:44334) 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 learned_addr learned my addr 172.21.15.134:0/2542886867 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6e00a5700 auth_method 0 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:54374/0 (socket says 172.21.15.134:54374) 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6e00a44c0 auth_method 0 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6e0136080 auth_method 0 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6e00a44c0 auth_method 0 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6e00a5700 auth_method 0 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:28.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6e0136080 auth_method 0 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:28.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:28.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6e00a5700 auth_method 0 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:29.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6e00a44c0 auth_method 0 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6e0136080 auth_method 0 2024-09-16T15:18:29.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:29.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:29.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:30.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6e00a5700 auth_method 0 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6e00a44c0 auth_method 0 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:30.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6e0136080 auth_method 0 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.157+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:30.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:30.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:31.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): tick 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 msgr2=0x7fb6e00a4920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6e00a44c0 0x7fb6e00a4920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 msgr2=0x7fb6e0136460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6e0136080 0x7fb6e0136460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 msgr2=0x7fb6e00a3f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6e00a5700 0x7fb6e00a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient: _add_conns ranks=[6,4,9,1,7,0,3,10] 2024-09-16T15:18:31.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-d con 0x7fb6c8003820 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-c con 0x7fb6c8006210 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-a con 0x7fb6c8008d20 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6c8008d20 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6c8006210 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009c6a0 con 0x7fb6c8003820 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6ddffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:31.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8003820 auth_method 0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8008d20 auth_method 0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8006210 auth_method 0 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:31.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:31.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.754+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8008d20 auth_method 0 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:31.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8003820 auth_method 0 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8006210 auth_method 0 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:31.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:31.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:31.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:31.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:31.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:31.955+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8006210 auth_method 0 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:32.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8008d20 auth_method 0 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8003820 auth_method 0 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:32.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:32.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:32.356+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:33.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:33.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:33.159 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8008d20 auth_method 0 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8003820 auth_method 0 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8006210 auth_method 0 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:33.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:33.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:33.157+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:34.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): tick 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 msgr2=0x7fb6c8005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8003820 0x7fb6c8005c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 msgr2=0x7fb6c80086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8006210 0x7fb6c80086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8008d20 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:34.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient: _add_conns ranks=[1,7,0,3,10] 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 reap_dead start 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb6c800bdc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-i con 0x7fb6c800f9a0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb6c8012480 addr v1:172.21.15.134:6789/0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e0073250 con 0x7fb6c8012480 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c8017220 con 0x7fb6c800bdc0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6c800f9a0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6ddffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.754+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 reap_dead start 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:34.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.757+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 legacy=0x7fb6c8014940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:34.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.757+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 legacy=0x7fb6c800e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:34.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:34.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:34.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:35.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:35.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:35.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:36.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.159+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.159+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.159+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:36.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:36.159+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:37.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.012+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 legacy=0x7fb6c800e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T15:18:37.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.012+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 legacy=0x7fb6c800e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:18:37.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.012+0000 7fb6dffff640 1 --1- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 0x7fb6c800e1b0 :-1 s=OPENED pgs=86 cs=1 l=1).handle_message read tag failed 2024-09-16T15:18:37.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.012+0000 7fb6dffff640 1 --1- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 0x7fb6c800e1b0 :-1 s=OPENED pgs=86 cs=1 l=1).fault on lossy channel, failing 2024-09-16T15:18:37.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.012+0000 7fb6deffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.162:6789/0 2024-09-16T15:18:37.036 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.035+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 legacy=0x7fb6c8014940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-09-16T15:18:37.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.035+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 legacy=0x7fb6c8014940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:18:37.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.035+0000 7fb6e4cd0640 1 --1- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 0x7fb6c8014940 :-1 s=OPENED pgs=80 cs=1 l=1).handle_message read tag failed 2024-09-16T15:18:37.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.035+0000 7fb6e4cd0640 1 --1- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 0x7fb6c8014940 :-1 s=OPENED pgs=80 cs=1 l=1).fault on lossy channel, failing 2024-09-16T15:18:37.037 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.035+0000 7fb6deffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.134:6789/0 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): tick 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8011dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c800f9a0 0x7fb6c8011dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c800bdc0 legacy=0x7fb6c800e1b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:37.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.134:6789/0 conn(0x7fb6c8012480 legacy=0x7fb6c8014940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient: _add_conns ranks=[3,10] 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-g con 0x7fb6c8017430 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-f con 0x7fb6c8017d50 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6c8017d50 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6c8017430 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6ddffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:37.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8017430 auth_method 0 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8017d50 auth_method 0 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:37.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:37.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:37.957 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8017d50 auth_method 0 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8017430 auth_method 0 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:37.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:37.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:37.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:37.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:37.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:38.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8017430 auth_method 0 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8017d50 auth_method 0 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:38.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:38.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:38.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:38.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:38.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:38.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8017d50 auth_method 0 2024-09-16T15:18:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:39.160 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8017430 auth_method 0 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:39.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:39.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): tick 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:40.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6c8017430 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 msgr2=0x7fb6c800b1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.162:3300/0 conn(0x7fb6c8017d50 0x7fb6c800b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 reap_dead start 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient: _add_conns ranks=[5,4,8,7,9,0,3,2,6,10,1] 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-h con 0x7fb6c800f9a0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-c con 0x7fb6c8012480 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-e con 0x7fb6c80102c0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:40.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6c8012480 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6c800f9a0 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009c6a0 con 0x7fb6c80102c0 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6ddffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8012480 auth_method 0 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c80102c0 auth_method 0 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.755+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:40.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c80102c0 auth_method 0 2024-09-16T15:18:40.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8012480 auth_method 0 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:40.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:40.956+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:41.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:41.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c80102c0 auth_method 0 2024-09-16T15:18:41.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8012480 auth_method 0 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:41.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:41.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:41.357+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c80102c0 auth_method 0 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c8012480 auth_method 0 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:42.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c800f9a0 auth_method 0 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:42.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:42.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:42.158+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): tick 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 monclient(hunting): continuing hunt 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 msgr2=0x7fb6c800b1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fb6c80102c0 0x7fb6c800b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 msgr2=0x7fb6c8017810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6c800f9a0 0x7fb6c8017810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 msgr2=0x7fb6c800fd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6c8012480 0x7fb6c800fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient: _add_conns ranks=[1,7,9,3,0,10,2,6] 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb6c8010a20 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:43.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-i con 0x7fb6c80086e0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): picked mon.noname-a con 0x7fb6c8011a50 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c8003cb0 con 0x7fb6c8010a20 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6c8011a50 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6c80086e0 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.755+0000 7fb6ddffb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8011a50 auth_method 0 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:43.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c80086e0 auth_method 0 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:43.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.756+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:43.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.758+0000 7fb6dffff640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c8010a20 legacy=0x7fb6c8010e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8011a50 auth_method 0 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:43.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c80086e0 auth_method 0 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:43.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:43.957+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:18:44.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:44.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8011a50 auth_method 0 2024-09-16T15:18:44.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:44.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c80086e0 auth_method 0 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:44.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:44.358+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:18:45.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6c8011a50 auth_method 0 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.159+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6c80086e0 auth_method 0 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:45.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:18:45.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:18:45.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:45.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.160+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:18:45.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939008129 0 0) 0x7fb6c8003cb0 con 0x7fb6c8010a20 2024-09-16T15:18:45.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): _negotiate 2024-09-16T15:18:45.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:45.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): my global_id is 124102 2024-09-16T15:18:45.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6cc003520 con 0x7fb6c8010a20 2024-09-16T15:18:45.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478250473 0 0) 0x7fb6cc003520 con 0x7fb6c8010a20 2024-09-16T15:18:45.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 msgr2=0x7fb6c8011510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:45.666 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6c80086e0 0x7fb6c8011510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 msgr2=0x7fb6c80138c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 --2- 172.21.15.134:0/2542886867 >> v2:172.21.15.134:3300/0 conn(0x7fb6c8011a50 0x7fb6c80138c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _finish_hunting 0 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 monclient: found mon.noname-f-legacy 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.162:6789/0 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8014e10 con 0x7fb6c8010a20 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 reap_dead start 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fb6d0015bb0 con 0x7fb6c8010a20 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:45.667 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: dump: 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:45.668 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 monclient: mon.f has (v2) addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] but i'm connected to v1:172.21.15.162:6789/0, reconnecting 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 >> v1:172.21.15.162:6789/0 conn(0x7fb6c8010a20 legacy=0x7fb6c8010e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient: _add_conns ranks=[2,5,7,6,3,0,4,1,8] 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6cc004200 0x7fb6cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): picked mon.b con 0x7fb6cc004200 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6cc009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): picked mon.h con 0x7fb6cc004e50 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 0x7fb6cc00c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): picked mon.i con 0x7fb6cc009eb0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009bb00 con 0x7fb6cc004200 2024-09-16T15:18:45.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e009b4b0 con 0x7fb6cc004e50 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc00c9d0 con 0x7fb6cc009eb0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.663+0000 7fb6deffd640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 0x7fb6cc00c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6cc009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6cc009eb0 auth_method 0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request con 0x7fb6cc004e50 auth_method 0 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6cc004200 0x7fb6cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6cc004200 auth_method 0 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:45.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_done global_id 124102 payload 293 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 msgr2=0x7fb6cc009970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6cc009970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6cc004200 msgr2=0x7fb6cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6cc004200 0x7fb6cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient: _finish_hunting 0 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 monclient: found mon.i 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc00cc50 con 0x7fb6cc009eb0 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient: _finish_auth 0 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 10 monclient: _check_auth_tickets 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e4cd0640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e6733640 5 monclient: authenticate success, global_id 124102 2024-09-16T15:18:45.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.664+0000 7fb6e6733640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.667+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 msgr2=0x7fb6cc00c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.667+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 0x7fb6cc00c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00a51a0 tx=0x7fb6cc0057f0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=e13fd0b2444122a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d80348a0 con 0x7fb6cc009eb0 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: dump: 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.676 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8034a40 con 0x7fb6cc009eb0 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6e6733640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6e6733640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:45.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 1 -- 172.21.15.134:0/2542886867 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d8034bb0 con 0x7fb6cc009eb0 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6e6733640 4 set_mon_vals no callback set 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.668+0000 7fb6deffd640 10 monclient: dump: 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:45.678 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 10 monclient: shutdown 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 -- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 msgr2=0x7fb6cc00c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 0x7fb6cc00c2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00a51a0 tx=0x7fb6cc0057f0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 -- 172.21.15.134:0/2542886867 shutdown_connections 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fb6cc009eb0 0x7fb6cc00c2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6cc009970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 --2- 172.21.15.134:0/2542886867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6cc004200 0x7fb6cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 -- 172.21.15.134:0/2542886867 >> 172.21.15.134:0/2542886867 conn(0x7fb6e00a99a0 msgr2=0x7fb6e00a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 -- 172.21.15.134:0/2542886867 shutdown_connections 2024-09-16T15:18:45.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.669+0000 7fb6e6733640 1 -- 172.21.15.134:0/2542886867 wait complete. 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.670+0000 7fb6e6733640 10 monclient: build_initial_monmap 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.670+0000 7fb6e6733640 10 monclient: monmap: 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:45.680 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 1 Processor -- start 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 1 -- start start 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 10 monclient: init 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 10 monclient: _add_conns ranks=[6,4,5,3,1,8,2,7,0] 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8010a20 0x7fb6e01cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-d con 0x7fb6c8010a20 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.671+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6cc004200 0x7fb6e01cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-c con 0x7fb6cc004200 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:45.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): picked mon.noname-h con 0x7fb6cc004e50 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01221e0 con 0x7fb6cc004200 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0122710 con 0x7fb6cc004e50 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01229f0 con 0x7fb6c8010a20 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e6733640 10 monclient(hunting): authenticate will time out at 1862.261180s 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3302/0 says I am v2:172.21.15.134:57530/0 (socket says 172.21.15.134:57530) 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/4104746793 learned_addr learned my addr 172.21.15.134:0/4104746793 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request con 0x7fb6cc004e50 auth_method 0 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8010a20 0x7fb6e01cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): get_auth_request con 0x7fb6c8010a20 auth_method 0 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.672+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 10 monclient(hunting): handle_auth_done global_id 45293 payload 824 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6df7fe640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6cc004200 0x7fb6e01cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8010a20 msgr2=0x7fb6e01cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8010a20 0x7fb6e01cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6cc004200 msgr2=0x7fb6e01cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6cc004200 0x7fb6e01cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 10 monclient: _finish_hunting 0 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 monclient: found mon.noname-h 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:18:45.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.673+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/4104746793 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e00a8c60 con 0x7fb6cc004e50 2024-09-16T15:18:45.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e4cd0640 1 -- 172.21.15.134:0/4104746793 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 msgr2=0x7fb6e01cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:45.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e4cd0640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8000f80 tx=0x7fb6d8004ab0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:45.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d8034870 con 0x7fb6cc004e50 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: dump: 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:45.687 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: _finish_auth 0 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: _check_auth_tickets 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8040aa0 con 0x7fb6cc004e50 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 5 monclient: authenticate success, global_id 45293 2024-09-16T15:18:45.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 10 monclient: _renew_subs 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0121720 con 0x7fb6cc004e50 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6d8040c10 con 0x7fb6cc004e50 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6dcff9640 10 monclient: dump: 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:45.689 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 10 monclient: _renew_subs 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0121260 con 0x7fb6cc004e50 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.675+0000 7fb6de7fc640 4 set_mon_vals no callback set 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb6d803c040 con 0x7fb6cc004e50 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6e6733640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:45.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6e6733640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6e6733640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e00a5b60 con 0x7fb6cc004e50 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.676+0000 7fb6dcff9640 1 --2- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 0x7fb6c803d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.677+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 5 ==== osd_map(820..820 src has 1..820) v4 ==== 3963+0+0 (secure 0 0 0) 0x7fb6d806b7b0 con 0x7fb6cc004e50 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.677+0000 7fb6dffff640 1 --2- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 0x7fb6c803d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.677+0000 7fb6dffff640 10 monclient: get_auth_request con 0x7fb6c803b280 auth_method 0 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.680+0000 7fb6dffff640 1 --2- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 0x7fb6c803d740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006df0 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.682+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb6d804a960 con 0x7fb6cc004e50 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.682+0000 7fb6dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:45.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.682+0000 7fb6dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:45.730 INFO:tasks.rados.rados.0.smithi162.stdout:3813: done (2 left) 2024-09-16T15:18:45.730 INFO:tasks.rados.rados.0.smithi162.stdout:3814: done (1 left) 2024-09-16T15:18:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:3815: done (0 left) 2024-09-16T15:18:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:3816: read oid 11 snap -1 2024-09-16T15:18:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:3816: expect deleted 2024-09-16T15:18:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:3817: write (excl) oid 34 current snap is 400 2024-09-16T15:18:45.731 INFO:tasks.rados.rados.0.smithi162.stdout:3817: seq_num 1297 ranges {598679=577963,1947263=638108,3351151=447528} 2024-09-16T15:18:45.737 INFO:tasks.rados.rados.0.smithi162.stdout:3817: writing smithi16234619-34 from 598679 to 1176642 tid 1 2024-09-16T15:18:45.744 INFO:tasks.rados.rados.0.smithi162.stdout:3817: writing smithi16234619-34 from 1947263 to 2585371 tid 2 2024-09-16T15:18:45.751 INFO:tasks.rados.rados.0.smithi162.stdout:3817: writing smithi16234619-34 from 3351151 to 3798679 tid 3 2024-09-16T15:18:45.751 INFO:tasks.rados.rados.0.smithi162.stdout:3818: snap_remove snap 396 2024-09-16T15:18:45.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6e6733640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:45.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6e6733640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:45.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6e6733640 10 monclient: _send_mon_message to mon.h at v2:172.21.15.162:3302/0 2024-09-16T15:18:45.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb6e00a5b60 con 0x7fb6cc004e50 2024-09-16T15:18:45.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6dcff9640 1 -- 172.21.15.134:0/4104746793 <== mon.5 v2:172.21.15.162:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3190 (secure 0 0 0) 0x7fb6d803a020 con 0x7fb6cc004e50 2024-09-16T15:18:45.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6dcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:45.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.924+0000 7fb6dcff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:45.926 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":152,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:45.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 msgr2=0x7fb6c803d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 0x7fb6c803d740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006df0 tx=0x7fb6d0008040 comp rx=0 tx=0).stop 2024-09-16T15:18:45.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 10 monclient: shutdown 2024-09-16T15:18:45.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:45.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 msgr2=0x7fb6e01cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8000f80 tx=0x7fb6d8004ab0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 shutdown_connections 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6c803b280 0x7fb6c803d740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb6cc004e50 0x7fb6e01cbc10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb6cc004200 0x7fb6e01cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 --2- 172.21.15.134:0/4104746793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb6c8010a20 0x7fb6e01cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.926+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 >> 172.21.15.134:0/4104746793 conn(0x7fb6e00a99a0 msgr2=0x7fb6e00a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.927+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 shutdown_connections 2024-09-16T15:18:45.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:45.927+0000 7fb6e6733640 1 -- 172.21.15.134:0/4104746793 wait complete. 2024-09-16T15:18:47.678 INFO:tasks.rados.rados.0.smithi162.stdout:3818: done (2 left) 2024-09-16T15:18:47.678 INFO:tasks.rados.rados.0.smithi162.stdout:3819: read oid 15 snap -1 2024-09-16T15:18:47.678 INFO:tasks.rados.rados.0.smithi162.stdout:3819: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:47.678 INFO:tasks.rados.rados.0.smithi162.stdout:3820: write (excl) oid 4 current snap is 400 2024-09-16T15:18:47.678 INFO:tasks.rados.rados.0.smithi162.stdout:3820: seq_num 1298 ranges {615486=653212,1889392=498119,2978574=475308} 2024-09-16T15:18:47.684 INFO:tasks.rados.rados.0.smithi162.stdout:3820: writing smithi16234619-4 from 615486 to 1268698 tid 1 2024-09-16T15:18:47.691 INFO:tasks.rados.rados.0.smithi162.stdout:3820: writing smithi16234619-4 from 1889392 to 2387511 tid 2 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3820: writing smithi16234619-4 from 2978574 to 3453882 tid 3 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3821: snap_create 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3817: finishing write tid 1 to smithi16234619-34 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3817: finishing write tid 2 to smithi16234619-34 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3817: finishing write tid 3 to smithi16234619-34 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3817: finishing write tid 4 to smithi16234619-34 2024-09-16T15:18:47.697 INFO:tasks.rados.rados.0.smithi162.stdout:3817: finishing write tid 5 to smithi16234619-34 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3817: oid 34 updating version 0 to 2156 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3817: oid 34 updating version 2156 to 2157 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3817: oid 34 version 2157 is already newer than 2155 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3817: oid 34 updating version 2157 to 2158 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 2158 (ObjNum 1297 snap 400 seq_num 1297) dirty exists 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3817: left oid 34 (ObjNum 1297 snap 400 seq_num 1297) 2024-09-16T15:18:47.698 INFO:tasks.rados.rados.0.smithi162.stdout:3819: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:47.764 INFO:tasks.rados.rados.0.smithi162.stdout:3820: finishing write tid 1 to smithi16234619-4 2024-09-16T15:18:47.764 INFO:tasks.rados.rados.0.smithi162.stdout:3820: finishing write tid 2 to smithi16234619-4 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: finishing write tid 3 to smithi16234619-4 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: finishing write tid 4 to smithi16234619-4 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: finishing write tid 5 to smithi16234619-4 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: oid 4 updating version 0 to 2338 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: oid 4 version 2338 is already newer than 2337 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: oid 4 version 2338 is already newer than 2335 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: oid 4 version 2338 is already newer than 2336 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 4 v 2338 (ObjNum 1298 snap 400 seq_num 1298) dirty exists 2024-09-16T15:18:47.765 INFO:tasks.rados.rados.0.smithi162.stdout:3820: left oid 4 (ObjNum 1298 snap 400 seq_num 1298) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3816: done (4 left) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3817: done (3 left) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3819: done (2 left) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3820: done (1 left) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3821: done (0 left) 2024-09-16T15:18:48.692 INFO:tasks.rados.rados.0.smithi162.stdout:3822: delete oid 28 current snap is 401 2024-09-16T15:18:48.697 INFO:tasks.rados.rados.0.smithi162.stdout:3822: done (0 left) 2024-09-16T15:18:48.697 INFO:tasks.rados.rados.0.smithi162.stdout:3823: snap_remove snap 365 2024-09-16T15:18:48.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:49.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.039+0000 7f77facf0640 10 monclient: get_monmap_and_config 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.039+0000 7f77facf0640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.039+0000 7f77facf0640 10 monclient: monmap: 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:49.040404+0000 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:49.040404+0000 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.041 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:49.042 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.040+0000 7f77facf0640 1 Processor -- start 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.040+0000 7f77facf0640 1 -- start start 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.040+0000 7f77facf0640 10 monclient: init 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient: _add_conns ranks=[1,9,4,6,7,8,5,3,2,10,0] 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f77f4136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:49.045 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-a con 0x7f77f409bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-c con 0x7f77f413bf80 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f4073250 con 0x7f77f4136030 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f409b600 con 0x7f77f409bfe0 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40a5740 con 0x7f77f413bf80 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77facf0640 10 monclient(hunting): authenticate will time out at 1865.630214s 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38658/0 (socket says 172.21.15.134:38658) 2024-09-16T15:18:49.046 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 -- 172.21.15.134:0/655624213 learned_addr learned my addr 172.21.15.134:0/655624213 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33436/0 (socket says 172.21.15.134:33436) 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): get_auth_request con 0x7f77f413bf80 auth_method 0 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): get_auth_request con 0x7f77f409bfe0 auth_method 0 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient(hunting): handle_auth_done global_id 124105 payload 274 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 -- 172.21.15.134:0/655624213 >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 msgr2=0x7f77f409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 --2- 172.21.15.134:0/655624213 >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 -- 172.21.15.134:0/655624213 >> v1:172.21.15.162:6789/0 conn(0x7f77f4136030 legacy=0x7f77f4136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 monclient: found mon.noname-c 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f928d640 1 -- 172.21.15.134:0/655624213 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40a52c0 con 0x7f77f413bf80 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.041+0000 7f77f3fff640 1 --2- 172.21.15.134:0/655624213 >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f928d640 1 -- 172.21.15.134:0/655624213 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 msgr2=0x7f77f413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f928d640 1 --2- 172.21.15.134:0/655624213 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009e50 tx=0x7f77ec02a5b0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=f67c5e0a64165d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 1 -- 172.21.15.134:0/655624213 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77ec007a70 con 0x7f77f413bf80 2024-09-16T15:18:49.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.054 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: dump: 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.055 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77facf0640 5 monclient: authenticate success, global_id 124105 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 1 -- 172.21.15.134:0/655624213 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec02d050 con 0x7f77f413bf80 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77facf0640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 1 -- 172.21.15.134:0/655624213 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77ec007bf0 con 0x7f77f413bf80 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77facf0640 4 set_mon_vals no callback set 2024-09-16T15:18:49.056 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.044+0000 7f77f37fe640 10 monclient: dump: 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.057 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 10 monclient: shutdown 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 -- 172.21.15.134:0/655624213 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 msgr2=0x7f77f413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 --2- 172.21.15.134:0/655624213 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009e50 tx=0x7f77ec02a5b0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 -- 172.21.15.134:0/655624213 shutdown_connections 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 --2- 172.21.15.134:0/655624213 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f77f413bf80 0x7f77f413e370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 --2- 172.21.15.134:0/655624213 >> v2:172.21.15.134:3300/0 conn(0x7f77f409bfe0 0x7f77f409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 -- 172.21.15.134:0/655624213 >> 172.21.15.134:0/655624213 conn(0x7f77f40a9ea0 msgr2=0x7f77f40a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 -- 172.21.15.134:0/655624213 shutdown_connections 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 1 -- 172.21.15.134:0/655624213 wait complete. 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.045+0000 7f77facf0640 10 monclient: monmap: 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.059 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 Processor -- start 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 -- start start 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient: init 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient: _add_conns ranks=[2,6,5,7,1,0,3,8,4] 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-b con 0x7f77f409bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-d con 0x7f77f4136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:49.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f77f413bf80 0x7f77f4138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): picked mon.noname-h con 0x7f77f413bf80 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41cc210 con 0x7f77f409bfe0 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4120020 con 0x7f77f413bf80 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77f411fa80 con 0x7f77f4136030 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77facf0640 10 monclient(hunting): authenticate will time out at 1865.635897s 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77f8a8c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.046+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40346/0 (socket says 172.21.15.134:40346) 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47176/0 (socket says 172.21.15.134:47176) 2024-09-16T15:18:49.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 -- 172.21.15.134:0/4085119912 learned_addr learned my addr 172.21.15.134:0/4085119912 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): get_auth_request con 0x7f77f4136030 auth_method 0 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): get_auth_request con 0x7f77f409bfe0 auth_method 0 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f77f413bf80 0x7f77f4138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): get_auth_request con 0x7f77f413bf80 auth_method 0 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f928d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient(hunting): handle_auth_done global_id 94506 payload 824 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 -- 172.21.15.134:0/4085119912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f77f413bf80 msgr2=0x7f77f4138b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f77f413bf80 0x7f77f4138b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 -- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 msgr2=0x7f77f41cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 monclient: found mon.noname-d 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f3fff640 1 -- 172.21.15.134:0/4085119912 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009400 con 0x7f77f4136030 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.047+0000 7f77f8a8c640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f3fff640 1 -- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 msgr2=0x7f77f4133e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f3fff640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f77e000ec10 tx=0x7f77e000c520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77e0009760 con 0x7f77f4136030 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: dump: 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.064 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.065 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 5 monclient: authenticate success, global_id 94506 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e000ce40 con 0x7f77f4136030 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77e0013d50 con 0x7f77f4136030 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 10 monclient: _renew_subs 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec009b30 con 0x7f77f4136030 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f2ffd640 4 set_mon_vals no callback set 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77f17fa640 10 monclient: dump: 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.066 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 10 monclient: _renew_subs 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.049+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40a7fe0 con 0x7f77f4136030 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f77e0013470 con 0x7f77f4136030 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77facf0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77facf0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77facf0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f409fa20 con 0x7f77f4136030 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77f17fa640 1 --2- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 0x7f77d803ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.050+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(823..823 src has 1..823) v4 ==== 3979+0+0 (secure 0 0 0) 0x7f77e004a520 con 0x7f77f4136030 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.051+0000 7f77f8a8c640 1 --2- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 0x7f77d803ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.051+0000 7f77f8a8c640 10 monclient: get_auth_request con 0x7f77d80387b0 auth_method 0 2024-09-16T15:18:49.068 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.053+0000 7f77f8a8c640 1 --2- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 0x7f77d803ac70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f77e8004320 tx=0x7f77e8007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.055+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f77e001c7d0 con 0x7f77f4136030 2024-09-16T15:18:49.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.055+0000 7f77f17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.055+0000 7f77f17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:49.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77facf0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77facf0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77facf0640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f77f409c400 con 0x7f77f4136030 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77f17fa640 1 -- 172.21.15.134:0/4085119912 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f77e0017020 con 0x7f77f4136030 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77f17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.279+0000 7f77f17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 msgr2=0x7f77d803ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 0x7f77d803ac70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f77e8004320 tx=0x7f77e8007e30 comp rx=0 tx=0).stop 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 10 monclient: shutdown 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 msgr2=0x7f77f4133e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f77e000ec10 tx=0x7f77e000c520 comp rx=0 tx=0).stop 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 shutdown_connections 2024-09-16T15:18:49.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> v2:172.21.15.162:6800/980547263 conn(0x7f77d80387b0 0x7f77d803ac70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f77f413bf80 0x7f77f4138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f77f4136030 0x7f77f4133e30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 --2- 172.21.15.134:0/4085119912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77f409bfe0 0x7f77f41cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 >> 172.21.15.134:0/4085119912 conn(0x7f77f40a9ea0 msgr2=0x7f77f4134e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 shutdown_connections 2024-09-16T15:18:49.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.281+0000 7f77facf0640 1 -- 172.21.15.134:0/4085119912 wait complete. 2024-09-16T15:18:49.293 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:18:49.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:18:49.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:18:49.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.389+0000 7f5535472640 10 monclient: get_monmap_and_config 2024-09-16T15:18:49.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.389+0000 7f5535472640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.389+0000 7f5535472640 10 monclient: monmap: 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:49.390324+0000 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:49.390324+0000 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.391 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.392 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.392 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:49.392 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:49.392 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.389+0000 7f5535472640 1 Processor -- start 2024-09-16T15:18:49.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 -- start start 2024-09-16T15:18:49.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient: init 2024-09-16T15:18:49.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient: _add_conns ranks=[4,1,6,10,3,7,9,0,5,8,2] 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): picked mon.noname-c con 0x7f5530136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5530137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 0x7f553013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): picked mon.noname-d con 0x7f553013c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5530073250 con 0x7f5530137220 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55300a56e0 con 0x7f5530136e40 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f553009abc0 con 0x7f553013c370 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f5535472640 10 monclient(hunting): authenticate will time out at 1865.979563s 2024-09-16T15:18:49.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 0x7f553013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38660/0 (socket says 172.21.15.134:38660) 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 1 -- 172.21.15.134:0/3849669793 learned_addr learned my addr 172.21.15.134:0/3849669793 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): get_auth_request con 0x7f5530136e40 auth_method 0 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): get_auth_request con 0x7f553013c370 auth_method 0 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient(hunting): handle_auth_done global_id 94515 payload 274 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 -- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 msgr2=0x7f553009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 -- 172.21.15.134:0/3849669793 >> v1:172.21.15.162:6789/0 conn(0x7f5530137220 legacy=0x7f5530137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 monclient: found mon.noname-d 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.390+0000 7f552f7fe640 1 -- 172.21.15.134:0/3849669793 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553009ba50 con 0x7f553013c370 2024-09-16T15:18:49.399 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.391+0000 7f552effd640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552f7fe640 1 -- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 msgr2=0x7f553013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552f7fe640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 0x7f553013e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55200097d0 tx=0x7f552002a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b9fb61c1e5b3540a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 1 -- 172.21.15.134:0/3849669793 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f552002ad20 con 0x7f553013c370 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: dump: 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.406 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 1 -- 172.21.15.134:0/3849669793 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55200307b0 con 0x7f553013c370 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f5535472640 5 monclient: authenticate success, global_id 94515 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f5535472640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f5535472640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f5535472640 4 set_mon_vals no callback set 2024-09-16T15:18:49.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 1 -- 172.21.15.134:0/3849669793 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f552002aea0 con 0x7f553013c370 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.393+0000 7f552dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f552dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f552dffb640 10 monclient: dump: 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.408 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 10 monclient: shutdown 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 -- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 msgr2=0x7f553013e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 0x7f553013e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f55200097d0 tx=0x7f552002a000 comp rx=0 tx=0).stop 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 -- 172.21.15.134:0/3849669793 shutdown_connections 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f553013c370 0x7f553013e760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 --2- 172.21.15.134:0/3849669793 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f5530136e40 0x7f553009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 -- 172.21.15.134:0/3849669793 >> 172.21.15.134:0/3849669793 conn(0x7f55300a6110 msgr2=0x7f55300a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 -- 172.21.15.134:0/3849669793 shutdown_connections 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.394+0000 7f5535472640 1 -- 172.21.15.134:0/3849669793 wait complete. 2024-09-16T15:18:49.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.395+0000 7f5535472640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.395+0000 7f5535472640 10 monclient: monmap: 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.410 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.395+0000 7f5535472640 1 Processor -- start 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.395+0000 7f5535472640 1 -- start start 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.395+0000 7f5535472640 10 monclient: init 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient: _add_conns ranks=[1,3,7,5,2,0,6,4,8] 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5530136e40 0x7f553011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): picked mon.noname-f con 0x7f5530136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 0x7f55301201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): picked mon.noname-g con 0x7f5530137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 0x7f55301206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): picked mon.noname-i con 0x7f553013c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300a7960 con 0x7f5530136e40 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55300a7410 con 0x7f5530137220 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f5530121e50 con 0x7f553013c370 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f5535472640 10 monclient(hunting): authenticate will time out at 1865.985099s 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 0x7f55301201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 0x7f55301206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5530136e40 0x7f553011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 0x7f55301201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:48092/0 (socket says 172.21.15.134:48092) 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 1 -- 172.21.15.134:0/2184926973 learned_addr learned my addr 172.21.15.134:0/2184926973 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): get_auth_request con 0x7f5530137220 auth_method 0 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): get_auth_request con 0x7f553013c370 auth_method 0 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): get_auth_request con 0x7f5530136e40 auth_method 0 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient(hunting): handle_auth_done global_id 114316 payload 824 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 msgr2=0x7f55301201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 0x7f55301201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5530136e40 msgr2=0x7f553011fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5530136e40 0x7f553011fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 monclient: found mon.noname-i 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009b30 con 0x7f553013c370 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552e7fc640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5530137220 0x7f55301201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.396+0000 7f552effd640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5530136e40 0x7f553011fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 msgr2=0x7f55301206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f552f7fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 0x7f55301206e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5520034000 tx=0x7f55200046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5520003610 con 0x7f553013c370 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: dump: 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.415 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520003770 con 0x7f553013c370 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5520003d50 con 0x7f553013c370 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:49.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f55277fe640 10 monclient: dump: 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 5 monclient: authenticate success, global_id 114316 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 10 monclient: _renew_subs 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f552d7fa640 4 set_mon_vals no callback set 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5520009400 con 0x7f553013c370 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 10 monclient: _renew_subs 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.399+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5530120ef0 con 0x7f553013c370 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f5535472640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f5535472640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 0x7f5530135b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530071b20 con 0x7f5530135720 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 0x7f5530135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient: get_auth_request con 0x7f5530135720 auth_method 0 2024-09-16T15:18:49.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f55200038e0 con 0x7f553013c370 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 10 monclient(hunting): handle_auth_done global_id 124110 payload 824 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552effd640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5530061ae0 con 0x7f5530135720 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f55277fe640 1 --2- 172.21.15.134:0/2184926973 >> v2:172.21.15.162:6800/980547263 conn(0x7f5510038880 0x7f551003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(823..823 src has 1..823) v4 ==== 3979+0+0 (secure 0 0 0) 0x7f552006a950 con 0x7f553013c370 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552e7fc640 1 --2- 172.21.15.134:0/2184926973 >> v2:172.21.15.162:6800/980547263 conn(0x7f5510038880 0x7f551003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.400+0000 7f552e7fc640 10 monclient: get_auth_request con 0x7f5510038880 auth_method 0 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f552effd640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 msgr2=0x7f5530135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f552effd640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 0x7f5530135b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f552800ef40 tx=0x7f5528007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5528015ce0 con 0x7f5530135720 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f55277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f55277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.402+0000 7f55277fe640 10 monclient: dump: 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.420 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f552800f040 con 0x7f5530135720 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 msgr2=0x7f5530135b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 0x7f5530135b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f552800ef40 tx=0x7f5528007880 comp rx=0 tx=0).stop 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 msgr2=0x7f5530135b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f55277fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530135720 0x7f5530135b00 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f552800ef40 tx=0x7f5528007880 comp rx=0 tx=0).stop 2024-09-16T15:18:49.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.403+0000 7f552e7fc640 1 --2- 172.21.15.134:0/2184926973 >> v2:172.21.15.162:6800/980547263 conn(0x7f5510038880 0x7f551003ad40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55180043e0 tx=0x7f5518007ef0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f5535472640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f5535472640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 0x7f553009e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f5535472640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553007a680 con 0x7f5530136040 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 0x7f553009e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient: get_auth_request con 0x7f5530136040 auth_method 0 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 10 monclient(hunting): handle_auth_done global_id 124119 payload 824 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.409+0000 7f552effd640 1 -- 172.21.15.134:0/2184926973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f55300a53c0 con 0x7f5530136040 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f552effd640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 msgr2=0x7f553009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f552effd640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 0x7f553009e520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f552800b010 tx=0x7f5528004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5528021320 con 0x7f5530136040 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f55277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f55277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.411+0000 7f55277fe640 10 monclient: dump: 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.424 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9198 (secure 0 0 0) 0x7f5528021a60 con 0x7f5530136040 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:49.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 msgr2=0x7f553009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 0x7f553009e520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f552800b010 tx=0x7f5528004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 msgr2=0x7f553009e520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f55277fe640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5530136040 0x7f553009e520 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f552800b010 tx=0x7f5528004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:49.426 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.427 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.428 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.429 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.430 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.431 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.433 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.434 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.435 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.437 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:49.439 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.440 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.441 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.442 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.443 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.444 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.445 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.446 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:49.447 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 >> v2:172.21.15.162:6800/980547263 conn(0x7f5510038880 msgr2=0x7f551003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 --2- 172.21.15.134:0/2184926973 >> v2:172.21.15.162:6800/980547263 conn(0x7f5510038880 0x7f551003ad40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f55180043e0 tx=0x7f5518007ef0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 10 monclient: shutdown 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 msgr2=0x7f55301206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 --2- 172.21.15.134:0/2184926973 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f553013c370 0x7f55301206e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5520034000 tx=0x7f55200046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 reap_dead start 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f552f7fe640 1 -- 172.21.15.134:0/2184926973 reap_dead start 2024-09-16T15:18:49.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 shutdown_connections 2024-09-16T15:18:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 >> 172.21.15.134:0/2184926973 conn(0x7f55300a6110 msgr2=0x7f553006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 shutdown_connections 2024-09-16T15:18:49.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.412+0000 7f5535472640 1 -- 172.21.15.134:0/2184926973 wait complete. 2024-09-16T15:18:49.449 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:49.449 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:18:49.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.545+0000 7f4731d3e640 10 monclient: get_monmap_and_config 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.545+0000 7f4731d3e640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.546+0000 7f4731d3e640 10 monclient: monmap: 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.547 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:49.546939+0000 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:49.546939+0000 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.548 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.549 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:49.549 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:49.549 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.546+0000 7f4731d3e640 1 Processor -- start 2024-09-16T15:18:49.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.546+0000 7f4731d3e640 1 -- start start 2024-09-16T15:18:49.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.546+0000 7f4731d3e640 10 monclient: init 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient: _add_conns ranks=[3,8,9,10,6,2,7,5,1,0,4] 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c136e40 0x7f472c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-g con 0x7f472c136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-e con 0x7f472c137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 0x7f472c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-a con 0x7f472c135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472c09abc0 con 0x7f472c135dc0 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c09ba50 con 0x7f472c136e40 2024-09-16T15:18:49.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f472c0a53c0 con 0x7f472c137220 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f4731d3e640 10 monclient(hunting): authenticate will time out at 1866.136270s 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 0x7f472c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41700/0 (socket says 172.21.15.134:41700) 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): get_auth_request con 0x7f472c135dc0 auth_method 0 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 -- 172.21.15.134:0/233006414 learned_addr learned my addr 172.21.15.134:0/233006414 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): get_auth_request con 0x7f472c137220 auth_method 0 2024-09-16T15:18:49.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472b7fe640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c136e40 0x7f472c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472b7fe640 10 monclient(hunting): get_auth_request con 0x7f472c136e40 auth_method 0 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient(hunting): handle_auth_done global_id 84842 payload 274 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 -- 172.21.15.134:0/233006414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c136e40 msgr2=0x7f472c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c136e40 0x7f472c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 -- 172.21.15.134:0/233006414 >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 msgr2=0x7f472c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 --2- 172.21.15.134:0/233006414 >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 0x7f472c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 monclient: found mon.noname-e 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472affd640 1 -- 172.21.15.134:0/233006414 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c073250 con 0x7f472c137220 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.547+0000 7f472bfff640 1 --2- 172.21.15.134:0/233006414 >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 0x7f472c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472affd640 1 -- 172.21.15.134:0/233006414 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 msgr2=0x7f472c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472affd640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f472002a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=1ecb33ad622f19e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 1 -- 172.21.15.134:0/233006414 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f472002adf0 con 0x7f472c137220 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: dump: 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.561 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 1 -- 172.21.15.134:0/233006414 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47200307b0 con 0x7f472c137220 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f4731d3e640 5 monclient: authenticate success, global_id 84842 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f4731d3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f4731d3e640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f4731d3e640 4 set_mon_vals no callback set 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 1 -- 172.21.15.134:0/233006414 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4720030d90 con 0x7f472c137220 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.550+0000 7f472a7fc640 10 monclient: dump: 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.563 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.564 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 10 monclient: shutdown 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 -- 172.21.15.134:0/233006414 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 msgr2=0x7f472c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f472002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 -- 172.21.15.134:0/233006414 shutdown_connections 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 --2- 172.21.15.134:0/233006414 >> v2:172.21.15.134:3300/0 conn(0x7f472c135dc0 0x7f472c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c137600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 --2- 172.21.15.134:0/233006414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c136e40 0x7f472c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 -- 172.21.15.134:0/233006414 >> 172.21.15.134:0/233006414 conn(0x7f472c0a6110 msgr2=0x7f472c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 -- 172.21.15.134:0/233006414 shutdown_connections 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 1 -- 172.21.15.134:0/233006414 wait complete. 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.565 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.551+0000 7f4731d3e640 10 monclient: monmap: 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.566 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 Processor -- start 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 -- start start 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient: init 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient: _add_conns ranks=[3,1,8,4,5,0,2,7,6] 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c135dc0 0x7f472c09f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-g con 0x7f472c135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f472c136e40 0x7f472c134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-f con 0x7f472c136e40 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:49.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): picked mon.noname-e con 0x7f472c137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c122330 con 0x7f472c136e40 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c121190 con 0x7f472c135dc0 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1363b0 con 0x7f472c137220 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.552+0000 7f4731d3e640 10 monclient(hunting): authenticate will time out at 1866.141959s 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41716/0 (socket says 172.21.15.134:41716) 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 learned_addr learned my addr 172.21.15.134:0/27420487 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): get_auth_request con 0x7f472c137220 auth_method 0 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c135dc0 0x7f472c09f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f472c136e40 0x7f472c134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): get_auth_request con 0x7f472c135dc0 auth_method 0 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 10 monclient(hunting): get_auth_request con 0x7f472c136e40 auth_method 0 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient(hunting): handle_auth_done global_id 84851 payload 824 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c135dc0 msgr2=0x7f472c09f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f472c135dc0 0x7f472c09f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f472c136e40 msgr2=0x7f472c134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f472c136e40 0x7f472c134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 monclient: found mon.noname-e 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724009400 con 0x7f472c137220 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f472c136e40 0x7f472c134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 msgr2=0x7f472c1cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 msgr2=0x7f472c1cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 secure :-1 s=AUTH_CONNECTING_SIGN pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f472400e930 tx=0x7f472400edf0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:18:49.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.553+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:18:49.703 INFO:tasks.rados.rados.0.smithi162.stdout:3823: done (0 left) 2024-09-16T15:18:49.704 INFO:tasks.rados.rados.0.smithi162.stdout:3824: read oid 15 snap -1 2024-09-16T15:18:49.704 INFO:tasks.rados.rados.0.smithi162.stdout:3824: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:49.704 INFO:tasks.rados.rados.0.smithi162.stdout:3825: rollback oid 46 current snap is 401 2024-09-16T15:18:49.704 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 46 to 395 2024-09-16T15:18:49.704 INFO:tasks.rados.rados.0.smithi162.stdout:3826: snap_remove snap 388 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient: get_auth_request con 0x7f472c137220 auth_method 0 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 0 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 msgr2=0x7f472c1cb430 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f472c137220 0x7f472c1cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient: _add_conns ranks=[7,2,0,5,6,4] 2024-09-16T15:18:49.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47180035d0 0x7f4718005a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): picked mon.noname-i con 0x7f47180035d0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 0x7f4718008a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): picked mon.noname-b con 0x7f47180065c0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4718009630 0x7f471800baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): picked mon.noname-a con 0x7f4718009630 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471800c6e0 con 0x7f4718009630 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471800c820 con 0x7f47180065c0 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f471800c9e0 con 0x7f47180035d0 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f4713fff640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 0x7f4718008a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): get_auth_request con 0x7f47180065c0 auth_method 0 2024-09-16T15:18:49.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4718009630 0x7f471800baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): get_auth_request con 0x7f4718009630 auth_method 0 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47180035d0 0x7f4718005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:49.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): get_auth_request con 0x7f47180035d0 auth_method 0 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient(hunting): handle_auth_done global_id 84851 payload 293 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47180035d0 msgr2=0x7f4718005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f47180035d0 0x7f4718005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4718009630 msgr2=0x7f471800baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4718009630 0x7f471800baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 monclient: found mon.noname-b 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472affd640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471800cc60 con 0x7f47180065c0 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4718009630 0x7f471800baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.754+0000 7f472bfff640 1 -- 172.21.15.134:0/27420487 reap_dead start 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f472affd640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 msgr2=0x7f4718008a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f472affd640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 0x7f4718008a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4720031000 tx=0x7f4724009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f47200044b0 con 0x7f47180065c0 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: dump: 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.764 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720030920 con 0x7f47180065c0 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4720003610 con 0x7f47180065c0 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4713fff640 10 monclient: dump: 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.766 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4731d3e640 5 monclient: authenticate success, global_id 84851 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4731d3e640 10 monclient: _renew_subs 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4731d3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47200094d0 con 0x7f47180065c0 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.756+0000 7f4729ffb640 4 set_mon_vals no callback set 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 10 monclient: _renew_subs 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:18:49.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c122050 con 0x7f47180065c0 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4720039050 con 0x7f47180065c0 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c071b20 con 0x7f472c122570 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 0x7f471803f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f472b7fe640 10 monclient: get_auth_request con 0x7f472c122570 auth_method 0 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f472b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f472b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.757+0000 7f472b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f4720069360 con 0x7f47180065c0 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472b7fe640 10 monclient(hunting): handle_auth_done global_id 114365 payload 824 2024-09-16T15:18:49.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472b7fe640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f472c061ae0 con 0x7f472c122570 2024-09-16T15:18:49.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 0x7f471803f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.758+0000 7f472bfff640 10 monclient: get_auth_request con 0x7f471803cfe0 auth_method 0 2024-09-16T15:18:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f472b7fe640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 msgr2=0x7f472c09f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f471c017af0 con 0x7f472c122570 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 10 monclient: dump: 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.772 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f472bfff640 1 --2- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 0x7f471803f4a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f472c09fdd0 tx=0x7f472400edf0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f471c017c50 con 0x7f472c122570 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 msgr2=0x7f472c09f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c007700 comp rx=0 tx=0).stop 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 msgr2=0x7f472c09f4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.760+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c007700 comp rx=0 tx=0).stop 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.766+0000 7f4731d3e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:49.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.766+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.766+0000 7f4731d3e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.766+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c07a680 con 0x7f472c1cb430 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient: get_auth_request con 0x7f472c1cb430 auth_method 0 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 10 monclient(hunting): handle_auth_done global_id 114374 payload 824 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.767+0000 7f472b7fe640 1 -- 172.21.15.134:0/27420487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f472c09b490 con 0x7f472c1cb430 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f472b7fe640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 msgr2=0x7f472c1cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f472b7fe640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f471c022d80 con 0x7f472c1cb430 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 10 monclient: dump: 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.775 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f471c020420 con 0x7f472c1cb430 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 msgr2=0x7f472c1cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 msgr2=0x7f472c1cb810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.769+0000 7f4713fff640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f471c004630 tx=0x7f471c004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:49.776 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:49.777 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.778 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.779 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:49.780 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.781 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:49.785 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.787 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.788 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.789 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:49.790 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.792 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.793 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:49.794 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.796 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:18:49.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:49.798 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 msgr2=0x7f471803f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 0x7f471803f4a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f472c09fdd0 tx=0x7f472400edf0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 10 monclient: shutdown 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 msgr2=0x7f4718008a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 0x7f4718008a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4720031000 tx=0x7f4724009190 comp rx=0 tx=0).stop 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 shutdown_connections 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> v2:172.21.15.162:6800/980547263 conn(0x7f471803cfe0 0x7f471803f4a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180065c0 0x7f4718008a50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c122570 0x7f472c09f4c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 --2- 172.21.15.134:0/27420487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f472c1cb430 0x7f472c1cb810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 >> 172.21.15.134:0/27420487 conn(0x7f472c0a6110 msgr2=0x7f472c0a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 shutdown_connections 2024-09-16T15:18:49.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.770+0000 7f4731d3e640 1 -- 172.21.15.134:0/27420487 wait complete. 2024-09-16T15:18:49.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:49.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:18:49.801 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:18:49.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.897+0000 7f51a43ce640 10 monclient: get_monmap_and_config 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.897+0000 7f51a43ce640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.897+0000 7f51a43ce640 10 monclient: monmap: 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:49.898516+0000 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:49.898516+0000 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:49.899 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:49.900 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 Processor -- start 2024-09-16T15:18:49.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 -- start start 2024-09-16T15:18:49.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient: init 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient: _add_conns ranks=[2,7,3,8,0,6,1,5,4,10,9] 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-b con 0x7f519c136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f519c137220 0x7f519c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-i con 0x7f519c137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 0x7f519c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-g con 0x7f519c135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c09abc0 con 0x7f519c136e40 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c09ba50 con 0x7f519c135dc0 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0a53c0 con 0x7f519c137220 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a43ce640 10 monclient(hunting): authenticate will time out at 1866.487795s 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47212/0 (socket says 172.21.15.134:47212) 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 1 -- 172.21.15.134:0/3443803581 learned_addr learned my addr 172.21.15.134:0/3443803581 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 10 monclient(hunting): get_auth_request con 0x7f519c136e40 auth_method 0 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.898+0000 7f51a216a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a1969640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f519c137220 0x7f519c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a296b640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 0x7f519c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a1969640 10 monclient(hunting): get_auth_request con 0x7f519c137220 auth_method 0 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a1969640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a1969640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a1969640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient(hunting): handle_auth_done global_id 114383 payload 274 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 -- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f519c137220 msgr2=0x7f519c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f519c137220 0x7f519c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 -- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 msgr2=0x7f519c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 0x7f519c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 monclient: found mon.noname-b 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a216a640 1 -- 172.21.15.134:0/3443803581 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c073250 con 0x7f519c136e40 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a296b640 10 monclient: get_auth_request con 0x7f519c135dc0 auth_method 0 2024-09-16T15:18:49.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.899+0000 7f51a296b640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 0x7f519c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:18:49.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.901+0000 7f51a216a640 1 -- 172.21.15.134:0/3443803581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 msgr2=0x7f519c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.901+0000 7f51a216a640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51980098a0 tx=0x7f519802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debb4c863a506756 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 1 -- 172.21.15.134:0/3443803581 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f519802ade0 con 0x7f519c136e40 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: dump: 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.913 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 1 -- 172.21.15.134:0/3443803581 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51980307b0 con 0x7f519c136e40 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 5 monclient: authenticate success, global_id 114383 2024-09-16T15:18:49.914 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 1 -- 172.21.15.134:0/3443803581 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5198030d90 con 0x7f519c136e40 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 4 set_mon_vals no callback set 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a1168640 10 monclient: dump: 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.915 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 10 monclient: shutdown 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 -- 172.21.15.134:0/3443803581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 msgr2=0x7f519c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51980098a0 tx=0x7f519802a080 comp rx=0 tx=0).stop 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 -- 172.21.15.134:0/3443803581 shutdown_connections 2024-09-16T15:18:49.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c135dc0 0x7f519c136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f519c137220 0x7f519c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3443803581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f519c136e40 0x7f519c09c150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 -- 172.21.15.134:0/3443803581 >> 172.21.15.134:0/3443803581 conn(0x7f519c0a6110 msgr2=0x7f519c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 -- 172.21.15.134:0/3443803581 shutdown_connections 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.902+0000 7f51a43ce640 1 -- 172.21.15.134:0/3443803581 wait complete. 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.903+0000 7f51a43ce640 10 monclient: build_initial_monmap 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.903+0000 7f51a43ce640 10 monclient: monmap: 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:49.917 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.903+0000 7f51a43ce640 1 Processor -- start 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.903+0000 7f51a43ce640 1 -- start start 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.903+0000 7f51a43ce640 10 monclient: init 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:49.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient: _add_conns ranks=[4,6,3,0,5,2,7,8,1] 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 0x7f519c134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-c con 0x7f519c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 0x7f519c1352e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-d con 0x7f519c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c137220 0x7f519c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): picked mon.noname-g con 0x7f519c137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1cc1e0 con 0x7f519c137220 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f519c122230 con 0x7f519c135dc0 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f519c122480 con 0x7f519c136e40 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a43ce640 10 monclient(hunting): authenticate will time out at 1866.493230s 2024-09-16T15:18:49.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 0x7f519c134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 0x7f519c1352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 0x7f519c134da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38668/0 (socket says 172.21.15.134:38668) 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 1 -- 172.21.15.134:0/3394665304 learned_addr learned my addr 172.21.15.134:0/3394665304 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): get_auth_request con 0x7f519c135dc0 auth_method 0 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): get_auth_request con 0x7f519c136e40 auth_method 0 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a296b640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c137220 0x7f519c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient(hunting): handle_auth_done global_id 94524 payload 824 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 msgr2=0x7f519c134da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 0x7f519c134da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c137220 msgr2=0x7f519c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c137220 0x7f519c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient: _finish_hunting 0 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 monclient: found mon.noname-d 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a1969640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980094d0 con 0x7f519c136e40 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a296b640 10 monclient: get_auth_request con 0x7f519c137220 auth_method 0 2024-09-16T15:18:49.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a296b640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f519c137220 0x7f519c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.904+0000 7f51a216a640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c135dc0 0x7f519c134da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f51a1969640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 msgr2=0x7f519c1352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f51a1969640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 0x7f519c1352e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d580 tx=0x7f518c00da40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f518c012c70 con 0x7f519c136e40 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f5192ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f5192ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.906+0000 7f5192ffd640 10 monclient: dump: 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.922 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 5 monclient: authenticate success, global_id 94524 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c012e10 con 0x7f519c136e40 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 monclient: _renew_subs 2024-09-16T15:18:49.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f518c0148a0 con 0x7f519c136e40 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c136970 con 0x7f519c136e40 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a0967640 4 set_mon_vals no callback set 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 10 monclient: dump: 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.924 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 monclient: _renew_subs 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c136420 con 0x7f519c136e40 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 0x7f519c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f518c014430 con 0x7f519c136e40 2024-09-16T15:18:49.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f519c071b20 con 0x7f519c122610 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.907+0000 7f51a216a640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 0x7f519c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient: get_auth_request con 0x7f519c122610 auth_method 0 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f5192ffd640 1 --2- 172.21.15.134:0/3394665304 >> v2:172.21.15.162:6800/980547263 conn(0x7f51800388b0 0x7f518003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f518c04a5c0 con 0x7f519c136e40 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 10 monclient(hunting): handle_auth_done global_id 124132 payload 824 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a216a640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f519c061ae0 con 0x7f519c122610 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a296b640 1 --2- 172.21.15.134:0/3394665304 >> v2:172.21.15.162:6800/980547263 conn(0x7f51800388b0 0x7f518003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.908+0000 7f51a296b640 10 monclient: get_auth_request con 0x7f51800388b0 auth_method 0 2024-09-16T15:18:49.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f51a216a640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 msgr2=0x7f519c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f51a216a640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 0x7f519c02b260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51980097c0 tx=0x7f51980046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5198030e90 con 0x7f519c122610 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 10 monclient: dump: 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.927 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f51a296b640 1 --2- 172.21.15.134:0/3394665304 >> v2:172.21.15.162:6800/980547263 conn(0x7f51800388b0 0x7f518003ad70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f519c0a71d0 tx=0x7f51940074b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f519802ade0 con 0x7f519c122610 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 msgr2=0x7f519c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 0x7f519c02b260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51980097c0 tx=0x7f51980046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 msgr2=0x7f519c02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.910+0000 7f5192ffd640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c122610 0x7f519c02b260 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51980097c0 tx=0x7f51980046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.916+0000 7f51a43ce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:49.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.916+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 0x7f519c02b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:49.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.916+0000 7f51a43ce640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:49.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.916+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f519c122050 con 0x7f519c0a2ba0 2024-09-16T15:18:49.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 0x7f519c02b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient: get_auth_request con 0x7f519c0a2ba0 auth_method 0 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 10 monclient(hunting): handle_auth_done global_id 124141 payload 824 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.917+0000 7f51a216a640 1 -- 172.21.15.134:0/3394665304 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f519c07a680 con 0x7f519c0a2ba0 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f51a216a640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 msgr2=0x7f519c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f51a216a640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 0x7f519c02b7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198008030 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5198032bb0 con 0x7f519c0a2ba0 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:49.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 10 monclient: dump: 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:49.932 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f5198032d10 con 0x7f519c0a2ba0 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 msgr2=0x7f519c02b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 0x7f519c02b7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198008030 comp rx=0 tx=0).stop 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 msgr2=0x7f519c02b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.919+0000 7f5192ffd640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f519c0a2ba0 0x7f519c02b7a0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5198000c00 tx=0x7f5198008030 comp rx=0 tx=0).stop 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:49.933 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:49.934 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.935 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.936 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:49.937 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:49.938 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.941 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.942 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.943 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.944 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.945 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.946 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:49.947 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.948 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:49.949 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.950 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.951 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.953 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:49.954 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:49.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 >> v2:172.21.15.162:6800/980547263 conn(0x7f51800388b0 msgr2=0x7f518003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3394665304 >> v2:172.21.15.162:6800/980547263 conn(0x7f51800388b0 0x7f518003ad70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f519c0a71d0 tx=0x7f51940074b0 comp rx=0 tx=0).stop 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 10 monclient: shutdown 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 msgr2=0x7f519c1352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 --2- 172.21.15.134:0/3394665304 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f519c136e40 0x7f519c1352e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d580 tx=0x7f518c00da40 comp rx=0 tx=0).stop 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a296b640 1 -- 172.21.15.134:0/3394665304 reap_dead start 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a296b640 1 -- 172.21.15.134:0/3394665304 reap_dead start 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 shutdown_connections 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 >> 172.21.15.134:0/3394665304 conn(0x7f519c0a6110 msgr2=0x7f519c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 shutdown_connections 2024-09-16T15:18:49.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:49.920+0000 7f51a43ce640 1 -- 172.21.15.134:0/3394665304 wait complete. 2024-09-16T15:18:49.956 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:49.957 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:18:49.957 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.053+0000 7f9a903f4640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.053+0000 7f9a903f4640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.053+0000 7f9a903f4640 10 monclient: monmap: 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.054725+0000 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.054725+0000 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.055 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.056 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 1 Processor -- start 2024-09-16T15:18:50.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 1 -- start start 2024-09-16T15:18:50.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 10 monclient: init 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 10 monclient: _add_conns ranks=[2,1,4,8,10,0,7,3,6,5,9] 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 0x7f9a8809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.054+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-b con 0x7f9a88136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f9a88137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-c con 0x7f9a8813c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a88073250 con 0x7f9a88137220 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880a56e0 con 0x7f9a88136e40 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8809abc0 con 0x7f9a8813c370 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a903f4640 10 monclient(hunting): authenticate will time out at 1866.644047s 2024-09-16T15:18:50.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38692/0 (socket says 172.21.15.134:38692) 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 -- 172.21.15.134:0/1632731055 learned_addr learned my addr 172.21.15.134:0/1632731055 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): get_auth_request con 0x7f9a8813c370 auth_method 0 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 0x7f9a8809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): get_auth_request con 0x7f9a88136e40 auth_method 0 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient(hunting): handle_auth_done global_id 124150 payload 274 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 -- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 msgr2=0x7f9a8809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 0x7f9a8809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 -- 172.21.15.134:0/1632731055 >> v1:172.21.15.162:6789/0 conn(0x7f9a88137220 legacy=0x7f9a88137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 monclient: found mon.noname-c 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e991640 1 -- 172.21.15.134:0/1632731055 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8809ba50 con 0x7f9a8813c370 2024-09-16T15:18:50.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.055+0000 7f9a8e190640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 0x7f9a8809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8e991640 1 -- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 msgr2=0x7f9a8813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8e991640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009e80 tx=0x7f9a8002a580 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d83466794f9939e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 1 -- 172.21.15.134:0/1632731055 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a80007a70 con 0x7f9a8813c370 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:50.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: dump: 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.068 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 1 -- 172.21.15.134:0/1632731055 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8002d050 con 0x7f9a8813c370 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 5 monclient: authenticate success, global_id 124150 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 1 -- 172.21.15.134:0/1632731055 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a80007bf0 con 0x7f9a8813c370 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 4 set_mon_vals no callback set 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a8d18e640 10 monclient: dump: 2024-09-16T15:18:50.069 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.070 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 10 monclient: shutdown 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 1 -- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 msgr2=0x7f9a8813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009e80 tx=0x7f9a8002a580 comp rx=0 tx=0).stop 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 1 -- 172.21.15.134:0/1632731055 shutdown_connections 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9a8813c370 0x7f9a8813e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.058+0000 7f9a903f4640 1 --2- 172.21.15.134:0/1632731055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88136e40 0x7f9a8809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 1 -- 172.21.15.134:0/1632731055 >> 172.21.15.134:0/1632731055 conn(0x7f9a880a6110 msgr2=0x7f9a880a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 1 -- 172.21.15.134:0/1632731055 shutdown_connections 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 1 -- 172.21.15.134:0/1632731055 wait complete. 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 10 monclient: monmap: 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.071 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 1 Processor -- start 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 1 -- start start 2024-09-16T15:18:50.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.059+0000 7f9a903f4640 10 monclient: init 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient: _add_conns ranks=[6,2,3,7,1,8,0,4,5] 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 0x7f9a88135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-d con 0x7f9a88136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 0x7f9a881363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-b con 0x7f9a88137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a8813c370 0x7f9a8811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): picked mon.noname-g con 0x7f9a8813c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880a7b10 con 0x7f9a88137220 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8813b530 con 0x7f9a8813c370 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880a75c0 con 0x7f9a88136e40 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a903f4640 10 monclient(hunting): authenticate will time out at 1866.649360s 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 0x7f9a881363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 0x7f9a88135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 0x7f9a88135eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40380/0 (socket says 172.21.15.134:40380) 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 0x7f9a881363f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47230/0 (socket says 172.21.15.134:47230) 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 -- 172.21.15.134:0/772881253 learned_addr learned my addr 172.21.15.134:0/772881253 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): get_auth_request con 0x7f9a88136e40 auth_method 0 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): get_auth_request con 0x7f9a88137220 auth_method 0 2024-09-16T15:18:50.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a8813c370 0x7f9a8811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 10 monclient(hunting): get_auth_request con 0x7f9a8813c370 auth_method 0 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient(hunting): handle_auth_done global_id 94533 payload 824 2024-09-16T15:18:50.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a8813c370 msgr2=0x7f9a8811fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a8813c370 0x7f9a8811fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 msgr2=0x7f9a881363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 0x7f9a881363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 monclient: found mon.noname-d 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e190640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009400 con 0x7f9a88136e40 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8e991640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a8813c370 0x7f9a8811fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.060+0000 7f9a8d98f640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a88137220 0x7f9a881363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.062+0000 7f9a8e190640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 msgr2=0x7f9a88135eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.062+0000 7f9a8e190640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 0x7f9a88135eb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400ebe0 tx=0x7f9a8400c520 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a84009760 con 0x7f9a88136e40 2024-09-16T15:18:50.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: dump: 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.077 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400ce40 con 0x7f9a88136e40 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 5 monclient: authenticate success, global_id 94533 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 10 monclient: _renew_subs 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a80009b30 con 0x7f9a88136e40 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a8400eee0 con 0x7f9a88136e40 2024-09-16T15:18:50.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a7effd640 10 monclient: dump: 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.079 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a8c98d640 4 set_mon_vals no callback set 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 10 monclient: _renew_subs 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.063+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a88122880 con 0x7f9a88136e40 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a903f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9a84013470 con 0x7f9a88136e40 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a903f4640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 0x7f9a8802b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88071b20 con 0x7f9a8802b260 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 0x7f9a8802b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient: get_auth_request con 0x7f9a8802b260 auth_method 0 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a7effd640 1 --2- 172.21.15.134:0/772881253 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a6c0388b0 0x7f9a6c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f9a8404a520 con 0x7f9a88136e40 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_done global_id 94542 payload 824 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8d98f640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a88121fe0 con 0x7f9a8802b260 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8e991640 1 --2- 172.21.15.134:0/772881253 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a6c0388b0 0x7f9a6c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.064+0000 7f9a8e991640 10 monclient: get_auth_request con 0x7f9a6c0388b0 auth_method 0 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a8d98f640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 msgr2=0x7f9a8802b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a8d98f640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 0x7f9a8802b640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780044c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7800dd90 con 0x7f9a8802b260 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a7effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.066+0000 7f9a7effd640 10 monclient: dump: 2024-09-16T15:18:50.083 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.084 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9a78014470 con 0x7f9a8802b260 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 msgr2=0x7f9a8802b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 0x7f9a8802b640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780044c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 msgr2=0x7f9a8802b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a7effd640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802b260 0x7f9a8802b640 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780044c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.067+0000 7f9a8e991640 1 --2- 172.21.15.134:0/772881253 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a6c0388b0 0x7f9a6c03ad70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009e80 tx=0x7f9a80009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a903f4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a903f4640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 0x7f9a8802bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a903f4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88061ae0 con 0x7f9a8802bb80 2024-09-16T15:18:50.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 0x7f9a8802bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient: get_auth_request con 0x7f9a8802bb80 auth_method 0 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 10 monclient(hunting): handle_auth_done global_id 94551 payload 824 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.073+0000 7f9a8d98f640 1 -- 172.21.15.134:0/772881253 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a8807a680 con 0x7f9a8802bb80 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a8d98f640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 msgr2=0x7f9a8802bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a8d98f640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 0x7f9a8802bf90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780036f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7801f2e0 con 0x7f9a8802bb80 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a7effd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a7effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:50.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.075+0000 7f9a7effd640 10 monclient: dump: 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.087 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f9a78014b40 con 0x7f9a8802bb80 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 msgr2=0x7f9a8802bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 0x7f9a8802bf90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780036f0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 msgr2=0x7f9a8802bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a7effd640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a8802bb80 0x7f9a8802bf90 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800fb10 tx=0x7f9a780036f0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.088 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.089 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.090 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.091 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.092 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.093 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.094 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.095 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.096 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.097 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.098 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.099 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.100 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.101 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.102 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.103 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.104 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.105 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.106 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.107 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:50.108 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.109 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.110 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a6c0388b0 msgr2=0x7f9a6c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 --2- 172.21.15.134:0/772881253 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a6c0388b0 0x7f9a6c03ad70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009e80 tx=0x7f9a80009000 comp rx=0 tx=0).stop 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 10 monclient: shutdown 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 msgr2=0x7f9a88135eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 --2- 172.21.15.134:0/772881253 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a88136e40 0x7f9a88135eb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400ebe0 tx=0x7f9a8400c520 comp rx=0 tx=0).stop 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a8e991640 1 -- 172.21.15.134:0/772881253 reap_dead start 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a8e991640 1 -- 172.21.15.134:0/772881253 reap_dead start 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 shutdown_connections 2024-09-16T15:18:50.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 >> 172.21.15.134:0/772881253 conn(0x7f9a880a6110 msgr2=0x7f9a8813a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 shutdown_connections 2024-09-16T15:18:50.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.076+0000 7f9a903f4640 1 -- 172.21.15.134:0/772881253 wait complete. 2024-09-16T15:18:50.112 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.112 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:18:50.112 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.209+0000 7faade90f640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.209+0000 7faade90f640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.209+0000 7faade90f640 10 monclient: monmap: 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.210212+0000 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.210212+0000 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.215 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.209+0000 7faade90f640 1 Processor -- start 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 -- start start 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient: init 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient: _add_conns ranks=[10,0,9,4,7,8,5,3,6,1,2] 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faad8136e40 0x7faad809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): picked mon.noname-f con 0x7faad8136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faad8137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faad813c370 0x7faad813e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): picked mon.noname-a con 0x7faad813c370 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad8073250 con 0x7faad8137220 2024-09-16T15:18:50.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad80a56e0 con 0x7faad813c370 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad809abc0 con 0x7faad8136e40 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faade90f640 10 monclient(hunting): authenticate will time out at 1866.799530s 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faad813c370 0x7faad813e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faad813c370 0x7faad813e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33474/0 (socket says 172.21.15.134:33474) 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 -- 172.21.15.134:0/2254110001 learned_addr learned my addr 172.21.15.134:0/2254110001 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): get_auth_request con 0x7faad813c370 auth_method 0 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad77fe640 1 --1- 172.21.15.134:0/2254110001 >> v1:172.21.15.134:6789/0 conn(0x7faad8137220 0x7faad8137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:46940/0 (socket says 172.21.15.134:46940) 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.217 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad7fff640 1 --2- 172.21.15.134:0/2254110001 >> v2:172.21.15.162:3300/0 conn(0x7faad8136e40 0x7faad809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad7fff640 10 monclient(hunting): get_auth_request con 0x7faad8136e40 auth_method 0 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient(hunting): handle_auth_done global_id 124137 payload 274 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 -- 172.21.15.134:0/2254110001 >> v2:172.21.15.162:3300/0 conn(0x7faad8136e40 msgr2=0x7faad809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 --2- 172.21.15.134:0/2254110001 >> v2:172.21.15.162:3300/0 conn(0x7faad8136e40 0x7faad809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 -- 172.21.15.134:0/2254110001 >> v1:172.21.15.134:6789/0 conn(0x7faad8137220 legacy=0x7faad8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 monclient: found mon.noname-a 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:50.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.210+0000 7faadceac640 1 -- 172.21.15.134:0/2254110001 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad809ba50 con 0x7faad813c370 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.213+0000 7faadceac640 1 -- 172.21.15.134:0/2254110001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad813c370 msgr2=0x7faad813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.213+0000 7faadceac640 1 --2- 172.21.15.134:0/2254110001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad813c370 0x7faad813e760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faad8137f00 tx=0x7faad002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc7d32be8d54c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 1 -- 172.21.15.134:0/2254110001 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad0007a70 con 0x7faad813c370 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: dump: 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.226 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 1 -- 172.21.15.134:0/2254110001 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad002d050 con 0x7faad813c370 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 5 monclient: authenticate success, global_id 124137 2024-09-16T15:18:50.227 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 1 -- 172.21.15.134:0/2254110001 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad0007bf0 con 0x7faad813c370 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 4 set_mon_vals no callback set 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faad6ffd640 10 monclient: dump: 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.228 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 10 monclient: shutdown 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 -- 172.21.15.134:0/2254110001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad813c370 msgr2=0x7faad813e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.229 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 --2- 172.21.15.134:0/2254110001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad813c370 0x7faad813e760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faad8137f00 tx=0x7faad002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 -- 172.21.15.134:0/2254110001 shutdown_connections 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 --2- 172.21.15.134:0/2254110001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad813c370 0x7faad813e760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 --2- 172.21.15.134:0/2254110001 >> v2:172.21.15.162:3300/0 conn(0x7faad8136e40 0x7faad809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 -- 172.21.15.134:0/2254110001 >> 172.21.15.134:0/2254110001 conn(0x7faad80a6110 msgr2=0x7faad80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 -- 172.21.15.134:0/2254110001 shutdown_connections 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.214+0000 7faade90f640 1 -- 172.21.15.134:0/2254110001 wait complete. 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.215+0000 7faade90f640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.215+0000 7faade90f640 10 monclient: monmap: 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.230 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.215+0000 7faade90f640 1 Processor -- start 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.215+0000 7faade90f640 1 -- start start 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.215+0000 7faade90f640 10 monclient: init 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.231 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient: _add_conns ranks=[2,1,8,3,7,0,6,4,5] 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 0x7faad811fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): picked mon.noname-b con 0x7faad8136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faad8137220 0x7faad81201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): picked mon.noname-f con 0x7faad8137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 0x7faad81206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): picked mon.noname-e con 0x7faad813c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8122420 con 0x7faad8137220 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad8122670 con 0x7faad8136e40 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faad80a77f0 con 0x7faad813c370 2024-09-16T15:18:50.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faade90f640 10 monclient(hunting): authenticate will time out at 1866.805273s 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 0x7faad811fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 0x7faad811fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47244/0 (socket says 172.21.15.134:47244) 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 1 -- 172.21.15.134:0/3711366340 learned_addr learned my addr 172.21.15.134:0/3711366340 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): get_auth_request con 0x7faad8136e40 auth_method 0 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 0x7faad81206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): get_auth_request con 0x7faad813c370 auth_method 0 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faad8137220 0x7faad81201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): get_auth_request con 0x7faad8137220 auth_method 0 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient(hunting): handle_auth_done global_id 84860 payload 824 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 msgr2=0x7faad811fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 0x7faad811fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faad8137220 msgr2=0x7faad81201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faad8137220 0x7faad81201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 monclient: found mon.noname-e 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0009b30 con 0x7faad813c370 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.216+0000 7faad7fff640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faad8136e40 0x7faad811fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.218+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 msgr2=0x7faad81206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.218+0000 7faadceac640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 0x7faad81206e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad0030000 tx=0x7faad0004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad0008ce0 con 0x7faad813c370 2024-09-16T15:18:50.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: dump: 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.236 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad0008e80 con 0x7faad813c370 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 5 monclient: authenticate success, global_id 84860 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 10 monclient: _renew_subs 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad80a7410 con 0x7faad813c370 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faad003d450 con 0x7faad813c370 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad4ff9640 10 monclient: dump: 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.238 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 10 monclient: _renew_subs 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad8121dd0 con 0x7faad813c370 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.219+0000 7faad67fc640 4 set_mon_vals no callback set 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faade90f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faade90f640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 0x7faad8135b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faad8122800 con 0x7faad8135750 2024-09-16T15:18:50.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7faad0004040 con 0x7faad813c370 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 0x7faad8135b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient: get_auth_request con 0x7faad8135750 auth_method 0 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad4ff9640 1 --2- 172.21.15.134:0/3711366340 >> v2:172.21.15.162:6800/980547263 conn(0x7faab8038880 0x7faab803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7faad006abe0 con 0x7faad813c370 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 10 monclient(hunting): handle_auth_done global_id 84869 payload 824 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad7fff640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faad8061ae0 con 0x7faad8135750 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad77fe640 1 --2- 172.21.15.134:0/3711366340 >> v2:172.21.15.162:6800/980547263 conn(0x7faab8038880 0x7faab803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.220+0000 7faad77fe640 10 monclient: get_auth_request con 0x7faab8038880 auth_method 0 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad7fff640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 msgr2=0x7faad8135b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad7fff640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 0x7faad8135b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faac8010c80 tx=0x7faac800e5a0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faac800eed0 con 0x7faad8135750 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.222+0000 7faad4ff9640 10 monclient: dump: 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.241 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad77fe640 1 --2- 172.21.15.134:0/3711366340 >> v2:172.21.15.162:6800/980547263 conn(0x7faab8038880 0x7faab803ad40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac0007e60 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faac8014640 con 0x7faad8135750 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 msgr2=0x7faad8135b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.242 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 0x7faad8135b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faac8010c80 tx=0x7faac800e5a0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 msgr2=0x7faad8135b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.223+0000 7faad4ff9640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8135750 0x7faad8135b30 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faac8010c80 tx=0x7faac800e5a0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faade90f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faade90f640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 0x7faad809d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faade90f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7faad8071b20 con 0x7faad8136070 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 0x7faad809d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient: get_auth_request con 0x7faad8136070 auth_method 0 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 10 monclient(hunting): handle_auth_done global_id 84878 payload 824 2024-09-16T15:18:50.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.229+0000 7faad7fff640 1 -- 172.21.15.134:0/3711366340 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faad807a680 con 0x7faad8136070 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad7fff640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 msgr2=0x7faad809d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad7fff640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 0x7faad809d4f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faac801a000 tx=0x7faac80046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7faac802b070 con 0x7faad8136070 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad4ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad4ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.231+0000 7faad4ff9640 10 monclient: dump: 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.246 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7faac8023040 con 0x7faad8136070 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 msgr2=0x7faad809d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 0x7faad809d4f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faac801a000 tx=0x7faac80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 msgr2=0x7faad809d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faad4ff9640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad8136070 0x7faad809d4f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.248 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.249 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.250 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.251 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.252 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.253 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.254 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.255 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.256 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.257 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.258 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.259 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.260 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.261 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.262 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.263 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.264 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.265 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.266 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.267 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.268 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.269 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 >> v2:172.21.15.162:6800/980547263 conn(0x7faab8038880 msgr2=0x7faab803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 --2- 172.21.15.134:0/3711366340 >> v2:172.21.15.162:6800/980547263 conn(0x7faab8038880 0x7faab803ad40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac0007e60 comp rx=0 tx=0).stop 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 10 monclient: shutdown 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 msgr2=0x7faad81206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 --2- 172.21.15.134:0/3711366340 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7faad813c370 0x7faad81206e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faad0030000 tx=0x7faad0004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 reap_dead start 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faadceac640 1 -- 172.21.15.134:0/3711366340 reap_dead start 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 shutdown_connections 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 >> 172.21.15.134:0/3711366340 conn(0x7faad80a6110 msgr2=0x7faad813b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 shutdown_connections 2024-09-16T15:18:50.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.232+0000 7faade90f640 1 -- 172.21.15.134:0/3711366340 wait complete. 2024-09-16T15:18:50.272 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.272 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:18:50.272 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.367+0000 7fe3c6776640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.367+0000 7fe3c6776640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.367+0000 7fe3c6776640 10 monclient: monmap: 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.368602+0000 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.368602+0000 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.369 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 Processor -- start 2024-09-16T15:18:50.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 -- start start 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient: init 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient: _add_conns ranks=[3,6,9,4,10,8,5,7,2,1,0] 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3c0136e40 0x7fe3c009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-g con 0x7fe3c0136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-d con 0x7fe3c0137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe3c0135dc0 0x7fe3c0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-a con 0x7fe3c0135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c009abc0 con 0x7fe3c0135dc0 2024-09-16T15:18:50.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c009ba50 con 0x7fe3c0136e40 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00a53c0 con 0x7fe3c0137220 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.368+0000 7fe3c6776640 10 monclient(hunting): authenticate will time out at 1866.958000s 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe3c0135dc0 0x7fe3c0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe3c0135dc0 0x7fe3c0136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33476/0 (socket says 172.21.15.134:33476) 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40398/0 (socket says 172.21.15.134:40398) 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 -- 172.21.15.134:0/2834931402 learned_addr learned my addr 172.21.15.134:0/2834931402 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): get_auth_request con 0x7fe3c0135dc0 auth_method 0 2024-09-16T15:18:50.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): get_auth_request con 0x7fe3c0137220 auth_method 0 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bffff640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3c0136e40 0x7fe3c009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bffff640 10 monclient(hunting): get_auth_request con 0x7fe3c0136e40 auth_method 0 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_done global_id 124146 payload 274 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 -- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 msgr2=0x7fe3c0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 -- 172.21.15.134:0/2834931402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3c0136e40 msgr2=0x7fe3c009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3c0136e40 0x7fe3c009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 monclient: found mon.noname-a 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:50.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3c4d13640 1 -- 172.21.15.134:0/2834931402 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0073250 con 0x7fe3c0135dc0 2024-09-16T15:18:50.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.369+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c4d13640 1 -- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c0135dc0 msgr2=0x7fe3c0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c0135dc0 0x7fe3c0136220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e417ba3a4da36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 1 -- 172.21.15.134:0/2834931402 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b8007a70 con 0x7fe3c0135dc0 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: dump: 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.384 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 1 -- 172.21.15.134:0/2834931402 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b802d050 con 0x7fe3c0135dc0 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 1 -- 172.21.15.134:0/2834931402 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b8007bf0 con 0x7fe3c0135dc0 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 5 monclient: authenticate success, global_id 124146 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 4 set_mon_vals no callback set 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3beffd640 10 monclient: dump: 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.386 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 10 monclient: shutdown 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 -- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c0135dc0 msgr2=0x7fe3c0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c0135dc0 0x7fe3c0136220 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b802a560 comp rx=0 tx=0).stop 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 -- 172.21.15.134:0/2834931402 shutdown_connections 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3c0135dc0 0x7fe3c0136220 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0137220 0x7fe3c0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 --2- 172.21.15.134:0/2834931402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3c0136e40 0x7fe3c009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.372+0000 7fe3c6776640 1 -- 172.21.15.134:0/2834931402 >> 172.21.15.134:0/2834931402 conn(0x7fe3c00a6110 msgr2=0x7fe3c00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.373+0000 7fe3c6776640 1 -- 172.21.15.134:0/2834931402 shutdown_connections 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.373+0000 7fe3c6776640 1 -- 172.21.15.134:0/2834931402 wait complete. 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.373+0000 7fe3c6776640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.373+0000 7fe3c6776640 10 monclient: monmap: 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.388 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.373+0000 7fe3c6776640 1 Processor -- start 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 -- start start 2024-09-16T15:18:50.389 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient: init 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient: _add_conns ranks=[6,7,8,5,4,1,2,0,3] 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 0x7fe3c0134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-d con 0x7fe3c0135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe3c0136e40 0x7fe3c0134fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-i con 0x7fe3c0136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe3c0137220 0x7fe3c0121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): picked mon.noname-e con 0x7fe3c0137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00a8930 con 0x7fe3c0135dc0 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c01367f0 con 0x7fe3c0136e40 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.390 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c01362a0 con 0x7fe3c0137220 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c6776640 10 monclient(hunting): authenticate will time out at 1866.963637s 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 0x7fe3c0134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 0x7fe3c0134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40406/0 (socket says 172.21.15.134:40406) 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 1 -- 172.21.15.134:0/214628786 learned_addr learned my addr 172.21.15.134:0/214628786 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): get_auth_request con 0x7fe3c0135dc0 auth_method 0 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe3c0137220 0x7fe3c0121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): get_auth_request con 0x7fe3c0137220 auth_method 0 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.391 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3c4d13640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe3c0136e40 0x7fe3c0134fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.374+0000 7fe3bffff640 10 monclient(hunting): handle_auth_done global_id 94569 payload 824 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe3c0137220 msgr2=0x7fe3c0121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe3c0137220 0x7fe3c0121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe3c0136e40 msgr2=0x7fe3c0134fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe3c0136e40 0x7fe3c0134fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 monclient: found mon.noname-d 2024-09-16T15:18:50.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bffff640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009400 con 0x7fe3c0135dc0 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bf7fe640 10 monclient: get_auth_request con 0x7fe3c0136e40 auth_method 0 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.375+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe3c0136e40 0x7fe3c0134fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bffff640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 msgr2=0x7fe3c0134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bffff640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 0x7fe3c0134aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d800 tx=0x7fe3b000dcc0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b000bd40 con 0x7fe3c0135dc0 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: dump: 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.393 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.394 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 5 monclient: authenticate success, global_id 94569 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0014430 con 0x7fe3c0135dc0 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3b0014a10 con 0x7fe3c0135dc0 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3be7fc640 4 set_mon_vals no callback set 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3bcff9640 10 monclient: dump: 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.395 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 10 monclient: _renew_subs 2024-09-16T15:18:50.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b8009b30 con 0x7fe3c0135dc0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 10 monclient: _renew_subs 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.377+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c0125780 con 0x7fe3c0135dc0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c6776640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c6776640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 0x7fe3c0122130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0125910 con 0x7fe3c0121d50 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe3b00145a0 con 0x7fe3c0135dc0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bcff9640 1 --2- 172.21.15.134:0/214628786 >> v2:172.21.15.162:6800/980547263 conn(0x7fe3a0038880 0x7fe3a003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7fe3b004a220 con 0x7fe3c0135dc0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 0x7fe3c0122130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient: get_auth_request con 0x7fe3c0121d50 auth_method 0 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/214628786 >> v2:172.21.15.162:6800/980547263 conn(0x7fe3a0038880 0x7fe3a003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3c4d13640 10 monclient: get_auth_request con 0x7fe3a0038880 auth_method 0 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.378+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.379+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_done global_id 124147 payload 824 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.379+0000 7fe3bf7fe640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3c0061ae0 con 0x7fe3c0121d50 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3c4d13640 1 --2- 172.21.15.134:0/214628786 >> v2:172.21.15.162:6800/980547263 conn(0x7fe3a0038880 0x7fe3a003ad40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b8009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bf7fe640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 msgr2=0x7fe3c0122130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 0x7fe3c0122130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0122f30 tx=0x7fe3a8010c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3a8010e70 con 0x7fe3c0121d50 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bcff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:18:50.398 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.381+0000 7fe3bcff9640 10 monclient: dump: 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.399 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe3a800ebe0 con 0x7fe3c0121d50 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 msgr2=0x7fe3c0122130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 0x7fe3c0122130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0122f30 tx=0x7fe3a8010c20 comp rx=0 tx=0).stop 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 msgr2=0x7fe3c0122130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.400 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.382+0000 7fe3bcff9640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0121d50 0x7fe3c0122130 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0122f30 tx=0x7fe3a8010c20 comp rx=0 tx=0).stop 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3c6776640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3c6776640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 0x7fe3c00a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3c6776640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0071b20 con 0x7fe3c0122670 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 0x7fe3c00a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient: get_auth_request con 0x7fe3c0122670 auth_method 0 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 10 monclient(hunting): handle_auth_done global_id 124156 payload 824 2024-09-16T15:18:50.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.388+0000 7fe3bf7fe640 1 -- 172.21.15.134:0/214628786 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe3c007a680 con 0x7fe3c0122670 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bf7fe640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 msgr2=0x7fe3c00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bf7fe640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 0x7fe3c00a1a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8008010 tx=0x7fe3a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe3a8022070 con 0x7fe3c0122670 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bcff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bcff9640 10 monclient: dump: 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.403 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.391+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fe3a80067c0 con 0x7fe3c0122670 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 msgr2=0x7fe3c00a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 0x7fe3c00a1a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8008010 tx=0x7fe3a80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 msgr2=0x7fe3c00a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3bcff9640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3c0122670 0x7fe3c00a1a20 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8008010 tx=0x7fe3a80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.405 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 2, 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.406 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.409 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.410 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.411 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.412 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.413 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.414 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.415 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.415 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.415 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.415 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.416 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.417 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.418 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.419 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.420 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.421 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.422 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.423 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.424 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.425 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.426 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 >> v2:172.21.15.162:6800/980547263 conn(0x7fe3a0038880 msgr2=0x7fe3a003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 --2- 172.21.15.134:0/214628786 >> v2:172.21.15.162:6800/980547263 conn(0x7fe3a0038880 0x7fe3a003ad40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009e50 tx=0x7fe3b8009000 comp rx=0 tx=0).stop 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 10 monclient: shutdown 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 msgr2=0x7fe3c0134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 --2- 172.21.15.134:0/214628786 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe3c0135dc0 0x7fe3c0134aa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d800 tx=0x7fe3b000dcc0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c4d13640 1 -- 172.21.15.134:0/214628786 reap_dead start 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c4d13640 1 -- 172.21.15.134:0/214628786 reap_dead start 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 shutdown_connections 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 >> 172.21.15.134:0/214628786 conn(0x7fe3c00a6110 msgr2=0x7fe3c006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 shutdown_connections 2024-09-16T15:18:50.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.392+0000 7fe3c6776640 1 -- 172.21.15.134:0/214628786 wait complete. 2024-09-16T15:18:50.429 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.429 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:18:50.429 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.525+0000 7f0cb2d83640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.525+0000 7f0cb2d83640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.525+0000 7f0cb2d83640 10 monclient: monmap: 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.526778+0000 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.526778+0000 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.527 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.528 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 1 Processor -- start 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 1 -- start start 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 10 monclient: init 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 10 monclient: _add_conns ranks=[10,0,2,4,6,1,9,8,7,5,3] 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.526+0000 7f0cb2d83640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0cac136e40 0x7f0cac09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-f con 0x7f0cac136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0cac137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-b con 0x7f0cac13c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cac073250 con 0x7f0cac137220 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cac0a56e0 con 0x7f0cac136e40 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac09abc0 con 0x7f0cac13c370 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb2d83640 10 monclient(hunting): authenticate will time out at 1867.116053s 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cabfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0cac137220 0x7f0cac137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:46944/0 (socket says 172.21.15.134:46944) 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47248/0 (socket says 172.21.15.134:47248) 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cabfff640 1 -- 172.21.15.134:0/2028029278 learned_addr learned my addr 172.21.15.134:0/2028029278 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): get_auth_request con 0x7f0cac13c370 auth_method 0 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/2028029278 >> v2:172.21.15.162:3300/0 conn(0x7f0cac136e40 0x7f0cac09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request con 0x7f0cac136e40 auth_method 0 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient(hunting): handle_auth_done global_id 114419 payload 274 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 -- 172.21.15.134:0/2028029278 >> v2:172.21.15.162:3300/0 conn(0x7f0cac136e40 msgr2=0x7f0cac09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 --2- 172.21.15.134:0/2028029278 >> v2:172.21.15.162:3300/0 conn(0x7f0cac136e40 0x7f0cac09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 -- 172.21.15.134:0/2028029278 >> v1:172.21.15.134:6789/0 conn(0x7f0cac137220 legacy=0x7f0cac137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 monclient: found mon.noname-b 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:50.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.527+0000 7f0cb1320640 1 -- 172.21.15.134:0/2028029278 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac09ba50 con 0x7f0cac13c370 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb1320640 1 -- 172.21.15.134:0/2028029278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 msgr2=0x7f0cac13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb1320640 1 --2- 172.21.15.134:0/2028029278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009d90 tx=0x7f0ca402a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7a3aaeb15dfa7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 1 -- 172.21.15.134:0/2028029278 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ca4007bc0 con 0x7f0cac13c370 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: dump: 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.536 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 5 monclient: authenticate success, global_id 114419 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 1 -- 172.21.15.134:0/2028029278 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca402d050 con 0x7f0cac13c370 2024-09-16T15:18:50.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 1 -- 172.21.15.134:0/2028029278 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ca4007d40 con 0x7f0cac13c370 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 4 set_mon_vals no callback set 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cab7fe640 10 monclient: dump: 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.538 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 10 monclient: shutdown 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 -- 172.21.15.134:0/2028029278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 msgr2=0x7f0cac13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/2028029278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009d90 tx=0x7f0ca402a4e0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 -- 172.21.15.134:0/2028029278 shutdown_connections 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/2028029278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cac13c370 0x7f0cac13e760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/2028029278 >> v2:172.21.15.162:3300/0 conn(0x7f0cac136e40 0x7f0cac09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 -- 172.21.15.134:0/2028029278 >> 172.21.15.134:0/2028029278 conn(0x7f0cac0a6110 msgr2=0x7f0cac0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.530+0000 7f0cb2d83640 1 -- 172.21.15.134:0/2028029278 shutdown_connections 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 1 -- 172.21.15.134:0/2028029278 wait complete. 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 10 monclient: monmap: 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.540 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 1 Processor -- start 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 1 -- start start 2024-09-16T15:18:50.541 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.531+0000 7f0cb2d83640 10 monclient: init 2024-09-16T15:18:50.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient: _add_conns ranks=[3,4,7,1,5,8,0,6,2] 2024-09-16T15:18:50.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac136e40 0x7f0cac11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.547 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-g con 0x7f0cac136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-c con 0x7f0cac137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0cac13c370 0x7f0cac1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): picked mon.noname-i con 0x7f0cac13c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac121dd0 con 0x7f0cac136e40 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0a37b0 con 0x7f0cac137220 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0a7410 con 0x7f0cac13c370 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb2d83640 10 monclient(hunting): authenticate will time out at 1867.121506s 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38704/0 (socket says 172.21.15.134:38704) 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 -- 172.21.15.134:0/3200864497 learned_addr learned my addr 172.21.15.134:0/3200864497 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): get_auth_request con 0x7f0cac137220 auth_method 0 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac136e40 0x7f0cac11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request con 0x7f0cac136e40 auth_method 0 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0cac13c370 0x7f0cac1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): get_auth_request con 0x7f0cac13c370 auth_method 0 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.549 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.550 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cb1320640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient(hunting): handle_auth_done global_id 124159 payload 824 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0cac13c370 msgr2=0x7f0cac1206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0cac13c370 0x7f0cac1206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac136e40 msgr2=0x7f0cac11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac136e40 0x7f0cac11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 monclient: found mon.noname-c 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.532+0000 7f0cabfff640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009a70 con 0x7f0cac137220 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cabfff640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 msgr2=0x7f0cac1201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cabfff640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c460 tx=0x7f0c9800c920 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c98005cf0 con 0x7f0cac137220 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: dump: 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.552 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98005e90 con 0x7f0cac137220 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c980074e0 con 0x7f0cac137220 2024-09-16T15:18:50.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0ca97fa640 10 monclient: dump: 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.554 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 5 monclient: authenticate success, global_id 124159 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 10 monclient: _renew_subs 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0caaffd640 4 set_mon_vals no callback set 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac122670 con 0x7f0cac137220 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 10 monclient: _renew_subs 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.535+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac0a78f0 con 0x7f0cac137220 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb2d83640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0c98007070 con 0x7f0cac137220 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 0x7f0cac135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.555 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac071b20 con 0x7f0cac0a29c0 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0ca97fa640 1 --2- 172.21.15.134:0/3200864497 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c90038880 0x7f0c9003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 0x7f0cac135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient: get_auth_request con 0x7f0cac0a29c0 auth_method 0 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f0c98049c40 con 0x7f0cac137220 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb1320640 1 --2- 172.21.15.134:0/3200864497 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c90038880 0x7f0c9003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb1320640 10 monclient: get_auth_request con 0x7f0c90038880 auth_method 0 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_done global_id 84711 payload 824 2024-09-16T15:18:50.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.536+0000 7f0cb0b1f640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cac061ae0 con 0x7f0cac0a29c0 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.538+0000 7f0cb0b1f640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 msgr2=0x7f0cac135630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.538+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 0x7f0cac135630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00118e0 tx=0x7f0ca0011da0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0cb1320640 1 --2- 172.21.15.134:0/3200864497 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c90038880 0x7f0c9003ad40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0cac06b550 tx=0x7f0ca402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ca000fdc0 con 0x7f0cac0a29c0 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 10 monclient: dump: 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.557 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0ca000f950 con 0x7f0cac0a29c0 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 msgr2=0x7f0cac135630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 0x7f0cac135630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00118e0 tx=0x7f0ca0011da0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 msgr2=0x7f0cac135630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.539+0000 7f0ca97fa640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac0a29c0 0x7f0cac135630 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00118e0 tx=0x7f0ca0011da0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb2d83640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 0x7f0cac135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb2d83640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac07a680 con 0x7f0cac135b70 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 0x7f0cac135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient: get_auth_request con 0x7f0cac135b70 auth_method 0 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 10 monclient(hunting): handle_auth_done global_id 84720 payload 824 2024-09-16T15:18:50.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.545+0000 7f0cb0b1f640 1 -- 172.21.15.134:0/3200864497 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0cac0a53c0 con 0x7f0cac135b70 2024-09-16T15:18:50.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0cb0b1f640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 msgr2=0x7f0cac135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0cb0b1f640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 0x7f0cac135f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00118e0 tx=0x7f0ca00046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ca002c070 con 0x7f0cac135b70 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 10 monclient: dump: 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.561 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f0ca0024040 con 0x7f0cac135b70 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 msgr2=0x7f0cac135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 0x7f0cac135f80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00118e0 tx=0x7f0ca00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 msgr2=0x7f0cac135f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.562 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.548+0000 7f0ca97fa640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0cac135b70 0x7f0cac135f80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.563 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.564 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.565 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.566 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.567 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.568 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.569 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.570 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.571 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.572 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.573 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.574 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.575 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.576 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.577 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.578 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.579 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.580 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.581 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.582 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.583 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c90038880 msgr2=0x7f0c9003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/3200864497 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c90038880 0x7f0c9003ad40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0cac06b550 tx=0x7f0ca402f000 comp rx=0 tx=0).stop 2024-09-16T15:18:50.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 10 monclient: shutdown 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 msgr2=0x7f0cac1201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb1320640 1 -- 172.21.15.134:0/3200864497 reap_dead start 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c460 tx=0x7f0c9800c920 comp rx=0 tx=0).stop 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 shutdown_connections 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 --2- 172.21.15.134:0/3200864497 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0cac137220 0x7f0cac1201a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 >> 172.21.15.134:0/3200864497 conn(0x7f0cac0a6110 msgr2=0x7f0cac13b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 shutdown_connections 2024-09-16T15:18:50.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.549+0000 7f0cb2d83640 1 -- 172.21.15.134:0/3200864497 wait complete. 2024-09-16T15:18:50.585 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:18:50.586 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:18:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.684+0000 7fe2b5bb5640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.684+0000 7fe2b5bb5640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.684+0000 7fe2b5bb5640 10 monclient: monmap: 2024-09-16T15:18:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.685 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.685105+0000 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.685105+0000 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.686 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.687 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.684+0000 7fe2b5bb5640 1 Processor -- start 2024-09-16T15:18:50.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 -- start start 2024-09-16T15:18:50.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient: init 2024-09-16T15:18:50.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient: _add_conns ranks=[10,7,3,2,8,6,9,5,4,1,0] 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 0x7fe2b0130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-f con 0x7fe2b0130ab0 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-i con 0x7fe2b01305f0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-g con 0x7fe2b0134030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b009f600 con 0x7fe2b0130ab0 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00673e0 con 0x7fe2b0134030 2024-09-16T15:18:50.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b009e740 con 0x7fe2b01305f0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2b5bb5640 10 monclient(hunting): authenticate will time out at 1867.274524s 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:41108/0 (socket says 172.21.15.134:41108) 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:48178/0 (socket says 172.21.15.134:48178) 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 1 -- 172.21.15.134:0/1935457766 learned_addr learned my addr 172.21.15.134:0/1935457766 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): get_auth_request con 0x7fe2b01305f0 auth_method 0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/1935457766 >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 0x7fe2b0130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): get_auth_request con 0x7fe2b0134030 auth_method 0 2024-09-16T15:18:50.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): get_auth_request con 0x7fe2b0130ab0 auth_method 0 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.685+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 10 monclient(hunting): handle_auth_done global_id 114343 payload 274 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 -- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 msgr2=0x7fe2b01344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 -- 172.21.15.134:0/1935457766 >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 msgr2=0x7fe2b0130e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/1935457766 >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 0x7fe2b0130e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 monclient: found mon.noname-i 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2aeffd640 1 -- 172.21.15.134:0/1935457766 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0073250 con 0x7fe2b01305f0 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2affff640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.686+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/1935457766 >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 0x7fe2b0130e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.688+0000 7fe2aeffd640 1 -- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 msgr2=0x7fe2b0133af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.688+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400bf20 tx=0x7fe2a40093b0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=849af527204025d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 1 -- 172.21.15.134:0/1935457766 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a400b2e0 con 0x7fe2b01305f0 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: dump: 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.700 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 1 -- 172.21.15.134:0/1935457766 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a402d050 con 0x7fe2b01305f0 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 5 monclient: authenticate success, global_id 114343 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 1 -- 172.21.15.134:0/1935457766 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a400b460 con 0x7fe2b01305f0 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 4 set_mon_vals no callback set 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2ae7fc640 10 monclient: dump: 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.702 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 10 monclient: shutdown 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 msgr2=0x7fe2b0133af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400bf20 tx=0x7fe2a40093b0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/1935457766 shutdown_connections 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2b0134030 0x7fe2b01344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/1935457766 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b0133af0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/1935457766 >> v2:172.21.15.162:3300/0 conn(0x7fe2b0130ab0 0x7fe2b0130e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/1935457766 >> 172.21.15.134:0/1935457766 conn(0x7fe2b00a6400 msgr2=0x7fe2b00a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/1935457766 shutdown_connections 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.689+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/1935457766 wait complete. 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.690+0000 7fe2b5bb5640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.690+0000 7fe2b5bb5640 10 monclient: monmap: 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.704 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.690+0000 7fe2b5bb5640 1 Processor -- start 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.690+0000 7fe2b5bb5640 1 -- start start 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.690+0000 7fe2b5bb5640 10 monclient: init 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.705 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient: _add_conns ranks=[7,5,2,1,6,4,8,0,3] 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-i con 0x7fe2b01305f0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b0130ab0 0x7fe2b0070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-h con 0x7fe2b0130ab0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): picked mon.noname-b con 0x7fe2b0134030 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b012ca80 con 0x7fe2b0134030 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b012c1e0 con 0x7fe2b0130ab0 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b012c7a0 con 0x7fe2b01305f0 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.706 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2b5bb5640 10 monclient(hunting): authenticate will time out at 1867.280389s 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47262/0 (socket says 172.21.15.134:47262) 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 learned_addr learned my addr 172.21.15.134:0/2828785743 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): get_auth_request con 0x7fe2b0134030 auth_method 0 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b0130ab0 0x7fe2b0070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient(hunting): handle_auth_done global_id 114428 payload 824 2024-09-16T15:18:50.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 msgr2=0x7fe2b006fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b006fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b0130ab0 msgr2=0x7fe2b0070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b0130ab0 0x7fe2b0070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 monclient: found mon.noname-b 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0009400 con 0x7fe2b0134030 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2af7fe640 10 monclient: get_auth_request con 0x7fe2b01305f0 auth_method 0 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2aeffd640 10 monclient: get_auth_request con 0x7fe2b0130ab0 auth_method 0 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe2b01305f0 0x7fe2b006fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.691+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b0130ab0 0x7fe2b0070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.693+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 msgr2=0x7fe2b0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2affff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012fc0 tx=0x7fe2a8010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a801c070 con 0x7fe2b0134030 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: dump: 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8013040 con 0x7fe2b0134030 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 5 monclient: authenticate success, global_id 114428 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a800d2e0 con 0x7fe2b0134030 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 10 monclient: _renew_subs 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a800d8a0 con 0x7fe2b0134030 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.710 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe297fff640 10 monclient: dump: 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.711 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2adffb640 4 set_mon_vals no callback set 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 10 monclient: _renew_subs 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.694+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a400bb50 con 0x7fe2b0134030 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe2a8010d30 con 0x7fe2b0134030 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2b5bb5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 0x7fe2b00610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006b5d0 con 0x7fe2b002c3e0 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe297fff640 1 --2- 172.21.15.134:0/2828785743 >> v2:172.21.15.162:6800/980547263 conn(0x7fe29c0388b0 0x7fe29c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.712 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(824..824 src has 1..824) v4 ==== 4023+0+0 (secure 0 0 0) 0x7fe2a804df20 con 0x7fe2b0134030 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 0x7fe2b00610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient: get_auth_request con 0x7fe2b002c3e0 auth_method 0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/2828785743 >> v2:172.21.15.162:6800/980547263 conn(0x7fe29c0388b0 0x7fe29c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2aeffd640 10 monclient: get_auth_request con 0x7fe29c0388b0 auth_method 0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_done global_id 45311 payload 824 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.695+0000 7fe2af7fe640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2b012cbf0 con 0x7fe2b002c3e0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.698+0000 7fe2af7fe640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 msgr2=0x7fe2b00610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.698+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 0x7fe2b00610b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0010900 tx=0x7fe2a0010dc0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.713 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.698+0000 7fe2aeffd640 1 --2- 172.21.15.134:0/2828785743 >> v2:172.21.15.162:6800/980547263 conn(0x7fe29c0388b0 0x7fe29c03ad70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2b009d5c0 tx=0x7fe2a400b000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a000ec50 con 0x7fe2b002c3e0 2024-09-16T15:18:50.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:50.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 10 monclient: dump: 2024-09-16T15:18:50.716 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.717 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe2a000edb0 con 0x7fe2b002c3e0 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 msgr2=0x7fe2b00610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 0x7fe2b00610b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0010900 tx=0x7fe2a0010dc0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 msgr2=0x7fe2b00610b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.701+0000 7fe297fff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b002c3e0 0x7fe2b00610b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2b5bb5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 0x7fe2b01d1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2b5bb5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b012c350 con 0x7fe2b00615f0 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 0x7fe2b01d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient: get_auth_request con 0x7fe2b00615f0 auth_method 0 2024-09-16T15:18:50.718 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.708+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.709+0000 7fe2af7fe640 10 monclient(hunting): handle_auth_done global_id 45320 payload 824 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.709+0000 7fe2af7fe640 1 -- 172.21.15.134:0/2828785743 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2b0061ae0 con 0x7fe2b00615f0 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.711+0000 7fe2af7fe640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 msgr2=0x7fe2b01d1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.711+0000 7fe2af7fe640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 0x7fe2b01d1980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0008010 tx=0x7fe2a00046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe2a00219f0 con 0x7fe2b00615f0 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 10 monclient: dump: 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.719 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7fe2a0021b50 con 0x7fe2b00615f0 2024-09-16T15:18:50.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 msgr2=0x7fe2b01d1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 0x7fe2b01d1980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0008010 tx=0x7fe2a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 msgr2=0x7fe2b01d1980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.713+0000 7fe297fff640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe2b00615f0 0x7fe2b01d1980 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0008010 tx=0x7fe2a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.721 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.722 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.723 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.724 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.725 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.726 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.727 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.728 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.729 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.730 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.731 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.732 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.733 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.734 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.735 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.736 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.737 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.738 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.739 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.740 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:50.741 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 >> v2:172.21.15.162:6800/980547263 conn(0x7fe29c0388b0 msgr2=0x7fe29c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/2828785743 >> v2:172.21.15.162:6800/980547263 conn(0x7fe29c0388b0 0x7fe29c03ad70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe2b009d5c0 tx=0x7fe2a400b000 comp rx=0 tx=0).stop 2024-09-16T15:18:50.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2affff640 1 -- 172.21.15.134:0/2828785743 reap_dead start 2024-09-16T15:18:50.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 10 monclient: shutdown 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 msgr2=0x7fe2b0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012fc0 tx=0x7fe2a8010490 comp rx=0 tx=0).stop 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 shutdown_connections 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 --2- 172.21.15.134:0/2828785743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2b0134030 0x7fe2b0070960 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.715+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 >> 172.21.15.134:0/2828785743 conn(0x7fe2b00a6400 msgr2=0x7fe2b0099740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.716+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 shutdown_connections 2024-09-16T15:18:50.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.716+0000 7fe2b5bb5640 1 -- 172.21.15.134:0/2828785743 wait complete. 2024-09-16T15:18:50.744 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.744 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:18:50.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:18:50.745 INFO:tasks.rados.rados.0.smithi162.stdout:3826: done (2 left) 2024-09-16T15:18:50.745 INFO:tasks.rados.rados.0.smithi162.stdout:3827: snap_create 2024-09-16T15:18:50.745 INFO:tasks.rados.rados.0.smithi162.stdout:3824: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:18:50.785 INFO:tasks.rados.rados.0.smithi162.stdout:3825: finishing rollback tid 0 to smithi16234619-46 2024-09-16T15:18:50.785 INFO:tasks.rados.rados.0.smithi162.stdout:3825: finishing rollback tid 1 to smithi16234619-46 2024-09-16T15:18:50.785 INFO:tasks.rados.rados.0.smithi162.stdout:3825: finishing rollback tid 2 to smithi16234619-46 2024-09-16T15:18:50.785 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1236 (ObjNum 1261 snap 392 seq_num 1261) dirty exists 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.840+0000 7fec0d020640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.840+0000 7fec0d020640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.840+0000 7fec0d020640 10 monclient: monmap: 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.841605+0000 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.841605+0000 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.842 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.843 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 Processor -- start 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 -- start start 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient: init 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient: _add_conns ranks=[4,3,2,10,0,9,7,6,1,5,8] 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-c con 0x7fec08136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec08137220 0x7fec08137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.845 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-g con 0x7fec08137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-b con 0x7fec08135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0809abc0 con 0x7fec08135dc0 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0809ba50 con 0x7fec08137220 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec080a53c0 con 0x7fec08136e40 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.841+0000 7fec0d020640 10 monclient(hunting): authenticate will time out at 1867.430897s 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38720/0 (socket says 172.21.15.134:38720) 2024-09-16T15:18:50.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47274/0 (socket says 172.21.15.134:47274) 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 -- 172.21.15.134:0/3842563833 learned_addr learned my addr 172.21.15.134:0/3842563833 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): get_auth_request con 0x7fec08136e40 auth_method 0 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): get_auth_request con 0x7fec08135dc0 auth_method 0 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7febfffff640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec08137220 0x7fec08137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient(hunting): handle_auth_done global_id 124168 payload 274 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 -- 172.21.15.134:0/3842563833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec08137220 msgr2=0x7fec08137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec08137220 0x7fec08137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 -- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 msgr2=0x7fec08136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 monclient: found mon.noname-c 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec06d9d640 1 -- 172.21.15.134:0/3842563833 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08073250 con 0x7fec08136e40 2024-09-16T15:18:50.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.842+0000 7fec0759e640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec06d9d640 1 -- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 msgr2=0x7fec0809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec06d9d640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febf80098a0 tx=0x7febf802a080 comp rx=0 tx=0).ready entity=mon.4 client_cookie=b1d8c7f256fda737 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.855 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 1 -- 172.21.15.134:0/3842563833 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf802ade0 con 0x7fec08136e40 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: dump: 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.856 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 5 monclient: authenticate success, global_id 124168 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 1 -- 172.21.15.134:0/3842563833 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf80307b0 con 0x7fec08136e40 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 1 -- 172.21.15.134:0/3842563833 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf8030d90 con 0x7fec08136e40 2024-09-16T15:18:50.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 4 set_mon_vals no callback set 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0659c640 10 monclient: dump: 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.858 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 10 monclient: shutdown 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 -- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 msgr2=0x7fec0809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febf80098a0 tx=0x7febf802a080 comp rx=0 tx=0).stop 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 -- 172.21.15.134:0/3842563833 shutdown_connections 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08135dc0 0x7fec08136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec08137220 0x7fec08137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 --2- 172.21.15.134:0/3842563833 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fec08136e40 0x7fec0809c150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 -- 172.21.15.134:0/3842563833 >> 172.21.15.134:0/3842563833 conn(0x7fec080a6110 msgr2=0x7fec080a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 -- 172.21.15.134:0/3842563833 shutdown_connections 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.845+0000 7fec0d020640 1 -- 172.21.15.134:0/3842563833 wait complete. 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.846+0000 7fec0d020640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.846+0000 7fec0d020640 10 monclient: monmap: 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.860 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.846+0000 7fec0d020640 1 Processor -- start 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.846+0000 7fec0d020640 1 -- start start 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.846+0000 7fec0d020640 10 monclient: init 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient: _add_conns ranks=[6,5,2,4,0,3,7,1,8] 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-d con 0x7fec08135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fec08136e40 0x7fec08135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.861 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-h con 0x7fec08136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08137220 0x7fec081cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): picked mon.noname-b con 0x7fec08137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec081363b0 con 0x7fec08137220 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec081233c0 con 0x7fec08136e40 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fec08124660 con 0x7fec08135dc0 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0d020640 10 monclient(hunting): authenticate will time out at 1867.436350s 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08137220 0x7fec081cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40418/0 (socket says 172.21.15.134:40418) 2024-09-16T15:18:50.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 -- 172.21.15.134:0/2194047516 learned_addr learned my addr 172.21.15.134:0/2194047516 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): get_auth_request con 0x7fec08135dc0 auth_method 0 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): get_auth_request con 0x7fec08137220 auth_method 0 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fec08136e40 0x7fec08135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): get_auth_request con 0x7fec08136e40 auth_method 0 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient(hunting): handle_auth_done global_id 94578 payload 824 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fec08136e40 msgr2=0x7fec08135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fec08136e40 0x7fec08135470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08137220 msgr2=0x7fec081cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08137220 0x7fec081cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 monclient: found mon.noname-d 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec06d9d640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec00009a70 con 0x7fec08135dc0 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.847+0000 7fec0759e640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec08137220 0x7fec081cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.849+0000 7fec06d9d640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 msgr2=0x7fec081c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.849+0000 7fec06d9d640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7febf80090b0 tx=0x7febf80046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf8003610 con 0x7fec08135dc0 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: dump: 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.865 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:50.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf80037d0 con 0x7fec08135dc0 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 5 monclient: authenticate success, global_id 94578 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 10 monclient: _renew_subs 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf80094d0 con 0x7fec08135dc0 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf8003db0 con 0x7fec08135dc0 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7febff7fe640 10 monclient: dump: 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.867 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec05d9b640 4 set_mon_vals no callback set 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 10 monclient: _renew_subs 2024-09-16T15:18:50.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.850+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec08125e60 con 0x7fec08135dc0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0d020640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec06d9d640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7febf80090b0 tx=0x7febf80046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7febf806a870 osd_map(825..825 src has 1..825) v4 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0d020640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 0x7fec0802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7febf8039040 con 0x7fec08135dc0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fec08067a80 con 0x7fec081359b0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febff7fe640 1 --2- 172.21.15.134:0/2194047516 >> v2:172.21.15.162:6800/980547263 conn(0x7febe8038880 0x7febe803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 0x7fec0802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient: get_auth_request con 0x7fec081359b0 auth_method 0 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0759e640 1 --2- 172.21.15.134:0/2194047516 >> v2:172.21.15.162:6800/980547263 conn(0x7febe8038880 0x7febe803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7fec0759e640 10 monclient: get_auth_request con 0x7febe8038880 auth_method 0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 10 monclient(hunting): handle_auth_done global_id 114352 payload 824 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.851+0000 7febfffff640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec08061ae0 con 0x7fec081359b0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febfffff640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 msgr2=0x7fec0802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febfffff640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 0x7fec0802b260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec08121eb0 tx=0x7febf000fc70 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7fec0759e640 1 --2- 172.21.15.134:0/2194047516 >> v2:172.21.15.162:6800/980547263 conn(0x7febe8038880 0x7febe803ad40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fec00009fd0 tx=0x7fec0000b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf000de60 con 0x7fec081359b0 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 10 monclient: dump: 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.870 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7febf000c040 con 0x7fec081359b0 2024-09-16T15:18:50.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:50.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:50.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 msgr2=0x7fec0802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 0x7fec0802b260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec08121eb0 tx=0x7febf000fc70 comp rx=0 tx=0).stop 2024-09-16T15:18:50.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 msgr2=0x7fec0802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.854+0000 7febff7fe640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec081359b0 0x7fec0802b260 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec08121eb0 tx=0x7febf000fc70 comp rx=0 tx=0).stop 2024-09-16T15:18:50.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7fec0d020640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:50.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7fec0d020640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 0x7fec0802bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:50.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7fec0d020640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:50.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fec08071b20 con 0x7fec0802b7a0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7febfffff640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 0x7fec0802bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7febfffff640 10 monclient: get_auth_request con 0x7fec0802b7a0 auth_method 0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7febfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7febfffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.860+0000 7febfffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.861+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.861+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.861+0000 7febfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.861+0000 7febfffff640 10 monclient(hunting): handle_auth_done global_id 114361 payload 824 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.861+0000 7febfffff640 1 -- 172.21.15.134:0/2194047516 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fec0807a680 con 0x7fec0802b7a0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.863+0000 7febfffff640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 msgr2=0x7fec0802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.863+0000 7febfffff640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 0x7fec0802bbb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febf000afd0 tx=0x7febf00046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7febf0021070 con 0x7fec0802b7a0 2024-09-16T15:18:50.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 10 monclient: dump: 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:50.875 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7febf0003a30 con 0x7fec0802b7a0 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 msgr2=0x7fec0802bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 0x7fec0802bbb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febf000afd0 tx=0x7febf00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 msgr2=0x7fec0802bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fec0802b7a0 0x7fec0802bbb0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febf000afd0 tx=0x7febf00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.876 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:50.877 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.878 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.879 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:50.880 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.881 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.882 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:50.883 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.884 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:50.885 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.886 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.887 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.888 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.889 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:50.890 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.891 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:50.892 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.893 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:50.894 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.895 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.896 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:50.897 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 >> v2:172.21.15.162:6800/980547263 conn(0x7febe8038880 msgr2=0x7febe803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 --2- 172.21.15.134:0/2194047516 >> v2:172.21.15.162:6800/980547263 conn(0x7febe8038880 0x7febe803ad40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fec00009fd0 tx=0x7fec0000b040 comp rx=0 tx=0).stop 2024-09-16T15:18:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 10 monclient: shutdown 2024-09-16T15:18:50.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 msgr2=0x7fec081c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 --2- 172.21.15.134:0/2194047516 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fec08135dc0 0x7fec081c8180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7febf80090b0 tx=0x7febf80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0759e640 1 -- 172.21.15.134:0/2194047516 reap_dead start 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7febff7fe640 1 -- 172.21.15.134:0/2194047516 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(825..825 src has 1..825) v4 ==== 4023+0+0 (unknown 0 0 0) 0x7febf806a870 con 0x7fec08135dc0 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0759e640 1 -- 172.21.15.134:0/2194047516 reap_dead start 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 shutdown_connections 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 >> 172.21.15.134:0/2194047516 conn(0x7fec080a6110 msgr2=0x7fec0809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 shutdown_connections 2024-09-16T15:18:50.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.866+0000 7fec0d020640 1 -- 172.21.15.134:0/2194047516 wait complete. 2024-09-16T15:18:50.899 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:50.899 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:18:50.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 10 monclient: get_monmap_and_config 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 10 monclient: build_initial_monmap 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 10 monclient: monmap: 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:50.995993+0000 2024-09-16T15:18:50.996 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:50.995993+0000 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:50.997 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:50.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 1 Processor -- start 2024-09-16T15:18:50.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 1 -- start start 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.995+0000 7f4ddcf11640 10 monclient: init 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient: _add_conns ranks=[7,9,1,4,10,5,0,8,2,3,6] 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136030 0x7f4dd8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-i con 0x7f4dd8136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dd809bfe0 0x7f4dd809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-a con 0x7f4dd809bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f4dd8136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd8073250 con 0x7f4dd8136c60 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd809b600 con 0x7f4dd809bfe0 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:51.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80a5740 con 0x7f4dd8136030 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4ddcf11640 10 monclient(hunting): authenticate will time out at 1867.585426s 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dd809bfe0 0x7f4dd809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4dd809bfe0 0x7f4dd809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33486/0 (socket says 172.21.15.134:33486) 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 -- 172.21.15.134:0/469818182 learned_addr learned my addr 172.21.15.134:0/469818182 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f4dd809bfe0 auth_method 0 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136030 0x7f4dd8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): get_auth_request con 0x7f4dd8136030 auth_method 0 2024-09-16T15:18:51.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_done global_id 124155 payload 274 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 -- 172.21.15.134:0/469818182 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136030 msgr2=0x7f4dd8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136030 0x7f4dd8136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 -- 172.21.15.134:0/469818182 >> v1:172.21.15.162:6789/0 conn(0x7f4dd8136c60 legacy=0x7f4dd8137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 monclient: found mon.noname-a 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.996+0000 7f4dd5d9b640 1 -- 172.21.15.134:0/469818182 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80a52c0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd5d9b640 1 -- 172.21.15.134:0/469818182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 msgr2=0x7f4dd809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd5d9b640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd809c400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009870 tx=0x7f4dcc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3159e717453e8937 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 1 -- 172.21.15.134:0/469818182 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dcc02ae20 con 0x7f4dd809bfe0 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: dump: 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:51.008 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:51.009 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: _finish_auth 0 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: _check_auth_tickets 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 1 -- 172.21.15.134:0/469818182 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0307b0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4ddcf11640 5 monclient: authenticate success, global_id 124155 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4ddcf11640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4ddcf11640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 1 -- 172.21.15.134:0/469818182 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dcc030dd0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4ddcf11640 4 set_mon_vals no callback set 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:50.999+0000 7f4dd559a640 10 monclient: dump: 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:51.010 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:51.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 10 monclient: shutdown 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 -- 172.21.15.134:0/469818182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 msgr2=0x7f4dd809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd809c400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009870 tx=0x7f4dcc02a100 comp rx=0 tx=0).stop 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 -- 172.21.15.134:0/469818182 shutdown_connections 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd809c400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/469818182 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136030 0x7f4dd8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 -- 172.21.15.134:0/469818182 >> 172.21.15.134:0/469818182 conn(0x7f4dd80a9ea0 msgr2=0x7f4dd80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 -- 172.21.15.134:0/469818182 shutdown_connections 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.000+0000 7f4ddcf11640 1 -- 172.21.15.134:0/469818182 wait complete. 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient: build_initial_monmap 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient: monmap: 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:51.012 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 Processor -- start 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 -- start start 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient: init 2024-09-16T15:18:51.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient: _add_conns ranks=[0,3,7,8,4,6,1,5,2] 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-a con 0x7f4dd809bfe0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dd8136030 0x7f4dd8134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-g con 0x7f4dd8136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136c60 0x7f4dd8135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): picked mon.noname-i con 0x7f4dd8136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80a7e40 con 0x7f4dd809bfe0 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd809f9a0 con 0x7f4dd8136030 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8120df0 con 0x7f4dd8136c60 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:51.014 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4ddcf11640 10 monclient(hunting): authenticate will time out at 1867.590900s 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.001+0000 7f4dd659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33492/0 (socket says 172.21.15.134:33492) 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 -- 172.21.15.134:0/2158582174 learned_addr learned my addr 172.21.15.134:0/2158582174 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): get_auth_request con 0x7f4dd809bfe0 auth_method 0 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dd8136030 0x7f4dd8134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): get_auth_request con 0x7f4dd8136030 auth_method 0 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136c60 0x7f4dd8135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:51.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): get_auth_request con 0x7f4dd8136c60 auth_method 0 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd5d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient(hunting): handle_auth_done global_id 124164 payload 824 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 -- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136c60 msgr2=0x7f4dd8135100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136c60 0x7f4dd8135100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 -- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dd8136030 msgr2=0x7f4dd8134bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dd8136030 0x7f4dd8134bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 monclient: found mon.noname-a 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.002+0000 7f4dd659c640 1 -- 172.21.15.134:0/2158582174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0094d0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dd659c640 1 -- 172.21.15.134:0/2158582174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 msgr2=0x7f4dd80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dd659c640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800efc0 tx=0x7f4dc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dc80148f0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: dump: 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:51.017 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:51.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 5 monclient: authenticate success, global_id 124164 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc800f040 con 0x7f4dd809bfe0 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4dc80133e0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dc6ffd640 10 monclient: dump: 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:51.019 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4dd4d99640 4 set_mon_vals no callback set 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 10 monclient: _renew_subs 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd8121ce0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 10 monclient: _renew_subs 2024-09-16T15:18:51.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.004+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd81222a0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4dc8014480 con 0x7f4dd809bfe0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4ddcf11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4ddcf11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4ddcf11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd809c400 con 0x7f4dd809bfe0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4dc6ffd640 1 --2- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 0x7f4db803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(825..825 src has 1..825) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f4dc804a770 con 0x7f4dd809bfe0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4dd5d9b640 1 --2- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 0x7f4db803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.005+0000 7f4dd5d9b640 10 monclient: get_auth_request con 0x7f4db80388b0 auth_method 0 2024-09-16T15:18:51.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.008+0000 7f4dd5d9b640 1 --2- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 0x7f4db803ad70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc034000 tx=0x7f4dcc02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:51.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.010+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4dc8014730 con 0x7f4dd809bfe0 2024-09-16T15:18:51.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.010+0000 7f4dc6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:51.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.010+0000 7f4dc6ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.231+0000 7f4ddcf11640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:18:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.231+0000 7f4ddcf11640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:18:51.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.231+0000 7f4ddcf11640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.231+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4dd8074df0 con 0x7f4dd809bfe0 2024-09-16T15:18:51.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.232+0000 7f4dc6ffd640 1 -- 172.21.15.134:0/2158582174 <== mon.0 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4dc8018020 con 0x7f4dd809bfe0 2024-09-16T15:18:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.232+0000 7f4dc6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:18:51.234 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.232+0000 7f4dc6ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=11,logm=55} crc {auth=3773645957,config=2586702516,health=1564785003,kv=2983865385,logm=2700063411}) 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=3190276872,config=2586702516,health=1564785003,kv=3324725102,logm=1245804793}) 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.236 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=12,kv=12,logm=53} crc {auth=4150984348,config=2586702516,health=1564785003,kv=1090497770,logm=2366519117}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=956236439,config=2586702516,health=1564785004,kv=426406850,logm=1614386262}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=21,config=2,health=11,kv=12,logm=54} crc {auth=897809554,config=2586702516,health=2923331907,kv=1090497770,logm=1527444522}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=22,config=2,health=9,kv=10,logm=57} crc {auth=920009119,config=2586702516,health=3894801024,kv=2983865385,logm=361084526}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1315662316,config=2586702516,health=1564785003,kv=1090497770,logm=2846295071}) 2024-09-16T15:18:51.237 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=1,health=12,kv=11,logm=56} crc {auth=360927243,config=1747058506,health=1564785003,kv=1671578860,logm=2504392389}) 2024-09-16T15:18:51.238 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.233+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=21,config=2,health=11,kv=11,logm=55} crc {auth=3226331108,config=2586702516,health=2325320394,kv=1302766305,logm=4072462265}) 2024-09-16T15:18:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 msgr2=0x7f4db803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:51.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 0x7f4db803ad70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc034000 tx=0x7f4dcc02e000 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 10 monclient: shutdown 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 msgr2=0x7f4dd80a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800efc0 tx=0x7f4dc800c490 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 shutdown_connections 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4dd8136c60 0x7f4dd8135100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> v2:172.21.15.162:6800/980547263 conn(0x7f4db80388b0 0x7f4db803ad70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4dd8136030 0x7f4dd8134bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 --2- 172.21.15.134:0/2158582174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd809bfe0 0x7f4dd80a2610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 >> 172.21.15.134:0/2158582174 conn(0x7f4dd80a9ea0 msgr2=0x7f4dd80a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 shutdown_connections 2024-09-16T15:18:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:51.235+0000 7f4ddcf11640 1 -- 172.21.15.134:0/2158582174 wait complete. 2024-09-16T15:18:51.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:18:51.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2465177987}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=831253460}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=689670678}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=973731267}) 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.656 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432072371}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2149036179}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3188151595}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2892625762}) 2024-09-16T15:18:51.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3305799779}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=899066431}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2285564404}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.659 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4007296165}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=113599442}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=913838436}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2834329656}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984673370}) 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124075911}) 2024-09-16T15:18:51.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.655+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3209769332}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=75135989}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3737698783}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2829292852}) 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=989806598}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1844939799}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3907413843}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2564720950}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=478006404}) 2024-09-16T15:18:51.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3601828036}) 2024-09-16T15:18:51.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4081049105}) 2024-09-16T15:18:51.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3755577836}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1453435431}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1658204949}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=177359876}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3212053514}) 2024-09-16T15:18:51.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3845673556}) 2024-09-16T15:18:51.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1545355095}) 2024-09-16T15:18:51.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.712+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2518269488}) 2024-09-16T15:18:51.716 INFO:tasks.rados.rados.0.smithi162.stdout:3824: done (2 left) 2024-09-16T15:18:51.716 INFO:tasks.rados.rados.0.smithi162.stdout:3825: done (1 left) 2024-09-16T15:18:51.716 INFO:tasks.rados.rados.0.smithi162.stdout:3827: done (0 left) 2024-09-16T15:18:51.717 INFO:tasks.rados.rados.0.smithi162.stdout:3828: read oid 6 snap -1 2024-09-16T15:18:51.717 INFO:tasks.rados.rados.0.smithi162.stdout:3828: expect (ObjNum 1217 snap 374 seq_num 1217) 2024-09-16T15:18:51.717 INFO:tasks.rados.rados.0.smithi162.stdout:3829: snap_create 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1185763003}) 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1009775997}) 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=142267964}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1410615591}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2548523352}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2276494357}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1177167179}) 2024-09-16T15:18:51.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:51.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=130067559}) 2024-09-16T15:18:51.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:51.721+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1321389304}) 2024-09-16T15:18:51.727 INFO:tasks.rados.rados.0.smithi162.stdout:3828: expect (ObjNum 1217 snap 374 seq_num 1217) 2024-09-16T15:18:52.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=714127163}) 2024-09-16T15:18:52.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.040 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3162987273}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1186605813}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3784285094}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112483424}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=726193703}) 2024-09-16T15:18:52.041 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3143913571}) 2024-09-16T15:18:52.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2921387965}) 2024-09-16T15:18:52.042 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.037+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3841805400}) 2024-09-16T15:18:52.045 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.042+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1387326598}) 2024-09-16T15:18:52.047 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1489658577}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=122997745}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2221625740}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1649046190}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.048 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3983927835}) 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=676548485}) 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=371958169}) 2024-09-16T15:18:52.049 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.043+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1980463025}) 2024-09-16T15:18:52.049 INFO:tasks.rados.rados.0.smithi162.stdout:3828: done (1 left) 2024-09-16T15:18:52.049 INFO:tasks.rados.rados.0.smithi162.stdout:3829: done (0 left) 2024-09-16T15:18:52.049 INFO:tasks.rados.rados.0.smithi162.stdout:3830: snap_create 2024-09-16T15:18:52.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3229273131}) 2024-09-16T15:18:52.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3845287219}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3481501922}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3067671734}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3634040752}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2384950863}) 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145363450}) 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950796895}) 2024-09-16T15:18:52.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.053+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=575142815}) 2024-09-16T15:18:52.248 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:18:52.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:52.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.266 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1263135327}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=302362585}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2029515256}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=124668212}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.267 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1573499586}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4039382231}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4025977239}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705196652}) 2024-09-16T15:18:52.268 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.264+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3156306878}) 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.345+0000 7f5898926640 10 monclient: get_monmap_and_config 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.345+0000 7f5898926640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.345+0000 7f5898926640 10 monclient: monmap: 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:52.346736+0000 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:52.346736+0000 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.347 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:52.348 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.346+0000 7f5898926640 1 Processor -- start 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.346+0000 7f5898926640 1 -- start start 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.346+0000 7f5898926640 10 monclient: init 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.346+0000 7f5898926640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.346+0000 7f5898926640 10 monclient: _add_conns ranks=[1,9,2,3,0,4,10,8,6,7,5] 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f5894136080 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): picked mon.noname-a con 0x7f58940a5700 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): picked mon.noname-b con 0x7f589413c610 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5894073250 con 0x7f5894136080 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589409bb00 con 0x7f58940a5700 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589409b4b0 con 0x7f589413c610 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5898926640 10 monclient(hunting): authenticate will time out at 1868.936096s 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47296/0 (socket says 172.21.15.134:47296) 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 -- 172.21.15.134:0/1523229625 learned_addr learned my addr 172.21.15.134:0/1523229625 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): get_auth_request con 0x7f589413c610 auth_method 0 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 1 --2- 172.21.15.134:0/1523229625 >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33496/0 (socket says 172.21.15.134:33496) 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): get_auth_request con 0x7f58940a5700 auth_method 0 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient(hunting): handle_auth_done global_id 114455 payload 274 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 -- 172.21.15.134:0/1523229625 >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 msgr2=0x7f589413c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 --2- 172.21.15.134:0/1523229625 >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 -- 172.21.15.134:0/1523229625 >> v1:172.21.15.162:6789/0 conn(0x7f5894136080 legacy=0x7f5894136460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.353 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 monclient: found mon.noname-b 2024-09-16T15:18:52.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:52.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5892d9d640 1 -- 172.21.15.134:0/1523229625 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894136b70 con 0x7f589413c610 2024-09-16T15:18:52.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.347+0000 7f5889d9b640 1 --2- 172.21.15.134:0/1523229625 >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:52.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5892d9d640 1 -- 172.21.15.134:0/1523229625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 msgr2=0x7f589413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5892d9d640 1 --2- 172.21.15.134:0/1523229625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f588c009e50 tx=0x7f588c02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdfa4092797b030 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 1 -- 172.21.15.134:0/1523229625 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f588c007a70 con 0x7f589413c610 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: dump: 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.360 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 1 -- 172.21.15.134:0/1523229625 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c02d050 con 0x7f589413c610 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 5 monclient: authenticate success, global_id 114455 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 1 -- 172.21.15.134:0/1523229625 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f588c007c30 con 0x7f589413c610 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 4 set_mon_vals no callback set 2024-09-16T15:18:52.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5891d9b640 10 monclient: dump: 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.362 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 10 monclient: shutdown 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 -- 172.21.15.134:0/1523229625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 msgr2=0x7f589413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 --2- 172.21.15.134:0/1523229625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f588c009e50 tx=0x7f588c02a5b0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 -- 172.21.15.134:0/1523229625 shutdown_connections 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 --2- 172.21.15.134:0/1523229625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f589413c610 0x7f589413ea00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 --2- 172.21.15.134:0/1523229625 >> v2:172.21.15.134:3300/0 conn(0x7f58940a5700 0x7f589413c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.350+0000 7f5898926640 1 -- 172.21.15.134:0/1523229625 >> 172.21.15.134:0/1523229625 conn(0x7f58940a99a0 msgr2=0x7f58940a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.351+0000 7f5898926640 1 -- 172.21.15.134:0/1523229625 shutdown_connections 2024-09-16T15:18:52.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.351+0000 7f5898926640 1 -- 172.21.15.134:0/1523229625 wait complete. 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.351+0000 7f5898926640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.351+0000 7f5898926640 10 monclient: monmap: 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.364 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 Processor -- start 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 -- start start 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient: init 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient: _add_conns ranks=[4,8,5,2,0,1,6,3,7] 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): picked mon.noname-c con 0x7f58940a5700 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): picked mon.noname-e con 0x7f5894136080 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f589413c610 0x7f58941398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): picked mon.noname-h con 0x7f589413c610 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:52.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5894121230 con 0x7f58940a5700 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5894121480 con 0x7f589413c610 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f589413c1d0 con 0x7f5894136080 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5898926640 10 monclient(hunting): authenticate will time out at 1868.941766s 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38724/0 (socket says 172.21.15.134:38724) 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41778/0 (socket says 172.21.15.134:41778) 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 1 -- 172.21.15.134:0/137297218 learned_addr learned my addr 172.21.15.134:0/137297218 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): get_auth_request con 0x7f58940a5700 auth_method 0 2024-09-16T15:18:52.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 10 monclient(hunting): get_auth_request con 0x7f5894136080 auth_method 0 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f5889d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.352+0000 7f589259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5889d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f589413c610 0x7f58941398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): get_auth_request con 0x7f589413c610 auth_method 0 2024-09-16T15:18:52.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5892d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 10 monclient(hunting): handle_auth_done global_id 124177 payload 824 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 -- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 msgr2=0x7f58941cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 -- 172.21.15.134:0/137297218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f589413c610 msgr2=0x7f58941398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f589413c610 0x7f58941398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 monclient: found mon.noname-c 2024-09-16T15:18:52.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f589259c640 1 -- 172.21.15.134:0/137297218 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009400 con 0x7f58940a5700 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.353+0000 7f5889d9b640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f589259c640 1 -- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 msgr2=0x7f58941cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f589259c640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f588400c430 tx=0x7f588400c8f0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5884003860 con 0x7f58940a5700 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: dump: 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.369 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884011c10 con 0x7f58940a5700 2024-09-16T15:18:52.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 5 monclient: authenticate success, global_id 124177 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5884011dc0 con 0x7f58940a5700 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f588b7fe640 10 monclient: dump: 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.371 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 10 monclient: _renew_subs 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f589159a640 4 set_mon_vals no callback set 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c009b30 con 0x7f58940a5700 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 10 monclient: _renew_subs 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.355+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58940a3200 con 0x7f58940a5700 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f58840117a0 con 0x7f58940a5700 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f5898926640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f5898926640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f5898926640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5894120d00 con 0x7f58940a5700 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f588b7fe640 1 --2- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 0x7f587403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.356+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(827..827 src has 1..827) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f5884049f60 con 0x7f58940a5700 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.357+0000 7f5889d9b640 1 --2- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 0x7f587403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.357+0000 7f5889d9b640 10 monclient: get_auth_request con 0x7f5874038880 auth_method 0 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.359+0000 7f5889d9b640 1 --2- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 0x7f587403ad40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5894120b10 tx=0x7f587c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.361+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f5884011a50 con 0x7f58940a5700 2024-09-16T15:18:52.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.361+0000 7f588b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.361+0000 7f588b7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:52.562 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=972236316}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=327489323}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2999446556}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=425502959}) 2024-09-16T15:18:52.563 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1714643087}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=668999031}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1862055335}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2484650249}) 2024-09-16T15:18:52.564 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.547+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4134342367}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=654773501}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=30370248}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3638701013}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1075063309}) 2024-09-16T15:18:52.565 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2807041271}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1924317488}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2920402311}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3969164294}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.549+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2092892335}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3031064883}) 2024-09-16T15:18:52.566 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=995118016}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1688492598}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=291453961}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1417462932}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=137385378}) 2024-09-16T15:18:52.567 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4218808590}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381323492}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.550+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2449700841}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=651570253}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1322409321}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.568 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=417494615}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.551+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=915850203}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=838602247}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4267424287}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2286353087}) 2024-09-16T15:18:52.569 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3856295120}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.552+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4143863334}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3449075081}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1730709353}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3403247127}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1007968266}) 2024-09-16T15:18:52.570 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1841229525}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1706290410}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2382113837}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=97892746}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.553+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3083363561}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2009084484}) 2024-09-16T15:18:52.571 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=41250813}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3096857235}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1230894134}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2251751544}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4115188060}) 2024-09-16T15:18:52.572 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.554+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.555+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.555+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2076635660}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.555+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.555+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1900260296}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.555+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2743542022}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1689827620}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1052543023}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2246663197}) 2024-09-16T15:18:52.573 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2531761026}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3464778978}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445254876}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2859006746}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1738586235}) 2024-09-16T15:18:52.574 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.556+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1972154055}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4200550755}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3407220870}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=791950296}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=919300534}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.575 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1971947462}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1214429584}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.557+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1228438878}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.558+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.558+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=164450563}) 2024-09-16T15:18:52.576 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.558+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1500248844}) 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.590+0000 7f5898926640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.590+0000 7f5898926640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.590+0000 7f5898926640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.590+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5894120fe0 con 0x7f58940a5700 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.591+0000 7f588b7fe640 1 -- 172.21.15.134:0/137297218 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f5884019b50 con 0x7f58940a5700 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.591+0000 7f588b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:52.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.591+0000 7f588b7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:52.593 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:52.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 msgr2=0x7f587403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 0x7f587403ad40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5894120b10 tx=0x7f587c009190 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 10 monclient: shutdown 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 msgr2=0x7f58941cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f588400c430 tx=0x7f588400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 shutdown_connections 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> v2:172.21.15.162:6800/980547263 conn(0x7f5874038880 0x7f587403ad40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f589413c610 0x7f58941398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f5894136080 0x7f58941cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 --2- 172.21.15.134:0/137297218 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f58940a5700 0x7f58941cb980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 >> 172.21.15.134:0/137297218 conn(0x7f58940a99a0 msgr2=0x7f589406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 shutdown_connections 2024-09-16T15:18:52.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.593+0000 7f5898926640 1 -- 172.21.15.134:0/137297218 wait complete. 2024-09-16T15:18:52.605 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:18:52.605 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-16T15:18:52.605 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:18:52.605 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2685841226}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1006771866}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2913871141}) 2024-09-16T15:18:52.654 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3305712138}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3955509469}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4061141767}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3909324236}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668096134}) 2024-09-16T15:18:52.655 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.652+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2726003119}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=739907014}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1250520965}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.653+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=346749539}) 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.657 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2602016081}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2229694056}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2607584658}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1800192917}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3950185210}) 2024-09-16T15:18:52.658 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.654+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1815644288}) 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=141792112}) 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3177923694}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3835959918}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1583400341}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1776563855}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3519597378}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2980586673}) 2024-09-16T15:18:52.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2014184025}) 2024-09-16T15:18:52.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.679+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2268806659}) 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3571028243}) 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=601734135}) 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=54955706}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3607944001}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1769959635}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.680+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=314529377}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=309001003}) 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4230986773}) 2024-09-16T15:18:52.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.681+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1066862421}) 2024-09-16T15:18:52.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2107931458}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1642144124}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3563416470}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3809222244}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3066727274}) 2024-09-16T15:18:52.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4155458668}) 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=530384492}) 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1348173637}) 2024-09-16T15:18:52.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.720+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=302853822}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2521541170}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1103680168}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3289363584}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3827259736}) 2024-09-16T15:18:52.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1893939010}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2204526021}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2999074545}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4224982573}) 2024-09-16T15:18:52.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.737+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3575383100}) 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.753+0000 7f6efbac9640 10 monclient: get_monmap_and_config 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.754+0000 7f6efbac9640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.754+0000 7f6efbac9640 10 monclient: monmap: 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:52.754959+0000 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:52.754959+0000 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.755 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:52.756 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.754+0000 7f6efbac9640 1 Processor -- start 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.754+0000 7f6efbac9640 1 -- start start 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.754+0000 7f6efbac9640 10 monclient: init 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient: _add_conns ranks=[0,3,1,10,8,5,2,6,7,9,4] 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6ef4136e40 addr v1:172.21.15.134:6789/0 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 0x7f6ef4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-g con 0x7f6ef4137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f6ef413c370 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef4073250 con 0x7f6ef4136e40 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef409ba50 con 0x7f6ef413c370 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40a56e0 con 0x7f6ef4137220 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6efbac9640 10 monclient(hunting): authenticate will time out at 1869.344304s 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9865640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6ef4136e40 0x7f6ef409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:46954/0 (socket says 172.21.15.134:46954) 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9865640 1 -- 172.21.15.134:0/3750139835 learned_addr learned my addr 172.21.15.134:0/3750139835 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 1 --2- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 0x7f6ef4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): get_auth_request con 0x7f6ef4137220 auth_method 0 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient(hunting): handle_auth_done global_id 84747 payload 274 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 1 -- 172.21.15.134:0/3750139835 >> v1:172.21.15.162:6789/0 conn(0x7f6ef413c370 legacy=0x7f6ef413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 1 -- 172.21.15.134:0/3750139835 >> v1:172.21.15.134:6789/0 conn(0x7f6ef4136e40 legacy=0x7f6ef409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 1 monclient: found mon.noname-g 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:18:52.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.755+0000 7f6ef9064640 1 -- 172.21.15.134:0/3750139835 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40a53c0 con 0x7f6ef4137220 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.758+0000 7f6ef9064640 1 -- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 msgr2=0x7f6ef4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.758+0000 7f6ef9064640 1 --2- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 0x7f6ef4137600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80097d0 tx=0x7f6ee802a040 comp rx=0 tx=0).ready entity=mon.3 client_cookie=bdbd39622d12cdca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 1 -- 172.21.15.134:0/3750139835 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ee802ae40 con 0x7f6ef4137220 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: dump: 2024-09-16T15:18:52.764 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.765 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 1 -- 172.21.15.134:0/3750139835 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee80307b0 con 0x7f6ef4137220 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 1 -- 172.21.15.134:0/3750139835 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ee8030d90 con 0x7f6ef4137220 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 5 monclient: authenticate success, global_id 84747 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 4 set_mon_vals no callback set 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6ef8863640 10 monclient: dump: 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.766 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 10 monclient: shutdown 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 -- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 msgr2=0x7f6ef4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 --2- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 0x7f6ef4137600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80097d0 tx=0x7f6ee802a040 comp rx=0 tx=0).stop 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 -- 172.21.15.134:0/3750139835 shutdown_connections 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 --2- 172.21.15.134:0/3750139835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ef4137220 0x7f6ef4137600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 -- 172.21.15.134:0/3750139835 >> 172.21.15.134:0/3750139835 conn(0x7f6ef40a6110 msgr2=0x7f6ef40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 -- 172.21.15.134:0/3750139835 shutdown_connections 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.759+0000 7f6efbac9640 1 -- 172.21.15.134:0/3750139835 wait complete. 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.760+0000 7f6efbac9640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.760+0000 7f6efbac9640 10 monclient: monmap: 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:52.768 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.760+0000 7f6efbac9640 1 Processor -- start 2024-09-16T15:18:52.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.760+0000 7f6efbac9640 1 -- start start 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.760+0000 7f6efbac9640 10 monclient: init 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient: _add_conns ranks=[7,0,4,1,5,2,3,6,8] 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6ef4136e40 0x7f6ef411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-i con 0x7f6ef4136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-a con 0x7f6ef4137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6ef413c370 0x7f6ef41206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): picked mon.noname-c con 0x7f6ef413c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40a75f0 con 0x7f6ef4137220 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4122790 con 0x7f6ef413c370 2024-09-16T15:18:52.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4121000 con 0x7f6ef4136e40 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efbac9640 10 monclient(hunting): authenticate will time out at 1869.350145s 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33502/0 (socket says 172.21.15.134:33502) 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 -- 172.21.15.134:0/2077741788 learned_addr learned my addr 172.21.15.134:0/2077741788 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): get_auth_request con 0x7f6ef4137220 auth_method 0 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6ef413c370 0x7f6ef41206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): get_auth_request con 0x7f6ef413c370 auth_method 0 2024-09-16T15:18:52.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6ef4136e40 0x7f6ef411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): get_auth_request con 0x7f6ef4136e40 auth_method 0 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9865640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient(hunting): handle_auth_done global_id 124182 payload 824 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6ef4136e40 msgr2=0x7f6ef411fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f6ef4136e40 0x7f6ef411fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6ef413c370 msgr2=0x7f6ef41206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6ef413c370 0x7f6ef41206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 monclient: found mon.noname-a 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6ef9064640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee8009400 con 0x7f6ef4137220 2024-09-16T15:18:52.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.761+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6ef413c370 0x7f6ef41206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef9064640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 msgr2=0x7f6ef41201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef9064640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802a080 tx=0x7f6ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ee8004500 con 0x7f6ef4137220 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.763+0000 7f6ef27fc640 10 monclient: dump: 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.778 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.779 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee8030d40 con 0x7f6ef4137220 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 5 monclient: authenticate success, global_id 124182 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ee803e450 con 0x7f6ef4137220 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 10 monclient: _renew_subs 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4122480 con 0x7f6ef4137220 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 10 monclient: dump: 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.781 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef3fff640 4 set_mon_vals no callback set 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 10 monclient: _renew_subs 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef413b530 con 0x7f6ef4137220 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6ee8003610 con 0x7f6ef4137220 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6ef27fc640 1 --2- 172.21.15.134:0/2077741788 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ee0038880 0x7f6ee003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.764+0000 7f6efbac9640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 0x7f6ef402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4071b20 con 0x7f6ef4136000 2024-09-16T15:18:52.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(827..827 src has 1..827) v4 ==== 4023+0+0 (secure 0 0 0) 0x7f6ee806ac70 con 0x7f6ef4137220 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 0x7f6ef402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient: get_auth_request con 0x7f6ef4136000 auth_method 0 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6ef9865640 1 --2- 172.21.15.134:0/2077741788 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ee0038880 0x7f6ee003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6ef9865640 10 monclient: get_auth_request con 0x7f6ee0038880 auth_method 0 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 10 monclient(hunting): handle_auth_done global_id 124191 payload 824 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.765+0000 7f6efa066640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ef41228d0 con 0x7f6ef4136000 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6efa066640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 msgr2=0x7f6ef402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 0x7f6ef402b260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40a2fb0 tx=0x7f6eec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eec00cc70 con 0x7f6ef4136000 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 10 monclient: dump: 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef9865640 1 --2- 172.21.15.134:0/2077741788 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ee0038880 0x7f6ee003ad40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4005620 tx=0x7f6ee40055b0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6eec00cdd0 con 0x7f6ef4136000 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 msgr2=0x7f6ef402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 0x7f6ef402b260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40a2fb0 tx=0x7f6eec00ed60 comp rx=0 tx=0).stop 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 msgr2=0x7f6ef402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.767+0000 7f6ef27fc640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4136000 0x7f6ef402b260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.773+0000 7f6efbac9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.773+0000 7f6efbac9640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 0x7f6ef402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efbac9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4061ae0 con 0x7f6ef41363e0 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 0x7f6ef402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient: get_auth_request con 0x7f6ef41363e0 auth_method 0 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 10 monclient(hunting): handle_auth_done global_id 124200 payload 824 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.774+0000 7f6efa066640 1 -- 172.21.15.134:0/2077741788 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ef407a680 con 0x7f6ef41363e0 2024-09-16T15:18:52.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6efa066640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 msgr2=0x7f6ef402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6efa066640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 0x7f6ef402b7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6eec018000 tx=0x7f6eec0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6eec022070 con 0x7f6ef41363e0 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 10 monclient: dump: 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.787 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9198 (secure 0 0 0) 0x7f6eec013680 con 0x7f6ef41363e0 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 msgr2=0x7f6ef402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.776+0000 7f6ef27fc640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 0x7f6ef402b7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6eec018000 tx=0x7f6eec0046f0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6ef27fc640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 msgr2=0x7f6ef402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6ef27fc640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef41363e0 0x7f6ef402b7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:52.789 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:52.790 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:52.791 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:52.792 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:52.793 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.794 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.795 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.796 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:52.797 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.798 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:52.799 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.800 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.801 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:52.802 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:52.803 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:52.804 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.806 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:52.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:52.808 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:52.809 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.809 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:52.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:52.813 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ee0038880 msgr2=0x7f6ee003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 --2- 172.21.15.134:0/2077741788 >> v2:172.21.15.162:6800/980547263 conn(0x7f6ee0038880 0x7f6ee003ad40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4005620 tx=0x7f6ee40055b0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 10 monclient: shutdown 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 msgr2=0x7f6ef41201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efa066640 1 -- 172.21.15.134:0/2077741788 reap_dead start 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ee802a080 tx=0x7f6ee80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 shutdown_connections 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 --2- 172.21.15.134:0/2077741788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef4137220 0x7f6ef41201a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 >> 172.21.15.134:0/2077741788 conn(0x7f6ef40a6110 msgr2=0x7f6ef413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 shutdown_connections 2024-09-16T15:18:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.777+0000 7f6efbac9640 1 -- 172.21.15.134:0/2077741788 wait complete. 2024-09-16T15:18:52.816 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:52.816 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:18:52.816 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3689004403}) 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=481760120}) 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.956 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1546635013}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2108891655}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2789557959}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4114617091}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1105912430}) 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.957 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2582447378}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.949+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3691816242}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1459264366}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3394389345}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2264835176}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.958 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2179483492}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=38341507}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=927062879}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1893319413}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=964519177}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.950+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=242795300}) 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.959 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1185087716}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2958656076}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2330239260}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1622829811}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.960 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2384278524}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1970090164}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3049395518}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=898023506}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.951+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1847868404}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4131170598}) 2024-09-16T15:18:52.961 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3952186275}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1003012303}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2729887344}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.952+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1587521855}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1442793968}) 2024-09-16T15:18:52.962 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=670834552}) 2024-09-16T15:18:52.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715561554}) 2024-09-16T15:18:52.963 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.953+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1013635110}) 2024-09-16T15:18:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.954+0000 7fca68a96640 10 monclient: get_monmap_and_config 2024-09-16T15:18:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.954+0000 7fca68a96640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.954+0000 7fca68a96640 10 monclient: monmap: 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:52.955491+0000 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:52.955491+0000 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 Processor -- start 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 -- start start 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient: init 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient: _add_conns ranks=[7,2,10,9,6,1,0,3,4,5,8] 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca6409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-i con 0x7fca64136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-b con 0x7fca64137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:52.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 0x7fca64136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-f con 0x7fca64135dc0 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6409abc0 con 0x7fca64135dc0 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6409ba50 con 0x7fca64137220 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fca640a53c0 con 0x7fca64136e40 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca68a96640 10 monclient(hunting): authenticate will time out at 1869.544845s 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca6259c640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca6409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47300/0 (socket says 172.21.15.134:47300) 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 1 -- 172.21.15.134:0/4051589247 learned_addr learned my addr 172.21.15.134:0/4051589247 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 10 monclient(hunting): get_auth_request con 0x7fca64137220 auth_method 0 2024-09-16T15:18:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca61d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): get_auth_request con 0x7fca64136e40 auth_method 0 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.955+0000 7fca62d9d640 1 --2- 172.21.15.134:0/4051589247 >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 0x7fca64136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): get_auth_request con 0x7fca64135dc0 auth_method 0 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient(hunting): handle_auth_done global_id 114464 payload 274 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 -- 172.21.15.134:0/4051589247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 msgr2=0x7fca6409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca6409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 -- 172.21.15.134:0/4051589247 >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 msgr2=0x7fca64136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 --2- 172.21.15.134:0/4051589247 >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 0x7fca64136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 monclient: found mon.noname-b 2024-09-16T15:18:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca61d9b640 1 -- 172.21.15.134:0/4051589247 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca64073250 con 0x7fca64137220 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca6259c640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca6409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.956+0000 7fca62d9d640 1 --2- 172.21.15.134:0/4051589247 >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 0x7fca64136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.958+0000 7fca61d9b640 1 -- 172.21.15.134:0/4051589247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 msgr2=0x7fca64137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.958+0000 7fca61d9b640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca580098a0 tx=0x7fca5802a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39c472f639f383 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 1 -- 172.21.15.134:0/4051589247 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca5802adf0 con 0x7fca64137220 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: dump: 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 5 monclient: authenticate success, global_id 114464 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 1 -- 172.21.15.134:0/4051589247 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca580307b0 con 0x7fca64137220 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 4 set_mon_vals no callback set 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 1 -- 172.21.15.134:0/4051589247 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca58030d90 con 0x7fca64137220 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca6159a640 10 monclient: dump: 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.979 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 10 monclient: shutdown 2024-09-16T15:18:52.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 -- 172.21.15.134:0/4051589247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 msgr2=0x7fca64137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fca580098a0 tx=0x7fca5802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 -- 172.21.15.134:0/4051589247 shutdown_connections 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 --2- 172.21.15.134:0/4051589247 >> v2:172.21.15.162:3300/0 conn(0x7fca64135dc0 0x7fca64136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64137220 0x7fca64137600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 --2- 172.21.15.134:0/4051589247 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca6409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 -- 172.21.15.134:0/4051589247 >> 172.21.15.134:0/4051589247 conn(0x7fca640a6110 msgr2=0x7fca640a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 -- 172.21.15.134:0/4051589247 shutdown_connections 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.959+0000 7fca68a96640 1 -- 172.21.15.134:0/4051589247 wait complete. 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.960+0000 7fca68a96640 10 monclient: build_initial_monmap 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.960+0000 7fca68a96640 10 monclient: monmap: 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:52.981 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.960+0000 7fca68a96640 1 Processor -- start 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 -- start start 2024-09-16T15:18:52.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient: init 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient: _add_conns ranks=[8,7,3,4,1,2,6,0,5] 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 0x7fca641cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-e con 0x7fca64135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca64134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-i con 0x7fca64136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca64137220 0x7fca64135470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): picked mon.noname-g con 0x7fca64137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca64125080 con 0x7fca64137220 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fca64123550 con 0x7fca64136e40 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fca641363b0 con 0x7fca64135dc0 2024-09-16T15:18:52.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca68a96640 10 monclient(hunting): authenticate will time out at 1869.550546s 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 0x7fca641cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 0x7fca641cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41782/0 (socket says 172.21.15.134:41782) 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 -- 172.21.15.134:0/2297181556 learned_addr learned my addr 172.21.15.134:0/2297181556 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): get_auth_request con 0x7fca64135dc0 auth_method 0 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca62d9d640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca64137220 0x7fca64135470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient(hunting): handle_auth_done global_id 84905 payload 824 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca61d9b640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca64134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 msgr2=0x7fca64134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fca64136e40 0x7fca64134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca64137220 msgr2=0x7fca64135470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca64137220 0x7fca64135470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 monclient: found mon.noname-e 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.961+0000 7fca6259c640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c009a70 con 0x7fca64135dc0 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca6259c640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 msgr2=0x7fca641cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca6259c640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 0x7fca641cb220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca5400ebe0 tx=0x7fca5400c520 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca5400eee0 con 0x7fca64135dc0 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:52.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: dump: 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.986 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400ce40 con 0x7fca64135dc0 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca54013d50 con 0x7fca64135dc0 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 5 monclient: authenticate success, global_id 84905 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca53fff640 4 set_mon_vals no callback set 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca52ffd640 10 monclient: dump: 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 10 monclient: _renew_subs 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca54009400 con 0x7fca64135dc0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 10 monclient: _renew_subs 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.964+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca580094d0 con 0x7fca64135dc0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca68a96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca68a96640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 0x7fca641220f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca64071b20 con 0x7fca64125530 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fca54013470 con 0x7fca64135dc0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 0x7fca641220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient: get_auth_request con 0x7fca64125530 auth_method 0 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:52.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca52ffd640 1 --2- 172.21.15.134:0/2297181556 >> v2:172.21.15.162:6800/980547263 conn(0x7fca440388b0 0x7fca4403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(827..827 src has 1..827) v4 ==== 4023+0+0 (secure 0 0 0) 0x7fca5404a640 con 0x7fca64135dc0 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 10 monclient(hunting): handle_auth_done global_id 114473 payload 824 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca61d9b640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca64123d70 con 0x7fca64125530 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca62d9d640 1 --2- 172.21.15.134:0/2297181556 >> v2:172.21.15.162:6800/980547263 conn(0x7fca440388b0 0x7fca4403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.965+0000 7fca62d9d640 10 monclient: get_auth_request con 0x7fca440388b0 auth_method 0 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca61d9b640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 msgr2=0x7fca641220f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca61d9b640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 0x7fca641220f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6406b550 tx=0x7fca580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca58004460 con 0x7fca64125530 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca52ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.967+0000 7fca52ffd640 10 monclient: dump: 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:52.991 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca62d9d640 1 --2- 172.21.15.134:0/2297181556 >> v2:172.21.15.162:6800/980547263 conn(0x7fca440388b0 0x7fca4403ad70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca641240b0 tx=0x7fca5c00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fca58030920 con 0x7fca64125530 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 msgr2=0x7fca641220f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 0x7fca641220f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6406b550 tx=0x7fca580046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 msgr2=0x7fca641220f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:52.992 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.968+0000 7fca52ffd640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca64125530 0x7fca641220f0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6406b550 tx=0x7fca580046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=148857003}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3005187165}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1774497396}) 2024-09-16T15:18:52.996 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1864129728}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342736658}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4175022253}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1470609840}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4002804636}) 2024-09-16T15:18:52.997 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.972+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3704739082}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=780636996}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4124128943}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=401045379}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1127837363}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.998 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1045820184}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2468475487}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3092782696}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1732086932}) 2024-09-16T15:18:52.999 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.974+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1371745790}) 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca68a96640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca68a96640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 0x7fca6402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca68a96640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca64061ae0 con 0x7fca6402b260 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 0x7fca6402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient: get_auth_request con 0x7fca6402b260 auth_method 0 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.001 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1167421200}) 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=964007468}) 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.002 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2637308537}) 2024-09-16T15:18:53.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=36388141}) 2024-09-16T15:18:53.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 10 monclient(hunting): handle_auth_done global_id 114482 payload 824 2024-09-16T15:18:53.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.003 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.975+0000 7fca61d9b640 1 -- 172.21.15.134:0/2297181556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fca6407a680 con 0x7fca6402b260 2024-09-16T15:18:53.006 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3206094195}) 2024-09-16T15:18:53.006 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4021909537}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3231999578}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3840641618}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.975+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2849368721}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2242325121}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3709092629}) 2024-09-16T15:18:53.007 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3813517765}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=904522411}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2463706236}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.976+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2918911000}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.008 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=623789535}) 2024-09-16T15:18:53.009 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.009 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3597487940}) 2024-09-16T15:18:53.009 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.977+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2215698149}) 2024-09-16T15:18:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.977+0000 7fca61d9b640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 msgr2=0x7fca6402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.977+0000 7fca61d9b640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 0x7fca6402b670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca580097c0 tx=0x7fca58008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.009 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca5804a070 con 0x7fca6402b260 2024-09-16T15:18:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:53.010 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1549878952}) 2024-09-16T15:18:53.011 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 10 monclient: dump: 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.011 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.012 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3237391752}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1564343651}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2755696627}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3186606144}) 2024-09-16T15:18:53.014 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=572554045}) 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=241746943}) 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2619282135}) 2024-09-16T15:18:53.015 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.978+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1830062260}) 2024-09-16T15:18:53.015 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7fca58003e60 con 0x7fca6402b260 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 msgr2=0x7fca6402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 0x7fca6402b670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca580097c0 tx=0x7fca58008030 comp rx=0 tx=0).stop 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 msgr2=0x7fca6402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.978+0000 7fca52ffd640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca6402b260 0x7fca6402b670 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca580097c0 tx=0x7fca58008030 comp rx=0 tx=0).stop 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.016 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.017 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.018 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.019 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.020 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.021 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.022 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.023 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.025 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.026 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.027 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.028 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.029 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.030 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.031 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.032 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.033 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.034 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.035 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.036 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.037 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.038 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 >> v2:172.21.15.162:6800/980547263 conn(0x7fca440388b0 msgr2=0x7fca4403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 --2- 172.21.15.134:0/2297181556 >> v2:172.21.15.162:6800/980547263 conn(0x7fca440388b0 0x7fca4403ad70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca641240b0 tx=0x7fca5c00b040 comp rx=0 tx=0).stop 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 10 monclient: shutdown 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 msgr2=0x7fca641cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.039 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 --2- 172.21.15.134:0/2297181556 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fca64135dc0 0x7fca641cb220 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fca5400ebe0 tx=0x7fca5400c520 comp rx=0 tx=0).stop 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca62d9d640 1 -- 172.21.15.134:0/2297181556 reap_dead start 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca62d9d640 1 -- 172.21.15.134:0/2297181556 reap_dead start 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 shutdown_connections 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 >> 172.21.15.134:0/2297181556 conn(0x7fca640a6110 msgr2=0x7fca6409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 shutdown_connections 2024-09-16T15:18:53.040 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:52.979+0000 7fca68a96640 1 -- 172.21.15.134:0/2297181556 wait complete. 2024-09-16T15:18:53.048 INFO:tasks.rados.rados.0.smithi162.stdout:3830: done (0 left) 2024-09-16T15:18:53.048 INFO:tasks.rados.rados.0.smithi162.stdout:3831: rollback oid 24 current snap is 404 2024-09-16T15:18:53.048 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 24 to 400 2024-09-16T15:18:53.048 INFO:tasks.rados.rados.0.smithi162.stdout:3832: snap_create 2024-09-16T15:18:53.048 INFO:tasks.rados.rados.0.smithi162.stdout:3831: finishing rollback tid 1 to smithi16234619-24 2024-09-16T15:18:53.049 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 24 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=916039580}) 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2950440301}) 2024-09-16T15:18:53.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.979+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=775871045}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2166143739}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2509577726}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=104272136}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.054 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=132614145}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3456461498}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.980+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1274850899}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3309590984}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2754197818}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1167129194}) 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1160947091}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3220725757}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2327343154}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1863102394}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=655825262}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.981+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=209151398}) 2024-09-16T15:18:53.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.982+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.982+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.982+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3016031940}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.982+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3083672317}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=968720936}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=994048580}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=487076838}) 2024-09-16T15:18:53.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1133570307}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1572890352}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1402363700}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.983+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2815769201}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1029587685}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.058 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=582627661}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2929105734}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4113916885}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3939346531}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4030342324}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.059 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1294709913}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4147605415}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3254711321}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4134026111}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1387819959}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2864421474}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.060 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3747268448}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1706600977}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4236173719}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=852050390}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2747711307}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2028002734}) 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.061 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3087831205}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2773264633}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1890407989}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1339510996}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1307317932}) 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.062 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3060589157}) 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2639344651}) 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2468748784}) 2024-09-16T15:18:53.063 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.987+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1825670376}) 2024-09-16T15:18:53.064 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.064 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:18:53.064 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.999+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.999+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:52.999+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2690327557}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=267688422}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3340197571}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2773369276}) 2024-09-16T15:18:53.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3261677130}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3865362067}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2927342633}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2863959958}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.000+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1190036514}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2869198597}) 2024-09-16T15:18:53.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=994657563}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1011482266}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=661429532}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=390181504}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1844046820}) 2024-09-16T15:18:53.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2125997340}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1581700254}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.001+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1671629503}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=462745883}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3584949552}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1350359123}) 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=438416670}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=764042579}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3812286198}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=890472041}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.002+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3232799078}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.003+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3516808770}) 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3936934486}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1470137543}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2930329382}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3088159458}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4191349701}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1082339656}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2036070176}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3989254914}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.004+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3094583156}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=461202667}) 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.101 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1744748833}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1259177245}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=886023602}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=914761079}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2235135084}) 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1838860354}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=355479580}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.005+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4109187766}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3300438408}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3019577371}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=159989874}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3309196800}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2884163883}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3692568000}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4044576941}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422907349}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.006+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1885765168}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1771343760}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3528810140}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=11777143}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2482478079}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3817891090}) 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.105 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=726501552}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.007+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.008+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1653611371}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.008+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.008+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=750179505}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.008+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699368358}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1521190821}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4028957652}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.106 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=443408448}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2063207374}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=593263425}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1617796345}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3193854438}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2908632692}) 2024-09-16T15:18:53.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.017+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=817935803}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1685079857}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1484356933}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3753141250}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=332435742}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=424120105}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1529012793}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2203090683}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2825097223}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.019+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=306260958}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1650329456}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1922324944}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=636367093}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2683090435}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3030508877}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1296313339}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=563655603}) 2024-09-16T15:18:53.110 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2145882425}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.020+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3364678085}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3743744576}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1228775749}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=563419751}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1183031218}) 2024-09-16T15:18:53.111 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2244710311}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2052307810}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.021+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2252679653}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.022+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.022+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=738176298}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.022+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2669505434}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3492929865}) 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.112 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=683920137}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1898872993}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2530138154}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1921417783}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1714343618}) 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1429816493}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4158673055}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.023+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2779676869}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=345224101}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=418692922}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1999912291}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.114 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=15475682}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1336793195}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1681698990}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3219529602}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2942492134}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.024+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3740626356}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=954472708}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=805373537}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=731700567}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=613388582}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=585158981}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=15290001}) 2024-09-16T15:18:53.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4049822980}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3680527318}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.040+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2180052881}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2536972198}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3215927623}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1650058279}) 2024-09-16T15:18:53.117 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2147008497}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2163148801}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.045+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=285912067}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4115233895}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2808724663}) 2024-09-16T15:18:53.118 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.046+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2366506571}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2977041482}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=618603004}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=570897111}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2162637335}) 2024-09-16T15:18:53.119 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1749767970}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2694958366}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3438137979}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2777312460}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.051+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3483392220}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=389342214}) 2024-09-16T15:18:53.120 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3059282513}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3392225014}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.073+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1457493951}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=840840756}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1421823381}) 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.121 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.122 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2231022361}) 2024-09-16T15:18:53.122 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.122 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2211779843}) 2024-09-16T15:18:53.122 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.074+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3673795734}) 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.188+0000 7f75344d0640 10 monclient: get_monmap_and_config 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.188+0000 7f75344d0640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.188+0000 7f75344d0640 10 monclient: monmap: 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.190 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:53.189469+0000 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:53.189469+0000 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:53.191 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 Processor -- start 2024-09-16T15:18:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 -- start start 2024-09-16T15:18:53.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient: init 2024-09-16T15:18:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient: _add_conns ranks=[10,1,4,9,5,8,6,7,0,3,2] 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f752c136e40 0x7f752c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-f con 0x7f752c136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f752c137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 0x7f752c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-c con 0x7f752c13c370 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c073250 con 0x7f752c137220 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c0a56e0 con 0x7f752c136e40 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f752c09abc0 con 0x7f752c13c370 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f75344d0640 10 monclient(hunting): authenticate will time out at 1869.778866s 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f7532a6d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 0x7f752c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.189+0000 7f753226c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f752c136e40 0x7f752c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7531a6b640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f752c137220 0x7f752c137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:57000/0 (socket says 172.21.15.134:57000) 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7531a6b640 1 -- 172.21.15.134:0/768959856 learned_addr learned my addr 172.21.15.134:0/768959856 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): get_auth_request con 0x7f752c136e40 auth_method 0 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): get_auth_request con 0x7f752c13c370 auth_method 0 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient(hunting): handle_auth_done global_id 124195 payload 274 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 1 -- 172.21.15.134:0/768959856 >> v2:172.21.15.162:3300/0 conn(0x7f752c136e40 msgr2=0x7f752c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 1 --2- 172.21.15.134:0/768959856 >> v2:172.21.15.162:3300/0 conn(0x7f752c136e40 0x7f752c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 1 -- 172.21.15.134:0/768959856 >> v1:172.21.15.162:6789/0 conn(0x7f752c137220 legacy=0x7f752c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 1 monclient: found mon.noname-c 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:53.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.190+0000 7f7532a6d640 1 -- 172.21.15.134:0/768959856 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c09ba50 con 0x7f752c13c370 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f7532a6d640 1 -- 172.21.15.134:0/768959856 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 msgr2=0x7f752c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f7532a6d640 1 --2- 172.21.15.134:0/768959856 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 0x7f752c13e760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7524009d90 tx=0x7f752402a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=e6c1e5406c6004fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 1 -- 172.21.15.134:0/768959856 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7524007bc0 con 0x7f752c13c370 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: dump: 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.202 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f75344d0640 5 monclient: authenticate success, global_id 124195 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f75344d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 1 -- 172.21.15.134:0/768959856 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752402d050 con 0x7f752c13c370 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f75344d0640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f75344d0640 4 set_mon_vals no callback set 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 1 -- 172.21.15.134:0/768959856 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7524007d40 con 0x7f752c13c370 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.193+0000 7f753126a640 10 monclient: dump: 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.204 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 10 monclient: shutdown 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 -- 172.21.15.134:0/768959856 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 msgr2=0x7f752c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 --2- 172.21.15.134:0/768959856 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 0x7f752c13e760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7524009d90 tx=0x7f752402a4e0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 -- 172.21.15.134:0/768959856 shutdown_connections 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 --2- 172.21.15.134:0/768959856 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c13c370 0x7f752c13e760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 --2- 172.21.15.134:0/768959856 >> v2:172.21.15.162:3300/0 conn(0x7f752c136e40 0x7f752c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 -- 172.21.15.134:0/768959856 >> 172.21.15.134:0/768959856 conn(0x7f752c0a6110 msgr2=0x7f752c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 -- 172.21.15.134:0/768959856 shutdown_connections 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 1 -- 172.21.15.134:0/768959856 wait complete. 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.194+0000 7f75344d0640 10 monclient: monmap: 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:53.206 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 Processor -- start 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 -- start start 2024-09-16T15:18:53.207 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient: init 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient: _add_conns ranks=[4,6,0,8,2,3,7,1,5] 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 0x7f752c0a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-c con 0x7f752c136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f752c137220 0x7f752c135e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-d con 0x7f752c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f752c13c370 0x7f752c136390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): picked mon.noname-a con 0x7f752c13c370 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0a75a0 con 0x7f752c13c370 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f752c121f60 con 0x7f752c136e40 2024-09-16T15:18:53.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f752c122240 con 0x7f752c137220 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f75344d0640 10 monclient(hunting): authenticate will time out at 1869.784531s 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 0x7f752c0a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 0x7f752c0a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38758/0 (socket says 172.21.15.134:38758) 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 -- 172.21.15.134:0/508343121 learned_addr learned my addr 172.21.15.134:0/508343121 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f752c13c370 0x7f752c136390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): get_auth_request con 0x7f752c136e40 auth_method 0 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f752c137220 0x7f752c135e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): get_auth_request con 0x7f752c13c370 auth_method 0 2024-09-16T15:18:53.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): get_auth_request con 0x7f752c137220 auth_method 0 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7532a6d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient(hunting): handle_auth_done global_id 124204 payload 824 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f752c137220 msgr2=0x7f752c135e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f752c137220 0x7f752c135e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f752c13c370 msgr2=0x7f752c136390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f752c13c370 0x7f752c136390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 monclient: found mon.noname-c 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.195+0000 7f753226c640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009a70 con 0x7f752c136e40 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f753226c640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 msgr2=0x7f752c0a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f753226c640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 0x7f752c0a2880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f752800e890 tx=0x7f752800ed50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7528013470 con 0x7f752c136e40 2024-09-16T15:18:53.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: dump: 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.212 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 5 monclient: authenticate success, global_id 124204 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7528013610 con 0x7f752c136e40 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7528013bf0 con 0x7f752c136e40 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 10 monclient: _renew_subs 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c122520 con 0x7f752c136e40 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.213 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7530a69640 4 set_mon_vals no callback set 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f7522ffd640 10 monclient: dump: 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.214 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 10 monclient: _renew_subs 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.198+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1c1940 con 0x7f752c136e40 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f75344d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f75344d0640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 0x7f752c061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f752c071b20 con 0x7f752c0610b0 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7528013d50 con 0x7f752c136e40 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 0x7f752c061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient: get_auth_request con 0x7f752c0610b0 auth_method 0 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7522ffd640 1 --2- 172.21.15.134:0/508343121 >> v2:172.21.15.162:6800/980547263 conn(0x7f7510038880 0x7f751003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(828..828 src has 1..828) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f752804a0e0 con 0x7f752c136e40 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 10 monclient(hunting): handle_auth_done global_id 124213 payload 824 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7531a6b640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f752c1220d0 con 0x7f752c0610b0 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7532a6d640 1 --2- 172.21.15.134:0/508343121 >> v2:172.21.15.162:6800/980547263 conn(0x7f7510038880 0x7f751003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.199+0000 7f7532a6d640 10 monclient: get_auth_request con 0x7f7510038880 auth_method 0 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7531a6b640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 msgr2=0x7f752c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7531a6b640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 0x7f752c061490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f751c00ea70 tx=0x7f751c00ef30 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f751c00cc50 con 0x7f752c0610b0 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7522ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7522ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:53.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.201+0000 7f7522ffd640 10 monclient: dump: 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.219 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f751c00cdb0 con 0x7f752c0610b0 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 msgr2=0x7f752c061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 0x7f752c061490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f751c00ea70 tx=0x7f751c00ef30 comp rx=0 tx=0).stop 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 msgr2=0x7f752c061490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7522ffd640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c0610b0 0x7f752c061490 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f751c00ea70 tx=0x7f751c00ef30 comp rx=0 tx=0).stop 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.202+0000 7f7532a6d640 1 --2- 172.21.15.134:0/508343121 >> v2:172.21.15.162:6800/980547263 conn(0x7f7510038880 0x7f751003ad40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7524031000 tx=0x7f752402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f75344d0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f75344d0640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 0x7f752c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f75344d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f752c061ae0 con 0x7f752c02b260 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 0x7f752c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient: get_auth_request con 0x7f752c02b260 auth_method 0 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 10 monclient(hunting): handle_auth_done global_id 124222 payload 824 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.208+0000 7f7531a6b640 1 -- 172.21.15.134:0/508343121 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f752c07a680 con 0x7f752c02b260 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7531a6b640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 msgr2=0x7f752c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7531a6b640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 0x7f752c02b670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f751c017000 tx=0x7f751c004940 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f751c015cc0 con 0x7f752c02b260 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7522ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7522ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.210+0000 7f7522ffd640 10 monclient: dump: 2024-09-16T15:18:53.221 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.222 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f751c015850 con 0x7f752c02b260 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 msgr2=0x7f752c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 0x7f752c02b670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f751c017000 tx=0x7f751c004940 comp rx=0 tx=0).stop 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 msgr2=0x7f752c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7522ffd640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c02b260 0x7f752c02b670 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f751c017000 tx=0x7f751c004940 comp rx=0 tx=0).stop 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.223 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.224 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.225 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.226 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.227 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.228 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.229 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.230 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.231 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.232 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.233 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.234 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.235 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.236 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.237 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.238 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.239 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.240 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.241 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.242 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.243 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 >> v2:172.21.15.162:6800/980547263 conn(0x7f7510038880 msgr2=0x7f751003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 --2- 172.21.15.134:0/508343121 >> v2:172.21.15.162:6800/980547263 conn(0x7f7510038880 0x7f751003ad40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7524031000 tx=0x7f752402f000 comp rx=0 tx=0).stop 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 10 monclient: shutdown 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 msgr2=0x7f752c0a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 --2- 172.21.15.134:0/508343121 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f752c136e40 0x7f752c0a2880 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f752800e890 tx=0x7f752800ed50 comp rx=0 tx=0).stop 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7532a6d640 1 -- 172.21.15.134:0/508343121 reap_dead start 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f7532a6d640 1 -- 172.21.15.134:0/508343121 reap_dead start 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 shutdown_connections 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 >> 172.21.15.134:0/508343121 conn(0x7f752c0a6110 msgr2=0x7f752c13b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 shutdown_connections 2024-09-16T15:18:53.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.211+0000 7f75344d0640 1 -- 172.21.15.134:0/508343121 wait complete. 2024-09-16T15:18:53.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:18:53.246 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:18:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.343+0000 7ff72e79c640 10 monclient: get_monmap_and_config 2024-09-16T15:18:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.343+0000 7ff72e79c640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.343+0000 7ff72e79c640 10 monclient: monmap: 2024-09-16T15:18:53.344 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:53.344062+0000 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:53.344062+0000 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.345 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:53.346 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:53.346 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.343+0000 7ff72e79c640 1 Processor -- start 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 -- start start 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient: init 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient: _add_conns ranks=[3,9,6,8,5,4,1,10,7,0,2] 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff72809aa80 0x7ff72809ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-g con 0x7ff72809aa80 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff72809cfe0 0x7ff728133ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-a con 0x7ff72809cfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 0x7ff728134560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-d con 0x7ff7281340e0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72809b3a0 con 0x7ff72809cfe0 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72809c0e0 con 0x7ff72809aa80 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff72809c3c0 con 0x7ff7281340e0 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72e79c640 10 monclient(hunting): authenticate will time out at 1869.933522s 2024-09-16T15:18:53.349 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff72809cfe0 0x7ff728133ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 0x7ff728134560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff72809cfe0 0x7ff728133ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33546/0 (socket says 172.21.15.134:33546) 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 -- 172.21.15.134:0/235127471 learned_addr learned my addr 172.21.15.134:0/235127471 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): get_auth_request con 0x7ff7281340e0 auth_method 0 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): get_auth_request con 0x7ff72809cfe0 auth_method 0 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff72809aa80 0x7ff72809ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): get_auth_request con 0x7ff72809aa80 auth_method 0 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient(hunting): handle_auth_done global_id 124218 payload 274 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 -- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 msgr2=0x7ff728134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 0x7ff728134560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 -- 172.21.15.134:0/235127471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff72809aa80 msgr2=0x7ff72809ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff72809aa80 0x7ff72809ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 monclient: found mon.noname-a 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff71ffff640 1 -- 172.21.15.134:0/235127471 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff728073250 con 0x7ff72809cfe0 2024-09-16T15:18:53.352 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.344+0000 7ff72cd39640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 0x7ff728134560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:53.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff71ffff640 1 -- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72809cfe0 msgr2=0x7ff728133ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff71ffff640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72809cfe0 0x7ff728133ba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f75aee1909d1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 1 -- 172.21.15.134:0/235127471 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff71002ae00 con 0x7ff72809cfe0 2024-09-16T15:18:53.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.358 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: dump: 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.359 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 1 -- 172.21.15.134:0/235127471 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100307b0 con 0x7ff72809cfe0 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff72e79c640 5 monclient: authenticate success, global_id 124218 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff72e79c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff72e79c640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 1 -- 172.21.15.134:0/235127471 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff710030d90 con 0x7ff72809cfe0 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff72e79c640 4 set_mon_vals no callback set 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.347+0000 7ff7277fe640 10 monclient: dump: 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.361 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 10 monclient: shutdown 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 -- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72809cfe0 msgr2=0x7ff728133ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72809cfe0 0x7ff728133ba0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 -- 172.21.15.134:0/235127471 shutdown_connections 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff7281340e0 0x7ff728134560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff72809cfe0 0x7ff728133ba0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 --2- 172.21.15.134:0/235127471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff72809aa80 0x7ff72809ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 -- 172.21.15.134:0/235127471 >> 172.21.15.134:0/235127471 conn(0x7ff7280a2e20 msgr2=0x7ff7280a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 -- 172.21.15.134:0/235127471 shutdown_connections 2024-09-16T15:18:53.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.348+0000 7ff72e79c640 1 -- 172.21.15.134:0/235127471 wait complete. 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient: monmap: 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.363 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 1 Processor -- start 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 1 -- start start 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient: init 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient: _add_conns ranks=[6,4,8,3,0,2,5,7,1] 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 0x7ff72806e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-d con 0x7ff72809aa80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff72809cfe0 0x7ff72806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.349+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-c con 0x7ff72809cfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff7281340e0 0x7ff72806ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): picked mon.noname-e con 0x7ff7281340e0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:53.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff728070960 con 0x7ff72809cfe0 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff728066840 con 0x7ff72809aa80 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff72806f5f0 con 0x7ff7281340e0 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 0x7ff72806e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72e79c640 10 monclient(hunting): authenticate will time out at 1869.939057s 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 0x7ff72806e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40440/0 (socket says 172.21.15.134:40440) 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 -- 172.21.15.134:0/3610801421 learned_addr learned my addr 172.21.15.134:0/3610801421 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): get_auth_request con 0x7ff72809aa80 auth_method 0 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff72809cfe0 0x7ff72806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff7281340e0 0x7ff72806ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): get_auth_request con 0x7ff72809cfe0 auth_method 0 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): get_auth_request con 0x7ff7281340e0 auth_method 0 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient(hunting): handle_auth_done global_id 94605 payload 824 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff7281340e0 msgr2=0x7ff72806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff7281340e0 0x7ff72806ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff72809cfe0 msgr2=0x7ff72806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff72809cfe0 0x7ff72806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 monclient: found mon.noname-d 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff727fff640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72000bc80 con 0x7ff72809aa80 2024-09-16T15:18:53.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.350+0000 7ff71ffff640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff72809cfe0 0x7ff72806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff727fff640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 msgr2=0x7ff72806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff727fff640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 0x7ff72806e280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9b0 tx=0x7ff71800edb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff718009670 con 0x7ff72809aa80 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: dump: 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.368 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718013470 con 0x7ff72809aa80 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 5 monclient: authenticate success, global_id 94605 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 10 monclient: _renew_subs 2024-09-16T15:18:53.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7100094d0 con 0x7ff72809aa80 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff718013a50 con 0x7ff72809aa80 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff7257fa640 10 monclient: dump: 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.370 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff726ffd640 4 set_mon_vals no callback set 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 10 monclient: _renew_subs 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.352+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728099390 con 0x7ff72809aa80 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff718013bb0 con 0x7ff72809aa80 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff7257fa640 1 --2- 172.21.15.134:0/3610801421 >> v2:172.21.15.162:6800/980547263 conn(0x7ff708038880 0x7ff70803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff72e79c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(828..828 src has 1..828) v4 ==== 3931+0+0 (secure 0 0 0) 0x7ff71804a360 con 0x7ff72809aa80 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff72e79c640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 0x7ff728097630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.353+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff72806b5d0 con 0x7ff728097250 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 0x7ff728097630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff71ffff640 1 --2- 172.21.15.134:0/3610801421 >> v2:172.21.15.162:6800/980547263 conn(0x7ff708038880 0x7ff70803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient: get_auth_request con 0x7ff728097250 auth_method 0 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff71ffff640 10 monclient: get_auth_request con 0x7ff708038880 auth_method 0 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 10 monclient(hunting): handle_auth_done global_id 94614 payload 824 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.354+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff728061ae0 con 0x7ff728097250 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 msgr2=0x7ff728097630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff72cd39640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 0x7ff728097630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720013920 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff71ffff640 1 --2- 172.21.15.134:0/3610801421 >> v2:172.21.15.162:6800/980547263 conn(0x7ff708038880 0x7ff70803ad40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff7280a3bb0 tx=0x7ff71002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff720010040 con 0x7ff728097250 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 10 monclient: dump: 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.373 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff720011040 con 0x7ff728097250 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 msgr2=0x7ff728097630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 0x7ff728097630 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720013920 comp rx=0 tx=0).stop 2024-09-16T15:18:53.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 msgr2=0x7ff728097630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.356+0000 7ff7257fa640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097250 0x7ff728097630 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720013920 comp rx=0 tx=0).stop 2024-09-16T15:18:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.362+0000 7ff72e79c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.362+0000 7ff72e79c640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 0x7ff7281ba260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.362+0000 7ff72e79c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.362+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff72807a680 con 0x7ff728097b70 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 0x7ff7281ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient: get_auth_request con 0x7ff728097b70 auth_method 0 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 10 monclient(hunting): handle_auth_done global_id 94623 payload 824 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.363+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff72809cb30 con 0x7ff728097b70 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 msgr2=0x7ff7281ba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff72cd39640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 0x7ff7281ba260 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff720024070 con 0x7ff728097b70 2024-09-16T15:18:53.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 10 monclient: dump: 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.378 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7ff720009e40 con 0x7ff728097b70 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 msgr2=0x7ff7281ba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 0x7ff7281ba260 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 msgr2=0x7ff7281ba260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.365+0000 7ff7257fa640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff728097b70 0x7ff7281ba260 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff720013460 tx=0x7ff720004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.379 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.380 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 5, 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.381 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.382 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.383 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.384 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.385 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.386 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.387 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.389 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.390 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.391 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.392 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.393 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.394 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.395 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.396 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.397 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.398 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.399 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.400 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.401 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.401 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.401 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.401 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.401 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 >> v2:172.21.15.162:6800/980547263 conn(0x7ff708038880 msgr2=0x7ff70803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 --2- 172.21.15.134:0/3610801421 >> v2:172.21.15.162:6800/980547263 conn(0x7ff708038880 0x7ff70803ad40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff7280a3bb0 tx=0x7ff71002e000 comp rx=0 tx=0).stop 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 10 monclient: shutdown 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 msgr2=0x7ff72806e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 --2- 172.21.15.134:0/3610801421 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff72809aa80 0x7ff72806e280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff71800e9b0 tx=0x7ff71800edb0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 reap_dead start 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72cd39640 1 -- 172.21.15.134:0/3610801421 reap_dead start 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 shutdown_connections 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 >> 172.21.15.134:0/3610801421 conn(0x7ff7280a2e20 msgr2=0x7ff7280a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 shutdown_connections 2024-09-16T15:18:53.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.366+0000 7ff72e79c640 1 -- 172.21.15.134:0/3610801421 wait complete. 2024-09-16T15:18:53.403 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.403 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:18:53.403 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:18:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.500+0000 7f7be3bdb640 10 monclient: get_monmap_and_config 2024-09-16T15:18:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.500+0000 7f7be3bdb640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.500+0000 7f7be3bdb640 10 monclient: monmap: 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:53.501363+0000 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:53.501363+0000 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.502 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:53.503 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:53.503 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.500+0000 7f7be3bdb640 1 Processor -- start 2024-09-16T15:18:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 -- start start 2024-09-16T15:18:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient: init 2024-09-16T15:18:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient: _add_conns ranks=[3,8,2,9,1,10,6,7,0,5,4] 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bdc136e40 0x7f7bdc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-g con 0x7f7bdc136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-e con 0x7f7bdc137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-b con 0x7f7bdc135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc09abc0 con 0x7f7bdc135dc0 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc09ba50 con 0x7f7bdc136e40 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0a53c0 con 0x7f7bdc137220 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be3bdb640 10 monclient(hunting): authenticate will time out at 1870.090669s 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41804/0 (socket says 172.21.15.134:41804) 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47340/0 (socket says 172.21.15.134:47340) 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 1 -- 172.21.15.134:0/1519298011 learned_addr learned my addr 172.21.15.134:0/1519298011 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): get_auth_request con 0x7f7bdc137220 auth_method 0 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): get_auth_request con 0x7f7bdc135dc0 auth_method 0 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.501+0000 7f7be1176640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1977640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bdc136e40 0x7f7bdc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1977640 10 monclient(hunting): get_auth_request con 0x7f7bdc136e40 auth_method 0 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1977640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1977640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 10 monclient(hunting): handle_auth_done global_id 114491 payload 274 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 -- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 msgr2=0x7f7bdc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 -- 172.21.15.134:0/1519298011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bdc136e40 msgr2=0x7f7bdc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bdc136e40 0x7f7bdc09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 monclient: found mon.noname-b 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be2178640 1 -- 172.21.15.134:0/1519298011 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc073250 con 0x7f7bdc135dc0 2024-09-16T15:18:53.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.502+0000 7f7be1176640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:53.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.504+0000 7f7be2178640 1 -- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 msgr2=0x7f7bdc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.504+0000 7f7be2178640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b586b5e2682cfb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.504+0000 7f7be0975640 1 -- 172.21.15.134:0/1519298011 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd4007bc0 con 0x7f7bdc135dc0 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.504+0000 7f7be0975640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: dump: 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.516 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 5 monclient: authenticate success, global_id 114491 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 1 -- 172.21.15.134:0/1519298011 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd402d050 con 0x7f7bdc135dc0 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 4 set_mon_vals no callback set 2024-09-16T15:18:53.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 1 -- 172.21.15.134:0/1519298011 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd4007d80 con 0x7f7bdc135dc0 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be0975640 10 monclient: dump: 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 10 monclient: shutdown 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 msgr2=0x7f7bdc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd402a480 comp rx=0 tx=0).stop 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1519298011 shutdown_connections 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7bdc135dc0 0x7f7bdc136220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc137220 0x7f7bdc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1519298011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7bdc136e40 0x7f7bdc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1519298011 >> 172.21.15.134:0/1519298011 conn(0x7f7bdc0a6110 msgr2=0x7f7bdc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1519298011 shutdown_connections 2024-09-16T15:18:53.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.505+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1519298011 wait complete. 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient: monmap: 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 Processor -- start 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 -- start start 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient: init 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient: _add_conns ranks=[1,5,6,7,3,2,0,8,4] 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bdc135dc0 0x7f7bdc1cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-f con 0x7f7bdc135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7bdc136e40 0x7f7bdc134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-h con 0x7f7bdc136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): picked mon.noname-d con 0x7f7bdc137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc123660 con 0x7f7bdc135dc0 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc1cb760 con 0x7f7bdc136e40 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0a89a0 con 0x7f7bdc137220 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.506+0000 7f7be3bdb640 10 monclient(hunting): authenticate will time out at 1870.095978s 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40462/0 (socket says 172.21.15.134:40462) 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 learned_addr learned my addr 172.21.15.134:0/1241881641 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): get_auth_request con 0x7f7bdc137220 auth_method 0 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7bdc136e40 0x7f7bdc134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1977640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bdc135dc0 0x7f7bdc1cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 10 monclient(hunting): get_auth_request con 0x7f7bdc136e40 auth_method 0 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1977640 10 monclient(hunting): get_auth_request con 0x7f7bdc135dc0 auth_method 0 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1977640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1977640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient(hunting): handle_auth_done global_id 94632 payload 824 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7bdc136e40 msgr2=0x7f7bdc134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7bdc136e40 0x7f7bdc134ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bdc135dc0 msgr2=0x7f7bdc1cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7bdc135dc0 0x7f7bdc1cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 monclient: found mon.noname-d 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80094d0 con 0x7f7bdc137220 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.507+0000 7f7be1176640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f7bdc136e40 0x7f7bdc134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 msgr2=0x7f7bdc135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be2178640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd4004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd4007ea0 con 0x7f7bdc137220 2024-09-16T15:18:53.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: dump: 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.525 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd40044b0 con 0x7f7bdc137220 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 5 monclient: authenticate success, global_id 94632 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 10 monclient: _renew_subs 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd4004620 con 0x7f7bdc137220 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd4009a70 con 0x7f7bdc137220 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.526 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bce7fc640 10 monclient: dump: 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.527 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7bcffff640 4 set_mon_vals no callback set 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 10 monclient: _renew_subs 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.509+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc136310 con 0x7f7bdc137220 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be3bdb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be2178640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd4004a40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f7bd406b860 osd_map(828..828 src has 1..828) v4 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 0x7f7bdc121cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7bd4004040 con 0x7f7bdc137220 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc071b20 con 0x7f7bdc1218d0 2024-09-16T15:18:53.528 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 0x7f7bdc121cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient: get_auth_request con 0x7f7bdc1218d0 auth_method 0 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7bce7fc640 1 --2- 172.21.15.134:0/1241881641 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bc0038880 0x7f7bc003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 10 monclient(hunting): handle_auth_done global_id 84932 payload 824 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.510+0000 7f7be1977640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bdc061ae0 con 0x7f7bdc1218d0 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.511+0000 7f7be1176640 1 --2- 172.21.15.134:0/1241881641 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bc0038880 0x7f7bc003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.511+0000 7f7be1176640 10 monclient: get_auth_request con 0x7f7bc0038880 auth_method 0 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7be1977640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 msgr2=0x7f7bdc121cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7be1977640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 0x7f7bdc121cb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000f4a0 tx=0x7f7bd000f960 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd0014cf0 con 0x7f7bdc1218d0 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 10 monclient: dump: 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7bd0014e50 con 0x7f7bdc1218d0 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 msgr2=0x7f7bdc121cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 0x7f7bdc121cb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000f4a0 tx=0x7f7bd000f960 comp rx=0 tx=0).stop 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 msgr2=0x7f7bdc121cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7bce7fc640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1218d0 0x7f7bdc121cb0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000f4a0 tx=0x7f7bd000f960 comp rx=0 tx=0).stop 2024-09-16T15:18:53.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.513+0000 7f7be1176640 1 --2- 172.21.15.134:0/1241881641 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bc0038880 0x7f7bc003ad40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009c90 tx=0x7f7bd8008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be3bdb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 0x7f7bdc0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be3bdb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc1253d0 con 0x7f7bdc1221f0 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 0x7f7bdc0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient: get_auth_request con 0x7f7bdc1221f0 auth_method 0 2024-09-16T15:18:53.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.519+0000 7f7be1977640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.520+0000 7f7be1977640 10 monclient(hunting): handle_auth_done global_id 84941 payload 824 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.520+0000 7f7be1977640 1 -- 172.21.15.134:0/1241881641 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7bdc07a680 con 0x7f7bdc1221f0 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7be1977640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 msgr2=0x7f7bdc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7be1977640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 0x7f7bdc0a1a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0019000 tx=0x7f7bd00046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7bd00156e0 con 0x7f7bdc1221f0 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 10 monclient: dump: 2024-09-16T15:18:53.534 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.535 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f7bd0015840 con 0x7f7bdc1221f0 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 msgr2=0x7f7bdc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 0x7f7bdc0a1a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0019000 tx=0x7f7bd00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 msgr2=0x7f7bdc0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.522+0000 7f7bce7fc640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7bdc1221f0 0x7f7bdc0a1a20 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0019000 tx=0x7f7bd00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.536 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:18:53.537 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.538 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.539 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.540 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.541 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.542 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.543 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.544 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.545 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.546 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.547 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.548 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.549 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.550 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.551 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.552 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.553 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.554 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.555 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.556 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.557 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bc0038880 msgr2=0x7f7bc003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1241881641 >> v2:172.21.15.162:6800/980547263 conn(0x7f7bc0038880 0x7f7bc003ad40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009c90 tx=0x7f7bd8008040 comp rx=0 tx=0).stop 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 10 monclient: shutdown 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 msgr2=0x7f7bdc135400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 --2- 172.21.15.134:0/1241881641 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7bdc137220 0x7f7bdc135400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd4004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 reap_dead start 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be2178640 1 -- 172.21.15.134:0/1241881641 reap_dead start 2024-09-16T15:18:53.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7bce7fc640 1 -- 172.21.15.134:0/1241881641 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(828..828 src has 1..828) v4 ==== 3931+0+0 (unknown 0 0 0) 0x7f7bd406b860 con 0x7f7bdc137220 2024-09-16T15:18:53.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 shutdown_connections 2024-09-16T15:18:53.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 >> 172.21.15.134:0/1241881641 conn(0x7f7bdc0a6110 msgr2=0x7f7bdc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 shutdown_connections 2024-09-16T15:18:53.560 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.523+0000 7f7be3bdb640 1 -- 172.21.15.134:0/1241881641 wait complete. 2024-09-16T15:18:53.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.560 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:18:53.560 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:18:53.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.660 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1536523001}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2244020019}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=35530376}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=539704925}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4007812459}) 2024-09-16T15:18:53.661 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3434781756}) 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3571137192}) 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1499797082}) 2024-09-16T15:18:53.662 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=67531932}) 2024-09-16T15:18:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.657+0000 7fb3ba861640 10 monclient: get_monmap_and_config 2024-09-16T15:18:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.657+0000 7fb3ba861640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.658+0000 7fb3ba861640 10 monclient: monmap: 2024-09-16T15:18:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.668 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:53.658980+0000 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:53.658980+0000 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.669 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 Processor -- start 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 -- start start 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient: init 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient: _add_conns ranks=[1,6,10,7,9,5,3,8,4,2,0] 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb3b4136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-d con 0x7fb3b4137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3b413c370 0x7fb3b413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-f con 0x7fb3b413c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:18:53.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b4073250 con 0x7fb3b4136e40 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b40a56e0 con 0x7fb3b413c370 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b409abc0 con 0x7fb3b4137220 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3ba861640 10 monclient(hunting): authenticate will time out at 1870.248701s 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40474/0 (socket says 172.21.15.134:40474) 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 1 -- 172.21.15.134:0/2562542033 learned_addr learned my addr 172.21.15.134:0/2562542033 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request con 0x7fb3b4137220 auth_method 0 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 1 --2- 172.21.15.134:0/2562542033 >> v2:172.21.15.162:3300/0 conn(0x7fb3b413c370 0x7fb3b413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.671 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): get_auth_request con 0x7fb3b413c370 auth_method 0 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.659+0000 7fb3b8dfe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b8dfe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_done global_id 94641 payload 274 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 1 -- 172.21.15.134:0/2562542033 >> v2:172.21.15.162:3300/0 conn(0x7fb3b413c370 msgr2=0x7fb3b413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/2562542033 >> v2:172.21.15.162:3300/0 conn(0x7fb3b413c370 0x7fb3b413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.672 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 1 -- 172.21.15.134:0/2562542033 >> v1:172.21.15.162:6789/0 conn(0x7fb3b4136e40 legacy=0x7fb3b409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 1 monclient: found mon.noname-d 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.660+0000 7fb3b37fe640 1 -- 172.21.15.134:0/2562542033 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b409ba50 con 0x7fb3b4137220 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.662+0000 7fb3b37fe640 1 -- 172.21.15.134:0/2562542033 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 msgr2=0x7fb3b4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.662+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/2562542033 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009870 tx=0x7fb3a802a100 comp rx=0 tx=0).ready entity=mon.6 client_cookie=9ced345a3e5dde2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.662+0000 7fb3b2ffd640 1 -- 172.21.15.134:0/2562542033 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a802ae20 con 0x7fb3b4137220 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: dump: 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.673 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 1 -- 172.21.15.134:0/2562542033 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a80307b0 con 0x7fb3b4137220 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 5 monclient: authenticate success, global_id 94641 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 1 -- 172.21.15.134:0/2562542033 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a8030d90 con 0x7fb3b4137220 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 4 set_mon_vals no callback set 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3b2ffd640 10 monclient: dump: 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.675 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 10 monclient: shutdown 2024-09-16T15:18:53.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 -- 172.21.15.134:0/2562542033 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 msgr2=0x7fb3b4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 --2- 172.21.15.134:0/2562542033 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009870 tx=0x7fb3a802a100 comp rx=0 tx=0).stop 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 -- 172.21.15.134:0/2562542033 shutdown_connections 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 --2- 172.21.15.134:0/2562542033 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4137220 0x7fb3b4137600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 --2- 172.21.15.134:0/2562542033 >> v2:172.21.15.162:3300/0 conn(0x7fb3b413c370 0x7fb3b413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 -- 172.21.15.134:0/2562542033 >> 172.21.15.134:0/2562542033 conn(0x7fb3b40a6110 msgr2=0x7fb3b40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 -- 172.21.15.134:0/2562542033 shutdown_connections 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.663+0000 7fb3ba861640 1 -- 172.21.15.134:0/2562542033 wait complete. 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.664+0000 7fb3ba861640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.664+0000 7fb3ba861640 10 monclient: monmap: 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:53.677 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.664+0000 7fb3ba861640 1 Processor -- start 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.664+0000 7fb3ba861640 1 -- start start 2024-09-16T15:18:53.678 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.664+0000 7fb3ba861640 10 monclient: init 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient: _add_conns ranks=[6,5,3,7,2,4,1,8,0] 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 0x7fb3b411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-d con 0x7fb3b4136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb3b4137220 0x7fb3b40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-h con 0x7fb3b4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3b413c370 0x7fb3b4135f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): picked mon.noname-g con 0x7fb3b413c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4120d00 con 0x7fb3b413c370 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4122010 con 0x7fb3b4137220 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40a7b60 con 0x7fb3b4136e40 2024-09-16T15:18:53.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3ba861640 10 monclient(hunting): authenticate will time out at 1870.254378s 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 0x7fb3b411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 0x7fb3b411fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:40488/0 (socket says 172.21.15.134:40488) 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 -- 172.21.15.134:0/4090539649 learned_addr learned my addr 172.21.15.134:0/4090539649 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request con 0x7fb3b4136e40 auth_method 0 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient(hunting): handle_auth_done global_id 94650 payload 824 2024-09-16T15:18:53.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb3b4137220 msgr2=0x7fb3b40a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb3b4137220 0x7fb3b40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b8dfe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3b413c370 0x7fb3b4135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fb3b4137220 0x7fb3b40a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3b413c370 msgr2=0x7fb3b4135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3b413c370 0x7fb3b4135f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 monclient: found mon.noname-d 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.665+0000 7fb3b3fff640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80094d0 con 0x7fb3b4136e40 2024-09-16T15:18:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.667+0000 7fb3b3fff640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 msgr2=0x7fb3b411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.667+0000 7fb3b3fff640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 0x7fb3b411fc60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c430 tx=0x7fb3a400c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a4011c10 con 0x7fb3b4136e40 2024-09-16T15:18:53.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: dump: 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.686 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4011d90 con 0x7fb3b4136e40 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 5 monclient: authenticate success, global_id 94650 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 10 monclient: _renew_subs 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b40a75a0 con 0x7fb3b4136e40 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a4012890 con 0x7fb3b4136e40 2024-09-16T15:18:53.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b0ff9640 10 monclient: dump: 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.688 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 10 monclient: _renew_subs 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41228b0 con 0x7fb3b4136e40 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3b27fc640 4 set_mon_vals no callback set 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.668+0000 7fb3ba861640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 0x7fb3b41cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4071b20 con 0x7fb3b4136440 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb3a4012420 con 0x7fb3b4136e40 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 0x7fb3b41cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient: get_auth_request con 0x7fb3b4136440 auth_method 0 2024-09-16T15:18:53.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b0ff9640 1 --2- 172.21.15.134:0/4090539649 >> v2:172.21.15.162:6800/980547263 conn(0x7fb3a00388b0 0x7fb3a003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(828..828 src has 1..828) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fb3a404a100 con 0x7fb3b4136e40 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_done global_id 124219 payload 824 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b37fe640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3b4061ae0 con 0x7fb3b4136440 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b8dfe640 1 --2- 172.21.15.134:0/4090539649 >> v2:172.21.15.162:6800/980547263 conn(0x7fb3a00388b0 0x7fb3a003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.669+0000 7fb3b8dfe640 10 monclient: get_auth_request con 0x7fb3a00388b0 auth_method 0 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b37fe640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 msgr2=0x7fb3b41cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 0x7fb3b41cbe30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4137f00 tx=0x7fb3a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a80044a0 con 0x7fb3b4136440 2024-09-16T15:18:53.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b8dfe640 1 --2- 172.21.15.134:0/4090539649 >> v2:172.21.15.162:6800/980547263 conn(0x7fb3a00388b0 0x7fb3a003ad70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406b7d0 tx=0x7fb3ac00b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.671+0000 7fb3b0ff9640 10 monclient: dump: 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.691 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb3a8030d40 con 0x7fb3b4136440 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 msgr2=0x7fb3b41cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 0x7fb3b41cbe30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4137f00 tx=0x7fb3a80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 msgr2=0x7fb3b41cbe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.672+0000 7fb3b0ff9640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b4136440 0x7fb3b41cbe30 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4137f00 tx=0x7fb3a80046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3509712959}) 2024-09-16T15:18:53.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3598193446}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2770872157}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3274401657}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1962637308}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1979554392}) 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.673+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.674+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.674+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=866634851}) 2024-09-16T15:18:53.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.674+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.674+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3406103356}) 2024-09-16T15:18:53.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.674+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652133482}) 2024-09-16T15:18:53.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.678+0000 7fb3ba861640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.678+0000 7fb3ba861640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 0x7fb3b402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.678+0000 7fb3ba861640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.678+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4073e50 con 0x7fb3b402b260 2024-09-16T15:18:53.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 0x7fb3b402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient: get_auth_request con 0x7fb3b402b260 auth_method 0 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 10 monclient(hunting): handle_auth_done global_id 124228 payload 824 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.679+0000 7fb3b37fe640 1 -- 172.21.15.134:0/4090539649 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb3b407a680 con 0x7fb3b402b260 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b37fe640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 msgr2=0x7fb3b402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b37fe640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 0x7fb3b402b670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097c0 tx=0x7fb3a8009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb3a80328b0 con 0x7fb3b402b260 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.681+0000 7fb3b0ff9640 10 monclient: dump: 2024-09-16T15:18:53.698 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.699 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fb3a8032440 con 0x7fb3b402b260 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 msgr2=0x7fb3b402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 0x7fb3b402b670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097c0 tx=0x7fb3a8009240 comp rx=0 tx=0).stop 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 msgr2=0x7fb3b402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b0ff9640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3b402b260 0x7fb3b402b670 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097c0 tx=0x7fb3a8009240 comp rx=0 tx=0).stop 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.700 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:18:53.701 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.702 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.703 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.704 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.705 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.706 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.707 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.708 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.709 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.710 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.711 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.712 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.713 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.714 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.715 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.717 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.718 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.719 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.720 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.721 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.722 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.723 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.724 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.725 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 >> v2:172.21.15.162:6800/980547263 conn(0x7fb3a00388b0 msgr2=0x7fb3a003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 --2- 172.21.15.134:0/4090539649 >> v2:172.21.15.162:6800/980547263 conn(0x7fb3a00388b0 0x7fb3a003ad70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406b7d0 tx=0x7fb3ac00b040 comp rx=0 tx=0).stop 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 10 monclient: shutdown 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 msgr2=0x7fb3b411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 --2- 172.21.15.134:0/4090539649 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fb3b4136e40 0x7fb3b411fc60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c430 tx=0x7fb3a400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b8dfe640 1 -- 172.21.15.134:0/4090539649 reap_dead start 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3b8dfe640 1 -- 172.21.15.134:0/4090539649 reap_dead start 2024-09-16T15:18:53.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 shutdown_connections 2024-09-16T15:18:53.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 >> 172.21.15.134:0/4090539649 conn(0x7fb3b40a6110 msgr2=0x7fb3b413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 shutdown_connections 2024-09-16T15:18:53.728 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.682+0000 7fb3ba861640 1 -- 172.21.15.134:0/4090539649 wait complete. 2024-09-16T15:18:53.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:18:53.728 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2380699250}) 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1125146465}) 2024-09-16T15:18:53.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2193930551}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=336830339}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1415621109}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=833144465}) 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=452778834}) 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=336239289}) 2024-09-16T15:18:53.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318190200}) 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2284336638}) 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=818416427}) 2024-09-16T15:18:53.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1489783272}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1581540421}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=698631068}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4290964103}) 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4157173671}) 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901943332}) 2024-09-16T15:18:53.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.756+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=700241409}) 2024-09-16T15:18:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.842+0000 7fc0dbbaa640 10 monclient: get_monmap_and_config 2024-09-16T15:18:53.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.842+0000 7fc0dbbaa640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.842+0000 7fc0dbbaa640 10 monclient: monmap: 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:53.843206+0000 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:53.843206+0000 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.844 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:53.845 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.842+0000 7fc0dbbaa640 1 Processor -- start 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 -- start start 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient: init 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient: _add_conns ranks=[9,4,5,3,1,6,8,2,10,7,0] 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc0d4136e40 0x7fc0d409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-a con 0x7fc0d4136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 0x7fc0d4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.856 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-c con 0x7fc0d4137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc0d4135dc0 0x7fc0d4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-h con 0x7fc0d4135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d409abc0 con 0x7fc0d4136e40 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d409ba50 con 0x7fc0d4137220 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40a53c0 con 0x7fc0d4135dc0 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0dbbaa640 10 monclient(hunting): authenticate will time out at 1870.432512s 2024-09-16T15:18:53.857 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc0d4136e40 0x7fc0d409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 0x7fc0d4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc0d4136e40 0x7fc0d409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33568/0 (socket says 172.21.15.134:33568) 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 -- 172.21.15.134:0/2647794630 learned_addr learned my addr 172.21.15.134:0/2647794630 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): get_auth_request con 0x7fc0d4136e40 auth_method 0 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): get_auth_request con 0x7fc0d4137220 auth_method 0 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient(hunting): handle_auth_done global_id 124227 payload 274 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 -- 172.21.15.134:0/2647794630 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc0d4135dc0 msgr2=0x7fc0d4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc0d4135dc0 0x7fc0d4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 -- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 msgr2=0x7fc0d4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 0x7fc0d4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.859 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 monclient: found mon.noname-a 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9946640 1 -- 172.21.15.134:0/2647794630 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d4073250 con 0x7fc0d4136e40 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.843+0000 7fc0d9145640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 0x7fc0d4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.846+0000 7fc0d9946640 1 -- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 msgr2=0x7fc0d409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.846+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d409c150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009870 tx=0x7fc0d002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cff033a805e8fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.846+0000 7fc0d8944640 1 -- 172.21.15.134:0/2647794630 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0d002ae00 con 0x7fc0d4136e40 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.846+0000 7fc0d8944640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: dump: 2024-09-16T15:18:53.860 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.861 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 5 monclient: authenticate success, global_id 124227 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 1 -- 172.21.15.134:0/2647794630 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d00307b0 con 0x7fc0d4136e40 2024-09-16T15:18:53.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 4 set_mon_vals no callback set 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 1 -- 172.21.15.134:0/2647794630 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0d0030d90 con 0x7fc0d4136e40 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0d8944640 10 monclient: dump: 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.863 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.864 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 10 monclient: shutdown 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 msgr2=0x7fc0d409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d409c150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009870 tx=0x7fc0d002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2647794630 shutdown_connections 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc0d4135dc0 0x7fc0d4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc0d4137220 0x7fc0d4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2647794630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d409c150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2647794630 >> 172.21.15.134:0/2647794630 conn(0x7fc0d40a6110 msgr2=0x7fc0d40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2647794630 shutdown_connections 2024-09-16T15:18:53.865 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.847+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2647794630 wait complete. 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.848+0000 7fc0dbbaa640 10 monclient: build_initial_monmap 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.848+0000 7fc0dbbaa640 10 monclient: monmap: 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.848+0000 7fc0dbbaa640 1 Processor -- start 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.848+0000 7fc0dbbaa640 1 -- start start 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.848+0000 7fc0dbbaa640 10 monclient: init 2024-09-16T15:18:53.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient: _add_conns ranks=[7,0,2,8,1,3,5,4,6] 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc0d4135dc0 0x7fc0d41cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-i con 0x7fc0d4135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d41351b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-a con 0x7fc0d4136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0d4137220 0x7fc0d41356f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): picked mon.noname-b con 0x7fc0d4137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41cc410 con 0x7fc0d4136e40 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4136980 con 0x7fc0d4137220 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41363e0 con 0x7fc0d4135dc0 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0dbbaa640 10 monclient(hunting): authenticate will time out at 1870.438144s 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d41351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d41351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33578/0 (socket says 172.21.15.134:33578) 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0d4137220 0x7fc0d41356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 -- 172.21.15.134:0/2043063484 learned_addr learned my addr 172.21.15.134:0/2043063484 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): get_auth_request con 0x7fc0d4136e40 auth_method 0 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): get_auth_request con 0x7fc0d4137220 auth_method 0 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc0d4135dc0 0x7fc0d41cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0da147640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): get_auth_request con 0x7fc0d4135dc0 auth_method 0 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient(hunting): handle_auth_done global_id 124236 payload 824 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc0d4135dc0 msgr2=0x7fc0d41cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc0d4135dc0 0x7fc0d41cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0d4137220 msgr2=0x7fc0d41356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0d4137220 0x7fc0d41356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient: _finish_hunting 0 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 monclient: found mon.noname-a 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.849+0000 7fc0d9145640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009400 con 0x7fc0d4136e40 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0d9145640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 msgr2=0x7fc0d41351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0d9145640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d41351b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400e960 tx=0x7fc0c400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c400cc10 con 0x7fc0d4136e40 2024-09-16T15:18:53.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: dump: 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.873 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400cd90 con 0x7fc0d4136e40 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0dbbaa640 5 monclient: authenticate success, global_id 124236 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0dbbaa640 10 monclient: _renew_subs 2024-09-16T15:18:53.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0dbbaa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d00094d0 con 0x7fc0d4136e40 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0c4013d50 con 0x7fc0d4136e40 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0cbfff640 4 set_mon_vals no callback set 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.851+0000 7fc0ca7fc640 10 monclient: dump: 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 10 monclient: _renew_subs 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d4135c30 con 0x7fc0d4136e40 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc0c4013470 con 0x7fc0d4136e40 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 0x7fc0d402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4071b20 con 0x7fc0d41224f0 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0ca7fc640 1 --2- 172.21.15.134:0/2043063484 >> v2:172.21.15.162:6800/980547263 conn(0x7fc0b8038880 0x7fc0b803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.852+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(828..828 src has 1..828) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc0c404a4a0 con 0x7fc0d4136e40 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 0x7fc0d402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient: get_auth_request con 0x7fc0d41224f0 auth_method 0 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0da147640 1 --2- 172.21.15.134:0/2043063484 >> v2:172.21.15.162:6800/980547263 conn(0x7fc0b8038880 0x7fc0b803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0da147640 10 monclient: get_auth_request con 0x7fc0b8038880 auth_method 0 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 10 monclient(hunting): handle_auth_done global_id 84774 payload 824 2024-09-16T15:18:53.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.853+0000 7fc0d9946640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0d4061ae0 con 0x7fc0d41224f0 2024-09-16T15:18:53.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0d9946640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 msgr2=0x7fc0d402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 0x7fc0d402b260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4137f00 tx=0x7fc0d00046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0da147640 1 --2- 172.21.15.134:0/2043063484 >> v2:172.21.15.162:6800/980547263 conn(0x7fc0b8038880 0x7fc0b803ad40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009f60 tx=0x7fc0cc00d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0d0030d40 con 0x7fc0d41224f0 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0ca7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.855+0000 7fc0ca7fc640 10 monclient: dump: 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.882 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc0d0003610 con 0x7fc0d41224f0 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:53.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 msgr2=0x7fc0d402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 0x7fc0d402b260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4137f00 tx=0x7fc0d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 msgr2=0x7fc0d402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.856+0000 7fc0ca7fc640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41224f0 0x7fc0d402b260 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4137f00 tx=0x7fc0d00046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0dbbaa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 0x7fc0d402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0dbbaa640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4067960 con 0x7fc0d41228d0 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 0x7fc0d402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient: get_auth_request con 0x7fc0d41228d0 auth_method 0 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:53.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 10 monclient(hunting): handle_auth_done global_id 84783 payload 824 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.862+0000 7fc0d9946640 1 -- 172.21.15.134:0/2043063484 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc0d407a680 con 0x7fc0d41228d0 2024-09-16T15:18:53.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0d9946640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 msgr2=0x7fc0d402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:53.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0d9946640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 0x7fc0d402b7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0d002a0e0 tx=0x7fc0d0009240 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:53.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0d0003eb0 con 0x7fc0d41228d0 2024-09-16T15:18:53.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 10 monclient: dump: 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:53.888 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fc0d003cac0 con 0x7fc0d41228d0 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 msgr2=0x7fc0d402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 0x7fc0d402b7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0d002a0e0 tx=0x7fc0d0009240 comp rx=0 tx=0).stop 2024-09-16T15:18:53.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 msgr2=0x7fc0d402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0ca7fc640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d41228d0 0x7fc0d402b7a0 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0d002a0e0 tx=0x7fc0d0009240 comp rx=0 tx=0).stop 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.891 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.892 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.893 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.894 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:53.895 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.896 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.897 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.898 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.899 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.900 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.901 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.902 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.903 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.904 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.905 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.906 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:53.907 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:53.908 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.909 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:53.910 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:53.911 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:53.912 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:53.913 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.914 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:53.915 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:53.916 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:53.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 >> v2:172.21.15.162:6800/980547263 conn(0x7fc0b8038880 msgr2=0x7fc0b803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2043063484 >> v2:172.21.15.162:6800/980547263 conn(0x7fc0b8038880 0x7fc0b803ad40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009f60 tx=0x7fc0cc00d040 comp rx=0 tx=0).stop 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 10 monclient: shutdown 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 msgr2=0x7fc0d41351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 --2- 172.21.15.134:0/2043063484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0d4136e40 0x7fc0d41351b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400e960 tx=0x7fc0c400ee20 comp rx=0 tx=0).stop 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0da147640 1 -- 172.21.15.134:0/2043063484 reap_dead start 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0da147640 1 -- 172.21.15.134:0/2043063484 reap_dead start 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 shutdown_connections 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 >> 172.21.15.134:0/2043063484 conn(0x7fc0d40a6110 msgr2=0x7fc0d409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:53.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 shutdown_connections 2024-09-16T15:18:53.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:53.865+0000 7fc0dbbaa640 1 -- 172.21.15.134:0/2043063484 wait complete. 2024-09-16T15:18:53.919 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:53.919 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:18:53.919 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2529387539}) 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2669279642}) 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.932 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3659733047}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=765257745}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2272291707}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1998333594}) 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.933 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4244878209}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2624672104}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.897+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1131357125}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=164473164}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3732445990}) 2024-09-16T15:18:53.934 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2451834212}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=156881224}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3720205259}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2585521643}) 2024-09-16T15:18:53.935 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1943170349}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3118439324}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.898+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1429031412}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3164506109}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1195754858}) 2024-09-16T15:18:53.936 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1750003793}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1543701772}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3020756996}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=14102793}) 2024-09-16T15:18:53.937 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3612517513}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1881123474}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.900+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1220380341}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3932569068}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.938 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=607475201}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3955973174}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1598165871}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=210573525}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3810093780}) 2024-09-16T15:18:53.939 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1945801405}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=416326115}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.901+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2771290179}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=8360154}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3397960976}) 2024-09-16T15:18:53.940 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2742571830}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1868659537}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4290344464}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.902+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.941 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2420173327}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2260894428}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4119738485}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.903+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3672048000}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=973036204}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=978376614}) 2024-09-16T15:18:53.942 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1094140689}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=963645760}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3940399118}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.943 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1553551438}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3226726409}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1676470394}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.904+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3013515295}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3078608256}) 2024-09-16T15:18:53.944 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2500920875}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1941416212}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=691732978}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3277733739}) 2024-09-16T15:18:53.945 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1787823615}) 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1431692534}) 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=595035429}) 2024-09-16T15:18:53.946 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:53.905+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4176441320}) 2024-09-16T15:18:54.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1267132832}) 2024-09-16T15:18:54.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.051 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1890016861}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3171166470}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4268369738}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.052 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=199497578}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2003695512}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2188528999}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3307659107}) 2024-09-16T15:18:54.053 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.048+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3572457626}) 2024-09-16T15:18:54.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.055 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1371924079}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2302569106}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3831744216}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2705115398}) 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.056 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1707208736}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1275085310}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2751355688}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2493712346}) 2024-09-16T15:18:54.057 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.052+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1631732712}) 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.057+0000 7f4820b45640 10 monclient: get_monmap_and_config 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.057+0000 7f4820b45640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.057+0000 7f4820b45640 10 monclient: monmap: 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:54.058562+0000 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:54.058562+0000 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.059 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:54.060 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:54.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.061 INFO:tasks.rados.rados.0.smithi162.stdout:3831: done (1 left) 2024-09-16T15:18:54.062 INFO:tasks.rados.rados.0.smithi162.stdout:3832: done (0 left) 2024-09-16T15:18:54.062 INFO:tasks.rados.rados.0.smithi162.stdout:3833: write oid 43 current snap is 405 2024-09-16T15:18:54.062 INFO:tasks.rados.rados.0.smithi162.stdout:3833: seq_num 1300 ranges {649100=403710,1606356=501788,2633420=497852,3636005=213095} 2024-09-16T15:18:54.067 INFO:tasks.rados.rados.0.smithi162.stdout:3833: writing smithi16234619-43 from 649100 to 1052810 tid 1 2024-09-16T15:18:54.073 INFO:tasks.rados.rados.0.smithi162.stdout:3833: writing smithi16234619-43 from 1606356 to 2108144 tid 2 2024-09-16T15:18:54.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 Processor -- start 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 -- start start 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient: init 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient: _add_conns ranks=[2,3,7,6,0,5,4,1,8,10,9] 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-b con 0x7f481c130ab0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f481c1305f0 0x7f481c133af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-g con 0x7f481c1305f0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f481c134030 0x7f481c1344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-i con 0x7f481c134030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:54.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c09f600 con 0x7f481c130ab0 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0673e0 con 0x7f481c1305f0 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f481c09e740 con 0x7f481c134030 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.058+0000 7f4820b45640 10 monclient(hunting): authenticate will time out at 1870.647932s 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47366/0 (socket says 172.21.15.134:47366) 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 -- 172.21.15.134:0/2510966349 learned_addr learned my addr 172.21.15.134:0/2510966349 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): get_auth_request con 0x7f481c130ab0 auth_method 0 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f481c1305f0 0x7f481c133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f481c134030 0x7f481c1344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): get_auth_request con 0x7f481c1305f0 auth_method 0 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): get_auth_request con 0x7f481c134030 auth_method 0 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient(hunting): handle_auth_done global_id 114509 payload 274 2024-09-16T15:18:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 -- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f481c134030 msgr2=0x7f481c1344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f481c134030 0x7f481c1344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 -- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f481c1305f0 msgr2=0x7f481c133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f481c1305f0 0x7f481c133af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 monclient: found mon.noname-b 2024-09-16T15:18:54.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.059+0000 7f481a59c640 1 -- 172.21.15.134:0/2510966349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c073250 con 0x7f481c130ab0 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481a59c640 1 -- 172.21.15.134:0/2510966349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 msgr2=0x7f481c130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481a59c640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15bd41fe217a655f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 1 -- 172.21.15.134:0/2510966349 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f480c02ae60 con 0x7f481c130ab0 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: dump: 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 5 monclient: authenticate success, global_id 114509 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 1 -- 172.21.15.134:0/2510966349 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c0307b0 con 0x7f481c130ab0 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 4 set_mon_vals no callback set 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 1 -- 172.21.15.134:0/2510966349 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f480c030d90 con 0x7f481c130ab0 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f481959a640 10 monclient: dump: 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 10 monclient: shutdown 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 -- 172.21.15.134:0/2510966349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 msgr2=0x7f481c130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c02a100 comp rx=0 tx=0).stop 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 -- 172.21.15.134:0/2510966349 shutdown_connections 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f481c134030 0x7f481c1344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f481c1305f0 0x7f481c133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 --2- 172.21.15.134:0/2510966349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f481c130ab0 0x7f481c130e90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 -- 172.21.15.134:0/2510966349 >> 172.21.15.134:0/2510966349 conn(0x7f481c0a6400 msgr2=0x7f481c0a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 -- 172.21.15.134:0/2510966349 shutdown_connections 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.062+0000 7f4820b45640 1 -- 172.21.15.134:0/2510966349 wait complete. 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.063+0000 7f4820b45640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.063+0000 7f4820b45640 10 monclient: monmap: 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.086 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.063+0000 7f4820b45640 1 Processor -- start 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 -- start start 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient: init 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient: _add_conns ranks=[8,4,5,3,2,6,0,1,7] 2024-09-16T15:18:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-e con 0x7f481c1305f0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f481c130ab0 0x7f481c071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-c con 0x7f481c130ab0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c134030 0x7f481c071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): picked mon.noname-h con 0x7f481c134030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1329a0 con 0x7f481c130ab0 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f481c072060 con 0x7f481c134030 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f481c132f30 con 0x7f481c1305f0 2024-09-16T15:18:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4820b45640 10 monclient(hunting): authenticate will time out at 1870.653435s 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41846/0 (socket says 172.21.15.134:41846) 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 -- 172.21.15.134:0/1242441147 learned_addr learned my addr 172.21.15.134:0/1242441147 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f481c130ab0 0x7f481c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): get_auth_request con 0x7f481c1305f0 auth_method 0 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): get_auth_request con 0x7f481c130ab0 auth_method 0 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c134030 0x7f481c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): get_auth_request con 0x7f481c134030 auth_method 0 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient(hunting): handle_auth_done global_id 84950 payload 824 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c134030 msgr2=0x7f481c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c134030 0x7f481c071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f481c130ab0 msgr2=0x7f481c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f481c130ab0 0x7f481c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 monclient: found mon.noname-e 2024-09-16T15:18:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.064+0000 7f481a59c640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c0094d0 con 0x7f481c1305f0 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f481a59c640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 msgr2=0x7f481c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f481a59c640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f480c003610 con 0x7f481c1305f0 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: dump: 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c003770 con 0x7f481c1305f0 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 5 monclient: authenticate success, global_id 84950 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 10 monclient: _renew_subs 2024-09-16T15:18:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f480c003d50 con 0x7f481c1305f0 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c133270 con 0x7f481c1305f0 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4818d99640 4 set_mon_vals no callback set 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f480affd640 10 monclient: dump: 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 10 monclient: _renew_subs 2024-09-16T15:18:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.067+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c09a500 con 0x7f481c1305f0 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f480c039070 con 0x7f481c1305f0 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4820b45640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4820b45640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 0x7f481c0a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f481c073880 con 0x7f481c0a2d10 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f480affd640 1 --2- 172.21.15.134:0/1242441147 >> v2:172.21.15.162:6800/980547263 conn(0x7f47fc038880 0x7f47fc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f480c06a800 con 0x7f481c1305f0 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 0x7f481c0a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient: get_auth_request con 0x7f481c0a2d10 auth_method 0 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f481ad9d640 1 --2- 172.21.15.134:0/1242441147 >> v2:172.21.15.162:6800/980547263 conn(0x7f47fc038880 0x7f47fc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f481ad9d640 10 monclient: get_auth_request con 0x7f47fc038880 auth_method 0 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 10 monclient(hunting): handle_auth_done global_id 45365 payload 824 2024-09-16T15:18:54.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.068+0000 7f4819d9b640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f481c061ae0 con 0x7f481c0a2d10 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.070+0000 7f4819d9b640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 msgr2=0x7f481c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.070+0000 7f4819d9b640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 0x7f481c0a30f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f481c131a10 tx=0x7f4810011d40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.070+0000 7f481ad9d640 1 --2- 172.21.15.134:0/1242441147 >> v2:172.21.15.162:6800/980547263 conn(0x7f47fc038880 0x7f47fc03ad40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f481c06df60 tx=0x7f481400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48100044a0 con 0x7f481c0a2d10 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 10 monclient: dump: 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.099 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f481000fe20 con 0x7f481c0a2d10 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 msgr2=0x7f481c0a30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 0x7f481c0a30f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f481c131a10 tx=0x7f4810011d40 comp rx=0 tx=0).stop 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 msgr2=0x7f481c0a30f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.071+0000 7f480affd640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a2d10 0x7f481c0a30f0 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f481c131a10 tx=0x7f4810011d40 comp rx=0 tx=0).stop 2024-09-16T15:18:54.103 INFO:tasks.rados.rados.0.smithi162.stdout:3833: writing smithi16234619-43 from 2633420 to 3131272 tid 3 2024-09-16T15:18:54.103 INFO:tasks.rados.rados.0.smithi162.stdout:3833: writing smithi16234619-43 from 3636005 to 3849100 tid 4 2024-09-16T15:18:54.103 INFO:tasks.rados.rados.0.smithi162.stdout:3834: copy_from oid 11 from oid 17 current snap is 405 2024-09-16T15:18:54.103 INFO:tasks.rados.rados.0.smithi162.stdout:3835: copy_from oid 14 from oid 45 current snap is 405 2024-09-16T15:18:54.103 INFO:tasks.rados.rados.0.smithi162.stdout:3836: snap_create 2024-09-16T15:18:54.104 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 1 to smithi16234619-43 2024-09-16T15:18:54.104 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 2 to smithi16234619-43 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4820b45640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4820b45640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 0x7f481c1d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4820b45640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f481c06b5d0 con 0x7f481c0a3630 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 0x7f481c1d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient: get_auth_request con 0x7f481c0a3630 auth_method 0 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 10 monclient(hunting): handle_auth_done global_id 45374 payload 824 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.077+0000 7f4819d9b640 1 -- 172.21.15.134:0/1242441147 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f481c07a680 con 0x7f481c0a3630 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f4819d9b640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 msgr2=0x7f481c1d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f4819d9b640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 0x7f481c1d59c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f481001b000 tx=0x7f48100046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f481002b070 con 0x7f481c0a3630 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 10 monclient: dump: 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f4810023040 con 0x7f481c0a3630 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 msgr2=0x7f481c1d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 0x7f481c1d59c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f481001b000 tx=0x7f48100046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 msgr2=0x7f481c1d59c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.080+0000 7f480affd640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f481c0a3630 0x7f481c1d59c0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f481001b000 tx=0x7f48100046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:54.109 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:18:54.110 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.111 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:54.133 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:54.134 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:54.135 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:54.136 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:54.138 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 3 to smithi16234619-43 2024-09-16T15:18:54.138 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 4 to smithi16234619-43 2024-09-16T15:18:54.138 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 5 to smithi16234619-43 2024-09-16T15:18:54.138 INFO:tasks.rados.rados.0.smithi162.stdout:3833: finishing write tid 6 to smithi16234619-43 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: oid 43 updating version 0 to 2418 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: oid 43 updating version 2418 to 2419 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: oid 43 version 2419 is already newer than 2416 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: oid 43 version 2419 is already newer than 2415 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: oid 43 version 2419 is already newer than 2417 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 43 v 2419 (ObjNum 1300 snap 405 seq_num 1300) dirty exists 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3833: left oid 43 (ObjNum 1300 snap 405 seq_num 1300) 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3834: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3835: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3834: finishing copy_from to smithi16234619-11 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 1488 (ObjNum 1197 snap 365 seq_num 1197) dirty exists 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:3835: finishing copy_from to smithi16234619-14 2024-09-16T15:18:54.139 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 2458 (ObjNum 1265 snap 392 seq_num 1265) dirty exists 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 >> v2:172.21.15.162:6800/980547263 conn(0x7f47fc038880 msgr2=0x7f47fc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 --2- 172.21.15.134:0/1242441147 >> v2:172.21.15.162:6800/980547263 conn(0x7f47fc038880 0x7f47fc03ad40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f481c06df60 tx=0x7f481400b040 comp rx=0 tx=0).stop 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 10 monclient: shutdown 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 msgr2=0x7f481c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f481ad9d640 1 -- 172.21.15.134:0/1242441147 reap_dead start 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 shutdown_connections 2024-09-16T15:18:54.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 --2- 172.21.15.134:0/1242441147 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f481c1305f0 0x7f481c070bc0 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 >> 172.21.15.134:0/1242441147 conn(0x7f481c0a6400 msgr2=0x7f481c069e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:54.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 shutdown_connections 2024-09-16T15:18:54.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.081+0000 7f4820b45640 1 -- 172.21.15.134:0/1242441147 wait complete. 2024-09-16T15:18:54.141 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:54.141 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:18:54.141 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:18:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 10 monclient: get_monmap_and_config 2024-09-16T15:18:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 10 monclient: monmap: 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:54.239025+0000 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:54.239025+0000 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:54.241 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 1 Processor -- start 2024-09-16T15:18:54.245 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 1 -- start start 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.238+0000 7fdd8dfae640 10 monclient: init 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient: _add_conns ranks=[5,7,8,4,2,10,0,3,6,1,9] 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdd88133f10 0x7fdd881342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-h con 0x7fdd88133f10 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd88134830 0x7fdd88134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-i con 0x7fdd88134830 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-e con 0x7fdd88071bd0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.246 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880a3d50 con 0x7fdd88133f10 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806f630 con 0x7fdd88134830 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880a1e10 con 0x7fdd88071bd0 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd8dfae640 10 monclient(hunting): authenticate will time out at 1870.828392s 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41860/0 (socket says 172.21.15.134:41860) 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 -- 172.21.15.134:0/1859269715 learned_addr learned my addr 172.21.15.134:0/1859269715 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): get_auth_request con 0x7fdd88071bd0 auth_method 0 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdd88133f10 0x7fdd881342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd88134830 0x7fdd88134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): get_auth_request con 0x7fdd88133f10 auth_method 0 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): get_auth_request con 0x7fdd88134830 auth_method 0 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient(hunting): handle_auth_done global_id 84959 payload 274 2024-09-16T15:18:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 -- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd88134830 msgr2=0x7fdd88134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd88134830 0x7fdd88134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 -- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdd88133f10 msgr2=0x7fdd881342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdd88133f10 0x7fdd881342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 monclient: found mon.noname-e 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.239+0000 7fdd87fff640 1 -- 172.21.15.134:0/1859269715 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88073250 con 0x7fdd88071bd0 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd87fff640 1 -- 172.21.15.134:0/1859269715 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 msgr2=0x7fdd881351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e80 tx=0x7fdd8002a530 comp rx=0 tx=0).ready entity=mon.8 client_cookie=96583f9a9e11de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 1 -- 172.21.15.134:0/1859269715 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd80007a70 con 0x7fdd88071bd0 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: dump: 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 1 -- 172.21.15.134:0/1859269715 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8002d050 con 0x7fdd88071bd0 2024-09-16T15:18:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 1 -- 172.21.15.134:0/1859269715 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd80007c30 con 0x7fdd88071bd0 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd8dfae640 5 monclient: authenticate success, global_id 84959 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd8dfae640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd8dfae640 4 set_mon_vals no callback set 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.242+0000 7fdd867fc640 10 monclient: dump: 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 10 monclient: shutdown 2024-09-16T15:18:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1859269715 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 msgr2=0x7fdd881351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e80 tx=0x7fdd8002a530 comp rx=0 tx=0).stop 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1859269715 shutdown_connections 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88071bd0 0x7fdd881351f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd88134830 0x7fdd88134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1859269715 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fdd88133f10 0x7fdd881342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1859269715 >> 172.21.15.134:0/1859269715 conn(0x7fdd880a7410 msgr2=0x7fdd880a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1859269715 shutdown_connections 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.243+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1859269715 wait complete. 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.244+0000 7fdd8dfae640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.244+0000 7fdd8dfae640 10 monclient: monmap: 2024-09-16T15:18:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.244+0000 7fdd8dfae640 1 Processor -- start 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.244+0000 7fdd8dfae640 1 -- start start 2024-09-16T15:18:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.244+0000 7fdd8dfae640 10 monclient: init 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient: _add_conns ranks=[1,8,0,2,3,5,6,7,4] 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88071bd0 0x7fdd8806af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-f con 0x7fdd88071bd0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-e con 0x7fdd88133f10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88134830 0x7fdd8806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): picked mon.noname-a con 0x7fdd88134830 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:18:54.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806c9f0 con 0x7fdd88134830 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88122410 con 0x7fdd88071bd0 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880a6220 con 0x7fdd88133f10 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd8dfae640 10 monclient(hunting): authenticate will time out at 1870.834147s 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41876/0 (socket says 172.21.15.134:41876) 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88134830 0x7fdd8806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 -- 172.21.15.134:0/1537518342 learned_addr learned my addr 172.21.15.134:0/1537518342 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): get_auth_request con 0x7fdd88133f10 auth_method 0 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): get_auth_request con 0x7fdd88134830 auth_method 0 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88071bd0 0x7fdd8806af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): get_auth_request con 0x7fdd88071bd0 auth_method 0 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient(hunting): handle_auth_done global_id 84968 payload 824 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88071bd0 msgr2=0x7fdd8806af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88071bd0 0x7fdd8806af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88134830 msgr2=0x7fdd8806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88134830 0x7fdd8806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 monclient: found mon.noname-e 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd86ffd640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009b30 con 0x7fdd88133f10 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.245+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88134830 0x7fdd8806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd86ffd640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 msgr2=0x7fdd8806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd86ffd640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e870 tx=0x7fdd7c00ed30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd7c0094f0 con 0x7fdd88133f10 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 10 monclient: dump: 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c013470 con 0x7fdd88133f10 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd8dfae640 5 monclient: authenticate success, global_id 84968 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd8dfae640 10 monclient: _renew_subs 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.247+0000 7fdd8dfae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8809e060 con 0x7fdd88133f10 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd7c013a50 con 0x7fdd88133f10 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 10 monclient: dump: 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.272 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.273 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 10 monclient: _renew_subs 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd85ffb640 4 set_mon_vals no callback set 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd88122130 con 0x7fdd88133f10 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 0x7fdd88132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806a3f0 con 0x7fdd881328d0 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.248+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdd7c013c30 con 0x7fdd88133f10 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd6ffff640 1 --2- 172.21.15.134:0/1537518342 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd74038880 0x7fdd7403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fdd7c04a230 con 0x7fdd88133f10 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 0x7fdd88132cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient: get_auth_request con 0x7fdd881328d0 auth_method 0 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1537518342 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd74038880 0x7fdd7403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd87fff640 10 monclient: get_auth_request con 0x7fdd74038880 auth_method 0 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 10 monclient(hunting): handle_auth_done global_id 114433 payload 824 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.249+0000 7fdd877fe640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd88061ae0 con 0x7fdd881328d0 2024-09-16T15:18:54.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.251+0000 7fdd877fe640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 msgr2=0x7fdd88132cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.251+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 0x7fdd88132cb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdd780118e0 tx=0x7fdd78011da0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.251+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd7800fdc0 con 0x7fdd881328d0 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd87fff640 1 --2- 172.21.15.134:0/1537518342 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd74038880 0x7fdd7403ad40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd80033000 tx=0x7fdd80009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 10 monclient: dump: 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.276 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdd7800f950 con 0x7fdd881328d0 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 msgr2=0x7fdd88132cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 0x7fdd88132cb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdd780118e0 tx=0x7fdd78011da0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 msgr2=0x7fdd88132cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.252+0000 7fdd6ffff640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881328d0 0x7fdd88132cb0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdd780118e0 tx=0x7fdd78011da0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd8dfae640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 0x7fdd8809b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd8dfae640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8807a680 con 0x7fdd881331f0 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 0x7fdd8809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient: get_auth_request con 0x7fdd881331f0 auth_method 0 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.258+0000 7fdd877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.259+0000 7fdd877fe640 10 monclient(hunting): handle_auth_done global_id 114442 payload 824 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.259+0000 7fdd877fe640 1 -- 172.21.15.134:0/1537518342 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd88070970 con 0x7fdd881331f0 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd877fe640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 msgr2=0x7fdd8809b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd877fe640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 0x7fdd8809b410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd780118e0 tx=0x7fdd780046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd7802c070 con 0x7fdd881331f0 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 10 monclient: dump: 2024-09-16T15:18:54.279 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.280 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fdd78024040 con 0x7fdd881331f0 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 msgr2=0x7fdd8809b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 0x7fdd8809b410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd780118e0 tx=0x7fdd780046c0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 msgr2=0x7fdd8809b410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:18:54.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.261+0000 7fdd6ffff640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fdd881331f0 0x7fdd8809b410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 156, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:18:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 6, 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:18:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:18:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:18:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:18:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.290 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.291 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.292 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:18:54.293 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:18:54.294 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.295 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.296 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.297 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:18:54.301 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:18:54.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:18:54.304 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:18:54.305 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:18:54.306 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd74038880 msgr2=0x7fdd7403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1537518342 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd74038880 0x7fdd7403ad40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd80033000 tx=0x7fdd80009000 comp rx=0 tx=0).stop 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 10 monclient: shutdown 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 msgr2=0x7fdd8806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00e870 tx=0x7fdd7c00ed30 comp rx=0 tx=0).stop 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd87fff640 1 -- 172.21.15.134:0/1537518342 reap_dead start 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 shutdown_connections 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 --2- 172.21.15.134:0/1537518342 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fdd88133f10 0x7fdd8806b4a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 >> 172.21.15.134:0/1537518342 conn(0x7fdd880a7410 msgr2=0x7fdd881b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:54.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 shutdown_connections 2024-09-16T15:18:54.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.262+0000 7fdd8dfae640 1 -- 172.21.15.134:0/1537518342 wait complete. 2024-09-16T15:18:54.309 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:18:54.309 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i'] 2024-09-16T15:18:54.309 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-16T15:18:54.309 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-16T15:18:54.309 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:18:54.310 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-16T15:18:54.310 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:18:54.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2881675216}) 2024-09-16T15:18:54.321 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3455128355}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2505040937}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1299264236}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1157682061}) 2024-09-16T15:18:54.322 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4184492554}) 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3484085589}) 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1011084608}) 2024-09-16T15:18:54.323 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:18:54.318+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=583581692}) 2024-09-16T15:18:54.324 INFO:tasks.ceph.mon.i:Stopped 2024-09-16T15:18:54.324 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-16T15:18:54.324 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:54.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 10 monclient: get_monmap_and_config 2024-09-16T15:18:54.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 10 monclient: monmap: 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:54.419122+0000 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:54.419122+0000 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.420 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:54.421 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 1 Processor -- start 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 1 -- start start 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.418+0000 7fa5a6fe9640 10 monclient: init 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient: _add_conns ranks=[3,5,8,1,7,0,10,2,9,6,4] 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5a0136030 0x7fa5a0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-g con 0x7fa5a0136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 0x7fa5a009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-h con 0x7fa5a009bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-e con 0x7fa5a0136c60 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:54.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a009b600 con 0x7fa5a0136030 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00a5740 con 0x7fa5a009bfe0 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00a52c0 con 0x7fa5a0136c60 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a6fe9640 10 monclient(hunting): authenticate will time out at 1871.008422s 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41882/0 (socket says 172.21.15.134:41882) 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 -- 172.21.15.134:0/816958852 learned_addr learned my addr 172.21.15.134:0/816958852 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): get_auth_request con 0x7fa5a0136c60 auth_method 0 2024-09-16T15:18:54.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5a0136030 0x7fa5a0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 0x7fa5a009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): get_auth_request con 0x7fa5a0136030 auth_method 0 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 10 monclient(hunting): get_auth_request con 0x7fa5a009bfe0 auth_method 0 2024-09-16T15:18:54.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient(hunting): handle_auth_done global_id 84977 payload 274 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 -- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 msgr2=0x7fa5a009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 0x7fa5a009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 -- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5a0136030 msgr2=0x7fa5a0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5a0136030 0x7fa5a0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 monclient: found mon.noname-e 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa5a5586640 1 -- 172.21.15.134:0/816958852 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0073250 con 0x7fa5a0136c60 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.419+0000 7fa597fff640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 0x7fa5a009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a5586640 1 -- 172.21.15.134:0/816958852 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 msgr2=0x7fa5a01370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a5586640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa598009e80 tx=0x7fa59802a530 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c24ac30c6b6e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 1 -- 172.21.15.134:0/816958852 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa598007a70 con 0x7fa5a0136c60 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: dump: 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.436 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 1 -- 172.21.15.134:0/816958852 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59802d050 con 0x7fa5a0136c60 2024-09-16T15:18:54.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a6fe9640 5 monclient: authenticate success, global_id 84977 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a6fe9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a6fe9640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 1 -- 172.21.15.134:0/816958852 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa598007c30 con 0x7fa5a0136c60 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5a6fe9640 4 set_mon_vals no callback set 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.422+0000 7fa5977fe640 10 monclient: dump: 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.438 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.439 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 10 monclient: shutdown 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/816958852 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 msgr2=0x7fa5a01370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa598009e80 tx=0x7fa59802a530 comp rx=0 tx=0).stop 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/816958852 shutdown_connections 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136c60 0x7fa5a01370c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fa5a009bfe0 0x7fa5a009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/816958852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5a0136030 0x7fa5a0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/816958852 >> 172.21.15.134:0/816958852 conn(0x7fa5a00a9ea0 msgr2=0x7fa5a00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/816958852 shutdown_connections 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/816958852 wait complete. 2024-09-16T15:18:54.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.423+0000 7fa5a6fe9640 10 monclient: build_initial_monmap 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient: monmap: 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:54.441 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 Processor -- start 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 -- start start 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient: init 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient: _add_conns ranks=[1,8,6,3,5,2,0,4,7] 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a009bfe0 0x7fa5a0134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-f con 0x7fa5a009bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-e con 0x7fa5a0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): picked mon.noname-d con 0x7fa5a0136c60 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0122300 con 0x7fa5a009bfe0 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a009fda0 con 0x7fa5a0136c60 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a01cc1e0 con 0x7fa5a0136030 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:54.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.424+0000 7fa5a6fe9640 10 monclient(hunting): authenticate will time out at 1871.013999s 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:41884/0 (socket says 172.21.15.134:41884) 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 1 -- 172.21.15.134:0/15182614 learned_addr learned my addr 172.21.15.134:0/15182614 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): get_auth_request con 0x7fa5a0136030 auth_method 0 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): get_auth_request con 0x7fa5a0136c60 auth_method 0 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a009bfe0 0x7fa5a0134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): get_auth_request con 0x7fa5a009bfe0 auth_method 0 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:54.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a4d85640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient(hunting): handle_auth_done global_id 94659 payload 824 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 -- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 msgr2=0x7fa5a0135340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 -- 172.21.15.134:0/15182614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a009bfe0 msgr2=0x7fa5a0134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a009bfe0 0x7fa5a0134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient: _finish_hunting 0 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 monclient: found mon.noname-d 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa5a5586640 1 -- 172.21.15.134:0/15182614 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598009b30 con 0x7fa5a0136c60 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.425+0000 7fa597fff640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:18:54.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a5586640 1 -- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 msgr2=0x7fa5a011fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a5586640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00a5400 tx=0x7fa598004a40 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa598004690 con 0x7fa5a0136c60 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: dump: 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.447 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: _finish_auth 0 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: _check_auth_tickets 2024-09-16T15:18:54.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa598004830 con 0x7fa5a0136c60 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 5 monclient: authenticate success, global_id 94659 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 10 monclient: _renew_subs 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a01cb760 con 0x7fa5a0136c60 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa59803b410 con 0x7fa5a0136c60 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5957fa640 10 monclient: dump: 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:54.449 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:54.450 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 10 monclient: _renew_subs 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a00a7e10 con 0x7fa5a0136c60 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.427+0000 7fa596ffd640 4 set_mon_vals no callback set 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5a5586640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00a5400 tx=0x7fa598004a40 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fa598038070 mgrmap(e 5) v1 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa598038070 con 0x7fa5a0136c60 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5a6fe9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5a6fe9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5a6fe9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a0122440 con 0x7fa5a0136c60 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.428+0000 7fa5957fa640 1 --2- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 0x7fa58c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.429+0000 7fa5a4d85640 1 --2- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 0x7fa58c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.429+0000 7fa5a4d85640 10 monclient: get_auth_request con 0x7fa58c038880 auth_method 0 2024-09-16T15:18:54.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.431+0000 7fa5a4d85640 1 --2- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 0x7fa58c03ad40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ac0 tx=0x7fa59c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:54.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:54.433+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fa59806a410 con 0x7fa5a0136c60 2024-09-16T15:18:55.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.256+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa5a0122440 con 0x7fa5a0136c60 2024-09-16T15:18:55.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.256+0000 7fa5957fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:55.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.256+0000 7fa5957fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:55.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.483+0000 7fa5a6fe9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:55.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.483+0000 7fa5a6fe9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:55.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.483+0000 7fa5a6fe9640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:18:55.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.483+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa5a0122440 con 0x7fa5a0136c60 2024-09-16T15:18:55.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.484+0000 7fa5957fa640 1 -- 172.21.15.134:0/15182614 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7fa59803fc90 con 0x7fa5a0136c60 2024-09-16T15:18:55.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.484+0000 7fa5957fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:55.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.484+0000 7fa5957fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:55.486 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 msgr2=0x7fa58c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 0x7fa58c03ad40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ac0 tx=0x7fa59c009190 comp rx=0 tx=0).stop 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 10 monclient: shutdown 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 msgr2=0x7fa5a011fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00a5400 tx=0x7fa598004a40 comp rx=0 tx=0).stop 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 shutdown_connections 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> v2:172.21.15.162:6800/980547263 conn(0x7fa58c038880 0x7fa58c03ad40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:55.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa5a0136c60 0x7fa5a011fa80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:55.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa5a0136030 0x7fa5a0135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:55.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 --2- 172.21.15.134:0/15182614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5a009bfe0 0x7fa5a0134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:55.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 >> 172.21.15.134:0/15182614 conn(0x7fa5a00a9ea0 msgr2=0x7fa5a00a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:55.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 shutdown_connections 2024-09-16T15:18:55.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:55.486+0000 7fa5a6fe9640 1 -- 172.21.15.134:0/15182614 wait complete. 2024-09-16T15:18:58.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.605+0000 7fee61145640 10 monclient: get_monmap_and_config 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.605+0000 7fee61145640 10 monclient: build_initial_monmap 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.605+0000 7fee61145640 10 monclient: monmap: 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:18:58.606789+0000 2024-09-16T15:18:58.607 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:18:58.606789+0000 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:18:58.608 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:18:58.609 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 1 Processor -- start 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 1 -- start start 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 10 monclient: init 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 10 monclient: _add_conns ranks=[0,3,6,5,8,7,10,4,9,2,1] 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.606+0000 7fee61145640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fee5c136030 addr v1:172.21.15.134:6789/0 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 0x7fee5c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): picked mon.noname-g con 0x7fee5c09bfe0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:18:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 0x7fee5c13e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): picked mon.noname-d con 0x7fee5c13bf80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c073250 con 0x7fee5c136030 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c09b600 con 0x7fee5c09bfe0 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0a5740 con 0x7fee5c13bf80 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee61145640 10 monclient(hunting): authenticate will time out at 1875.196057s 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5ad9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fee5c136030 0x7fee5c136410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:44384/0 (socket says 172.21.15.134:44384) 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 0x7fee5c13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1238491885 learned_addr learned my addr 172.21.15.134:0/1238491885 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): get_auth_request con 0x7fee5c13bf80 auth_method 0 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 0x7fee5c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 10 monclient(hunting): get_auth_request con 0x7fee5c09bfe0 auth_method 0 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient(hunting): handle_auth_done global_id 94668 payload 274 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 -- 172.21.15.134:0/1238491885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 msgr2=0x7fee5c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 0x7fee5c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 -- 172.21.15.134:0/1238491885 >> v1:172.21.15.134:6789/0 conn(0x7fee5c136030 legacy=0x7fee5c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient: _finish_hunting 0 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 monclient: found mon.noname-d 2024-09-16T15:18:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5b59e640 1 -- 172.21.15.134:0/1238491885 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0a52c0 con 0x7fee5c13bf80 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.607+0000 7fee5259c640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 0x7fee5c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5b59e640 1 -- 172.21.15.134:0/1238491885 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 msgr2=0x7fee5c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5b59e640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 0x7fee5c13e370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fee54009dc0 tx=0x7fee5402a4d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=41ce4fd3987d97bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 1 -- 172.21.15.134:0/1238491885 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee54007bc0 con 0x7fee5c13bf80 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: dump: 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: _finish_auth 0 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: _check_auth_tickets 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee61145640 5 monclient: authenticate success, global_id 94668 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 1 -- 172.21.15.134:0/1238491885 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5402d050 con 0x7fee5c13bf80 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee61145640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee61145640 10 monclient: get_monmap_and_config success 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee61145640 4 set_mon_vals no callback set 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 1 -- 172.21.15.134:0/1238491885 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee54007d40 con 0x7fee5c13bf80 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.610+0000 7fee5a59c640 10 monclient: dump: 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 10 monclient: shutdown 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 -- 172.21.15.134:0/1238491885 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 msgr2=0x7fee5c13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 0x7fee5c13e370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fee54009dc0 tx=0x7fee5402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 -- 172.21.15.134:0/1238491885 shutdown_connections 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fee5c13bf80 0x7fee5c13e370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 --2- 172.21.15.134:0/1238491885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee5c09bfe0 0x7fee5c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 -- 172.21.15.134:0/1238491885 >> 172.21.15.134:0/1238491885 conn(0x7fee5c0a9ea0 msgr2=0x7fee5c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 -- 172.21.15.134:0/1238491885 shutdown_connections 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 1 -- 172.21.15.134:0/1238491885 wait complete. 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 10 monclient: build_initial_monmap 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.611+0000 7fee61145640 10 monclient: monmap: 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:18:58.621 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 Processor -- start 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 -- start start 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient: init 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient: _reopen_session rank -1 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient: _add_conns ranks=[4,5,1,2,6,3,7,8,0] 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.625 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): picked mon.noname-c con 0x7fee5c09bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fee5c136030 0x7fee5c133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): picked mon.noname-h con 0x7fee5c136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5c13bf80 0x7fee5c1389c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): picked mon.noname-f con 0x7fee5c13bf80 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0a3020 con 0x7fee5c13bf80 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1cc170 con 0x7fee5c09bfe0 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): start opening mon connection 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0a48c0 con 0x7fee5c136030 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): _renew_subs 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.612+0000 7fee61145640 10 monclient(hunting): authenticate will time out at 1875.201937s 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.626 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:40530/0 (socket says 172.21.15.134:40530) 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1439025912 learned_addr learned my addr 172.21.15.134:0/1439025912 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): get_auth_request con 0x7fee5c09bfe0 auth_method 0 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5b59e640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5c13bf80 0x7fee5c1389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5259c640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fee5c136030 0x7fee5c133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5b59e640 10 monclient(hunting): get_auth_request con 0x7fee5c13bf80 auth_method 0 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5b59e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5b59e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5b59e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:58.627 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5259c640 10 monclient(hunting): get_auth_request con 0x7fee5c136030 auth_method 0 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5259c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5259c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5259c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient(hunting): handle_auth_done global_id 124258 payload 824 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fee5c136030 msgr2=0x7fee5c133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fee5c136030 0x7fee5c133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5c13bf80 msgr2=0x7fee5c1389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5c13bf80 0x7fee5c1389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 monclient: found mon.noname-c 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:18:58.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.613+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1439025912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009a70 con 0x7fee5c09bfe0 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee5ad9d640 1 -- 172.21.15.134:0/1439025912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 msgr2=0x7fee5c1cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee5ad9d640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00c460 tx=0x7fee4c00c920 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee4c005cf0 con 0x7fee5c09bfe0 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: dump: 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:58.629 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: _finish_auth 0 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: _check_auth_tickets 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c005e90 con 0x7fee5c09bfe0 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 5 monclient: authenticate success, global_id 124258 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 10 monclient: _renew_subs 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:58.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c0a82d0 con 0x7fee5c09bfe0 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee4c0074e0 con 0x7fee5c09bfe0 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee53fff640 10 monclient: dump: 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:18:58.631 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee59d9b640 4 set_mon_vals no callback set 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 10 monclient: _renew_subs 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:58.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.615+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c120750 con 0x7fee5c09bfe0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fee4c007640 con 0x7fee5c09bfe0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee61145640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee61145640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee61145640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c120040 con 0x7fee5c09bfe0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee53fff640 1 --2- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 0x7fee3c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.616+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fee4c049c70 con 0x7fee5c09bfe0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.617+0000 7fee5259c640 1 --2- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 0x7fee3c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.617+0000 7fee5259c640 10 monclient: get_auth_request con 0x7fee3c038880 auth_method 0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.619+0000 7fee5259c640 1 --2- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 0x7fee3c03ad40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fee5c13a120 tx=0x7fee44008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.621+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fee4c007070 con 0x7fee5c09bfe0 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.621+0000 7fee53fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:18:58.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.621+0000 7fee53fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.846+0000 7fee61145640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.846+0000 7fee61145640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.846+0000 7fee61145640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.846+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fee5c09c400 con 0x7fee5c09bfe0 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.847+0000 7fee53fff640 1 -- 172.21.15.134:0/1439025912 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3197 (secure 0 0 0) 0x7fee4c010300 con 0x7fee5c09bfe0 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.847+0000 7fee53fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:18:58.848 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.847+0000 7fee53fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:18:58.849 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 msgr2=0x7fee3c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 0x7fee3c03ad40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fee5c13a120 tx=0x7fee44008040 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 10 monclient: shutdown 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 msgr2=0x7fee5c1cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00c460 tx=0x7fee4c00c920 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 shutdown_connections 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> v2:172.21.15.162:6800/980547263 conn(0x7fee3c038880 0x7fee3c03ad40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee5c13bf80 0x7fee5c1389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fee5c136030 0x7fee5c133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 --2- 172.21.15.134:0/1439025912 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fee5c09bfe0 0x7fee5c1cb280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:18:58.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 >> 172.21.15.134:0/1439025912 conn(0x7fee5c0a9ea0 msgr2=0x7fee5c0a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:18:58.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 shutdown_connections 2024-09-16T15:18:58.852 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:18:58.849+0000 7fee61145640 1 -- 172.21.15.134:0/1439025912 wait complete. 2024-09-16T15:19:01.862 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:19:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.967+0000 7fd9c7d08640 10 monclient: get_monmap_and_config 2024-09-16T15:19:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.967+0000 7fd9c7d08640 10 monclient: build_initial_monmap 2024-09-16T15:19:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.967+0000 7fd9c7d08640 10 monclient: monmap: 2024-09-16T15:19:01.969 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:01.969231+0000 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:01.969231+0000 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:01.970 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:01.971 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:01.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.967+0000 7fd9c7d08640 1 Processor -- start 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 -- start start 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient: init 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient: _add_conns ranks=[3,8,2,0,9,5,7,1,6,10,4] 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-g con 0x7fd9c0136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-e con 0x7fd9c009bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-b con 0x7fd9c0136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009b600 con 0x7fd9c0136c60 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00a5740 con 0x7fd9c0136030 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00a52c0 con 0x7fd9c009bfe0 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c7d08640 10 monclient(hunting): authenticate will time out at 1878.558634s 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:37714/0 (socket says 172.21.15.134:37714) 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:38596/0 (socket says 172.21.15.134:38596) 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 1 -- 172.21.15.134:0/3864135035 learned_addr learned my addr 172.21.15.134:0/3864135035 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): get_auth_request con 0x7fd9c0136c60 auth_method 0 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): get_auth_request con 0x7fd9c009bfe0 auth_method 0 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:01.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c62a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.968+0000 7fd9c5aa4640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c5aa4640 10 monclient(hunting): get_auth_request con 0x7fd9c0136030 auth_method 0 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c5aa4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c5aa4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c5aa4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 10 monclient(hunting): handle_auth_done global_id 114518 payload 274 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 -- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 msgr2=0x7fd9c009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:01.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 -- 172.21.15.134:0/3864135035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 msgr2=0x7fd9c0136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 10 monclient: _finish_hunting 0 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 monclient: found mon.noname-b 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c62a5640 1 -- 172.21.15.134:0/3864135035 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0073250 con 0x7fd9c0136c60 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.969+0000 7fd9c52a3640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.971+0000 7fd9c62a5640 1 -- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 msgr2=0x7fd9c01370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.971+0000 7fd9c62a5640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009e80 tx=0x7fd9b802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c87039435de2078 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 1 -- 172.21.15.134:0/3864135035 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9b8007a70 con 0x7fd9c0136c60 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: dump: 2024-09-16T15:19:01.982 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:01.983 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: _finish_auth 0 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: _check_auth_tickets 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 5 monclient: authenticate success, global_id 114518 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 1 -- 172.21.15.134:0/3864135035 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b802d050 con 0x7fd9c0136c60 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 4 set_mon_vals no callback set 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 1 -- 172.21.15.134:0/3864135035 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9b8007bf0 con 0x7fd9c0136c60 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:01.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c4aa2640 10 monclient: dump: 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:01.985 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 10 monclient: shutdown 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 -- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 msgr2=0x7fd9c01370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009e80 tx=0x7fd9b802a530 comp rx=0 tx=0).stop 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 -- 172.21.15.134:0/3864135035 shutdown_connections 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0136c60 0x7fd9c01370c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9c009bfe0 0x7fd9c009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/3864135035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 -- 172.21.15.134:0/3864135035 >> 172.21.15.134:0/3864135035 conn(0x7fd9c00a9ea0 msgr2=0x7fd9c00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 -- 172.21.15.134:0/3864135035 shutdown_connections 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.972+0000 7fd9c7d08640 1 -- 172.21.15.134:0/3864135035 wait complete. 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.973+0000 7fd9c7d08640 10 monclient: build_initial_monmap 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.973+0000 7fd9c7d08640 10 monclient: monmap: 2024-09-16T15:19:01.986 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:01.987 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.973+0000 7fd9c7d08640 1 Processor -- start 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.973+0000 7fd9c7d08640 1 -- start start 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.973+0000 7fd9c7d08640 10 monclient: init 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient: _add_conns ranks=[5,3,7,0,2,4,1,8,6] 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 0x7fd9c00a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-h con 0x7fd9c009bfe0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-g con 0x7fd9c0136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 0x7fd9c0135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): picked mon.noname-i con 0x7fd9c0136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01227c0 con 0x7fd9c0136030 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009fd50 con 0x7fd9c009bfe0 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009f990 con 0x7fd9c0136c60 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c7d08640 10 monclient(hunting): authenticate will time out at 1878.564199s 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c62a5640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 msgr2=0x7fd9c0135340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c62a5640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 0x7fd9c0135340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 0x7fd9c00a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:37288/0 (socket says 172.21.15.134:37288) 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 -- 172.21.15.134:0/223058507 learned_addr learned my addr 172.21.15.134:0/223058507 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): get_auth_request con 0x7fd9c0136030 auth_method 0 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:01.989 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): get_auth_request con 0x7fd9c009bfe0 auth_method 0 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient(hunting): handle_auth_done global_id 84828 payload 824 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 -- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 msgr2=0x7fd9c0135340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 0x7fd9c0135340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 -- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 msgr2=0x7fd9c00a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:01.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 0x7fd9c00a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient: _finish_hunting 0 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 monclient: found mon.noname-g 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c52a3640 1 -- 172.21.15.134:0/223058507 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0009400 con 0x7fd9c0136030 2024-09-16T15:19:01.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.974+0000 7fd9c5aa4640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 0x7fd9c00a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c52a3640 1 -- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 msgr2=0x7fd9c0134e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c52a3640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000ebb0 tx=0x7fd9b000c520 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9b000eeb0 con 0x7fd9c0136030 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: dump: 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:01.993 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: _finish_auth 0 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000ce40 con 0x7fd9c0136030 2024-09-16T15:19:01.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 5 monclient: authenticate success, global_id 84828 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 10 monclient: _renew_subs 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b8009b30 con 0x7fd9c0136030 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd9b0013d50 con 0x7fd9c0136030 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b67fc640 10 monclient: dump: 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:01.995 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 10 monclient: _renew_subs 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0120fc0 con 0x7fd9c0136030 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.977+0000 7fd9b7fff640 4 set_mon_vals no callback set 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9c7d08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9c7d08640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:01.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9c7d08640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c01cb7d0 con 0x7fd9c0136030 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd9b0013470 con 0x7fd9c0136030 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9b67fc640 1 --2- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 0x7fd9a403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fd9b004a510 con 0x7fd9c0136030 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.978+0000 7fd9c5aa4640 1 --2- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 0x7fd9a403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.979+0000 7fd9c5aa4640 10 monclient: get_auth_request con 0x7fd9a4038880 auth_method 0 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.981+0000 7fd9c5aa4640 1 --2- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 0x7fd9a403ad40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc006fd0 tx=0x7fd9bc008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.984+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd9c01cb7d0 con 0x7fd9c0136030 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.984+0000 7fd9b67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:01.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:01.984+0000 7fd9b67fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:02.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.209+0000 7fd9c7d08640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:19:02.211 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.209+0000 7fd9c7d08640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:19:02.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.209+0000 7fd9c7d08640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:19:02.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.209+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd9c01cb7d0 con 0x7fd9c0136030 2024-09-16T15:19:02.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.210+0000 7fd9b67fc640 1 -- 172.21.15.134:0/223058507 <== mon.3 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3197 (secure 0 0 0) 0x7fd9c01cb7d0 con 0x7fd9c0136030 2024-09-16T15:19:02.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.210+0000 7fd9b67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:19:02.212 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.210+0000 7fd9b67fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:02.213 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":156,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:19:02.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 msgr2=0x7fd9a403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:02.214 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 0x7fd9a403ad40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc006fd0 tx=0x7fd9bc008040 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 10 monclient: shutdown 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 msgr2=0x7fd9c0134e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000ebb0 tx=0x7fd9b000c520 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 shutdown_connections 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd9c0136c60 0x7fd9c0135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> v2:172.21.15.162:6800/980547263 conn(0x7fd9a4038880 0x7fd9a403ad40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9c0136030 0x7fd9c0134e00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 --2- 172.21.15.134:0/223058507 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd9c009bfe0 0x7fd9c00a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 >> 172.21.15.134:0/223058507 conn(0x7fd9c00a9ea0 msgr2=0x7fd9c006b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 shutdown_connections 2024-09-16T15:19:02.215 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:02.212+0000 7fd9c7d08640 1 -- 172.21.15.134:0/223058507 wait complete. 2024-09-16T15:19:05.225 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:19:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 10 monclient: get_monmap_and_config 2024-09-16T15:19:05.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 10 monclient: build_initial_monmap 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 10 monclient: monmap: 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:05.334035+0000 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:05.334035+0000 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:05.335 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:05.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 1 Processor -- start 2024-09-16T15:19:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 1 -- start start 2024-09-16T15:19:05.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.333+0000 7fc75fd2c640 10 monclient: init 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient: _add_conns ranks=[3,1,9,2,7,8,10,4,6,0,5] 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-g con 0x7fc758136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fc75809bfe0 addr v1:172.21.15.162:6789/0 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-a con 0x7fc75809c400 addr v2:172.21.15.134:3300/0 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc758073250 con 0x7fc75809bfe0 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75809b600 con 0x7fc75809c400 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580a5740 con 0x7fc758136030 2024-09-16T15:19:05.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75fd2c640 10 monclient(hunting): authenticate will time out at 1881.923461s 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:40284/0 (socket says 172.21.15.134:40284) 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 learned_addr learned my addr 172.21.15.134:0/367654586 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc75809c400 auth_method 0 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc758136030 auth_method 0 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.334+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:05.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.337+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> v1:172.21.15.162:6789/0 conn(0x7fc75809bfe0 legacy=0x7fc758136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:05.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc75809c400 auth_method 0 2024-09-16T15:19:05.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc758136030 auth_method 0 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.535+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:19:05.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc75809c400 auth_method 0 2024-09-16T15:19:05.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc758136030 auth_method 0 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:05.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:05.936+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc75809c400 auth_method 0 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:06.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.737+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc758136030 auth_method 0 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:06.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:06.738+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:19:08.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): tick 2024-09-16T15:19:08.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): _check_auth_tickets 2024-09-16T15:19:08.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 monclient(hunting): continuing hunt 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): _reopen_session rank -1 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 msgr2=0x7fc758136410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc758136030 0x7fc758136410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 msgr2=0x7fc758137370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 --2- 172.21.15.134:0/367654586 >> v2:172.21.15.134:3300/0 conn(0x7fc75809c400 0x7fc758137370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 >> v1:172.21.15.162:6789/0 conn(0x7fc75809bfe0 legacy=0x7fc758136c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient: _add_conns ranks=[7,8,0,6,2,4,5,10] 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 0x7fc73c005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): picked mon.noname-i con 0x7fc73c003870 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): picked mon.noname-e con 0x7fc73c006340 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:08.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc73c008e80 addr v1:172.21.15.134:6789/0 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7580a52c0 con 0x7fc73c008e80 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fc75809b600 con 0x7fc73c003870 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580a5740 con 0x7fc73c006340 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc74f7fe640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 msgr2=0x7fc73c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 0x7fc73c005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request con 0x7fc73c006340 auth_method 0 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:08.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:19:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.334+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.336+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 >> v1:172.21.15.134:6789/0 conn(0x7fc73c008e80 legacy=0x7fc73c00b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 msgr2=0x7fc73c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 0x7fc73c005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request con 0x7fc73c006340 auth_method 0 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:08.537 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:08.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:19:08.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:08.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:08.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.535+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:19:08.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75dac8640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 msgr2=0x7fc73c005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 0x7fc73c005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request con 0x7fc73c006340 auth_method 0 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:08.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:08.936+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.0 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818545256 0 0) 0x7fc7580a52c0 con 0x7fc73c008e80 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 10 monclient(hunting): _negotiate 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 10 monclient(hunting): my global_id is 124254 2024-09-16T15:19:09.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.075+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc744003520 con 0x7fc73c008e80 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.0 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575991083 0 0) 0x7fc744003520 con 0x7fc73c008e80 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 msgr2=0x7fc73c0087b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc73c006340 0x7fc73c0087b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 msgr2=0x7fc73c005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fc73c003870 0x7fc73c005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 monclient: found mon.noname-a-legacy 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:172.21.15.134:6789/0 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c00dd30 con 0x7fc73c008e80 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.0 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 2104958246 0 0) 0x7fc75000caf0 con 0x7fc73c008e80 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: dump: 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.081 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 monclient: mon.a has (v2) addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] but i'm connected to v1:172.21.15.134:6789/0, reconnecting 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 >> v1:172.21.15.134:6789/0 conn(0x7fc73c008e80 legacy=0x7fc73c00b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 reap_dead start 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient: _add_conns ranks=[1,2,8,7,5,6,3,4,0] 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744004170 0x7fc7440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): picked mon.f con 0x7fc744004170 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc744009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): picked mon.b con 0x7fc744004e20 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:09.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc744009eb0 0x7fc74400c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): picked mon.e con 0x7fc744009eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75809b600 con 0x7fc744004170 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580a5740 con 0x7fc744004e20 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc74400ca00 con 0x7fc744009eb0 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75cac6640 10 monclient(hunting): _finish_auth 0 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.076+0000 7fc75e2c9640 1 -- 172.21.15.134:0/367654586 reap_dead start 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc744009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744004170 0x7fc7440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request con 0x7fc744004e20 auth_method 0 2024-09-16T15:19:09.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc744004170 auth_method 0 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc744009eb0 0x7fc74400c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc744009eb0 auth_method 0 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:19:09.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_done global_id 124254 payload 293 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc744009eb0 msgr2=0x7fc74400c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc744009eb0 0x7fc74400c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744004170 msgr2=0x7fc7440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744004170 0x7fc7440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 monclient: found mon.b 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c00dd30 con 0x7fc744004e20 2024-09-16T15:19:09.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75d2c7640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75fd2c640 5 monclient: authenticate success, global_id 124254 2024-09-16T15:19:09.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.077+0000 7fc75fd2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:09.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.080+0000 7fc75d2c7640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 msgr2=0x7fc744009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.080+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc744009970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003740 tx=0x7fc7440057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385d7beea171dd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc748007540 con 0x7fc744004e20 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: dump: 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.091 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74802d050 con 0x7fc744004e20 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75fd2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75fd2c640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 1 -- 172.21.15.134:0/367654586 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc748005c30 con 0x7fc744004e20 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75fd2c640 4 set_mon_vals no callback set 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.082+0000 7fc75cac6640 10 monclient: dump: 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.093 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 10 monclient: shutdown 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 -- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 msgr2=0x7fc744009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc744009970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003740 tx=0x7fc7440057f0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 -- 172.21.15.134:0/367654586 shutdown_connections 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fc744009eb0 0x7fc74400c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc744009970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/367654586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744004170 0x7fc7440047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 -- 172.21.15.134:0/367654586 >> 172.21.15.134:0/367654586 conn(0x7fc7580a9ea0 msgr2=0x7fc7580a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 -- 172.21.15.134:0/367654586 shutdown_connections 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.083+0000 7fc75fd2c640 1 -- 172.21.15.134:0/367654586 wait complete. 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.084+0000 7fc75fd2c640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.084+0000 7fc75fd2c640 10 monclient: monmap: 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.094 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 Processor -- start 2024-09-16T15:19:09.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 -- start start 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient: init 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient: _add_conns ranks=[5,2,1,7,6,4,0,3,8] 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 0x7fc7581cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-h con 0x7fc744004170 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-b con 0x7fc744004e20 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 0x7fc75811ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): picked mon.noname-f con 0x7fc744009eb0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806c070 con 0x7fc744009eb0 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580a8490 con 0x7fc744004e20 2024-09-16T15:19:09.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc758121bb0 con 0x7fc744004170 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75fd2c640 10 monclient(hunting): authenticate will time out at 1885.674854s 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75d2c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75d2c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51514/0 (socket says 172.21.15.134:51514) 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.085+0000 7fc75d2c7640 1 -- 172.21.15.134:0/2861601513 learned_addr learned my addr 172.21.15.134:0/2861601513 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request con 0x7fc744004e20 auth_method 0 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 0x7fc7581cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): get_auth_request con 0x7fc744004170 auth_method 0 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 0x7fc75811ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request con 0x7fc744009eb0 auth_method 0 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient(hunting): handle_auth_done global_id 114527 payload 824 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 -- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 msgr2=0x7fc7581cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 0x7fc7581cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 -- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 msgr2=0x7fc75811ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 0x7fc75811ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 monclient: found mon.noname-b 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75d2c7640 1 -- 172.21.15.134:0/2861601513 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758123670 con 0x7fc744004e20 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75e2c9640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 0x7fc75811ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.086+0000 7fc75dac8640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 0x7fc7581cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:09.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.088+0000 7fc75d2c7640 1 -- 172.21.15.134:0/2861601513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 msgr2=0x7fc7581cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.088+0000 7fc75d2c7640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc748001980 tx=0x7fc748001520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc74803b880 con 0x7fc744004e20 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: dump: 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.100 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74803ba00 con 0x7fc744004e20 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc748031400 con 0x7fc744004e20 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 5 monclient: authenticate success, global_id 114527 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 10 monclient: _renew_subs 2024-09-16T15:19:09.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc758123f10 con 0x7fc744004e20 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74ffff640 4 set_mon_vals no callback set 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc74e7fc640 10 monclient: dump: 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.102 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 10 monclient: _renew_subs 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.089+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581cc150 con 0x7fc744004e20 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc748038070 con 0x7fc744004e20 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75fd2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75fd2c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75fd2c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc74e7fc640 1 --2- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 0x7fc73c03bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc758136410 con 0x7fc744004e20 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fc74806a210 con 0x7fc744004e20 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75dac8640 1 --2- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 0x7fc73c03bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.090+0000 7fc75dac8640 10 monclient: get_auth_request con 0x7fc73c039a90 auth_method 0 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.093+0000 7fc75dac8640 1 --2- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 0x7fc73c03bf50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc754002230 tx=0x7fc75400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.095+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fc74803f070 con 0x7fc744004e20 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.095+0000 7fc74e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.095+0000 7fc74e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:09.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc75fd2c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:19:09.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc75fd2c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc75fd2c640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc758072070 con 0x7fc744004e20 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc74e7fc640 1 -- 172.21.15.134:0/2861601513 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3190 (secure 0 0 0) 0x7fc748036030 con 0x7fc744004e20 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc74e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.324+0000 7fc74e7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:09.326 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":160,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 msgr2=0x7fc73c03bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 0x7fc73c03bf50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc754002230 tx=0x7fc75400b040 comp rx=0 tx=0).stop 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 10 monclient: shutdown 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 msgr2=0x7fc7581cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc748001980 tx=0x7fc748001520 comp rx=0 tx=0).stop 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 shutdown_connections 2024-09-16T15:19:09.328 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> v2:172.21.15.162:6800/980547263 conn(0x7fc73c039a90 0x7fc73c03bf50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc744009eb0 0x7fc75811ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc744004e20 0x7fc7581cbc10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 --2- 172.21.15.134:0/2861601513 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc744004170 0x7fc7581cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 >> 172.21.15.134:0/2861601513 conn(0x7fc7580a9ea0 msgr2=0x7fc75809e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 shutdown_connections 2024-09-16T15:19:09.329 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.326+0000 7fc75fd2c640 1 -- 172.21.15.134:0/2861601513 wait complete. 2024-09-16T15:19:09.338 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-16T15:19:09.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-16T15:19:09.339 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.435+0000 7f0dc286b640 10 monclient: get_monmap_and_config 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.435+0000 7f0dc286b640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.435+0000 7f0dc286b640 10 monclient: monmap: 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:09.436683+0000 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:09.436683+0000 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:09.437 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:09.438 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 1 Processor -- start 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 1 -- start start 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 10 monclient: init 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 10 monclient: _add_conns ranks=[2,0,8,6,9,10,1,7,4,5,3] 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 0x7f0dbc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-b con 0x7f0dbc136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.436+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0dbc137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0dbc13c370 0x7f0dbc13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-e con 0x7f0dbc13c370 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc073250 con 0x7f0dbc137220 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0a56e0 con 0x7f0dbc136e40 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc09abc0 con 0x7f0dbc13c370 2024-09-16T15:19:09.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc286b640 10 monclient(hunting): authenticate will time out at 1886.026044s 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 0x7f0dbc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbb7fe640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0dbc137220 0x7f0dbc137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:42562/0 (socket says 172.21.15.134:42562) 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/2256622251 learned_addr learned my addr 172.21.15.134:0/2256622251 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): get_auth_request con 0x7f0dbc136e40 auth_method 0 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0dbc13c370 0x7f0dbc13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): get_auth_request con 0x7f0dbc13c370 auth_method 0 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dc0e08640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_done global_id 114536 payload 274 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 -- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0dbc13c370 msgr2=0x7f0dbc13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 --2- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0dbc13c370 0x7f0dbc13e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 -- 172.21.15.134:0/2256622251 >> v1:172.21.15.134:6789/0 conn(0x7f0dbc137220 legacy=0x7f0dbc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 monclient: found mon.noname-b 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:09.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.437+0000 7f0dbbfff640 1 -- 172.21.15.134:0/2256622251 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc09ba50 con 0x7f0dbc136e40 2024-09-16T15:19:09.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbbfff640 1 -- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 msgr2=0x7f0dbc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbbfff640 1 --2- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 0x7f0dbc09c150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0097d0 tx=0x7f0dac02a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4d84a9634cde45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 1 -- 172.21.15.134:0/2256622251 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0dac02ad20 con 0x7f0dbc136e40 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: dump: 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.451 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 1 -- 172.21.15.134:0/2256622251 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac0307b0 con 0x7f0dbc136e40 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 5 monclient: authenticate success, global_id 114536 2024-09-16T15:19:09.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 1 -- 172.21.15.134:0/2256622251 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0dac02aea0 con 0x7f0dbc136e40 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 4 set_mon_vals no callback set 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dbaffd640 10 monclient: dump: 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.453 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 10 monclient: shutdown 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 -- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 msgr2=0x7f0dbc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 --2- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 0x7f0dbc09c150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0097d0 tx=0x7f0dac02a000 comp rx=0 tx=0).stop 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 -- 172.21.15.134:0/2256622251 shutdown_connections 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 --2- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0dbc13c370 0x7f0dbc13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 --2- 172.21.15.134:0/2256622251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dbc136e40 0x7f0dbc09c150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 -- 172.21.15.134:0/2256622251 >> 172.21.15.134:0/2256622251 conn(0x7f0dbc0a6110 msgr2=0x7f0dbc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 -- 172.21.15.134:0/2256622251 shutdown_connections 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.440+0000 7f0dc286b640 1 -- 172.21.15.134:0/2256622251 wait complete. 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.441+0000 7f0dc286b640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.441+0000 7f0dc286b640 10 monclient: monmap: 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.455 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.441+0000 7f0dc286b640 1 Processor -- start 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 -- start start 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient: init 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient: _add_conns ranks=[0,4,7,5,3,8,1,6,2] 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 0x7f0dbc135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-a con 0x7f0dbc136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0dbc137220 0x7f0dbc1363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-c con 0x7f0dbc137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0dbc13c370 0x7f0dbc11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): picked mon.noname-i con 0x7f0dbc13c370 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1cc4d0 con 0x7f0dbc136e40 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc122280 con 0x7f0dbc137220 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0a7b90 con 0x7f0dbc13c370 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc286b640 10 monclient(hunting): authenticate will time out at 1886.031442s 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 0x7f0dbc135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0dbc137220 0x7f0dbc1363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 0x7f0dbc135e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57602/0 (socket says 172.21.15.134:57602) 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 -- 172.21.15.134:0/1085991532 learned_addr learned my addr 172.21.15.134:0/1085991532 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): get_auth_request con 0x7f0dbc136e40 auth_method 0 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): get_auth_request con 0x7f0dbc137220 auth_method 0 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc0e08640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0dbc13c370 msgr2=0x7f0dbc11fc60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dc0e08640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0dbc13c370 0x7f0dbc11fc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.458 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient(hunting): handle_auth_done global_id 124263 payload 824 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0dbc13c370 msgr2=0x7f0dbc11fc60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f0dbc13c370 0x7f0dbc11fc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0dbc137220 msgr2=0x7f0dbc1363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0dbc137220 0x7f0dbc1363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 monclient: found mon.noname-a 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.442+0000 7f0dbbfff640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009400 con 0x7f0dbc136e40 2024-09-16T15:19:09.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dbbfff640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 msgr2=0x7f0dbc135e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dbbfff640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 0x7f0dbc135e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0dac035040 tx=0x7f0dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0dac004460 con 0x7f0dbc136e40 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: dump: 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.460 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac003a80 con 0x7f0dbc136e40 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 5 monclient: authenticate success, global_id 124263 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0dac003bf0 con 0x7f0dbc136e40 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 10 monclient: _renew_subs 2024-09-16T15:19:09.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc0a75f0 con 0x7f0dbc136e40 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0db8ff9640 10 monclient: dump: 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.462 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dba7fc640 4 set_mon_vals no callback set 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 10 monclient: _renew_subs 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.445+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc0a3510 con 0x7f0dbc136e40 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0dac039040 con 0x7f0dbc136e40 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc286b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc286b640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 0x7f0dbc120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc071b20 con 0x7f0dbc1201a0 2024-09-16T15:19:09.463 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0db8ff9640 1 --2- 172.21.15.134:0/1085991532 >> v2:172.21.15.162:6800/980547263 conn(0x7f0da8038880 0x7f0da803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 0x7f0dbc120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient: get_auth_request con 0x7f0dbc1201a0 auth_method 0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f0dac06a760 con 0x7f0dbc136e40 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_done global_id 124272 payload 824 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dbc061ae0 con 0x7f0dbc1201a0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc0e08640 1 --2- 172.21.15.134:0/1085991532 >> v2:172.21.15.162:6800/980547263 conn(0x7f0da8038880 0x7f0da803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.446+0000 7f0dc0e08640 10 monclient: get_auth_request con 0x7f0da8038880 auth_method 0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 msgr2=0x7f0dbc120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 0x7f0dbc120580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc13b4a0 tx=0x7f0db00052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0db0013d60 con 0x7f0dbc1201a0 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0db8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0db8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.448+0000 7f0db8ff9640 10 monclient: dump: 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.465 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0db00138f0 con 0x7f0dbc1201a0 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 msgr2=0x7f0dbc120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 0x7f0dbc120580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc13b4a0 tx=0x7f0db00052d0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 msgr2=0x7f0dbc120580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0db8ff9640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc1201a0 0x7f0dbc120580 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc13b4a0 tx=0x7f0db00052d0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.466 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.449+0000 7f0dc0e08640 1 --2- 172.21.15.134:0/1085991532 >> v2:172.21.15.162:6800/980547263 conn(0x7f0da8038880 0x7f0da803ad40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0a3040 tx=0x7f0db400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dc286b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:09.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dc286b640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dc286b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc07a680 con 0x7f0dbc09d4f0 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient: get_auth_request con 0x7f0dbc09d4f0 auth_method 0 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 10 monclient(hunting): handle_auth_done global_id 124281 payload 824 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.455+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0dbc0a53c0 con 0x7f0dbc09d4f0 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.457+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 msgr2=0x7f0dbc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.457+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 msgr2=0x7f0dbc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:19:09.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.457+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f0db0007920 tx=0x7f0db0004940 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:19:09.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.457+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: get_auth_request con 0x7f0dbc09d4f0 auth_method 0 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: _init_auth method 2 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: _init_auth already have auth, reseting 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: handle_auth_reply_more payload 9 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: handle_auth_reply_more responding with 132 bytes 2024-09-16T15:19:09.660 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.658+0000 7f0dbb7fe640 10 monclient: handle_auth_done global_id 124281 payload 293 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0dbb7fe640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 msgr2=0x7f0dbc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0dbb7fe640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0db0020000 tx=0x7f0db0009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0db0023070 con 0x7f0dbc09d4f0 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0db8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0db8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:19:09.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.660+0000 7f0db8ff9640 10 monclient: dump: 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.663 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9187 (secure 0 0 0) 0x7f0db0006030 con 0x7f0dbc09d4f0 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 msgr2=0x7f0dbc09d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0db0020000 tx=0x7f0db0009030 comp rx=0 tx=0).stop 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 msgr2=0x7f0dbc09d900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0db8ff9640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc09d4f0 0x7f0dbc09d900 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0db0020000 tx=0x7f0db0009030 comp rx=0 tx=0).stop 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:09.664 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.665 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:09.666 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:09.667 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.668 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.669 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:09.670 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.673 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:09.674 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.675 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.677 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.678 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.679 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:09.680 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.681 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.682 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:09.683 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:09.684 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 >> v2:172.21.15.162:6800/980547263 conn(0x7f0da8038880 msgr2=0x7f0da803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 --2- 172.21.15.134:0/1085991532 >> v2:172.21.15.162:6800/980547263 conn(0x7f0da8038880 0x7f0da803ad40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0a3040 tx=0x7f0db400b040 comp rx=0 tx=0).stop 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 10 monclient: shutdown 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 msgr2=0x7f0dbc135e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 --2- 172.21.15.134:0/1085991532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dbc136e40 0x7f0dbc135e90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0dac035040 tx=0x7f0dac0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc0e08640 1 -- 172.21.15.134:0/1085991532 reap_dead start 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc0e08640 1 -- 172.21.15.134:0/1085991532 reap_dead start 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 shutdown_connections 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 >> 172.21.15.134:0/1085991532 conn(0x7f0dbc0a6110 msgr2=0x7f0dbc13b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 shutdown_connections 2024-09-16T15:19:09.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.661+0000 7f0dc286b640 1 -- 172.21.15.134:0/1085991532 wait complete. 2024-09-16T15:19:09.687 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:09.687 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-16T15:19:09.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.784+0000 7f7fd20d0640 10 monclient: get_monmap_and_config 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.784+0000 7f7fd20d0640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.784+0000 7f7fd20d0640 10 monclient: monmap: 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:09.785681+0000 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:09.785681+0000 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.786 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:09.787 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.785+0000 7f7fd20d0640 1 Processor -- start 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.785+0000 7f7fd20d0640 1 -- start start 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.785+0000 7f7fd20d0640 10 monclient: init 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient: _add_conns ranks=[6,8,3,0,7,10,4,1,5,9,2] 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-d con 0x7f7fcc136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 0x7f7fcc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-e con 0x7f7fcc137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fcc135dc0 0x7f7fcc136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-g con 0x7f7fcc135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc09abc0 con 0x7f7fcc135dc0 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc09ba50 con 0x7f7fcc136e40 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0a53c0 con 0x7f7fcc137220 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fd20d0640 10 monclient(hunting): authenticate will time out at 1886.375188s 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 0x7f7fcc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52802/0 (socket says 172.21.15.134:52802) 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): get_auth_request con 0x7f7fcc137220 auth_method 0 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/1698921061 learned_addr learned my addr 172.21.15.134:0/1698921061 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): get_auth_request con 0x7f7fcc136e40 auth_method 0 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcbfff640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fcc135dc0 0x7f7fcc136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcbfff640 10 monclient(hunting): get_auth_request con 0x7f7fcc135dc0 auth_method 0 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_done global_id 94677 payload 274 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 msgr2=0x7f7fcc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 0x7f7fcc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/1698921061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fcc135dc0 msgr2=0x7f7fcc136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fcc135dc0 0x7f7fcc136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 monclient: found mon.noname-d 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/1698921061 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc073250 con 0x7f7fcc136e40 2024-09-16T15:19:09.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.786+0000 7f7fc3fff640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 0x7f7fcc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:09.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 msgr2=0x7f7fcc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc02a080 comp rx=0 tx=0).ready entity=mon.6 client_cookie=f16cb68feadc3f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 1 -- 172.21.15.134:0/1698921061 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc02ade0 con 0x7f7fcc136e40 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: dump: 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.800 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 1 -- 172.21.15.134:0/1698921061 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0307b0 con 0x7f7fcc136e40 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fd20d0640 5 monclient: authenticate success, global_id 94677 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fd20d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fd20d0640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fd20d0640 4 set_mon_vals no callback set 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 1 -- 172.21.15.134:0/1698921061 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc030d90 con 0x7f7fcc136e40 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.789+0000 7f7fcaffd640 10 monclient: dump: 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.802 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 10 monclient: shutdown 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 -- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 msgr2=0x7f7fcc09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc02a080 comp rx=0 tx=0).stop 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 -- 172.21.15.134:0/1698921061 shutdown_connections 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fcc135dc0 0x7f7fcc136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc137220 0x7f7fcc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/1698921061 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f7fcc136e40 0x7f7fcc09c150 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 -- 172.21.15.134:0/1698921061 >> 172.21.15.134:0/1698921061 conn(0x7f7fcc0a6110 msgr2=0x7f7fcc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 -- 172.21.15.134:0/1698921061 shutdown_connections 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 1 -- 172.21.15.134:0/1698921061 wait complete. 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.790+0000 7f7fd20d0640 10 monclient: monmap: 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:09.804 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 Processor -- start 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 -- start start 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient: init 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient: _add_conns ranks=[1,8,2,3,5,4,0,7,6] 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fcc135dc0 0x7f7fcc1cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-f con 0x7f7fcc135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 0x7f7fcc135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-e con 0x7f7fcc136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc137220 0x7f7fcc135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): picked mon.noname-b con 0x7f7fcc137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1257d0 con 0x7f7fcc135dc0 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc125a20 con 0x7f7fcc137220 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc136850 con 0x7f7fcc136e40 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fd20d0640 10 monclient(hunting): authenticate will time out at 1886.380538s 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 0x7f7fcc135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 0x7f7fcc135010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:40136/0 (socket says 172.21.15.134:40136) 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 -- 172.21.15.134:0/219812549 learned_addr learned my addr 172.21.15.134:0/219812549 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): get_auth_request con 0x7f7fcc136e40 auth_method 0 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc137220 0x7f7fcc135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): get_auth_request con 0x7f7fcc137220 auth_method 0 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient(hunting): handle_auth_done global_id 85022 payload 824 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc137220 msgr2=0x7f7fcc135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc137220 0x7f7fcc135550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fcc135dc0 0x7f7fcc1cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fcc135dc0 msgr2=0x7f7fcc1cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fcc135dc0 0x7f7fcc1cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 monclient: found mon.noname-e 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.791+0000 7f7fc3fff640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4009400 con 0x7f7fcc136e40 2024-09-16T15:19:09.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc3fff640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 msgr2=0x7f7fcc135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc3fff640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 0x7f7fcc135010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400e990 tx=0x7f7fb400ee50 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fb400cbf0 con 0x7f7fcc136e40 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: dump: 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.809 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb400cd70 con 0x7f7fcc136e40 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 5 monclient: authenticate success, global_id 85022 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 10 monclient: _renew_subs 2024-09-16T15:19:09.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc0094d0 con 0x7f7fcc136e40 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fb400cee0 con 0x7f7fcc136e40 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fc8ff9640 10 monclient: dump: 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.811 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fca7fc640 4 set_mon_vals no callback set 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 10 monclient: _renew_subs 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.794+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc136320 con 0x7f7fcc136e40 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fd20d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 0x7f7fcc121e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0735d0 con 0x7f7fcc121a70 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f7fb4013470 con 0x7f7fcc136e40 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 0x7f7fcc121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient: get_auth_request con 0x7f7fcc121a70 auth_method 0 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fc8ff9640 1 --2- 172.21.15.134:0/219812549 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fac038880 0x7f7fac03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f7fb404a3d0 con 0x7f7fcc136e40 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_done global_id 114554 payload 824 2024-09-16T15:19:09.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fcc061ae0 con 0x7f7fcc121a70 2024-09-16T15:19:09.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcbfff640 1 --2- 172.21.15.134:0/219812549 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fac038880 0x7f7fac03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.795+0000 7f7fcbfff640 10 monclient: get_auth_request con 0x7f7fac038880 auth_method 0 2024-09-16T15:19:09.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 msgr2=0x7f7fcc121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 0x7f7fcc121e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc034000 tx=0x7f7fbc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc02ade0 con 0x7f7fcc121a70 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fc8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fc8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.797+0000 7f7fc8ff9640 10 monclient: dump: 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.816 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7fbc030b30 con 0x7f7fcc121a70 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 msgr2=0x7f7fcc121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 0x7f7fcc121e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc034000 tx=0x7f7fbc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 msgr2=0x7f7fcc121e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fc8ff9640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc121a70 0x7f7fcc121e50 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc034000 tx=0x7f7fbc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.798+0000 7f7fcbfff640 1 --2- 172.21.15.134:0/219812549 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fac038880 0x7f7fac03ad40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc06b550 tx=0x7f7fc4009820 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fd20d0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 0x7f7fcc0a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fd20d0640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc125bb0 con 0x7f7fcc122390 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 0x7f7fcc0a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient: get_auth_request con 0x7f7fcc122390 auth_method 0 2024-09-16T15:19:09.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 10 monclient(hunting): handle_auth_done global_id 114563 payload 824 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.804+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/219812549 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7fcc07a680 con 0x7f7fcc122390 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fcb7fe640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 msgr2=0x7f7fcc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fcb7fe640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 0x7f7fcc0a1a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fbc04b070 con 0x7f7fcc122390 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fc8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fc8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fc8ff9640 10 monclient: dump: 2024-09-16T15:19:09.820 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.821 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.806+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9517 (secure 0 0 0) 0x7f7fbc032440 con 0x7f7fcc122390 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 msgr2=0x7f7fcc0a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 0x7f7fcc0a1a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc009240 comp rx=0 tx=0).stop 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 msgr2=0x7f7fcc0a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fc8ff9640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7fcc122390 0x7f7fcc0a1a20 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0098a0 tx=0x7f7fbc009240 comp rx=0 tx=0).stop 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:09.822 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:09.823 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:09.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:09.825 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:09.826 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:09.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:09.829 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.830 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.833 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.835 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.837 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:09.839 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.841 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.842 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.843 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 3 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:09.844 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fac038880 msgr2=0x7f7fac03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/219812549 >> v2:172.21.15.162:6800/980547263 conn(0x7f7fac038880 0x7f7fac03ad40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc06b550 tx=0x7f7fc4009820 comp rx=0 tx=0).stop 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 10 monclient: shutdown 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 msgr2=0x7f7fcc135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.846 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 --2- 172.21.15.134:0/219812549 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f7fcc136e40 0x7f7fcc135010 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400e990 tx=0x7f7fb400ee50 comp rx=0 tx=0).stop 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fcbfff640 1 -- 172.21.15.134:0/219812549 reap_dead start 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fcbfff640 1 -- 172.21.15.134:0/219812549 reap_dead start 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 shutdown_connections 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 >> 172.21.15.134:0/219812549 conn(0x7f7fcc0a6110 msgr2=0x7f7fcc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 shutdown_connections 2024-09-16T15:19:09.847 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.807+0000 7f7fd20d0640 1 -- 172.21.15.134:0/219812549 wait complete. 2024-09-16T15:19:09.847 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:09.847 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-16T15:19:09.847 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:19:09.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.944+0000 7fe78e872640 10 monclient: get_monmap_and_config 2024-09-16T15:19:09.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.944+0000 7fe78e872640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.944+0000 7fe78e872640 10 monclient: monmap: 2024-09-16T15:19:09.945 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.945 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:09.945172+0000 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:09.945172+0000 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:09.946 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:09.947 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.944+0000 7fe78e872640 1 Processor -- start 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 -- start start 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient: init 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient: _add_conns ranks=[4,6,7,10,0,8,3,1,5,2,9] 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-c con 0x7fe788136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-d con 0x7fe788137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:09.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 0x7fe788136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-i con 0x7fe788135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78809abc0 con 0x7fe788136e40 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe78809ba50 con 0x7fe788137220 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880a53c0 con 0x7fe788135dc0 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78e872640 10 monclient(hunting): authenticate will time out at 1886.534503s 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:36976/0 (socket says 172.21.15.134:36976) 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52816/0 (socket says 172.21.15.134:52816) 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 -- 172.21.15.134:0/4192540731 learned_addr learned my addr 172.21.15.134:0/4192540731 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78ce0f640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 msgr2=0x7fe788136220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): get_auth_request con 0x7fe788136e40 auth_method 0 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): get_auth_request con 0x7fe788137220 auth_method 0 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe78ce0f640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 0x7fe788136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient(hunting): handle_auth_done global_id 124276 payload 274 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 -- 172.21.15.134:0/4192540731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 msgr2=0x7fe788136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 0x7fe788136220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 -- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 msgr2=0x7fe788137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 monclient: found mon.noname-c 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe787fff640 1 -- 172.21.15.134:0/4192540731 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788073250 con 0x7fe788136e40 2024-09-16T15:19:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.945+0000 7fe7877fe640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe787fff640 1 -- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 msgr2=0x7fe78809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe787fff640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098a0 tx=0x7fe77802a0a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=aa6385caae3e0800 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 1 -- 172.21.15.134:0/4192540731 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe77802ae00 con 0x7fe788136e40 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: dump: 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.959 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 1 -- 172.21.15.134:0/4192540731 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7780307b0 con 0x7fe788136e40 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe78e872640 5 monclient: authenticate success, global_id 124276 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe78e872640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe78e872640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 1 -- 172.21.15.134:0/4192540731 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe778030d90 con 0x7fe788136e40 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe78e872640 4 set_mon_vals no callback set 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.948+0000 7fe786ffd640 10 monclient: dump: 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.961 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 10 monclient: shutdown 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 -- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 msgr2=0x7fe78809c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098a0 tx=0x7fe77802a0a0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 -- 172.21.15.134:0/4192540731 shutdown_connections 2024-09-16T15:19:09.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe788135dc0 0x7fe788136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe788137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 --2- 172.21.15.134:0/4192540731 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe78809c150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 -- 172.21.15.134:0/4192540731 >> 172.21.15.134:0/4192540731 conn(0x7fe7880a6110 msgr2=0x7fe7880a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 -- 172.21.15.134:0/4192540731 shutdown_connections 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 1 -- 172.21.15.134:0/4192540731 wait complete. 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 10 monclient: build_initial_monmap 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.949+0000 7fe78e872640 10 monclient: monmap: 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:09.963 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 1 Processor -- start 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 1 -- start start 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 10 monclient: init 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 10 monclient: _add_conns ranks=[0,4,6,7,5,3,8,1,2] 2024-09-16T15:19:09.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe788135dc0 0x7fe7880a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-a con 0x7fe788135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-c con 0x7fe788136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.950+0000 7fe78e872640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe7881cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): picked mon.noname-d con 0x7fe788137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7881228d0 con 0x7fe788135dc0 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe788136490 con 0x7fe788136e40 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880a3570 con 0x7fe788137220 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78e872640 10 monclient(hunting): authenticate will time out at 1886.540048s 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:36988/0 (socket says 172.21.15.134:36988) 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 -- 172.21.15.134:0/3435242927 learned_addr learned my addr 172.21.15.134:0/3435242927 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): get_auth_request con 0x7fe788136e40 auth_method 0 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe7881cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe788135dc0 0x7fe7880a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): get_auth_request con 0x7fe788135dc0 auth_method 0 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): get_auth_request con 0x7fe788137220 auth_method 0 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe78ce0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient(hunting): handle_auth_done global_id 124285 payload 824 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 msgr2=0x7fe7881cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe788137220 0x7fe7881cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe788135dc0 msgr2=0x7fe7880a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe788135dc0 0x7fe7880a2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient: _finish_hunting 0 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 monclient: found mon.noname-c 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe7877fe640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009400 con 0x7fe788136e40 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.951+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe788135dc0 0x7fe7880a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe7877fe640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 msgr2=0x7fe788134e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe7877fe640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe77000e990 tx=0x7fe77000ee50 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe77000cbf0 con 0x7fe788136e40 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: dump: 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.968 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: _finish_auth 0 2024-09-16T15:19:09.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000cd70 con 0x7fe788136e40 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 5 monclient: authenticate success, global_id 124285 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 10 monclient: _renew_subs 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7780094d0 con 0x7fe788136e40 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe77000cee0 con 0x7fe788136e40 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe784ff9640 10 monclient: dump: 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.970 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe7867fc640 4 set_mon_vals no callback set 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 10 monclient: _renew_subs 2024-09-16T15:19:09.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.953+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788121fa0 con 0x7fe788136e40 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe78e872640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe770013470 con 0x7fe788136e40 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe78e872640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 0x7fe78802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe788071b20 con 0x7fe7881353c0 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 0x7fe78802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient: get_auth_request con 0x7fe7881353c0 auth_method 0 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe784ff9640 1 --2- 172.21.15.134:0/3435242927 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7680388b0 0x7fe76803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(829..829 src has 1..829) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fe77004a420 con 0x7fe788136e40 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 10 monclient(hunting): handle_auth_done global_id 124294 payload 824 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.954+0000 7fe787fff640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe788061ae0 con 0x7fe7881353c0 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.955+0000 7fe78ce0f640 1 --2- 172.21.15.134:0/3435242927 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7680388b0 0x7fe76803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.955+0000 7fe78ce0f640 10 monclient: get_auth_request con 0x7fe7680388b0 auth_method 0 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe787fff640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 msgr2=0x7fe78802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 0x7fe78802b260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe7780046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe778030e90 con 0x7fe7881353c0 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 10 monclient: dump: 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.973 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe778003820 con 0x7fe7881353c0 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:09.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 msgr2=0x7fe78802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 0x7fe78802b260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe7780046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 msgr2=0x7fe78802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe784ff9640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881353c0 0x7fe78802b260 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe7780046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:09.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.957+0000 7fe78ce0f640 1 --2- 172.21.15.134:0/3435242927 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7680388b0 0x7fe76803ad70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe78806b550 tx=0x7fe7800095f0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe78e872640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe78e872640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 0x7fe78802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe78e872640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe78807a680 con 0x7fe7881210b0 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 0x7fe78802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient: get_auth_request con 0x7fe7881210b0 auth_method 0 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:09.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 10 monclient(hunting): handle_auth_done global_id 124303 payload 824 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.963+0000 7fe787fff640 1 -- 172.21.15.134:0/3435242927 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe78809b490 con 0x7fe7881210b0 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe787fff640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 msgr2=0x7fe78802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe787fff640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 0x7fe78802b7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097c0 tx=0x7fe778009240 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe77803c750 con 0x7fe7881210b0 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 10 monclient: dump: 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:09.978 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fe778049a70 con 0x7fe7881210b0 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 msgr2=0x7fe78802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 0x7fe78802b7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097c0 tx=0x7fe778009240 comp rx=0 tx=0).stop 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 msgr2=0x7fe78802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:09.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe784ff9640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe7881210b0 0x7fe78802b7a0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097c0 tx=0x7fe778009240 comp rx=0 tx=0).stop 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:09.980 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 0, 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.981 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:09.983 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.984 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.985 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:09.987 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.988 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.989 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:09.991 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.993 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.995 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.997 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:09.998 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:09.999 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.000 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.000 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7680388b0 msgr2=0x7fe76803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 --2- 172.21.15.134:0/3435242927 >> v2:172.21.15.162:6800/980547263 conn(0x7fe7680388b0 0x7fe76803ad70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe78806b550 tx=0x7fe7800095f0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 10 monclient: shutdown 2024-09-16T15:19:10.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 msgr2=0x7fe788134e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe77000e990 tx=0x7fe77000ee50 comp rx=0 tx=0).stop 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78ce0f640 1 -- 172.21.15.134:0/3435242927 reap_dead start 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 shutdown_connections 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 --2- 172.21.15.134:0/3435242927 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fe788136e40 0x7fe788134e80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.966+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 >> 172.21.15.134:0/3435242927 conn(0x7fe7880a6110 msgr2=0x7fe78809f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.967+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 shutdown_connections 2024-09-16T15:19:10.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:09.967+0000 7fe78e872640 1 -- 172.21.15.134:0/3435242927 wait complete. 2024-09-16T15:19:10.002 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.002 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-16T15:19:10.003 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3833: done (3 left) 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3834: done (2 left) 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3835: done (1 left) 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3836: done (0 left) 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3837: read oid 25 snap -1 2024-09-16T15:19:10.084 INFO:tasks.rados.rados.0.smithi162.stdout:3837: expect (ObjNum 1276 snap 396 seq_num 1276) 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3838: read oid 24 snap -1 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3838: expect deleted 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3839: read oid 33 snap -1 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3839: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3840: read oid 28 snap 402 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3840: expect deleted 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3841: write (excl) oid 31 current snap is 406 2024-09-16T15:19:10.085 INFO:tasks.rados.rados.0.smithi162.stdout:3841: seq_num 1301 ranges {665907=595312,1781191=645446,3177196=688711} 2024-09-16T15:19:10.091 INFO:tasks.rados.rados.0.smithi162.stdout:3841: writing smithi16234619-31 from 665907 to 1261219 tid 1 2024-09-16T15:19:10.097 INFO:tasks.rados.rados.0.smithi162.stdout:3841: writing smithi16234619-31 from 1781191 to 2426637 tid 2 2024-09-16T15:19:10.105 INFO:tasks.rados.rados.0.smithi162.stdout:3841: writing smithi16234619-31 from 3177196 to 3865907 tid 3 2024-09-16T15:19:10.105 INFO:tasks.rados.rados.0.smithi162.stdout:3842: delete oid 7 current snap is 406 2024-09-16T15:19:10.105 INFO:tasks.rados.rados.0.smithi162.stdout:3839: expect (ObjNum 1248 snap 386 seq_num 1248) 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.105+0000 7fd546e3a640 10 monclient: get_monmap_and_config 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.105+0000 7fd546e3a640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.105+0000 7fd546e3a640 10 monclient: monmap: 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:10.106162+0000 2024-09-16T15:19:10.106 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:10.106162+0000 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:10.107 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.105+0000 7fd546e3a640 1 Processor -- start 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 -- start start 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient: init 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient: _add_conns ranks=[2,10,1,6,3,5,0,4,8,9,7] 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-b con 0x7fd540136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:10.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd540137220 0x7fd540137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-f con 0x7fd540137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd540135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540073250 con 0x7fd540135dc0 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54009abc0 con 0x7fd540137220 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54009ba50 con 0x7fd540136e40 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd546e3a640 10 monclient(hunting): authenticate will time out at 1886.695730s 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51542/0 (socket says 172.21.15.134:51542) 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 1 -- 172.21.15.134:0/919413322 learned_addr learned my addr 172.21.15.134:0/919413322 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 10 monclient(hunting): get_auth_request con 0x7fd540136e40 auth_method 0 2024-09-16T15:19:10.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd544bd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.106+0000 7fd537fff640 1 --2- 172.21.15.134:0/919413322 >> v2:172.21.15.162:3300/0 conn(0x7fd540137220 0x7fd540137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd537fff640 10 monclient(hunting): get_auth_request con 0x7fd540137220 auth_method 0 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd537fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd537fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient(hunting): handle_auth_done global_id 114572 payload 274 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 1 -- 172.21.15.134:0/919413322 >> v2:172.21.15.162:3300/0 conn(0x7fd540137220 msgr2=0x7fd540137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 1 --2- 172.21.15.134:0/919413322 >> v2:172.21.15.162:3300/0 conn(0x7fd540137220 0x7fd540137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 1 -- 172.21.15.134:0/919413322 >> v1:172.21.15.162:6789/0 conn(0x7fd540135dc0 legacy=0x7fd540136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 1 monclient: found mon.noname-b 2024-09-16T15:19:10.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:10.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.107+0000 7fd544bd6640 1 -- 172.21.15.134:0/919413322 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400a53c0 con 0x7fd540136e40 2024-09-16T15:19:10.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd544bd6640 1 -- 172.21.15.134:0/919413322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 msgr2=0x7fd54009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd544bd6640 1 --2- 172.21.15.134:0/919413322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0098a0 tx=0x7fd53c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3a4799677faf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 1 -- 172.21.15.134:0/919413322 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd53c02ae30 con 0x7fd540136e40 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: dump: 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.119 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 1 -- 172.21.15.134:0/919413322 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c0307b0 con 0x7fd540136e40 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 5 monclient: authenticate success, global_id 114572 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:10.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 1 -- 172.21.15.134:0/919413322 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd53c030d90 con 0x7fd540136e40 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 4 set_mon_vals no callback set 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd5377fe640 10 monclient: dump: 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.121 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 10 monclient: shutdown 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 1 -- 172.21.15.134:0/919413322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 msgr2=0x7fd54009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.110+0000 7fd546e3a640 1 --2- 172.21.15.134:0/919413322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0098a0 tx=0x7fd53c02a110 comp rx=0 tx=0).stop 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 -- 172.21.15.134:0/919413322 shutdown_connections 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 --2- 172.21.15.134:0/919413322 >> v2:172.21.15.162:3300/0 conn(0x7fd540137220 0x7fd540137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 --2- 172.21.15.134:0/919413322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd540136e40 0x7fd54009c150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 -- 172.21.15.134:0/919413322 >> 172.21.15.134:0/919413322 conn(0x7fd5400a6110 msgr2=0x7fd5400a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 -- 172.21.15.134:0/919413322 shutdown_connections 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 1 -- 172.21.15.134:0/919413322 wait complete. 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.111+0000 7fd546e3a640 10 monclient: monmap: 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.123 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 Processor -- start 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 -- start start 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient: init 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient: _add_conns ranks=[6,5,3,2,0,1,4,7,8] 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 0x7fd540134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-d con 0x7fd540135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:10.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd540136e40 0x7fd540135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-h con 0x7fd540136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd540137220 0x7fd5401cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): picked mon.noname-g con 0x7fd540137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540120d20 con 0x7fd540137220 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401225f0 con 0x7fd540136e40 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401364e0 con 0x7fd540135dc0 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd546e3a640 10 monclient(hunting): authenticate will time out at 1886.701777s 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 0x7fd540134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 0x7fd540134f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52842/0 (socket says 172.21.15.134:52842) 2024-09-16T15:19:10.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 1 -- 172.21.15.134:0/4008458412 learned_addr learned my addr 172.21.15.134:0/4008458412 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): get_auth_request con 0x7fd540135dc0 auth_method 0 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd537fff640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd540136e40 0x7fd540135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.112+0000 7fd5453d7640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd540137220 0x7fd5401cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): get_auth_request con 0x7fd540136e40 auth_method 0 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): get_auth_request con 0x7fd540137220 auth_method 0 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd537fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 10 monclient(hunting): handle_auth_done global_id 94704 payload 824 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd540136e40 msgr2=0x7fd540135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd540136e40 0x7fd540135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd540137220 msgr2=0x7fd5401cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd540137220 0x7fd5401cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 monclient: found mon.noname-d 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.113+0000 7fd544bd6640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c0094d0 con 0x7fd540135dc0 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd544bd6640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 msgr2=0x7fd540134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd544bd6640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 0x7fd540134f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0090b0 tx=0x7fd53c0046c0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd53c030d40 con 0x7fd540135dc0 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: dump: 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.128 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c003a80 con 0x7fd540135dc0 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 5 monclient: authenticate success, global_id 94704 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd53c003bf0 con 0x7fd540135dc0 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd5357fa640 10 monclient: dump: 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.130 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 10 monclient: _renew_subs 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd536ffd640 4 set_mon_vals no callback set 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400a3560 con 0x7fd540135dc0 2024-09-16T15:19:10.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 10 monclient: _renew_subs 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.115+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd540122310 con 0x7fd540135dc0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd53c039070 con 0x7fd540135dc0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd546e3a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd5357fa640 1 --2- 172.21.15.134:0/4008458412 >> v2:172.21.15.162:6800/980547263 conn(0x7fd52c0388e0 0x7fd52c03ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd546e3a640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 0x7fd54002b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd5401224a0 con 0x7fd540120130 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.116+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(830..830 src has 1..830) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fd53c06aa50 con 0x7fd540135dc0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 0x7fd54002b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient: get_auth_request con 0x7fd540120130 auth_method 0 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd537fff640 1 --2- 172.21.15.134:0/4008458412 >> v2:172.21.15.162:6800/980547263 conn(0x7fd52c0388e0 0x7fd52c03ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd537fff640 10 monclient: get_auth_request con 0x7fd52c0388e0 auth_method 0 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 10 monclient(hunting): handle_auth_done global_id 94713 payload 824 2024-09-16T15:19:10.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.117+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd540061ae0 con 0x7fd540120130 2024-09-16T15:19:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 msgr2=0x7fd54002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5453d7640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 0x7fd54002b260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd53800f890 tx=0x7fd53800fd50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.135 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd53800dca0 con 0x7fd540120130 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd537fff640 1 --2- 172.21.15.134:0/4008458412 >> v2:172.21.15.162:6800/980547263 conn(0x7fd52c0388e0 0x7fd52c03ada0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd5400a29d0 tx=0x7fd530008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 10 monclient: dump: 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.136 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd53800de00 con 0x7fd540120130 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 msgr2=0x7fd54002b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 0x7fd54002b260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd53800f890 tx=0x7fd53800fd50 comp rx=0 tx=0).stop 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 msgr2=0x7fd54002b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.119+0000 7fd5357fa640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540120130 0x7fd54002b260 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd53800f890 tx=0x7fd53800fd50 comp rx=0 tx=0).stop 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd546e3a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:10.137 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd546e3a640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 0x7fd54002bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd546e3a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd540071b20 con 0x7fd54002b7a0 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 0x7fd54002bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient: get_auth_request con 0x7fd54002b7a0 auth_method 0 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 10 monclient(hunting): handle_auth_done global_id 94722 payload 824 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.127+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd54007a680 con 0x7fd54002b7a0 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.129+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 msgr2=0x7fd54002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.129+0000 7fd5453d7640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 0x7fd54002bbb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd538018000 tx=0x7fd538003ae0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd538022070 con 0x7fd54002b7a0 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 10 monclient: dump: 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.139 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9351 (secure 0 0 0) 0x7fd538014bf0 con 0x7fd54002b7a0 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 msgr2=0x7fd54002bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 0x7fd54002bbb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd538018000 tx=0x7fd538003ae0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 msgr2=0x7fd54002bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.140 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5357fa640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd54002b7a0 0x7fd54002bbb0 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd538018000 tx=0x7fd538003ae0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:10.141 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.142 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:10.144 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.145 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.147 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.154 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.155 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.156 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:10.157 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.158 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.159 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.160 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.161 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 >> v2:172.21.15.162:6800/980547263 conn(0x7fd52c0388e0 msgr2=0x7fd52c03ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 1 --2- 172.21.15.134:0/4008458412 >> v2:172.21.15.162:6800/980547263 conn(0x7fd52c0388e0 0x7fd52c03ada0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd5400a29d0 tx=0x7fd530008040 comp rx=0 tx=0).stop 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 10 monclient: shutdown 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 msgr2=0x7fd540134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd546e3a640 1 --2- 172.21.15.134:0/4008458412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd540135dc0 0x7fd540134f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0090b0 tx=0x7fd53c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 reap_dead start 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.130+0000 7fd5453d7640 1 -- 172.21.15.134:0/4008458412 reap_dead start 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.131+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 shutdown_connections 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.131+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 >> 172.21.15.134:0/4008458412 conn(0x7fd5400a6110 msgr2=0x7fd54009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.131+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 shutdown_connections 2024-09-16T15:19:10.164 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.131+0000 7fd546e3a640 1 -- 172.21.15.134:0/4008458412 wait complete. 2024-09-16T15:19:10.164 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.164 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-16T15:19:10.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:19:10.200 INFO:tasks.rados.rados.0.smithi162.stdout:3837: expect (ObjNum 1276 snap 396 seq_num 1276) 2024-09-16T15:19:10.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 10 monclient: get_monmap_and_config 2024-09-16T15:19:10.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 10 monclient: monmap: 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:10.262289+0000 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:10.262289+0000 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.263 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.264 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:10.264 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:10.264 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 1 Processor -- start 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 1 -- start start 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.261+0000 7ff6e5296640 10 monclient: init 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient: _add_conns ranks=[2,1,6,9,8,3,0,10,5,7,4] 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-b con 0x7ff6e0136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff6e0137220 addr v1:172.21.15.162:6789/0 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-d con 0x7ff6e013c370 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e0073250 con 0x7ff6e0137220 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00a56e0 con 0x7ff6e0136e40 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e009abc0 con 0x7ff6e013c370 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6e5296640 10 monclient(hunting): authenticate will time out at 1886.851525s 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51544/0 (socket says 172.21.15.134:51544) 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52854/0 (socket says 172.21.15.134:52854) 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 -- 172.21.15.134:0/1970885412 learned_addr learned my addr 172.21.15.134:0/1970885412 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): get_auth_request con 0x7ff6e0136e40 auth_method 0 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): get_auth_request con 0x7ff6e013c370 auth_method 0 2024-09-16T15:19:10.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient(hunting): handle_auth_done global_id 114581 payload 274 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 -- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 msgr2=0x7ff6e013e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 -- 172.21.15.134:0/1970885412 >> v1:172.21.15.162:6789/0 conn(0x7ff6e0137220 legacy=0x7ff6e0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 monclient: found mon.noname-b 2024-09-16T15:19:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6deffd640 1 -- 172.21.15.134:0/1970885412 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e009ba50 con 0x7ff6e0136e40 2024-09-16T15:19:10.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.262+0000 7ff6df7fe640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6deffd640 1 -- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 msgr2=0x7ff6e009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6deffd640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009e80 tx=0x7ff6d002a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a19ee6810889fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 1 -- 172.21.15.134:0/1970885412 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0007a70 con 0x7ff6e0136e40 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: dump: 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.275 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6e5296640 5 monclient: authenticate success, global_id 114581 2024-09-16T15:19:10.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 1 -- 172.21.15.134:0/1970885412 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d002d050 con 0x7ff6e0136e40 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 1 -- 172.21.15.134:0/1970885412 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0007bf0 con 0x7ff6e0136e40 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6ddffb640 10 monclient: dump: 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.277 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6e5296640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.265+0000 7ff6e5296640 4 set_mon_vals no callback set 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 10 monclient: shutdown 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 -- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 msgr2=0x7ff6e009c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009e80 tx=0x7ff6d002a580 comp rx=0 tx=0).stop 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 -- 172.21.15.134:0/1970885412 shutdown_connections 2024-09-16T15:19:10.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7ff6e013c370 0x7ff6e013e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 --2- 172.21.15.134:0/1970885412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e0136e40 0x7ff6e009c150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 -- 172.21.15.134:0/1970885412 >> 172.21.15.134:0/1970885412 conn(0x7ff6e00a6110 msgr2=0x7ff6e00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 -- 172.21.15.134:0/1970885412 shutdown_connections 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.266+0000 7ff6e5296640 1 -- 172.21.15.134:0/1970885412 wait complete. 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 10 monclient: monmap: 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.279 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 1 Processor -- start 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 1 -- start start 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 10 monclient: init 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.267+0000 7ff6e5296640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient: _add_conns ranks=[5,4,2,6,7,0,3,1,8] 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6e0136e40 0x7ff6e0135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-h con 0x7ff6e0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6e0137220 0x7ff6e01363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-c con 0x7ff6e0137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 0x7ff6e011fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): picked mon.noname-b con 0x7ff6e013c370 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01228b0 con 0x7ff6e013c370 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00a7c40 con 0x7ff6e0137220 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00a7710 con 0x7ff6e0136e40 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6e5296640 10 monclient(hunting): authenticate will time out at 1886.857073s 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 0x7ff6e011fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6e0137220 0x7ff6e01363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 0x7ff6e011fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51548/0 (socket says 172.21.15.134:51548) 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 learned_addr learned my addr 172.21.15.134:0/927911081 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): get_auth_request con 0x7ff6e013c370 auth_method 0 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): get_auth_request con 0x7ff6e0137220 auth_method 0 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6e0136e40 0x7ff6e0135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): get_auth_request con 0x7ff6e0136e40 auth_method 0 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6de7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient(hunting): handle_auth_done global_id 114590 payload 824 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6e0136e40 msgr2=0x7ff6e0135e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7ff6e0136e40 0x7ff6e0135e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6e0137220 msgr2=0x7ff6e01363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ff6e0137220 0x7ff6e01363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 monclient: found mon.noname-b 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.268+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8009400 con 0x7ff6e013c370 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 msgr2=0x7ff6e011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6df7fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 0x7ff6e011fc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800e850 tx=0x7ff6d800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d8009640 con 0x7ff6e013c370 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: dump: 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.284 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8013470 con 0x7ff6e013c370 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 5 monclient: authenticate success, global_id 114590 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 10 monclient: _renew_subs 2024-09-16T15:19:10.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0009b30 con 0x7ff6e013c370 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d8013a50 con 0x7ff6e013c370 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6d77fe640 10 monclient: dump: 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.286 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 10 monclient: _renew_subs 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6dd7fa640 4 set_mon_vals no callback set 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.270+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e0121fa0 con 0x7ff6e013c370 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6e5296640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6e5296640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 0x7ff6e01201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0071b20 con 0x7ff6e01221c0 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff6d8013bb0 con 0x7ff6e013c370 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 0x7ff6e01201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient: get_auth_request con 0x7ff6e01221c0 auth_method 0 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.271+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/927911081 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c0038880 0x7ff6c003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(830..830 src has 1..830) v4 ==== 3931+0+0 (secure 0 0 0) 0x7ff6d804a7d0 con 0x7ff6e013c370 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6deffd640 10 monclient(hunting): handle_auth_done global_id 85031 payload 824 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6deffd640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6e0061ae0 con 0x7ff6e01221c0 2024-09-16T15:19:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6de7fc640 1 --2- 172.21.15.134:0/927911081 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c0038880 0x7ff6c003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.272+0000 7ff6de7fc640 10 monclient: get_auth_request con 0x7ff6c0038880 auth_method 0 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6deffd640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 msgr2=0x7ff6e01201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6deffd640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 0x7ff6e01201a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0033000 tx=0x7ff6d0004a40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0007d20 con 0x7ff6e01221c0 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 10 monclient: dump: 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.291 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6de7fc640 1 --2- 172.21.15.134:0/927911081 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c0038880 0x7ff6c003ad40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00a3020 tx=0x7ff6c8007e30 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff6d0007e80 con 0x7ff6e01221c0 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 msgr2=0x7ff6e01201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 0x7ff6e01201a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0033000 tx=0x7ff6d0004a40 comp rx=0 tx=0).stop 2024-09-16T15:19:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 msgr2=0x7ff6e01201a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.274+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01221c0 0x7ff6e01201a0 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0033000 tx=0x7ff6d0004a40 comp rx=0 tx=0).stop 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6e5296640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6e5296640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 0x7ff6e009d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6e5296640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e007a680 con 0x7ff6e01206e0 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 0x7ff6e009d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient: get_auth_request con 0x7ff6e01206e0 auth_method 0 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.280+0000 7ff6deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.281+0000 7ff6deffd640 10 monclient(hunting): handle_auth_done global_id 85040 payload 824 2024-09-16T15:19:10.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.281+0000 7ff6deffd640 1 -- 172.21.15.134:0/927911081 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff6e00a53c0 con 0x7ff6e01206e0 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6deffd640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 msgr2=0x7ff6e009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6deffd640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 0x7ff6e009d4f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000f80 tx=0x7ff6d002f000 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff6d0004790 con 0x7ff6e01206e0 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 10 monclient: dump: 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.294 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7ff6d00048f0 con 0x7ff6e01206e0 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 msgr2=0x7ff6e009d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 0x7ff6e009d4f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000f80 tx=0x7ff6d002f000 comp rx=0 tx=0).stop 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 msgr2=0x7ff6e009d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6d77fe640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ff6e01206e0 0x7ff6e009d4f0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000f80 tx=0x7ff6d002f000 comp rx=0 tx=0).stop 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:10.296 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.297 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.298 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:10.299 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.300 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:10.301 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.304 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.306 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.307 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:10.308 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.310 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.311 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.312 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.313 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.314 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.315 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.316 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.317 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c0038880 msgr2=0x7ff6c003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 1 --2- 172.21.15.134:0/927911081 >> v2:172.21.15.162:6800/980547263 conn(0x7ff6c0038880 0x7ff6c003ad40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00a3020 tx=0x7ff6c8007e30 comp rx=0 tx=0).stop 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 10 monclient: shutdown 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 msgr2=0x7ff6e011fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6e5296640 1 --2- 172.21.15.134:0/927911081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6e013c370 0x7ff6e011fc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800e850 tx=0x7ff6d800ed10 comp rx=0 tx=0).stop 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.283+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 reap_dead start 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.284+0000 7ff6df7fe640 1 -- 172.21.15.134:0/927911081 reap_dead start 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.284+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 shutdown_connections 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.284+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 >> 172.21.15.134:0/927911081 conn(0x7ff6e00a6110 msgr2=0x7ff6e013b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.284+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 shutdown_connections 2024-09-16T15:19:10.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.284+0000 7ff6e5296640 1 -- 172.21.15.134:0/927911081 wait complete. 2024-09-16T15:19:10.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.320 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-16T15:19:10.320 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.416+0000 7f66eacec640 10 monclient: get_monmap_and_config 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.416+0000 7f66eacec640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.416+0000 7f66eacec640 10 monclient: monmap: 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:10.417803+0000 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:10.417803+0000 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.418 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:10.419 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.417+0000 7f66eacec640 1 Processor -- start 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.417+0000 7f66eacec640 1 -- start start 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.417+0000 7f66eacec640 10 monclient: init 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient: _add_conns ranks=[1,4,10,6,3,9,8,7,5,2,0] 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f66e4136e40 addr v1:172.21.15.162:6789/0 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-c con 0x7f66e4137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66e413c370 0x7f66e413e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-f con 0x7f66e413c370 addr v2:172.21.15.162:3300/0 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e4073250 con 0x7f66e4136e40 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e40a56e0 con 0x7f66e413c370 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66e409abc0 con 0x7f66e4137220 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66eacec640 10 monclient(hunting): authenticate will time out at 1887.007196s 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e8a88640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f66e4136e40 0x7f66e409c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.134:55846/0 (socket says 172.21.15.134:55846) 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:37030/0 (socket says 172.21.15.134:37030) 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e8a88640 1 -- 172.21.15.134:0/3664054716 learned_addr learned my addr 172.21.15.134:0/3664054716 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): get_auth_request con 0x7f66e4137220 auth_method 0 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e9289640 1 --2- 172.21.15.134:0/3664054716 >> v2:172.21.15.162:3300/0 conn(0x7f66e413c370 0x7f66e413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e9289640 10 monclient(hunting): get_auth_request con 0x7f66e413c370 auth_method 0 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e9289640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e9289640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66e9289640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient(hunting): handle_auth_done global_id 124321 payload 274 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 -- 172.21.15.134:0/3664054716 >> v2:172.21.15.162:3300/0 conn(0x7f66e413c370 msgr2=0x7f66e413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 --2- 172.21.15.134:0/3664054716 >> v2:172.21.15.162:3300/0 conn(0x7f66e413c370 0x7f66e413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 -- 172.21.15.134:0/3664054716 >> v1:172.21.15.162:6789/0 conn(0x7f66e4136e40 legacy=0x7f66e409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 monclient: found mon.noname-c 2024-09-16T15:19:10.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:19:10.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.418+0000 7f66dbfff640 1 -- 172.21.15.134:0/3664054716 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e409ba50 con 0x7f66e4137220 2024-09-16T15:19:10.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66dbfff640 1 -- 172.21.15.134:0/3664054716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 msgr2=0x7f66e4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66dbfff640 1 --2- 172.21.15.134:0/3664054716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402a010 comp rx=0 tx=0).ready entity=mon.4 client_cookie=13ecb417e294ad07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 1 -- 172.21.15.134:0/3664054716 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d402ad30 con 0x7f66e4137220 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: dump: 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.431 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 1 -- 172.21.15.134:0/3664054716 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d40307b0 con 0x7f66e4137220 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66eacec640 5 monclient: authenticate success, global_id 124321 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66eacec640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 1 -- 172.21.15.134:0/3664054716 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d4030d90 con 0x7f66e4137220 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66db7fe640 10 monclient: dump: 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.433 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66eacec640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.421+0000 7f66eacec640 4 set_mon_vals no callback set 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 10 monclient: shutdown 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 -- 172.21.15.134:0/3664054716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 msgr2=0x7f66e4137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 --2- 172.21.15.134:0/3664054716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402a010 comp rx=0 tx=0).stop 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 -- 172.21.15.134:0/3664054716 shutdown_connections 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 --2- 172.21.15.134:0/3664054716 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f66e4137220 0x7f66e4137600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 --2- 172.21.15.134:0/3664054716 >> v2:172.21.15.162:3300/0 conn(0x7f66e413c370 0x7f66e413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 -- 172.21.15.134:0/3664054716 >> 172.21.15.134:0/3664054716 conn(0x7f66e40a6110 msgr2=0x7f66e40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 -- 172.21.15.134:0/3664054716 shutdown_connections 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 1 -- 172.21.15.134:0/3664054716 wait complete. 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.422+0000 7f66eacec640 10 monclient: monmap: 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.435 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 Processor -- start 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 -- start start 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient: init 2024-09-16T15:19:10.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient: _add_conns ranks=[8,5,3,6,0,2,1,7,4] 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 0x7f66e411fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-e con 0x7f66e4136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f66e4137220 0x7f66e40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-h con 0x7f66e4137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66e413c370 0x7f66e4135f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): picked mon.noname-g con 0x7f66e413c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4120d00 con 0x7f66e413c370 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4122010 con 0x7f66e4137220 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40a7b60 con 0x7f66e4136e40 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66eacec640 10 monclient(hunting): authenticate will time out at 1887.012853s 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 0x7f66e411fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 0x7f66e411fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:40170/0 (socket says 172.21.15.134:40170) 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 1 -- 172.21.15.134:0/2530956347 learned_addr learned my addr 172.21.15.134:0/2530956347 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 10 monclient(hunting): get_auth_request con 0x7f66e4136e40 auth_method 0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.423+0000 7f66e8a88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f66e4137220 0x7f66e40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): get_auth_request con 0x7f66e4137220 auth_method 0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66e413c370 0x7f66e4135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): get_auth_request con 0x7f66e413c370 auth_method 0 2024-09-16T15:19:10.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e9289640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient(hunting): handle_auth_done global_id 85049 payload 824 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f66e4137220 msgr2=0x7f66e40a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f66e4137220 0x7f66e40a2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66e413c370 msgr2=0x7f66e4135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66e413c370 0x7f66e4135f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 monclient: found mon.noname-e 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.424+0000 7f66e8a88640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d4009400 con 0x7f66e4136e40 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66e8a88640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 msgr2=0x7f66e411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66e8a88640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 0x7f66e411fc60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c460 tx=0x7f66e000c920 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66e0005cf0 con 0x7f66e4136e40 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:19:10.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: dump: 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.441 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e0005e90 con 0x7f66e4136e40 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 5 monclient: authenticate success, global_id 85049 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66e00074e0 con 0x7f66e4136e40 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 10 monclient: _renew_subs 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e40a75a0 con 0x7f66e4136e40 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:10.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66daffd640 4 set_mon_vals no callback set 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66d97fa640 10 monclient: dump: 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.443 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 10 monclient: _renew_subs 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.426+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e41228b0 con 0x7f66e4136e40 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66eacec640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66eacec640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 0x7f66e41cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4073e50 con 0x7f66e4136440 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f66e0007070 con 0x7f66e4136e40 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 0x7f66e41cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient: get_auth_request con 0x7f66e4136440 auth_method 0 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66d97fa640 1 --2- 172.21.15.134:0/2530956347 >> v2:172.21.15.162:6800/980547263 conn(0x7f66d00388b0 0x7f66d003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(830..830 src has 1..830) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f66e0049be0 con 0x7f66e4136e40 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.427+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.428+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.428+0000 7f66e9289640 1 --2- 172.21.15.134:0/2530956347 >> v2:172.21.15.162:6800/980547263 conn(0x7f66d00388b0 0x7f66d003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.428+0000 7f66e9289640 10 monclient: get_auth_request con 0x7f66d00388b0 auth_method 0 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.428+0000 7f66dbfff640 10 monclient(hunting): handle_auth_done global_id 124291 payload 824 2024-09-16T15:19:10.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.428+0000 7f66dbfff640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66e4061ae0 con 0x7f66e4136440 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3841: finishing write tid 1 to smithi16234619-31 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3841: finishing write tid 2 to smithi16234619-31 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3841: finishing write tid 3 to smithi16234619-31 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3841: finishing write tid 4 to smithi16234619-31 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3837: done (5 left) 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3838: done (4 left) 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3839: done (3 left) 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3840: done (2 left) 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3842: done (1 left) 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3843: write oid 16 current snap is 406 2024-09-16T15:19:10.447 INFO:tasks.rados.rados.0.smithi162.stdout:3841: finishing write tid 5 to smithi16234619-31 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3841: oid 31 updating version 0 to 2113 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3841: oid 31 updating version 2113 to 2116 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3841: oid 31 version 2116 is already newer than 2114 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3841: oid 31 version 2116 is already newer than 2115 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 31 v 2116 (ObjNum 1301 snap 406 seq_num 1301) dirty exists 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3841: left oid 31 (ObjNum 1301 snap 406 seq_num 1301) 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3843: seq_num 1303 ranges {699521=745810,2181802=649115,3515689=383832} 2024-09-16T15:19:10.448 INFO:tasks.rados.rados.0.smithi162.stdout:3843: writing smithi16234619-16 from 699521 to 1445331 tid 1 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66dbfff640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 msgr2=0x7f66e41cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66dbfff640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 0x7f66e41cbe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66d4034000 tx=0x7f66d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66e9289640 1 --2- 172.21.15.134:0/2530956347 >> v2:172.21.15.162:6800/980547263 conn(0x7f66d00388b0 0x7f66d003ad70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f66e406b7d0 tx=0x7f66dc009780 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d4030d40 con 0x7f66e4136440 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:19:10.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 10 monclient: dump: 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.451 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f66d4003610 con 0x7f66e4136440 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 msgr2=0x7f66e41cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 0x7f66e41cbe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66d4034000 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 msgr2=0x7f66e41cbe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.430+0000 7f66d97fa640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e4136440 0x7f66e41cbe30 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66d4034000 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.436+0000 7f66eacec640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.436+0000 7f66eacec640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 0x7f66e402b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66eacec640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4071b20 con 0x7f66e402b260 2024-09-16T15:19:10.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 0x7f66e402b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient: get_auth_request con 0x7f66e402b260 auth_method 0 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 10 monclient(hunting): handle_auth_done global_id 124300 payload 824 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.437+0000 7f66dbfff640 1 -- 172.21.15.134:0/2530956347 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66e407a680 con 0x7f66e402b260 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66dbfff640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 msgr2=0x7f66e402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66dbfff640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 0x7f66e402b670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f66d40096f0 tx=0x7f66d4009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f66d404d070 con 0x7f66e402b260 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.439+0000 7f66d97fa640 10 monclient: dump: 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.454 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7f66d4032440 con 0x7f66e402b260 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 msgr2=0x7f66e402b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 0x7f66e402b670 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f66d40096f0 tx=0x7f66d4009190 comp rx=0 tx=0).stop 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 msgr2=0x7f66e402b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66d97fa640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66e402b260 0x7f66e402b670 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f66d40096f0 tx=0x7f66d4009190 comp rx=0 tx=0).stop 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.455 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:19:10.456 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:10.457 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:10.458 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.459 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:10.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:10.462 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.463 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.464 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.465 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.466 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.467 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.468 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.469 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.471 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.472 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.473 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:10.474 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.475 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 >> v2:172.21.15.162:6800/980547263 conn(0x7f66d00388b0 msgr2=0x7f66d003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 --2- 172.21.15.134:0/2530956347 >> v2:172.21.15.162:6800/980547263 conn(0x7f66d00388b0 0x7f66d003ad70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f66e406b7d0 tx=0x7f66dc009780 comp rx=0 tx=0).stop 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 10 monclient: shutdown 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 msgr2=0x7f66e411fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 --2- 172.21.15.134:0/2530956347 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f66e4136e40 0x7f66e411fc60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c460 tx=0x7f66e000c920 comp rx=0 tx=0).stop 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66e9289640 1 -- 172.21.15.134:0/2530956347 reap_dead start 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66e9289640 1 -- 172.21.15.134:0/2530956347 reap_dead start 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 shutdown_connections 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 >> 172.21.15.134:0/2530956347 conn(0x7f66e40a6110 msgr2=0x7f66e413a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 shutdown_connections 2024-09-16T15:19:10.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.440+0000 7f66eacec640 1 -- 172.21.15.134:0/2530956347 wait complete. 2024-09-16T15:19:10.478 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.478 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-16T15:19:10.478 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:19:10.479 INFO:tasks.rados.rados.0.smithi162.stdout:3843: writing smithi16234619-16 from 2181802 to 2830917 tid 2 2024-09-16T15:19:10.479 INFO:tasks.rados.rados.0.smithi162.stdout:3843: writing smithi16234619-16 from 3515689 to 3899521 tid 3 2024-09-16T15:19:10.479 INFO:tasks.rados.rados.0.smithi162.stdout:3841: done (1 left) 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3844: write oid 18 current snap is 406 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3844: seq_num 1304 ranges {716328=421059,1840284=509126,3143112=659293} 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3844: writing smithi16234619-18 from 716328 to 1137387 tid 1 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3844: writing smithi16234619-18 from 1840284 to 2349410 tid 2 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3844: writing smithi16234619-18 from 3143112 to 3802405 tid 3 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3845: write oid 6 current snap is 406 2024-09-16T15:19:10.480 INFO:tasks.rados.rados.0.smithi162.stdout:3845: seq_num 1305 ranges {733135=496308,1898766=652784,3170535=470565} 2024-09-16T15:19:10.482 INFO:tasks.rados.rados.0.smithi162.stdout:3845: writing smithi16234619-6 from 733135 to 1229443 tid 1 2024-09-16T15:19:10.490 INFO:tasks.rados.rados.0.smithi162.stdout:3845: writing smithi16234619-6 from 1898766 to 2551550 tid 2 2024-09-16T15:19:10.496 INFO:tasks.rados.rados.0.smithi162.stdout:3845: writing smithi16234619-6 from 3170535 to 3641100 tid 3 2024-09-16T15:19:10.496 INFO:tasks.rados.rados.0.smithi162.stdout:3846: delete oid 13 current snap is 406 2024-09-16T15:19:10.496 INFO:tasks.rados.rados.0.smithi162.stdout:3844: finishing write tid 1 to smithi16234619-18 2024-09-16T15:19:10.501 INFO:tasks.rados.rados.0.smithi162.stdout:3844: finishing write tid 2 to smithi16234619-18 2024-09-16T15:19:10.503 INFO:tasks.rados.rados.0.smithi162.stdout:3844: finishing write tid 3 to smithi16234619-18 2024-09-16T15:19:10.503 INFO:tasks.rados.rados.0.smithi162.stdout:3844: finishing write tid 4 to smithi16234619-18 2024-09-16T15:19:10.503 INFO:tasks.rados.rados.0.smithi162.stdout:3844: finishing write tid 5 to smithi16234619-18 2024-09-16T15:19:10.503 INFO:tasks.rados.rados.0.smithi162.stdout:3844: oid 18 updating version 0 to 2119 2024-09-16T15:19:10.504 INFO:tasks.rados.rados.0.smithi162.stdout:3844: oid 18 updating version 2119 to 2121 2024-09-16T15:19:10.504 INFO:tasks.rados.rados.0.smithi162.stdout:3844: oid 18 version 2121 is already newer than 2120 2024-09-16T15:19:10.504 INFO:tasks.rados.rados.0.smithi162.stdout:3844: oid 18 version 2121 is already newer than 2118 2024-09-16T15:19:10.504 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 18 v 2121 (ObjNum 1304 snap 406 seq_num 1304) dirty exists 2024-09-16T15:19:10.504 INFO:tasks.rados.rados.0.smithi162.stdout:3844: left oid 18 (ObjNum 1304 snap 406 seq_num 1304) 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.577+0000 7fe14303a640 10 monclient: get_monmap_and_config 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.577+0000 7fe14303a640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.577+0000 7fe14303a640 10 monclient: monmap: 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:10.578837+0000 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:10.578837+0000 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.579 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:10.580 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.578+0000 7fe14303a640 1 Processor -- start 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.578+0000 7fe14303a640 1 -- start start 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.578+0000 7fe14303a640 10 monclient: init 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient: _add_conns ranks=[5,6,1,4,9,3,2,10,0,8,7] 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe13c136e40 0x7fe13c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-h con 0x7fe13c136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-d con 0x7fe13c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fe13c135dc0 addr v1:172.21.15.162:6789/0 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13c073250 con 0x7fe13c135dc0 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c09abc0 con 0x7fe13c136e40 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c09ba50 con 0x7fe13c137220 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe14303a640 10 monclient(hunting): authenticate will time out at 1887.168209s 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52868/0 (socket says 172.21.15.134:52868) 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 -- 172.21.15.134:0/1835862333 learned_addr learned my addr 172.21.15.134:0/1835862333 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): get_auth_request con 0x7fe13c137220 auth_method 0 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe13c136e40 0x7fe13c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): get_auth_request con 0x7fe13c136e40 auth_method 0 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient(hunting): handle_auth_done global_id 94740 payload 274 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 -- 172.21.15.134:0/1835862333 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe13c136e40 msgr2=0x7fe13c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe13c136e40 0x7fe13c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 -- 172.21.15.134:0/1835862333 >> v1:172.21.15.162:6789/0 conn(0x7fe13c135dc0 legacy=0x7fe13c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 monclient: found mon.noname-d 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:19:10.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.579+0000 7fe13bfff640 1 -- 172.21.15.134:0/1835862333 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c0a53c0 con 0x7fe13c137220 2024-09-16T15:19:10.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13bfff640 1 -- 172.21.15.134:0/1835862333 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 msgr2=0x7fe13c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13bfff640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe1300098d0 tx=0x7fe13002a100 comp rx=0 tx=0).ready entity=mon.6 client_cookie=b52a2c6bf2eee23d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 1 -- 172.21.15.134:0/1835862333 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe13002ae20 con 0x7fe13c137220 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: dump: 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.592 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 1 -- 172.21.15.134:0/1835862333 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300307b0 con 0x7fe13c137220 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe14303a640 5 monclient: authenticate success, global_id 94740 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe14303a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.593 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe14303a640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe14303a640 4 set_mon_vals no callback set 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 1 -- 172.21.15.134:0/1835862333 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe130030d90 con 0x7fe13c137220 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.582+0000 7fe13b7fe640 10 monclient: dump: 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.594 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 10 monclient: shutdown 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 -- 172.21.15.134:0/1835862333 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 msgr2=0x7fe13c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe1300098d0 tx=0x7fe13002a100 comp rx=0 tx=0).stop 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 -- 172.21.15.134:0/1835862333 shutdown_connections 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c137600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 --2- 172.21.15.134:0/1835862333 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fe13c136e40 0x7fe13c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 -- 172.21.15.134:0/1835862333 >> 172.21.15.134:0/1835862333 conn(0x7fe13c0a6110 msgr2=0x7fe13c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 -- 172.21.15.134:0/1835862333 shutdown_connections 2024-09-16T15:19:10.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 1 -- 172.21.15.134:0/1835862333 wait complete. 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.583+0000 7fe14303a640 10 monclient: monmap: 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.596 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 Processor -- start 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 -- start start 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient: init 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient: _add_conns ranks=[7,8,6,5,1,2,0,3,4] 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe13c135dc0 0x7fe13c134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-i con 0x7fe13c135dc0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 0x7fe13c1353a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-e con 0x7fe13c136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c1cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): picked mon.noname-d con 0x7fe13c137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:10.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c120c30 con 0x7fe13c137220 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c122260 con 0x7fe13c135dc0 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c1369a0 con 0x7fe13c136e40 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe14303a640 10 monclient(hunting): authenticate will time out at 1887.173810s 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe140dd6640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe13c135dc0 msgr2=0x7fe13c134e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c1cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe140dd6640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe13c135dc0 0x7fe13c134e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe13bfff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 0x7fe13c1353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c1cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:52884/0 (socket says 172.21.15.134:52884) 2024-09-16T15:19:10.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 1 -- 172.21.15.134:0/736486240 learned_addr learned my addr 172.21.15.134:0/736486240 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe13bfff640 10 monclient(hunting): get_auth_request con 0x7fe13c136e40 auth_method 0 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe13bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe13bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe13bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 10 monclient(hunting): get_auth_request con 0x7fe13c137220 auth_method 0 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.584+0000 7fe1415d7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe1415d7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe1415d7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe1415d7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.599 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient(hunting): handle_auth_done global_id 85058 payload 824 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe13c135dc0 msgr2=0x7fe13c134e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fe13c135dc0 0x7fe13c134e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 msgr2=0x7fe13c1cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c1cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 monclient: found mon.noname-e 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe13bfff640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300094d0 con 0x7fe13c136e40 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.585+0000 7fe1415d7640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fe13c137220 0x7fe13c1cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe13bfff640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 msgr2=0x7fe13c1353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe13bfff640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 0x7fe13c1353a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe130031000 tx=0x7fe1300046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe130004530 con 0x7fe13c136e40 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: dump: 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.601 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130030d40 con 0x7fe13c136e40 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 5 monclient: authenticate success, global_id 85058 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe13003d450 con 0x7fe13c136e40 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 10 monclient: _renew_subs 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c136470 con 0x7fe13c136e40 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe1397fa640 10 monclient: dump: 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.603 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe13affd640 4 set_mon_vals no callback set 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 10 monclient: _renew_subs 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.587+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c121f80 con 0x7fe13c136e40 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe14303a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe14303a640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 0x7fe13c02b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c0736c0 con 0x7fe13c121020 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fe130039070 con 0x7fe13c136e40 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe1397fa640 1 --2- 172.21.15.134:0/736486240 >> v2:172.21.15.162:6800/980547263 conn(0x7fe128038880 0x7fe12803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(830..830 src has 1..830) v4 ==== 3931+0+0 (secure 0 0 0) 0x7fe13006a870 con 0x7fe13c136e40 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 0x7fe13c02b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient: get_auth_request con 0x7fe13c121020 auth_method 0 2024-09-16T15:19:10.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe1415d7640 1 --2- 172.21.15.134:0/736486240 >> v2:172.21.15.162:6800/980547263 conn(0x7fe128038880 0x7fe12803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.588+0000 7fe1415d7640 10 monclient: get_auth_request con 0x7fe128038880 auth_method 0 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.589+0000 7fe140dd6640 10 monclient(hunting): handle_auth_done global_id 84864 payload 824 2024-09-16T15:19:10.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.589+0000 7fe140dd6640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe13c061ae0 con 0x7fe13c121020 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe140dd6640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 msgr2=0x7fe13c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe140dd6640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 0x7fe13c02b260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009fd0 tx=0x7fe12c00ec70 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1415d7640 1 --2- 172.21.15.134:0/736486240 >> v2:172.21.15.162:6800/980547263 conn(0x7fe128038880 0x7fe12803ad40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0a8ca0 tx=0x7fe134003de0 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe12c009610 con 0x7fe13c121020 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 10 monclient: dump: 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.608 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe12c00cbc0 con 0x7fe13c121020 2024-09-16T15:19:10.609 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 msgr2=0x7fe13c02b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 0x7fe13c02b260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009fd0 tx=0x7fe12c00ec70 comp rx=0 tx=0).stop 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 msgr2=0x7fe13c02b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.591+0000 7fe1397fa640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c121020 0x7fe13c02b260 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009fd0 tx=0x7fe12c00ec70 comp rx=0 tx=0).stop 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.597+0000 7fe14303a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.597+0000 7fe14303a640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 0x7fe13c02bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.597+0000 7fe14303a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.597+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c065cf0 con 0x7fe13c02b7a0 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 0x7fe13c02bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient: get_auth_request con 0x7fe13c02b7a0 auth_method 0 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.610 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 10 monclient(hunting): handle_auth_done global_id 84873 payload 824 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.598+0000 7fe140dd6640 1 -- 172.21.15.134:0/736486240 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe13c07a680 con 0x7fe13c02b7a0 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe140dd6640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 msgr2=0x7fe13c02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe140dd6640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 0x7fe13c02bbb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe12c018000 tx=0x7fe12c004730 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe12c022070 con 0x7fe13c02b7a0 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe1397fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe1397fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.600+0000 7fe1397fa640 10 monclient: dump: 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.611 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9185 (secure 0 0 0) 0x7fe12c005e90 con 0x7fe13c02b7a0 2024-09-16T15:19:10.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 msgr2=0x7fe13c02bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 0x7fe13c02bbb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe12c018000 tx=0x7fe12c004730 comp rx=0 tx=0).stop 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 msgr2=0x7fe13c02bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1397fa640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe13c02b7a0 0x7fe13c02bbb0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe12c018000 tx=0x7fe12c004730 comp rx=0 tx=0).stop 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:10.613 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:10.614 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:10.615 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:10.616 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:10.617 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.618 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.619 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.620 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.621 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.622 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.623 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.624 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.625 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.626 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.627 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:10.628 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.629 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.630 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.631 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.632 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.633 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.634 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 >> v2:172.21.15.162:6800/980547263 conn(0x7fe128038880 msgr2=0x7fe12803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 --2- 172.21.15.134:0/736486240 >> v2:172.21.15.162:6800/980547263 conn(0x7fe128038880 0x7fe12803ad40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0a8ca0 tx=0x7fe134003de0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 10 monclient: shutdown 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 msgr2=0x7fe13c1353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 --2- 172.21.15.134:0/736486240 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fe13c136e40 0x7fe13c1353a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe130031000 tx=0x7fe1300046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1415d7640 1 -- 172.21.15.134:0/736486240 reap_dead start 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe1415d7640 1 -- 172.21.15.134:0/736486240 reap_dead start 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 shutdown_connections 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 >> 172.21.15.134:0/736486240 conn(0x7fe13c0a6110 msgr2=0x7fe13c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 shutdown_connections 2024-09-16T15:19:10.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.601+0000 7fe14303a640 1 -- 172.21.15.134:0/736486240 wait complete. 2024-09-16T15:19:10.637 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-16T15:19:10.637 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:19:10.655 INFO:tasks.rados.rados.0.smithi162.stdout:3843: finishing write tid 1 to smithi16234619-16 2024-09-16T15:19:10.658 INFO:tasks.rados.rados.0.smithi162.stdout:3844: done (3 left) 2024-09-16T15:19:10.658 INFO:tasks.rados.rados.0.smithi162.stdout:3846: done (2 left) 2024-09-16T15:19:10.658 INFO:tasks.rados.rados.0.smithi162.stdout:3847: write oid 47 current snap is 406 2024-09-16T15:19:10.658 INFO:tasks.rados.rados.0.smithi162.stdout:3847: seq_num 1307 ranges {766749=646806,1899377=656453,3109028=609462} 2024-09-16T15:19:10.665 INFO:tasks.rados.rados.0.smithi162.stdout:3847: writing smithi16234619-47 from 766749 to 1413555 tid 1 2024-09-16T15:19:10.673 INFO:tasks.rados.rados.0.smithi162.stdout:3847: writing smithi16234619-47 from 1899377 to 2555830 tid 2 2024-09-16T15:19:10.681 INFO:tasks.rados.rados.0.smithi162.stdout:3847: writing smithi16234619-47 from 3109028 to 3718490 tid 3 2024-09-16T15:19:10.681 INFO:tasks.rados.rados.0.smithi162.stdout:3848: snap_create 2024-09-16T15:19:10.723 INFO:tasks.rados.rados.0.smithi162.stdout:3847: finishing write tid 1 to smithi16234619-47 2024-09-16T15:19:10.723 INFO:tasks.rados.rados.0.smithi162.stdout:3847: finishing write tid 2 to smithi16234619-47 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: finishing write tid 3 to smithi16234619-47 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: finishing write tid 4 to smithi16234619-47 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: finishing write tid 5 to smithi16234619-47 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: oid 47 updating version 0 to 2422 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: oid 47 version 2422 is already newer than 2421 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: oid 47 version 2422 is already newer than 2420 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:3847: oid 47 updating version 2422 to 2423 2024-09-16T15:19:10.725 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 2423 (ObjNum 1307 snap 406 seq_num 1307) dirty exists 2024-09-16T15:19:10.726 INFO:tasks.rados.rados.0.smithi162.stdout:3847: left oid 47 (ObjNum 1307 snap 406 seq_num 1307) 2024-09-16T15:19:10.728 INFO:tasks.rados.rados.0.smithi162.stdout:3843: finishing write tid 2 to smithi16234619-16 2024-09-16T15:19:10.728 INFO:tasks.rados.rados.0.smithi162.stdout:3843: finishing write tid 3 to smithi16234619-16 2024-09-16T15:19:10.728 INFO:tasks.rados.rados.0.smithi162.stdout:3843: finishing write tid 4 to smithi16234619-16 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: finishing write tid 5 to smithi16234619-16 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: oid 16 updating version 0 to 1852 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: oid 16 version 1852 is already newer than 1851 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: oid 16 updating version 1852 to 1853 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: oid 16 version 1853 is already newer than 1850 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1853 (ObjNum 1303 snap 406 seq_num 1303) dirty exists 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3843: left oid 16 (ObjNum 1303 snap 406 seq_num 1303) 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3845: finishing write tid 1 to smithi16234619-6 2024-09-16T15:19:10.729 INFO:tasks.rados.rados.0.smithi162.stdout:3845: finishing write tid 2 to smithi16234619-6 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: finishing write tid 3 to smithi16234619-6 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: finishing write tid 4 to smithi16234619-6 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: finishing write tid 5 to smithi16234619-6 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: oid 6 updating version 0 to 1854 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: oid 6 updating version 1854 to 1855 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: oid 6 updating version 1855 to 1856 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: oid 6 updating version 1856 to 1857 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 6 v 1857 (ObjNum 1305 snap 406 seq_num 1305) dirty exists 2024-09-16T15:19:10.730 INFO:tasks.rados.rados.0.smithi162.stdout:3845: left oid 6 (ObjNum 1305 snap 406 seq_num 1305) 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.734+0000 7f1c105ed640 10 monclient: get_monmap_and_config 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.734+0000 7f1c105ed640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.734+0000 7f1c105ed640 10 monclient: monmap: 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:10.735440+0000 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:10.735440+0000 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.736 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 Processor -- start 2024-09-16T15:19:10.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 -- start start 2024-09-16T15:19:10.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient: init 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient: _add_conns ranks=[2,9,4,8,1,10,0,5,7,6,3] 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-b con 0x7f1c08136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c08137220 0x7f1c08137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-a con 0x7f1c08137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-c con 0x7f1c08135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0809abc0 con 0x7f1c08137220 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0809ba50 con 0x7f1c08136e40 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080a53c0 con 0x7f1c08135dc0 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c105ed640 10 monclient(hunting): authenticate will time out at 1887.324791s 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c08137220 0x7f1c08137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:37034/0 (socket says 172.21.15.134:37034) 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:51564/0 (socket says 172.21.15.134:51564) 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c08137220 0x7f1c08137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57652/0 (socket says 172.21.15.134:57652) 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 1 -- 172.21.15.134:0/2256648104 learned_addr learned my addr 172.21.15.134:0/2256648104 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 10 monclient(hunting): get_auth_request con 0x7f1c08135dc0 auth_method 0 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0eb8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 10 monclient(hunting): get_auth_request con 0x7f1c08136e40 auth_method 0 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0e389640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 10 monclient(hunting): get_auth_request con 0x7f1c08137220 auth_method 0 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.735+0000 7f1c0db88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient(hunting): handle_auth_done global_id 124299 payload 274 2024-09-16T15:19:10.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 -- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 msgr2=0x7f1c08136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 -- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 msgr2=0x7f1c0809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 monclient: found mon.noname-a 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0db88640 1 -- 172.21.15.134:0/2256648104 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08073250 con 0x7f1c08137220 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0eb8a640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.736+0000 7f1c0e389640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0db88640 1 -- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08137220 msgr2=0x7f1c08137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08137220 0x7f1c08137600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0097a0 tx=0x7f1bfc029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e61d03c360e656 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 1 -- 172.21.15.134:0/2256648104 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1bfc02ad30 con 0x7f1c08137220 2024-09-16T15:19:10.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: dump: 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.746 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 5 monclient: authenticate success, global_id 124299 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 1 -- 172.21.15.134:0/2256648104 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc0307b0 con 0x7f1c08137220 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 4 set_mon_vals no callback set 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 1 -- 172.21.15.134:0/2256648104 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1bfc030d90 con 0x7f1c08137220 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c0d387640 10 monclient: dump: 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.748 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 10 monclient: shutdown 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 -- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08137220 msgr2=0x7f1c08137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08137220 0x7f1c08137600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0097a0 tx=0x7f1bfc029fd0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 -- 172.21.15.134:0/2256648104 shutdown_connections 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f1c08135dc0 0x7f1c08136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08137220 0x7f1c08137600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2256648104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c0809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 -- 172.21.15.134:0/2256648104 >> 172.21.15.134:0/2256648104 conn(0x7f1c080a6110 msgr2=0x7f1c080a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 -- 172.21.15.134:0/2256648104 shutdown_connections 2024-09-16T15:19:10.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.739+0000 7f1c105ed640 1 -- 172.21.15.134:0/2256648104 wait complete. 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.740+0000 7f1c105ed640 10 monclient: build_initial_monmap 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.740+0000 7f1c105ed640 10 monclient: monmap: 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:10.750 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:10.751 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:10.751 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:10.751 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.740+0000 7f1c105ed640 1 Processor -- start 2024-09-16T15:19:10.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.740+0000 7f1c105ed640 1 -- start start 2024-09-16T15:19:10.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.740+0000 7f1c105ed640 10 monclient: init 2024-09-16T15:19:10.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:10.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient: _add_conns ranks=[0,2,6,4,3,1,7,8,5] 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 0x7f1c081cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-a con 0x7f1c08135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c08134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-b con 0x7f1c08136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1c08137220 0x7f1c081354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): picked mon.noname-d con 0x7f1c08137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081cb7f0 con 0x7f1c08135dc0 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081254c0 con 0x7f1c08136e40 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081259a0 con 0x7f1c08137220 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c105ed640 10 monclient(hunting): authenticate will time out at 1887.330214s 2024-09-16T15:19:10.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 0x7f1c081cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c08134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 0x7f1c081cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57668/0 (socket says 172.21.15.134:57668) 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 -- 172.21.15.134:0/2941303165 learned_addr learned my addr 172.21.15.134:0/2941303165 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1c08137220 0x7f1c081354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): get_auth_request con 0x7f1c08135dc0 auth_method 0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): get_auth_request con 0x7f1c08136e40 auth_method 0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): get_auth_request con 0x7f1c08137220 auth_method 0 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0eb8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient(hunting): handle_auth_done global_id 124308 payload 824 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1c08137220 msgr2=0x7f1c081354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1c08137220 0x7f1c081354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 msgr2=0x7f1c08134fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c08136e40 0x7f1c08134fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient: _finish_hunting 0 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 monclient: found mon.noname-a 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.741+0000 7f1c0e389640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf80094d0 con 0x7f1c08135dc0 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c0e389640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 msgr2=0x7f1c081cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c0e389640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 0x7f1c081cb2b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009f60 tx=0x7f1c00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c00014b00 con 0x7f1c08135dc0 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 10 monclient: dump: 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.760 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 10 monclient: _finish_auth 0 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.743+0000 7f1c06ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:19:10.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0000f040 con 0x7f1c08135dc0 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c00014c80 con 0x7f1c08135dc0 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 5 monclient: authenticate success, global_id 124308 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 monclient: _renew_subs 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c00009a70 con 0x7f1c08135dc0 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c0cb86640 4 set_mon_vals no callback set 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 10 monclient: dump: 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.762 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 monclient: _renew_subs 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc009400 con 0x7f1c08135dc0 2024-09-16T15:19:10.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 0x7f1c0802b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08123c10 con 0x7f1c081221d0 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1c00014690 con 0x7f1c08135dc0 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.744+0000 7f1c06ffd640 1 --2- 172.21.15.134:0/2941303165 >> v2:172.21.15.162:6800/980547263 conn(0x7f1bf4038880 0x7f1bf403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(830..830 src has 1..830) v4 ==== 3931+0+0 (secure 0 0 0) 0x7f1c0004aad0 con 0x7f1c08135dc0 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 0x7f1c0802b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient: get_auth_request con 0x7f1c081221d0 auth_method 0 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0eb8a640 1 --2- 172.21.15.134:0/2941303165 >> v2:172.21.15.162:6800/980547263 conn(0x7f1bf4038880 0x7f1bf403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0eb8a640 10 monclient: get_auth_request con 0x7f1bf4038880 auth_method 0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 10 monclient(hunting): handle_auth_done global_id 45464 payload 824 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.745+0000 7f1c0db88640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c08061ae0 con 0x7f1c081221d0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c0db88640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 msgr2=0x7f1c0802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 0x7f1c0802b260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc034000 tx=0x7f1bfc0046c0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c0eb8a640 1 --2- 172.21.15.134:0/2941303165 >> v2:172.21.15.162:6800/980547263 conn(0x7f1bf4038880 0x7f1bf403ad40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8006cc0 tx=0x7f1bf8009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1bfc003610 con 0x7f1c081221d0 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c06ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.747+0000 7f1c06ffd640 10 monclient: dump: 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.765 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1bfc030b30 con 0x7f1c081221d0 2024-09-16T15:19:10.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 msgr2=0x7f1c0802b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 0x7f1c0802b260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc034000 tx=0x7f1bfc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 msgr2=0x7f1c0802b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.748+0000 7f1c06ffd640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081221d0 0x7f1c0802b260 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c105ed640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 0x7f1c0802b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c105ed640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c08071b20 con 0x7f1c081225b0 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 0x7f1c0802b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient: get_auth_request con 0x7f1c081225b0 auth_method 0 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:10.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:10.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:10.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 10 monclient(hunting): handle_auth_done global_id 45473 payload 824 2024-09-16T15:19:10.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.754+0000 7f1c0db88640 1 -- 172.21.15.134:0/2941303165 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1c0807a680 con 0x7f1c081225b0 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c0db88640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 msgr2=0x7f1c0802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c0db88640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 0x7f1c0802b7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc042000 tx=0x7f1bfc009190 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1bfc04e070 con 0x7f1c081225b0 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 10 monclient: dump: 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:10.769 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9351 (secure 0 0 0) 0x7f1bfc03dcc0 con 0x7f1c081225b0 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:10.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 msgr2=0x7f1c0802b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 0x7f1c0802b7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc042000 tx=0x7f1bfc009190 comp rx=0 tx=0).stop 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 msgr2=0x7f1c0802b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.757+0000 7f1c06ffd640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f1c081225b0 0x7f1c0802b7a0 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc042000 tx=0x7f1bfc009190 comp rx=0 tx=0).stop 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 160, 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:19:10.771 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 1, 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.772 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.773 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:19:10.774 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:19:10.775 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:19:10.776 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.777 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.778 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.779 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.780 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:19:10.781 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.782 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.783 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.784 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.785 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.786 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:19:10.787 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:19:10.788 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.789 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.790 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:19:10.791 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:19:10.792 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 >> v2:172.21.15.162:6800/980547263 conn(0x7f1bf4038880 msgr2=0x7f1bf403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2941303165 >> v2:172.21.15.162:6800/980547263 conn(0x7f1bf4038880 0x7f1bf403ad40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8006cc0 tx=0x7f1bf8009240 comp rx=0 tx=0).stop 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 10 monclient: shutdown 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 msgr2=0x7f1c081cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 --2- 172.21.15.134:0/2941303165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c08135dc0 0x7f1c081cb2b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009f60 tx=0x7f1c00009f90 comp rx=0 tx=0).stop 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c0eb8a640 1 -- 172.21.15.134:0/2941303165 reap_dead start 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c0eb8a640 1 -- 172.21.15.134:0/2941303165 reap_dead start 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 shutdown_connections 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 >> 172.21.15.134:0/2941303165 conn(0x7f1c080a6110 msgr2=0x7f1c0809f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 shutdown_connections 2024-09-16T15:19:10.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:10.758+0000 7f1c105ed640 1 -- 172.21.15.134:0/2941303165 wait complete. 2024-09-16T15:19:10.794 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-16T15:19:10.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3843: done (3 left) 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3845: done (2 left) 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3847: done (1 left) 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3848: done (0 left) 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3849: read oid 16 snap -1 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3849: expect (ObjNum 1303 snap 406 seq_num 1303) 2024-09-16T15:19:11.092 INFO:tasks.rados.rados.0.smithi162.stdout:3850: copy_from oid 41 from oid 34 current snap is 407 2024-09-16T15:19:11.093 INFO:tasks.rados.rados.0.smithi162.stdout:3851: rollback oid 32 current snap is 407 2024-09-16T15:19:11.093 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 32 to 397 2024-09-16T15:19:11.093 INFO:tasks.rados.rados.0.smithi162.stdout:3852: read oid 47 snap -1 2024-09-16T15:19:11.093 INFO:tasks.rados.rados.0.smithi162.stdout:3852: expect (ObjNum 1307 snap 406 seq_num 1307) 2024-09-16T15:19:11.093 INFO:tasks.rados.rados.0.smithi162.stdout:3853: snap_remove snap 393 2024-09-16T15:19:12.049 INFO:tasks.rados.rados.0.smithi162.stdout:3849: expect (ObjNum 1303 snap 406 seq_num 1303) 2024-09-16T15:19:12.177 INFO:tasks.rados.rados.0.smithi162.stdout:3852: expect (ObjNum 1307 snap 406 seq_num 1307) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3851: finishing rollback tid 1 to smithi16234619-32 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 2331 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3850: finishing copy_from racing read to smithi16234619-41 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3850: finishing copy_from to smithi16234619-41 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 2158 (ObjNum 1297 snap 400 seq_num 1297) dirty exists 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3849: done (4 left) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3850: done (3 left) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3851: done (2 left) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3852: done (1 left) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3853: done (0 left) 2024-09-16T15:19:12.556 INFO:tasks.rados.rados.0.smithi162.stdout:3854: read oid 50 snap -1 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3854: expect (ObjNum 1293 snap 398 seq_num 1293) 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3855: read oid 21 snap 382 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3855: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3856: read oid 14 snap 395 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3856: expect (ObjNum 1246 snap 386 seq_num 1246) 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3857: write (excl) oid 2 current snap is 407 2024-09-16T15:19:12.557 INFO:tasks.rados.rados.0.smithi162.stdout:3857: seq_num 1308 ranges {783556=438408,1674212=400111,2736451=420734,3882270=101286} 2024-09-16T15:19:12.562 INFO:tasks.rados.rados.0.smithi162.stdout:3857: writing smithi16234619-2 from 783556 to 1221964 tid 1 2024-09-16T15:19:12.567 INFO:tasks.rados.rados.0.smithi162.stdout:3857: writing smithi16234619-2 from 1674212 to 2074323 tid 2 2024-09-16T15:19:12.573 INFO:tasks.rados.rados.0.smithi162.stdout:3857: writing smithi16234619-2 from 2736451 to 3157185 tid 3 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3857: writing smithi16234619-2 from 3882270 to 3983556 tid 4 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3858: read oid 45 snap -1 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3858: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3859: read oid 44 snap -1 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3859: expect deleted 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3860: snap_create 2024-09-16T15:19:12.578 INFO:tasks.rados.rados.0.smithi162.stdout:3855: expect (ObjNum 1196 snap 365 seq_num 1196) 2024-09-16T15:19:12.650 INFO:tasks.rados.rados.0.smithi162.stdout:3854: expect (ObjNum 1293 snap 398 seq_num 1293) 2024-09-16T15:19:13.011 INFO:tasks.rados.rados.0.smithi162.stdout:3856: expect (ObjNum 1246 snap 386 seq_num 1246) 2024-09-16T15:19:13.103 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 1 to smithi16234619-2 2024-09-16T15:19:13.103 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 2 to smithi16234619-2 2024-09-16T15:19:13.103 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 3 to smithi16234619-2 2024-09-16T15:19:13.103 INFO:tasks.rados.rados.0.smithi162.stdout:3858: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 4 to smithi16234619-2 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 5 to smithi16234619-2 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: finishing write tid 6 to smithi16234619-2 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: oid 2 updating version 0 to 2535 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: oid 2 version 2535 is already newer than 2532 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: oid 2 version 2535 is already newer than 2534 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: oid 2 version 2535 is already newer than 2531 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: oid 2 version 2535 is already newer than 2533 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2535 (ObjNum 1308 snap 407 seq_num 1308) dirty exists 2024-09-16T15:19:13.206 INFO:tasks.rados.rados.0.smithi162.stdout:3857: left oid 2 (ObjNum 1308 snap 407 seq_num 1308) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3854: done (6 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3855: done (5 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3856: done (4 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3857: done (3 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3858: done (2 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3859: done (1 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3860: done (0 left) 2024-09-16T15:19:13.536 INFO:tasks.rados.rados.0.smithi162.stdout:3861: snap_create 2024-09-16T15:19:14.670 INFO:tasks.rados.rados.0.smithi162.stdout:3861: done (0 left) 2024-09-16T15:19:14.670 INFO:tasks.rados.rados.0.smithi162.stdout:3862: delete oid 36 current snap is 409 2024-09-16T15:19:14.675 INFO:tasks.rados.rados.0.smithi162.stdout:3862: done (0 left) 2024-09-16T15:19:14.675 INFO:tasks.rados.rados.0.smithi162.stdout:3863: write (excl) oid 33 current snap is 409 2024-09-16T15:19:14.675 INFO:tasks.rados.rados.0.smithi162.stdout:3863: seq_num 1310 ranges {417170=588906,1674823=342347} 2024-09-16T15:19:14.680 INFO:tasks.rados.rados.0.smithi162.stdout:3863: writing smithi16234619-33 from 417170 to 1006076 tid 1 2024-09-16T15:19:14.687 INFO:tasks.rados.rados.0.smithi162.stdout:3863: writing smithi16234619-33 from 1674823 to 2017170 tid 2 2024-09-16T15:19:14.687 INFO:tasks.rados.rados.0.smithi162.stdout:3864: read oid 14 snap -1 2024-09-16T15:19:14.687 INFO:tasks.rados.rados.0.smithi162.stdout:3864: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:19:14.687 INFO:tasks.rados.rados.0.smithi162.stdout:3865: rollback oid 49 current snap is 409 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 49 to 400 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:3866: copy_from oid 35 from oid 44 current snap is 409 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:3867: read oid 28 snap -1 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:3867: expect deleted 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:3868: write oid 42 current snap is 409 2024-09-16T15:19:14.688 INFO:tasks.rados.rados.0.smithi162.stdout:3868: seq_num 1311 ranges {433977=664155,1733305=300672} 2024-09-16T15:19:14.694 INFO:tasks.rados.rados.0.smithi162.stdout:3868: writing smithi16234619-42 from 433977 to 1098132 tid 1 2024-09-16T15:19:14.700 INFO:tasks.rados.rados.0.smithi162.stdout:3868: writing smithi16234619-42 from 1733305 to 2033977 tid 2 2024-09-16T15:19:14.700 INFO:tasks.rados.rados.0.smithi162.stdout:3869: snap_remove snap 397 2024-09-16T15:19:15.676 INFO:tasks.rados.rados.0.smithi162.stdout:3869: done (6 left) 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3870: copy_from oid 30 from oid 12 current snap is 409 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3866: finishing copy_from to smithi16234619-35 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3866: got expected ENOENT (src dne) 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3866: finishing copy_from racing read to smithi16234619-35 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3865: finishing rollback tid 0 to smithi16234619-49 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3865: finishing rollback tid 1 to smithi16234619-49 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3865: finishing rollback tid 2 to smithi16234619-49 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 2332 (ObjNum 1250 snap 386 seq_num 1250) dirty exists 2024-09-16T15:19:15.677 INFO:tasks.rados.rados.0.smithi162.stdout:3864: expect (ObjNum 1265 snap 392 seq_num 1265) 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3863: finishing write tid 1 to smithi16234619-33 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3863: finishing write tid 2 to smithi16234619-33 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3864: done (6 left) 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3865: done (5 left) 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3866: done (4 left) 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3867: done (3 left) 2024-09-16T15:19:15.784 INFO:tasks.rados.rados.0.smithi162.stdout:3871: delete oid 48 current snap is 409 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: finishing write tid 3 to smithi16234619-33 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: finishing write tid 4 to smithi16234619-33 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: oid 33 updating version 0 to 1725 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: oid 33 version 1725 is already newer than 1724 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: oid 33 version 1725 is already newer than 1723 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 33 v 1725 (ObjNum 1310 snap 409 seq_num 1310) dirty exists 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3863: left oid 33 (ObjNum 1310 snap 409 seq_num 1310) 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3868: finishing write tid 1 to smithi16234619-42 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3868: finishing write tid 2 to smithi16234619-42 2024-09-16T15:19:15.785 INFO:tasks.rados.rados.0.smithi162.stdout:3868: finishing write tid 3 to smithi16234619-42 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3868: finishing write tid 4 to smithi16234619-42 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3868: oid 42 updating version 0 to 2276 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3868: oid 42 updating version 2276 to 2277 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3868: oid 42 updating version 2277 to 2278 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 2278 (ObjNum 1311 snap 409 seq_num 1311) dirty exists 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3868: left oid 42 (ObjNum 1311 snap 409 seq_num 1311) 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3870: finishing copy_from racing read to smithi16234619-30 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:3870: finishing copy_from to smithi16234619-30 2024-09-16T15:19:15.786 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 2326 (ObjNum 1253 snap 387 seq_num 1253) dirty exists 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3863: done (3 left) 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3868: done (2 left) 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3870: done (1 left) 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3871: done (0 left) 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3872: write (excl) oid 35 current snap is 409 2024-09-16T15:19:15.787 INFO:tasks.rados.rados.0.smithi162.stdout:3872: seq_num 1313 ranges {467591=414653,1450269=617322} 2024-09-16T15:19:15.792 INFO:tasks.rados.rados.0.smithi162.stdout:3872: writing smithi16234619-35 from 467591 to 882244 tid 1 2024-09-16T15:19:15.798 INFO:tasks.rados.rados.0.smithi162.stdout:3872: writing smithi16234619-35 from 1450269 to 2067591 tid 2 2024-09-16T15:19:15.798 INFO:tasks.rados.rados.0.smithi162.stdout:3873: write (excl) oid 49 current snap is 409 2024-09-16T15:19:15.798 INFO:tasks.rados.rados.0.smithi162.stdout:3873: seq_num 1314 ranges {484398=489902,1392398=411118} 2024-09-16T15:19:15.804 INFO:tasks.rados.rados.0.smithi162.stdout:3873: writing smithi16234619-49 from 484398 to 974300 tid 1 2024-09-16T15:19:15.809 INFO:tasks.rados.rados.0.smithi162.stdout:3873: writing smithi16234619-49 from 1392398 to 1803516 tid 2 2024-09-16T15:19:15.809 INFO:tasks.rados.rados.0.smithi162.stdout:3874: snap_remove snap 382 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3874: done (2 left) 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3875: read oid 25 snap -1 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3875: expect (ObjNum 1276 snap 396 seq_num 1276) 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3876: snap_create 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3872: finishing write tid 1 to smithi16234619-35 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3872: finishing write tid 2 to smithi16234619-35 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3872: finishing write tid 3 to smithi16234619-35 2024-09-16T15:19:16.689 INFO:tasks.rados.rados.0.smithi162.stdout:3872: finishing write tid 4 to smithi16234619-35 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3872: oid 35 updating version 0 to 1859 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3872: oid 35 version 1859 is already newer than 1858 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3872: oid 35 updating version 1859 to 1860 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 35 v 1860 (ObjNum 1313 snap 409 seq_num 1313) dirty exists 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3872: left oid 35 (ObjNum 1313 snap 409 seq_num 1313) 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: finishing write tid 1 to smithi16234619-49 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: finishing write tid 2 to smithi16234619-49 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: finishing write tid 3 to smithi16234619-49 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: finishing write tid 4 to smithi16234619-49 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: oid 49 updating version 0 to 2333 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: oid 49 updating version 2333 to 2334 2024-09-16T15:19:16.690 INFO:tasks.rados.rados.0.smithi162.stdout:3873: oid 49 updating version 2334 to 2335 2024-09-16T15:19:16.691 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 49 v 2335 (ObjNum 1314 snap 409 seq_num 1314) dirty exists 2024-09-16T15:19:16.691 INFO:tasks.rados.rados.0.smithi162.stdout:3873: left oid 49 (ObjNum 1314 snap 409 seq_num 1314) 2024-09-16T15:19:16.715 INFO:tasks.rados.rados.0.smithi162.stdout:3875: expect (ObjNum 1276 snap 396 seq_num 1276) 2024-09-16T15:19:17.492 INFO:tasks.rados.rados.0.smithi162.stdout:3872: done (3 left) 2024-09-16T15:19:17.492 INFO:tasks.rados.rados.0.smithi162.stdout:3873: done (2 left) 2024-09-16T15:19:17.492 INFO:tasks.rados.rados.0.smithi162.stdout:3875: done (1 left) 2024-09-16T15:19:17.492 INFO:tasks.rados.rados.0.smithi162.stdout:3876: done (0 left) 2024-09-16T15:19:17.492 INFO:tasks.rados.rados.0.smithi162.stdout:3877: delete oid 18 current snap is 410 2024-09-16T15:19:17.499 INFO:tasks.rados.rados.0.smithi162.stdout:3877: done (0 left) 2024-09-16T15:19:17.499 INFO:tasks.rados.rados.0.smithi162.stdout:3878: snap_remove snap 398 2024-09-16T15:19:18.503 INFO:tasks.rados.rados.0.smithi162.stdout:3878: done (0 left) 2024-09-16T15:19:18.503 INFO:tasks.rados.rados.0.smithi162.stdout:3879: read oid 4 snap -1 2024-09-16T15:19:18.503 INFO:tasks.rados.rados.0.smithi162.stdout:3879: expect (ObjNum 1298 snap 400 seq_num 1298) 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3880: copy_from oid 14 from oid 11 current snap is 410 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3881: copy_from oid 44 from oid 32 current snap is 410 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3882: rollback oid 42 current snap is 410 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 42 to 400 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3883: read oid 31 snap -1 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3883: expect (ObjNum 1301 snap 406 seq_num 1301) 2024-09-16T15:19:18.504 INFO:tasks.rados.rados.0.smithi162.stdout:3884: snap_create 2024-09-16T15:19:18.509 INFO:tasks.rados.rados.0.smithi162.stdout:3881: finishing copy_from to smithi16234619-44 2024-09-16T15:19:18.509 INFO:tasks.rados.rados.0.smithi162.stdout:3881: got expected ENOENT (src dne) 2024-09-16T15:19:18.510 INFO:tasks.rados.rados.0.smithi162.stdout:3881: finishing copy_from racing read to smithi16234619-44 2024-09-16T15:19:18.515 INFO:tasks.rados.rados.0.smithi162.stdout:3882: finishing rollback tid 0 to smithi16234619-42 2024-09-16T15:19:18.516 INFO:tasks.rados.rados.0.smithi162.stdout:3882: finishing rollback tid 1 to smithi16234619-42 2024-09-16T15:19:18.516 INFO:tasks.rados.rados.0.smithi162.stdout:3882: finishing rollback tid 2 to smithi16234619-42 2024-09-16T15:19:18.517 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 42 v 2298 (ObjNum 1151 snap 343 seq_num 1151) dirty exists 2024-09-16T15:19:18.518 INFO:tasks.rados.rados.0.smithi162.stdout:3879: expect (ObjNum 1298 snap 400 seq_num 1298) 2024-09-16T15:19:18.738 INFO:tasks.rados.rados.0.smithi162.stdout:3883: expect (ObjNum 1301 snap 406 seq_num 1301) 2024-09-16T15:19:18.895 INFO:tasks.rados.rados.0.smithi162.stdout:3880: finishing copy_from racing read to smithi16234619-14 2024-09-16T15:19:18.895 INFO:tasks.rados.rados.0.smithi162.stdout:3880: finishing copy_from to smithi16234619-14 2024-09-16T15:19:18.895 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 1488 (ObjNum 1197 snap 365 seq_num 1197) dirty exists 2024-09-16T15:19:19.683 INFO:tasks.rados.rados.0.smithi162.stdout:3879: done (5 left) 2024-09-16T15:19:19.683 INFO:tasks.rados.rados.0.smithi162.stdout:3880: done (4 left) 2024-09-16T15:19:19.683 INFO:tasks.rados.rados.0.smithi162.stdout:3881: done (3 left) 2024-09-16T15:19:19.683 INFO:tasks.rados.rados.0.smithi162.stdout:3882: done (2 left) 2024-09-16T15:19:19.683 INFO:tasks.rados.rados.0.smithi162.stdout:3883: done (1 left) 2024-09-16T15:19:19.684 INFO:tasks.rados.rados.0.smithi162.stdout:3884: done (0 left) 2024-09-16T15:19:19.684 INFO:tasks.rados.rados.0.smithi162.stdout:3885: write (excl) oid 16 current snap is 411 2024-09-16T15:19:19.684 INFO:tasks.rados.rados.0.smithi162.stdout:3885: seq_num 1316 ranges {518012=756753,2025715=92297} 2024-09-16T15:19:19.690 INFO:tasks.rados.rados.0.smithi162.stdout:3885: writing smithi16234619-16 from 518012 to 1274765 tid 1 2024-09-16T15:19:19.694 INFO:tasks.rados.rados.0.smithi162.stdout:3885: writing smithi16234619-16 from 2025715 to 2118012 tid 2 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3886: copy_from oid 25 from oid 21 current snap is 411 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3887: read oid 27 snap -1 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3887: expect deleted 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3888: copy_from oid 22 from oid 39 current snap is 411 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3889: copy_from oid 6 from oid 19 current snap is 411 2024-09-16T15:19:19.695 INFO:tasks.rados.rados.0.smithi162.stdout:3890: delete oid 1 current snap is 411 2024-09-16T15:19:19.700 INFO:tasks.rados.rados.0.smithi162.stdout:3887: done (5 left) 2024-09-16T15:19:19.700 INFO:tasks.rados.rados.0.smithi162.stdout:3890: done (4 left) 2024-09-16T15:19:19.700 INFO:tasks.rados.rados.0.smithi162.stdout:3891: write (excl) oid 45 current snap is 411 2024-09-16T15:19:19.700 INFO:tasks.rados.rados.0.smithi162.stdout:3891: seq_num 1318 ranges {551626=507251,1626326=418456} 2024-09-16T15:19:19.706 INFO:tasks.rados.rados.0.smithi162.stdout:3891: writing smithi16234619-45 from 551626 to 1058877 tid 1 2024-09-16T15:19:19.712 INFO:tasks.rados.rados.0.smithi162.stdout:3891: writing smithi16234619-45 from 1626326 to 2044782 tid 2 2024-09-16T15:19:19.713 INFO:tasks.rados.rados.0.smithi162.stdout:3892: write (excl) oid 2 current snap is 411 2024-09-16T15:19:19.713 INFO:tasks.rados.rados.0.smithi162.stdout:3892: seq_num 1319 ranges {568433=582500,1684808=483625} 2024-09-16T15:19:19.720 INFO:tasks.rados.rados.0.smithi162.stdout:3892: writing smithi16234619-2 from 568433 to 1150933 tid 1 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3892: writing smithi16234619-2 from 1684808 to 2168433 tid 2 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3893: read oid 42 snap -1 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3893: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3894: delete oid 30 current snap is 411 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: finishing write tid 1 to smithi16234619-16 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: finishing write tid 2 to smithi16234619-16 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: finishing write tid 3 to smithi16234619-16 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: finishing write tid 4 to smithi16234619-16 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: oid 16 updating version 0 to 1862 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: oid 16 version 1862 is already newer than 1861 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: oid 16 updating version 1862 to 1863 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 16 v 1863 (ObjNum 1316 snap 411 seq_num 1316) dirty exists 2024-09-16T15:19:19.727 INFO:tasks.rados.rados.0.smithi162.stdout:3885: left oid 16 (ObjNum 1316 snap 411 seq_num 1316) 2024-09-16T15:19:19.728 INFO:tasks.rados.rados.0.smithi162.stdout:3891: finishing write tid 1 to smithi16234619-45 2024-09-16T15:19:19.730 INFO:tasks.rados.rados.0.smithi162.stdout:3885: done (7 left) 2024-09-16T15:19:19.730 INFO:tasks.rados.rados.0.smithi162.stdout:3894: done (6 left) 2024-09-16T15:19:19.730 INFO:tasks.rados.rados.0.smithi162.stdout:3895: read oid 29 snap -1 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3895: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3896: copy_from oid 50 from oid 12 current snap is 411 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3897: read oid 38 snap -1 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3897: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3898: read oid 35 snap -1 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3898: expect (ObjNum 1313 snap 409 seq_num 1313) 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3899: write (excl) oid 36 current snap is 411 2024-09-16T15:19:19.731 INFO:tasks.rados.rados.0.smithi162.stdout:3899: seq_num 1321 ranges {602047=732998,1801772=400275} 2024-09-16T15:19:19.739 INFO:tasks.rados.rados.0.smithi162.stdout:3899: writing smithi16234619-36 from 602047 to 1335045 tid 1 2024-09-16T15:19:19.744 INFO:tasks.rados.rados.0.smithi162.stdout:3899: writing smithi16234619-36 from 1801772 to 2202047 tid 2 2024-09-16T15:19:19.744 INFO:tasks.rados.rados.0.smithi162.stdout:3900: delete oid 8 current snap is 411 2024-09-16T15:19:19.744 INFO:tasks.rados.rados.0.smithi162.stdout:3893: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:19.986 INFO:tasks.rados.rados.0.smithi162.stdout:3897: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:19:20.110 INFO:tasks.rados.rados.0.smithi162.stdout:3898: expect (ObjNum 1313 snap 409 seq_num 1313) 2024-09-16T15:19:20.231 INFO:tasks.rados.rados.0.smithi162.stdout:3895: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:20.443 INFO:tasks.rados.rados.0.smithi162.stdout:3886: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3891: finishing write tid 2 to smithi16234619-45 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3891: finishing write tid 3 to smithi16234619-45 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3893: done (11 left) 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3895: done (10 left) 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3897: done (9 left) 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3898: done (8 left) 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3900: done (7 left) 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3901: write oid 47 current snap is 411 2024-09-16T15:19:20.444 INFO:tasks.rados.rados.0.smithi162.stdout:3891: finishing write tid 4 to smithi16234619-45 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3891: oid 45 updating version 0 to 2547 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3891: oid 45 updating version 2547 to 2549 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3891: oid 45 version 2549 is already newer than 2548 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 2549 (ObjNum 1318 snap 411 seq_num 1318) dirty exists 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3891: left oid 45 (ObjNum 1318 snap 411 seq_num 1318) 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3888: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3892: finishing write tid 1 to smithi16234619-2 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:3888: finishing copy_from to smithi16234619-22 2024-09-16T15:19:20.445 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 2181 (ObjNum 1269 snap 396 seq_num 1269) dirty exists 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: finishing write tid 2 to smithi16234619-2 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: finishing write tid 3 to smithi16234619-2 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: finishing write tid 4 to smithi16234619-2 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: oid 2 updating version 0 to 2551 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: oid 2 updating version 2551 to 2553 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: oid 2 version 2553 is already newer than 2552 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 2 v 2553 (ObjNum 1319 snap 411 seq_num 1319) dirty exists 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3892: left oid 2 (ObjNum 1319 snap 411 seq_num 1319) 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3896: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3889: finishing copy_from to smithi16234619-6 2024-09-16T15:19:20.446 INFO:tasks.rados.rados.0.smithi162.stdout:3889: got expected ENOENT (src dne) 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3889: finishing copy_from racing read to smithi16234619-6 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3886: finishing copy_from to smithi16234619-25 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 2405 (ObjNum 1037 snap 311 seq_num 1037) dirty exists 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3899: finishing write tid 1 to smithi16234619-36 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3899: finishing write tid 2 to smithi16234619-36 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3896: finishing copy_from to smithi16234619-50 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 2326 (ObjNum 1253 snap 387 seq_num 1253) dirty exists 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3899: finishing write tid 3 to smithi16234619-36 2024-09-16T15:19:20.447 INFO:tasks.rados.rados.0.smithi162.stdout:3899: finishing write tid 4 to smithi16234619-36 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:3899: oid 36 updating version 0 to 2461 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:3899: oid 36 version 2461 is already newer than 2460 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:3899: oid 36 version 2461 is already newer than 2459 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 2461 (ObjNum 1321 snap 411 seq_num 1321) dirty exists 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:3899: left oid 36 (ObjNum 1321 snap 411 seq_num 1321) 2024-09-16T15:19:20.448 INFO:tasks.rados.rados.0.smithi162.stdout:3901: seq_num 1323 ranges {635661=599849,1918736=316925} 2024-09-16T15:19:20.449 INFO:tasks.rados.rados.0.smithi162.stdout:3901: writing smithi16234619-47 from 635661 to 1235510 tid 1 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3901: writing smithi16234619-47 from 1918736 to 2235661 tid 2 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3886: done (7 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3888: done (6 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3889: done (5 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3891: done (4 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3892: done (3 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3896: done (2 left) 2024-09-16T15:19:20.455 INFO:tasks.rados.rados.0.smithi162.stdout:3899: done (1 left) 2024-09-16T15:19:20.456 INFO:tasks.rados.rados.0.smithi162.stdout:3902: write (excl) oid 34 current snap is 411 2024-09-16T15:19:20.456 INFO:tasks.rados.rados.0.smithi162.stdout:3902: seq_num 1324 ranges {652468=675098,1977218=275250} 2024-09-16T15:19:20.461 INFO:tasks.rados.rados.0.smithi162.stdout:3902: writing smithi16234619-34 from 652468 to 1327566 tid 1 2024-09-16T15:19:20.467 INFO:tasks.rados.rados.0.smithi162.stdout:3902: writing smithi16234619-34 from 1977218 to 2252468 tid 2 2024-09-16T15:19:20.467 INFO:tasks.rados.rados.0.smithi162.stdout:3903: write oid 29 current snap is 411 2024-09-16T15:19:20.467 INFO:tasks.rados.rados.0.smithi162.stdout:3903: seq_num 1325 ranges {669275=750347,2035700=233575} 2024-09-16T15:19:20.475 INFO:tasks.rados.rados.0.smithi162.stdout:3903: writing smithi16234619-29 from 669275 to 1419622 tid 1 2024-09-16T15:19:20.480 INFO:tasks.rados.rados.0.smithi162.stdout:3903: writing smithi16234619-29 from 2035700 to 2269275 tid 2 2024-09-16T15:19:20.480 INFO:tasks.rados.rados.0.smithi162.stdout:3904: snap_remove snap 409 2024-09-16T15:19:20.918 INFO:tasks.rados.rados.0.smithi162.stdout:3904: done (3 left) 2024-09-16T15:19:20.918 INFO:tasks.rados.rados.0.smithi162.stdout:3905: write (excl) oid 25 current snap is 411 2024-09-16T15:19:20.918 INFO:tasks.rados.rados.0.smithi162.stdout:3905: seq_num 1326 ranges {686082=425596,1577829=433132} 2024-09-16T15:19:20.924 INFO:tasks.rados.rados.0.smithi162.stdout:3905: writing smithi16234619-25 from 686082 to 1111678 tid 1 2024-09-16T15:19:20.930 INFO:tasks.rados.rados.0.smithi162.stdout:3905: writing smithi16234619-25 from 1577829 to 2010961 tid 2 2024-09-16T15:19:20.930 INFO:tasks.rados.rados.0.smithi162.stdout:3906: read oid 12 snap 395 2024-09-16T15:19:20.930 INFO:tasks.rados.rados.0.smithi162.stdout:3906: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:20.930 INFO:tasks.rados.rados.0.smithi162.stdout:3907: delete oid 23 current snap is 411 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: finishing write tid 1 to smithi16234619-47 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: finishing write tid 2 to smithi16234619-47 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: finishing write tid 3 to smithi16234619-47 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: finishing write tid 4 to smithi16234619-47 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: oid 47 updating version 0 to 2426 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: oid 47 version 2426 is already newer than 2424 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:3901: oid 47 version 2426 is already newer than 2425 2024-09-16T15:19:20.931 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 2426 (ObjNum 1323 snap 411 seq_num 1323) dirty exists 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3901: left oid 47 (ObjNum 1323 snap 411 seq_num 1323) 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: finishing write tid 1 to smithi16234619-34 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: finishing write tid 2 to smithi16234619-34 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: finishing write tid 3 to smithi16234619-34 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: finishing write tid 4 to smithi16234619-34 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: oid 34 updating version 0 to 2462 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: oid 34 updating version 2462 to 2463 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:3902: oid 34 updating version 2463 to 2464 2024-09-16T15:19:20.932 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 2464 (ObjNum 1324 snap 411 seq_num 1324) dirty exists 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3902: left oid 34 (ObjNum 1324 snap 411 seq_num 1324) 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: finishing write tid 1 to smithi16234619-29 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: finishing write tid 2 to smithi16234619-29 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: finishing write tid 3 to smithi16234619-29 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: finishing write tid 4 to smithi16234619-29 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: oid 29 updating version 0 to 2302 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: oid 29 version 2302 is already newer than 2301 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:3903: oid 29 version 2302 is already newer than 2300 2024-09-16T15:19:20.933 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 29 v 2302 (ObjNum 1325 snap 411 seq_num 1325) dirty exists 2024-09-16T15:19:20.934 INFO:tasks.rados.rados.0.smithi162.stdout:3903: left oid 29 (ObjNum 1325 snap 411 seq_num 1325) 2024-09-16T15:19:20.939 INFO:tasks.rados.rados.0.smithi162.stdout:3906: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:21.168 INFO:tasks.rados.rados.0.smithi162.stdout:3905: finishing write tid 1 to smithi16234619-25 2024-09-16T15:19:21.168 INFO:tasks.rados.rados.0.smithi162.stdout:3905: finishing write tid 2 to smithi16234619-25 2024-09-16T15:19:21.168 INFO:tasks.rados.rados.0.smithi162.stdout:3905: finishing write tid 3 to smithi16234619-25 2024-09-16T15:19:21.168 INFO:tasks.rados.rados.0.smithi162.stdout:3905: finishing write tid 4 to smithi16234619-25 2024-09-16T15:19:21.168 INFO:tasks.rados.rados.0.smithi162.stdout:3905: oid 25 updating version 0 to 2465 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3905: oid 25 updating version 2465 to 2467 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3905: oid 25 version 2467 is already newer than 2466 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 25 v 2467 (ObjNum 1326 snap 411 seq_num 1326) dirty exists 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3905: left oid 25 (ObjNum 1326 snap 411 seq_num 1326) 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3901: done (5 left) 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3902: done (4 left) 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3903: done (3 left) 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3905: done (2 left) 2024-09-16T15:19:21.169 INFO:tasks.rados.rados.0.smithi162.stdout:3906: done (1 left) 2024-09-16T15:19:21.170 INFO:tasks.rados.rados.0.smithi162.stdout:3907: done (0 left) 2024-09-16T15:19:21.170 INFO:tasks.rados.rados.0.smithi162.stdout:3908: write (excl) oid 15 current snap is 411 2024-09-16T15:19:21.170 INFO:tasks.rados.rados.0.smithi162.stdout:3908: seq_num 1328 ranges {719696=576094,2094793=224903} 2024-09-16T15:19:21.174 INFO:tasks.rados.rados.0.smithi162.stdout:3908: writing smithi16234619-15 from 719696 to 1295790 tid 1 2024-09-16T15:19:21.180 INFO:tasks.rados.rados.0.smithi162.stdout:3908: writing smithi16234619-15 from 2094793 to 2319696 tid 2 2024-09-16T15:19:21.180 INFO:tasks.rados.rados.0.smithi162.stdout:3909: read oid 39 snap -1 2024-09-16T15:19:21.180 INFO:tasks.rados.rados.0.smithi162.stdout:3909: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:21.181 INFO:tasks.rados.rados.0.smithi162.stdout:3910: write (excl) oid 8 current snap is 411 2024-09-16T15:19:21.181 INFO:tasks.rados.rados.0.smithi162.stdout:3910: seq_num 1329 ranges {736503=651343,2153275=183228} 2024-09-16T15:19:21.187 INFO:tasks.rados.rados.0.smithi162.stdout:3910: writing smithi16234619-8 from 736503 to 1387846 tid 1 2024-09-16T15:19:21.193 INFO:tasks.rados.rados.0.smithi162.stdout:3910: writing smithi16234619-8 from 2153275 to 2336503 tid 2 2024-09-16T15:19:21.193 INFO:tasks.rados.rados.0.smithi162.stdout:3911: copy_from oid 50 from oid 3 current snap is 411 2024-09-16T15:19:21.193 INFO:tasks.rados.rados.0.smithi162.stdout:3909: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:21.407 INFO:tasks.rados.rados.0.smithi162.stdout:3908: finishing write tid 1 to smithi16234619-15 2024-09-16T15:19:21.407 INFO:tasks.rados.rados.0.smithi162.stdout:3908: finishing write tid 2 to smithi16234619-15 2024-09-16T15:19:21.407 INFO:tasks.rados.rados.0.smithi162.stdout:3908: finishing write tid 3 to smithi16234619-15 2024-09-16T15:19:21.407 INFO:tasks.rados.rados.0.smithi162.stdout:3908: finishing write tid 4 to smithi16234619-15 2024-09-16T15:19:21.407 INFO:tasks.rados.rados.0.smithi162.stdout:3908: oid 15 updating version 0 to 2330 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3908: oid 15 version 2330 is already newer than 2328 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3908: oid 15 version 2330 is already newer than 2329 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 15 v 2330 (ObjNum 1328 snap 411 seq_num 1328) dirty exists 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3908: left oid 15 (ObjNum 1328 snap 411 seq_num 1328) 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3910: finishing write tid 1 to smithi16234619-8 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3908: done (3 left) 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3909: done (2 left) 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3912: snap_create 2024-09-16T15:19:21.408 INFO:tasks.rados.rados.0.smithi162.stdout:3910: finishing write tid 2 to smithi16234619-8 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: finishing write tid 3 to smithi16234619-8 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: finishing write tid 4 to smithi16234619-8 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: oid 8 updating version 0 to 2125 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: oid 8 version 2125 is already newer than 2124 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: oid 8 updating version 2125 to 2126 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 8 v 2126 (ObjNum 1329 snap 411 seq_num 1329) dirty exists 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3910: left oid 8 (ObjNum 1329 snap 411 seq_num 1329) 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3911: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:3911: finishing copy_from to smithi16234619-50 2024-09-16T15:19:21.409 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 50 v 1816 (ObjNum 1292 snap 397 seq_num 1292) dirty exists 2024-09-16T15:19:21.920 INFO:tasks.rados.rados.0.smithi162.stdout:3910: done (2 left) 2024-09-16T15:19:21.920 INFO:tasks.rados.rados.0.smithi162.stdout:3911: done (1 left) 2024-09-16T15:19:21.920 INFO:tasks.rados.rados.0.smithi162.stdout:3912: done (0 left) 2024-09-16T15:19:21.920 INFO:tasks.rados.rados.0.smithi162.stdout:3913: snap_create 2024-09-16T15:19:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:3913: done (0 left) 2024-09-16T15:19:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:3914: write (excl) oid 14 current snap is 413 2024-09-16T15:19:23.675 INFO:tasks.rados.rados.0.smithi162.stdout:3914: seq_num 1330 ranges {753310=726592,2095404=257906} 2024-09-16T15:19:23.683 INFO:tasks.rados.rados.0.smithi162.stdout:3914: writing smithi16234619-14 from 753310 to 1479902 tid 1 2024-09-16T15:19:23.689 INFO:tasks.rados.rados.0.smithi162.stdout:3914: writing smithi16234619-14 from 2095404 to 2353310 tid 2 2024-09-16T15:19:23.689 INFO:tasks.rados.rados.0.smithi162.stdout:3915: rollback oid 45 current snap is 413 2024-09-16T15:19:23.689 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 408 2024-09-16T15:19:23.689 INFO:tasks.rados.rados.0.smithi162.stdout:3916: rollback oid 20 current snap is 413 2024-09-16T15:19:23.689 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 20 to 386 2024-09-16T15:19:23.691 INFO:tasks.rados.rados.0.smithi162.stdout:3917: read oid 23 snap -1 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3917: expect deleted 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3918: copy_from oid 22 from oid 32 current snap is 413 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3919: read oid 40 snap 404 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3919: expect (ObjNum 1267 snap 394 seq_num 1267) 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3920: read oid 39 snap -1 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3920: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3921: write (excl) oid 34 current snap is 413 2024-09-16T15:19:23.692 INFO:tasks.rados.rados.0.smithi162.stdout:3921: seq_num 1331 ranges {770117=518194,1870239=499878} 2024-09-16T15:19:23.696 INFO:tasks.rados.rados.0.smithi162.stdout:3921: writing smithi16234619-34 from 770117 to 1288311 tid 1 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3921: writing smithi16234619-34 from 1870239 to 2370117 tid 2 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3922: read oid 38 snap -1 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3922: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3923: read oid 26 snap 412 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3923: expect (ObjNum 1243 snap 386 seq_num 1243) 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3924: delete oid 41 current snap is 413 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3918: finishing copy_from to smithi16234619-22 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3918: got expected ENOENT (src dne) 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3918: finishing copy_from racing read to smithi16234619-22 2024-09-16T15:19:23.703 INFO:tasks.rados.rados.0.smithi162.stdout:3920: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:23.915 INFO:tasks.rados.rados.0.smithi162.stdout:3916: finishing rollback tid 1 to smithi16234619-20 2024-09-16T15:19:23.915 INFO:tasks.rados.rados.0.smithi162.stdout:3916: finishing rollback tid 2 to smithi16234619-20 2024-09-16T15:19:23.915 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1727 (ObjNum 1138 snap 340 seq_num 1138) dirty exists 2024-09-16T15:19:23.916 INFO:tasks.rados.rados.0.smithi162.stdout:3922: expect (ObjNum 1123 snap 336 seq_num 1123) 2024-09-16T15:19:24.039 INFO:tasks.rados.rados.0.smithi162.stdout:3919: expect (ObjNum 1267 snap 394 seq_num 1267) 2024-09-16T15:19:24.269 INFO:tasks.rados.rados.0.smithi162.stdout:3923: expect (ObjNum 1243 snap 386 seq_num 1243) 2024-09-16T15:19:24.407 INFO:tasks.rados.rados.0.smithi162.stdout:3915: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:19:24.407 INFO:tasks.rados.rados.0.smithi162.stdout:3915: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3915: finishing rollback tid 2 to smithi16234619-45 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 2561 (ObjNum 1265 snap 392 seq_num 1265) dirty exists 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: finishing write tid 1 to smithi16234619-14 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: finishing write tid 2 to smithi16234619-14 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: finishing write tid 3 to smithi16234619-14 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: finishing write tid 4 to smithi16234619-14 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: oid 14 updating version 0 to 2470 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: oid 14 version 2470 is already newer than 2469 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: oid 14 version 2470 is already newer than 2468 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 14 v 2470 (ObjNum 1330 snap 413 seq_num 1330) dirty exists 2024-09-16T15:19:24.408 INFO:tasks.rados.rados.0.smithi162.stdout:3914: left oid 14 (ObjNum 1330 snap 413 seq_num 1330) 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: finishing write tid 1 to smithi16234619-34 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: finishing write tid 2 to smithi16234619-34 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: finishing write tid 3 to smithi16234619-34 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: finishing write tid 4 to smithi16234619-34 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: oid 34 updating version 0 to 2471 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: oid 34 updating version 2471 to 2473 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: oid 34 version 2473 is already newer than 2472 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 34 v 2473 (ObjNum 1331 snap 413 seq_num 1331) dirty exists 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3921: left oid 34 (ObjNum 1331 snap 413 seq_num 1331) 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3914: done (10 left) 2024-09-16T15:19:24.409 INFO:tasks.rados.rados.0.smithi162.stdout:3915: done (9 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3916: done (8 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3917: done (7 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3918: done (6 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3919: done (5 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3920: done (4 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3921: done (3 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3922: done (2 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3923: done (1 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3924: done (0 left) 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3925: read oid 1 snap -1 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3925: expect deleted 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3926: write oid 46 current snap is 413 2024-09-16T15:19:24.410 INFO:tasks.rados.rados.0.smithi162.stdout:3926: seq_num 1333 ranges {403731=668692,1587203=587797} 2024-09-16T15:19:24.412 INFO:tasks.rados.rados.0.smithi162.stdout:3926: writing smithi16234619-46 from 403731 to 1072423 tid 1 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3926: writing smithi16234619-46 from 1587203 to 2175000 tid 2 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3927: copy_from oid 40 from oid 38 current snap is 413 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3925: done (2 left) 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3928: read oid 25 snap -1 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3928: expect (ObjNum 1326 snap 411 seq_num 1326) 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3929: read oid 21 snap -1 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3929: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3930: read oid 39 snap 405 2024-09-16T15:19:24.419 INFO:tasks.rados.rados.0.smithi162.stdout:3930: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:24.420 INFO:tasks.rados.rados.0.smithi162.stdout:3931: write (excl) oid 36 current snap is 413 2024-09-16T15:19:24.420 INFO:tasks.rados.rados.0.smithi162.stdout:3931: seq_num 1334 ranges {420538=743941,1929332=491206} 2024-09-16T15:19:24.425 INFO:tasks.rados.rados.0.smithi162.stdout:3931: writing smithi16234619-36 from 420538 to 1164479 tid 1 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3931: writing smithi16234619-36 from 1929332 to 2420538 tid 2 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3932: read oid 32 snap -1 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3932: expect deleted 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3933: read oid 11 snap 408 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3933: expect (ObjNum 1197 snap 365 seq_num 1197) 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3934: read oid 18 snap -1 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3934: expect deleted 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3935: write (excl) oid 9 current snap is 413 2024-09-16T15:19:24.434 INFO:tasks.rados.rados.0.smithi162.stdout:3935: seq_num 1335 ranges {437345=419190,1587814=591466} 2024-09-16T15:19:24.438 INFO:tasks.rados.rados.0.smithi162.stdout:3935: writing smithi16234619-9 from 437345 to 856535 tid 1 2024-09-16T15:19:24.445 INFO:tasks.rados.rados.0.smithi162.stdout:3935: writing smithi16234619-9 from 1587814 to 2179280 tid 2 2024-09-16T15:19:24.446 INFO:tasks.rados.rados.0.smithi162.stdout:3936: write (excl) oid 41 current snap is 413 2024-09-16T15:19:24.446 INFO:tasks.rados.rados.0.smithi162.stdout:3936: seq_num 1336 ranges {454152=494439,1646296=735124} 2024-09-16T15:19:24.451 INFO:tasks.rados.rados.0.smithi162.stdout:3936: writing smithi16234619-41 from 454152 to 948591 tid 1 2024-09-16T15:19:24.459 INFO:tasks.rados.rados.0.smithi162.stdout:3936: writing smithi16234619-41 from 1646296 to 2381420 tid 2 2024-09-16T15:19:24.459 INFO:tasks.rados.rados.0.smithi162.stdout:3937: write oid 37 current snap is 413 2024-09-16T15:19:24.459 INFO:tasks.rados.rados.0.smithi162.stdout:3937: seq_num 1337 ranges {470959=569688,1704778=595135} 2024-09-16T15:19:24.464 INFO:tasks.rados.rados.0.smithi162.stdout:3937: writing smithi16234619-37 from 470959 to 1040647 tid 1 2024-09-16T15:19:24.471 INFO:tasks.rados.rados.0.smithi162.stdout:3937: writing smithi16234619-37 from 1704778 to 2299913 tid 2 2024-09-16T15:19:24.471 INFO:tasks.rados.rados.0.smithi162.stdout:3938: delete oid 7 current snap is 413 2024-09-16T15:19:24.471 INFO:tasks.rados.rados.0.smithi162.stdout:3930: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:24.687 INFO:tasks.rados.rados.0.smithi162.stdout:3929: expect (ObjNum 1037 snap 311 seq_num 1037) 2024-09-16T15:19:24.795 INFO:tasks.rados.rados.0.smithi162.stdout:3928: expect (ObjNum 1326 snap 411 seq_num 1326) 2024-09-16T15:19:24.899 INFO:tasks.rados.rados.0.smithi162.stdout:3933: expect (ObjNum 1197 snap 365 seq_num 1197) 2024-09-16T15:19:24.966 INFO:tasks.rados.rados.0.smithi162.stdout:3935: finishing write tid 1 to smithi16234619-9 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: finishing write tid 2 to smithi16234619-9 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3931: finishing write tid 1 to smithi16234619-36 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3931: finishing write tid 2 to smithi16234619-36 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: finishing write tid 3 to smithi16234619-9 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3926: finishing write tid 1 to smithi16234619-46 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3931: finishing write tid 3 to smithi16234619-36 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: finishing write tid 4 to smithi16234619-9 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: oid 9 updating version 0 to 2341 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: oid 9 updating version 2341 to 2342 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:3935: oid 9 updating version 2342 to 2343 2024-09-16T15:19:24.967 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 9 v 2343 (ObjNum 1335 snap 413 seq_num 1335) dirty exists 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3935: left oid 9 (ObjNum 1335 snap 413 seq_num 1335) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: finishing write tid 4 to smithi16234619-36 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: oid 36 updating version 0 to 2474 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: oid 36 updating version 2474 to 2475 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: oid 36 updating version 2475 to 2476 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 2476 (ObjNum 1334 snap 413 seq_num 1334) dirty exists 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: left oid 36 (ObjNum 1334 snap 413 seq_num 1334) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3926: finishing write tid 2 to smithi16234619-46 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3928: done (12 left) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3929: done (11 left) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3930: done (10 left) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3931: done (9 left) 2024-09-16T15:19:24.968 INFO:tasks.rados.rados.0.smithi162.stdout:3933: done (8 left) 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3934: done (7 left) 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3935: done (6 left) 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3938: done (5 left) 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3939: delete oid 27 current snap is 413 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: finishing write tid 3 to smithi16234619-46 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: finishing write tid 4 to smithi16234619-46 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: oid 46 updating version 0 to 1239 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: oid 46 version 1239 is already newer than 1238 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: oid 46 version 1239 is already newer than 1237 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 46 v 1239 (ObjNum 1333 snap 413 seq_num 1333) dirty exists 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3926: left oid 46 (ObjNum 1333 snap 413 seq_num 1333) 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3927: finishing copy_from racing read to smithi16234619-40 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:3927: finishing copy_from to smithi16234619-40 2024-09-16T15:19:24.969 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 40 v 1830 (ObjNum 1123 snap 336 seq_num 1123) dirty exists 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: finishing write tid 1 to smithi16234619-41 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: finishing write tid 2 to smithi16234619-41 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: finishing write tid 3 to smithi16234619-41 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: finishing write tid 4 to smithi16234619-41 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: oid 41 updating version 0 to 2337 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: oid 41 updating version 2337 to 2339 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: oid 41 version 2339 is already newer than 2338 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 41 v 2339 (ObjNum 1336 snap 413 seq_num 1336) dirty exists 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3936: left oid 41 (ObjNum 1336 snap 413 seq_num 1336) 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: finishing write tid 1 to smithi16234619-37 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: finishing write tid 2 to smithi16234619-37 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: finishing write tid 3 to smithi16234619-37 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: finishing write tid 4 to smithi16234619-37 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: oid 37 updating version 0 to 2308 2024-09-16T15:19:24.970 INFO:tasks.rados.rados.0.smithi162.stdout:3937: oid 37 updating version 2308 to 2309 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3937: oid 37 version 2309 is already newer than 2307 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 37 v 2309 (ObjNum 1337 snap 413 seq_num 1337) dirty exists 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3937: left oid 37 (ObjNum 1337 snap 413 seq_num 1337) 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3926: done (5 left) 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3927: done (4 left) 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3932: done (3 left) 2024-09-16T15:19:24.971 INFO:tasks.rados.rados.0.smithi162.stdout:3936: done (2 left) 2024-09-16T15:19:24.972 INFO:tasks.rados.rados.0.smithi162.stdout:3937: done (1 left) 2024-09-16T15:19:24.972 INFO:tasks.rados.rados.0.smithi162.stdout:3939: done (0 left) 2024-09-16T15:19:24.972 INFO:tasks.rados.rados.0.smithi162.stdout:3940: snap_create 2024-09-16T15:19:25.691 INFO:tasks.rados.rados.0.smithi162.stdout:3940: done (0 left) 2024-09-16T15:19:25.691 INFO:tasks.rados.rados.0.smithi162.stdout:3941: read oid 45 snap 386 2024-09-16T15:19:25.691 INFO:tasks.rados.rados.0.smithi162.stdout:3941: expect deleted 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3942: read oid 12 snap 403 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3942: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3943: read oid 24 snap 411 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3943: expect deleted 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3944: read oid 43 snap -1 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3944: expect (ObjNum 1300 snap 405 seq_num 1300) 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3945: write (excl) oid 1 current snap is 414 2024-09-16T15:19:25.692 INFO:tasks.rados.rados.0.smithi162.stdout:3945: seq_num 1340 ranges {521380=511788,1480224=742462} 2024-09-16T15:19:25.696 INFO:tasks.rados.rados.0.smithi162.stdout:3945: writing smithi16234619-1 from 521380 to 1033168 tid 1 2024-09-16T15:19:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:3945: writing smithi16234619-1 from 1480224 to 2222686 tid 2 2024-09-16T15:19:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:3946: write (excl) oid 5 current snap is 414 2024-09-16T15:19:25.703 INFO:tasks.rados.rados.0.smithi162.stdout:3946: seq_num 1341 ranges {538187=587037,1538706=602473} 2024-09-16T15:19:25.709 INFO:tasks.rados.rados.0.smithi162.stdout:3946: writing smithi16234619-5 from 538187 to 1125224 tid 1 2024-09-16T15:19:25.716 INFO:tasks.rados.rados.0.smithi162.stdout:3946: writing smithi16234619-5 from 1538706 to 2141179 tid 2 2024-09-16T15:19:25.716 INFO:tasks.rados.rados.0.smithi162.stdout:3947: delete oid 4 current snap is 414 2024-09-16T15:19:25.716 INFO:tasks.rados.rados.0.smithi162.stdout:3944: expect (ObjNum 1300 snap 405 seq_num 1300) 2024-09-16T15:19:25.850 INFO:tasks.rados.rados.0.smithi162.stdout:3942: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:26.079 INFO:tasks.rados.rados.0.smithi162.stdout:3945: finishing write tid 1 to smithi16234619-1 2024-09-16T15:19:26.079 INFO:tasks.rados.rados.0.smithi162.stdout:3945: finishing write tid 2 to smithi16234619-1 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: finishing write tid 3 to smithi16234619-1 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: finishing write tid 4 to smithi16234619-1 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: oid 1 updating version 0 to 2344 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: oid 1 updating version 2344 to 2345 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: oid 1 updating version 2345 to 2346 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 1 v 2346 (ObjNum 1340 snap 414 seq_num 1340) dirty exists 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: left oid 1 (ObjNum 1340 snap 414 seq_num 1340) 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3946: finishing write tid 1 to smithi16234619-5 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3941: done (6 left) 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3942: done (5 left) 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3943: done (4 left) 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3944: done (3 left) 2024-09-16T15:19:26.080 INFO:tasks.rados.rados.0.smithi162.stdout:3945: done (2 left) 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3947: done (1 left) 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3948: snap_create 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: finishing write tid 2 to smithi16234619-5 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: finishing write tid 3 to smithi16234619-5 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: finishing write tid 4 to smithi16234619-5 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: oid 5 updating version 0 to 2311 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: oid 5 updating version 2311 to 2312 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: oid 5 version 2312 is already newer than 2310 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 5 v 2312 (ObjNum 1341 snap 414 seq_num 1341) dirty exists 2024-09-16T15:19:26.081 INFO:tasks.rados.rados.0.smithi162.stdout:3946: left oid 5 (ObjNum 1341 snap 414 seq_num 1341) 2024-09-16T15:19:26.701 INFO:tasks.rados.rados.0.smithi162.stdout:3946: done (1 left) 2024-09-16T15:19:26.701 INFO:tasks.rados.rados.0.smithi162.stdout:3948: done (0 left) 2024-09-16T15:19:26.702 INFO:tasks.rados.rados.0.smithi162.stdout:3949: copy_from oid 32 from oid 49 current snap is 415 2024-09-16T15:19:26.702 INFO:tasks.rados.rados.0.smithi162.stdout:3950: write oid 36 current snap is 415 2024-09-16T15:19:26.702 INFO:tasks.rados.rados.0.smithi162.stdout:3950: seq_num 1343 ranges {571801=737535,1939317=489789} 2024-09-16T15:19:26.708 INFO:tasks.rados.rados.0.smithi162.stdout:3950: writing smithi16234619-36 from 571801 to 1309336 tid 1 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3950: writing smithi16234619-36 from 1939317 to 2429106 tid 2 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3951: read oid 39 snap -1 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3951: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3952: read oid 46 snap 402 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3952: expect (ObjNum 1261 snap 392 seq_num 1261) 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3953: read oid 3 snap -1 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3953: expect (ObjNum 1292 snap 397 seq_num 1292) 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3954: write oid 7 current snap is 415 2024-09-16T15:19:26.715 INFO:tasks.rados.rados.0.smithi162.stdout:3954: seq_num 1344 ranges {588608=412784,1597799=749800} 2024-09-16T15:19:26.720 INFO:tasks.rados.rados.0.smithi162.stdout:3954: writing smithi16234619-7 from 588608 to 1001392 tid 1 2024-09-16T15:19:26.728 INFO:tasks.rados.rados.0.smithi162.stdout:3954: writing smithi16234619-7 from 1597799 to 2347599 tid 2 2024-09-16T15:19:26.728 INFO:tasks.rados.rados.0.smithi162.stdout:3955: write (excl) oid 13 current snap is 415 2024-09-16T15:19:26.728 INFO:tasks.rados.rados.0.smithi162.stdout:3955: seq_num 1345 ranges {605415=488033,1656281=493458} 2024-09-16T15:19:26.734 INFO:tasks.rados.rados.0.smithi162.stdout:3955: writing smithi16234619-13 from 605415 to 1093448 tid 1 2024-09-16T15:19:26.740 INFO:tasks.rados.rados.0.smithi162.stdout:3955: writing smithi16234619-13 from 1656281 to 2149739 tid 2 2024-09-16T15:19:26.740 INFO:tasks.rados.rados.0.smithi162.stdout:3956: read oid 30 snap -1 2024-09-16T15:19:26.740 INFO:tasks.rados.rados.0.smithi162.stdout:3956: expect deleted 2024-09-16T15:19:26.741 INFO:tasks.rados.rados.0.smithi162.stdout:3957: copy_from oid 25 from oid 24 current snap is 415 2024-09-16T15:19:26.741 INFO:tasks.rados.rados.0.smithi162.stdout:3951: expect (ObjNum 1269 snap 396 seq_num 1269) 2024-09-16T15:19:26.955 INFO:tasks.rados.rados.0.smithi162.stdout:3953: expect (ObjNum 1292 snap 397 seq_num 1292) 2024-09-16T15:19:27.127 INFO:tasks.rados.rados.0.smithi162.stdout:3952: expect (ObjNum 1261 snap 392 seq_num 1261) 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: finishing write tid 1 to smithi16234619-36 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: finishing write tid 2 to smithi16234619-36 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: finishing write tid 3 to smithi16234619-36 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: finishing write tid 4 to smithi16234619-36 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: oid 36 updating version 0 to 2477 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: oid 36 updating version 2477 to 2479 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: oid 36 version 2479 is already newer than 2478 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 36 v 2479 (ObjNum 1343 snap 415 seq_num 1343) dirty exists 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3950: left oid 36 (ObjNum 1343 snap 415 seq_num 1343) 2024-09-16T15:19:27.358 INFO:tasks.rados.rados.0.smithi162.stdout:3957: finishing copy_from to smithi16234619-25 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3957: got expected ENOENT (src dne) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3957: finishing copy_from racing read to smithi16234619-25 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3949: finishing copy_from racing read to smithi16234619-32 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3950: done (8 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3951: done (7 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3952: done (6 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3953: done (5 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3956: done (4 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3957: done (3 left) 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3958: write (excl) oid 26 current snap is 415 2024-09-16T15:19:27.359 INFO:tasks.rados.rados.0.smithi162.stdout:3958: seq_num 1346 ranges {622222=679635,1831116=753469} 2024-09-16T15:19:27.364 INFO:tasks.rados.rados.0.smithi162.stdout:3958: writing smithi16234619-26 from 622222 to 1301857 tid 1 2024-09-16T15:19:27.371 INFO:tasks.rados.rados.0.smithi162.stdout:3958: writing smithi16234619-26 from 1831116 to 2584585 tid 2 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:3959: snap_remove snap 413 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:3954: finishing write tid 1 to smithi16234619-7 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:3949: finishing copy_from to smithi16234619-32 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 32 v 2335 (ObjNum 1314 snap 409 seq_num 1314) dirty exists 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:3954: finishing write tid 2 to smithi16234619-7 2024-09-16T15:19:27.372 INFO:tasks.rados.rados.0.smithi162.stdout:3954: finishing write tid 3 to smithi16234619-7 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3949: done (4 left) 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3959: done (3 left) 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3960: copy_from oid 11 from oid 9 current snap is 415 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3954: finishing write tid 4 to smithi16234619-7 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3954: oid 7 updating version 0 to 2129 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3954: oid 7 version 2129 is already newer than 2128 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3954: oid 7 version 2129 is already newer than 2127 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 2129 (ObjNum 1344 snap 415 seq_num 1344) dirty exists 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3954: left oid 7 (ObjNum 1344 snap 415 seq_num 1344) 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3955: finishing write tid 1 to smithi16234619-13 2024-09-16T15:19:28.609 INFO:tasks.rados.rados.0.smithi162.stdout:3955: finishing write tid 2 to smithi16234619-13 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: finishing write tid 3 to smithi16234619-13 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: finishing write tid 4 to smithi16234619-13 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: oid 13 updating version 0 to 2313 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: oid 13 updating version 2313 to 2314 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: oid 13 updating version 2314 to 2315 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 2315 (ObjNum 1345 snap 415 seq_num 1345) dirty exists 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3955: left oid 13 (ObjNum 1345 snap 415 seq_num 1345) 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: finishing write tid 1 to smithi16234619-26 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: finishing write tid 2 to smithi16234619-26 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: finishing write tid 3 to smithi16234619-26 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: finishing write tid 4 to smithi16234619-26 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: oid 26 updating version 0 to 2563 2024-09-16T15:19:28.610 INFO:tasks.rados.rados.0.smithi162.stdout:3958: oid 26 updating version 2563 to 2565 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3958: oid 26 version 2565 is already newer than 2564 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 26 v 2565 (ObjNum 1346 snap 415 seq_num 1346) dirty exists 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3958: left oid 26 (ObjNum 1346 snap 415 seq_num 1346) 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3954: done (3 left) 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3955: done (2 left) 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3958: done (1 left) 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3961: write oid 22 current snap is 415 2024-09-16T15:19:28.611 INFO:tasks.rados.rados.0.smithi162.stdout:3961: seq_num 1347 ranges {639029=754884,2173245=465784} 2024-09-16T15:19:28.616 INFO:tasks.rados.rados.0.smithi162.stdout:3961: writing smithi16234619-22 from 639029 to 1393913 tid 1 2024-09-16T15:19:28.625 INFO:tasks.rados.rados.0.smithi162.stdout:3961: writing smithi16234619-22 from 2173245 to 2639029 tid 2 2024-09-16T15:19:28.625 INFO:tasks.rados.rados.0.smithi162.stdout:3962: snap_remove snap 402 2024-09-16T15:19:29.684 INFO:tasks.rados.rados.0.smithi162.stdout:3962: done (2 left) 2024-09-16T15:19:29.684 INFO:tasks.rados.rados.0.smithi162.stdout:3963: write oid 13 current snap is 415 2024-09-16T15:19:29.684 INFO:tasks.rados.rados.0.smithi162.stdout:3963: seq_num 1348 ranges {655836=430133,1831727=757138} 2024-09-16T15:19:29.690 INFO:tasks.rados.rados.0.smithi162.stdout:3963: writing smithi16234619-13 from 655836 to 1085969 tid 1 2024-09-16T15:19:29.699 INFO:tasks.rados.rados.0.smithi162.stdout:3963: writing smithi16234619-13 from 1831727 to 2588865 tid 2 2024-09-16T15:19:29.699 INFO:tasks.rados.rados.0.smithi162.stdout:3964: snap_remove snap 400 2024-09-16T15:19:30.869 INFO:tasks.rados.rados.0.smithi162.stdout:3964: done (3 left) 2024-09-16T15:19:30.869 INFO:tasks.rados.rados.0.smithi162.stdout:3965: snap_create 2024-09-16T15:19:30.869 INFO:tasks.rados.rados.0.smithi162.stdout:3961: finishing write tid 1 to smithi16234619-22 2024-09-16T15:19:30.869 INFO:tasks.rados.rados.0.smithi162.stdout:3961: finishing write tid 2 to smithi16234619-22 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: finishing write tid 3 to smithi16234619-22 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: finishing write tid 4 to smithi16234619-22 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: oid 22 updating version 0 to 2342 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: oid 22 version 2342 is already newer than 2341 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: oid 22 version 2342 is already newer than 2340 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 2342 (ObjNum 1347 snap 415 seq_num 1347) dirty exists 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3961: left oid 22 (ObjNum 1347 snap 415 seq_num 1347) 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3960: finishing copy_from racing read to smithi16234619-11 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3960: finishing copy_from to smithi16234619-11 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 11 v 2343 (ObjNum 1335 snap 413 seq_num 1335) dirty exists 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3963: finishing write tid 1 to smithi16234619-13 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3963: finishing write tid 2 to smithi16234619-13 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3963: finishing write tid 3 to smithi16234619-13 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3963: finishing write tid 4 to smithi16234619-13 2024-09-16T15:19:30.870 INFO:tasks.rados.rados.0.smithi162.stdout:3963: oid 13 updating version 0 to 2318 2024-09-16T15:19:30.871 INFO:tasks.rados.rados.0.smithi162.stdout:3963: oid 13 version 2318 is already newer than 2316 2024-09-16T15:19:30.871 INFO:tasks.rados.rados.0.smithi162.stdout:3963: oid 13 version 2318 is already newer than 2317 2024-09-16T15:19:30.871 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 2318 (ObjNum 1348 snap 415 seq_num 1348) dirty exists 2024-09-16T15:19:30.871 INFO:tasks.rados.rados.0.smithi162.stdout:3963: left oid 13 (ObjNum 1348 snap 415 seq_num 1348) 2024-09-16T15:19:32.122 INFO:tasks.rados.rados.0.smithi162.stdout:3960: done (3 left) 2024-09-16T15:19:32.122 INFO:tasks.rados.rados.0.smithi162.stdout:3961: done (2 left) 2024-09-16T15:19:32.122 INFO:tasks.rados.rados.0.smithi162.stdout:3963: done (1 left) 2024-09-16T15:19:32.122 INFO:tasks.rados.rados.0.smithi162.stdout:3965: done (0 left) 2024-09-16T15:19:32.123 INFO:tasks.rados.rados.0.smithi162.stdout:3966: snap_create 2024-09-16T15:19:33.665 INFO:tasks.rados.rados.0.smithi162.stdout:3966: done (0 left) 2024-09-16T15:19:33.665 INFO:tasks.rados.rados.0.smithi162.stdout:3967: rollback oid 47 current snap is 417 2024-09-16T15:19:33.665 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 47 to 410 2024-09-16T15:19:33.665 INFO:tasks.rados.rados.0.smithi162.stdout:3968: snap_remove snap 407 2024-09-16T15:19:34.280 INFO:tasks.rados.rados.0.smithi162.stdout:3968: done (1 left) 2024-09-16T15:19:34.280 INFO:tasks.rados.rados.0.smithi162.stdout:3969: snap_remove snap 411 2024-09-16T15:19:35.283 INFO:tasks.rados.rados.0.smithi162.stdout:3969: done (1 left) 2024-09-16T15:19:35.283 INFO:tasks.rados.rados.0.smithi162.stdout:3970: snap_remove snap 416 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3970: done (1 left) 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3971: rollback oid 30 current snap is 417 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 387 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3967: finishing rollback tid 0 to smithi16234619-47 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3967: finishing rollback tid 1 to smithi16234619-47 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3967: finishing rollback tid 2 to smithi16234619-47 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 47 v 2429 (ObjNum 1307 snap 406 seq_num 1307) dirty exists 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3967: done (1 left) 2024-09-16T15:19:36.285 INFO:tasks.rados.rados.0.smithi162.stdout:3972: snap_remove snap 399 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:3971: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:3971: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 2332 (ObjNum 1219 snap 375 seq_num 1219) dirty exists 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:3971: done (1 left) 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:3972: done (0 left) 2024-09-16T15:19:37.053 INFO:tasks.rados.rados.0.smithi162.stdout:3973: read oid 12 snap -1 2024-09-16T15:19:37.054 INFO:tasks.rados.rados.0.smithi162.stdout:3973: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:37.054 INFO:tasks.rados.rados.0.smithi162.stdout:3974: write oid 13 current snap is 417 2024-09-16T15:19:37.054 INFO:tasks.rados.rados.0.smithi162.stdout:3974: seq_num 1349 ranges {672643=505382,1890209=500796} 2024-09-16T15:19:37.060 INFO:tasks.rados.rados.0.smithi162.stdout:3974: writing smithi16234619-13 from 672643 to 1178025 tid 1 2024-09-16T15:19:37.067 INFO:tasks.rados.rados.0.smithi162.stdout:3974: writing smithi16234619-13 from 1890209 to 2391005 tid 2 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3975: read oid 4 snap -1 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3975: expect deleted 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3976: read oid 46 snap -1 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3976: expect (ObjNum 1333 snap 413 seq_num 1333) 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3977: read oid 1 snap 395 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3977: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3978: read oid 42 snap -1 2024-09-16T15:19:37.068 INFO:tasks.rados.rados.0.smithi162.stdout:3978: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3979: copy_from oid 50 from oid 48 current snap is 417 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3980: read oid 15 snap -1 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3980: expect (ObjNum 1328 snap 411 seq_num 1328) 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3981: rollback oid 30 current snap is 417 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 30 to 394 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3982: write oid 19 current snap is 417 2024-09-16T15:19:37.069 INFO:tasks.rados.rados.0.smithi162.stdout:3982: seq_num 1350 ranges {689450=580631,1948691=740759} 2024-09-16T15:19:37.075 INFO:tasks.rados.rados.0.smithi162.stdout:3982: writing smithi16234619-19 from 689450 to 1270081 tid 1 2024-09-16T15:19:37.084 INFO:tasks.rados.rados.0.smithi162.stdout:3982: writing smithi16234619-19 from 1948691 to 2689450 tid 2 2024-09-16T15:19:37.084 INFO:tasks.rados.rados.0.smithi162.stdout:3973: expect (ObjNum 1253 snap 387 seq_num 1253) 2024-09-16T15:19:37.316 INFO:tasks.rados.rados.0.smithi162.stdout:3977: expect (ObjNum 1241 snap 384 seq_num 1241) 2024-09-16T15:19:37.551 INFO:tasks.rados.rados.0.smithi162.stdout:3979: finishing copy_from to smithi16234619-50 2024-09-16T15:19:37.551 INFO:tasks.rados.rados.0.smithi162.stdout:3979: got expected ENOENT (src dne) 2024-09-16T15:19:37.551 INFO:tasks.rados.rados.0.smithi162.stdout:3979: finishing copy_from racing read to smithi16234619-50 2024-09-16T15:19:37.551 INFO:tasks.rados.rados.0.smithi162.stdout:3976: expect (ObjNum 1333 snap 413 seq_num 1333) 2024-09-16T15:19:37.715 INFO:tasks.rados.rados.0.smithi162.stdout:3980: expect (ObjNum 1328 snap 411 seq_num 1328) 2024-09-16T15:19:37.817 INFO:tasks.rados.rados.0.smithi162.stdout:3978: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3981: finishing rollback tid 0 to smithi16234619-30 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3981: finishing rollback tid 1 to smithi16234619-30 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3981: finishing rollback tid 2 to smithi16234619-30 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 30 v 2335 (ObjNum 1219 snap 375 seq_num 1219) dirty exists 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: finishing write tid 1 to smithi16234619-13 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: finishing write tid 2 to smithi16234619-13 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: finishing write tid 3 to smithi16234619-13 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: finishing write tid 4 to smithi16234619-13 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: oid 13 updating version 0 to 2333 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: oid 13 updating version 2333 to 2334 2024-09-16T15:19:38.056 INFO:tasks.rados.rados.0.smithi162.stdout:3974: oid 13 version 2334 is already newer than 2332 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 13 v 2334 (ObjNum 1349 snap 417 seq_num 1349) dirty exists 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3974: left oid 13 (ObjNum 1349 snap 417 seq_num 1349) 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: finishing write tid 1 to smithi16234619-19 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: finishing write tid 2 to smithi16234619-19 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: finishing write tid 3 to smithi16234619-19 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: finishing write tid 4 to smithi16234619-19 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: oid 19 updating version 0 to 2481 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: oid 19 updating version 2481 to 2482 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: oid 19 version 2482 is already newer than 2480 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 19 v 2482 (ObjNum 1350 snap 417 seq_num 1350) dirty exists 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3982: left oid 19 (ObjNum 1350 snap 417 seq_num 1350) 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3973: done (9 left) 2024-09-16T15:19:38.057 INFO:tasks.rados.rados.0.smithi162.stdout:3974: done (8 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3975: done (7 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3976: done (6 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3977: done (5 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3978: done (4 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3979: done (3 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3980: done (2 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3981: done (1 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3982: done (0 left) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3983: read oid 8 snap -1 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3983: expect (ObjNum 1329 snap 411 seq_num 1329) 2024-09-16T15:19:38.058 INFO:tasks.rados.rados.0.smithi162.stdout:3984: snap_remove snap 415 2024-09-16T15:19:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3984: done (1 left) 2024-09-16T15:19:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3985: write oid 20 current snap is 417 2024-09-16T15:19:38.555 INFO:tasks.rados.rados.0.smithi162.stdout:3985: seq_num 1351 ranges {706257=655880,1890820=504465} 2024-09-16T15:19:38.563 INFO:tasks.rados.rados.0.smithi162.stdout:3985: writing smithi16234619-20 from 706257 to 1362137 tid 1 2024-09-16T15:19:38.570 INFO:tasks.rados.rados.0.smithi162.stdout:3985: writing smithi16234619-20 from 1890820 to 2395285 tid 2 2024-09-16T15:19:38.570 INFO:tasks.rados.rados.0.smithi162.stdout:3986: read oid 42 snap -1 2024-09-16T15:19:38.570 INFO:tasks.rados.rados.0.smithi162.stdout:3986: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:38.570 INFO:tasks.rados.rados.0.smithi162.stdout:3987: write (excl) oid 7 current snap is 417 2024-09-16T15:19:38.570 INFO:tasks.rados.rados.0.smithi162.stdout:3987: seq_num 1352 ranges {723064=731129,1949302=764476} 2024-09-16T15:19:38.578 INFO:tasks.rados.rados.0.smithi162.stdout:3987: writing smithi16234619-7 from 723064 to 1454193 tid 1 2024-09-16T15:19:38.585 INFO:tasks.rados.rados.0.smithi162.stdout:3987: writing smithi16234619-7 from 1949302 to 2713778 tid 2 2024-09-16T15:19:38.586 INFO:tasks.rados.rados.0.smithi162.stdout:3988: write oid 23 current snap is 417 2024-09-16T15:19:38.586 INFO:tasks.rados.rados.0.smithi162.stdout:3988: seq_num 1353 ranges {739871=406378,1607784=508134} 2024-09-16T15:19:38.592 INFO:tasks.rados.rados.0.smithi162.stdout:3988: writing smithi16234619-23 from 739871 to 1146249 tid 1 2024-09-16T15:19:38.597 INFO:tasks.rados.rados.0.smithi162.stdout:3988: writing smithi16234619-23 from 1607784 to 2115918 tid 2 2024-09-16T15:19:38.597 INFO:tasks.rados.rados.0.smithi162.stdout:3989: snap_remove snap 414 2024-09-16T15:19:39.689 INFO:tasks.rados.rados.0.smithi162.stdout:3989: done (5 left) 2024-09-16T15:19:39.689 INFO:tasks.rados.rados.0.smithi162.stdout:3990: snap_remove snap 405 2024-09-16T15:19:40.695 INFO:tasks.rados.rados.0.smithi162.stdout:3983: expect (ObjNum 1329 snap 411 seq_num 1329) 2024-09-16T15:19:40.775 INFO:tasks.rados.rados.0.smithi162.stdout:3986: expect (ObjNum 1151 snap 343 seq_num 1151) 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: finishing write tid 1 to smithi16234619-20 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: finishing write tid 2 to smithi16234619-20 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: finishing write tid 3 to smithi16234619-20 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: finishing write tid 4 to smithi16234619-20 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: oid 20 updating version 0 to 1730 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: oid 20 version 1730 is already newer than 1729 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:3985: oid 20 version 1730 is already newer than 1728 2024-09-16T15:19:41.014 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 20 v 1730 (ObjNum 1351 snap 417 seq_num 1351) dirty exists 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3985: left oid 20 (ObjNum 1351 snap 417 seq_num 1351) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: finishing write tid 1 to smithi16234619-23 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3983: done (5 left) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3985: done (4 left) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3986: done (3 left) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3990: done (2 left) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3991: snap_create 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: finishing write tid 2 to smithi16234619-23 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: finishing write tid 3 to smithi16234619-23 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: finishing write tid 4 to smithi16234619-23 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: oid 23 updating version 0 to 1493 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: oid 23 version 1493 is already newer than 1492 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: oid 23 version 1493 is already newer than 1491 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 23 v 1493 (ObjNum 1353 snap 417 seq_num 1353) dirty exists 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3988: left oid 23 (ObjNum 1353 snap 417 seq_num 1353) 2024-09-16T15:19:41.015 INFO:tasks.rados.rados.0.smithi162.stdout:3987: finishing write tid 1 to smithi16234619-7 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: finishing write tid 2 to smithi16234619-7 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: finishing write tid 3 to smithi16234619-7 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: finishing write tid 4 to smithi16234619-7 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: oid 7 updating version 0 to 2130 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: oid 7 updating version 2130 to 2132 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: oid 7 version 2132 is already newer than 2131 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 7 v 2132 (ObjNum 1352 snap 417 seq_num 1352) dirty exists 2024-09-16T15:19:41.016 INFO:tasks.rados.rados.0.smithi162.stdout:3987: left oid 7 (ObjNum 1352 snap 417 seq_num 1352) 2024-09-16T15:19:41.696 INFO:tasks.rados.rados.0.smithi162.stdout:3987: done (2 left) 2024-09-16T15:19:41.697 INFO:tasks.rados.rados.0.smithi162.stdout:3988: done (1 left) 2024-09-16T15:19:41.697 INFO:tasks.rados.rados.0.smithi162.stdout:3991: done (0 left) 2024-09-16T15:19:41.697 INFO:tasks.rados.rados.0.smithi162.stdout:3992: read oid 37 snap -1 2024-09-16T15:19:41.697 INFO:tasks.rados.rados.0.smithi162.stdout:3992: expect (ObjNum 1337 snap 413 seq_num 1337) 2024-09-16T15:19:41.697 INFO:tasks.rados.rados.0.smithi162.stdout:3993: delete oid 13 current snap is 418 2024-09-16T15:19:41.705 INFO:tasks.rados.rados.0.smithi162.stdout:3992: expect (ObjNum 1337 snap 413 seq_num 1337) 2024-09-16T15:19:41.785 INFO:tasks.rados.rados.0.smithi162.stdout:3992: done (1 left) 2024-09-16T15:19:41.785 INFO:tasks.rados.rados.0.smithi162.stdout:3993: done (0 left) 2024-09-16T15:19:41.785 INFO:tasks.rados.rados.0.smithi162.stdout:3994: snap_remove snap 380 2024-09-16T15:19:42.058 INFO:tasks.rados.rados.0.smithi162.stdout:3994: done (0 left) 2024-09-16T15:19:42.058 INFO:tasks.rados.rados.0.smithi162.stdout:3995: read oid 37 snap -1 2024-09-16T15:19:42.058 INFO:tasks.rados.rados.0.smithi162.stdout:3995: expect (ObjNum 1337 snap 413 seq_num 1337) 2024-09-16T15:19:42.058 INFO:tasks.rados.rados.0.smithi162.stdout:3996: rollback oid 22 current snap is 418 2024-09-16T15:19:42.058 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 22 to 412 2024-09-16T15:19:42.059 INFO:tasks.rados.rados.0.smithi162.stdout:3997: snap_remove snap 406 2024-09-16T15:19:43.074 INFO:tasks.rados.rados.0.smithi162.stdout:3997: done (2 left) 2024-09-16T15:19:43.075 INFO:tasks.rados.rados.0.smithi162.stdout:3998: write oid 10 current snap is 418 2024-09-16T15:19:43.075 INFO:tasks.rados.rados.0.smithi162.stdout:3998: seq_num 1355 ranges {773485=673229,2124748=511803} 2024-09-16T15:19:43.083 INFO:tasks.rados.rados.0.smithi162.stdout:3998: writing smithi16234619-10 from 773485 to 1446714 tid 1 2024-09-16T15:19:43.091 INFO:tasks.rados.rados.0.smithi162.stdout:3998: writing smithi16234619-10 from 2124748 to 2636551 tid 2 2024-09-16T15:19:43.091 INFO:tasks.rados.rados.0.smithi162.stdout:3999: rollback oid 45 current snap is 418 2024-09-16T15:19:43.091 INFO:tasks.rados.rados.0.smithi162.stdout:rollback oid 45 to 386 2024-09-16T15:19:43.091 INFO:tasks.rados.rados.0.smithi162.stdout:3995: expect (ObjNum 1337 snap 413 seq_num 1337) 2024-09-16T15:19:43.174 INFO:tasks.rados.rados.0.smithi162.stdout:3996: finishing rollback tid 0 to smithi16234619-22 2024-09-16T15:19:43.174 INFO:tasks.rados.rados.0.smithi162.stdout:3996: finishing rollback tid 1 to smithi16234619-22 2024-09-16T15:19:43.174 INFO:tasks.rados.rados.0.smithi162.stdout:3995: done (3 left) 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 3 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3996: finishing rollback tid 2 to smithi16234619-22 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 22 v 2345 (ObjNum 1269 snap 396 seq_num 1269) dirty exists 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3996: done (2 left) 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 2 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: finishing write tid 1 to smithi16234619-10 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: finishing write tid 2 to smithi16234619-10 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: finishing write tid 3 to smithi16234619-10 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: finishing write tid 4 to smithi16234619-10 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: oid 10 updating version 0 to 2613 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: oid 10 version 2613 is already newer than 2612 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: oid 10 version 2613 is already newer than 2611 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 10 v 2613 (ObjNum 1355 snap 418 seq_num 1355) dirty exists 2024-09-16T15:19:43.175 INFO:tasks.rados.rados.0.smithi162.stdout:3998: left oid 10 (ObjNum 1355 snap 418 seq_num 1355) 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout:3999: finishing rollback tid 0 to smithi16234619-45 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout:3998: done (1 left) 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout: waiting on 1 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout:3999: finishing rollback tid 1 to smithi16234619-45 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout:update_object_version oid 45 v 2616 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stdout:3999: done (0 left) 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stderr:0 errors. 2024-09-16T15:19:43.176 INFO:tasks.rados.rados.0.smithi162.stderr: 2024-09-16T15:19:43.178 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:19:43.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.273+0000 7feea6f1f640 10 monclient: get_monmap_and_config 2024-09-16T15:19:43.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.273+0000 7feea6f1f640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.273+0000 7feea6f1f640 10 monclient: monmap: 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:43.275244+0000 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:43.275244+0000 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.276 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:43.277 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 Processor -- start 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 -- start start 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient: init 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient: _add_conns ranks=[8,10,7,0,9,4,3,6,2,1,5] 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-e con 0x7feea0136030 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feea009bfe0 0x7feea009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-f con 0x7feea009bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 0x7feea01370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-i con 0x7feea0136c60 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea009b600 con 0x7feea009bfe0 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feea00a5740 con 0x7feea0136c60 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7feea00a52c0 con 0x7feea0136030 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea6f1f640 10 monclient(hunting): authenticate will time out at 1919.864635s 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea54bc640 1 -- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 msgr2=0x7feea01370e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.162:3303/0 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea54bc640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 0x7feea01370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:53988/0 (socket says 172.21.15.134:53988) 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 1 -- 172.21.15.134:0/3461561747 learned_addr learned my addr 172.21.15.134:0/3461561747 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): get_auth_request con 0x7feea0136030 auth_method 0 2024-09-16T15:19:43.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 1 --2- 172.21.15.134:0/3461561747 >> v2:172.21.15.162:3300/0 conn(0x7feea009bfe0 0x7feea009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): get_auth_request con 0x7feea009bfe0 auth_method 0 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.274+0000 7fee9ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 10 monclient(hunting): handle_auth_done global_id 85067 payload 274 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 -- 172.21.15.134:0/3461561747 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 msgr2=0x7feea01370e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3461561747 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 0x7feea01370e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 -- 172.21.15.134:0/3461561747 >> v2:172.21.15.162:3300/0 conn(0x7feea009bfe0 msgr2=0x7feea009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3461561747 >> v2:172.21.15.162:3300/0 conn(0x7feea009bfe0 0x7feea009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 monclient: found mon.noname-e 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.275+0000 7feea4cbb640 1 -- 172.21.15.134:0/3461561747 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0073250 con 0x7feea0136030 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7feea4cbb640 1 -- 172.21.15.134:0/3461561747 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 msgr2=0x7feea0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3461561747 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee90009870 tx=0x7fee9002a0d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=c26aab7cc8999f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7fee9f7fe640 1 -- 172.21.15.134:0/3461561747 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee9002ae30 con 0x7feea0136030 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7fee9f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7fee9f7fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.277+0000 7fee9f7fe640 10 monclient: dump: 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.286 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 1 -- 172.21.15.134:0/3461561747 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee900307b0 con 0x7feea0136030 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 5 monclient: authenticate success, global_id 85067 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 1 -- 172.21.15.134:0/3461561747 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee90030d90 con 0x7feea0136030 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 4 set_mon_vals no callback set 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:19:43.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7fee9f7fe640 10 monclient: dump: 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.288 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 10 monclient: shutdown 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 -- 172.21.15.134:0/3461561747 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 msgr2=0x7feea0136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3461561747 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee90009870 tx=0x7fee9002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 -- 172.21.15.134:0/3461561747 shutdown_connections 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3461561747 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7feea0136c60 0x7feea01370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3461561747 >> v2:172.21.15.162:3300/0 conn(0x7feea009bfe0 0x7feea009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3461561747 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7feea0136030 0x7feea0136410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 -- 172.21.15.134:0/3461561747 >> 172.21.15.134:0/3461561747 conn(0x7feea00a9ea0 msgr2=0x7feea00a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 -- 172.21.15.134:0/3461561747 shutdown_connections 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.278+0000 7feea6f1f640 1 -- 172.21.15.134:0/3461561747 wait complete. 2024-09-16T15:19:43.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.279+0000 7feea6f1f640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.279+0000 7feea6f1f640 10 monclient: monmap: 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.290 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.291 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.279+0000 7feea6f1f640 1 Processor -- start 2024-09-16T15:19:43.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.279+0000 7feea6f1f640 1 -- start start 2024-09-16T15:19:43.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.279+0000 7feea6f1f640 10 monclient: init 2024-09-16T15:19:43.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient: _add_conns ranks=[6,5,4,3,8,2,0,1,7] 2024-09-16T15:19:43.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-d con 0x7feea009bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:43.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 0x7feea01352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-h con 0x7feea0136030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 0x7feea01cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): picked mon.noname-c con 0x7feea0136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feea0120ba0 con 0x7feea0136c60 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feea0121eb0 con 0x7feea0136030 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7feea0122470 con 0x7feea009bfe0 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea6f1f640 10 monclient(hunting): authenticate will time out at 1919.870210s 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 0x7feea01cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:60328/0 (socket says 172.21.15.134:60328) 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 1 -- 172.21.15.134:0/3401364714 learned_addr learned my addr 172.21.15.134:0/3401364714 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): get_auth_request con 0x7feea009bfe0 auth_method 0 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): get_auth_request con 0x7feea0136c60 auth_method 0 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 0x7feea01352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 10 monclient(hunting): get_auth_request con 0x7feea0136030 auth_method 0 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient(hunting): handle_auth_done global_id 124339 payload 824 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 -- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 msgr2=0x7feea0134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 -- 172.21.15.134:0/3401364714 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 msgr2=0x7feea01352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 0x7feea01352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 monclient: found mon.noname-c 2024-09-16T15:19:43.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea54bc640 1 -- 172.21.15.134:0/3401364714 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee900094d0 con 0x7feea0136c60 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.280+0000 7fee9ffff640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 0x7feea01352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.282+0000 7feea54bc640 1 -- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 msgr2=0x7feea01cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.282+0000 7feea54bc640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 0x7feea01cb290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee9800b360 tx=0x7fee9800b820 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.282+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee98003d80 con 0x7feea0136c60 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: dump: 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.299 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98006540 con 0x7feea0136c60 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 5 monclient: authenticate success, global_id 124339 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 10 monclient: _renew_subs 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea00a7a80 con 0x7feea0136c60 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fee980066b0 con 0x7feea0136c60 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 10 monclient: dump: 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.301 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 10 monclient: _renew_subs 2024-09-16T15:19:43.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9effd640 4 set_mon_vals no callback set 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea01bf800 con 0x7feea0136c60 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.283+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fee980060d0 con 0x7feea0136c60 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea6f1f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea6f1f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea6f1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea009c400 con 0x7feea0136c60 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7fee9d7fa640 1 --2- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 0x7fee8403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(860..860 src has 1..860) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fee980491c0 con 0x7feea0136c60 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 0x7fee8403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.284+0000 7feea4cbb640 10 monclient: get_auth_request con 0x7fee840388b0 auth_method 0 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.286+0000 7feea4cbb640 1 --2- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 0x7fee8403ad70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee90035040 tx=0x7fee9002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.303 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.288+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fee98006380 con 0x7feea0136c60 2024-09-16T15:19:43.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.288+0000 7fee9d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.288+0000 7fee9d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:43.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.453+0000 7feea6f1f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:19:43.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.453+0000 7feea6f1f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:19:43.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.453+0000 7feea6f1f640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:19:43.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.453+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea0121ff0 con 0x7feea0136c60 2024-09-16T15:19:43.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.454+0000 7fee9d7fa640 1 -- 172.21.15.134:0/3401364714 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v860) v1 ==== 74+0+12736 (secure 0 0 0) 0x7fee98017030 con 0x7feea0136c60 2024-09-16T15:19:43.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.454+0000 7fee9d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:19:43.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.454+0000 7fee9d7fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:19:43.457 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:19:43.457 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":860,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:19:43.058372+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:00:45.513777+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-16T15:00:56.161730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"860","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":827,"snap_epoch":860,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.343686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.259639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.265227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.302963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":748,"length":1},{"begin":789,"length":1},{"begin":798,"length":2},{"begin":812,"length":1},{"begin":816,"length":2}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":799,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 msgr2=0x7fee8403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 0x7fee8403ad70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee90035040 tx=0x7fee9002e000 comp rx=0 tx=0).stop 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 10 monclient: shutdown 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 msgr2=0x7feea01cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 0x7feea01cb290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee9800b360 tx=0x7fee9800b820 comp rx=0 tx=0).stop 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 shutdown_connections 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> v2:172.21.15.162:6800/980547263 conn(0x7fee840388b0 0x7fee8403ad70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7feea0136c60 0x7feea01cb290 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7feea0136030 0x7feea01352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 --2- 172.21.15.134:0/3401364714 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7feea009bfe0 0x7feea0134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 >> 172.21.15.134:0/3401364714 conn(0x7feea00a9ea0 msgr2=0x7feea006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:43.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 shutdown_connections 2024-09-16T15:19:43.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.456+0000 7feea6f1f640 1 -- 172.21.15.134:0/3401364714 wait complete. 2024-09-16T15:19:43.469 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:19:43.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 10 monclient: get_monmap_and_config 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 10 monclient: monmap: 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:43.565972+0000 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:43.565972+0000 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.567 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 1 Processor -- start 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 1 -- start start 2024-09-16T15:19:43.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.564+0000 7fd37fb7d640 10 monclient: init 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient: _add_conns ranks=[6,2,0,10,4,1,3,9,8,7,5] 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 0x7fd378136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-d con 0x7fd378136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd37809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-b con 0x7fd37809bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd378136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378073250 con 0x7fd378136c60 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37809b600 con 0x7fd37809bfe0 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780a5740 con 0x7fd378136030 2024-09-16T15:19:43.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37fb7d640 10 monclient(hunting): authenticate will time out at 1920.155415s 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd37809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 0x7fd378136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37e11a640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd378136c60 0x7fd3781370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:37900/0 (socket says 172.21.15.134:37900) 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37e11a640 1 -- 172.21.15.134:0/139870612 learned_addr learned my addr 172.21.15.134:0/139870612 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): get_auth_request con 0x7fd378136030 auth_method 0 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): get_auth_request con 0x7fd37809bfe0 auth_method 0 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient(hunting): handle_auth_done global_id 94776 payload 274 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 -- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 msgr2=0x7fd37809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd37809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 -- 172.21.15.134:0/139870612 >> v1:172.21.15.134:6789/0 conn(0x7fd378136c60 legacy=0x7fd3781370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 monclient: found mon.noname-d 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d919640 1 -- 172.21.15.134:0/139870612 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780a52c0 con 0x7fd378136030 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.565+0000 7fd37d118640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd37809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37d919640 1 -- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 msgr2=0x7fd378136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37d919640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 0x7fd378136410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd3740097d0 tx=0x7fd37402a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2a03c92795080a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 1 -- 172.21.15.134:0/139870612 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd37402ad20 con 0x7fd378136030 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: dump: 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.575 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 1 -- 172.21.15.134:0/139870612 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3740307b0 con 0x7fd378136030 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37fb7d640 5 monclient: authenticate success, global_id 94776 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37fb7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:43.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37fb7d640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 1 -- 172.21.15.134:0/139870612 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd37402aea0 con 0x7fd378136030 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37fb7d640 4 set_mon_vals no callback set 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.568+0000 7fd37c917640 10 monclient: dump: 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.577 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 10 monclient: shutdown 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 -- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 msgr2=0x7fd378136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 0x7fd378136410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd3740097d0 tx=0x7fd37402a000 comp rx=0 tx=0).stop 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 -- 172.21.15.134:0/139870612 shutdown_connections 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd37809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/139870612 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fd378136030 0x7fd378136410 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 -- 172.21.15.134:0/139870612 >> 172.21.15.134:0/139870612 conn(0x7fd3780a9ea0 msgr2=0x7fd3780a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 -- 172.21.15.134:0/139870612 shutdown_connections 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 1 -- 172.21.15.134:0/139870612 wait complete. 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.569+0000 7fd37fb7d640 10 monclient: monmap: 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.579 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.580 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.580 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.580 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 Processor -- start 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 -- start start 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient: init 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient: _add_conns ranks=[2,3,4,5,1,6,0,7,8] 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-b con 0x7fd37809bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd378136030 0x7fd378135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-g con 0x7fd378136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd378136c60 0x7fd37811fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): picked mon.noname-c con 0x7fd378136c60 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd378135aa0 con 0x7fd37809bfe0 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd378122420 con 0x7fd378136030 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780a3450 con 0x7fd378136c60 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37fb7d640 10 monclient(hunting): authenticate will time out at 1920.160832s 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47628/0 (socket says 172.21.15.134:47628) 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 1 -- 172.21.15.134:0/2185326966 learned_addr learned my addr 172.21.15.134:0/2185326966 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): get_auth_request con 0x7fd37809bfe0 auth_method 0 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37e11a640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd378136c60 0x7fd37811fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37e11a640 10 monclient(hunting): get_auth_request con 0x7fd378136c60 auth_method 0 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37e11a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37e11a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37e11a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.570+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37e11a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37e11a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37e11a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 10 monclient(hunting): handle_auth_done global_id 114626 payload 824 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d118640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd378136030 0x7fd378135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 -- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd378136c60 msgr2=0x7fd37811fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd378136c60 0x7fd37811fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 -- 172.21.15.134:0/2185326966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd378136030 msgr2=0x7fd378135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd378136030 0x7fd378135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 monclient: found mon.noname-b 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.571+0000 7fd37d919640 1 -- 172.21.15.134:0/2185326966 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368009400 con 0x7fd37809bfe0 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37d919640 1 -- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 msgr2=0x7fd378135020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37d919640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd374035040 tx=0x7fd3740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd374004460 con 0x7fd37809bfe0 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: dump: 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.587 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374003a80 con 0x7fd37809bfe0 2024-09-16T15:19:43.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 5 monclient: authenticate success, global_id 114626 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 10 monclient: _renew_subs 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd374009400 con 0x7fd37809bfe0 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd374003bf0 con 0x7fd37809bfe0 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36e7fc640 10 monclient: dump: 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.589 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd36ffff640 4 set_mon_vals no callback set 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 10 monclient: _renew_subs 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.573+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3780a7d00 con 0x7fd37809bfe0 2024-09-16T15:19:43.590 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd374039040 con 0x7fd37809bfe0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37fb7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37fb7d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37fb7d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780a0330 con 0x7fd37809bfe0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd36e7fc640 1 --2- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 0x7fd35c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(860..860 src has 1..860) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fd37406a860 con 0x7fd37809bfe0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37d118640 1 --2- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 0x7fd35c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.574+0000 7fd37d118640 10 monclient: get_auth_request con 0x7fd35c0388b0 auth_method 0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.577+0000 7fd37d118640 1 --2- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 0x7fd35c03ad70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3780a5950 tx=0x7fd368008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.579+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd37403a980 con 0x7fd37809bfe0 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.579+0000 7fd36e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.591 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.579+0000 7fd36e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:43.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.741+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd378061ae0 con 0x7fd35c0388b0 2024-09-16T15:19:43.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.744+0000 7fd36e7fc640 1 -- 172.21.15.134:0/2185326966 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+99501 (secure 0 0 0) 0x7fd3680109f0 con 0x7fd35c0388b0 2024-09-16T15:19:43.748 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:19:43.748 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 msgr2=0x7fd35c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 0x7fd35c03ad70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd3780a5950 tx=0x7fd368008040 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 10 monclient: shutdown 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 msgr2=0x7fd378135020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd374035040 tx=0x7fd3740046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 shutdown_connections 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd378136c60 0x7fd37811fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> v2:172.21.15.162:6800/980547263 conn(0x7fd35c0388b0 0x7fd35c03ad70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd378136030 0x7fd378135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 --2- 172.21.15.134:0/2185326966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd37809bfe0 0x7fd378135020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 >> 172.21.15.134:0/2185326966 conn(0x7fd3780a9ea0 msgr2=0x7fd37809d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:43.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 shutdown_connections 2024-09-16T15:19:43.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.747+0000 7fd37fb7d640 1 -- 172.21.15.134:0/2185326966 wait complete. 2024-09-16T15:19:43.759 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1424,"stamp":"2024-09-16T15:19:43.074538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346543931,"num_objects":159,"num_object_clones":107,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":5,"num_read":7985,"num_read_kb":7831100,"num_write":5673,"num_write_kb":2181713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18173,"ondisk_log_size":18173,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":784944,"kb_used_data":466372,"kb_used_omap":9,"kb_used_meta":317814,"kb_avail":565446096,"statfs":{"total":579820584960,"available":579016802304,"internally_reserved":0,"allocated":477564928,"data_stored":582910251,"data_compressed":13024798,"data_compressed_allocated":9625600,"data_compressed_original":29526088,"omap_allocated":9562,"internal_metadata":325442214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":790372,"num_objects":-4,"num_object_clones":-4,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":-3,"num_read":59,"num_read_kb":72106,"num_write":23,"num_write_kb":5765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.388680"},"pg_stats":[{"pgid":"3.f","version":"856'745","reported_seq":3409,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.450081+0000","last_change":"2024-09-16T15:19:40.450355+0000","last_active":"2024-09-16T15:19:40.450081+0000","last_peered":"2024-09-16T15:19:40.450081+0000","last_clean":"2024-09-16T15:19:40.450081+0000","last_became_active":"2024-09-16T15:00:57.697659+0000","last_became_peered":"2024-09-16T15:00:57.697659+0000","last_unstale":"2024-09-16T15:19:40.450081+0000","last_undegraded":"2024-09-16T15:19:40.450081+0000","last_fullsized":"2024-09-16T15:19:40.450081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:33.648611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002740391,"stat_sum":{"num_bytes":18678929,"num_objects":7,"num_object_clones":5,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":341,"num_read_kb":357296,"num_write":214,"num_write_kb":89380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.e","version":"854'1605","reported_seq":4280,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.450688+0000","last_change":"2024-09-16T15:19:40.450865+0000","last_active":"2024-09-16T15:19:40.450688+0000","last_peered":"2024-09-16T15:19:40.450688+0000","last_clean":"2024-09-16T15:19:40.450688+0000","last_became_active":"2024-09-16T15:00:57.697265+0000","last_became_peered":"2024-09-16T15:00:57.697265+0000","last_unstale":"2024-09-16T15:19:40.450688+0000","last_undegraded":"2024-09-16T15:19:40.450688+0000","last_fullsized":"2024-09-16T15:19:40.450688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":1605,"log_dups_size":0,"ondisk_log_size":1605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:50.341202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029921029999999999,"stat_sum":{"num_bytes":21831141,"num_objects":11,"num_object_clones":6,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":761,"num_read_kb":777478,"num_write":502,"num_write_kb":202195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.d","version":"836'332","reported_seq":2989,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.694351+0000","last_change":"2024-09-16T15:19:40.694638+0000","last_active":"2024-09-16T15:19:40.694351+0000","last_peered":"2024-09-16T15:19:40.694351+0000","last_clean":"2024-09-16T15:19:40.694351+0000","last_became_active":"2024-09-16T15:00:57.694245+0000","last_became_peered":"2024-09-16T15:00:57.694245+0000","last_unstale":"2024-09-16T15:19:40.694351+0000","last_undegraded":"2024-09-16T15:19:40.694351+0000","last_fullsized":"2024-09-16T15:19:40.694351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:26:31.012316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053767400000000003,"stat_sum":{"num_bytes":5252852,"num_objects":4,"num_object_clones":3,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":136,"num_read_kb":133586,"num_write":102,"num_write_kb":34636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.c","version":"856'2591","reported_seq":5181,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.450954+0000","last_change":"2024-09-16T15:19:40.451204+0000","last_active":"2024-09-16T15:19:40.450954+0000","last_peered":"2024-09-16T15:19:40.450954+0000","last_clean":"2024-09-16T15:19:40.450954+0000","last_became_active":"2024-09-16T15:00:57.697164+0000","last_became_peered":"2024-09-16T15:00:57.697164+0000","last_unstale":"2024-09-16T15:19:40.450954+0000","last_undegraded":"2024-09-16T15:19:40.450954+0000","last_fullsized":"2024-09-16T15:19:40.450954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":2591,"log_dups_size":0,"ondisk_log_size":2591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:22.677312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036327830000000001,"stat_sum":{"num_bytes":38074277,"num_objects":16,"num_object_clones":10,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1059,"num_read_kb":1028375,"num_write":659,"num_write_kb":285836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.b","version":"857'2305","reported_seq":4923,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.712317+0000","last_change":"2024-09-16T15:19:40.712492+0000","last_active":"2024-09-16T15:19:40.712317+0000","last_peered":"2024-09-16T15:19:40.712317+0000","last_clean":"2024-09-16T15:19:40.712317+0000","last_became_active":"2024-09-16T15:00:57.690892+0000","last_became_peered":"2024-09-16T15:00:57.690892+0000","last_unstale":"2024-09-16T15:19:40.712317+0000","last_undegraded":"2024-09-16T15:19:40.712317+0000","last_fullsized":"2024-09-16T15:19:40.712317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:20.568717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011709923000000001,"stat_sum":{"num_bytes":51035044,"num_objects":23,"num_object_clones":17,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":942,"num_read_kb":883865,"num_write":729,"num_write_kb":251761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.a","version":"857'745","reported_seq":3430,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.697557+0000","last_change":"2024-09-16T15:19:40.697794+0000","last_active":"2024-09-16T15:19:40.697557+0000","last_peered":"2024-09-16T15:19:40.697557+0000","last_clean":"2024-09-16T15:19:40.697557+0000","last_became_active":"2024-09-16T15:00:57.690642+0000","last_became_peered":"2024-09-16T15:00:57.690642+0000","last_unstale":"2024-09-16T15:19:40.697557+0000","last_undegraded":"2024-09-16T15:19:40.697557+0000","last_fullsized":"2024-09-16T15:19:40.697557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:06.290380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034877290000000002,"stat_sum":{"num_bytes":9739012,"num_objects":5,"num_object_clones":3,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":388,"num_read_kb":366168,"num_write":216,"num_write_kb":93058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.9","version":"0'0","reported_seq":2608,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:39.687477+0000","last_change":"2024-09-16T15:19:39.687724+0000","last_active":"2024-09-16T15:19:39.687477+0000","last_peered":"2024-09-16T15:19:39.687477+0000","last_clean":"2024-09-16T15:19:39.687477+0000","last_became_active":"2024-09-16T15:00:57.694316+0000","last_became_peered":"2024-09-16T15:00:57.694316+0000","last_unstale":"2024-09-16T15:19:39.687477+0000","last_undegraded":"2024-09-16T15:19:39.687477+0000","last_fullsized":"2024-09-16T15:19:39.687477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:24.303289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337509,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.8","version":"857'1226","reported_seq":3920,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.710999+0000","last_change":"2024-09-16T15:19:40.711208+0000","last_active":"2024-09-16T15:19:40.710999+0000","last_peered":"2024-09-16T15:19:40.710999+0000","last_clean":"2024-09-16T15:19:40.710999+0000","last_became_active":"2024-09-16T15:00:57.695297+0000","last_became_peered":"2024-09-16T15:00:57.695297+0000","last_unstale":"2024-09-16T15:19:40.710999+0000","last_undegraded":"2024-09-16T15:19:40.710999+0000","last_fullsized":"2024-09-16T15:19:40.710999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:10.572589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078099479999999997,"stat_sum":{"num_bytes":39688892,"num_objects":17,"num_object_clones":13,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":622,"num_read_kb":645657,"num_write":390,"num_write_kb":154243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"1.0","version":"16'117","reported_seq":1003,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.447755+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:19:40.447755+0000","last_peered":"2024-09-16T15:19:40.447755+0000","last_clean":"2024-09-16T15:19:40.447755+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:19:40.447755+0000","last_undegraded":"2024-09-16T15:19:40.447755+0000","last_fullsized":"2024-09-16T15:19:40.447755+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":859,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-09-16T15:19:38.554924+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:19:38.554924+0000","last_peered":"2024-09-16T15:19:38.554924+0000","last_clean":"2024-09-16T15:19:38.554924+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:19:38.554924+0000","last_undegraded":"2024-09-16T15:19:38.554924+0000","last_fullsized":"2024-09-16T15:19:38.554924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"857'376","reported_seq":3041,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.704011+0000","last_change":"2024-09-16T15:19:40.704265+0000","last_active":"2024-09-16T15:19:40.704011+0000","last_peered":"2024-09-16T15:19:40.704011+0000","last_clean":"2024-09-16T15:19:40.704011+0000","last_became_active":"2024-09-16T15:00:57.692706+0000","last_became_peered":"2024-09-16T15:00:57.692706+0000","last_unstale":"2024-09-16T15:19:40.704011+0000","last_undegraded":"2024-09-16T15:19:40.704011+0000","last_fullsized":"2024-09-16T15:19:40.704011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:59.565000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034848769999999999,"stat_sum":{"num_bytes":6460673,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":194798,"num_write":117,"num_write_kb":32468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":857,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.447671+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:19:40.447671+0000","last_peered":"2024-09-16T15:19:40.447671+0000","last_clean":"2024-09-16T15:19:40.447671+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:19:40.447671+0000","last_undegraded":"2024-09-16T15:19:40.447671+0000","last_fullsized":"2024-09-16T15:19:40.447671+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"852'395","reported_seq":3027,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-09-16T15:19:37.054258+0000","last_change":"2024-09-16T15:19:37.054464+0000","last_active":"2024-09-16T15:19:37.054258+0000","last_peered":"2024-09-16T15:19:37.054258+0000","last_clean":"2024-09-16T15:19:37.054258+0000","last_became_active":"2024-09-16T15:00:57.690494+0000","last_became_peered":"2024-09-16T15:00:57.690494+0000","last_unstale":"2024-09-16T15:19:37.054258+0000","last_undegraded":"2024-09-16T15:19:37.054258+0000","last_fullsized":"2024-09-16T15:19:37.054258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:11.805387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093466199999999995,"stat_sum":{"num_bytes":6992636,"num_objects":3,"num_object_clones":2,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":114065,"num_write":131,"num_write_kb":53339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"331","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":855,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.447833+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:19:40.447833+0000","last_peered":"2024-09-16T15:19:40.447833+0000","last_clean":"2024-09-16T15:19:40.447833+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:19:40.447833+0000","last_undegraded":"2024-09-16T15:19:40.447833+0000","last_fullsized":"2024-09-16T15:19:40.447833+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"857'983","reported_seq":3662,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.709066+0000","last_change":"2024-09-16T15:19:40.709255+0000","last_active":"2024-09-16T15:19:40.709066+0000","last_peered":"2024-09-16T15:19:40.709066+0000","last_clean":"2024-09-16T15:19:40.709066+0000","last_became_active":"2024-09-16T15:00:57.694164+0000","last_became_peered":"2024-09-16T15:00:57.694164+0000","last_unstale":"2024-09-16T15:19:40.709066+0000","last_undegraded":"2024-09-16T15:19:40.709066+0000","last_fullsized":"2024-09-16T15:19:40.709066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:25.627469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082964049999999998,"stat_sum":{"num_bytes":22012298,"num_objects":10,"num_object_clones":7,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":514,"num_read_kb":550714,"num_write":286,"num_write_kb":105559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"2.2","version":"17'2","reported_seq":860,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-09-16T15:19:38.554907+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:19:38.554907+0000","last_peered":"2024-09-16T15:19:38.554907+0000","last_clean":"2024-09-16T15:19:38.554907+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:19:38.554907+0000","last_undegraded":"2024-09-16T15:19:38.554907+0000","last_fullsized":"2024-09-16T15:19:38.554907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"854'1751","reported_seq":4349,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:39.687514+0000","last_change":"2024-09-16T15:19:39.687716+0000","last_active":"2024-09-16T15:19:39.687514+0000","last_peered":"2024-09-16T15:19:39.687514+0000","last_clean":"2024-09-16T15:19:39.687514+0000","last_became_active":"2024-09-16T15:00:57.691235+0000","last_became_peered":"2024-09-16T15:00:57.691235+0000","last_unstale":"2024-09-16T15:19:39.687514+0000","last_undegraded":"2024-09-16T15:19:39.687514+0000","last_fullsized":"2024-09-16T15:19:39.687514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":1751,"log_dups_size":0,"ondisk_log_size":1751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:08.224322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029533800000000002,"stat_sum":{"num_bytes":30271964,"num_objects":14,"num_object_clones":10,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":606,"num_read_kb":549014,"num_write":601,"num_write_kb":254960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"3.5","version":"855'2340","reported_seq":4956,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.450422+0000","last_change":"2024-09-16T15:19:40.450623+0000","last_active":"2024-09-16T15:19:40.450422+0000","last_peered":"2024-09-16T15:19:40.450422+0000","last_clean":"2024-09-16T15:19:40.450422+0000","last_became_active":"2024-09-16T15:00:57.694253+0000","last_became_peered":"2024-09-16T15:00:57.694253+0000","last_unstale":"2024-09-16T15:19:40.450422+0000","last_undegraded":"2024-09-16T15:19:40.450422+0000","last_fullsized":"2024-09-16T15:19:40.450422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":2340,"log_dups_size":0,"ondisk_log_size":2340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:10.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029589400000000002,"stat_sum":{"num_bytes":46528029,"num_objects":19,"num_object_clones":13,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":1017,"num_read_kb":1072624,"num_write":699,"num_write_kb":281595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":850,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:39.687358+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:19:39.687358+0000","last_peered":"2024-09-16T15:19:39.687358+0000","last_clean":"2024-09-16T15:19:39.687358+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:19:39.687358+0000","last_undegraded":"2024-09-16T15:19:39.687358+0000","last_fullsized":"2024-09-16T15:19:39.687358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"857'1200","reported_seq":3845,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.706666+0000","last_change":"2024-09-16T15:19:40.706966+0000","last_active":"2024-09-16T15:19:40.706666+0000","last_peered":"2024-09-16T15:19:40.706666+0000","last_clean":"2024-09-16T15:19:40.706666+0000","last_became_active":"2024-09-16T15:00:57.694179+0000","last_became_peered":"2024-09-16T15:00:57.694179+0000","last_unstale":"2024-09-16T15:19:40.706666+0000","last_undegraded":"2024-09-16T15:19:40.706666+0000","last_fullsized":"2024-09-16T15:19:40.706666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":1200,"log_dups_size":0,"ondisk_log_size":1200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:17.389162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062655580000000001,"stat_sum":{"num_bytes":17428443,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":534,"num_read_kb":516899,"num_write":392,"num_write_kb":162068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":857,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.447785+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:19:40.447785+0000","last_peered":"2024-09-16T15:19:40.447785+0000","last_clean":"2024-09-16T15:19:40.447785+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:19:40.447785+0000","last_undegraded":"2024-09-16T15:19:40.447785+0000","last_fullsized":"2024-09-16T15:19:40.447785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"857'995","reported_seq":3663,"reported_epoch":857,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.705803+0000","last_change":"2024-09-16T15:19:40.706478+0000","last_active":"2024-09-16T15:19:40.705803+0000","last_peered":"2024-09-16T15:19:40.705803+0000","last_clean":"2024-09-16T15:19:40.705803+0000","last_became_active":"2024-09-16T15:00:57.695262+0000","last_became_peered":"2024-09-16T15:00:57.695262+0000","last_unstale":"2024-09-16T15:19:40.705803+0000","last_undegraded":"2024-09-16T15:19:40.705803+0000","last_fullsized":"2024-09-16T15:19:40.705803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":995,"log_dups_size":0,"ondisk_log_size":995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:23.560802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031236480000000001,"stat_sum":{"num_bytes":23014868,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":429,"num_read_kb":412628,"num_write":338,"num_write_kb":132545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"31e","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"32c","length":"1"},{"start":"330","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":849,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:39.686861+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:19:39.686861+0000","last_peered":"2024-09-16T15:19:39.686861+0000","last_clean":"2024-09-16T15:19:39.686861+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:19:39.686861+0000","last_undegraded":"2024-09-16T15:19:39.686861+0000","last_fullsized":"2024-09-16T15:19:39.686861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"854'464","reported_seq":3120,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-09-16T15:19:37.085130+0000","last_change":"2024-09-16T15:19:37.085307+0000","last_active":"2024-09-16T15:19:37.085130+0000","last_peered":"2024-09-16T15:19:37.085130+0000","last_clean":"2024-09-16T15:19:37.085130+0000","last_became_active":"2024-09-16T15:00:57.691016+0000","last_became_peered":"2024-09-16T15:00:57.691016+0000","last_unstale":"2024-09-16T15:19:37.085130+0000","last_undegraded":"2024-09-16T15:19:37.085130+0000","last_fullsized":"2024-09-16T15:19:37.085130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:56.516673+0000","last_clean_scrub_stamp":"2024-09-16T15:00:56.516673+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:30.046612+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.031769451999999997,"stat_sum":{"num_bytes":8944486,"num_objects":5,"num_object_clones":4,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":215,"num_read_kb":227833,"num_write":136,"num_write_kb":45652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"315","length":"1"},{"start":"317","length":"1"},{"start":"321","length":"1"},{"start":"329","length":"1"},{"start":"331","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":856,"reported_epoch":856,"state":"active+clean","last_fresh":"2024-09-16T15:19:40.447985+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:19:40.447985+0000","last_peered":"2024-09-16T15:19:40.447985+0000","last_clean":"2024-09-16T15:19:40.447985+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:19:40.447985+0000","last_undegraded":"2024-09-16T15:19:40.447985+0000","last_fullsized":"2024-09-16T15:19:40.447985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":345953544,"num_objects":155,"num_object_clones":107,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":5,"num_read":7869,"num_read_kb":7831000,"num_write":5512,"num_write_kb":2179295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":435363840,"data_stored":556881236,"data_compressed":13002263,"data_compressed_allocated":9560064,"data_compressed_original":28478233,"omap_allocated":0,"internal_metadata":0},"log_size":18053,"ondisk_log_size":18053,"up":32,"acting":32,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575079,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140656,"kb_used_data":83296,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94231184,"statfs":{"total":96636764160,"available":96492732416,"internally_reserved":0,"allocated":85295104,"data_stored":105498609,"data_compressed":2852764,"data_compressed_allocated":2084864,"data_compressed_original":6172560,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46000000000000002,"15min":0.46700000000000003},"min":{"1min":0.38400000000000001,"5min":0.26200000000000001,"15min":0.19400000000000001},"max":{"1min":0.61899999999999999,"5min":0.69899999999999995,"15min":0.72799999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.28499999999999998,"15min":0.24199999999999999},"max":{"1min":0.59799999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.35999999999999999}]},{"osd":1,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.49399999999999999},"min":{"1min":0.40000000000000002,"5min":0.27000000000000002,"15min":0.23499999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":1.0860000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.47199999999999998},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.25700000000000001},"max":{"1min":0.63400000000000001,"5min":0.65800000000000003,"15min":0.94999999999999996},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Sep 16 15:19:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.99299999999999999},"last":0.373},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.47699999999999998},"min":{"1min":0.33600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":1.046},"last":0.441}]},{"osd":3,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.34699999999999998},"min":{"1min":0.29099999999999998,"5min":0.23100000000000001,"15min":0.19900000000000001},"max":{"1min":0.47799999999999998,"5min":0.56799999999999995,"15min":0.61899999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.27000000000000002,"5min":0.222,"15min":0.21199999999999999},"max":{"1min":0.58599999999999997,"5min":0.628,"15min":0.63900000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"min":{"1min":0.25900000000000001,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.50900000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.54500000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575082,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64904,"kb_used_data":25596,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94306936,"statfs":{"total":96636764160,"available":96570302464,"internally_reserved":0,"allocated":26210304,"data_stored":36834613,"data_compressed":589471,"data_compressed_allocated":417792,"data_compressed_original":2156429,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.39600000000000002,"5min":0.28000000000000003,"15min":0.22900000000000001},"max":{"1min":0.61599999999999999,"5min":0.81000000000000005,"15min":1.151},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51700000000000002,"15min":0.503},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.57699999999999996,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53500000000000003}]},{"osd":1,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56299999999999994,"5min":0.68400000000000005,"15min":1.2669999999999999},"last":0.504},{"interface":"front","average":{"1min":0.438,"5min":0.49399999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.318,"15min":0.24399999999999999},"max":{"1min":0.59499999999999997,"5min":0.753,"15min":0.88300000000000001},"last":0.437}]},{"osd":2,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.59799999999999998,"5min":0.69599999999999995,"15min":1.391},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.41999999999999998}]},{"osd":4,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34000000000000002,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.193,"15min":0.193},"max":{"1min":0.53100000000000003,"5min":0.65500000000000003,"15min":1.6100000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.38,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53300000000000003,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.29899999999999999}]},{"osd":5,"last update":"Mon Sep 16 15:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.37,"15min":0.38},"min":{"1min":0.23999999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54800000000000004,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.254},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.438},"min":{"1min":0.313,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.58299999999999996,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.32700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575079,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":158784,"kb_used_data":100760,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94213056,"statfs":{"total":96636764160,"available":96474169344,"internally_reserved":0,"allocated":103178240,"data_stored":128209260,"data_compressed":3201145,"data_compressed_allocated":2179072,"data_compressed_original":6548495,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34899999999999998,"15min":0.32900000000000001},"min":{"1min":0.22,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45400000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.378,"15min":0.35399999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.18099999999999999},"max":{"1min":0.46600000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.438}]},{"osd":1,"last update":"Mon Sep 16 15:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.34499999999999997,"15min":0.36099999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.438,"5min":0.59899999999999998,"15min":2.2669999999999999},"last":0.375},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.34599999999999997,"15min":0.34100000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42999999999999999,"5min":0.58899999999999997,"15min":0.72399999999999998},"last":0.34499999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.65500000000000003,"5min":0.68899999999999995,"15min":0.80300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.499,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.751,"15min":0.80000000000000004},"last":0.52800000000000002}]},{"osd":4,"last update":"Mon Sep 16 15:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68700000000000006,"5min":0.76500000000000001,"15min":0.83999999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.28699999999999998},"max":{"1min":0.65100000000000002,"5min":0.68000000000000005,"15min":0.79900000000000004},"last":0.42199999999999999}]},{"osd":5,"last update":"Mon Sep 16 15:19:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.33800000000000002,"5min":0.32700000000000001,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.30299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.81399999999999995},"last":0.53800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171132,"kb_used_data":113776,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94200708,"statfs":{"total":96636764160,"available":96461524992,"internally_reserved":0,"allocated":116506624,"data_stored":140964838,"data_compressed":2705600,"data_compressed_allocated":2129920,"data_compressed_original":6138015,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.32500000000000001,"15min":0.33000000000000002},"min":{"1min":0.187,"5min":0.184,"15min":0.156},"max":{"1min":0.48999999999999999,"5min":0.49099999999999999,"15min":0.88900000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.26700000000000002,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.53800000000000003,"5min":0.54100000000000004,"15min":1.262},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Sep 16 15:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.35399999999999998},"min":{"1min":0.20799999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.59399999999999997,"5min":0.879,"15min":4.6829999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.39600000000000002},"min":{"1min":0.20899999999999999,"5min":0.19800000000000001,"15min":0.17000000000000001},"max":{"1min":0.59699999999999998,"5min":0.90700000000000003,"15min":12.194000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Sep 16 15:19:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.27600000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":1.024},"last":0.377},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.5,"15min":0.50700000000000001},"min":{"1min":0.317,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.75600000000000001,"5min":0.79600000000000004,"15min":1.129},"last":0.626}]},{"osd":4,"last update":"Mon Sep 16 15:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.28599999999999998},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":4.6020000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.316},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Sep 16 15:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.78700000000000003,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.36899999999999999,"5min":0.33200000000000002,"15min":0.29399999999999998},"max":{"1min":0.83999999999999997,"5min":1.198,"15min":1.2050000000000001},"last":0.69299999999999995}]}]},{"osd":4,"up_from":11,"seq":47244640485,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98740,"kb_used_data":49576,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94273100,"statfs":{"total":96636764160,"available":96535654400,"internally_reserved":0,"allocated":50765824,"data_stored":58869458,"data_compressed":1050775,"data_compressed_allocated":774144,"data_compressed_original":2272593,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.745,"5min":0.745,"15min":1.0900000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46200000000000002,"15min":0.46700000000000003},"min":{"1min":0.245,"5min":0.223,"15min":0.223},"max":{"1min":0.60999999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 15:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59599999999999997,"5min":0.629,"15min":0.73499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48699999999999999,"15min":0.5},"min":{"1min":0.27400000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.998},"last":0.495}]},{"osd":2,"last update":"Mon Sep 16 15:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.77800000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Sep 16 15:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38300000000000001,"15min":0.376},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.53100000000000003,"5min":0.65900000000000003,"15min":0.67000000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.29699999999999999,"5min":0.25600000000000001,"15min":0.215},"max":{"1min":0.67000000000000004,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.40799999999999997}]},{"osd":5,"last update":"Mon Sep 16 15:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.37,"15min":0.371},"min":{"1min":0.25800000000000001,"5min":0.22,"15min":0.20399999999999999},"max":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.26300000000000001,"5min":0.23699999999999999,"15min":0.22900000000000001},"max":{"1min":0.60099999999999998,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.373}]}]},{"osd":0,"up_from":11,"seq":47244640486,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150728,"kb_used_data":93368,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94221112,"statfs":{"total":96636764160,"available":96482418688,"internally_reserved":0,"allocated":95608832,"data_stored":112533473,"data_compressed":2625043,"data_compressed_allocated":2039808,"data_compressed_original":6237996,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.32300000000000001,"15min":0.31900000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.17899999999999999},"max":{"1min":0.47699999999999998,"5min":0.57299999999999995,"15min":0.59799999999999998},"last":0.16900000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.23799999999999999,"5min":0.217,"15min":0.182},"max":{"1min":0.502,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.20799999999999999}]},{"osd":2,"last update":"Mon Sep 16 15:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.251,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.65500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.246},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36599999999999999,"15min":0.38500000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.63800000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.25600000000000001}]},{"osd":3,"last update":"Mon Sep 16 15:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.29699999999999999,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.70699999999999996,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57999999999999996,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":1.387,"15min":1.387},"last":0.37}]},{"osd":4,"last update":"Mon Sep 16 15:19:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54000000000000004,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.30599999999999999,"15min":0.28399999999999997},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.29399999999999998}]},{"osd":5,"last update":"Mon Sep 16 15:19:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.505,"15min":0.498},"min":{"1min":0.316,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":0.65200000000000002,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.30199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":88551424,"data_stored":108180250,"data_compressed":2619651,"data_compressed_allocated":2007040,"data_compressed_original":5725996,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109486080,"data_stored":137132554,"data_compressed":2705600,"data_compressed_allocated":2129920,"data_compressed_original":6138015,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96157696,"data_stored":124375831,"data_compressed":3201145,"data_compressed_allocated":2179072,"data_compressed_original":6548495,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19144704,"data_stored":30486836,"data_compressed":572328,"data_compressed_allocated":385024,"data_compressed_original":1620574,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43753472,"data_stored":55040604,"data_compressed":1050775,"data_compressed_allocated":774144,"data_compressed_original":2272593,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":78270464,"data_stored":101665161,"data_compressed":2852764,"data_compressed_allocated":2084864,"data_compressed_original":6172560,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:19:43.762 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-16T15:19:43.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-16T15:19:43.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.860+0000 7f4ef146b640 10 monclient: get_monmap_and_config 2024-09-16T15:19:43.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.860+0000 7f4ef146b640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.862 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.860+0000 7f4ef146b640 10 monclient: monmap: 2024-09-16T15:19:43.862 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:19:43.862380+0000 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:19:43.862380+0000 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.863 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:19:43.864 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:19:43.864 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 Processor -- start 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 -- start start 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient: init 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient: _add_conns ranks=[2,10,3,1,7,4,5,6,8,0,9] 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-b con 0x7f4eec0a4220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4eec09c720 0x7f4eec0a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-f con 0x7f4eec09c720 addr v2:172.21.15.162:3300/0 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4eec09b1a0 0x7f4eec136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-g con 0x7f4eec09b1a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec09c390 con 0x7f4eec09c720 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0a5080 con 0x7f4eec0a4220 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0a5e20 con 0x7f4eec09b1a0 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4ef146b640 10 monclient(hunting): authenticate will time out at 1920.451652s 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47638/0 (socket says 172.21.15.134:47638) 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 1 -- 172.21.15.134:0/936898470 learned_addr learned my addr 172.21.15.134:0/936898470 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): get_auth_request con 0x7f4eec0a4220 auth_method 0 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eea7fc640 1 --2- 172.21.15.134:0/936898470 >> v2:172.21.15.162:3300/0 conn(0x7f4eec09c720 0x7f4eec0a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eeb7fe640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4eec09b1a0 0x7f4eec136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eea7fc640 10 monclient(hunting): get_auth_request con 0x7f4eec09c720 auth_method 0 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.861+0000 7f4eea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeb7fe640 10 monclient(hunting): get_auth_request con 0x7f4eec09b1a0 auth_method 0 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_done global_id 114635 payload 274 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 -- 172.21.15.134:0/936898470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4eec09b1a0 msgr2=0x7f4eec136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4eec09b1a0 0x7f4eec136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 -- 172.21.15.134:0/936898470 >> v2:172.21.15.162:3300/0 conn(0x7f4eec09c720 msgr2=0x7f4eec0a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:19:43.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/936898470 >> v2:172.21.15.162:3300/0 conn(0x7f4eec09c720 0x7f4eec0a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 monclient: found mon.noname-b 2024-09-16T15:19:43.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.862+0000 7f4eeaffd640 1 -- 172.21.15.134:0/936898470 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec04e730 con 0x7f4eec0a4220 2024-09-16T15:19:43.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.864+0000 7f4eeaffd640 1 -- 172.21.15.134:0/936898470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 msgr2=0x7f4eec0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.864+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0098a0 tx=0x7f4edc02a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12fc61a522ebfdd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 1 -- 172.21.15.134:0/936898470 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4edc02adf0 con 0x7f4eec0a4220 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: dump: 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.871 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 1 -- 172.21.15.134:0/936898470 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc0307b0 con 0x7f4eec0a4220 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 5 monclient: authenticate success, global_id 114635 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 1 -- 172.21.15.134:0/936898470 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4edc030d90 con 0x7f4eec0a4220 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 10 monclient: get_monmap_and_config success 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 4 set_mon_vals no callback set 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ee9ffb640 10 monclient: dump: 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.873 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 10 monclient: shutdown 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 -- 172.21.15.134:0/936898470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 msgr2=0x7f4eec0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0098a0 tx=0x7f4edc02a0d0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 -- 172.21.15.134:0/936898470 shutdown_connections 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4eec09b1a0 0x7f4eec136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 --2- 172.21.15.134:0/936898470 >> v2:172.21.15.162:3300/0 conn(0x7f4eec09c720 0x7f4eec0a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 --2- 172.21.15.134:0/936898470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec0a4220 0x7f4eec0a4600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 -- 172.21.15.134:0/936898470 >> 172.21.15.134:0/936898470 conn(0x7f4eec0a9820 msgr2=0x7f4eec0a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 -- 172.21.15.134:0/936898470 shutdown_connections 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.867+0000 7f4ef146b640 1 -- 172.21.15.134:0/936898470 wait complete. 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.868+0000 7f4ef146b640 10 monclient: build_initial_monmap 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.868+0000 7f4ef146b640 10 monclient: monmap: 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:19:43.875 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.868+0000 7f4ef146b640 1 Processor -- start 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.868+0000 7f4ef146b640 1 -- start start 2024-09-16T15:19:43.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.868+0000 7f4ef146b640 10 monclient: init 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient: _reopen_session rank -1 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient: _add_conns ranks=[2,8,4,5,0,7,3,1,6] 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-b con 0x7f4eec09b1a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 0x7f4eec137260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-e con 0x7f4eec0a4220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 0x7f4eec123d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): picked mon.noname-c con 0x7f4eec1377a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1260b0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec125880 con 0x7f4eec1377a0 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): start opening mon connection 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1273b0 con 0x7f4eec0a4220 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): _renew_subs 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4ef146b640 10 monclient(hunting): authenticate will time out at 1920.459319s 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 0x7f4eec137260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:47654/0 (socket says 172.21.15.134:47654) 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 -- 172.21.15.134:0/3720151329 learned_addr learned my addr 172.21.15.134:0/3720151329 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): get_auth_request con 0x7f4eec09b1a0 auth_method 0 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): get_auth_request con 0x7f4eec0a4220 auth_method 0 2024-09-16T15:19:43.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 0x7f4eec123d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): get_auth_request con 0x7f4eec1377a0 auth_method 0 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:19:43.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient(hunting): handle_auth_done global_id 114644 payload 824 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 -- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 msgr2=0x7f4eec137260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 0x7f4eec137260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 -- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 msgr2=0x7f4eec123d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 0x7f4eec123d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient: _finish_hunting 0 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 monclient: found mon.noname-b 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeaffd640 1 -- 172.21.15.134:0/3720151329 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0094d0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eea7fc640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 0x7f4eec137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.869+0000 7f4eeb7fe640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 0x7f4eec123d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.871+0000 7f4eeaffd640 1 -- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 msgr2=0x7f4eec1cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:19:43.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.871+0000 7f4eeaffd640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0097c0 tx=0x7f4edc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4edc003610 con 0x7f4eec09b1a0 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: dump: 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.884 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: _finish_auth 0 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc0037d0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 5 monclient: authenticate success, global_id 114644 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _renew_subs 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec127c50 con 0x7f4eec09b1a0 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4edc003db0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ed37fe640 10 monclient: dump: 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:19:43.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _renew_subs 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec0a1dc0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ee97fa640 4 set_mon_vals no callback set 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.872+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec0a8ea0 con 0x7f4eec09b1a0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.873+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4edc003940 con 0x7f4eec09b1a0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.873+0000 7f4ed37fe640 1 --2- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 0x7f4ed803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.873+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(860..860 src has 1..860) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f4edc06a800 con 0x7f4eec09b1a0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.873+0000 7f4eea7fc640 1 --2- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 0x7f4ed803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.873+0000 7f4eea7fc640 10 monclient: get_auth_request con 0x7f4ed8038880 auth_method 0 2024-09-16T15:19:43.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.876+0000 7f4eea7fc640 1 --2- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 0x7f4ed803ad40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006fd0 tx=0x7f4ee0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:19:43.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.877+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4edc030b30 con 0x7f4eec09b1a0 2024-09-16T15:19:43.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.877+0000 7f4ed37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:19:43.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:43.877+0000 7f4ed37fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:19:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.039+0000 7f4ef146b640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-16T15:19:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.039+0000 7f4ef146b640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-16T15:19:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.039+0000 7f4ef146b640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:19:44.041 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.039+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4eec09ae30 con 0x7f4eec09b1a0 2024-09-16T15:19:44.716 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.714+0000 7f4ed37fe640 1 -- 172.21.15.134:0/3720151329 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]=0 pool 'unique_pool_0' removed v861) v1 ==== 175+0+0 (secure 0 0 0) 0x7f4edc0370d0 con 0x7f4eec09b1a0 2024-09-16T15:19:44.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.714+0000 7f4ed37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-16T15:19:44.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.714+0000 7f4ed37fe640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' removed 2024-09-16T15:19:44.717 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'unique_pool_0' removed 2024-09-16T15:19:44.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 msgr2=0x7f4ed803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:44.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 0x7f4ed803ad40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006fd0 tx=0x7f4ee0008040 comp rx=0 tx=0).stop 2024-09-16T15:19:44.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 10 monclient: shutdown 2024-09-16T15:19:44.720 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 msgr2=0x7f4eec1cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0097c0 tx=0x7f4edc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 shutdown_connections 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> v2:172.21.15.162:6800/980547263 conn(0x7f4ed8038880 0x7f4ed803ad40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4eec1377a0 0x7f4eec123d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4eec0a4220 0x7f4eec137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 --2- 172.21.15.134:0/3720151329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4eec09b1a0 0x7f4eec1cb2e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 >> 172.21.15.134:0/3720151329 conn(0x7f4eec0a9820 msgr2=0x7f4eec0a7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 shutdown_connections 2024-09-16T15:19:44.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:19:44.717+0000 7f4ef146b640 1 -- 172.21.15.134:0/3720151329 wait complete. 2024-09-16T15:19:44.736 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-16T15:19:44.746 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-16T15:20:40.794 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-16T15:20:40.795 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-16T15:20:40.795 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-16T15:20:40.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-16T15:20:40.799 INFO:tasks.ceph.mon.i:Started 2024-09-16T15:20:40.799 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:20:40.799 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.903+0000 7fa81cde2640 10 monclient: get_monmap_and_config 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.903+0000 7fa81cde2640 10 monclient: build_initial_monmap 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.903+0000 7fa81cde2640 10 monclient: monmap: 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:40.904883+0000 2024-09-16T15:20:40.905 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:40.904883+0000 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:40.906 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.904+0000 7fa81cde2640 1 Processor -- start 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.904+0000 7fa81cde2640 1 -- start start 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.904+0000 7fa81cde2640 10 monclient: init 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient: _add_conns ranks=[1,9,0,6,3,5,2,4,8,7,10] 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fa818136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa81809bfe0 0x7fa81809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-a con 0x7fa81809bfe0 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa81813bf80 addr v1:172.21.15.134:6789/0 2024-09-16T15:20:40.909 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa818073250 con 0x7fa81813bf80 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81809b600 con 0x7fa818136030 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8180a5740 con 0x7fa81809bfe0 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa81cde2640 10 monclient(hunting): authenticate will time out at 1977.494331s 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa81809bfe0 0x7fa81809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa816d9d640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa81813bf80 0x7fa81813e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45926/0 (socket says 172.21.15.134:45926) 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa81809bfe0 0x7fa81809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58616/0 (socket says 172.21.15.134:58616) 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa816d9d640 1 -- 172.21.15.134:0/2535195059 learned_addr learned my addr 172.21.15.134:0/2535195059 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): get_auth_request con 0x7fa81809bfe0 auth_method 0 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:40.910 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient(hunting): handle_auth_done global_id 124317 payload 274 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 -- 172.21.15.134:0/2535195059 >> v1:172.21.15.162:6789/0 conn(0x7fa818136030 legacy=0x7fa818136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 -- 172.21.15.134:0/2535195059 >> v1:172.21.15.134:6789/0 conn(0x7fa81813bf80 legacy=0x7fa81813e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 monclient: found mon.noname-a 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:40.911 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.905+0000 7fa815d9b640 1 -- 172.21.15.134:0/2535195059 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8180a52c0 con 0x7fa81809bfe0 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa815d9b640 1 -- 172.21.15.134:0/2535195059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81809bfe0 msgr2=0x7fa81809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa815d9b640 1 --2- 172.21.15.134:0/2535195059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81809bfe0 0x7fa81809c400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0097a0 tx=0x7fa80c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdca90bc24a60ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 1 -- 172.21.15.134:0/2535195059 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa80c02ae60 con 0x7fa81809bfe0 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: dump: 2024-09-16T15:20:40.917 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:40.918 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: _finish_auth 0 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: _check_auth_tickets 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 1 -- 172.21.15.134:0/2535195059 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c0307b0 con 0x7fa81809bfe0 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 1 -- 172.21.15.134:0/2535195059 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa80c030d90 con 0x7fa81809bfe0 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81cde2640 5 monclient: authenticate success, global_id 124317 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81cde2640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:40.919 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81cde2640 4 set_mon_vals no callback set 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.908+0000 7fa81559a640 10 monclient: dump: 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:40.920 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 10 monclient: shutdown 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:40.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 -- 172.21.15.134:0/2535195059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81809bfe0 msgr2=0x7fa81809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 --2- 172.21.15.134:0/2535195059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81809bfe0 0x7fa81809c400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0097a0 tx=0x7fa80c02a060 comp rx=0 tx=0).stop 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 -- 172.21.15.134:0/2535195059 shutdown_connections 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 --2- 172.21.15.134:0/2535195059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa81809bfe0 0x7fa81809c400 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 -- 172.21.15.134:0/2535195059 >> 172.21.15.134:0/2535195059 conn(0x7fa8180a9ea0 msgr2=0x7fa8180a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 -- 172.21.15.134:0/2535195059 shutdown_connections 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.909+0000 7fa81cde2640 1 -- 172.21.15.134:0/2535195059 wait complete. 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.910+0000 7fa81cde2640 10 monclient: build_initial_monmap 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.910+0000 7fa81cde2640 10 monclient: monmap: 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:40.922 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.910+0000 7fa81cde2640 1 Processor -- start 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.910+0000 7fa81cde2640 1 -- start start 2024-09-16T15:20:40.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.910+0000 7fa81cde2640 10 monclient: init 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient: _add_conns ranks=[6,1,8,0,4,3,5,7,2] 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-d con 0x7fa81809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa818136030 0x7fa8181341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-f con 0x7fa818136030 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa81813bf80 0x7fa818138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): picked mon.noname-e con 0x7fa81813bf80 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81811ff60 con 0x7fa818136030 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fa81809fd40 con 0x7fa81809bfe0 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:40.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fa81813b250 con 0x7fa81813bf80 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81cde2640 10 monclient(hunting): authenticate will time out at 1977.500156s 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa81813bf80 0x7fa818138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:43484/0 (socket says 172.21.15.134:43484) 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 -- 172.21.15.134:0/3875084348 learned_addr learned my addr 172.21.15.134:0/3875084348 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): get_auth_request con 0x7fa81809bfe0 auth_method 0 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:40.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): get_auth_request con 0x7fa81813bf80 auth_method 0 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa818136030 0x7fa8181341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): get_auth_request con 0x7fa818136030 auth_method 0 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:40.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa816d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa815d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient(hunting): handle_auth_done global_id 94785 payload 824 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 -- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa81813bf80 msgr2=0x7fa818138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa81813bf80 0x7fa818138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 -- 172.21.15.134:0/3875084348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa818136030 msgr2=0x7fa8181341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa818136030 0x7fa8181341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient: _finish_hunting 0 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 monclient: found mon.noname-d 2024-09-16T15:20:40.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.911+0000 7fa81659c640 1 -- 172.21.15.134:0/3875084348 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c009400 con 0x7fa81809bfe0 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa81659c640 1 -- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 msgr2=0x7fa818133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa81659c640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c430 tx=0x7fa80800c8f0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa808003860 con 0x7fa81809bfe0 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 10 monclient: dump: 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:40.928 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.929 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 10 monclient: _finish_auth 0 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.913+0000 7fa806ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808011c10 con 0x7fa81809bfe0 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 5 monclient: authenticate success, global_id 94785 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 10 monclient: _renew_subs 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180a0290 con 0x7fa81809bfe0 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa808011dc0 con 0x7fa81809bfe0 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa806ffd640 10 monclient: dump: 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:40.930 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa814d99640 4 set_mon_vals no callback set 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 10 monclient: _renew_subs 2024-09-16T15:20:40.931 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.914+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81813a850 con 0x7fa81809bfe0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fa8080117a0 con 0x7fa81809bfe0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa81cde2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa81cde2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa81cde2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa818120190 con 0x7fa81809bfe0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa806ffd640 1 --2- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 0x7fa7f803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fa808049dc0 con 0x7fa81809bfe0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa815d9b640 1 --2- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 0x7fa7f803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:40.932 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.915+0000 7fa815d9b640 10 monclient: get_auth_request con 0x7fa7f8038880 auth_method 0 2024-09-16T15:20:40.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.918+0000 7fa815d9b640 1 --2- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 0x7fa7f803ad40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa81809cdf0 tx=0x7fa80c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:40.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.919+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fa808011a50 con 0x7fa81809bfe0 2024-09-16T15:20:40.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.919+0000 7fa806ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:40.933 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:40.919+0000 7fa806ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:41.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa81cde2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa81cde2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa81cde2640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa81809c400 con 0x7fa81809bfe0 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa806ffd640 1 -- 172.21.15.134:0/3875084348 <== mon.6 v2:172.21.15.134:3303/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3191 (secure 0 0 0) 0x7fa808019ba0 con 0x7fa81809bfe0 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa806ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.147+0000 7fa806ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:41.149 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":160,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":92,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:20:41.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 msgr2=0x7fa7f803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:41.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 0x7fa7f803ad40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa81809cdf0 tx=0x7fa80c02e000 comp rx=0 tx=0).stop 2024-09-16T15:20:41.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 10 monclient: shutdown 2024-09-16T15:20:41.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:41.151 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 msgr2=0x7fa818133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c430 tx=0x7fa80800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 shutdown_connections 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> v2:172.21.15.162:6800/980547263 conn(0x7fa7f8038880 0x7fa7f803ad40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fa81813bf80 0x7fa818138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa818136030 0x7fa8181341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 --2- 172.21.15.134:0/3875084348 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fa81809bfe0 0x7fa818133c70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 >> 172.21.15.134:0/3875084348 conn(0x7fa8180a9ea0 msgr2=0x7fa81809d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 shutdown_connections 2024-09-16T15:20:41.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:41.149+0000 7fa81cde2640 1 -- 172.21.15.134:0/3875084348 wait complete. 2024-09-16T15:20:44.162 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.268+0000 7fb70de0d640 10 monclient: get_monmap_and_config 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.268+0000 7fb70de0d640 10 monclient: build_initial_monmap 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.269+0000 7fb70de0d640 10 monclient: monmap: 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:44.269947+0000 2024-09-16T15:20:44.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:44.269947+0000 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:44.271 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.269+0000 7fb70de0d640 1 Processor -- start 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.269+0000 7fb70de0d640 1 -- start start 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.269+0000 7fb70de0d640 10 monclient: init 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient: _add_conns ranks=[4,9,1,6,10,2,0,8,3,7,5] 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-c con 0x7fb708136cd0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-a con 0x7fb7081370b0 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fb708135c50 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb708073250 con 0x7fb708135c50 2024-09-16T15:20:44.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70809b590 con 0x7fb7081370b0 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080a5060 con 0x7fb708136cd0 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb70de0d640 10 monclient(hunting): authenticate will time out at 1980.859451s 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58632/0 (socket says 172.21.15.134:58632) 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 1 -- 172.21.15.134:0/1278165774 learned_addr learned my addr 172.21.15.134:0/1278165774 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:44.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:56128/0 (socket says 172.21.15.134:56128) 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): get_auth_request con 0x7fb7081370b0 auth_method 0 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): get_auth_request con 0x7fb708136cd0 auth_method 0 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:44.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient(hunting): handle_auth_done global_id 124366 payload 274 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 -- 172.21.15.134:0/1278165774 >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 msgr2=0x7fb7081374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1278165774 >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 -- 172.21.15.134:0/1278165774 >> v1:172.21.15.162:6789/0 conn(0x7fb708135c50 legacy=0x7fb7081360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 monclient: found mon.noname-c 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb7077fe640 1 -- 172.21.15.134:0/1278165774 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70809b2b0 con 0x7fb708136cd0 2024-09-16T15:20:44.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.270+0000 7fb706ffd640 1 --2- 172.21.15.134:0/1278165774 >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7077fe640 1 -- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 msgr2=0x7fb70809bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009d90 tx=0x7fb6f802a4e0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=9f22d051d0ac9c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 1 -- 172.21.15.134:0/1278165774 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f8007bc0 con 0x7fb708136cd0 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: dump: 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:44.283 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:44.284 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb70de0d640 5 monclient: authenticate success, global_id 124366 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 1 -- 172.21.15.134:0/1278165774 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f802d050 con 0x7fb708136cd0 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb70de0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb70de0d640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 1 -- 172.21.15.134:0/1278165774 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6f8007d40 con 0x7fb708136cd0 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb70de0d640 4 set_mon_vals no callback set 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.273+0000 7fb7067fc640 10 monclient: dump: 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:44.285 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 10 monclient: shutdown 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 -- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 msgr2=0x7fb70809bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009d90 tx=0x7fb6f802a4e0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 -- 172.21.15.134:0/1278165774 shutdown_connections 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1278165774 >> v2:172.21.15.134:3300/0 conn(0x7fb7081370b0 0x7fb7081374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1278165774 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb708136cd0 0x7fb70809bf70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 -- 172.21.15.134:0/1278165774 >> 172.21.15.134:0/1278165774 conn(0x7fb7080a5fa0 msgr2=0x7fb7080a6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 -- 172.21.15.134:0/1278165774 shutdown_connections 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.274+0000 7fb70de0d640 1 -- 172.21.15.134:0/1278165774 wait complete. 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.275+0000 7fb70de0d640 10 monclient: build_initial_monmap 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.275+0000 7fb70de0d640 10 monclient: monmap: 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:44.287 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.275+0000 7fb70de0d640 1 Processor -- start 2024-09-16T15:20:44.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.275+0000 7fb70de0d640 1 -- start start 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.275+0000 7fb70de0d640 10 monclient: init 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient: _add_conns ranks=[0,2,4,5,3,6,8,7,1] 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-a con 0x7fb708135c50 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 0x7fb708135660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-b con 0x7fb708136cd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 0x7fb70811faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): picked mon.noname-c con 0x7fb7081370b0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb708122890 con 0x7fb708135c50 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:44.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb708122ae0 con 0x7fb708136cd0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080a8a20 con 0x7fb7081370b0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb70de0d640 10 monclient(hunting): authenticate will time out at 1980.865192s 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 0x7fb70811faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 0x7fb708135660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58648/0 (socket says 172.21.15.134:58648) 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): get_auth_request con 0x7fb708136cd0 auth_method 0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 1 -- 172.21.15.134:0/1559779015 learned_addr learned my addr 172.21.15.134:0/1559779015 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): get_auth_request con 0x7fb7081370b0 auth_method 0 2024-09-16T15:20:44.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): get_auth_request con 0x7fb708135c50 auth_method 0 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:44.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient(hunting): handle_auth_done global_id 114653 payload 824 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 -- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 msgr2=0x7fb70811faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 0x7fb70811faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 -- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 msgr2=0x7fb708135120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 monclient: found mon.noname-b 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb706ffd640 1 -- 172.21.15.134:0/1559779015 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0094d0 con 0x7fb708136cd0 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb707fff640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 0x7fb70811faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.276+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:44.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.278+0000 7fb706ffd640 1 -- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 msgr2=0x7fb708135660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.278+0000 7fb706ffd640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 0x7fb708135660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e840 tx=0x7fb6fc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.278+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6fc013470 con 0x7fb708136cd0 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: dump: 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:44.293 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: _finish_auth 0 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: _check_auth_tickets 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 5 monclient: authenticate success, global_id 114653 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc013630 con 0x7fb708136cd0 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb6fc013c10 con 0x7fb708136cd0 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 10 monclient: _renew_subs 2024-09-16T15:20:44.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f8009a70 con 0x7fb708136cd0 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb705ffb640 4 set_mon_vals no callback set 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 10 monclient: dump: 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:44.295 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 10 monclient: _renew_subs 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7081362a0 con 0x7fb708136cd0 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.279+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fb6fc0137a0 con 0x7fb708136cd0 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb70de0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb70de0d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb70de0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7080a33b0 con 0x7fb708136cd0 2024-09-16T15:20:44.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb6effff640 1 --2- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 0x7fb6f403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fb6fc04a2b0 con 0x7fb708136cd0 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 0x7fb6f403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.280+0000 7fb7077fe640 10 monclient: get_auth_request con 0x7fb6f4038880 auth_method 0 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.283+0000 7fb7077fe640 1 --2- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 0x7fb6f403ad40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a520 tx=0x7fb6f802f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.285+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fb6fc013d70 con 0x7fb708136cd0 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.285+0000 7fb6effff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:44.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.285+0000 7fb6effff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:44.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.510+0000 7fb70de0d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:20:44.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.510+0000 7fb70de0d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:44.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.510+0000 7fb70de0d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:44.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:44.510+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb708122c20 con 0x7fb708136cd0 2024-09-16T15:20:47.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:47.276+0000 7fb6eeffd640 10 monclient: tick 2024-09-16T15:20:47.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:47.276+0000 7fb6eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:47.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:47.276+0000 7fb6eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:47.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:47.276+0000 7fb6eeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:48.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:48.276+0000 7fb6eeffd640 10 monclient: tick 2024-09-16T15:20:48.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:48.276+0000 7fb6eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:48.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:48.276+0000 7fb6eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.277+0000 7fb6eeffd640 10 monclient: tick 2024-09-16T15:20:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.277+0000 7fb6eeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:49.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.277+0000 7fb6eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:49.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.315+0000 7fb6effff640 1 -- 172.21.15.134:0/1559779015 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3190 (secure 0 0 0) 0x7fb708122c20 con 0x7fb708136cd0 2024-09-16T15:20:49.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.315+0000 7fb6effff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:49.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.315+0000 7fb6effff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:49.317 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":164,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 msgr2=0x7fb6f403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 0x7fb6f403ad40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a520 tx=0x7fb6f802f000 comp rx=0 tx=0).stop 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 10 monclient: shutdown 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 msgr2=0x7fb708135660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 0x7fb708135660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e840 tx=0x7fb6fc00ed00 comp rx=0 tx=0).stop 2024-09-16T15:20:49.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 shutdown_connections 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> v2:172.21.15.162:6800/980547263 conn(0x7fb6f4038880 0x7fb6f403ad40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fb7081370b0 0x7fb70811faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb708136cd0 0x7fb708135660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 --2- 172.21.15.134:0/1559779015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb708135c50 0x7fb708135120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 >> 172.21.15.134:0/1559779015 conn(0x7fb7080a5fa0 msgr2=0x7fb7080a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 shutdown_connections 2024-09-16T15:20:49.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:49.319+0000 7fb70de0d640 1 -- 172.21.15.134:0/1559779015 wait complete. 2024-09-16T15:20:52.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:20:52.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.432+0000 7f0c55384640 10 monclient: get_monmap_and_config 2024-09-16T15:20:52.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.432+0000 7f0c55384640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.432+0000 7f0c55384640 10 monclient: monmap: 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:52.433849+0000 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:52.433849+0000 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.435 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.433+0000 7f0c55384640 1 Processor -- start 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.433+0000 7f0c55384640 1 -- start start 2024-09-16T15:20:52.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.433+0000 7f0c55384640 10 monclient: init 2024-09-16T15:20:52.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient: _add_conns ranks=[9,4,0,10,5,7,6,2,3,1,8] 2024-09-16T15:20:52.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 0x7f0c50136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-a con 0x7f0c50136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:52.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-c con 0x7f0c5009bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c50136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c50073250 con 0x7f0c50136c60 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5009b600 con 0x7f0c50136030 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c500a5740 con 0x7f0c5009bfe0 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c55384640 10 monclient(hunting): authenticate will time out at 1989.023102s 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 0x7f0c50136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 0x7f0c50136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56208/0 (socket says 172.21.15.134:56208) 2024-09-16T15:20:52.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38924/0 (socket says 172.21.15.134:38924) 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 1 -- 172.21.15.134:0/4121885409 learned_addr learned my addr 172.21.15.134:0/4121885409 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): get_auth_request con 0x7f0c50136030 auth_method 0 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): get_auth_request con 0x7f0c5009bfe0 auth_method 0 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient(hunting): handle_auth_done global_id 124384 payload 274 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 -- 172.21.15.134:0/4121885409 >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 msgr2=0x7f0c50136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 --2- 172.21.15.134:0/4121885409 >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 0x7f0c50136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 -- 172.21.15.134:0/4121885409 >> v1:172.21.15.134:6789/0 conn(0x7f0c50136c60 legacy=0x7f0c50137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 monclient: found mon.noname-c 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.434+0000 7f0c4e7fc640 1 -- 172.21.15.134:0/4121885409 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c500a52c0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4e7fc640 1 -- 172.21.15.134:0/4121885409 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 msgr2=0x7f0c5009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4e7fc640 1 --2- 172.21.15.134:0/4121885409 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c440097a0 tx=0x7f0c4402a010 comp rx=0 tx=0).ready entity=mon.4 client_cookie=64291b977d022f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 1 -- 172.21.15.134:0/4121885409 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c4402ad30 con 0x7f0c5009bfe0 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: dump: 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.446 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 1 -- 172.21.15.134:0/4121885409 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c440307b0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c55384640 5 monclient: authenticate success, global_id 124384 2024-09-16T15:20:52.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c55384640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c55384640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 1 -- 172.21.15.134:0/4121885409 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c44030d90 con 0x7f0c5009bfe0 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c55384640 4 set_mon_vals no callback set 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.437+0000 7f0c4dffb640 10 monclient: dump: 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.448 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 10 monclient: shutdown 2024-09-16T15:20:52.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 -- 172.21.15.134:0/4121885409 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 msgr2=0x7f0c5009c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 --2- 172.21.15.134:0/4121885409 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c440097a0 tx=0x7f0c4402a010 comp rx=0 tx=0).stop 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 -- 172.21.15.134:0/4121885409 shutdown_connections 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 --2- 172.21.15.134:0/4121885409 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c5009c400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 --2- 172.21.15.134:0/4121885409 >> v2:172.21.15.134:3300/0 conn(0x7f0c50136030 0x7f0c50136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 -- 172.21.15.134:0/4121885409 >> 172.21.15.134:0/4121885409 conn(0x7f0c500a9ea0 msgr2=0x7f0c500a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 -- 172.21.15.134:0/4121885409 shutdown_connections 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 1 -- 172.21.15.134:0/4121885409 wait complete. 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.438+0000 7f0c55384640 10 monclient: monmap: 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:52.450 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 Processor -- start 2024-09-16T15:20:52.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 -- start start 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient: init 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient: _add_conns ranks=[4,3,5,7,2,1,6,0,8] 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-c con 0x7f0c5009bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c50136030 0x7f0c50134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-g con 0x7f0c50136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c50136c60 0x7f0c501cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): picked mon.noname-h con 0x7f0c50136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5009fde0 con 0x7f0c50136030 2024-09-16T15:20:52.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c501bf7a0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c50122190 con 0x7f0c50136c60 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c55384640 10 monclient(hunting): authenticate will time out at 1989.028765s 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38928/0 (socket says 172.21.15.134:38928) 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 1 -- 172.21.15.134:0/920768311 learned_addr learned my addr 172.21.15.134:0/920768311 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): get_auth_request con 0x7f0c5009bfe0 auth_method 0 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.439+0000 7f0c4effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 10 monclient(hunting): handle_auth_done global_id 124393 payload 824 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4e7fc640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c50136030 0x7f0c50134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 -- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c50136c60 msgr2=0x7f0c501cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c50136c60 0x7f0c501cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 -- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c50136030 msgr2=0x7f0c50134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c50136030 0x7f0c50134fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 monclient: found mon.noname-c 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.454 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.440+0000 7f0c4effd640 1 -- 172.21.15.134:0/920768311 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009400 con 0x7f0c5009bfe0 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c4effd640 1 -- 172.21.15.134:0/920768311 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 msgr2=0x7f0c501bec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c4effd640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009d90 tx=0x7f0c4000c490 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c4000cdb0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: dump: 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.457 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.458 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4000f040 con 0x7f0c5009bfe0 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 5 monclient: authenticate success, global_id 124393 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 10 monclient: _renew_subs 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5009fa00 con 0x7f0c5009bfe0 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0c40014d60 con 0x7f0c5009bfe0 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:52.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c377fe640 10 monclient: dump: 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.460 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c4d7fa640 4 set_mon_vals no callback set 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 10 monclient: _renew_subs 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.442+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c50122750 con 0x7f0c5009bfe0 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0c40014480 con 0x7f0c5009bfe0 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c55384640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c55384640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c55384640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5009c400 con 0x7f0c5009bfe0 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c377fe640 1 --2- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 0x7f0c3c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f0c4004a320 con 0x7f0c5009bfe0 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c4e7fc640 1 --2- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 0x7f0c3c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.443+0000 7f0c4e7fc640 10 monclient: get_auth_request con 0x7f0c3c0388b0 auth_method 0 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.446+0000 7f0c4e7fc640 1 --2- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 0x7f0c3c03ad70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c5006ccb0 tx=0x7f0c4402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.447+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f0c40014b10 con 0x7f0c5009bfe0 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.447+0000 7f0c377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.447+0000 7f0c377fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:52.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.671+0000 7f0c55384640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:20:52.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.671+0000 7f0c55384640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:52.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.671+0000 7f0c55384640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.673 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.671+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0c501bfcb0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.672+0000 7f0c377fe640 1 -- 172.21.15.134:0/920768311 <== mon.4 v2:172.21.15.134:3302/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7f0c400127b0 con 0x7f0c5009bfe0 2024-09-16T15:20:52.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.672+0000 7f0c377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:52.674 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.672+0000 7f0c377fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:52.674 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":166,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 msgr2=0x7f0c3c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 0x7f0c3c03ad70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c5006ccb0 tx=0x7f0c4402e000 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 10 monclient: shutdown 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 msgr2=0x7f0c501bec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009d90 tx=0x7f0c4000c490 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 shutdown_connections 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f0c50136c60 0x7f0c501cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> v2:172.21.15.162:6800/980547263 conn(0x7f0c3c0388b0 0x7f0c3c03ad70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c50136030 0x7f0c50134fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 --2- 172.21.15.134:0/920768311 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f0c5009bfe0 0x7f0c501bec70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 >> 172.21.15.134:0/920768311 conn(0x7f0c500a9ea0 msgr2=0x7f0c5006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:52.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 shutdown_connections 2024-09-16T15:20:52.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.674+0000 7f0c55384640 1 -- 172.21.15.134:0/920768311 wait complete. 2024-09-16T15:20:52.686 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:20:52.686 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-16T15:20:52.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.782+0000 7f3899620640 10 monclient: get_monmap_and_config 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.782+0000 7f3899620640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.782+0000 7f3899620640 10 monclient: monmap: 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:52.783781+0000 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:52.783781+0000 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.784 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:52.785 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.783+0000 7f3899620640 1 Processor -- start 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.783+0000 7f3899620640 1 -- start start 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.783+0000 7f3899620640 10 monclient: init 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient: _add_conns ranks=[8,6,3,4,10,1,9,0,2,7,5] 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): picked mon.noname-e con 0x7f3894136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): picked mon.noname-d con 0x7f3894137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 0x7f3894136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): picked mon.noname-g con 0x7f3894135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389409abc0 con 0x7f3894135dc0 2024-09-16T15:20:52.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f389409ba50 con 0x7f3894137220 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f38940a53c0 con 0x7f3894136e40 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3899620640 10 monclient(hunting): authenticate will time out at 1989.373178s 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52474/0 (socket says 172.21.15.134:52474) 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51370/0 (socket says 172.21.15.134:51370) 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 -- 172.21.15.134:0/3643394884 learned_addr learned my addr 172.21.15.134:0/3643394884 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): get_auth_request con 0x7f3894136e40 auth_method 0 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): get_auth_request con 0x7f3894137220 auth_method 0 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 0x7f3894136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 10 monclient(hunting): get_auth_request con 0x7f3894135dc0 auth_method 0 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient(hunting): handle_auth_done global_id 85094 payload 274 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 -- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 msgr2=0x7f3894137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 -- 172.21.15.134:0/3643394884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 msgr2=0x7f3894136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 0x7f3894136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 monclient: found mon.noname-e 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f3892ffd640 1 -- 172.21.15.134:0/3643394884 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894073250 con 0x7f3894136e40 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f388a7fc640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.784+0000 7f38937fe640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 0x7f3894136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3892ffd640 1 -- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 msgr2=0x7f389409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3892ffd640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38840098a0 tx=0x7f388402a080 comp rx=0 tx=0).ready entity=mon.8 client_cookie=d8945db506da23f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 1 -- 172.21.15.134:0/3643394884 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f388402ade0 con 0x7f3894136e40 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: dump: 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.793 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 1 -- 172.21.15.134:0/3643394884 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38840307b0 con 0x7f3894136e40 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3899620640 5 monclient: authenticate success, global_id 85094 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3899620640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3899620640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:52.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 1 -- 172.21.15.134:0/3643394884 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3884030dd0 con 0x7f3894136e40 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f3899620640 4 set_mon_vals no callback set 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.787+0000 7f38927fc640 10 monclient: dump: 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.795 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.796 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.796 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.796 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.796 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.796 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 10 monclient: shutdown 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 -- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 msgr2=0x7f389409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38840098a0 tx=0x7f388402a080 comp rx=0 tx=0).stop 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 -- 172.21.15.134:0/3643394884 shutdown_connections 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894135dc0 0x7f3894136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f3894137220 0x7f3894137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 --2- 172.21.15.134:0/3643394884 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f3894136e40 0x7f389409c150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 -- 172.21.15.134:0/3643394884 >> 172.21.15.134:0/3643394884 conn(0x7f38940a6110 msgr2=0x7f38940a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 -- 172.21.15.134:0/3643394884 shutdown_connections 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 1 -- 172.21.15.134:0/3643394884 wait complete. 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.788+0000 7f3899620640 10 monclient: monmap: 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:52.803 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 Processor -- start 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 -- start start 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient: init 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient: _add_conns ranks=[1,5,3,0,2,7,6,4,8] 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3894135dc0 0x7f38941cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): picked mon.noname-f con 0x7f3894135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3894136e40 0x7f3894135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): picked mon.noname-h con 0x7f3894136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): picked mon.noname-g con 0x7f3894137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38941257d0 con 0x7f3894135dc0 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3894125a20 con 0x7f3894137220 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3894136850 con 0x7f3894136e40 2024-09-16T15:20:52.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3899620640 10 monclient(hunting): authenticate will time out at 1989.378762s 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f38937fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3894135dc0 0x7f38941cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f38937fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.134:60002/0 (socket says 172.21.15.134:60002) 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 learned_addr learned my addr 172.21.15.134:0/4195554522 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): get_auth_request con 0x7f3894135dc0 auth_method 0 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.789+0000 7f388a7fc640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3894136e40 0x7f3894135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): get_auth_request con 0x7f3894137220 auth_method 0 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): get_auth_request con 0x7f3894136e40 auth_method 0 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient(hunting): handle_auth_done global_id 84900 payload 824 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3894136e40 msgr2=0x7f3894135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3894136e40 0x7f3894135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3894135dc0 msgr2=0x7f38941cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3894135dc0 0x7f38941cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 monclient: found mon.noname-g 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.162:3301/0 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c009400 con 0x7f3894137220 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f388a7fc640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f3894136e40 0x7f3894135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:52.808 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.790+0000 7f3892ffd640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3894135dc0 0x7f38941cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 msgr2=0x7f3894135550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f38937fe640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f388c007f20 tx=0x7f388c00d560 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f388c0129d0 con 0x7f3894137220 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: dump: 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.809 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.3 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c00f040 con 0x7f3894137220 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3899620640 5 monclient: authenticate success, global_id 84900 2024-09-16T15:20:52.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3899620640 10 monclient: _renew_subs 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3899620640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38840094d0 con 0x7f3894137220 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.3 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f388c0143a0 con 0x7f3894137220 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f388bfff640 10 monclient: dump: 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.811 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3899620640 10 monclient: _renew_subs 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 10 monclient: _send_mon_message to mon.g at v2:172.21.15.162:3301/0 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3894136320 con 0x7f3894137220 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.792+0000 7f3891ffb640 4 set_mon_vals no callback set 2024-09-16T15:20:52.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 0x7f3894121e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940735d0 con 0x7f3894121a70 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.3 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f388c012b50 con 0x7f3894137220 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 0x7f3894121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient: get_auth_request con 0x7f3894121a70 auth_method 0 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.793+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f388bfff640 1 --2- 172.21.15.134:0/4195554522 >> v2:172.21.15.162:6800/980547263 conn(0x7f3874038880 0x7f387403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.3 v2:172.21.15.162:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f388c049f30 con 0x7f3894137220 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f3892ffd640 10 monclient(hunting): handle_auth_done global_id 124353 payload 824 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f3892ffd640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3894061ae0 con 0x7f3894121a70 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f388a7fc640 1 --2- 172.21.15.134:0/4195554522 >> v2:172.21.15.162:6800/980547263 conn(0x7f3874038880 0x7f387403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.794+0000 7f388a7fc640 10 monclient: get_auth_request con 0x7f3874038880 auth_method 0 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f3892ffd640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 msgr2=0x7f3894121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f3892ffd640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 0x7f3894121e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3884034000 tx=0x7f38840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f388402ade0 con 0x7f3894121a70 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 10 monclient: dump: 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.814 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388a7fc640 1 --2- 172.21.15.134:0/4195554522 >> v2:172.21.15.162:6800/980547263 conn(0x7f3874038880 0x7f387403ad40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f38941231d0 tx=0x7f387c009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3884003610 con 0x7f3894121a70 2024-09-16T15:20:52.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:52.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 msgr2=0x7f3894121e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 0x7f3894121e50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3884034000 tx=0x7f38840046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 msgr2=0x7f3894121e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:52.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.796+0000 7f388bfff640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894121a70 0x7f3894121e50 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3884034000 tx=0x7f38840046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3899620640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:52.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3899620640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 0x7f38940a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3899620640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894125bb0 con 0x7f3894122390 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3892ffd640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 0x7f38940a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3892ffd640 10 monclient: get_auth_request con 0x7f3894122390 auth_method 0 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3892ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3892ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.802+0000 7f3892ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.803+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.803+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.803+0000 7f3892ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.803+0000 7f3892ffd640 10 monclient(hunting): handle_auth_done global_id 124362 payload 824 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.803+0000 7f3892ffd640 1 -- 172.21.15.134:0/4195554522 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f389407a680 con 0x7f3894122390 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f3892ffd640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 msgr2=0x7f38940a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f3892ffd640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 0x7f38940a1a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f388402a0c0 tx=0x7f3884008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3884003ee0 con 0x7f3894122390 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 10 monclient: dump: 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.819 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 <== mon.0 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9198 (secure 0 0 0) 0x7f388404c790 con 0x7f3894122390 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 msgr2=0x7f38940a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 0x7f38940a1a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f388402a0c0 tx=0x7f3884008030 comp rx=0 tx=0).stop 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 msgr2=0x7f38940a1a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:52.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.805+0000 7f388bfff640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3894122390 0x7f38940a1a20 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f388402a0c0 tx=0x7f3884008030 comp rx=0 tx=0).stop 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "leader", 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:52.821 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.822 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.823 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:52.824 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.825 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.826 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.827 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.828 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.829 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.830 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.831 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.832 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.833 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:52.834 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:52.835 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:52.836 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.837 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.838 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.839 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.840 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:52.841 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:52.842 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:52.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 >> v2:172.21.15.162:6800/980547263 conn(0x7f3874038880 msgr2=0x7f387403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 --2- 172.21.15.134:0/4195554522 >> v2:172.21.15.162:6800/980547263 conn(0x7f3874038880 0x7f387403ad40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f38941231d0 tx=0x7f387c009190 comp rx=0 tx=0).stop 2024-09-16T15:20:52.843 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 10 monclient: shutdown 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 msgr2=0x7f3894135550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f38937fe640 1 -- 172.21.15.134:0/4195554522 reap_dead start 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f388c007f20 tx=0x7f388c00d560 comp rx=0 tx=0).stop 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 shutdown_connections 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 --2- 172.21.15.134:0/4195554522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3894137220 0x7f3894135550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 >> 172.21.15.134:0/4195554522 conn(0x7f38940a6110 msgr2=0x7f389406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 shutdown_connections 2024-09-16T15:20:52.844 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.806+0000 7f3899620640 1 -- 172.21.15.134:0/4195554522 wait complete. 2024-09-16T15:20:52.844 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:52.845 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-16T15:20:52.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.940+0000 7f4af1933640 10 monclient: get_monmap_and_config 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.940+0000 7f4af1933640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.941+0000 7f4af1933640 10 monclient: monmap: 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:52.941933+0000 2024-09-16T15:20:52.942 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:52.941933+0000 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:52.943 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.941+0000 7f4af1933640 1 Processor -- start 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.941+0000 7f4af1933640 1 -- start start 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.941+0000 7f4af1933640 10 monclient: init 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient: _add_conns ranks=[2,9,4,7,0,8,5,1,3,6,10] 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-b con 0x7f4aec136e40 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 0x7f4aec137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-a con 0x7f4aec137220 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:52.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-c con 0x7f4aec135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec09abc0 con 0x7f4aec137220 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec09ba50 con 0x7f4aec136e40 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0a53c0 con 0x7f4aec135dc0 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4af1933640 10 monclient(hunting): authenticate will time out at 1989.531254s 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38942/0 (socket says 172.21.15.134:38942) 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/3409344239 learned_addr learned my addr 172.21.15.134:0/3409344239 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45002/0 (socket says 172.21.15.134:45002) 2024-09-16T15:20:52.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): get_auth_request con 0x7f4aec135dc0 auth_method 0 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 1 --2- 172.21.15.134:0/3409344239 >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 0x7f4aec137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request con 0x7f4aec136e40 auth_method 0 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): get_auth_request con 0x7f4aec137220 auth_method 0 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_done global_id 124402 payload 274 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 msgr2=0x7f4aec09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/3409344239 >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 msgr2=0x7f4aec137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/3409344239 >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 0x7f4aec137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 monclient: found mon.noname-c 2024-09-16T15:20:52.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/3409344239 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec073250 con 0x7f4aec135dc0 2024-09-16T15:20:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4ae3fff640 1 --2- 172.21.15.134:0/3409344239 >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 0x7f4aec137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:52.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.942+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 msgr2=0x7f4aec136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009e80 tx=0x7f4ae402a580 comp rx=0 tx=0).ready entity=mon.4 client_cookie=84e7969ded06b78b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 1 -- 172.21.15.134:0/3409344239 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ae4007a70 con 0x7f4aec135dc0 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:52.956 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: dump: 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.957 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 1 -- 172.21.15.134:0/3409344239 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae402d050 con 0x7f4aec135dc0 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4af1933640 5 monclient: authenticate success, global_id 124402 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4af1933640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4af1933640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 1 -- 172.21.15.134:0/3409344239 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ae4007bf0 con 0x7f4aec135dc0 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4af1933640 4 set_mon_vals no callback set 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:52.958 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.945+0000 7f4aea7fc640 10 monclient: dump: 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.959 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 10 monclient: shutdown 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 -- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 msgr2=0x7f4aec136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009e80 tx=0x7f4ae402a580 comp rx=0 tx=0).stop 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 -- 172.21.15.134:0/3409344239 shutdown_connections 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4aec135dc0 0x7f4aec136220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 --2- 172.21.15.134:0/3409344239 >> v2:172.21.15.134:3300/0 conn(0x7f4aec137220 0x7f4aec137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 --2- 172.21.15.134:0/3409344239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec136e40 0x7f4aec09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 -- 172.21.15.134:0/3409344239 >> 172.21.15.134:0/3409344239 conn(0x7f4aec0a6110 msgr2=0x7f4aec0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 -- 172.21.15.134:0/3409344239 shutdown_connections 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 1 -- 172.21.15.134:0/3409344239 wait complete. 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 10 monclient: build_initial_monmap 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.946+0000 7f4af1933640 10 monclient: monmap: 2024-09-16T15:20:52.960 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:52.961 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 Processor -- start 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 -- start start 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient: init 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient: _add_conns ranks=[5,7,2,6,1,0,4,3,8] 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4aec135dc0 0x7f4aec134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-h con 0x7f4aec135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4aec136e40 0x7f4aec1354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-i con 0x7f4aec136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): picked mon.noname-b con 0x7f4aec137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1cc3e0 con 0x7f4aec137220 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.962 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec122090 con 0x7f4aec135dc0 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec122650 con 0x7f4aec136e40 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4af1933640 10 monclient(hunting): authenticate will time out at 1989.536905s 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.947+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45016/0 (socket says 172.21.15.134:45016) 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 learned_addr learned my addr 172.21.15.134:0/1347164052 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): get_auth_request con 0x7f4aec137220 auth_method 0 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4aec135dc0 0x7f4aec134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request con 0x7f4aec135dc0 auth_method 0 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4aec136e40 0x7f4aec1354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): get_auth_request con 0x7f4aec136e40 auth_method 0 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4ae3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient(hunting): handle_auth_done global_id 114671 payload 824 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4aec136e40 msgr2=0x7f4aec1354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4aec136e40 0x7f4aec1354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4aec135dc0 msgr2=0x7f4aec134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4aec135dc0 0x7f4aec134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 monclient: found mon.noname-b 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.948+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc009400 con 0x7f4aec137220 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 msgr2=0x7f4aec11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4aeb7fe640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fa0 tx=0x7f4ae4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ae4003e50 con 0x7f4aec137220 2024-09-16T15:20:52.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: dump: 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.966 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: _finish_auth 0 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4004790 con 0x7f4aec137220 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 5 monclient: authenticate success, global_id 114671 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 10 monclient: _renew_subs 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad4009400 con 0x7f4aec137220 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ae403b400 con 0x7f4aec137220 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:52.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae8ff9640 10 monclient: dump: 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.968 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4ae9ffb640 4 set_mon_vals no callback set 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 10 monclient: _renew_subs 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.950+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae4009b30 con 0x7f4aec137220 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4af1933640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4af1933640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 0x7f4aec02b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4ae4007600 con 0x7f4aec137220 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec071b20 con 0x7f4aec02b260 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 0x7f4aec02b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient: get_auth_request con 0x7f4aec02b260 auth_method 0 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4ae8ff9640 1 --2- 172.21.15.134:0/1347164052 >> v2:172.21.15.162:6800/980547263 conn(0x7f4acc038880 0x7f4acc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f4ae406a9f0 con 0x7f4aec137220 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_done global_id 114680 payload 824 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4aeaffd640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4aec061ae0 con 0x7f4aec02b260 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4ae3fff640 1 --2- 172.21.15.134:0/1347164052 >> v2:172.21.15.162:6800/980547263 conn(0x7f4acc038880 0x7f4acc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.951+0000 7f4ae3fff640 10 monclient: get_auth_request con 0x7f4acc038880 auth_method 0 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.953+0000 7f4aeaffd640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 msgr2=0x7f4aec02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 0x7f4aec02b640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc00f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4adc014ac0 con 0x7f4aec02b260 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 10 monclient: dump: 2024-09-16T15:20:52.970 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4adc014c20 con 0x7f4aec02b260 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 msgr2=0x7f4aec02b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 0x7f4aec02b640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc00f610 comp rx=0 tx=0).stop 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 msgr2=0x7f4aec02b640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae8ff9640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02b260 0x7f4aec02b640 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc00f610 comp rx=0 tx=0).stop 2024-09-16T15:20:52.972 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.954+0000 7f4ae3fff640 1 --2- 172.21.15.134:0/1347164052 >> v2:172.21.15.162:6800/980547263 conn(0x7f4acc038880 0x7f4acc03ad40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4aec075e40 tx=0x7f4ad4009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4af1933640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4af1933640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 0x7f4aec02bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4af1933640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec07a680 con 0x7f4aec02bb80 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 0x7f4aec02bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient: get_auth_request con 0x7f4aec02bb80 auth_method 0 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:52.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 10 monclient(hunting): handle_auth_done global_id 114689 payload 824 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.960+0000 7f4aeaffd640 1 -- 172.21.15.134:0/1347164052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4aec09b490 con 0x7f4aec02bb80 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.962+0000 7f4aeaffd640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 msgr2=0x7f4aec02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4aeaffd640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 0x7f4aec02bf90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4adc022e10 con 0x7f4aec02bb80 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 10 monclient: dump: 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:52.975 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9528 (secure 0 0 0) 0x7f4adc026070 con 0x7f4aec02bb80 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 msgr2=0x7f4aec02bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.976 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 0x7f4aec02bf90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 msgr2=0x7f4aec02bf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4ae8ff9640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec02bb80 0x7f4aec02bf90 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4adc011c00 tx=0x7f4adc0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:52.977 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.978 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.979 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:52.980 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:52.981 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.982 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.983 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.984 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.985 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.986 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.987 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:52.988 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.989 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:52.991 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:52.992 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.993 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:52.994 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:52.995 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 4 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "mgr": [ 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:52.997 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:52.998 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 >> v2:172.21.15.162:6800/980547263 conn(0x7f4acc038880 msgr2=0x7f4acc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 --2- 172.21.15.134:0/1347164052 >> v2:172.21.15.162:6800/980547263 conn(0x7f4acc038880 0x7f4acc03ad40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4aec075e40 tx=0x7f4ad4009190 comp rx=0 tx=0).stop 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 10 monclient: shutdown 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 msgr2=0x7f4aec11fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4009fa0 tx=0x7f4ae4004a40 comp rx=0 tx=0).stop 2024-09-16T15:20:52.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4aeb7fe640 1 -- 172.21.15.134:0/1347164052 reap_dead start 2024-09-16T15:20:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 shutdown_connections 2024-09-16T15:20:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 --2- 172.21.15.134:0/1347164052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aec137220 0x7f4aec11fc60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 >> 172.21.15.134:0/1347164052 conn(0x7f4aec0a6110 msgr2=0x7f4aec09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 shutdown_connections 2024-09-16T15:20:53.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:52.963+0000 7f4af1933640 1 -- 172.21.15.134:0/1347164052 wait complete. 2024-09-16T15:20:53.000 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.000 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-16T15:20:53.000 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.100+0000 7fd280e44640 10 monclient: get_monmap_and_config 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.100+0000 7fd280e44640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.100+0000 7fd280e44640 10 monclient: monmap: 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:53.101775+0000 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:53.101775+0000 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:53.102 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:53.103 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.101+0000 7fd280e44640 1 Processor -- start 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.101+0000 7fd280e44640 1 -- start start 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.101+0000 7fd280e44640 10 monclient: init 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient: _add_conns ranks=[4,0,5,1,7,3,2,8,9,10,6] 2024-09-16T15:20:53.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-c con 0x7fd27c136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd27c137220 addr v1:172.21.15.134:6789/0 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c13c370 0x7fd27c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-h con 0x7fd27c13c370 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27c073250 con 0x7fd27c137220 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0a56e0 con 0x7fd27c136e40 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c09abc0 con 0x7fd27c13c370 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd280e44640 10 monclient(hunting): authenticate will time out at 1989.691180s 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd273fff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd27c137220 0x7fd27c137600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:45756/0 (socket says 172.21.15.134:45756) 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38944/0 (socket says 172.21.15.134:38944) 2024-09-16T15:20:53.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd273fff640 1 -- 172.21.15.134:0/38084873 learned_addr learned my addr 172.21.15.134:0/38084873 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): get_auth_request con 0x7fd27c136e40 auth_method 0 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient(hunting): handle_auth_done global_id 124411 payload 274 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 -- 172.21.15.134:0/38084873 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c13c370 msgr2=0x7fd27c13e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27ad9d640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c13c370 0x7fd27c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c13c370 0x7fd27c13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 -- 172.21.15.134:0/38084873 >> v1:172.21.15.134:6789/0 conn(0x7fd27c137220 legacy=0x7fd27c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 monclient: found mon.noname-c 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:53.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.102+0000 7fd27a59c640 1 -- 172.21.15.134:0/38084873 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c09ba50 con 0x7fd27c136e40 2024-09-16T15:20:53.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd27a59c640 1 -- 172.21.15.134:0/38084873 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 msgr2=0x7fd27c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd27a59c640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0097d0 tx=0x7fd26c02a000 comp rx=0 tx=0).ready entity=mon.4 client_cookie=2c80e0124feaedcc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 1 -- 172.21.15.134:0/38084873 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd26c02ae00 con 0x7fd27c136e40 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: dump: 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.116 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd280e44640 5 monclient: authenticate success, global_id 124411 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd280e44640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 1 -- 172.21.15.134:0/38084873 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c0307b0 con 0x7fd27c136e40 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd280e44640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd280e44640 4 set_mon_vals no callback set 2024-09-16T15:20:53.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 1 -- 172.21.15.134:0/38084873 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd26c030d90 con 0x7fd27c136e40 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.105+0000 7fd279d9b640 10 monclient: dump: 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.118 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 10 monclient: shutdown 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 -- 172.21.15.134:0/38084873 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 msgr2=0x7fd27c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0097d0 tx=0x7fd26c02a000 comp rx=0 tx=0).stop 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 -- 172.21.15.134:0/38084873 shutdown_connections 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c13c370 0x7fd27c13e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 --2- 172.21.15.134:0/38084873 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c136e40 0x7fd27c09c150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 -- 172.21.15.134:0/38084873 >> 172.21.15.134:0/38084873 conn(0x7fd27c0a6110 msgr2=0x7fd27c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 -- 172.21.15.134:0/38084873 shutdown_connections 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 1 -- 172.21.15.134:0/38084873 wait complete. 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.106+0000 7fd280e44640 10 monclient: monmap: 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.120 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 Processor -- start 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 -- start start 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient: init 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient: _add_conns ranks=[5,2,3,6,4,7,8,0,1] 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c136e40 0x7fd27c0a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-h con 0x7fd27c136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 0x7fd27c135cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-b con 0x7fd27c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27c13c370 0x7fd27c136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): picked mon.noname-g con 0x7fd27c13c370 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c121000 con 0x7fd27c137220 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0a76a0 con 0x7fd27c13c370 2024-09-16T15:20:53.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c13b610 con 0x7fd27c136e40 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd280e44640 10 monclient(hunting): authenticate will time out at 1989.696656s 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 0x7fd27c135cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 0x7fd27c135cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45040/0 (socket says 172.21.15.134:45040) 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 1 -- 172.21.15.134:0/4103954504 learned_addr learned my addr 172.21.15.134:0/4103954504 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): get_auth_request con 0x7fd27c137220 auth_method 0 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd273fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c136e40 0x7fd27c0a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27ad9d640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27c13c370 0x7fd27c136210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): get_auth_request con 0x7fd27c136e40 auth_method 0 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27ad9d640 10 monclient(hunting): get_auth_request con 0x7fd27c13c370 auth_method 0 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27ad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27ad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27ad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.107+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd27ad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd27ad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd27ad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 10 monclient(hunting): handle_auth_done global_id 114698 payload 824 2024-09-16T15:20:53.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c136e40 msgr2=0x7fd27c0a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd27c136e40 0x7fd27c0a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27c13c370 msgr2=0x7fd27c136210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27c13c370 0x7fd27c136210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 monclient: found mon.noname-b 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.108+0000 7fd273fff640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c009400 con 0x7fd27c137220 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd273fff640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 msgr2=0x7fd27c135cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd273fff640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 0x7fd27c135cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c460 tx=0x7fd26400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd264005cf0 con 0x7fd27c137220 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: dump: 2024-09-16T15:20:53.124 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.125 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 5 monclient: authenticate success, global_id 114698 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264005e90 con 0x7fd27c137220 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd2640074e0 con 0x7fd27c137220 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 10 monclient: _renew_subs 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c0a7bd0 con 0x7fd27c137220 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd27959a640 4 set_mon_vals no callback set 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd272ffd640 10 monclient: dump: 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.126 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 10 monclient: _renew_subs 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.110+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c121de0 con 0x7fd27c137220 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd280e44640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd280e44640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 0x7fd27c120420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c071b20 con 0x7fd27c122000 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd264007070 con 0x7fd27c137220 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 0x7fd27c120420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient: get_auth_request con 0x7fd27c122000 auth_method 0 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd272ffd640 1 --2- 172.21.15.134:0/4103954504 >> v2:172.21.15.162:6800/980547263 conn(0x7fd25c038880 0x7fd25c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 10 monclient(hunting): handle_auth_done global_id 124420 payload 824 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fd264049ac0 con 0x7fd27c137220 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.111+0000 7fd27a59c640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27c061ae0 con 0x7fd27c122000 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.112+0000 7fd27ad9d640 1 --2- 172.21.15.134:0/4103954504 >> v2:172.21.15.162:6800/980547263 conn(0x7fd25c038880 0x7fd25c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.112+0000 7fd27ad9d640 10 monclient: get_auth_request con 0x7fd25c038880 auth_method 0 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.113+0000 7fd27a59c640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 msgr2=0x7fd27c120420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.113+0000 7fd27a59c640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 0x7fd27c120420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c0046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd26c003a80 con 0x7fd27c122000 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 10 monclient: dump: 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.129 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd26c003be0 con 0x7fd27c122000 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:53.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 msgr2=0x7fd27c120420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 0x7fd27c120420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 msgr2=0x7fd27c120420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd272ffd640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c122000 0x7fd27c120420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.131 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.114+0000 7fd27ad9d640 1 --2- 172.21.15.134:0/4103954504 >> v2:172.21.15.162:6800/980547263 conn(0x7fd25c038880 0x7fd25c03ad40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06b7d0 tx=0x7fd274007650 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd280e44640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd280e44640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 0x7fd27c02b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd280e44640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0a2d00 con 0x7fd27c02b260 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 0x7fd27c02b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient: get_auth_request con 0x7fd27c02b260 auth_method 0 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.132 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 10 monclient(hunting): handle_auth_done global_id 124429 payload 824 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.120+0000 7fd27a59c640 1 -- 172.21.15.134:0/4103954504 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd27c07a680 con 0x7fd27c02b260 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd27a59c640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 msgr2=0x7fd27c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd27a59c640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 0x7fd27c02b670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0096f0 tx=0x7fd26c008030 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd26c048e40 con 0x7fd27c02b260 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 10 monclient: dump: 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.133 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.134 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:53.135 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.136 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.137 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:53.138 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:53.139 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.140 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.141 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.142 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.143 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.144 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.145 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:53.146 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.147 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.148 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.149 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.150 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.151 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.152 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:53.153 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.154 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:53.155 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 <== mon.4 v2:172.21.15.134:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fd26c04c070 con 0x7fd27c02b260 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 msgr2=0x7fd27c02b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 0x7fd27c02b670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0096f0 tx=0x7fd26c008030 comp rx=0 tx=0).stop 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 msgr2=0x7fd27c02b670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd272ffd640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd27c02b260 0x7fd27c02b670 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0096f0 tx=0x7fd26c008030 comp rx=0 tx=0).stop 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 >> v2:172.21.15.162:6800/980547263 conn(0x7fd25c038880 msgr2=0x7fd25c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 --2- 172.21.15.134:0/4103954504 >> v2:172.21.15.162:6800/980547263 conn(0x7fd25c038880 0x7fd25c03ad40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06b7d0 tx=0x7fd274007650 comp rx=0 tx=0).stop 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 10 monclient: shutdown 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 msgr2=0x7fd27c135cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 --2- 172.21.15.134:0/4103954504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd27c137220 0x7fd27c135cd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c460 tx=0x7fd26400c920 comp rx=0 tx=0).stop 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd27ad9d640 1 -- 172.21.15.134:0/4103954504 reap_dead start 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd27ad9d640 1 -- 172.21.15.134:0/4103954504 reap_dead start 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 shutdown_connections 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 >> 172.21.15.134:0/4103954504 conn(0x7fd27c0a6110 msgr2=0x7fd27c13a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 shutdown_connections 2024-09-16T15:20:53.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.123+0000 7fd280e44640 1 -- 172.21.15.134:0/4103954504 wait complete. 2024-09-16T15:20:53.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.158 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-16T15:20:53.158 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 10 monclient: get_monmap_and_config 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 10 monclient: monmap: 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:53.256025+0000 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:53.256025+0000 2024-09-16T15:20:53.256 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:53.257 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 1 Processor -- start 2024-09-16T15:20:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 1 -- start start 2024-09-16T15:20:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.255+0000 7f671d60d640 10 monclient: init 2024-09-16T15:20:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient: _add_conns ranks=[9,4,3,7,10,1,5,0,8,2,6] 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 0x7f67181342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-a con 0x7f6718133f10 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-c con 0x7f6718134830 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6718071bd0 0x7f67181351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-g con 0x7f6718071bd0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67180a3d50 con 0x7f6718133f10 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671806f630 con 0x7f6718071bd0 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67180a1e10 con 0x7f6718134830 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f671d60d640 10 monclient(hunting): authenticate will time out at 1989.845175s 2024-09-16T15:20:53.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 0x7f67181342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38974/0 (socket says 172.21.15.134:38974) 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 -- 172.21.15.134:0/490779699 learned_addr learned my addr 172.21.15.134:0/490779699 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): get_auth_request con 0x7f6718134830 auth_method 0 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): get_auth_request con 0x7f6718133f10 auth_method 0 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67177fe640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6718071bd0 0x7f67181351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67177fe640 10 monclient(hunting): get_auth_request con 0x7f6718071bd0 auth_method 0 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67177fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient(hunting): handle_auth_done global_id 124438 payload 274 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 -- 172.21.15.134:0/490779699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6718071bd0 msgr2=0x7f67181351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6718071bd0 0x7f67181351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 -- 172.21.15.134:0/490779699 >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 msgr2=0x7f67181342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 --2- 172.21.15.134:0/490779699 >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 0x7f67181342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 monclient: found mon.noname-c 2024-09-16T15:20:53.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f67167fc640 1 -- 172.21.15.134:0/490779699 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718073250 con 0x7f6718134830 2024-09-16T15:20:53.263 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.256+0000 7f6716ffd640 1 --2- 172.21.15.134:0/490779699 >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 0x7f67181342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f67167fc640 1 -- 172.21.15.134:0/490779699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 msgr2=0x7f6718134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f67167fc640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6700009870 tx=0x7f670002a0a0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=d06da8515cd7c258 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 1 -- 172.21.15.134:0/490779699 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f670002ae00 con 0x7f6718134830 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: dump: 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.270 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f671d60d640 5 monclient: authenticate success, global_id 124438 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 1 -- 172.21.15.134:0/490779699 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67000307b0 con 0x7f6718134830 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f671d60d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f671d60d640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f671d60d640 4 set_mon_vals no callback set 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 1 -- 172.21.15.134:0/490779699 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6700030d90 con 0x7f6718134830 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.259+0000 7f6715ffb640 10 monclient: dump: 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.272 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 10 monclient: shutdown 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 -- 172.21.15.134:0/490779699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 msgr2=0x7f6718134cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6700009870 tx=0x7f670002a0a0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 -- 172.21.15.134:0/490779699 shutdown_connections 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6718071bd0 0x7f67181351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 --2- 172.21.15.134:0/490779699 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6718134830 0x7f6718134cb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 --2- 172.21.15.134:0/490779699 >> v2:172.21.15.134:3300/0 conn(0x7f6718133f10 0x7f67181342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.273 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 -- 172.21.15.134:0/490779699 >> 172.21.15.134:0/490779699 conn(0x7f67180a7410 msgr2=0x7f67180a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 -- 172.21.15.134:0/490779699 shutdown_connections 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 1 -- 172.21.15.134:0/490779699 wait complete. 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.260+0000 7f671d60d640 10 monclient: monmap: 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.274 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 Processor -- start 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 -- start start 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient: init 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient: _add_conns ranks=[2,6,5,3,0,1,4,8,7] 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-b con 0x7f6718071bd0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6718133f10 0x7f6718132c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-d con 0x7f6718133f10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6718134830 0x7f67181331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): picked mon.noname-h con 0x7f6718134830 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:53.275 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718120160 con 0x7f6718071bd0 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67180a1200 con 0x7f6718134830 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f67180a4b10 con 0x7f6718133f10 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f671d60d640 10 monclient(hunting): authenticate will time out at 1989.850768s 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6718133f10 0x7f6718132c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45048/0 (socket says 172.21.15.134:45048) 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 1 -- 172.21.15.134:0/2333313094 learned_addr learned my addr 172.21.15.134:0/2333313094 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): get_auth_request con 0x7f6718071bd0 auth_method 0 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.276 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): get_auth_request con 0x7f6718133f10 auth_method 0 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f6716ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.261+0000 7f67167fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6718134830 0x7f67181331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 10 monclient(hunting): handle_auth_done global_id 114707 payload 824 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6718133f10 msgr2=0x7f6718132c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6718133f10 0x7f6718132c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.277 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6718134830 msgr2=0x7f67181331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6718134830 0x7f67181331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 monclient: found mon.noname-b 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f6716ffd640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009a70 con 0x7f6718071bd0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f67177fe640 10 monclient: get_auth_request con 0x7f6718134830 auth_method 0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f67167fc640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f6718133f10 0x7f6718132c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.262+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6718134830 0x7f67181331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f6716ffd640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 msgr2=0x7f67180a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f6716ffd640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f671000efc0 tx=0x7f671000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f671000ced0 con 0x7f6718071bd0 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: dump: 2024-09-16T15:20:53.278 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.279 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000f040 con 0x7f6718071bd0 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 5 monclient: authenticate success, global_id 114707 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 10 monclient: _renew_subs 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000094d0 con 0x7f6718071bd0 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67100133e0 con 0x7f6718071bd0 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f670f7fe640 10 monclient: dump: 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.280 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f67157fa640 4 set_mon_vals no callback set 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 10 monclient: _renew_subs 2024-09-16T15:20:53.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.264+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671811df20 con 0x7f6718071bd0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6710014690 con 0x7f6718071bd0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f671d60d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f670f7fe640 1 --2- 172.21.15.134:0/2333313094 >> v2:172.21.15.162:6800/980547263 conn(0x7f66f8038880 0x7f66f803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f671d60d640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 0x7f67181c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f671806a3f0 con 0x7f671811ee20 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f671004a9c0 con 0x7f6718071bd0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 0x7f67181c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient: get_auth_request con 0x7f671811ee20 auth_method 0 2024-09-16T15:20:53.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67167fc640 1 --2- 172.21.15.134:0/2333313094 >> v2:172.21.15.162:6800/980547263 conn(0x7f66f8038880 0x7f66f803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.265+0000 7f67167fc640 10 monclient: get_auth_request con 0x7f66f8038880 auth_method 0 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.266+0000 7f67177fe640 10 monclient(hunting): handle_auth_done global_id 94812 payload 824 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.266+0000 7f67177fe640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6718061ae0 con 0x7f671811ee20 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f67177fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 msgr2=0x7f67181c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 0x7f67181c9060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f670800e710 tx=0x7f670800ebd0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67080044a0 con 0x7f671811ee20 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 10 monclient: dump: 2024-09-16T15:20:53.283 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f67167fc640 1 --2- 172.21.15.134:0/2333313094 >> v2:172.21.15.162:6800/980547263 conn(0x7f66f8038880 0x7f66f803ad40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6718135d70 tx=0x7f670002e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6708004030 con 0x7f671811ee20 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 msgr2=0x7f67181c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 0x7f67181c9060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f670800e710 tx=0x7f670800ebd0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 msgr2=0x7f67181c9060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.268+0000 7f670f7fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f671811ee20 0x7f67181c9060 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f670800e710 tx=0x7f670800ebd0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f671d60d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f671d60d640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 0x7f67181c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f671d60d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f671807a680 con 0x7f67181c95a0 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 0x7f67181c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient: get_auth_request con 0x7f67181c95a0 auth_method 0 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.274+0000 7f67177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.275+0000 7f67177fe640 10 monclient(hunting): handle_auth_done global_id 94821 payload 824 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.275+0000 7f67177fe640 1 -- 172.21.15.134:0/2333313094 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6718070970 con 0x7f67181c95a0 2024-09-16T15:20:53.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f67177fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 msgr2=0x7f67181c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f67177fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 0x7f67181c99b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f670800e710 tx=0x7f6708004730 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f670801e2e0 con 0x7f67181c95a0 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 10 monclient: dump: 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.288 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 <== mon.6 v2:172.21.15.134:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7f670800c950 con 0x7f67181c95a0 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 msgr2=0x7f67181c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 0x7f67181c99b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f670800e710 tx=0x7f6708004730 comp rx=0 tx=0).stop 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 msgr2=0x7f67181c99b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.277+0000 7f670f7fe640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f67181c95a0 0x7f67181c99b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:53.290 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.291 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.292 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:53.293 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.294 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.295 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.296 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.297 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.298 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.299 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.300 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.301 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.302 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.303 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.304 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.305 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.306 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.307 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.308 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.309 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:53.310 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 >> v2:172.21.15.162:6800/980547263 conn(0x7f66f8038880 msgr2=0x7f66f803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 --2- 172.21.15.134:0/2333313094 >> v2:172.21.15.162:6800/980547263 conn(0x7f66f8038880 0x7f66f803ad40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6718135d70 tx=0x7f670002e000 comp rx=0 tx=0).stop 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 10 monclient: shutdown 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 msgr2=0x7f67180a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f67177fe640 1 -- 172.21.15.134:0/2333313094 reap_dead start 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f671000efc0 tx=0x7f671000c490 comp rx=0 tx=0).stop 2024-09-16T15:20:53.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 shutdown_connections 2024-09-16T15:20:53.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 --2- 172.21.15.134:0/2333313094 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718071bd0 0x7f67180a0700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 >> 172.21.15.134:0/2333313094 conn(0x7f67180a7410 msgr2=0x7f6718134c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 shutdown_connections 2024-09-16T15:20:53.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.278+0000 7f671d60d640 1 -- 172.21.15.134:0/2333313094 wait complete. 2024-09-16T15:20:53.312 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.312 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-16T15:20:53.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.408+0000 7fbdaafc7640 10 monclient: get_monmap_and_config 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.408+0000 7fbdaafc7640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.408+0000 7fbdaafc7640 10 monclient: monmap: 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:53.409362+0000 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:53.409362+0000 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.410 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:53.411 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 Processor -- start 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 -- start start 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient: init 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient: _add_conns ranks=[9,8,6,2,0,1,4,7,10,5,3] 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.413 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-a con 0x7fbda4136e40 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-e con 0x7fbda4137220 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-d con 0x7fbda4135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda409abc0 con 0x7fbda4136e40 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbda409ba50 con 0x7fbda4135dc0 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40a53c0 con 0x7fbda4137220 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbdaafc7640 10 monclient(hunting): authenticate will time out at 1989.998743s 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda9564640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56252/0 (socket says 172.21.15.134:56252) 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 1 -- 172.21.15.134:0/2232935152 learned_addr learned my addr 172.21.15.134:0/2232935152 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): get_auth_request con 0x7fbda4136e40 auth_method 0 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda9564640 10 monclient(hunting): get_auth_request con 0x7fbda4135dc0 auth_method 0 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda9564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda9564640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda9564640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda3fff640 10 monclient(hunting): get_auth_request con 0x7fbda4137220 auth_method 0 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.409+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 10 monclient(hunting): handle_auth_done global_id 124389 payload 274 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 -- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 msgr2=0x7fbda4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 -- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 monclient: found mon.noname-a 2024-09-16T15:20:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda8d63640 1 -- 172.21.15.134:0/2232935152 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4073250 con 0x7fbda4136e40 2024-09-16T15:20:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda3fff640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.410+0000 7fbda9564640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.412+0000 7fbda8d63640 1 -- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4136e40 msgr2=0x7fbda409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.412+0000 7fbda8d63640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4136e40 0x7fbda409c150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009870 tx=0x7fbd9c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c9ef9a0ffe66d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 1 -- 172.21.15.134:0/2232935152 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c02ae00 con 0x7fbda4136e40 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: dump: 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.424 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 1 -- 172.21.15.134:0/2232935152 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0307b0 con 0x7fbda4136e40 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 5 monclient: authenticate success, global_id 124389 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:53.425 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 1 -- 172.21.15.134:0/2232935152 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c030d90 con 0x7fbda4136e40 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 4 set_mon_vals no callback set 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbda37fe640 10 monclient: dump: 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.426 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 10 monclient: shutdown 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 -- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4136e40 msgr2=0x7fbda409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4136e40 0x7fbda409c150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009870 tx=0x7fbd9c02a0a0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 -- 172.21.15.134:0/2232935152 shutdown_connections 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4137220 0x7fbda4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4135dc0 0x7fbda4136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/2232935152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbda4136e40 0x7fbda409c150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 -- 172.21.15.134:0/2232935152 >> 172.21.15.134:0/2232935152 conn(0x7fbda40a6110 msgr2=0x7fbda40a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 -- 172.21.15.134:0/2232935152 shutdown_connections 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.413+0000 7fbdaafc7640 1 -- 172.21.15.134:0/2232935152 wait complete. 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.414+0000 7fbdaafc7640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.414+0000 7fbdaafc7640 10 monclient: monmap: 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.428 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.414+0000 7fbdaafc7640 1 Processor -- start 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.414+0000 7fbdaafc7640 1 -- start start 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.414+0000 7fbdaafc7640 10 monclient: init 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient: _add_conns ranks=[1,8,6,7,4,3,0,2,5] 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda4135dc0 0x7fbda4134e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-f con 0x7fbda4135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 0x7fbda4135370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-e con 0x7fbda4136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): picked mon.noname-d con 0x7fbda4137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4124f80 con 0x7fbda4135dc0 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41bd570 con 0x7fbda4137220 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4136830 con 0x7fbda4136e40 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbdaafc7640 10 monclient(hunting): authenticate will time out at 1990.004260s 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 0x7fbda4135370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 0x7fbda4135370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52484/0 (socket says 172.21.15.134:52484) 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51406/0 (socket says 172.21.15.134:51406) 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 1 -- 172.21.15.134:0/3031221000 learned_addr learned my addr 172.21.15.134:0/3031221000 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): get_auth_request con 0x7fbda4136e40 auth_method 0 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): get_auth_request con 0x7fbda4137220 auth_method 0 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda8d63640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda4135dc0 0x7fbda4134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda8d63640 10 monclient(hunting): get_auth_request con 0x7fbda4135dc0 auth_method 0 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient(hunting): handle_auth_done global_id 94839 payload 824 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 msgr2=0x7fbda4135370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 0x7fbda4135370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda4135dc0 msgr2=0x7fbda4134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbda4135dc0 0x7fbda4134e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 monclient: found mon.noname-d 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda9564640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009400 con 0x7fbda4137220 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.415+0000 7fbda3fff640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4136e40 0x7fbda4135370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.417+0000 7fbda9564640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 msgr2=0x7fbda41218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.417+0000 7fbda9564640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e8b0 tx=0x7fbd8c00ed70 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.417+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd8c013470 con 0x7fbda4137220 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:53.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: dump: 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.433 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c013610 con 0x7fbda4137220 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 5 monclient: authenticate success, global_id 94839 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 monclient: _renew_subs 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd94009a70 con 0x7fbda4137220 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd8c013bf0 con 0x7fbda4137220 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda17fa640 10 monclient: dump: 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.434 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 monclient: _renew_subs 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbda2ffd640 4 set_mon_vals no callback set 2024-09-16T15:20:53.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0094d0 con 0x7fbda4137220 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 0x7fbda402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.418+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4071b20 con 0x7fbda41225b0 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbd8c013780 con 0x7fbda4137220 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 0x7fbda402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient: get_auth_request con 0x7fbda41225b0 auth_method 0 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda17fa640 1 --2- 172.21.15.134:0/3031221000 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd84038880 0x7fbd8403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 10 monclient(hunting): handle_auth_done global_id 85121 payload 824 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda8d63640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbda4061ae0 con 0x7fbda41225b0 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fbd8c049f50 con 0x7fbda4137220 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda3fff640 1 --2- 172.21.15.134:0/3031221000 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd84038880 0x7fbd8403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.419+0000 7fbda3fff640 10 monclient: get_auth_request con 0x7fbd84038880 auth_method 0 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda8d63640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 msgr2=0x7fbda402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda8d63640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 0x7fbda402b260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbda4137f00 tx=0x7fbd9c0046c0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c003610 con 0x7fbda41225b0 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 10 monclient: dump: 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.437 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbd9c030b30 con 0x7fbda41225b0 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 msgr2=0x7fbda402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 0x7fbda402b260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbda4137f00 tx=0x7fbd9c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 msgr2=0x7fbda402b260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.421+0000 7fbda17fa640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda41225b0 0x7fbda402b260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.422+0000 7fbda3fff640 1 --2- 172.21.15.134:0/3031221000 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd84038880 0x7fbd8403ad40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd94003de0 tx=0x7fbd9400b040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.427+0000 7fbdaafc7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:53.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 0x7fbda402b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbdaafc7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbda407a680 con 0x7fbda4122990 2024-09-16T15:20:53.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 0x7fbda402b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient: get_auth_request con 0x7fbda4122990 auth_method 0 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 10 monclient(hunting): handle_auth_done global_id 85130 payload 824 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.428+0000 7fbda8d63640 1 -- 172.21.15.134:0/3031221000 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbda409b490 con 0x7fbda4122990 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda8d63640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 msgr2=0x7fbda402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda8d63640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 0x7fbda402b7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097c0 tx=0x7fbd9c008030 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbd9c048300 con 0x7fbda4122990 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:53.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 10 monclient: dump: 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.442 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 <== mon.8 v2:172.21.15.134:3304/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fbd9c003c80 con 0x7fbda4122990 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 msgr2=0x7fbda402b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 0x7fbda402b7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097c0 tx=0x7fbd9c008030 comp rx=0 tx=0).stop 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 msgr2=0x7fbda402b7a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.430+0000 7fbda17fa640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbda4122990 0x7fbda402b7a0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097c0 tx=0x7fbd9c008030 comp rx=0 tx=0).stop 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:53.443 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 3, 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:53.444 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.445 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:53.446 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.448 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.449 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.450 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:53.451 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.452 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.453 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.454 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.455 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.456 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.457 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.458 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.459 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.460 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.461 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:53.462 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:53.463 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd84038880 msgr2=0x7fbd8403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/3031221000 >> v2:172.21.15.162:6800/980547263 conn(0x7fbd84038880 0x7fbd8403ad40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd94003de0 tx=0x7fbd9400b040 comp rx=0 tx=0).stop 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 10 monclient: shutdown 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 msgr2=0x7fbda41218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbda9564640 1 -- 172.21.15.134:0/3031221000 reap_dead start 2024-09-16T15:20:53.464 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e8b0 tx=0x7fbd8c00ed70 comp rx=0 tx=0).stop 2024-09-16T15:20:53.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 shutdown_connections 2024-09-16T15:20:53.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 --2- 172.21.15.134:0/3031221000 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbda4137220 0x7fbda41218b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 >> 172.21.15.134:0/3031221000 conn(0x7fbda40a6110 msgr2=0x7fbda40a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 shutdown_connections 2024-09-16T15:20:53.465 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.431+0000 7fbdaafc7640 1 -- 172.21.15.134:0/3031221000 wait complete. 2024-09-16T15:20:53.465 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.465 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-16T15:20:53.465 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.562+0000 7fce917c7640 10 monclient: get_monmap_and_config 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.562+0000 7fce917c7640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.562+0000 7fce917c7640 10 monclient: monmap: 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:53.563703+0000 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:53.563703+0000 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:53.564 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:53.565 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 Processor -- start 2024-09-16T15:20:53.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 -- start start 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient: init 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient: _add_conns ranks=[6,2,5,9,7,1,0,3,8,4,10] 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-d con 0x7fce8c136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 0x7fce8c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-b con 0x7fce8c137220 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c135dc0 0x7fce8c136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-h con 0x7fce8c135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c09abc0 con 0x7fce8c137220 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c09ba50 con 0x7fce8c135dc0 2024-09-16T15:20:53.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0a53c0 con 0x7fce8c136e40 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.563+0000 7fce917c7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce917c7640 10 monclient(hunting): authenticate will time out at 1990.153015s 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 0x7fce8c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51412/0 (socket says 172.21.15.134:51412) 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 learned_addr learned my addr 172.21.15.134:0/4141356220 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): get_auth_request con 0x7fce8c136e40 auth_method 0 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 0x7fce8c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45058/0 (socket says 172.21.15.134:45058) 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): get_auth_request con 0x7fce8c137220 auth_method 0 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c135dc0 0x7fce8c136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request con 0x7fce8c135dc0 auth_method 0 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient(hunting): handle_auth_done global_id 94848 payload 274 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c135dc0 msgr2=0x7fce8c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c135dc0 0x7fce8c136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 msgr2=0x7fce8c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 0x7fce8c137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 monclient: found mon.noname-d 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c073250 con 0x7fce8c136e40 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.564+0000 7fce83fff640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fce8c137220 0x7fce8c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.565+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 msgr2=0x7fce8c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.565+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 msgr2=0x7fce8c09c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.565+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0x7fce7c0098a0 tx=0x7fce7c02a080 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:20:53.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.565+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:20:53.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient: get_auth_request con 0x7fce8c136e40 auth_method 0 2024-09-16T15:20:53.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 0 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T15:20:53.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient: ms_handle_reset current mon [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 msgr2=0x7fce8c09c150 unknown :-1 s=STATE_CLOSED l=0).mark_down 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c136e40 0x7fce8c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient: _add_conns ranks=[9,1,8,10,7,4,3,0] 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 --2- 172.21.15.134:0/4141356220 >> v2:172.21.15.134:3300/0 conn(0x7fce78003630 0x7fce78005a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): picked mon.noname-a con 0x7fce78003630 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fce780062d0 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce7800a280 0x7fce7800c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): picked mon.noname-e con 0x7fce7800a280 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7800d0c0 con 0x7fce780062d0 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7800e2e0 con 0x7fce78003630 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fce7800e470 con 0x7fce7800a280 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8a7fc640 10 monclient(hunting): ms_handle_reset stray mon [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> v2:172.21.15.134:3300/0 conn(0x7fce78003630 0x7fce78005a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): get_auth_request con 0x7fce78003630 auth_method 0 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce7800a280 0x7fce7800c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:20:53.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request con 0x7fce7800a280 auth_method 0 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient(hunting): handle_auth_done global_id 94848 payload 293 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce7800a280 msgr2=0x7fce7800c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fce7800a280 0x7fce7800c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> v1:172.21.15.162:6789/0 conn(0x7fce780062d0 legacy=0x7fce78008790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 monclient: found mon.noname-a 2024-09-16T15:20:53.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7800e6f0 con 0x7fce78003630 2024-09-16T15:20:53.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.766+0000 7fce8b7fe640 1 -- 172.21.15.134:0/4141356220 reap_dead start 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.768+0000 7fce8affd640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 msgr2=0x7fce78005a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.768+0000 7fce8affd640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce78005a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0a5550 tx=0x7fce7c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4173fb1ac5066ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7c0044a0 con 0x7fce78003630 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: dump: 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.779 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 5 monclient: authenticate success, global_id 94848 2024-09-16T15:20:53.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c0328b0 con 0x7fce78003630 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 1 -- 172.21.15.134:0/4141356220 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7c032e90 con 0x7fce78003630 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 4 set_mon_vals no callback set 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce8a7fc640 10 monclient: dump: 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.781 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 10 monclient: shutdown 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 -- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 msgr2=0x7fce78005a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce78005a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0a5550 tx=0x7fce7c009240 comp rx=0 tx=0).stop 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 -- 172.21.15.134:0/4141356220 shutdown_connections 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 --2- 172.21.15.134:0/4141356220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce78005a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 -- 172.21.15.134:0/4141356220 >> 172.21.15.134:0/4141356220 conn(0x7fce8c0a6110 msgr2=0x7fce8c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 -- 172.21.15.134:0/4141356220 shutdown_connections 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.769+0000 7fce917c7640 1 -- 172.21.15.134:0/4141356220 wait complete. 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.770+0000 7fce917c7640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.770+0000 7fce917c7640 10 monclient: monmap: 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.783 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.770+0000 7fce917c7640 1 Processor -- start 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.770+0000 7fce917c7640 1 -- start start 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.770+0000 7fce917c7640 10 monclient: init 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient: _add_conns ranks=[0,6,5,3,4,1,2,8,7] 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-a con 0x7fce78003630 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c134d30 0x7fce8c135110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-d con 0x7fce8c134d30 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c11fc60 0x7fce8c1200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): picked mon.noname-h con 0x7fce8c11fc60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1226f0 con 0x7fce78003630 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1364f0 con 0x7fce8c11fc60 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c136a20 con 0x7fce8c134d30 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce917c7640 10 monclient(hunting): authenticate will time out at 1990.360364s 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56274/0 (socket says 172.21.15.134:56274) 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 -- 172.21.15.134:0/1343994848 learned_addr learned my addr 172.21.15.134:0/1343994848 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): get_auth_request con 0x7fce78003630 auth_method 0 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c134d30 0x7fce8c135110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): get_auth_request con 0x7fce8c134d30 auth_method 0 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce83fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient(hunting): handle_auth_done global_id 124398 payload 824 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c11fc60 0x7fce8c1200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c134d30 msgr2=0x7fce8c135110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fce8c134d30 0x7fce8c135110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c11fc60 msgr2=0x7fce8c1200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fce8c11fc60 0x7fce8c1200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 monclient: found mon.noname-a 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.771+0000 7fce8affd640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c121e50 con 0x7fce78003630 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.773+0000 7fce8affd640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 msgr2=0x7fce8c0a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.773+0000 7fce8affd640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009280 tx=0x7fce7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7c02a700 con 0x7fce78003630 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: dump: 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.788 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c032e70 con 0x7fce78003630 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 5 monclient: authenticate success, global_id 124398 2024-09-16T15:20:53.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 10 monclient: _renew_subs 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c135650 con 0x7fce78003630 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce7c003c90 con 0x7fce78003630 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 10 monclient: dump: 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.790 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 10 monclient: _renew_subs 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce89ffb640 4 set_mon_vals no callback set 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c120620 con 0x7fce78003630 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.774+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fce7c031040 con 0x7fce78003630 2024-09-16T15:20:53.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce837fe640 1 --2- 172.21.15.134:0/1343994848 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce917c7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fce7c06b200 con 0x7fce78003630 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce917c7640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 0x7fce8c1bd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1364f0 con 0x7fce8c1208d0 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce83fff640 1 --2- 172.21.15.134:0/1343994848 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 0x7fce8c1bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce83fff640 10 monclient: get_auth_request con 0x7fce6c038880 auth_method 0 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient: get_auth_request con 0x7fce8c1208d0 auth_method 0 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_done global_id 124354 payload 824 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.775+0000 7fce8b7fe640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce8c061ae0 con 0x7fce8c1208d0 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.777+0000 7fce83fff640 1 --2- 172.21.15.134:0/1343994848 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fce74009c00 tx=0x7fce74009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce8b7fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 msgr2=0x7fce8c1bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 0x7fce8c1bd060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06b430 tx=0x7fce840115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce84007e80 con 0x7fce8c1208d0 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 10 monclient: dump: 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.795 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fce84011ec0 con 0x7fce8c1208d0 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 msgr2=0x7fce8c1bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 0x7fce8c1bd060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06b430 tx=0x7fce840115a0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 msgr2=0x7fce8c1bd060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.778+0000 7fce837fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1208d0 0x7fce8c1bd060 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce8c06b430 tx=0x7fce840115a0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.784+0000 7fce917c7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.784+0000 7fce917c7640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 0x7fce8c1bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.784+0000 7fce917c7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.784+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c071b20 con 0x7fce8c1bd5a0 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 0x7fce8c1bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient: get_auth_request con 0x7fce8c1bd5a0 auth_method 0 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 10 monclient(hunting): handle_auth_done global_id 124363 payload 824 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.785+0000 7fce8b7fe640 1 -- 172.21.15.134:0/1343994848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce8c07a680 con 0x7fce8c1bd5a0 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.787+0000 7fce8b7fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 msgr2=0x7fce8c1bd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.787+0000 7fce8b7fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 0x7fce8c1bd9b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f80 tx=0x7fce84004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fce8402c430 con 0x7fce8c1bd5a0 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 10 monclient: dump: 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.798 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7fce84026070 con 0x7fce8c1bd5a0 2024-09-16T15:20:53.799 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 msgr2=0x7fce8c1bd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 0x7fce8c1bd9b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f80 tx=0x7fce84004ab0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 msgr2=0x7fce8c1bd9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce837fe640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce8c1bd5a0 0x7fce8c1bd9b0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f80 tx=0x7fce84004ab0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:53.801 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:53.802 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:53.803 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.804 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:53.805 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.806 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.807 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.808 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.809 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.810 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.811 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:53.812 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.813 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.814 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.815 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.816 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.817 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:53.818 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.819 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:20:53.820 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.821 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:53.821 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 msgr2=0x7fce6c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 --2- 172.21.15.134:0/1343994848 >> v2:172.21.15.162:6800/980547263 conn(0x7fce6c038880 0x7fce6c03ad40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fce74009c00 tx=0x7fce74009190 comp rx=0 tx=0).stop 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 10 monclient: shutdown 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 msgr2=0x7fce8c0a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce8b7fe640 1 -- 172.21.15.134:0/1343994848 reap_dead start 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009280 tx=0x7fce7c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 shutdown_connections 2024-09-16T15:20:53.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 --2- 172.21.15.134:0/1343994848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce78003630 0x7fce8c0a28c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 >> 172.21.15.134:0/1343994848 conn(0x7fce8c0a6110 msgr2=0x7fce8c0a8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 shutdown_connections 2024-09-16T15:20:53.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.788+0000 7fce917c7640 1 -- 172.21.15.134:0/1343994848 wait complete. 2024-09-16T15:20:53.823 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.823 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-16T15:20:53.823 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.919+0000 7f069b0c9640 10 monclient: get_monmap_and_config 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.919+0000 7f069b0c9640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.919+0000 7f069b0c9640 10 monclient: monmap: 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:53.920712+0000 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:53.920712+0000 2024-09-16T15:20:53.921 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:53.922 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 1 Processor -- start 2024-09-16T15:20:53.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 1 -- start start 2024-09-16T15:20:53.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 10 monclient: init 2024-09-16T15:20:53.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.924 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 10 monclient: _add_conns ranks=[9,3,1,6,2,10,8,7,4,5,0] 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0694133f10 0x7f06941342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.920+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-a con 0x7f0694133f10 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-g con 0x7f0694134830 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f0694071bd0 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0694073250 con 0x7f0694071bd0 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940a3d50 con 0x7f0694133f10 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069406f630 con 0x7f0694134830 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f069b0c9640 10 monclient(hunting): authenticate will time out at 1990.510049s 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0694133f10 0x7f06941342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0694133f10 0x7f06941342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56276/0 (socket says 172.21.15.134:56276) 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 -- 172.21.15.134:0/2811043467 learned_addr learned my addr 172.21.15.134:0/2811043467 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): get_auth_request con 0x7f0694133f10 auth_method 0 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.926 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): get_auth_request con 0x7f0694134830 auth_method 0 2024-09-16T15:20:53.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient(hunting): handle_auth_done global_id 124407 payload 274 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 -- 172.21.15.134:0/2811043467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 msgr2=0x7f0694134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 -- 172.21.15.134:0/2811043467 >> v1:172.21.15.162:6789/0 conn(0x7f0694071bd0 legacy=0x7f06941351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 monclient: found mon.noname-a 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:53.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.921+0000 7f0698e65640 1 -- 172.21.15.134:0/2811043467 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940a1e10 con 0x7f0694133f10 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f0698e65640 1 -- 172.21.15.134:0/2811043467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694133f10 msgr2=0x7f06941342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f0698e65640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694133f10 0x7f06941342f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f068802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c248b3a95e2e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 1 -- 172.21.15.134:0/2811043467 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f068802ae20 con 0x7f0694133f10 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: dump: 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.935 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 1 -- 172.21.15.134:0/2811043467 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06880307b0 con 0x7f0694133f10 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 5 monclient: authenticate success, global_id 124407 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:53.936 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 1 -- 172.21.15.134:0/2811043467 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0688030d90 con 0x7f0694133f10 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 4 set_mon_vals no callback set 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f06937fe640 10 monclient: dump: 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.937 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 10 monclient: shutdown 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 -- 172.21.15.134:0/2811043467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694133f10 msgr2=0x7f06941342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694133f10 0x7f06941342f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f068802a100 comp rx=0 tx=0).stop 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 -- 172.21.15.134:0/2811043467 shutdown_connections 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 --2- 172.21.15.134:0/2811043467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694133f10 0x7f06941342f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 -- 172.21.15.134:0/2811043467 >> 172.21.15.134:0/2811043467 conn(0x7f06940a7410 msgr2=0x7f06940a77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.938 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 -- 172.21.15.134:0/2811043467 shutdown_connections 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.924+0000 7f069b0c9640 1 -- 172.21.15.134:0/2811043467 wait complete. 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.925+0000 7f069b0c9640 10 monclient: build_initial_monmap 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.925+0000 7f069b0c9640 10 monclient: monmap: 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:53.939 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.925+0000 7f069b0c9640 1 Processor -- start 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.925+0000 7f069b0c9640 1 -- start start 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.925+0000 7f069b0c9640 10 monclient: init 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient: _add_conns ranks=[1,8,3,0,5,7,6,2,4] 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0694071bd0 0x7f0694121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-f con 0x7f0694071bd0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 0x7f06941215b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-e con 0x7f0694133f10 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694121af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): picked mon.noname-g con 0x7f0694134830 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:53.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0694122b60 con 0x7f0694071bd0 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0694124a00 con 0x7f0694134830 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f069409e1b0 con 0x7f0694133f10 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f069b0c9640 10 monclient(hunting): authenticate will time out at 1990.515535s 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 0x7f06941215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 0x7f06941215b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52508/0 (socket says 172.21.15.134:52508) 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 -- 172.21.15.134:0/542665734 learned_addr learned my addr 172.21.15.134:0/542665734 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): get_auth_request con 0x7f0694133f10 auth_method 0 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.941 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0699666640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694121af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient(hunting): handle_auth_done global_id 85139 payload 824 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0698e65640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0694071bd0 0x7f0694121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 msgr2=0x7f0694121af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694121af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0694071bd0 msgr2=0x7f0694121070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0694071bd0 0x7f0694121070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient: _finish_hunting 0 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 monclient: found mon.noname-e 2024-09-16T15:20:53.942 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0693fff640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880094d0 con 0x7f0694133f10 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0699666640 10 monclient: get_auth_request con 0x7f0694134830 auth_method 0 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.926+0000 7f0699666640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694134830 0x7f0694121af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f0693fff640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 msgr2=0x7f06941215b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f0693fff640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 0x7f06941215b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f068000e870 tx=0x7f068000ed30 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06800094f0 con 0x7f0694133f10 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: dump: 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.943 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: _finish_auth 0 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: _check_auth_tickets 2024-09-16T15:20:53.944 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680013470 con 0x7f0694133f10 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 5 monclient: authenticate success, global_id 85139 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 10 monclient: _renew_subs 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0694123950 con 0x7f0694133f10 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0680013a50 con 0x7f0694133f10 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f06917fa640 10 monclient: dump: 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.945 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f0692ffd640 4 set_mon_vals no callback set 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 10 monclient: _renew_subs 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:53.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.929+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0694124f30 con 0x7f0694133f10 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f0680013bf0 con 0x7f0694133f10 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f069b0c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f069b0c9640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 0x7f069411da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0694073880 con 0x7f0694122030 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f06917fa640 1 --2- 172.21.15.134:0/542665734 >> v2:172.21.15.162:6800/980547263 conn(0x7f0678038880 0x7f067803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f068004a2d0 con 0x7f0694133f10 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 0x7f069411da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient: get_auth_request con 0x7f0694122030 auth_method 0 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0699666640 1 --2- 172.21.15.134:0/542665734 >> v2:172.21.15.162:6800/980547263 conn(0x7f0678038880 0x7f067803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0699666640 10 monclient: get_auth_request con 0x7f0678038880 auth_method 0 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 10 monclient(hunting): handle_auth_done global_id 84936 payload 824 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.930+0000 7f0698e65640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0694061ae0 con 0x7f0694122030 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f0699666640 1 --2- 172.21.15.134:0/542665734 >> v2:172.21.15.162:6800/980547263 conn(0x7f0678038880 0x7f067803ad40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0694072080 tx=0x7f068c009040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f0698e65640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 msgr2=0x7f069411da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f0698e65640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 0x7f069411da60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0688034000 tx=0x7f06880046c0 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06880044a0 con 0x7f0694122030 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 10 monclient: dump: 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.948 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.949 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0688030d40 con 0x7f0694122030 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 msgr2=0x7f069411da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 0x7f069411da60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0688034000 tx=0x7f06880046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 msgr2=0x7f069411da60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.933+0000 7f06917fa640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0694122030 0x7f069411da60 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0688034000 tx=0x7f06880046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f069b0c9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:53.951 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f069b0c9640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 0x7f069411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f069b0c9640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069406a3f0 con 0x7f069411dfa0 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 0x7f069411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient: get_auth_request con 0x7f069411dfa0 auth_method 0 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 10 monclient(hunting): handle_auth_done global_id 84945 payload 824 2024-09-16T15:20:53.952 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.940+0000 7f0698e65640 1 -- 172.21.15.134:0/542665734 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f069407a680 con 0x7f069411dfa0 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.942+0000 7f0698e65640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 msgr2=0x7f069411e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.942+0000 7f0698e65640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 0x7f069411e3b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f0688009240 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.3 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0688032cd0 con 0x7f069411dfa0 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 10 monclient: dump: 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:53.953 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 <== mon.3 v2:172.21.15.162:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f0688032e30 con 0x7f069411dfa0 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 msgr2=0x7f069411e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 0x7f069411e3b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f0688009240 comp rx=0 tx=0).stop 2024-09-16T15:20:53.954 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 msgr2=0x7f069411e3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f06917fa640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f069411dfa0 0x7f069411e3b0 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f0688009240 comp rx=0 tx=0).stop 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:53.955 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.956 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.957 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.958 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:53.959 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.960 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:53.961 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.962 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:53.963 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.964 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.965 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.967 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.968 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.969 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.970 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.971 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.972 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.973 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:53.974 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:53.975 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 >> v2:172.21.15.162:6800/980547263 conn(0x7f0678038880 msgr2=0x7f067803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 --2- 172.21.15.134:0/542665734 >> v2:172.21.15.162:6800/980547263 conn(0x7f0678038880 0x7f067803ad40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0694072080 tx=0x7f068c009040 comp rx=0 tx=0).stop 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 10 monclient: shutdown 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 msgr2=0x7f06941215b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 --2- 172.21.15.134:0/542665734 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f0694133f10 0x7f06941215b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f068000e870 tx=0x7f068000ed30 comp rx=0 tx=0).stop 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f0699666640 1 -- 172.21.15.134:0/542665734 reap_dead start 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f0699666640 1 -- 172.21.15.134:0/542665734 reap_dead start 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 shutdown_connections 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.943+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 >> 172.21.15.134:0/542665734 conn(0x7f06940a7410 msgr2=0x7f06940a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.944+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 shutdown_connections 2024-09-16T15:20:53.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:53.944+0000 7f069b0c9640 1 -- 172.21.15.134:0/542665734 wait complete. 2024-09-16T15:20:53.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:53.978 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-16T15:20:53.978 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-16T15:20:54.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.074+0000 7fbffcd26640 10 monclient: get_monmap_and_config 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.074+0000 7fbffcd26640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.074+0000 7fbffcd26640 10 monclient: monmap: 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:54.075527+0000 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:54.075527+0000 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.076 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:54.077 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 Processor -- start 2024-09-16T15:20:54.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 -- start start 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient: init 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient: _add_conns ranks=[7,10,8,1,3,6,9,4,0,2,5] 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbff8136e40 0x7fbff809c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-i con 0x7fbff8136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 0x7fbff8137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-f con 0x7fbff8137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-e con 0x7fbff8135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff809abc0 con 0x7fbff8137220 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbff809ba50 con 0x7fbff8136e40 2024-09-16T15:20:54.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80a53c0 con 0x7fbff8135dc0 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbffcd26640 10 monclient(hunting): authenticate will time out at 1990.664833s 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.075+0000 7fbff6d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52520/0 (socket says 172.21.15.134:52520) 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1275941054 learned_addr learned my addr 172.21.15.134:0/1275941054 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request con 0x7fbff8135dc0 auth_method 0 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 1 --2- 172.21.15.134:0/1275941054 >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 0x7fbff8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff659c640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbff8136e40 0x7fbff809c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 10 monclient(hunting): get_auth_request con 0x7fbff8137220 auth_method 0 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff659c640 10 monclient(hunting): get_auth_request con 0x7fbff8136e40 auth_method 0 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_done global_id 85148 payload 274 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1275941054 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbff8136e40 msgr2=0x7fbff809c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbff8136e40 0x7fbff809c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1275941054 >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 msgr2=0x7fbff8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1275941054 >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 0x7fbff8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.082 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 monclient: found mon.noname-e 2024-09-16T15:20:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1275941054 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8073250 con 0x7fbff8135dc0 2024-09-16T15:20:54.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.076+0000 7fbfeffff640 1 --2- 172.21.15.134:0/1275941054 >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 0x7fbff8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.079+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1275941054 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 msgr2=0x7fbff8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.079+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e80 tx=0x7fbff002a580 comp rx=0 tx=0).ready entity=mon.8 client_cookie=efabdd863a0d62b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 1 -- 172.21.15.134:0/1275941054 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff0007a70 con 0x7fbff8135dc0 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: dump: 2024-09-16T15:20:54.087 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.088 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbffcd26640 5 monclient: authenticate success, global_id 85148 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbffcd26640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 1 -- 172.21.15.134:0/1275941054 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff002d050 con 0x7fbff8135dc0 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbffcd26640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbffcd26640 4 set_mon_vals no callback set 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 1 -- 172.21.15.134:0/1275941054 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff0007c30 con 0x7fbff8135dc0 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbff5d9b640 10 monclient: dump: 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.089 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.081+0000 7fbffcd26640 10 monclient: shutdown 2024-09-16T15:20:54.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 -- 172.21.15.134:0/1275941054 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 msgr2=0x7fbff8136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009e80 tx=0x7fbff002a580 comp rx=0 tx=0).stop 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 -- 172.21.15.134:0/1275941054 shutdown_connections 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8135dc0 0x7fbff8136220 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1275941054 >> v2:172.21.15.162:3300/0 conn(0x7fbff8137220 0x7fbff8137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1275941054 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fbff8136e40 0x7fbff809c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 -- 172.21.15.134:0/1275941054 >> 172.21.15.134:0/1275941054 conn(0x7fbff80a6110 msgr2=0x7fbff80a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 -- 172.21.15.134:0/1275941054 shutdown_connections 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 1 -- 172.21.15.134:0/1275941054 wait complete. 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.082+0000 7fbffcd26640 10 monclient: monmap: 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 Processor -- start 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 -- start start 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient: init 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient: _add_conns ranks=[6,8,1,2,7,3,4,5,0] 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-d con 0x7fbff8135dc0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 0x7fbff81352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-e con 0x7fbff8136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff8137220 0x7fbff8121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): picked mon.noname-f con 0x7fbff8137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff81368a0 con 0x7fbff8137220 2024-09-16T15:20:54.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8123fd0 con 0x7fbff8135dc0 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8123910 con 0x7fbff8136e40 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbffcd26640 10 monclient(hunting): authenticate will time out at 1990.672703s 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 0x7fbff81352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 0x7fbff81352b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52530/0 (socket says 172.21.15.134:52530) 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:51450/0 (socket says 172.21.15.134:51450) 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 1 -- 172.21.15.134:0/1976335718 learned_addr learned my addr 172.21.15.134:0/1976335718 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): get_auth_request con 0x7fbff8136e40 auth_method 0 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): get_auth_request con 0x7fbff8135dc0 auth_method 0 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbfeffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff659c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff8137220 0x7fbff8121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request con 0x7fbff8137220 auth_method 0 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.083+0000 7fbff6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 10 monclient(hunting): handle_auth_done global_id 94866 payload 824 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 msgr2=0x7fbff81352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 0x7fbff81352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff8137220 msgr2=0x7fbff8121730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff8137220 0x7fbff8121730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 monclient: found mon.noname-d 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbff659c640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009b30 con 0x7fbff8135dc0 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.084+0000 7fbfeffff640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fbff8136e40 0x7fbff81352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:54.096 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbff659c640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 msgr2=0x7fbff8134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.097 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbff659c640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbff8123ae0 tx=0x7fbfe800da80 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfe8012c70 con 0x7fbff8135dc0 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: dump: 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe8012e10 con 0x7fbff8135dc0 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbfe80148a0 con 0x7fbff8135dc0 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:54.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbfeeffd640 10 monclient: dump: 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.103 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.086+0000 7fbff559a640 4 set_mon_vals no callback set 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 5 monclient: authenticate success, global_id 94866 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 10 monclient: _renew_subs 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff80a8a50 con 0x7fbff8135dc0 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 10 monclient: _renew_subs 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff8125480 con 0x7fbff8135dc0 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbff659c640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbff8123ae0 tx=0x7fbfe800da80 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fbfe804a520 osd_map(861..861 src has 1..861) v4 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.087+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbfe8014430 con 0x7fbff8135dc0 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbfeeffd640 1 --2- 172.21.15.134:0/1976335718 >> v2:172.21.15.162:6800/980547263 conn(0x7fbfd8038880 0x7fbfd803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbffcd26640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 0x7fbff80a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8071b20 con 0x7fbff80a1a20 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 0x7fbff80a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient: get_auth_request con 0x7fbff80a1a20 auth_method 0 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbfeffff640 1 --2- 172.21.15.134:0/1976335718 >> v2:172.21.15.162:6800/980547263 conn(0x7fbfd8038880 0x7fbfd803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbfeffff640 10 monclient: get_auth_request con 0x7fbfd8038880 auth_method 0 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_done global_id 45527 payload 824 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.088+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbff8061ae0 con 0x7fbff80a1a20 2024-09-16T15:20:54.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.090+0000 7fbfeffff640 1 --2- 172.21.15.134:0/1976335718 >> v2:172.21.15.162:6800/980547263 conn(0x7fbfd8038880 0x7fbfd803ad40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006d40 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.090+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 msgr2=0x7fbff80a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.090+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 0x7fbff80a1e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbff806cb00 tx=0x7fbff0004a40 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff00046e0 con 0x7fbff80a1a20 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 10 monclient: dump: 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.106 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbff0004840 con 0x7fbff80a1a20 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 msgr2=0x7fbff80a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 0x7fbff80a1e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbff806cb00 tx=0x7fbff0004a40 comp rx=0 tx=0).stop 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 msgr2=0x7fbff80a1e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.091+0000 7fbfeeffd640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a1a20 0x7fbff80a1e00 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbff806cb00 tx=0x7fbff0004a40 comp rx=0 tx=0).stop 2024-09-16T15:20:54.107 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbffcd26640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 0x7fbff8121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbffcd26640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbff807a680 con 0x7fbff80a2340 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 0x7fbff8121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient: get_auth_request con 0x7fbff80a2340 auth_method 0 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.097+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.098+0000 7fbff6d9d640 10 monclient(hunting): handle_auth_done global_id 45536 payload 824 2024-09-16T15:20:54.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.098+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbff809b490 con 0x7fbff80a2340 2024-09-16T15:20:54.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 msgr2=0x7fbff8121c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbff6d9d640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 0x7fbff8121c70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbff0008000 tx=0x7fbff002f000 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbff0049580 con 0x7fbff80a2340 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 10 monclient: dump: 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.110 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.5 v2:172.21.15.162:3302/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9362 (secure 0 0 0) 0x7fbff0007b90 con 0x7fbff80a2340 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.100+0000 7fbfeeffd640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 msgr2=0x7fbff8121c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbfeeffd640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 0x7fbff8121c70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbff0008000 tx=0x7fbff002f000 comp rx=0 tx=0).stop 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 msgr2=0x7fbff8121c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbfeeffd640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fbff80a2340 0x7fbff8121c70 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbff0008000 tx=0x7fbff002f000 comp rx=0 tx=0).stop 2024-09-16T15:20:54.111 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:54.112 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.113 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.114 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:54.115 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:54.116 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.117 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:54.118 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.119 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.120 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.121 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.123 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.124 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.125 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.126 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.127 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.128 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.129 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.130 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "osd": [ 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 2 2024-09-16T15:20:54.131 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:54.132 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:54.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 >> v2:172.21.15.162:6800/980547263 conn(0x7fbfd8038880 msgr2=0x7fbfd803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1976335718 >> v2:172.21.15.162:6800/980547263 conn(0x7fbfd8038880 0x7fbfd803ad40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006d40 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 10 monclient: shutdown 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 msgr2=0x7fbff8134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 --2- 172.21.15.134:0/1976335718 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7fbff8135dc0 0x7fbff8134d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbff8123ae0 tx=0x7fbfe800da80 comp rx=0 tx=0).stop 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 reap_dead start 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbff6d9d640 1 -- 172.21.15.134:0/1976335718 reap_dead start 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbfeeffd640 1 -- 172.21.15.134:0/1976335718 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (unknown 0 0 0) 0x7fbfe804a520 con 0x7fbff8135dc0 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 shutdown_connections 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 >> 172.21.15.134:0/1976335718 conn(0x7fbff80a6110 msgr2=0x7fbff806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 shutdown_connections 2024-09-16T15:20:54.134 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.101+0000 7fbffcd26640 1 -- 172.21.15.134:0/1976335718 wait complete. 2024-09-16T15:20:54.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:54.135 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-16T15:20:54.135 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-16T15:20:54.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.232+0000 7f9f7bb5a640 10 monclient: get_monmap_and_config 2024-09-16T15:20:54.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.232+0000 7f9f7bb5a640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.233 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.232+0000 7f9f7bb5a640 10 monclient: monmap: 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:54.233239+0000 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:54.233239+0000 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.234 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.235 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.235 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.235 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:54.235 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:54.235 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.232+0000 7f9f7bb5a640 1 Processor -- start 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 -- start start 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient: init 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient: _add_conns ranks=[4,5,9,10,3,1,7,2,0,8,6] 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-c con 0x7f9f74136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f74137220 0x7f9f74137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-h con 0x7f9f74137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9f74135dc0 0x7f9f74136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-a con 0x7f9f74135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7409abc0 con 0x7f9f74135dc0 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7409ba50 con 0x7f9f74136e40 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740a53c0 con 0x7f9f74137220 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7bb5a640 10 monclient(hunting): authenticate will time out at 1990.822601s 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9f74135dc0 0x7f9f74136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.238 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38984/0 (socket says 172.21.15.134:38984) 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 1 -- 172.21.15.134:0/3785641570 learned_addr learned my addr 172.21.15.134:0/3785641570 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): get_auth_request con 0x7f9f74135dc0 auth_method 0 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): get_auth_request con 0x7f9f74136e40 auth_method 0 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f790f5640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f74137220 0x7f9f74137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f790f5640 10 monclient(hunting): get_auth_request con 0x7f9f74137220 auth_method 0 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f790f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f790f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f790f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_done global_id 124416 payload 274 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/3785641570 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f74137220 msgr2=0x7f9f74137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f74137220 0x7f9f74137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 msgr2=0x7f9f7409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.233+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f7a0f7640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f7a0f7640 1 monclient: found mon.noname-a 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f7a0f7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f7a0f7640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/3785641570 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74073250 con 0x7f9f74135dc0 2024-09-16T15:20:54.241 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.234+0000 7f9f798f6640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74135dc0 msgr2=0x7f9f74136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74135dc0 0x7f9f74136220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009d90 tx=0x7f9f6402a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a79075ff5c49b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 1 -- 172.21.15.134:0/3785641570 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f64007bc0 con 0x7f9f74135dc0 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: dump: 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.248 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 1 -- 172.21.15.134:0/3785641570 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6402d050 con 0x7f9f74135dc0 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7bb5a640 5 monclient: authenticate success, global_id 124416 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7bb5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7bb5a640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 1 -- 172.21.15.134:0/3785641570 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f64007d40 con 0x7f9f74135dc0 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f7bb5a640 4 set_mon_vals no callback set 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.236+0000 7f9f788f4640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f788f4640 10 monclient: dump: 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.250 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 10 monclient: shutdown 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74135dc0 msgr2=0x7f9f74136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74135dc0 0x7f9f74136220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009d90 tx=0x7f9f6402a490 comp rx=0 tx=0).stop 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/3785641570 shutdown_connections 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74135dc0 0x7f9f74136220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9f74137220 0x7f9f74137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/3785641570 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f9f74136e40 0x7f9f7409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/3785641570 >> 172.21.15.134:0/3785641570 conn(0x7f9f740a6110 msgr2=0x7f9f740a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/3785641570 shutdown_connections 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.237+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/3785641570 wait complete. 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient: monmap: 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.252 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 Processor -- start 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 -- start start 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient: init 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient: _add_conns ranks=[8,0,7,4,2,5,1,3,6] 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9f74135dc0 0x7f9f74134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-e con 0x7f9f74135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-a con 0x7f9f74136e40 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f74137220 0x7f9f74121970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): picked mon.noname-i con 0x7f9f74137220 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740a8a50 con 0x7f9f74136e40 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9f741368a0 con 0x7f9f74137220 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f9f741cc0d0 con 0x7f9f74135dc0 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.238+0000 7f9f7bb5a640 10 monclient(hunting): authenticate will time out at 1990.827968s 2024-09-16T15:20:54.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9f74135dc0 0x7f9f74134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56306/0 (socket says 172.21.15.134:56306) 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 -- 172.21.15.134:0/1987548042 learned_addr learned my addr 172.21.15.134:0/1987548042 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): get_auth_request con 0x7f9f74135dc0 auth_method 0 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): get_auth_request con 0x7f9f74136e40 auth_method 0 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f74137220 0x7f9f74121970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): get_auth_request con 0x7f9f74137220 auth_method 0 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f7a0f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient(hunting): handle_auth_done global_id 124425 payload 824 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9f74135dc0 msgr2=0x7f9f74134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9f74135dc0 0x7f9f74134ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f74137220 msgr2=0x7f9f74121970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f74137220 0x7f9f74121970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 monclient: found mon.noname-a 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f790f5640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0094d0 con 0x7f9f74136e40 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.239+0000 7f9f798f6640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f9f74135dc0 0x7f9f74134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f790f5640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 msgr2=0x7f9f74135420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f790f5640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800d620 tx=0x7f9f6800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f6800bca0 con 0x7f9f74136e40 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: dump: 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.257 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6800be40 con 0x7f9f74136e40 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 5 monclient: authenticate success, global_id 124425 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 10 monclient: _renew_subs 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f64009a70 con 0x7f9f74136e40 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f680148a0 con 0x7f9f74136e40 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f727fc640 10 monclient: dump: 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.259 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 10 monclient: _renew_subs 2024-09-16T15:20:54.260 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f73fff640 4 set_mon_vals no callback set 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.241+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f741258d0 con 0x7f9f74136e40 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f7bb5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 0x7f9f7402b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74071b20 con 0x7f9f741cb420 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f790f5640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800d620 tx=0x7f9f6800dae0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f9f68014430 mgrmap(e 5) v1 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9f68014430 con 0x7f9f74136e40 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 0x7f9f7402b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient: get_auth_request con 0x7f9f741cb420 auth_method 0 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.261 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f727fc640 1 --2- 172.21.15.134:0/1987548042 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f60038880 0x7f9f6003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.242+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.243+0000 7f9f798f6640 10 monclient(hunting): handle_auth_done global_id 134134 payload 824 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.243+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/1987548042 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f60038880 0x7f9f6003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.243+0000 7f9f798f6640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f74061ae0 con 0x7f9f741cb420 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.243+0000 7f9f7a0f7640 10 monclient: get_auth_request con 0x7f9f60038880 auth_method 0 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f798f6640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 msgr2=0x7f9f7402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f798f6640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 0x7f9f7402b260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c00f610 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f6c00fe50 con 0x7f9f741cb420 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 10 monclient: dump: 2024-09-16T15:20:54.262 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.263 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f7a0f7640 1 --2- 172.21.15.134:0/1987548042 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f60038880 0x7f9f6003ad40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009d90 tx=0x7f9f6402f000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9f6c014860 con 0x7f9f741cb420 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 msgr2=0x7f9f7402b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 0x7f9f7402b260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c00f610 comp rx=0 tx=0).stop 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 msgr2=0x7f9f7402b260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:54.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.245+0000 7f9f727fc640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f741cb420 0x7f9f7402b260 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c00f610 comp rx=0 tx=0).stop 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.251+0000 7f9f7bb5a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.251+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 0x7f9f7402bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.251+0000 7f9f7bb5a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.251+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7407a680 con 0x7f9f7402b7a0 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 0x7f9f7402bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient: get_auth_request con 0x7f9f7402b7a0 auth_method 0 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 10 monclient(hunting): handle_auth_done global_id 134143 payload 824 2024-09-16T15:20:54.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.252+0000 7f9f798f6640 1 -- 172.21.15.134:0/1987548042 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f7409b490 con 0x7f9f7402b7a0 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f798f6640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 msgr2=0x7f9f7402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f798f6640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 0x7f9f7402bbb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c0046c0 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9f6c004490 con 0x7f9f7402b7a0 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f727fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f727fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.254+0000 7f9f727fc640 10 monclient: dump: 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.267 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.7 v2:172.21.15.162:3303/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+9196 (secure 0 0 0) 0x7f9f6c026070 con 0x7f9f7402b7a0 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 msgr2=0x7f9f7402bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 0x7f9f7402bbb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 msgr2=0x7f9f7402bbb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9f7402b7a0 0x7f9f7402bbb0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011e70 tx=0x7f9f6c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: "state": "peon", 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: "election_epoch": 166, 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum": [ 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: 0, 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: 1, 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: 2, 2024-09-16T15:20:54.269 INFO:teuthology.orchestra.run.smithi134.stdout: 3, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: 4, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: 5, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: 6, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: 7, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: 8 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_age": 4, 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "required_con": "2449958758054445060", 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "required_mon": [ 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.270 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_con": "4540138322906710015", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "quorum_mon": [ 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.271 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "outside_quorum": [], 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "extra_probe_peers": [], 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "sync_provider": [], 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "monmap": { 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "epoch": 1, 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "fsid": "9b832ad6-88ab-4e38-a956-73c2de3b6492", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "modified": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "created": "2024-09-16T15:00:04.519049Z", 2024-09-16T15:20:54.272 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release": 18, 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "min_mon_release_name": "reef", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "election_strategy": 1, 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "disallowed_leaders: ": "", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false, 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "tiebreaker_mon": "", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "removed_ranks: ": "", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "features": { 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "persistent": [ 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "kraken", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "luminous", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "mimic", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap-prune", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "nautilus", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "octopus", 2024-09-16T15:20:54.273 INFO:teuthology.orchestra.run.smithi134.stdout: "pacific", 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "elector-pinging", 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "quincy", 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "reef" 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "optional": [] 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "mons": [ 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 0, 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "a", 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.274 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3300", 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789", 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6789/0", 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6789/0", 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.275 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 1, 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "f", 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3300", 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789", 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.276 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6789/0", 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6789/0", 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 2, 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "b", 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.277 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3301", 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790", 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6790/0", 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6790/0", 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.278 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 3, 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "g", 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3301", 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790", 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.279 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6790/0", 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6790/0", 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 4, 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "c", 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.280 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3302", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6791/0", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6791/0", 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.281 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 5, 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "h", 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3302", 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791", 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.282 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6791/0", 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6791/0", 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 6, 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "d", 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.283 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3303", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6792/0", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6792/0", 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.284 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 7, 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "i", 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:3303", 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.285 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792", 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.162:6792/0", 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.162:6792/0", 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "rank": 8, 2024-09-16T15:20:54.286 INFO:teuthology.orchestra.run.smithi134.stdout: "name": "e", 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addrs": { 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addrvec": [ 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v2", 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:3304", 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "type": "v1", 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793", 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "nonce": 0 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.287 INFO:teuthology.orchestra.run.smithi134.stdout: "addr": "172.21.15.134:6793/0", 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "public_addr": "172.21.15.134:6793/0", 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "priority": 0, 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "weight": 0, 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "crush_location": "{}" 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "feature_map": { 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "mon": [ 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:54.288 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: ], 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "client": [ 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: { 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "features": "0x3f01cfbffffdffff", 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "release": "reef", 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "num": 1 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout: "stretch_mode": false 2024-09-16T15:20:54.289 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:54.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f60038880 msgr2=0x7f9f6003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/1987548042 >> v2:172.21.15.162:6800/980547263 conn(0x7f9f60038880 0x7f9f6003ad40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009d90 tx=0x7f9f6402f000 comp rx=0 tx=0).stop 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 10 monclient: shutdown 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 msgr2=0x7f9f74135420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 --2- 172.21.15.134:0/1987548042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f74136e40 0x7f9f74135420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800d620 tx=0x7f9f6800dae0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/1987548042 reap_dead start 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f727fc640 1 -- 172.21.15.134:0/1987548042 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (unknown 0 0 0) 0x7f9f6804a2d0 con 0x7f9f74136e40 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7a0f7640 1 -- 172.21.15.134:0/1987548042 reap_dead start 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 shutdown_connections 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 >> 172.21.15.134:0/1987548042 conn(0x7f9f740a6110 msgr2=0x7f9f7409f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 shutdown_connections 2024-09-16T15:20:54.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.255+0000 7f9f7bb5a640 1 -- 172.21.15.134:0/1987548042 wait complete. 2024-09-16T15:20:54.291 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-16T15:20:54.292 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-16T15:20:54.292 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-16T15:20:54.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.387+0000 7fbb9fee5640 10 monclient: get_monmap_and_config 2024-09-16T15:20:54.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.387+0000 7fbb9fee5640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.387+0000 7fbb9fee5640 10 monclient: monmap: 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:54.388157+0000 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:54.388157+0000 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.389 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.390 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.390 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:54.390 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:54.390 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.387+0000 7fbb9fee5640 1 Processor -- start 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 -- start start 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient: init 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient: _add_conns ranks=[4,2,1,8,6,7,0,10,5,9,3] 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-c con 0x7fbb980a1d30 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-b con 0x7fbb98070e90 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fbb980a2650 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb98073250 con 0x7fbb980a2650 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980a3880 con 0x7fbb98070e90 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98070540 con 0x7fbb980a1d30 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9fee5640 10 monclient(hunting): authenticate will time out at 1990.977535s 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:38998/0 (socket says 172.21.15.134:38998) 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 -- 172.21.15.134:0/1886657197 learned_addr learned my addr 172.21.15.134:0/1886657197 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): get_auth_request con 0x7fbb980a1d30 auth_method 0 2024-09-16T15:20:54.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45064/0 (socket says 172.21.15.134:45064) 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): get_auth_request con 0x7fbb98070e90 auth_method 0 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient(hunting): handle_auth_done global_id 124456 payload 274 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 -- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 msgr2=0x7fbb980a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 -- 172.21.15.134:0/1886657197 >> v1:172.21.15.162:6789/0 conn(0x7fbb980a2650 legacy=0x7fbb98134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 monclient: found mon.noname-c 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9dc81640 1 -- 172.21.15.134:0/1886657197 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980a3a80 con 0x7fbb980a1d30 2024-09-16T15:20:54.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.388+0000 7fbb9d480640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.391+0000 7fbb9dc81640 1 -- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 msgr2=0x7fbb9806f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.391+0000 7fbb9dc81640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0098a0 tx=0x7fbb8c02a150 comp rx=0 tx=0).ready entity=mon.4 client_cookie=ef01615d4b3cc4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 1 -- 172.21.15.134:0/1886657197 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb8c02ae70 con 0x7fbb980a1d30 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: dump: 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.401 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.402 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 1 -- 172.21.15.134:0/1886657197 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0307b0 con 0x7fbb980a1d30 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 5 monclient: authenticate success, global_id 124456 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 1 -- 172.21.15.134:0/1886657197 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb8c030d90 con 0x7fbb980a1d30 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 4 set_mon_vals no callback set 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9cc7f640 10 monclient: dump: 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.403 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 10 monclient: shutdown 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.404 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 -- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 msgr2=0x7fbb9806f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0098a0 tx=0x7fbb8c02a150 comp rx=0 tx=0).stop 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 -- 172.21.15.134:0/1886657197 shutdown_connections 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb98070e90 0x7fbb980a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/1886657197 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fbb980a1d30 0x7fbb9806f550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 -- 172.21.15.134:0/1886657197 >> 172.21.15.134:0/1886657197 conn(0x7fbb9809abf0 msgr2=0x7fbb980a7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 -- 172.21.15.134:0/1886657197 shutdown_connections 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.392+0000 7fbb9fee5640 1 -- 172.21.15.134:0/1886657197 wait complete. 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.393+0000 7fbb9fee5640 10 monclient: build_initial_monmap 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.393+0000 7fbb9fee5640 10 monclient: monmap: 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:54.405 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.393+0000 7fbb9fee5640 1 Processor -- start 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.393+0000 7fbb9fee5640 1 -- start start 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.393+0000 7fbb9fee5640 10 monclient: init 2024-09-16T15:20:54.406 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient: _add_conns ranks=[3,0,2,1,8,5,7,4,6] 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb98070e90 0x7fbb98134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-g con 0x7fbb98070e90 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 0x7fbb98135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-a con 0x7fbb980a1d30 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): picked mon.noname-b con 0x7fbb980a2650 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb981ca090 con 0x7fbb980a1d30 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9809db10 con 0x7fbb980a2650 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb980a6a70 con 0x7fbb98070e90 2024-09-16T15:20:54.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9fee5640 10 monclient(hunting): authenticate will time out at 1990.983080s 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 0x7fbb98135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:45078/0 (socket says 172.21.15.134:45078) 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 -- 172.21.15.134:0/977444277 learned_addr learned my addr 172.21.15.134:0/977444277 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): get_auth_request con 0x7fbb980a1d30 auth_method 0 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): get_auth_request con 0x7fbb980a2650 auth_method 0 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9dc81640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb98070e90 0x7fbb98134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9dc81640 10 monclient(hunting): get_auth_request con 0x7fbb98070e90 auth_method 0 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9dc81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9dc81640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9dc81640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient(hunting): handle_auth_done global_id 114734 payload 824 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 -- 172.21.15.134:0/977444277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb98070e90 msgr2=0x7fbb98134c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb98070e90 0x7fbb98134c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 -- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 msgr2=0x7fbb98135140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 0x7fbb98135140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient: _finish_hunting 0 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 monclient: found mon.noname-b 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9e482640 1 -- 172.21.15.134:0/977444277 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94009400 con 0x7fbb980a2650 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.394+0000 7fbb9d480640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 0x7fbb98135140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9e482640 1 -- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 msgr2=0x7fbb9811e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9e482640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b970 tx=0x7fbb9000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb90007350 con 0x7fbb980a2650 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: dump: 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.410 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 5 monclient: authenticate success, global_id 114734 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb900074d0 con 0x7fbb980a2650 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 10 monclient: _renew_subs 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbb90007af0 con 0x7fbb980a2650 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0094d0 con 0x7fbb980a2650 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8bfff640 4 set_mon_vals no callback set 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb8a7fc640 10 monclient: dump: 2024-09-16T15:20:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:54.413 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 10 monclient: _renew_subs 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.396+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb980a6fc0 con 0x7fbb980a2650 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fbb90007c50 con 0x7fbb980a2650 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb9fee5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb9fee5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb9fee5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98122060 con 0x7fbb980a2650 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb8a7fc640 1 --2- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 0x7fbb7c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.397+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fbb900499c0 con 0x7fbb980a2650 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.398+0000 7fbb9dc81640 1 --2- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 0x7fbb7c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.398+0000 7fbb9dc81640 10 monclient: get_auth_request con 0x7fbb7c038880 auth_method 0 2024-09-16T15:20:54.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.400+0000 7fbb9dc81640 1 --2- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 0x7fbb7c03ad40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:54.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.402+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fbb98122060 con 0x7fbb980a2650 2024-09-16T15:20:54.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.402+0000 7fbb8a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:54.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.402+0000 7fbb8a7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:54.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.628+0000 7fbb9fee5640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-16T15:20:54.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.628+0000 7fbb9fee5640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-16T15:20:54.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.628+0000 7fbb9fee5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:54.630 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.628+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fbb98122060 con 0x7fbb980a2650 2024-09-16T15:20:54.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.630+0000 7fbb8a7fc640 1 -- 172.21.15.134:0/977444277 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fbb98122060 con 0x7fbb980a2650 2024-09-16T15:20:54.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.630+0000 7fbb8a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-16T15:20:54.631 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.630+0000 7fbb8a7fc640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 msgr2=0x7fbb7c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 0x7fbb7c03ad40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).stop 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 10 monclient: shutdown 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 msgr2=0x7fbb9811e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b970 tx=0x7fbb9000be30 comp rx=0 tx=0).stop 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 shutdown_connections 2024-09-16T15:20:54.634 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb980a2650 0x7fbb9811e7f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb980a1d30 0x7fbb98135140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> v2:172.21.15.162:6800/980547263 conn(0x7fbb7c038880 0x7fbb7c03ad40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 --2- 172.21.15.134:0/977444277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb98070e90 0x7fbb98134c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 >> 172.21.15.134:0/977444277 conn(0x7fbb9809abf0 msgr2=0x7fbb9809c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 shutdown_connections 2024-09-16T15:20:54.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:54.632+0000 7fbb9fee5640 1 -- 172.21.15.134:0/977444277 wait complete. 2024-09-16T15:20:54.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-16T15:20:54.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=2,health=12,kv=12,logm=51} crc {auth=1484096765,config=2586702516,health=46948609,kv=1090497770,logm=3362807713}) 2024-09-16T15:20:54.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=11,kv=9,logm=55} crc {auth=647159844,config=2586702516,health=2957007899,kv=1585245423,logm=1772690279}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=22,config=1,health=12,kv=11,logm=54} crc {auth=2503657034,config=1228700967,health=46948609,kv=174400417,logm=1876538727}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=24,config=2,health=11,kv=12,logm=51} crc {auth=388238746,config=2586702516,health=2961427103,kv=1090497770,logm=3512395338}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=11,logm=52} crc {auth=3781707964,config=2586702516,health=1415039288,kv=3549148692,logm=1684152849}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=11,kv=10,logm=53} crc {auth=3781707964,config=2586702516,health=146333975,kv=2095472101,logm=1586499202}) 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=23,config=2,health=11,kv=12,logm=52} crc {auth=2118751076,config=2586702516,health=3945847643,kv=1090497770,logm=3381460025}) 2024-09-16T15:20:54.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=12,logm=50} crc {auth=3781707964,config=2586702516,health=46948609,kv=1090497770,logm=2188088553}) 2024-09-16T15:20:54.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=24,config=1,health=11,kv=12,logm=52} crc {auth=3781707964,config=1228700967,health=2961427103,kv=1090497770,logm=3449798273}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=885839063}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3503520262}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2587804351}) 2024-09-16T15:20:54.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1066709990}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1546033924}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3666126898}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4133791786}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2608447693}) 2024-09-16T15:20:54.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.690+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3381818929}) 2024-09-16T15:20:54.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=195047258}) 2024-09-16T15:20:54.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3715278745}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=160875481}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2087074983}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2076546667}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1030424234}) 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1990164148}) 2024-09-16T15:20:54.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:54.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3687568217}) 2024-09-16T15:20:54.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:54.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3232474025}) 2024-09-16T15:20:55.089 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1495112220}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3768459483}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3103296793}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=534425002}) 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.090 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3986483258}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3110435176}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4132701311}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=344708216}) 2024-09-16T15:20:55.091 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4056933394}) 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2115981718}) 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4148052063}) 2024-09-16T15:20:55.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2005464085}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1731223684}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2980517787}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3628303487}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=792846146}) 2024-09-16T15:20:55.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3926814012}) 2024-09-16T15:20:55.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.099+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3988528985}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3338553972}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=654844126}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1661446010}) 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2351589864}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=459100275}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=421101597}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=842477121}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087047916}) 2024-09-16T15:20:55.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1097107091}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3708854516}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2521390212}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3754332701}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1256824621}) 2024-09-16T15:20:55.115 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2501736120}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3829298930}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2839023889}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163843647}) 2024-09-16T15:20:55.116 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.112+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3166691237}) 2024-09-16T15:20:55.645 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-16T15:20:55.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-16T15:20:55.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435851043}) 2024-09-16T15:20:55.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2223674440}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1353401674}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2152671671}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=653146193}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2891942775}) 2024-09-16T15:20:55.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=532760538}) 2024-09-16T15:20:55.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277402937}) 2024-09-16T15:20:55.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3950260568}) 2024-09-16T15:20:55.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1932169673}) 2024-09-16T15:20:55.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2218546286}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2064445589}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2240992136}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2452323376}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1845326511}) 2024-09-16T15:20:55.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3627726670}) 2024-09-16T15:20:55.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3569010211}) 2024-09-16T15:20:55.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=97227125}) 2024-09-16T15:20:55.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.745+0000 7ffb315db640 10 monclient: get_monmap_and_config 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.745+0000 7ffb315db640 10 monclient: build_initial_monmap 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.745+0000 7ffb315db640 10 monclient: monmap: 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:55.746542+0000 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:55.746542+0000 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:55.747 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:55.748 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 Processor -- start 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 -- start start 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient: init 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient: _add_conns ranks=[7,10,9,8,2,3,5,0,6,1,4] 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb2c136030 0x7ffb2c136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-i con 0x7ffb2c136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 0x7ffb2c09c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-f con 0x7ffb2c09bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb2c136c60 0x7ffb2c137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-a con 0x7ffb2c136c60 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c09b600 con 0x7ffb2c136c60 2024-09-16T15:20:55.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c0a5740 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0a52c0 con 0x7ffb2c136030 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb315db640 10 monclient(hunting): authenticate will time out at 1992.335743s 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb2c136c60 0x7ffb2c137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb2c136c60 0x7ffb2c137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56342/0 (socket says 172.21.15.134:56342) 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 1 -- 172.21.15.134:0/870678399 learned_addr learned my addr 172.21.15.134:0/870678399 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb2c136c60 auth_method 0 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:55.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.746+0000 7ffb2b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2affd640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb2c136030 0x7ffb2c136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2affd640 10 monclient(hunting): get_auth_request con 0x7ffb2c136030 auth_method 0 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2a7fc640 1 --2- 172.21.15.134:0/870678399 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 0x7ffb2c09c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 10 monclient(hunting): handle_auth_done global_id 124443 payload 274 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 -- 172.21.15.134:0/870678399 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb2c136030 msgr2=0x7ffb2c136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb2c136030 0x7ffb2c136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 -- 172.21.15.134:0/870678399 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 msgr2=0x7ffb2c09c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 --2- 172.21.15.134:0/870678399 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 0x7ffb2c09c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 monclient: found mon.noname-a 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2b7fe640 1 -- 172.21.15.134:0/870678399 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c073250 con 0x7ffb2c136c60 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2a7fc640 10 monclient: get_auth_request con 0x7ffb2c09bfe0 auth_method 0 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.747+0000 7ffb2a7fc640 1 --2- 172.21.15.134:0/870678399 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 0x7ffb2c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.749+0000 7ffb2b7fe640 1 -- 172.21.15.134:0/870678399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb2c136c60 msgr2=0x7ffb2c137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:55.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.749+0000 7ffb2b7fe640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb2c136c60 0x7ffb2c137110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0a68b0 tx=0x7ffb2402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61c2da46f841d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:55.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 1 -- 172.21.15.134:0/870678399 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb24007a70 con 0x7ffb2c136c60 2024-09-16T15:20:55.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:55.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:55.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: dump: 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:55.760 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: _finish_auth 0 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 5 monclient: authenticate success, global_id 124443 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 1 -- 172.21.15.134:0/870678399 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2402d050 con 0x7ffb2c136c60 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 4 set_mon_vals no callback set 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 1 -- 172.21.15.134:0/870678399 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb24007bf0 con 0x7ffb2c136c60 2024-09-16T15:20:55.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb29ffb640 10 monclient: dump: 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:55.762 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.763 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:55.763 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:55.763 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 10 monclient: shutdown 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 -- 172.21.15.134:0/870678399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb2c136c60 msgr2=0x7ffb2c137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb2c136c60 0x7ffb2c137110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0a68b0 tx=0x7ffb2402a570 comp rx=0 tx=0).stop 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 -- 172.21.15.134:0/870678399 shutdown_connections 2024-09-16T15:20:55.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb2c136c60 0x7ffb2c137110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 --2- 172.21.15.134:0/870678399 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c09bfe0 0x7ffb2c09c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 --2- 172.21.15.134:0/870678399 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7ffb2c136030 0x7ffb2c136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 -- 172.21.15.134:0/870678399 >> 172.21.15.134:0/870678399 conn(0x7ffb2c0a9ea0 msgr2=0x7ffb2c0a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 -- 172.21.15.134:0/870678399 shutdown_connections 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.750+0000 7ffb315db640 1 -- 172.21.15.134:0/870678399 wait complete. 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.751+0000 7ffb315db640 10 monclient: build_initial_monmap 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.751+0000 7ffb315db640 10 monclient: monmap: 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:55.765 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.751+0000 7ffb315db640 1 Processor -- start 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.751+0000 7ffb315db640 1 -- start start 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.751+0000 7ffb315db640 10 monclient: init 2024-09-16T15:20:55.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient: _add_conns ranks=[8,4,1,2,5,7,3,6,0] 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-e con 0x7ffb2c09bfe0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-c con 0x7ffb2c136030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 0x7ffb2c135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): picked mon.noname-f con 0x7ffb2c136c60 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c1cb520 con 0x7ffb2c136c60 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0a80e0 con 0x7ffb2c136030 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c09fb60 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.767 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb315db640 10 monclient(hunting): authenticate will time out at 1992.341376s 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:52546/0 (socket says 172.21.15.134:52546) 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:39010/0 (socket says 172.21.15.134:39010) 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 -- 172.21.15.134:0/2830752474 learned_addr learned my addr 172.21.15.134:0/2830752474 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): get_auth_request con 0x7ffb2c09bfe0 auth_method 0 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): get_auth_request con 0x7ffb2c136030 auth_method 0 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:55.768 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 0x7ffb2c135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 10 monclient(hunting): get_auth_request con 0x7ffb2c136c60 auth_method 0 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient(hunting): handle_auth_done global_id 85175 payload 824 2024-09-16T15:20:55.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 -- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 msgr2=0x7ffb2c134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 -- 172.21.15.134:0/2830752474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 msgr2=0x7ffb2c135340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 0x7ffb2c135340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient: _finish_hunting 0 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 monclient: found mon.noname-e 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2affd640 1 -- 172.21.15.134:0/2830752474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb24009b30 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2a7fc640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.752+0000 7ffb2b7fe640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 0x7ffb2c135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb2affd640 1 -- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 msgr2=0x7ffb2c1cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb2affd640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00d800 tx=0x7ffb1c00dcc0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:55.770 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb1c00bd40 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: dump: 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:55.771 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c014430 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 5 monclient: authenticate success, global_id 85175 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 10 monclient: _renew_subs 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c126820 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffb1c014a10 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb137fe640 10 monclient: dump: 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:55.773 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 10 monclient: _renew_subs 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c126540 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.755+0000 7ffb297fa640 4 set_mon_vals no callback set 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ffb1c014b90 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb315db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb315db640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb315db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:55.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c09c400 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb137fe640 1 --2- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 0x7ffb1803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7ffb1c04a000 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb2a7fc640 1 --2- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 0x7ffb1803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.756+0000 7ffb2a7fc640 10 monclient: get_auth_request con 0x7ffb18038880 auth_method 0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.759+0000 7ffb2a7fc640 1 --2- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 0x7ffb1803ad40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c125020 tx=0x7ffb20008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.760+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ffb1c014e40 con 0x7ffb2c09bfe0 2024-09-16T15:20:55.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.760+0000 7ffb137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:55.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:55.760+0000 7ffb137fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:55.809 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=310991825}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2590678737}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1597241076}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3380340467}) 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.810 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1569077178}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=401743396}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=679589236}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=254202057}) 2024-09-16T15:20:55.811 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.807+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3712931665}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=901992739}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2215674527}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=335314413}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.821 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=923136966}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2488392077}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=888624385}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1086856315}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2689668991}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.809+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3832306390}) 2024-09-16T15:20:55.822 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=940133569}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=137948463}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1872100485}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4115140523}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.823 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=281676552}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1407083024}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1173412406}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2120962129}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.810+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=115525677}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1751395436}) 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.824 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1846926166}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4180401926}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2839535138}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=875367358}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1304669119}) 2024-09-16T15:20:55.825 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2669020493}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4048976319}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.812+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1357046554}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.813+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=939751274}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=261580816}) 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.826 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=709031872}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4170117937}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2707415947}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2874649534}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2755299436}) 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.827 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1339817398}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.814+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4105374693}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2858022244}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2608352710}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2617196966}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1249647526}) 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.828 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3263061552}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1498553427}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2026857633}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2873274892}) 2024-09-16T15:20:55.829 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.815+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1144851863}) 2024-09-16T15:20:55.863 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.863 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.863 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1533688325}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4282496389}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3168201622}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1131477208}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=893071409}) 2024-09-16T15:20:55.864 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3578397980}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1016618473}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2769690660}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.823+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=685791701}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2500443219}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.865 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1913477320}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1589849308}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1056663663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1316443475}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2199641802}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.825+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.866 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.826+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3681377708}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.826+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.826+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3999790663}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.826+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795857679}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=85984690}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2534054434}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3994941188}) 2024-09-16T15:20:55.867 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2761929494}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1789627822}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=857066021}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2195979292}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3634479326}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.827+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=741375685}) 2024-09-16T15:20:55.868 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3796490711}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=738507591}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2296778016}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1375656047}) 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.869 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283216994}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2808357502}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2512746883}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1986088537}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.828+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3987779386}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.829+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.829+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.829+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1025663325}) 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.829+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.870 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.829+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1899895177}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1201636679}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2161086547}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4185922149}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3626534533}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.871 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=310733091}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526955602}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.830+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1041954862}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3569250911}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3864097935}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=380136489}) 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.872 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3374381427}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3062365535}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2108299413}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2047135309}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=133968764}) 2024-09-16T15:20:55.873 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.831+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3120459072}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=549653682}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=759021091}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3700224135}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1065935539}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.874 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=691016530}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3439886352}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1642336744}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380930075}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.832+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2627209638}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.833+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3831052089}) 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.875 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3505123247}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1656937656}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1347350498}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=845355234}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=742175671}) 2024-09-16T15:20:55.876 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=351315}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2722947129}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.834+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900710934}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3923789851}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3866117896}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2244425429}) 2024-09-16T15:20:55.877 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1069360633}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2165069500}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2460773473}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1876041706}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492083398}) 2024-09-16T15:20:55.878 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.835+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1638378594}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.836+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.836+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.836+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2074010380}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.836+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.836+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1031159037}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1962430076}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=982304195}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.879 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1780859707}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2858418910}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4182270296}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1170945454}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.837+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3885908808}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1109245829}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.880 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1854169646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2699682843}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3551927974}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2659466926}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2444782063}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.881 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2147951973}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885021646}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.844+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3643044177}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2107306290}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1554839843}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3603066115}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.882 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1203063655}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3373475117}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1775355278}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145172421}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1891375534}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.846+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=826651799}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.883 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=115934732}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3686367263}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3642864027}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2598506946}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3705186663}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.884 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3477112347}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2838069665}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3279330989}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.847+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901645942}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=616554576}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=293310557}) 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.885 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=772062395}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3018755523}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=515976755}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=255278472}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=611227620}) 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.886 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1606864708}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.848+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2959346913}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4172576398}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1181553587}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4099846968}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=658514518}) 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.887 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3304635634}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.849+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3227048967}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=531079311}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=489912769}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.850+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3500431291}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1619694516}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.888 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3966693405}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3119331578}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1596899023}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1586389025}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2666969428}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.889 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2812889676}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3821156368}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.851+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2583076437}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3292715495}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1232717064}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=847785490}) 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.890 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=662564278}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2029054011}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3108999847}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4163355802}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=356707095}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.852+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1572266438}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.891 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2746400561}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=918373722}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4261483360}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3656006688}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1230642731}) 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.892 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1825265025}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1040328952}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343746147}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.854+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3723481124}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2215001550}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=974140863}) 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.893 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1617749970}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=500394683}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2017503832}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3474261746}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3555133806}) 2024-09-16T15:20:55.894 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3882508463}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.855+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1271126986}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=233210178}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1323827912}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=733116631}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1656831143}) 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.895 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=840876247}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2332824629}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=909888423}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1874341897}) 2024-09-16T15:20:55.896 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:55.856+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2839341917}) 2024-09-16T15:20:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb315db640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-16T15:20:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb315db640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb315db640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:56.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffb2c126110 con 0x7ffb2c09bfe0 2024-09-16T15:20:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb137fe640 1 -- 172.21.15.134:0/2830752474 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+3196 (secure 0 0 0) 0x7ffb1c011340 con 0x7ffb2c09bfe0 2024-09-16T15:20:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-16T15:20:56.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.000+0000 7ffb137fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:56.002 INFO:teuthology.orchestra.run.smithi134.stdout:{"election_epoch":166,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","modified":"2024-09-16T15:00:04.519049Z","created":"2024-09-16T15:00:04.519049Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3301","nonce":0},{"type":"v1","addr":"172.21.15.162:6790","nonce":0}]},"addr":"172.21.15.162:6790/0","public_addr":"172.21.15.162:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3302","nonce":0},{"type":"v1","addr":"172.21.15.134:6791","nonce":0}]},"addr":"172.21.15.134:6791/0","public_addr":"172.21.15.134:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3302","nonce":0},{"type":"v1","addr":"172.21.15.162:6791","nonce":0}]},"addr":"172.21.15.162:6791/0","public_addr":"172.21.15.162:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3303","nonce":0},{"type":"v1","addr":"172.21.15.134:6792","nonce":0}]},"addr":"172.21.15.134:6792/0","public_addr":"172.21.15.134:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3303","nonce":0},{"type":"v1","addr":"172.21.15.162:6792","nonce":0}]},"addr":"172.21.15.162:6792/0","public_addr":"172.21.15.162:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3304","nonce":0},{"type":"v1","addr":"172.21.15.134:6793","nonce":0}]},"addr":"172.21.15.134:6793/0","public_addr":"172.21.15.134:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 msgr2=0x7ffb1803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 0x7ffb1803ad40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c125020 tx=0x7ffb20008040 comp rx=0 tx=0).stop 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 10 monclient: shutdown 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 msgr2=0x7ffb2c1cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00d800 tx=0x7ffb1c00dcc0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 shutdown_connections 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> v2:172.21.15.162:6800/980547263 conn(0x7ffb18038880 0x7ffb1803ad40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c136c60 0x7ffb2c135340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7ffb2c136030 0x7ffb2c134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 --2- 172.21.15.134:0/2830752474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7ffb2c09bfe0 0x7ffb2c1cafe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 >> 172.21.15.134:0/2830752474 conn(0x7ffb2c0a9ea0 msgr2=0x7ffb2c0a82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 shutdown_connections 2024-09-16T15:20:56.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.002+0000 7ffb315db640 1 -- 172.21.15.134:0/2830752474 wait complete. 2024-09-16T15:20:56.014 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-16T15:20:56.014 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T15:20:56.024 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:20:56.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3772568134}) 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.084 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3347527062}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1430324890}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3846941954}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1499088026}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2651784756}) 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.085 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=951903754}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3603529513}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.081+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1333578220}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3492572348}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3951764097}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4171145484}) 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.086 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3781962391}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3585861951}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3710099366}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=346157797}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=986765383}) 2024-09-16T15:20:56.087 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.082+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2600246349}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1496776163}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3743817264}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=322684750}) 2024-09-16T15:20:56.094 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1105103605}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3917417462}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1074755689}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370378337}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=21868178}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.084+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3454345043}) 2024-09-16T15:20:56.095 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1220813375}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3668785605}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1377058634}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=982128604}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4084038334}) 2024-09-16T15:20:56.096 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2993871060}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4208061698}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3253721710}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.085+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=119561940}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4043566569}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3960783415}) 2024-09-16T15:20:56.097 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1843608143}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1714255372}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1927805946}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2392223647}) 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.098 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1473005380}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1029903964}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.087+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2914879599}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1568265544}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=274304875}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4048562792}) 2024-09-16T15:20:56.099 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1170980914}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3601139317}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1753678786}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4179354877}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.088+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2812542790}) 2024-09-16T15:20:56.100 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2313673060}) 2024-09-16T15:20:56.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.102 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1638602084}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3066005644}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=650501670}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2179949136}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1016760910}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.103 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1131806687}) 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2605404716}) 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2422185262}) 2024-09-16T15:20:56.104 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.094+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2959530842}) 2024-09-16T15:20:56.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.107 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3641253709}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=961254249}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3303390782}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2359649468}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=838753300}) 2024-09-16T15:20:56.108 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=909939877}) 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2602398545}) 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1244194590}) 2024-09-16T15:20:56.109 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.105+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3584577858}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649659315}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2673030398}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3672551590}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2793405112}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=628047841}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2901485040}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1229596604}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707904751}) 2024-09-16T15:20:56.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.131+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=137010740}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1830838726}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3027195070}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1511949552}) 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2161010227}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3465006607}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4139156454}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4084400841}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1725716845}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.132+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=111636957}) 2024-09-16T15:20:56.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1380695509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2665003481}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=631491689}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1910849418}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1221974955}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1559836743}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=749473877}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3823563509}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2902378739}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1060909517}) 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1111120795}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=836661119}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2248583538}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2879742746}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=455508201}) 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4279362090}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1778314690}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3674269078}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1739568573}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674904469}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3600897228}) 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2153022093}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2348644797}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1531596515}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3465212144}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3030949757}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1612142567}) 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1943207159}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=471612324}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2760452436}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2076109170}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273471590}) 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3161813834}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2267959013}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65709589}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2534760105}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4096423022}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3385153535}) 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3190664037}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3194927669}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3559569078}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3946175917}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1672747871}) 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346799876}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.139+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3064420836}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3942371020}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2063794626}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3153349210}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1536837889}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1788926778}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1867409428}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3544841479}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1121729354}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=163164675}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3198016296}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1500194144}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4105737655}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2038624295}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=413844510}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.142+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2025895832}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=426127327}) 2024-09-16T15:20:56.164 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1748154021}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=260298943}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2921911903}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3537277199}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2212878715}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2385314113}) 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.165 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4177057851}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2494865812}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3683679495}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=939262660}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3245735126}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2775360499}) 2024-09-16T15:20:56.166 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=304858672}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=876443485}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1338748653}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3624318649}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.167 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3807363599}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1948053769}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432332466}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1738477563}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1660872622}) 2024-09-16T15:20:56.168 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1620811685}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=129510317}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3265522481}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=962206745}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2582953202}) 2024-09-16T15:20:56.169 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.170 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.170 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1913219201}) 2024-09-16T15:20:56.170 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.170 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1084384051}) 2024-09-16T15:20:56.170 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2767401207}) 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.170+0000 7f1b9afd4640 10 monclient: get_monmap_and_config 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.170+0000 7f1b9afd4640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.170+0000 7f1b9afd4640 10 monclient: monmap: 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:56.171274+0000 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:56.171274+0000 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.172 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 Processor -- start 2024-09-16T15:20:56.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 -- start start 2024-09-16T15:20:56.174 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient: init 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient: _add_conns ranks=[6,7,1,9,3,5,10,4,8,0,2] 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-d con 0x7f1b94136030 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b9409c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-i con 0x7f1b9409bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f1b94136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b94073250 con 0x7f1b94136c60 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9409b600 con 0x7f1b94136030 2024-09-16T15:20:56.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940a5740 con 0x7f1b9409bfe0 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b9afd4640 10 monclient(hunting): authenticate will time out at 1992.760598s 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34738/0 (socket says 172.21.15.134:34738) 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 -- 172.21.15.134:0/4163712495 learned_addr learned my addr 172.21.15.134:0/4163712495 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): get_auth_request con 0x7f1b94136030 auth_method 0 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b9409c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 10 monclient(hunting): get_auth_request con 0x7f1b9409bfe0 auth_method 0 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient(hunting): handle_auth_done global_id 94875 payload 274 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 -- 172.21.15.134:0/4163712495 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 msgr2=0x7f1b9409c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b9409c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 -- 172.21.15.134:0/4163712495 >> v1:172.21.15.162:6789/0 conn(0x7f1b94136c60 legacy=0x7f1b941370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 monclient: found mon.noname-d 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b98d70640 1 -- 172.21.15.134:0/4163712495 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940a52c0 con 0x7f1b94136030 2024-09-16T15:20:56.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.171+0000 7f1b93fff640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b9409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b98d70640 1 -- 172.21.15.134:0/4163712495 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 msgr2=0x7f1b94136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b98d70640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098a0 tx=0x7f1b8802a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=2c73281946cf131e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 1 -- 172.21.15.134:0/4163712495 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b8802adf0 con 0x7f1b94136030 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: dump: 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.180 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 1 -- 172.21.15.134:0/4163712495 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b880307b0 con 0x7f1b94136030 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b9afd4640 5 monclient: authenticate success, global_id 94875 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b9afd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:56.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b9afd4640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 1 -- 172.21.15.134:0/4163712495 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b88030d90 con 0x7f1b94136030 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b9afd4640 4 set_mon_vals no callback set 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.174+0000 7f1b937fe640 10 monclient: dump: 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.182 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 10 monclient: shutdown 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 -- 172.21.15.134:0/4163712495 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 msgr2=0x7f1b94136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b880098a0 tx=0x7f1b8802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 -- 172.21.15.134:0/4163712495 shutdown_connections 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b9409c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/4163712495 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136030 0x7f1b94136410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 -- 172.21.15.134:0/4163712495 >> 172.21.15.134:0/4163712495 conn(0x7f1b940a9ea0 msgr2=0x7f1b940a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 -- 172.21.15.134:0/4163712495 shutdown_connections 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.175+0000 7f1b9afd4640 1 -- 172.21.15.134:0/4163712495 wait complete. 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 10 monclient: monmap: 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.184 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.185 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 1 Processor -- start 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 1 -- start start 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 10 monclient: init 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.176+0000 7f1b9afd4640 10 monclient: _add_conns ranks=[7,0,6,1,8,4,2,3,5] 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b940a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-i con 0x7f1b9409bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 0x7f1b94134ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-a con 0x7f1b94136030 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): picked mon.noname-d con 0x7f1b94136c60 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9409fed0 con 0x7f1b94136030 2024-09-16T15:20:56.189 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94122560 con 0x7f1b94136c60 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940a7de0 con 0x7f1b9409bfe0 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b9afd4640 10 monclient(hunting): authenticate will time out at 1992.766050s 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 0x7f1b94134ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34750/0 (socket says 172.21.15.134:34750) 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 -- 172.21.15.134:0/621781099 learned_addr learned my addr 172.21.15.134:0/621781099 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): get_auth_request con 0x7f1b94136c60 auth_method 0 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 0x7f1b94134ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44162/0 (socket says 172.21.15.134:44162) 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): get_auth_request con 0x7f1b94136030 auth_method 0 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b940a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 10 monclient(hunting): get_auth_request con 0x7f1b9409bfe0 auth_method 0 2024-09-16T15:20:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient(hunting): handle_auth_done global_id 94884 payload 824 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 -- 172.21.15.134:0/621781099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 msgr2=0x7f1b940a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b940a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 -- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 msgr2=0x7f1b94134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 0x7f1b94134ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 monclient: found mon.noname-d 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b99571640 1 -- 172.21.15.134:0/621781099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880094d0 con 0x7f1b94136c60 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.177+0000 7f1b98d70640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b940a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b99571640 1 -- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 msgr2=0x7f1b941350e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b99571640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b970 tx=0x7f1b8c00be30 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b8c007350 con 0x7f1b94136c60 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: dump: 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c0074d0 con 0x7f1b94136c60 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 5 monclient: authenticate success, global_id 94884 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 10 monclient: _renew_subs 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b940a7a20 con 0x7f1b94136c60 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1b8c007af0 con 0x7f1b94136c60 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b917fa640 10 monclient: dump: 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.195 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b92ffd640 4 set_mon_vals no callback set 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 10 monclient: _renew_subs 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.179+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b940a3430 con 0x7f1b94136c60 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f1b8c007c50 con 0x7f1b94136c60 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b9afd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b9afd4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b9afd4640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94121cc0 con 0x7f1b94136c60 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.180+0000 7f1b917fa640 1 --2- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 0x7f1b7803ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.181+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f1b8c0499a0 con 0x7f1b94136c60 2024-09-16T15:20:56.196 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.181+0000 7f1b98d70640 1 --2- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 0x7f1b7803ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.181+0000 7f1b98d70640 10 monclient: get_auth_request con 0x7f1b78038880 auth_method 0 2024-09-16T15:20:56.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.183+0000 7f1b98d70640 1 --2- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 0x7f1b7803ad40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b9406ce90 tx=0x7f1b8802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.185+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f1b8c00cb50 con 0x7f1b94136c60 2024-09-16T15:20:56.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.185+0000 7f1b917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.185+0000 7f1b917fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:56.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.344+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b94067a30 con 0x7f1b78038880 2024-09-16T15:20:56.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.346+0000 7f1b917fa640 1 -- 172.21.15.134:0/621781099 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54033 (secure 0 0 0) 0x7f1b94067a30 con 0x7f1b78038880 2024-09-16T15:20:56.348 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:20:56.349 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 msgr2=0x7f1b7803ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 0x7f1b7803ad40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b9406ce90 tx=0x7f1b8802e000 comp rx=0 tx=0).stop 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 10 monclient: shutdown 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 msgr2=0x7f1b941350e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b970 tx=0x7f1b8c00be30 comp rx=0 tx=0).stop 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 shutdown_connections 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.348+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> v2:172.21.15.162:6800/980547263 conn(0x7f1b78038880 0x7f1b7803ad40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f1b94136c60 0x7f1b941350e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b94136030 0x7f1b94134ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 --2- 172.21.15.134:0/621781099 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f1b9409bfe0 0x7f1b940a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 >> 172.21.15.134:0/621781099 conn(0x7f1b940a9ea0 msgr2=0x7f1b941bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 shutdown_connections 2024-09-16T15:20:56.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.349+0000 7f1b9afd4640 1 -- 172.21.15.134:0/621781099 wait complete. 2024-09-16T15:20:56.360 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1461,"stamp":"2024-09-16T15:20:54.409593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":363504,"kb_used_data":41508,"kb_used_omap":9,"kb_used_meta":321910,"kb_avail":565867536,"statfs":{"total":579820584960,"available":579448356864,"internally_reserved":0,"allocated":42504192,"data_stored":24192514,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":329636518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002428"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":1008,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715831+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:19:44.715831+0000","last_peered":"2024-09-16T15:19:44.715831+0000","last_clean":"2024-09-16T15:19:44.715831+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:19:44.715831+0000","last_undegraded":"2024-09-16T15:19:44.715831+0000","last_fullsized":"2024-09-16T15:19:44.715831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":864,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076649+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:19:43.076649+0000","last_peered":"2024-09-16T15:19:43.076649+0000","last_clean":"2024-09-16T15:19:43.076649+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:19:43.076649+0000","last_undegraded":"2024-09-16T15:19:43.076649+0000","last_fullsized":"2024-09-16T15:19:43.076649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":862,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715726+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:19:44.715726+0000","last_peered":"2024-09-16T15:19:44.715726+0000","last_clean":"2024-09-16T15:19:44.715726+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:19:44.715726+0000","last_undegraded":"2024-09-16T15:19:44.715726+0000","last_fullsized":"2024-09-16T15:19:44.715726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":859,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.079648+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:19:43.079648+0000","last_peered":"2024-09-16T15:19:43.079648+0000","last_clean":"2024-09-16T15:19:43.079648+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:19:43.079648+0000","last_undegraded":"2024-09-16T15:19:43.079648+0000","last_fullsized":"2024-09-16T15:19:43.079648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":865,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076560+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:19:43.076560+0000","last_peered":"2024-09-16T15:19:43.076560+0000","last_clean":"2024-09-16T15:19:43.076560+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:19:43.076560+0000","last_undegraded":"2024-09-16T15:19:43.076560+0000","last_fullsized":"2024-09-16T15:19:43.076560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":854,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.075866+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:19:43.075866+0000","last_peered":"2024-09-16T15:19:43.075866+0000","last_clean":"2024-09-16T15:19:43.075866+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:19:43.075866+0000","last_undegraded":"2024-09-16T15:19:43.075866+0000","last_fullsized":"2024-09-16T15:19:43.075866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":861,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.080433+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:19:43.080433+0000","last_peered":"2024-09-16T15:19:43.080433+0000","last_clean":"2024-09-16T15:19:43.080433+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:19:43.080433+0000","last_undegraded":"2024-09-16T15:19:43.080433+0000","last_fullsized":"2024-09-16T15:19:43.080433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":854,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.736928+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:19:44.736928+0000","last_peered":"2024-09-16T15:19:44.736928+0000","last_clean":"2024-09-16T15:19:44.736928+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:19:44.736928+0000","last_undegraded":"2024-09-16T15:19:44.736928+0000","last_fullsized":"2024-09-16T15:19:44.736928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":861,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.717482+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:19:44.717482+0000","last_peered":"2024-09-16T15:19:44.717482+0000","last_clean":"2024-09-16T15:19:44.717482+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:19:44.717482+0000","last_undegraded":"2024-09-16T15:19:44.717482+0000","last_fullsized":"2024-09-16T15:19:44.717482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575093,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7069696,"data_stored":3855983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005}]},{"osd":1,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.23499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":1.0860000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.26900000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.94999999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Sep 16 15:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.99299999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":0.752,"15min":1.046},"last":0.52800000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.34699999999999998},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"last":0.437},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.60099999999999998,"5min":0.628,"15min":0.63900000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.35399999999999998},"min":{"1min":0.221,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.58499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.69599999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575096,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50356,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321484,"statfs":{"total":96636764160,"available":96585199616,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.81000000000000005,"15min":1.151},"last":0.623},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.625,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.374,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":1.2669999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.65000000000000002,"5min":0.753,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69199999999999995,"15min":1.391},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.57899999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.503}]},{"osd":4,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.217,"5min":0.193,"15min":0.193},"max":{"1min":0.48599999999999999,"5min":0.65500000000000003,"15min":1.6100000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.32900000000000001}]},{"osd":5,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.38200000000000001},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55800000000000005,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.312},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.442,"15min":0.44},"min":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.59299999999999997,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.36899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575093,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.18099999999999999},"max":{"1min":0.438,"5min":0.56399999999999995,"15min":0.70899999999999996},"last":0.35499999999999998}]},{"osd":1,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.33000000000000002,"15min":0.35499999999999998},"min":{"1min":0.20200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.39200000000000002,"5min":0.496,"15min":2.2669999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33600000000000002,"15min":0.33700000000000002},"min":{"1min":0.224,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42199999999999999,"5min":0.58899999999999997,"15min":0.72299999999999998},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.80000000000000004},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.33500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.83999999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.79900000000000004},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.70799999999999996,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575094,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7065600,"data_stored":3854819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.156},"max":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.88900000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":1.262},"last":0.40100000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.53600000000000003,"5min":0.879,"15min":4.6829999999999998},"last":0.314},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.39600000000000002},"min":{"1min":0.21099999999999999,"5min":0.19800000000000001,"15min":0.17000000000000001},"max":{"1min":0.55700000000000005,"5min":0.90700000000000003,"15min":12.194000000000001},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Sep 16 15:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.377,"5min":0.246,"15min":0.246},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.024},"last":0.629},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":1.129},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.748,"15min":4.6020000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.755,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.62},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":1.198,"15min":1.2050000000000001},"last":0.60999999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640499,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56064,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.39900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.60199999999999998,"5min":0.745,"15min":0.82099999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.46899999999999997},"min":{"1min":0.38300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.67500000000000004},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.36799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.73499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.76300000000000001},"last":0.505}]},{"osd":2,"last update":"Mon Sep 16 15:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.69899999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77800000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.28599999999999998,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.25700000000000001,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.443}]},{"osd":5,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.22900000000000001},"max":{"1min":0.625,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.35599999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64308,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307532,"statfs":{"total":96636764160,"available":96570912768,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.47399999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34000000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.26800000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.35199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.53100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.317},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36499999999999999,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.24399999999999999}]},{"osd":3,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.749,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.387,"15min":1.387},"last":0.432}]},{"osd":4,"last update":"Mon Sep 16 15:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.77400000000000002,"5min":1.093,"15min":1.093},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":1.113,"15min":1.113},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.77500000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.309}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:20:56.362 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:20:56.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.440+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.440+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.440+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1911666348}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4277396749}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2597595252}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3897219887}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=713807507}) 2024-09-16T15:20:56.448 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=547762273}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4005176874}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=598474782}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.441+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519985745}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2134969280}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2001472997}) 2024-09-16T15:20:56.449 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=814630957}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=811678241}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3113057948}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3398364186}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=651114766}) 2024-09-16T15:20:56.450 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3235620269}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.442+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1903497245}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=516965039}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=148236317}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1621894702}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1337250799}) 2024-09-16T15:20:56.451 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=81533648}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3936143769}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.445+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.445+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2250622253}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.445+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.445+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2341513965}) 2024-09-16T15:20:56.452 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.445+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2804775447}) 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 10 monclient: get_monmap_and_config 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 10 monclient: monmap: 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:56.499100+0000 2024-09-16T15:20:56.499 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:56.499100+0000 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:56.500 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 1 Processor -- start 2024-09-16T15:20:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 1 -- start start 2024-09-16T15:20:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.498+0000 7f8b7fcb7640 10 monclient: init 2024-09-16T15:20:56.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient: _add_conns ranks=[7,6,0,1,3,10,2,9,5,4,8] 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b78136030 0x7f8b78136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-i con 0x7f8b78136030 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-d con 0x7f8b7809bfe0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8b78136c60 addr v1:172.21.15.134:6789/0 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b78073250 con 0x7f8b78136c60 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7809b600 con 0x7f8b7809bfe0 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780a5740 con 0x7f8b78136030 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7fcb7640 10 monclient(hunting): authenticate will time out at 1993.088490s 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34766/0 (socket says 172.21.15.134:34766) 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 -- 172.21.15.134:0/806869730 learned_addr learned my addr 172.21.15.134:0/806869730 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): get_auth_request con 0x7f8b7809bfe0 auth_method 0 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7da53640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b78136030 0x7f8b78136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7da53640 10 monclient(hunting): get_auth_request con 0x7f8b78136030 auth_method 0 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7da53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7da53640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7da53640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient(hunting): handle_auth_done global_id 94893 payload 274 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 -- 172.21.15.134:0/806869730 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b78136030 msgr2=0x7f8b78136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b78136030 0x7f8b78136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 -- 172.21.15.134:0/806869730 >> v1:172.21.15.134:6789/0 conn(0x7f8b78136c60 legacy=0x7f8b781370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 monclient: found mon.noname-d 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:56.505 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.499+0000 7f8b7d252640 1 -- 172.21.15.134:0/806869730 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780a52c0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7d252640 1 -- 172.21.15.134:0/806869730 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 msgr2=0x7f8b7809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7d252640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098a0 tx=0x7f8b6802a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=e86917e775a01ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 1 -- 172.21.15.134:0/806869730 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b6802adf0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: dump: 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 1 -- 172.21.15.134:0/806869730 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b680307b0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7fcb7640 5 monclient: authenticate success, global_id 94893 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7fcb7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 1 -- 172.21.15.134:0/806869730 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b68030dd0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7fcb7640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7fcb7640 4 set_mon_vals no callback set 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.502+0000 7f8b7ca51640 10 monclient: dump: 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.513 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 10 monclient: shutdown 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.514 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/806869730 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 msgr2=0x7f8b7809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098a0 tx=0x7f8b6802a0d0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/806869730 shutdown_connections 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f8b7809bfe0 0x7f8b7809c400 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/806869730 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f8b78136030 0x7f8b78136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/806869730 >> 172.21.15.134:0/806869730 conn(0x7f8b780a9ea0 msgr2=0x7f8b780a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/806869730 shutdown_connections 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.503+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/806869730 wait complete. 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient: monmap: 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 Processor -- start 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 -- start start 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient: init 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient: _add_conns ranks=[4,1,5,6,7,2,0,8,3] 2024-09-16T15:20:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-c con 0x7f8b7809bfe0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b78136030 0x7f8b78134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-f con 0x7f8b78136030 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b78136c60 0x7f8b781cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): picked mon.noname-h con 0x7f8b78136c60 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7811ff30 con 0x7f8b78136030 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78121220 con 0x7f8b7809bfe0 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78135850 con 0x7f8b78136c60 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7fcb7640 10 monclient(hunting): authenticate will time out at 1993.093869s 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.504+0000 7f8b7da53640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33426/0 (socket says 172.21.15.134:33426) 2024-09-16T15:20:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 -- 172.21.15.134:0/1085814058 learned_addr learned my addr 172.21.15.134:0/1085814058 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): get_auth_request con 0x7f8b7809bfe0 auth_method 0 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b78136030 0x7f8b78134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b78136c60 0x7f8b781cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): get_auth_request con 0x7f8b78136030 auth_method 0 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): get_auth_request con 0x7f8b78136c60 auth_method 0 2024-09-16T15:20:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7d252640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7e254640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient(hunting): handle_auth_done global_id 124474 payload 824 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 -- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b78136c60 msgr2=0x7f8b781cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b78136c60 0x7f8b781cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 -- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b78136030 msgr2=0x7f8b78134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b78136030 0x7f8b78134ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 monclient: found mon.noname-c 2024-09-16T15:20:56.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.505+0000 7f8b7da53640 1 -- 172.21.15.134:0/1085814058 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680094d0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7da53640 1 -- 172.21.15.134:0/1085814058 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 msgr2=0x7f8b781bece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7da53640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d830 tx=0x7f8b7400dcf0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b7400bd40 con 0x7f8b7809bfe0 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: dump: 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.520 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 5 monclient: authenticate success, global_id 124474 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74014430 con 0x7f8b7809bfe0 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8b74014a10 con 0x7f8b7809bfe0 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6ffff640 4 set_mon_vals no callback set 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b6e7fc640 10 monclient: dump: 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.522 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 10 monclient: _renew_subs 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b780a7de0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 10 monclient: _renew_subs 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.507+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b781228a0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8b74014bb0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b7fcb7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b7fcb7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b7fcb7640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7809c400 con 0x7f8b7809bfe0 2024-09-16T15:20:56.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b6e7fc640 1 --2- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 0x7f8b5c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f8b74049ff0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.508+0000 7f8b7d252640 1 --2- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 0x7f8b5c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.509+0000 7f8b7d252640 10 monclient: get_auth_request con 0x7f8b5c038880 auth_method 0 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.511+0000 7f8b7d252640 1 --2- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 0x7f8b5c03ad40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b68034000 tx=0x7f8b6802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.513+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f8b7400b8d0 con 0x7f8b7809bfe0 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.513+0000 7f8b6e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.513+0000 7f8b6e7fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:56.676 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b78061ae0 con 0x7f8b5c038880 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2930805894}) 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3993184481}) 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3225605686}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2695519387}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2015270190}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1464585612}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2207419045}) 2024-09-16T15:20:56.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3323032065}) 2024-09-16T15:20:56.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1535460848}) 2024-09-16T15:20:56.680 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.677+0000 7f8b6e7fc640 1 -- 172.21.15.134:0/1085814058 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54074 (secure 0 0 0) 0x7f8b68004040 con 0x7f8b5c038880 2024-09-16T15:20:56.681 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:20:56.681 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:20:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 msgr2=0x7f8b5c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 0x7f8b5c03ad40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b68034000 tx=0x7f8b6802e000 comp rx=0 tx=0).stop 2024-09-16T15:20:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 10 monclient: shutdown 2024-09-16T15:20:56.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 msgr2=0x7f8b781bece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400d830 tx=0x7f8b7400dcf0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 shutdown_connections 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> v2:172.21.15.162:6800/980547263 conn(0x7f8b5c038880 0x7f8b5c03ad40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8b78136c60 0x7f8b781cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b78136030 0x7f8b78134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 --2- 172.21.15.134:0/1085814058 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8b7809bfe0 0x7f8b781bece0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 >> 172.21.15.134:0/1085814058 conn(0x7f8b780a9ea0 msgr2=0x7f8b780a0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 shutdown_connections 2024-09-16T15:20:56.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.680+0000 7f8b7fcb7640 1 -- 172.21.15.134:0/1085814058 wait complete. 2024-09-16T15:20:56.691 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1462,"stamp":"2024-09-16T15:20:56.410096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":363504,"kb_used_data":41508,"kb_used_omap":9,"kb_used_meta":321910,"kb_avail":565867536,"statfs":{"total":579820584960,"available":579448356864,"internally_reserved":0,"allocated":42504192,"data_stored":24192514,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":329636518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002517"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":1008,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715831+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:19:44.715831+0000","last_peered":"2024-09-16T15:19:44.715831+0000","last_clean":"2024-09-16T15:19:44.715831+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:19:44.715831+0000","last_undegraded":"2024-09-16T15:19:44.715831+0000","last_fullsized":"2024-09-16T15:19:44.715831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":864,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076649+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:19:43.076649+0000","last_peered":"2024-09-16T15:19:43.076649+0000","last_clean":"2024-09-16T15:19:43.076649+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:19:43.076649+0000","last_undegraded":"2024-09-16T15:19:43.076649+0000","last_fullsized":"2024-09-16T15:19:43.076649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":862,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715726+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:19:44.715726+0000","last_peered":"2024-09-16T15:19:44.715726+0000","last_clean":"2024-09-16T15:19:44.715726+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:19:44.715726+0000","last_undegraded":"2024-09-16T15:19:44.715726+0000","last_fullsized":"2024-09-16T15:19:44.715726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":859,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.079648+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:19:43.079648+0000","last_peered":"2024-09-16T15:19:43.079648+0000","last_clean":"2024-09-16T15:19:43.079648+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:19:43.079648+0000","last_undegraded":"2024-09-16T15:19:43.079648+0000","last_fullsized":"2024-09-16T15:19:43.079648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":865,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076560+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:19:43.076560+0000","last_peered":"2024-09-16T15:19:43.076560+0000","last_clean":"2024-09-16T15:19:43.076560+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:19:43.076560+0000","last_undegraded":"2024-09-16T15:19:43.076560+0000","last_fullsized":"2024-09-16T15:19:43.076560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":854,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.075866+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:19:43.075866+0000","last_peered":"2024-09-16T15:19:43.075866+0000","last_clean":"2024-09-16T15:19:43.075866+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:19:43.075866+0000","last_undegraded":"2024-09-16T15:19:43.075866+0000","last_fullsized":"2024-09-16T15:19:43.075866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":861,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.080433+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:19:43.080433+0000","last_peered":"2024-09-16T15:19:43.080433+0000","last_clean":"2024-09-16T15:19:43.080433+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:19:43.080433+0000","last_undegraded":"2024-09-16T15:19:43.080433+0000","last_fullsized":"2024-09-16T15:19:43.080433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":854,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.736928+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:19:44.736928+0000","last_peered":"2024-09-16T15:19:44.736928+0000","last_clean":"2024-09-16T15:19:44.736928+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:19:44.736928+0000","last_undegraded":"2024-09-16T15:19:44.736928+0000","last_fullsized":"2024-09-16T15:19:44.736928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":861,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.717482+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:19:44.717482+0000","last_peered":"2024-09-16T15:19:44.717482+0000","last_clean":"2024-09-16T15:19:44.717482+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:19:44.717482+0000","last_undegraded":"2024-09-16T15:19:44.717482+0000","last_fullsized":"2024-09-16T15:19:44.717482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575094,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7069696,"data_stored":3855983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.23499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":1.0860000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.26900000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.94999999999999996},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Sep 16 15:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.99299999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":0.752,"15min":1.046},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.34699999999999998},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.60099999999999998,"5min":0.628,"15min":0.63900000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.35399999999999998},"min":{"1min":0.221,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.58499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.69599999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575097,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50356,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321484,"statfs":{"total":96636764160,"available":96585199616,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.81000000000000005,"15min":1.151},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.625,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41199999999999998}]},{"osd":1,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.374,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":1.2669999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.65000000000000002,"5min":0.753,"15min":0.88300000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69199999999999995,"15min":1.391},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.57899999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.38500000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.217,"5min":0.193,"15min":0.193},"max":{"1min":0.48599999999999999,"5min":0.65500000000000003,"15min":1.6100000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.26800000000000002}]},{"osd":5,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.38200000000000001},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55800000000000005,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.442,"15min":0.44},"min":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.59299999999999997,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.32900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575094,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.18099999999999999},"max":{"1min":0.438,"5min":0.56399999999999995,"15min":0.70899999999999996},"last":0.46500000000000002}]},{"osd":1,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.33000000000000002,"15min":0.35499999999999998},"min":{"1min":0.20200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.39200000000000002,"5min":0.496,"15min":2.2669999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33600000000000002,"15min":0.33700000000000002},"min":{"1min":0.224,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42199999999999999,"5min":0.58899999999999997,"15min":0.72299999999999998},"last":0.434}]},{"osd":3,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.80000000000000004},"last":0.68000000000000005}]},{"osd":4,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.33500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.83999999999999997},"last":0.627},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.70799999999999996,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575095,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7065600,"data_stored":3854819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.156},"max":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.88900000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":1.262},"last":0.46100000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.53600000000000003,"5min":0.879,"15min":4.6829999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.39600000000000002},"min":{"1min":0.21099999999999999,"5min":0.19800000000000001,"15min":0.17000000000000001},"max":{"1min":0.55700000000000005,"5min":0.90700000000000003,"15min":12.194000000000001},"last":0.39300000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.377,"5min":0.246,"15min":0.246},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.024},"last":0.5},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":1.129},"last":0.44800000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.748,"15min":4.6020000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.755,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":1.198,"15min":1.2050000000000001},"last":0.49299999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56064,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.39900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.60199999999999998,"5min":0.745,"15min":0.82099999999999995},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.46899999999999997},"min":{"1min":0.38300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.67500000000000004},"last":0.434}]},{"osd":1,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.36799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.73499999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.76300000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Sep 16 15:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.69899999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77800000000000002},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.28599999999999998,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.25700000000000001,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.47099999999999997}]},{"osd":5,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.22900000000000001},"max":{"1min":0.625,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.54200000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64308,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307532,"statfs":{"total":96636764160,"available":96570912768,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.47399999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34000000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.26800000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.35199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.53100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.317},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36499999999999999,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.24399999999999999}]},{"osd":3,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.749,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.387,"15min":1.387},"last":0.432}]},{"osd":4,"last update":"Mon Sep 16 15:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.77400000000000002,"5min":1.093,"15min":1.093},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":1.113,"15min":1.113},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.77500000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.309}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:20:56.692 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:20:56.693 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=44696354}) 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.699+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1424108747}) 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2490196974}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3641592643}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2872420486}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3856514388}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1982665998}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=575533573}) 2024-09-16T15:20:56.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:56.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2151157077}) 2024-09-16T15:20:56.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.796+0000 7ff5ce1b5640 10 monclient: get_monmap_and_config 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.796+0000 7ff5ce1b5640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.796+0000 7ff5ce1b5640 10 monclient: monmap: 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:56.797465+0000 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:56.797465+0000 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.798 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:56.799 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 Processor -- start 2024-09-16T15:20:56.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 -- start start 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient: init 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient: _add_conns ranks=[9,10,1,3,2,5,8,6,0,4,7] 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff5c8136030 0x7ff5c8136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-a con 0x7ff5c8136030 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5c809bfe0 0x7ff5c809c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-f con 0x7ff5c809bfe0 addr v2:172.21.15.162:3300/0 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7ff5c8136c60 addr v1:172.21.15.162:6789/0 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c8073250 con 0x7ff5c8136c60 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c809b600 con 0x7ff5c8136030 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80a5740 con 0x7ff5c809bfe0 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5ce1b5640 10 monclient(hunting): authenticate will time out at 1993.386806s 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff5c8136030 0x7ff5c8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff5c8136030 0x7ff5c8136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44166/0 (socket says 172.21.15.134:44166) 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 1 -- 172.21.15.134:0/1943430777 learned_addr learned my addr 172.21.15.134:0/1943430777 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 10 monclient(hunting): get_auth_request con 0x7ff5c8136030 auth_method 0 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.797+0000 7ff5c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c6ffd640 1 --2- 172.21.15.134:0/1943430777 >> v2:172.21.15.162:3300/0 conn(0x7ff5c809bfe0 0x7ff5c809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c6ffd640 10 monclient(hunting): get_auth_request con 0x7ff5c809bfe0 auth_method 0 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_done global_id 124461 payload 274 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 1 -- 172.21.15.134:0/1943430777 >> v2:172.21.15.162:3300/0 conn(0x7ff5c809bfe0 msgr2=0x7ff5c809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 1 --2- 172.21.15.134:0/1943430777 >> v2:172.21.15.162:3300/0 conn(0x7ff5c809bfe0 0x7ff5c809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 1 -- 172.21.15.134:0/1943430777 >> v1:172.21.15.162:6789/0 conn(0x7ff5c8136c60 legacy=0x7ff5c81370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 1 monclient: found mon.noname-a 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:56.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.798+0000 7ff5c77fe640 1 -- 172.21.15.134:0/1943430777 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80a52c0 con 0x7ff5c8136030 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.800+0000 7ff5c77fe640 1 -- 172.21.15.134:0/1943430777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c8136030 msgr2=0x7ff5c8136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c77fe640 1 --2- 172.21.15.134:0/1943430777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c8136030 0x7ff5c8136410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80a5950 tx=0x7ff5b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990fde335d4cfdef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 1 -- 172.21.15.134:0/1943430777 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b802ae30 con 0x7ff5c8136030 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: dump: 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.810 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.811 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 1 -- 172.21.15.134:0/1943430777 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b80307b0 con 0x7ff5c8136030 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 5 monclient: authenticate success, global_id 124461 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 1 -- 172.21.15.134:0/1943430777 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5b8030d90 con 0x7ff5c8136030 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 4 set_mon_vals no callback set 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5c67fc640 10 monclient: dump: 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.812 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 10 monclient: shutdown 2024-09-16T15:20:56.813 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/1943430777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c8136030 msgr2=0x7ff5c8136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/1943430777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c8136030 0x7ff5c8136410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80a5950 tx=0x7ff5b802a110 comp rx=0 tx=0).stop 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/1943430777 shutdown_connections 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/1943430777 >> v2:172.21.15.162:3300/0 conn(0x7ff5c809bfe0 0x7ff5c809c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/1943430777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5c8136030 0x7ff5c8136410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/1943430777 >> 172.21.15.134:0/1943430777 conn(0x7ff5c80a9ea0 msgr2=0x7ff5c80a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/1943430777 shutdown_connections 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.801+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/1943430777 wait complete. 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.802+0000 7ff5ce1b5640 10 monclient: build_initial_monmap 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.802+0000 7ff5ce1b5640 10 monclient: monmap: 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:56.814 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.802+0000 7ff5ce1b5640 1 Processor -- start 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 -- start start 2024-09-16T15:20:56.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient: init 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient: _add_conns ranks=[1,3,2,8,0,4,6,7,5] 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c809bfe0 0x7ff5c80a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-f con 0x7ff5c809bfe0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c8136030 0x7ff5c8134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-g con 0x7ff5c8136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): picked mon.noname-b con 0x7ff5c8136c60 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80a3440 con 0x7ff5c809bfe0 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c809fd20 con 0x7ff5c8136c60 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:56.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80a81f0 con 0x7ff5c8136030 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5ce1b5640 10 monclient(hunting): authenticate will time out at 1993.392500s 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59014/0 (socket says 172.21.15.134:59014) 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 -- 172.21.15.134:0/4000280044 learned_addr learned my addr 172.21.15.134:0/4000280044 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): get_auth_request con 0x7ff5c8136c60 auth_method 0 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c8136030 0x7ff5c8134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c809bfe0 0x7ff5c80a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): get_auth_request con 0x7ff5c809bfe0 auth_method 0 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): get_auth_request con 0x7ff5c8136030 auth_method 0 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:56.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient(hunting): handle_auth_done global_id 114743 payload 824 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 -- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c8136030 msgr2=0x7ff5c8134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c8136030 0x7ff5c8134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 -- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c809bfe0 msgr2=0x7ff5c80a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c809bfe0 0x7ff5c80a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 monclient: found mon.noname-b 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.803+0000 7ff5c7fff640 1 -- 172.21.15.134:0/4000280044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b80094d0 con 0x7ff5c8136c60 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5c7fff640 1 -- 172.21.15.134:0/4000280044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 msgr2=0x7ff5c81352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5c7fff640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b970 tx=0x7ff5c000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5c0007350 con 0x7ff5c8136c60 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: dump: 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.820 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: _finish_auth 0 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: _check_auth_tickets 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c00074d0 con 0x7ff5c8136c60 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 5 monclient: authenticate success, global_id 114743 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 10 monclient: _renew_subs 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c8122750 con 0x7ff5c8136c60 2024-09-16T15:20:56.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff5c0007ab0 con 0x7ff5c8136c60 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5affff640 10 monclient: dump: 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:56.822 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 10 monclient: _renew_subs 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c8120d00 con 0x7ff5c8136c60 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.806+0000 7ff5c5ffb640 4 set_mon_vals no callback set 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7ff5c0007cd0 con 0x7ff5c8136c60 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5affff640 1 --2- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 0x7ff5b403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5ce1b5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5ce1b5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5ce1b5640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7ff5c0049a10 con 0x7ff5c8136c60 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c81cb7d0 con 0x7ff5c8136c60 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5c77fe640 1 --2- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 0x7ff5b403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.807+0000 7ff5c77fe640 10 monclient: get_auth_request con 0x7ff5b4038880 auth_method 0 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.810+0000 7ff5c77fe640 1 --2- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 0x7ff5b403ad40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8034000 tx=0x7ff5b802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.812+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7ff5c81cb7d0 con 0x7ff5c8136c60 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.812+0000 7ff5affff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.812+0000 7ff5affff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:56.971 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.970+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c8061ae0 con 0x7ff5b4038880 2024-09-16T15:20:56.974 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.973+0000 7ff5affff640 1 -- 172.21.15.134:0/4000280044 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54074 (secure 0 0 0) 0x7ff5b8004040 con 0x7ff5b4038880 2024-09-16T15:20:56.975 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:20:56.975 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 msgr2=0x7ff5b403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 0x7ff5b403ad40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8034000 tx=0x7ff5b802e000 comp rx=0 tx=0).stop 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 10 monclient: shutdown 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 msgr2=0x7ff5c81352d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b970 tx=0x7ff5c000be30 comp rx=0 tx=0).stop 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 shutdown_connections 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c8136c60 0x7ff5c81352d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> v2:172.21.15.162:6800/980547263 conn(0x7ff5b4038880 0x7ff5b403ad40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5c8136030 0x7ff5c8134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 --2- 172.21.15.134:0/4000280044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5c809bfe0 0x7ff5c80a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:56.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 >> 172.21.15.134:0/4000280044 conn(0x7ff5c80a9ea0 msgr2=0x7ff5c81bee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:56.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 shutdown_connections 2024-09-16T15:20:56.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:56.975+0000 7ff5ce1b5640 1 -- 172.21.15.134:0/4000280044 wait complete. 2024-09-16T15:20:56.988 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1462,"stamp":"2024-09-16T15:20:56.410096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":363504,"kb_used_data":41508,"kb_used_omap":9,"kb_used_meta":321910,"kb_avail":565867536,"statfs":{"total":579820584960,"available":579448356864,"internally_reserved":0,"allocated":42504192,"data_stored":24192514,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":329636518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002517"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":1008,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715831+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:19:44.715831+0000","last_peered":"2024-09-16T15:19:44.715831+0000","last_clean":"2024-09-16T15:19:44.715831+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:19:44.715831+0000","last_undegraded":"2024-09-16T15:19:44.715831+0000","last_fullsized":"2024-09-16T15:19:44.715831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":864,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076649+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:19:43.076649+0000","last_peered":"2024-09-16T15:19:43.076649+0000","last_clean":"2024-09-16T15:19:43.076649+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:19:43.076649+0000","last_undegraded":"2024-09-16T15:19:43.076649+0000","last_fullsized":"2024-09-16T15:19:43.076649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":862,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715726+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:19:44.715726+0000","last_peered":"2024-09-16T15:19:44.715726+0000","last_clean":"2024-09-16T15:19:44.715726+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:19:44.715726+0000","last_undegraded":"2024-09-16T15:19:44.715726+0000","last_fullsized":"2024-09-16T15:19:44.715726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":859,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.079648+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:19:43.079648+0000","last_peered":"2024-09-16T15:19:43.079648+0000","last_clean":"2024-09-16T15:19:43.079648+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:19:43.079648+0000","last_undegraded":"2024-09-16T15:19:43.079648+0000","last_fullsized":"2024-09-16T15:19:43.079648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":865,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076560+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:19:43.076560+0000","last_peered":"2024-09-16T15:19:43.076560+0000","last_clean":"2024-09-16T15:19:43.076560+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:19:43.076560+0000","last_undegraded":"2024-09-16T15:19:43.076560+0000","last_fullsized":"2024-09-16T15:19:43.076560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":854,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.075866+0000","last_change":"2024-09-16T15:00:49.320254+0000","last_active":"2024-09-16T15:19:43.075866+0000","last_peered":"2024-09-16T15:19:43.075866+0000","last_clean":"2024-09-16T15:19:43.075866+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:19:43.075866+0000","last_undegraded":"2024-09-16T15:19:43.075866+0000","last_fullsized":"2024-09-16T15:19:43.075866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:52.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":861,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.080433+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:19:43.080433+0000","last_peered":"2024-09-16T15:19:43.080433+0000","last_clean":"2024-09-16T15:19:43.080433+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:19:43.080433+0000","last_undegraded":"2024-09-16T15:19:43.080433+0000","last_fullsized":"2024-09-16T15:19:43.080433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":854,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.736928+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:19:44.736928+0000","last_peered":"2024-09-16T15:19:44.736928+0000","last_clean":"2024-09-16T15:19:44.736928+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:19:44.736928+0000","last_undegraded":"2024-09-16T15:19:44.736928+0000","last_fullsized":"2024-09-16T15:19:44.736928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:32.035217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":861,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.717482+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:19:44.717482+0000","last_peered":"2024-09-16T15:19:44.717482+0000","last_clean":"2024-09-16T15:19:44.717482+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:19:44.717482+0000","last_undegraded":"2024-09-16T15:19:44.717482+0000","last_fullsized":"2024-09-16T15:19:44.717482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575094,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7069696,"data_stored":3855983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66700000000000004}]},{"osd":1,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.23499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":1.0860000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.26900000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.94999999999999996},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Sep 16 15:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.99299999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":0.752,"15min":1.046},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.34699999999999998},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.60099999999999998,"5min":0.628,"15min":0.63900000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.35399999999999998},"min":{"1min":0.221,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.58499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.69599999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575097,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50356,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321484,"statfs":{"total":96636764160,"available":96585199616,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.81000000000000005,"15min":1.151},"last":0.436},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.625,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.41199999999999998}]},{"osd":1,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.374,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":1.2669999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.65000000000000002,"5min":0.753,"15min":0.88300000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69199999999999995,"15min":1.391},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.57899999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.38500000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.217,"5min":0.193,"15min":0.193},"max":{"1min":0.48599999999999999,"5min":0.65500000000000003,"15min":1.6100000000000001},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.26800000000000002}]},{"osd":5,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.38200000000000001},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55800000000000005,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.442,"15min":0.44},"min":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.59299999999999997,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.32900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575094,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.18099999999999999},"max":{"1min":0.438,"5min":0.56399999999999995,"15min":0.70899999999999996},"last":0.46500000000000002}]},{"osd":1,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.33000000000000002,"15min":0.35499999999999998},"min":{"1min":0.20200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.39200000000000002,"5min":0.496,"15min":2.2669999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.33600000000000002,"15min":0.33700000000000002},"min":{"1min":0.224,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42199999999999999,"5min":0.58899999999999997,"15min":0.72299999999999998},"last":0.434}]},{"osd":3,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.80000000000000004},"last":0.68000000000000005}]},{"osd":4,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.33500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.83999999999999997},"last":0.627},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.79900000000000004},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.70799999999999996,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67200000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575095,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7065600,"data_stored":3854819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.156},"max":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.88900000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":1.262},"last":0.46100000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.53600000000000003,"5min":0.879,"15min":4.6829999999999998},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.39600000000000002},"min":{"1min":0.21099999999999999,"5min":0.19800000000000001,"15min":0.17000000000000001},"max":{"1min":0.55700000000000005,"5min":0.90700000000000003,"15min":12.194000000000001},"last":0.39300000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.377,"5min":0.246,"15min":0.246},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.024},"last":0.5},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":1.129},"last":0.44800000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.748,"15min":4.6020000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.755,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":1.198,"15min":1.2050000000000001},"last":0.49299999999999999}]}]},{"osd":4,"up_from":11,"seq":47244640500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56064,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.39900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.60199999999999998,"5min":0.745,"15min":0.82099999999999995},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.46899999999999997},"min":{"1min":0.38300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.67500000000000004},"last":0.434}]},{"osd":1,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.36799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.73499999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.76300000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Mon Sep 16 15:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.69899999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77800000000000002},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.28599999999999998,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.25700000000000001,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.47099999999999997}]},{"osd":5,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.22900000000000001},"max":{"1min":0.625,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.54200000000000004}]}]},{"osd":0,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64308,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307532,"statfs":{"total":96636764160,"available":96570912768,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.47399999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34000000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.26800000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.35199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.53100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.317},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36499999999999999,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.24399999999999999}]},{"osd":3,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.749,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.387,"15min":1.387},"last":0.432}]},{"osd":4,"last update":"Mon Sep 16 15:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.77400000000000002,"5min":1.093,"15min":1.093},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":1.113,"15min":1.113},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.77500000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.309}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:20:56.989 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4028153573}) 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=888278913}) 2024-09-16T15:20:57.064 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3651522137}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1544465473}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=292952347}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=969827727}) 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2909772089}) 2024-09-16T15:20:57.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3839328373}) 2024-09-16T15:20:57.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.061+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1797343207}) 2024-09-16T15:20:57.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2583726699}) 2024-09-16T15:20:57.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1587782257}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2980172453}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3962280909}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3336497510}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2189201921}) 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=327094120}) 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2363250722}) 2024-09-16T15:20:57.071 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.067+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2646433212}) 2024-09-16T15:20:57.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.103+0000 7f6af0f91640 10 monclient: get_monmap_and_config 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.103+0000 7f6af0f91640 10 monclient: build_initial_monmap 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.103+0000 7f6af0f91640 10 monclient: monmap: 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:57.104502+0000 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:57.104502+0000 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:57.105 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:57.106 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 Processor -- start 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 -- start start 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient: init 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient: _add_conns ranks=[0,5,4,8,3,2,10,7,1,6,9] 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f6aec071900 addr v1:172.21.15.134:6789/0 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6aec0a1bf0 0x7f6aec1392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-h con 0x7f6aec0a1bf0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 0x7f6aec13b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-c con 0x7f6aec0a1ff0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec073250 con 0x7f6aec071900 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec071ce0 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec06e7d0 con 0x7f6aec0a1bf0 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6af0f91640 10 monclient(hunting): authenticate will time out at 1993.693841s 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aea59c640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f6aec071900 0x7f6aec0a14e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59246/0 (socket says 172.21.15.134:59246) 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aead9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 0x7f6aec13b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aea59c640 1 -- 172.21.15.134:0/2037096448 learned_addr learned my addr 172.21.15.134:0/2037096448 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aead9d640 10 monclient(hunting): get_auth_request con 0x7f6aec0a1ff0 auth_method 0 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:57.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aead9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.104+0000 7f6aead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient(hunting): handle_auth_done global_id 124483 payload 274 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6ae1d9b640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6aec0a1bf0 0x7f6aec1392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 1 -- 172.21.15.134:0/2037096448 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6aec0a1bf0 msgr2=0x7f6aec1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6aec0a1bf0 0x7f6aec1392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 1 -- 172.21.15.134:0/2037096448 >> v1:172.21.15.134:6789/0 conn(0x7f6aec071900 legacy=0x7f6aec0a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient: _finish_hunting 0 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 1 monclient: found mon.noname-c 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:20:57.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.105+0000 7f6aead9d640 1 -- 172.21.15.134:0/2037096448 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06f9e0 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6aead9d640 1 -- 172.21.15.134:0/2037096448 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 msgr2=0x7f6aec13b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6aead9d640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 0x7f6aec13b820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009dc0 tx=0x7f6ae402a4d0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=7cf06acb2427d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 1 -- 172.21.15.134:0/2037096448 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae4007ca0 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: dump: 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.116 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: _finish_auth 0 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:57.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 5 monclient: authenticate success, global_id 124483 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 1 -- 172.21.15.134:0/2037096448 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae402d050 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 4 set_mon_vals no callback set 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 1 -- 172.21.15.134:0/2037096448 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ae4007e60 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6ae9d9b640 10 monclient: dump: 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:57.118 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 10 monclient: shutdown 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 -- 172.21.15.134:0/2037096448 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 msgr2=0x7f6aec13b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 0x7f6aec13b820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009dc0 tx=0x7f6ae402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 -- 172.21.15.134:0/2037096448 shutdown_connections 2024-09-16T15:20:57.119 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f6aec0a1ff0 0x7f6aec13b820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 --2- 172.21.15.134:0/2037096448 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f6aec0a1bf0 0x7f6aec1392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 -- 172.21.15.134:0/2037096448 >> 172.21.15.134:0/2037096448 conn(0x7f6aec0a25d0 msgr2=0x7f6aec0a29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 -- 172.21.15.134:0/2037096448 shutdown_connections 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.108+0000 7f6af0f91640 1 -- 172.21.15.134:0/2037096448 wait complete. 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.109+0000 7f6af0f91640 10 monclient: build_initial_monmap 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.109+0000 7f6af0f91640 10 monclient: monmap: 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:57.120 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.109+0000 7f6af0f91640 1 Processor -- start 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 -- start start 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient: init 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient: _add_conns ranks=[8,1,3,0,6,4,7,5,2] 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.121 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-e con 0x7f6aec071900 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aec0a1bf0 0x7f6aec1339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-f con 0x7f6aec0a1bf0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aec0a1ff0 0x7f6aec1cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): picked mon.noname-g con 0x7f6aec0a1ff0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec11e340 con 0x7f6aec0a1bf0 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec135460 con 0x7f6aec0a1ff0 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0a17b0 con 0x7f6aec071900 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6af0f91640 10 monclient(hunting): authenticate will time out at 1993.699496s 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:35426/0 (socket says 172.21.15.134:35426) 2024-09-16T15:20:57.122 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 -- 172.21.15.134:0/906639911 learned_addr learned my addr 172.21.15.134:0/906639911 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): get_auth_request con 0x7f6aec071900 auth_method 0 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aec0a1bf0 0x7f6aec1339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aec0a1ff0 0x7f6aec1cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): get_auth_request con 0x7f6aec0a1ff0 auth_method 0 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): get_auth_request con 0x7f6aec0a1bf0 auth_method 0 2024-09-16T15:20:57.123 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aead9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6ae1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient(hunting): handle_auth_done global_id 85184 payload 824 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 -- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aec0a1ff0 msgr2=0x7f6aec1cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aec0a1ff0 0x7f6aec1cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 -- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aec0a1bf0 msgr2=0x7f6aec1339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aec0a1bf0 0x7f6aec1339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient: _finish_hunting 0 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 monclient: found mon.noname-e 2024-09-16T15:20:57.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.110+0000 7f6aea59c640 1 -- 172.21.15.134:0/906639911 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4009a70 con 0x7f6aec071900 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6aea59c640 1 -- 172.21.15.134:0/906639911 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 msgr2=0x7f6aec1c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6aea59c640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c460 tx=0x7f6adc00c920 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6adc005cf0 con 0x7f6aec071900 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: dump: 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.125 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 5 monclient: authenticate success, global_id 85184 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc005e90 con 0x7f6aec071900 2024-09-16T15:20:57.126 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 10 monclient: _renew_subs 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0a1360 con 0x7f6aec071900 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6adc0074e0 con 0x7f6aec071900 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae37fe640 10 monclient: dump: 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:57.127 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 10 monclient: _renew_subs 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6ae959a640 4 set_mon_vals no callback set 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.113+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec11e060 con 0x7f6aec071900 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f6adc007640 con 0x7f6aec071900 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6af0f91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:57.128 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6af0f91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6af0f91640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec1c87a0 con 0x7f6aec071900 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6ae37fe640 1 --2- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 0x7f6acc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f6adc049a80 con 0x7f6aec071900 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6ae1d9b640 1 --2- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 0x7f6acc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.114+0000 7f6ae1d9b640 10 monclient: get_auth_request con 0x7f6acc038880 auth_method 0 2024-09-16T15:20:57.129 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.117+0000 7f6ae1d9b640 1 --2- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 0x7f6acc03ad40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6aec13d110 tx=0x7f6ad4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.119+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f6aec1c87a0 con 0x7f6aec071900 2024-09-16T15:20:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f6ae37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:57.130 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f6ae37fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4290076109}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=622964369}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1831341935}) 2024-09-16T15:20:57.132 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3926802625}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1276346673}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1831868180}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3840885211}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2728729316}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.120+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3058028112}) 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.126+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.133 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.126+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.126+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2081363007}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.126+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.126+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3236241073}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2537436628}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3104411204}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1129488324}) 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.134 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3498680310}) 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=42866431}) 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1759290075}) 2024-09-16T15:20:57.135 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.127+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4210075412}) 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=123905653}) 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1922885305}) 2024-09-16T15:20:57.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1560056969}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3443406913}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.133+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=136887339}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4183378979}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1614085305}) 2024-09-16T15:20:57.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.139 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3081168495}) 2024-09-16T15:20:57.139 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.134+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3004303466}) 2024-09-16T15:20:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.295+0000 7f6af0f91640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:20:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.295+0000 7f6af0f91640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:20:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.295+0000 7f6af0f91640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:20:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.295+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6aec1c87a0 con 0x7f6aec071900 2024-09-16T15:20:57.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.296+0000 7f6ae37fe640 1 -- 172.21.15.134:0/906639911 <== mon.8 v2:172.21.15.134:3304/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v861) v1 ==== 74+0+10916 (secure 0 0 0) 0x7f6aec1c87a0 con 0x7f6aec071900 2024-09-16T15:20:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.296+0000 7f6ae37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-16T15:20:57.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.296+0000 7f6ae37fe640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:20:57.298 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:20:57.298 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":861,"fsid":"9b832ad6-88ab-4e38-a956-73c2de3b6492","created":"2024-09-16T15:00:30.672727+0000","modified":"2024-09-16T15:19:44.702224+0000","last_up_change":"2024-09-16T15:00:44.667324+0000","last_in_change":"2024-09-16T15:00:36.156025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:00:44.203747+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:00:45.513777+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8700f1de-9dc0-436a-a2f6-7e955103c5c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4141541690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6805","nonce":4141541690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6807","nonce":4141541690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4141541690}]},"public_addr":"172.21.15.134:6804/4141541690","cluster_addr":"172.21.15.134:6805/4141541690","heartbeat_back_addr":"172.21.15.134:6807/4141541690","heartbeat_front_addr":"172.21.15.134:6806/4141541690","state":["exists","up"]},{"osd":1,"uuid":"92341517-5f3d-458a-904f-eae7c1d454a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":443532050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6809","nonce":443532050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":443532050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":443532050}]},"public_addr":"172.21.15.134:6808/443532050","cluster_addr":"172.21.15.134:6809/443532050","heartbeat_back_addr":"172.21.15.134:6811/443532050","heartbeat_front_addr":"172.21.15.134:6810/443532050","state":["exists","up"]},{"osd":2,"uuid":"12bf62b9-5ecd-4476-8a55-c620754c3c85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1264438958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6801","nonce":1264438958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6803","nonce":1264438958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1264438958}]},"public_addr":"172.21.15.134:6800/1264438958","cluster_addr":"172.21.15.134:6801/1264438958","heartbeat_back_addr":"172.21.15.134:6803/1264438958","heartbeat_front_addr":"172.21.15.134:6802/1264438958","state":["exists","up"]},{"osd":3,"uuid":"91058726-d953-423c-8958-65f169388831","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":3922313654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":3922313654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":3922313654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3922313654}]},"public_addr":"172.21.15.162:6805/3922313654","cluster_addr":"172.21.15.162:6806/3922313654","heartbeat_back_addr":"172.21.15.162:6808/3922313654","heartbeat_front_addr":"172.21.15.162:6807/3922313654","state":["exists","up"]},{"osd":4,"uuid":"8d7393b8-3f8f-41f7-ab05-4de36eef0dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":1098590432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":1098590432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":1098590432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":1098590432}]},"public_addr":"172.21.15.162:6801/1098590432","cluster_addr":"172.21.15.162:6802/1098590432","heartbeat_back_addr":"172.21.15.162:6804/1098590432","heartbeat_front_addr":"172.21.15.162:6803/1098590432","state":["exists","up"]},{"osd":5,"uuid":"7c88ec95-db14-4be8-87a5-3151b85a4058","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6809","nonce":518841521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":518841521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":518841521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6811","nonce":518841521}]},"public_addr":"172.21.15.162:6809/518841521","cluster_addr":"172.21.15.162:6810/518841521","heartbeat_back_addr":"172.21.15.162:6812/518841521","heartbeat_front_addr":"172.21.15.162:6811/518841521","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.306416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.343686+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.259639+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.265227+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.168850+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:00:41.302963+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":748,"length":1},{"begin":789,"length":1},{"begin":798,"length":2},{"begin":812,"length":1},{"begin":816,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 msgr2=0x7f6acc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 0x7f6acc03ad40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6aec13d110 tx=0x7f6ad4008040 comp rx=0 tx=0).stop 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 10 monclient: shutdown 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 msgr2=0x7f6aec1c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c460 tx=0x7f6adc00c920 comp rx=0 tx=0).stop 2024-09-16T15:20:57.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 shutdown_connections 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> v2:172.21.15.162:6800/980547263 conn(0x7f6acc038880 0x7f6acc03ad40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6aec0a1ff0 0x7f6aec1cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6aec0a1bf0 0x7f6aec1339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 --2- 172.21.15.134:0/906639911 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f6aec071900 0x7f6aec1c7db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 >> 172.21.15.134:0/906639911 conn(0x7f6aec0a25d0 msgr2=0x7f6aec1349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 shutdown_connections 2024-09-16T15:20:57.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:57.298+0000 7f6af0f91640 1 -- 172.21.15.134:0/906639911 wait complete. 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1154672430}) 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3861343945}) 2024-09-16T15:20:57.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2043412244}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=134202190}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2360780707}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856792530}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3326053887}) 2024-09-16T15:20:57.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=891175584}) 2024-09-16T15:20:57.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3217106877}) 2024-09-16T15:20:57.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.709 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2344425581}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4068267880}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3775646180}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4227528923}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2153485369}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.710 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=515594376}) 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2045798033}) 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=202426755}) 2024-09-16T15:20:57.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:57.707+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3239103288}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1287356004}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3637388688}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2703464876}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=48290428}) 2024-09-16T15:20:58.092 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2995496652}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=618258146}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3548470900}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4235634059}) 2024-09-16T15:20:58.093 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.089+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3533799489}) 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3345603936}) 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2191391300}) 2024-09-16T15:20:58.140 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3069191495}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3417820108}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4147370969}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3981718089}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.141 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3621323434}) 2024-09-16T15:20:58.142 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.142 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2476854166}) 2024-09-16T15:20:58.142 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1135265098}) 2024-09-16T15:20:58.148 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.148 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2547626936}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1840914408}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1111671566}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2442258670}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1119333290}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.149 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3471038759}) 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2028490208}) 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2845016610}) 2024-09-16T15:20:58.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1250845931}) 2024-09-16T15:20:58.312 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T15:20:58.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 10 monclient: get_monmap_and_config 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 10 monclient: build_initial_monmap 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 10 monclient: monmap: 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:58.416254+0000 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:58.416254+0000 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:58.417 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:58.418 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 1 Processor -- start 2024-09-16T15:20:58.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 1 -- start start 2024-09-16T15:20:58.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.415+0000 7f359d00d640 10 monclient: init 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient: _add_conns ranks=[3,10,2,8,6,5,0,1,7,4,9] 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 0x7f359806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-g con 0x7f35980a1e80 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 0x7f35980a2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-f con 0x7f359806b230 addr v2:172.21.15.162:3300/0 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-b con 0x7f35980a27a0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359806bee0 con 0x7f359806b230 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359806b610 con 0x7f35980a27a0 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359806c7f0 con 0x7f35980a1e80 2024-09-16T15:20:58.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359d00d640 10 monclient(hunting): authenticate will time out at 1995.005504s 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59018/0 (socket says 172.21.15.134:59018) 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 -- 172.21.15.134:0/3721681087 learned_addr learned my addr 172.21.15.134:0/3721681087 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): get_auth_request con 0x7f35980a27a0 auth_method 0 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f3596d9d640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 0x7f359806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359659c640 1 --2- 172.21.15.134:0/3721681087 >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 0x7f35980a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient(hunting): handle_auth_done global_id 114752 payload 274 2024-09-16T15:20:58.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 -- 172.21.15.134:0/3721681087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 msgr2=0x7f359806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 0x7f359806cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 -- 172.21.15.134:0/3721681087 >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 msgr2=0x7f35980a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 --2- 172.21.15.134:0/3721681087 >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 0x7f35980a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient: _finish_hunting 0 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 monclient: found mon.noname-b 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359759e640 1 -- 172.21.15.134:0/3721681087 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359804e730 con 0x7f35980a27a0 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f3596d9d640 10 monclient: get_auth_request con 0x7f35980a1e80 auth_method 0 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359659c640 10 monclient: get_auth_request con 0x7f359806b230 auth_method 0 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f3596d9d640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 0x7f359806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:58.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.416+0000 7f359659c640 1 --2- 172.21.15.134:0/3721681087 >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 0x7f35980a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:58.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.419+0000 7f359759e640 1 -- 172.21.15.134:0/3721681087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 msgr2=0x7f3598137030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:58.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.419+0000 7f359759e640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3590009e50 tx=0x7f359002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcbd8dfb8ea70a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 1 -- 172.21.15.134:0/3721681087 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3590007a70 con 0x7f35980a27a0 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: dump: 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.428 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: _finish_auth 0 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 1 -- 172.21.15.134:0/3721681087 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359002d050 con 0x7f35980a27a0 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 5 monclient: authenticate success, global_id 114752 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:58.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 1 -- 172.21.15.134:0/3721681087 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3590007bf0 con 0x7f35980a27a0 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 4 set_mon_vals no callback set 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f3595d9b640 10 monclient: dump: 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.430 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 10 monclient: shutdown 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 -- 172.21.15.134:0/3721681087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 msgr2=0x7f3598137030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3590009e50 tx=0x7f359002a560 comp rx=0 tx=0).stop 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 -- 172.21.15.134:0/3721681087 shutdown_connections 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35980a27a0 0x7f3598137030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 --2- 172.21.15.134:0/3721681087 >> v2:172.21.15.162:3300/0 conn(0x7f359806b230 0x7f35980a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 --2- 172.21.15.134:0/3721681087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a1e80 0x7f359806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 -- 172.21.15.134:0/3721681087 >> 172.21.15.134:0/3721681087 conn(0x7f35980a7480 msgr2=0x7f35980a7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 -- 172.21.15.134:0/3721681087 shutdown_connections 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.420+0000 7f359d00d640 1 -- 172.21.15.134:0/3721681087 wait complete. 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.421+0000 7f359d00d640 10 monclient: build_initial_monmap 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.421+0000 7f359d00d640 10 monclient: monmap: 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:58.432 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.421+0000 7f359d00d640 1 Processor -- start 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.421+0000 7f359d00d640 1 -- start start 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.421+0000 7f359d00d640 10 monclient: init 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient: _add_conns ranks=[2,7,3,1,5,0,8,6,4] 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 0x7f35981bed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-b con 0x7f359806b230 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:58.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35980a1e80 0x7f35981bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-i con 0x7f35980a1e80 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a27a0 0x7f35981a8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): picked mon.noname-g con 0x7f35980a27a0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35980a4880 con 0x7f359806b230 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35981ac450 con 0x7f35980a27a0 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f35980a6c40 con 0x7f35980a1e80 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359d00d640 10 monclient(hunting): authenticate will time out at 1995.011403s 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 0x7f35981bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 0x7f35981bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59034/0 (socket says 172.21.15.134:59034) 2024-09-16T15:20:58.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 -- 172.21.15.134:0/3399424793 learned_addr learned my addr 172.21.15.134:0/3399424793 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): get_auth_request con 0x7f359806b230 auth_method 0 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359659c640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35980a1e80 0x7f35981bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359659c640 10 monclient(hunting): get_auth_request con 0x7f35980a1e80 auth_method 0 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359659c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359659c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359659c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient(hunting): handle_auth_done global_id 114761 payload 824 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359759e640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a27a0 0x7f35981a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 -- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35980a1e80 msgr2=0x7f35981bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f35980a1e80 0x7f35981bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 -- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a27a0 msgr2=0x7f35981a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a27a0 0x7f35981a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient: _finish_hunting 0 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 monclient: found mon.noname-b 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f3596d9d640 1 -- 172.21.15.134:0/3399424793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3590009b30 con 0x7f359806b230 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359759e640 10 monclient: get_auth_request con 0x7f35980a27a0 auth_method 0 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.422+0000 7f359759e640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35980a27a0 0x7f35981a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.424+0000 7f3596d9d640 1 -- 172.21.15.134:0/3399424793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 msgr2=0x7f35981bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.424+0000 7f3596d9d640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 0x7f35981bed70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f358800c7f0 tx=0x7f358800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f358800ce40 con 0x7f359806b230 2024-09-16T15:20:58.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: dump: 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.437 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 5 monclient: authenticate success, global_id 114761 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 10 monclient: _renew_subs 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35880113f0 con 0x7f359806b230 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35980a4d20 con 0x7f359806b230 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f35880119d0 con 0x7f359806b230 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359559a640 4 set_mon_vals no callback set 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f358f7fe640 10 monclient: dump: 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.439 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 10 monclient: _renew_subs 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981aacb0 con 0x7f359806b230 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.425+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3598061ae0 con 0x7f359806b230 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3588011560 con 0x7f359806b230 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.162:6800/980547263 conn(0x7f35780388b0 0x7f357803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f358804a030 con 0x7f359806b230 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.134:6804/4141541690 conn(0x7f357803e3d0 0x7f35780407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 --> v2:172.21.15.134:6804/4141541690 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3578040e60 con 0x7f357803e3d0 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358804c5c0 con 0x7f359806b230 2024-09-16T15:20:58.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f358f7fe640 10 monclient: handle_get_version_reply finishing 1 version 861 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f359659c640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.162:6800/980547263 conn(0x7f35780388b0 0x7f357803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f359659c640 10 monclient: get_auth_request con 0x7f35780388b0 auth_method 0 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f359759e640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.134:6804/4141541690 conn(0x7f357803e3d0 0x7f35780407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.426+0000 7f359759e640 10 monclient: get_auth_request con 0x7f357803e3d0 auth_method 0 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.429+0000 7f359659c640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.162:6800/980547263 conn(0x7f35780388b0 0x7f357803ad70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f359806dfa0 tx=0x7f3580008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.441 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.430+0000 7f359759e640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.134:6804/4141541690 conn(0x7f357803e3d0 0x7f35780407f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.431+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== osd.0 v2:172.21.15.134:6804/4141541690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3578040e60 con 0x7f357803e3d0 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675214301}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3088976450}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1553831875}) 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.445 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 --> v2:172.21.15.134:6804/4141541690 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f359807a680 con 0x7f357803e3d0 2024-09-16T15:20:58.446 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=643382269}) 2024-09-16T15:20:58.446 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.446 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.446 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556516043}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=987314319}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2883426460}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3705311369}) 2024-09-16T15:20:58.447 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3030806675}) 2024-09-16T15:20:58.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f358f7fe640 1 -- 172.21.15.134:0/3399424793 <== osd.0 v2:172.21.15.134:6804/4141541690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3590007600 con 0x7f357803e3d0 2024-09-16T15:20:58.448 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:58.448 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:20:58.448 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 >> v2:172.21.15.134:6804/4141541690 conn(0x7f357803e3d0 msgr2=0x7f35780407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.443+0000 7f359d00d640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.134:6804/4141541690 conn(0x7f357803e3d0 0x7f35780407f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 >> v2:172.21.15.162:6800/980547263 conn(0x7f35780388b0 msgr2=0x7f357803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 --2- 172.21.15.134:0/3399424793 >> v2:172.21.15.162:6800/980547263 conn(0x7f35780388b0 0x7f357803ad70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f359806dfa0 tx=0x7f3580008040 comp rx=0 tx=0).stop 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 10 monclient: shutdown 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 msgr2=0x7f35981bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 --2- 172.21.15.134:0/3399424793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f359806b230 0x7f35981bed70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f358800c7f0 tx=0x7f358800cbf0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.449 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359759e640 1 -- 172.21.15.134:0/3399424793 reap_dead start 2024-09-16T15:20:58.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 shutdown_connections 2024-09-16T15:20:58.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 >> 172.21.15.134:0/3399424793 conn(0x7f35980a7480 msgr2=0x7f359809d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:58.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 shutdown_connections 2024-09-16T15:20:58.450 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f359d00d640 1 -- 172.21.15.134:0/3399424793 wait complete. 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1924672879}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=414949495}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4258415195}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3321777216}) 2024-09-16T15:20:58.455 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3821599459}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1272549230}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=211479570}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1448699903}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.444+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=538182586}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.456 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=840539231}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=642447052}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2727754413}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2658029527}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1739303025}) 2024-09-16T15:20:58.457 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=292296682}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2562199964}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3457340991}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.446+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1280469629}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1904559671}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=110477646}) 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.458 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1793606688}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2160089888}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1496549267}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=579985784}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2170600846}) 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.459 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578427285}) 2024-09-16T15:20:58.460 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.447+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3582490557}) 2024-09-16T15:20:58.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T15:20:58.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.596+0000 7fd6e0f01640 10 monclient: get_monmap_and_config 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.596+0000 7fd6e0f01640 10 monclient: build_initial_monmap 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.596+0000 7fd6e0f01640 10 monclient: monmap: 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:58.597578+0000 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:58.597578+0000 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:58.598 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:58.599 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 Processor -- start 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 -- start start 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient: init 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient: _add_conns ranks=[10,3,5,4,8,0,2,1,6,9,7] 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-f con 0x7fd6dc136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6dc137220 0x7fd6dc137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-g con 0x7fd6dc137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 0x7fd6dc136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-h con 0x7fd6dc135dc0 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6dc09abc0 con 0x7fd6dc136e40 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc09ba50 con 0x7fd6dc137220 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0a53c0 con 0x7fd6dc135dc0 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.597+0000 7fd6e0f01640 10 monclient(hunting): authenticate will time out at 1995.186747s 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6dc137220 0x7fd6dc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 0x7fd6dc136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.134:38540/0 (socket says 172.21.15.134:38540) 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 1 -- 172.21.15.134:0/3691455929 learned_addr learned my addr 172.21.15.134:0/3691455929 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): get_auth_request con 0x7fd6dc135dc0 auth_method 0 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): get_auth_request con 0x7fd6dc136e40 auth_method 0 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): get_auth_request con 0x7fd6dc137220 auth_method 0 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_done global_id 45563 payload 274 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 -- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6dc137220 msgr2=0x7fd6dc137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 --2- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6dc137220 0x7fd6dc137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 -- 172.21.15.134:0/3691455929 >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 msgr2=0x7fd6dc09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 --2- 172.21.15.134:0/3691455929 >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient: _finish_hunting 0 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 monclient: found mon.noname-h 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.162:3302/0 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6dad9d640 1 -- 172.21.15.134:0/3691455929 --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc073250 con 0x7fd6dc135dc0 2024-09-16T15:20:58.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.598+0000 7fd6da59c640 1 --2- 172.21.15.134:0/3691455929 >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6dad9d640 1 -- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 msgr2=0x7fd6dc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6dad9d640 1 --2- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 0x7fd6dc136220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009dc0 tx=0x7fd6d402a4d0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=24796cedadd7addd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 1 -- 172.21.15.134:0/3691455929 <== mon.5 v2:172.21.15.162:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6d4007bc0 con 0x7fd6dc135dc0 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: dump: 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.611 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: _finish_auth 0 2024-09-16T15:20:58.612 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: _check_auth_tickets 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 1 -- 172.21.15.134:0/3691455929 <== mon.5 v2:172.21.15.162:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d402d050 con 0x7fd6dc135dc0 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 5 monclient: authenticate success, global_id 45563 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 4 set_mon_vals no callback set 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 1 -- 172.21.15.134:0/3691455929 <== mon.5 v2:172.21.15.162:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6d4007d40 con 0x7fd6dc135dc0 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6d9d9b640 10 monclient: dump: 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.613 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 10 monclient: shutdown 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 1 -- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 msgr2=0x7fd6dc136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.614 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.601+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 0x7fd6dc136220 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009dc0 tx=0x7fd6d402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 -- 172.21.15.134:0/3691455929 shutdown_connections 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc135dc0 0x7fd6dc136220 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/3691455929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6dc137220 0x7fd6dc137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/3691455929 >> v2:172.21.15.162:3300/0 conn(0x7fd6dc136e40 0x7fd6dc09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 -- 172.21.15.134:0/3691455929 >> 172.21.15.134:0/3691455929 conn(0x7fd6dc0a6110 msgr2=0x7fd6dc0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 -- 172.21.15.134:0/3691455929 shutdown_connections 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 1 -- 172.21.15.134:0/3691455929 wait complete. 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 10 monclient: build_initial_monmap 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.602+0000 7fd6e0f01640 10 monclient: monmap: 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:58.615 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 Processor -- start 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 -- start start 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient: init 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:58.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient: _add_conns ranks=[1,7,5,3,8,6,0,2,4] 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6dc135dc0 0x7fd6dc134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-f con 0x7fd6dc135dc0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-i con 0x7fd6dc136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc137220 0x7fd6dc1218f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): picked mon.noname-h con 0x7fd6dc137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0a8a10 con 0x7fd6dc135dc0 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1362c0 con 0x7fd6dc137220 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1234d0 con 0x7fd6dc136e40 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6e0f01640 10 monclient(hunting): authenticate will time out at 1995.192744s 2024-09-16T15:20:58.617 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6d1d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6da59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6dc135dc0 0x7fd6dc134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6dad9d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc137220 0x7fd6dc1218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.603+0000 7fd6d1d9b640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3303/0 says I am v2:172.21.15.134:35154/0 (socket says 172.21.15.134:35154) 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 -- 172.21.15.134:0/4155388528 learned_addr learned my addr 172.21.15.134:0/4155388528 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): get_auth_request con 0x7fd6dc135dc0 auth_method 0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): get_auth_request con 0x7fd6dc136e40 auth_method 0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): get_auth_request con 0x7fd6dc137220 auth_method 0 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:58.618 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6dad9d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6da59c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient(hunting): handle_auth_done global_id 134197 payload 824 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 -- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc137220 msgr2=0x7fd6dc1218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc137220 0x7fd6dc1218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 -- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6dc135dc0 msgr2=0x7fd6dc134eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6dc135dc0 0x7fd6dc134eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:58.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient: _finish_hunting 0 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 monclient: found mon.noname-i 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.162:3303/0 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.604+0000 7fd6d1d9b640 1 -- 172.21.15.134:0/4155388528 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0094d0 con 0x7fd6dc136e40 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d1d9b640 1 -- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 msgr2=0x7fd6dc1353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d1d9b640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e870 tx=0x7fd6c400ed30 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6c40094f0 con 0x7fd6dc136e40 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: dump: 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.620 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: _finish_auth 0 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 5 monclient: authenticate success, global_id 134197 2024-09-16T15:20:58.621 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d1d9b640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e870 tx=0x7fd6c400ed30 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fd6c4013a50 mon_map magic: 0 v1 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4013470 con 0x7fd6dc136e40 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 10 monclient: _renew_subs 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d4009a70 con 0x7fd6dc136e40 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6d959a640 4 set_mon_vals no callback set 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 10 monclient: _renew_subs 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.606+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc1242a0 con 0x7fd6dc136e40 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd6c4013a50 con 0x7fd6dc136e40 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6e0f01640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 10 monclient: dump: 2024-09-16T15:20:58.622 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:58.623 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd6c4013bb0 con 0x7fd6dc136e40 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6e0f01640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6e0f01640 10 monclient: _send_mon_message to mon.i at v2:172.21.15.162:3303/0 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc09b770 con 0x7fd6dc136e40 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.607+0000 7fd6d37fe640 1 --2- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 0x7fd6bc03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.608+0000 7fd6da59c640 1 --2- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 0x7fd6bc03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.608+0000 7fd6da59c640 10 monclient: get_auth_request con 0x7fd6bc038880 auth_method 0 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.610+0000 7fd6da59c640 1 --2- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 0x7fd6bc03ad40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc006ca0 tx=0x7fd6cc009240 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:58.624 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:58.612+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fd6c404a640 con 0x7fd6dc136e40 2024-09-16T15:20:58.674 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2287869842}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=735071673}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1070184856}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3034300619}) 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.675 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2357729933}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2192037585}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1829140405}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4100294653}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.656+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3290752042}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1997988388}) 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.676 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2963991651}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=480928151}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3289020362}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.657+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2660870655}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2128743639}) 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.677 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3934458037}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1602275817}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.658+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3216001029}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1497749798}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2676289926}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2702779836}) 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3659896406}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=912236828}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1903675173}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3846436349}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1914210580}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.659+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=252499675}) 2024-09-16T15:20:58.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3965488063}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=455196644}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.660+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3795553659}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4277447908}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=823227466}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=208014632}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=153545300}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1906324467}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.661+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=316901573}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4169738357}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.681 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1322218204}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1375105554}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1374146654}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=661890737}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2574940667}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.682 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=165705785}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=434093132}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.662+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=461856875}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2200952650}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=514214564}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3268696122}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.683 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=849514421}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1480457792}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1371338410}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3607940915}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1356427}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.663+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4014503571}) 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5927147}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2638306180}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1040921787}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3272845499}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2984243423}) 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1758431781}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=398048590}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1507402481}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.665+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=470632384}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1557696617}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1193893473}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2898963586}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3243123179}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342155351}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3334134085}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2669997548}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2966241770}) 2024-09-16T15:20:58.687 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.666+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3003314470}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=873970642}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2623125474}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3955498876}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=409120210}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.688 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3724031497}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2907933584}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2785462607}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1766611152}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.668+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1770689579}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1774275673}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.689 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1113108422}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=589204594}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1353482638}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4013449659}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3067365496}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.690 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=356030283}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3209114892}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.669+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=829315956}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2551718991}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2280783656}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1792785095}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2553916570}) 2024-09-16T15:20:58.691 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2691984597}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3373747209}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=114281571}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786742088}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.670+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3776810141}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.692 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3026145333}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2755635772}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2820175582}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=472933916}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1005440299}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.693 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4211780071}) 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978845785}) 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4013720970}) 2024-09-16T15:20:58.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.672+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2107997673}) 2024-09-16T15:20:58.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.698 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=717447861}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2322521190}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1707214763}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=813520955}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3756709140}) 2024-09-16T15:20:58.699 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2735411262}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4283764857}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3054057711}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408710762}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=764822809}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4183891842}) 2024-09-16T15:20:58.700 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3407584252}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3462463100}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2181051073}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4003350459}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3902265057}) 2024-09-16T15:20:58.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3796472708}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.684+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3854228212}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2976331809}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2540990873}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4083830139}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=402318918}) 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.702 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3109985158}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4228139936}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3363518272}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1597584453}) 2024-09-16T15:20:58.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1213936351}) 2024-09-16T15:20:58.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2850499353}) 2024-09-16T15:20:58.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1554115334}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1594877323}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=163216902}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2661329185}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1769930156}) 2024-09-16T15:20:58.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.707 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.707 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=300552857}) 2024-09-16T15:20:58.707 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:58.707 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3265265393}) 2024-09-16T15:20:58.707 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:58.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2717883030}) 2024-09-16T15:20:59.143 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.143 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3632224823}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2433905389}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1983415012}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2665343342}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2608799369}) 2024-09-16T15:20:59.144 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3902148591}) 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3410723386}) 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3399824548}) 2024-09-16T15:20:59.145 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3608220428}) 2024-09-16T15:20:59.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.435+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mon.7 v2:172.21.15.162:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fd6c4013e60 con 0x7fd6dc136e40 2024-09-16T15:20:59.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.435+0000 7fd6d37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:59.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.435+0000 7fd6d37fe640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=934019361}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2489274055}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1938074956}) 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.487 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4068715085}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=269760227}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2171996643}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1851659767}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2509873059}) 2024-09-16T15:20:59.488 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3568645223}) 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1180692301}) 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.510 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3338356503}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3127234566}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2393442961}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1414876566}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2919805536}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.511 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3547579938}) 2024-09-16T15:20:59.512 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.512 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701557914}) 2024-09-16T15:20:59.512 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.507+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3315994105}) 2024-09-16T15:20:59.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.615+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6dc061ae0 con 0x7fd6bc038880 2024-09-16T15:20:59.616 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.615+0000 7fd6d37fe640 1 -- 172.21.15.134:0/4155388528 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd6cc0099e0 con 0x7fd6bc038880 2024-09-16T15:20:59.617 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 msgr2=0x7fd6bc03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 0x7fd6bc03ad40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc006ca0 tx=0x7fd6cc009240 comp rx=0 tx=0).stop 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 10 monclient: shutdown 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 msgr2=0x7fd6dc1353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e870 tx=0x7fd6c400ed30 comp rx=0 tx=0).stop 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 shutdown_connections 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> v2:172.21.15.162:6800/980547263 conn(0x7fd6bc038880 0x7fd6bc03ad40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fd6dc137220 0x7fd6dc1218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7fd6dc136e40 0x7fd6dc1353f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 --2- 172.21.15.134:0/4155388528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6dc135dc0 0x7fd6dc134eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 >> 172.21.15.134:0/4155388528 conn(0x7fd6dc0a6110 msgr2=0x7fd6dc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:59.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 shutdown_connections 2024-09-16T15:20:59.620 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.617+0000 7fd6e0f01640 1 -- 172.21.15.134:0/4155388528 wait complete. 2024-09-16T15:20:59.629 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T15:20:59.629 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=608270228}) 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=963287370}) 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1674663646}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=534911464}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2726044074}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3651620155}) 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3495624270}) 2024-09-16T15:20:59.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=65177095}) 2024-09-16T15:20:59.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3482381379}) 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.731+0000 7f180a7fb640 10 monclient: get_monmap_and_config 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.731+0000 7f180a7fb640 10 monclient: build_initial_monmap 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.731+0000 7f180a7fb640 10 monclient: monmap: 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.732 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:59.732116+0000 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:59.732116+0000 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:59.733 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:59.734 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:59.734 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.731+0000 7f180a7fb640 1 Processor -- start 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 -- start start 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient: init 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient: _add_conns ranks=[9,6,8,10,0,1,7,3,5,2,4] 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-a con 0x7f18040a4220 addr v2:172.21.15.134:3300/0 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 0x7f18040a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-d con 0x7f180409c720 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-e con 0x7f180409b1a0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:20:59.736 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180409c390 con 0x7f18040a4220 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f18040a5080 con 0x7f180409c720 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f18040a5e20 con 0x7f180409b1a0 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f180a7fb640 10 monclient(hunting): authenticate will time out at 1996.321500s 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 0x7f18040a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44178/0 (socket says 172.21.15.134:44178) 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:35432/0 (socket says 172.21.15.134:35432) 2024-09-16T15:20:59.737 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 1 -- 172.21.15.134:0/2737706731 learned_addr learned my addr 172.21.15.134:0/2737706731 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): get_auth_request con 0x7f180409c720 auth_method 0 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): get_auth_request con 0x7f180409b1a0 auth_method 0 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): get_auth_request con 0x7f18040a4220 auth_method 0 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.738 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient(hunting): handle_auth_done global_id 85193 payload 274 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 -- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 msgr2=0x7f18040a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 0x7f18040a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 -- 172.21.15.134:0/2737706731 >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 msgr2=0x7f18040a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 --2- 172.21.15.134:0/2737706731 >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient: _finish_hunting 0 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 monclient: found mon.noname-e 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:20:59.739 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1808d98640 1 -- 172.21.15.134:0/2737706731 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180404e730 con 0x7f180409b1a0 2024-09-16T15:20:59.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f18037fe640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 0x7f18040a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:59.740 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.732+0000 7f1803fff640 1 --2- 172.21.15.134:0/2737706731 >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:59.744 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1808d98640 1 -- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 msgr2=0x7f1804136fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1808d98640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009dc0 tx=0x7f17fc02a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=3bb02fb8e95a3700 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 1 -- 172.21.15.134:0/2737706731 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17fc007bc0 con 0x7f180409b1a0 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: dump: 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.745 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: _finish_auth 0 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 1 -- 172.21.15.134:0/2737706731 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc02d050 con 0x7f180409b1a0 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f180a7fb640 5 monclient: authenticate success, global_id 85193 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f180a7fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f180a7fb640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:59.746 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 1 -- 172.21.15.134:0/2737706731 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17fc007d40 con 0x7f180409b1a0 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f180a7fb640 4 set_mon_vals no callback set 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.735+0000 7f1802ffd640 10 monclient: dump: 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.747 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 10 monclient: shutdown 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 -- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 msgr2=0x7f1804136fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009dc0 tx=0x7f17fc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 -- 172.21.15.134:0/2737706731 shutdown_connections 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f180409b1a0 0x7f1804136fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 --2- 172.21.15.134:0/2737706731 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f180409c720 0x7f18040a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 --2- 172.21.15.134:0/2737706731 >> v2:172.21.15.134:3300/0 conn(0x7f18040a4220 0x7f18040a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 -- 172.21.15.134:0/2737706731 >> 172.21.15.134:0/2737706731 conn(0x7f18040a9820 msgr2=0x7f18040a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 -- 172.21.15.134:0/2737706731 shutdown_connections 2024-09-16T15:20:59.748 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.736+0000 7f180a7fb640 1 -- 172.21.15.134:0/2737706731 wait complete. 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.737+0000 7f180a7fb640 10 monclient: build_initial_monmap 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.737+0000 7f180a7fb640 10 monclient: monmap: 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:59.749 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.737+0000 7f180a7fb640 1 Processor -- start 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.737+0000 7f180a7fb640 1 -- start start 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.737+0000 7f180a7fb640 10 monclient: init 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient: _add_conns ranks=[1,6,0,4,3,5,7,8,2] 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f180409b1a0 0x7f18041370b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-f con 0x7f180409b1a0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 0x7f18041375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-d con 0x7f18040a4220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 0x7f1804120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): picked mon.noname-a con 0x7f1804120b50 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.750 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1804123270 con 0x7f1804120b50 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040a1fd0 con 0x7f180409b1a0 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f18040a8db0 con 0x7f18040a4220 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f180a7fb640 10 monclient(hunting): authenticate will time out at 1996.327093s 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 0x7f18041375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 0x7f1804120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 0x7f18041375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34812/0 (socket says 172.21.15.134:34812) 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 0x7f1804120fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44184/0 (socket says 172.21.15.134:44184) 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 1 -- 172.21.15.134:0/3238241248 learned_addr learned my addr 172.21.15.134:0/3238241248 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): get_auth_request con 0x7f18040a4220 auth_method 0 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.751 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): get_auth_request con 0x7f1804120b50 auth_method 0 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient(hunting): handle_auth_done global_id 124479 payload 824 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1803fff640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f180409b1a0 0x7f18041370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 -- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 msgr2=0x7f18041375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.752 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 0x7f18041375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 -- 172.21.15.134:0/3238241248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f180409b1a0 msgr2=0x7f18041370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f180409b1a0 0x7f18041370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient: _finish_hunting 0 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 monclient: found mon.noname-a 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1808d98640 1 -- 172.21.15.134:0/3238241248 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8009400 con 0x7f1804120b50 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1803fff640 10 monclient: get_auth_request con 0x7f180409b1a0 auth_method 0 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f18037fe640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f18040a4220 0x7f18041375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.738+0000 7f1803fff640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f180409b1a0 0x7f18041370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1808d98640 1 -- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 msgr2=0x7f1804120fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1808d98640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 0x7f1804120fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17fc034000 tx=0x7f17fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17fc007ea0 con 0x7f1804120b50 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: dump: 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.754 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: _finish_auth 0 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0044b0 con 0x7f1804120b50 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 5 monclient: authenticate success, global_id 124479 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 10 monclient: _renew_subs 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f40094d0 con 0x7f1804120b50 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f17fc004620 con 0x7f1804120b50 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.755 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f1800ff9640 10 monclient: dump: 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.756 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 10 monclient: _renew_subs 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f18027fc640 4 set_mon_vals no callback set 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.740+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc009a70 con 0x7f1804120b50 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.741+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f17fc004040 con 0x7f1804120b50 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.741+0000 7f180a7fb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.741+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1804061ae0 con 0x7f1804120b50 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.162:6800/980547263 conn(0x7f17f00388b0 0x7f17f003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f17fc06af80 con 0x7f1804120b50 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.134:6808/443532050 conn(0x7f17f003e3d0 0x7f17f00407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 --> v2:172.21.15.134:6808/443532050 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17f0040e60 con 0x7f17f003e3d0 2024-09-16T15:20:59.757 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17fc0704d0 con 0x7f1804120b50 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1800ff9640 10 monclient: handle_get_version_reply finishing 1 version 861 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1803fff640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.162:6800/980547263 conn(0x7f17f00388b0 0x7f17f003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f18037fe640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.134:6808/443532050 conn(0x7f17f003e3d0 0x7f17f00407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f1803fff640 10 monclient: get_auth_request con 0x7f17f00388b0 auth_method 0 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.742+0000 7f18037fe640 10 monclient: get_auth_request con 0x7f17f003e3d0 auth_method 0 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.744+0000 7f1803fff640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.162:6800/980547263 conn(0x7f17f00388b0 0x7f17f003ad70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009b30 tx=0x7f17f4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.758 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.746+0000 7f18037fe640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.134:6808/443532050 conn(0x7f17f003e3d0 0x7f17f00407f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.759 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.747+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== osd.1 v2:172.21.15.134:6808/443532050 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f17f0040e60 con 0x7f17f003e3d0 2024-09-16T15:20:59.760 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 --> v2:172.21.15.134:6808/443532050 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f180409c960 con 0x7f17f003e3d0 2024-09-16T15:20:59.761 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f1800ff9640 1 -- 172.21.15.134:0/3238241248 <== osd.1 v2:172.21.15.134:6808/443532050 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f180409c960 con 0x7f17f003e3d0 2024-09-16T15:20:59.761 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:20:59.761 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:20:59.761 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 >> v2:172.21.15.134:6808/443532050 conn(0x7f17f003e3d0 msgr2=0x7f17f00407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.134:6808/443532050 conn(0x7f17f003e3d0 0x7f17f00407f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 >> v2:172.21.15.162:6800/980547263 conn(0x7f17f00388b0 msgr2=0x7f17f003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 --2- 172.21.15.134:0/3238241248 >> v2:172.21.15.162:6800/980547263 conn(0x7f17f00388b0 0x7f17f003ad70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009b30 tx=0x7f17f4008040 comp rx=0 tx=0).stop 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 10 monclient: shutdown 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 msgr2=0x7f1804120fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.759+0000 7f180a7fb640 1 --2- 172.21.15.134:0/3238241248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804120b50 0x7f1804120fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17fc034000 tx=0x7f17fc004a40 comp rx=0 tx=0).stop 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.760+0000 7f1808d98640 1 -- 172.21.15.134:0/3238241248 reap_dead start 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.760+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 shutdown_connections 2024-09-16T15:20:59.762 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.760+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 >> 172.21.15.134:0/3238241248 conn(0x7f18040a9820 msgr2=0x7f18041bf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:59.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.760+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 shutdown_connections 2024-09-16T15:20:59.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.760+0000 7f180a7fb640 1 -- 172.21.15.134:0/3238241248 wait complete. 2024-09-16T15:20:59.771 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T15:20:59.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.867+0000 7f9a8b93e640 10 monclient: get_monmap_and_config 2024-09-16T15:20:59.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.867+0000 7f9a8b93e640 10 monclient: build_initial_monmap 2024-09-16T15:20:59.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.867+0000 7f9a8b93e640 10 monclient: monmap: 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:20:59.868201+0000 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:20:59.868201+0000 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:59.869 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:59.870 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:20:59.870 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:20:59.870 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.867+0000 7f9a8b93e640 1 Processor -- start 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 -- start start 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient: init 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient: _add_conns ranks=[7,6,2,5,3,4,0,9,1,8,10] 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 0x7f9a8409c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-i con 0x7f9a84136e40 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-d con 0x7f9a84137220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:20:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-b con 0x7f9a84135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8409abc0 con 0x7f9a84135dc0 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8409ba50 con 0x7f9a84137220 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f9a840a53c0 con 0x7f9a84136e40 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a8b93e640 10 monclient(hunting): authenticate will time out at 1996.457523s 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59058/0 (socket says 172.21.15.134:59058) 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34814/0 (socket says 172.21.15.134:34814) 2024-09-16T15:20:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 1 -- 172.21.15.134:0/2067219608 learned_addr learned my addr 172.21.15.134:0/2067219608 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): get_auth_request con 0x7f9a84135dc0 auth_method 0 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): get_auth_request con 0x7f9a84137220 auth_method 0 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a896da640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 0x7f9a8409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_done global_id 94920 payload 274 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 -- 172.21.15.134:0/2067219608 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 msgr2=0x7f9a8409c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 0x7f9a8409c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 -- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 msgr2=0x7f9a84136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient: _finish_hunting 0 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 monclient: found mon.noname-d 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a88ed9640 1 -- 172.21.15.134:0/2067219608 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84073250 con 0x7f9a84137220 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a896da640 10 monclient: get_auth_request con 0x7f9a84136e40 auth_method 0 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a896da640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 0x7f9a8409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:20:59.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.868+0000 7f9a89edb640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a88ed9640 1 -- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 msgr2=0x7f9a84137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097d0 tx=0x7f9a78029fb0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=3fb460d437b17f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 1 -- 172.21.15.134:0/2067219608 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7802ad10 con 0x7f9a84137220 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: dump: 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.881 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: _finish_auth 0 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: _check_auth_tickets 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 1 -- 172.21.15.134:0/2067219608 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7802ae90 con 0x7f9a84137220 2024-09-16T15:20:59.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a8b93e640 5 monclient: authenticate success, global_id 94920 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a8b93e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 1 -- 172.21.15.134:0/2067219608 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a78030c20 con 0x7f9a84137220 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a83fff640 10 monclient: dump: 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.883 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a8b93e640 10 monclient: get_monmap_and_config success 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.871+0000 7f9a8b93e640 4 set_mon_vals no callback set 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 10 monclient: shutdown 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 -- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 msgr2=0x7f9a84137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097d0 tx=0x7f9a78029fb0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 -- 172.21.15.134:0/2067219608 shutdown_connections 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a84135dc0 0x7f9a84136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f9a84137220 0x7f9a84137600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/2067219608 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f9a84136e40 0x7f9a8409c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 -- 172.21.15.134:0/2067219608 >> 172.21.15.134:0/2067219608 conn(0x7f9a840a6110 msgr2=0x7f9a840a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 -- 172.21.15.134:0/2067219608 shutdown_connections 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.872+0000 7f9a8b93e640 1 -- 172.21.15.134:0/2067219608 wait complete. 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient: build_initial_monmap 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient: monmap: 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.885 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 Processor -- start 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 -- start start 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient: init 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient: _reopen_session rank -1 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient: _add_conns ranks=[0,3,5,6,7,1,4,2,8] 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-a con 0x7f9a84135dc0 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a84136e40 0x7f9a841352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-g con 0x7f9a84136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9a84137220 0x7f9a84121890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): picked mon.noname-h con 0x7f9a84137220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a840a8a70 con 0x7f9a84135dc0 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841251a0 con 0x7f9a84136e40 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): start opening mon connection 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a84124c20 con 0x7f9a84137220 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): _renew_subs 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.873+0000 7f9a8b93e640 10 monclient(hunting): authenticate will time out at 1996.462929s 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44200/0 (socket says 172.21.15.134:44200) 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 -- 172.21.15.134:0/895711897 learned_addr learned my addr 172.21.15.134:0/895711897 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): get_auth_request con 0x7f9a84135dc0 auth_method 0 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9a84137220 0x7f9a84121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): get_auth_request con 0x7f9a84137220 auth_method 0 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a84136e40 0x7f9a841352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): get_auth_request con 0x7f9a84136e40 auth_method 0 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a89edb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a88ed9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient(hunting): handle_auth_done global_id 124488 payload 824 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 -- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9a84137220 msgr2=0x7f9a84121890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9a84137220 0x7f9a84121890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 -- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a84136e40 msgr2=0x7f9a841352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:20:59.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a84136e40 0x7f9a841352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient: _finish_hunting 0 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 monclient: found mon.noname-a 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.874+0000 7f9a896da640 1 -- 172.21.15.134:0/895711897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009b30 con 0x7f9a84135dc0 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a896da640 1 -- 172.21.15.134:0/895711897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 msgr2=0x7f9a84134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a896da640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a84124df0 tx=0x7f9a7c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7c00ce10 con 0x7f9a84135dc0 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: dump: 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.890 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: _finish_auth 0 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: _check_auth_tickets 2024-09-16T15:20:59.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0113f0 con 0x7f9a84135dc0 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 5 monclient: authenticate success, global_id 124488 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a7c0119d0 con 0x7f9a84135dc0 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 10 monclient: _renew_subs 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78009400 con 0x7f9a84135dc0 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a81ffb640 10 monclient: dump: 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:20:59.892 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a837fe640 4 set_mon_vals no callback set 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 10 monclient: _renew_subs 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.876+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a84125480 con 0x7f9a84135dc0 2024-09-16T15:20:59.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f9a7c011560 con 0x7f9a84135dc0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a81ffb640 1 --2- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 0x7f9a7003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f9a7c04a030 con 0x7f9a84135dc0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a8b93e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a8b93e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a8b93e640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.877+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a841258a0 con 0x7f9a84135dc0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.878+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 0x7f9a7003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.878+0000 7f9a88ed9640 10 monclient: get_auth_request con 0x7f9a70038880 auth_method 0 2024-09-16T15:20:59.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.880+0000 7f9a88ed9640 1 --2- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 0x7f9a7003ad40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097a0 tx=0x7f9a7802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:20:59.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.882+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f9a841258a0 con 0x7f9a84135dc0 2024-09-16T15:20:59.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.882+0000 7f9a81ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:20:59.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:20:59.882+0000 7f9a81ffb640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:20:59.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4264151985}) 2024-09-16T15:20:59.986 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2520235988}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2746808098}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1957353178}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3904284948}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2061429894}) 2024-09-16T15:20:59.987 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2022214723}) 2024-09-16T15:20:59.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.984+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3871887772}) 2024-09-16T15:20:59.988 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.985+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1405985431}) 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=854675399}) 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.989 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3168186134}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1914253859}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1826695126}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3591780094}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2386350681}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.990 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4093148613}) 2024-09-16T15:20:59.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:20:59.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2774251550}) 2024-09-16T15:20:59.991 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:20:59.986+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2144758184}) 2024-09-16T15:21:00.047 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.046+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a84061ae0 con 0x7f9a70038880 2024-09-16T15:21:00.048 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.046+0000 7f9a81ffb640 1 -- 172.21.15.134:0/895711897 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9a780041f0 con 0x7f9a70038880 2024-09-16T15:21:00.048 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 msgr2=0x7f9a7003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 0x7f9a7003ad40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a780097a0 tx=0x7f9a7802e000 comp rx=0 tx=0).stop 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 10 monclient: shutdown 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 msgr2=0x7f9a84134d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.050 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a84124df0 tx=0x7f9a7c00cbc0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 shutdown_connections 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f9a84137220 0x7f9a84121890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> v2:172.21.15.162:6800/980547263 conn(0x7f9a70038880 0x7f9a7003ad40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a84136e40 0x7f9a841352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 --2- 172.21.15.134:0/895711897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a84135dc0 0x7f9a84134d70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 >> 172.21.15.134:0/895711897 conn(0x7f9a840a6110 msgr2=0x7f9a8406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 shutdown_connections 2024-09-16T15:21:00.051 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.048+0000 7f9a8b93e640 1 -- 172.21.15.134:0/895711897 wait complete. 2024-09-16T15:21:00.060 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T15:21:00.060 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.159+0000 7f54b56d7640 10 monclient: get_monmap_and_config 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.159+0000 7f54b56d7640 10 monclient: build_initial_monmap 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.159+0000 7f54b56d7640 10 monclient: monmap: 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:00.160480+0000 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:00.160480+0000 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:00.161 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 Processor -- start 2024-09-16T15:21:00.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 -- start start 2024-09-16T15:21:00.163 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient: init 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient: _add_conns ranks=[9,8,1,7,10,2,4,6,0,3,5] 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 0x7f54b00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-a con 0x7f54b00a4220 addr v2:172.21.15.134:3300/0 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-e con 0x7f54b009c720 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f54b009b1a0 addr v1:172.21.15.162:6789/0 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b004e730 con 0x7f54b009b1a0 2024-09-16T15:21:00.166 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b009c390 con 0x7f54b00a4220 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00a5080 con 0x7f54b009c720 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54b56d7640 10 monclient(hunting): authenticate will time out at 1996.749647s 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 0x7f54b00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:35448/0 (socket says 172.21.15.134:35448) 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 1 -- 172.21.15.134:0/3076786612 learned_addr learned my addr 172.21.15.134:0/3076786612 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): get_auth_request con 0x7f54b009c720 auth_method 0 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): get_auth_request con 0x7f54b00a4220 auth_method 0 2024-09-16T15:21:00.167 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.160+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_done global_id 85202 payload 274 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 1 -- 172.21.15.134:0/3076786612 >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 msgr2=0x7f54b00a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 1 --2- 172.21.15.134:0/3076786612 >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 0x7f54b00a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 1 -- 172.21.15.134:0/3076786612 >> v1:172.21.15.162:6789/0 conn(0x7f54b009b1a0 legacy=0x7f54b0136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:21:00.168 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 1 monclient: found mon.noname-e 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54ae7fc640 1 -- 172.21.15.134:0/3076786612 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00a5e20 con 0x7f54b009c720 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.161+0000 7f54aeffd640 1 --2- 172.21.15.134:0/3076786612 >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 0x7f54b00a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.163+0000 7f54ae7fc640 1 -- 172.21.15.134:0/3076786612 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 msgr2=0x7f54b00a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.163+0000 7f54ae7fc640 1 --2- 172.21.15.134:0/3076786612 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f54a40098a0 tx=0x7f54a402a110 comp rx=0 tx=0).ready entity=mon.8 client_cookie=51be656f8ef2bb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.163+0000 7f54adffb640 1 -- 172.21.15.134:0/3076786612 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54a402ae30 con 0x7f54b009c720 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: dump: 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.169 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: _finish_auth 0 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: _check_auth_tickets 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:00.170 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 1 -- 172.21.15.134:0/3076786612 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a40307b0 con 0x7f54b009c720 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 5 monclient: authenticate success, global_id 85202 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 1 -- 172.21.15.134:0/3076786612 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54a4030d90 con 0x7f54b009c720 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 4 set_mon_vals no callback set 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54adffb640 10 monclient: dump: 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.171 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 10 monclient: shutdown 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:00.172 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 -- 172.21.15.134:0/3076786612 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 msgr2=0x7f54b00a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 --2- 172.21.15.134:0/3076786612 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f54a40098a0 tx=0x7f54a402a110 comp rx=0 tx=0).stop 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 -- 172.21.15.134:0/3076786612 shutdown_connections 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 --2- 172.21.15.134:0/3076786612 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009c720 0x7f54b00a4b40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 --2- 172.21.15.134:0/3076786612 >> v2:172.21.15.134:3300/0 conn(0x7f54b00a4220 0x7f54b00a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 -- 172.21.15.134:0/3076786612 >> 172.21.15.134:0/3076786612 conn(0x7f54b00a9820 msgr2=0x7f54b00a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 -- 172.21.15.134:0/3076786612 shutdown_connections 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.164+0000 7f54b56d7640 1 -- 172.21.15.134:0/3076786612 wait complete. 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.165+0000 7f54b56d7640 10 monclient: build_initial_monmap 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.165+0000 7f54b56d7640 10 monclient: monmap: 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:00.173 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:00.174 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.165+0000 7f54b56d7640 1 Processor -- start 2024-09-16T15:21:00.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.165+0000 7f54b56d7640 1 -- start start 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.165+0000 7f54b56d7640 10 monclient: init 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient: _add_conns ranks=[8,0,1,6,4,2,5,7,3] 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 0x7f54b01cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-e con 0x7f54b009b1a0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 0x7f54b01371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-a con 0x7f54b00a4220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0137700 0x7f54b0120d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): picked mon.noname-f con 0x7f54b0137700 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01242e0 con 0x7f54b00a4220 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00a21a0 con 0x7f54b0137700 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0137b60 con 0x7f54b009b1a0 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54b56d7640 10 monclient(hunting): authenticate will time out at 1996.755189s 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 0x7f54b01cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 0x7f54b01371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 0x7f54b01cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:35460/0 (socket says 172.21.15.134:35460) 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 1 -- 172.21.15.134:0/517075646 learned_addr learned my addr 172.21.15.134:0/517075646 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): get_auth_request con 0x7f54b009b1a0 auth_method 0 2024-09-16T15:21:00.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): get_auth_request con 0x7f54b00a4220 auth_method 0 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient(hunting): handle_auth_done global_id 124506 payload 824 2024-09-16T15:21:00.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 -- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 msgr2=0x7f54b01cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 0x7f54b01cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 -- 172.21.15.134:0/517075646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0137700 msgr2=0x7f54b0120d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54af7fe640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0137700 0x7f54b0120d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0137700 0x7f54b0120d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient: _finish_hunting 0 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 monclient: found mon.noname-a 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54ae7fc640 1 -- 172.21.15.134:0/517075646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a0009400 con 0x7f54b00a4220 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54af7fe640 10 monclient: get_auth_request con 0x7f54b0137700 auth_method 0 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54aeffd640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f54b009b1a0 0x7f54b01cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.166+0000 7f54af7fe640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54b0137700 0x7f54b0120d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.168+0000 7f54ae7fc640 1 -- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 msgr2=0x7f54b01371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.168+0000 7f54ae7fc640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 0x7f54b01371c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54a4031000 tx=0x7f54a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.182 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.168+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54a4004530 con 0x7f54b00a4220 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.168+0000 7f54977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.168+0000 7f54977fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: dump: 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.183 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: _finish_auth 0 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 5 monclient: authenticate success, global_id 124506 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4030d40 con 0x7f54b00a4220 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f54a403d450 con 0x7f54b00a4220 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 10 monclient: _renew_subs 2024-09-16T15:21:00.184 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a40094d0 con 0x7f54b00a4220 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 10 monclient: dump: 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.185 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54ad7fa640 4 set_mon_vals no callback set 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 10 monclient: _renew_subs 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b00a1e70 con 0x7f54b00a4220 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f54a4039070 con 0x7f54b00a4220 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.169+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54b0061ae0 con 0x7f54b00a4220 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.162:6800/980547263 conn(0x7f549c0388b0 0x7f549c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f54a406a840 con 0x7f54b00a4220 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.134:6800/1264438958 conn(0x7f549c03e3d0 0x7f549c0407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 --> v2:172.21.15.134:6800/1264438958 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f549c040e60 con 0x7f549c03e3d0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a406cd60 con 0x7f54b00a4220 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54977fe640 10 monclient: handle_get_version_reply finishing 1 version 861 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54af7fe640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.134:6800/1264438958 conn(0x7f549c03e3d0 0x7f549c0407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54aeffd640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.162:6800/980547263 conn(0x7f549c0388b0 0x7f549c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54af7fe640 10 monclient: get_auth_request con 0x7f549c03e3d0 auth_method 0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.170+0000 7f54aeffd640 10 monclient: get_auth_request con 0x7f549c0388b0 auth_method 0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.172+0000 7f54aeffd640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.162:6800/980547263 conn(0x7f549c0388b0 0x7f549c03ad70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f54a00114b0 tx=0x7f54a0009190 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.174+0000 7f54af7fe640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.134:6800/1264438958 conn(0x7f549c03e3d0 0x7f549c0407f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.175+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== osd.2 v2:172.21.15.134:6800/1264438958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f549c040e60 con 0x7f549c03e3d0 2024-09-16T15:21:00.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 --> v2:172.21.15.134:6800/1264438958 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f54b007a680 con 0x7f549c03e3d0 2024-09-16T15:21:00.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54977fe640 1 -- 172.21.15.134:0/517075646 <== osd.2 v2:172.21.15.134:6800/1264438958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f54a80041d0 con 0x7f549c03e3d0 2024-09-16T15:21:00.189 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:21:00.189 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:21:00.189 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 >> v2:172.21.15.134:6800/1264438958 conn(0x7f549c03e3d0 msgr2=0x7f549c0407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.134:6800/1264438958 conn(0x7f549c03e3d0 0x7f549c0407f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 >> v2:172.21.15.162:6800/980547263 conn(0x7f549c0388b0 msgr2=0x7f549c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 --2- 172.21.15.134:0/517075646 >> v2:172.21.15.162:6800/980547263 conn(0x7f549c0388b0 0x7f549c03ad70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f54a00114b0 tx=0x7f54a0009190 comp rx=0 tx=0).stop 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 10 monclient: shutdown 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 msgr2=0x7f54b01371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 --2- 172.21.15.134:0/517075646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54b00a4220 0x7f54b01371c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54a4031000 tx=0x7f54a40046c0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54af7fe640 1 -- 172.21.15.134:0/517075646 reap_dead start 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 shutdown_connections 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 >> 172.21.15.134:0/517075646 conn(0x7f54b00a9820 msgr2=0x7f54b01bf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.187+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 shutdown_connections 2024-09-16T15:21:00.190 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.188+0000 7f54b56d7640 1 -- 172.21.15.134:0/517075646 wait complete. 2024-09-16T15:21:00.199 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T15:21:00.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.294+0000 7f57a619c640 10 monclient: get_monmap_and_config 2024-09-16T15:21:00.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.294+0000 7f57a619c640 10 monclient: build_initial_monmap 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.294+0000 7f57a619c640 10 monclient: monmap: 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:00.295383+0000 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:00.295383+0000 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:00.296 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:00.297 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:00.297 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:00.297 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:00.297 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.298 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 Processor -- start 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 -- start start 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient: init 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient: _add_conns ranks=[2,6,0,4,7,10,9,1,8,5,3] 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57a0136eb0 0x7f57a009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-b con 0x7f57a0136eb0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 0x7f57a0137690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-d con 0x7f57a0137290 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f57a0133d10 addr v1:172.21.15.134:6789/0 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a0073250 con 0x7f57a0133d10 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00a5560 con 0x7f57a0136eb0 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.299 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57a009b240 con 0x7f57a0137290 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f57a619c640 10 monclient(hunting): authenticate will time out at 1996.884707s 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57a0136eb0 0x7f57a009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 0x7f57a0137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579ffff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f57a0133d10 0x7f57a01341c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59262/0 (socket says 172.21.15.134:59262) 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579ffff640 1 -- 172.21.15.134:0/3197950434 learned_addr learned my addr 172.21.15.134:0/3197950434 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): get_auth_request con 0x7f57a0137290 auth_method 0 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): get_auth_request con 0x7f57a0136eb0 auth_method 0 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.295+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 10 monclient(hunting): handle_auth_done global_id 94929 payload 274 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 1 -- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57a0136eb0 msgr2=0x7f57a009bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 1 --2- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57a0136eb0 0x7f57a009bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 1 -- 172.21.15.134:0/3197950434 >> v1:172.21.15.134:6789/0 conn(0x7f57a0133d10 legacy=0x7f57a01341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 10 monclient: _finish_hunting 0 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 1 monclient: found mon.noname-d 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:21:00.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.296+0000 7f579effd640 1 -- 172.21.15.134:0/3197950434 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a009b520 con 0x7f57a0137290 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.298+0000 7f579effd640 1 -- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 msgr2=0x7f57a0137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.298+0000 7f579effd640 1 --2- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 0x7f57a0137690 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57940097d0 tx=0x7f579402a000 comp rx=0 tx=0).ready entity=mon.6 client_cookie=34dbb688b5eb981f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 1 -- 172.21.15.134:0/3197950434 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f579402ad20 con 0x7f57a0137290 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: dump: 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.306 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: _finish_auth 0 2024-09-16T15:21:00.307 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 1 -- 172.21.15.134:0/3197950434 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940307b0 con 0x7f57a0137290 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 5 monclient: authenticate success, global_id 94929 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 1 -- 172.21.15.134:0/3197950434 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f579402aea0 con 0x7f57a0137290 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 4 set_mon_vals no callback set 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f579e7fc640 10 monclient: dump: 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.308 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 10 monclient: shutdown 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 -- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 msgr2=0x7f57a0137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.309 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 --2- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 0x7f57a0137690 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57940097d0 tx=0x7f579402a000 comp rx=0 tx=0).stop 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 -- 172.21.15.134:0/3197950434 shutdown_connections 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 --2- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0137290 0x7f57a0137690 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 --2- 172.21.15.134:0/3197950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57a0136eb0 0x7f57a009bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 -- 172.21.15.134:0/3197950434 >> 172.21.15.134:0/3197950434 conn(0x7f57a00a99a0 msgr2=0x7f57a00a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 -- 172.21.15.134:0/3197950434 shutdown_connections 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.299+0000 7f57a619c640 1 -- 172.21.15.134:0/3197950434 wait complete. 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.300+0000 7f57a619c640 10 monclient: build_initial_monmap 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.300+0000 7f57a619c640 10 monclient: monmap: 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:00.310 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.300+0000 7f57a619c640 1 Processor -- start 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.300+0000 7f57a619c640 1 -- start start 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.300+0000 7f57a619c640 10 monclient: init 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient: _add_conns ranks=[6,8,0,4,2,3,1,7,5] 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-d con 0x7f57a0133d10 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:00.311 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57a0136eb0 0x7f57a00a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-e con 0x7f57a0136eb0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57a0137290 0x7f57a00a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): picked mon.noname-a con 0x7f57a0137290 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0120590 con 0x7f57a0137290 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01207e0 con 0x7f57a0133d10 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:00.312 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01363c0 con 0x7f57a0136eb0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f57a619c640 10 monclient(hunting): authenticate will time out at 1996.890260s 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57a0137290 0x7f57a00a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34832/0 (socket says 172.21.15.134:34832) 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 -- 172.21.15.134:0/3773298615 learned_addr learned my addr 172.21.15.134:0/3773298615 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): get_auth_request con 0x7f57a0133d10 auth_method 0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57a0136eb0 0x7f57a00a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): get_auth_request con 0x7f57a0137290 auth_method 0 2024-09-16T15:21:00.313 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): get_auth_request con 0x7f57a0136eb0 auth_method 0 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:00.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient(hunting): handle_auth_done global_id 94938 payload 824 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 -- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57a0136eb0 msgr2=0x7f57a00a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57a0136eb0 0x7f57a00a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 -- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57a0137290 msgr2=0x7f57a00a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57a0137290 0x7f57a00a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient: _finish_hunting 0 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 monclient: found mon.noname-d 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.301+0000 7f579f7fe640 1 -- 172.21.15.134:0/3773298615 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794009400 con 0x7f57a0133d10 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.303+0000 7f579f7fe640 1 -- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 msgr2=0x7f57a0132d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.303+0000 7f579f7fe640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f579000e9e0 tx=0x7f579000eea0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5790009670 con 0x7f57a0133d10 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:21:00.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: dump: 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.316 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: _finish_auth 0 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: _check_auth_tickets 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000cbe0 con 0x7f57a0133d10 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 5 monclient: authenticate success, global_id 94938 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _renew_subs 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00a7b40 con 0x7f57a0133d10 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5790013d50 con 0x7f57a0133d10 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 10 monclient: dump: 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:00.317 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _renew_subs 2024-09-16T15:21:00.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00a22c0 con 0x7f57a0133d10 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f579dffb640 4 set_mon_vals no callback set 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f579f7fe640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f579000e9e0 tx=0x7f579000eea0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7f579004a2d0 osd_map(861..861 src has 1..861) v4 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f579000cd50 con 0x7f57a0133d10 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.304+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0120060 con 0x7f57a0133d10 2024-09-16T15:21:00.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.305+0000 7f5787fff640 1 --2- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 0x7f578c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:00.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.305+0000 7f579effd640 1 --2- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 0x7f578c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:00.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.305+0000 7f579effd640 10 monclient: get_auth_request con 0x7f578c038880 auth_method 0 2024-09-16T15:21:00.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.308+0000 7f579effd640 1 --2- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 0x7f578c03ad40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5794034000 tx=0x7f579402e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:00.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:00.309+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f579004a2d0 con 0x7f57a0133d10 2024-09-16T15:21:00.483 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.483 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.483 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2784907013}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1054534947}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3132453835}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1991601271}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=428153545}) 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.484 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2291941422}) 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3352046096}) 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2105150244}) 2024-09-16T15:21:00.485 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3569902331}) 2024-09-16T15:21:00.492 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.492 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.492 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.482+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3991208549}) 2024-09-16T15:21:00.492 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.492 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3202620637}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2057488353}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=417239145}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=378734347}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.493 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2776215562}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=932807165}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1536592993}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.483+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3608504544}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2831063857}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794115920}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4233140160}) 2024-09-16T15:21:00.494 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3700234623}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2440646024}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2112940425}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2014513067}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1344916386}) 2024-09-16T15:21:00.495 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.484+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2019635857}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1452970956}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=675811128}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2175227357}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3951741909}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.496 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3636357731}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1618831692}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1115542602}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1404006363}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.485+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=801458879}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.486+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.486+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.486+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1388180631}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.486+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.497 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3784951166}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=488450858}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4208487450}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1260355712}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1858046769}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=652751938}) 2024-09-16T15:21:00.498 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2957187662}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.487+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1866774077}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2819279833}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3881057599}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=579558495}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1508911861}) 2024-09-16T15:21:00.499 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2433436598}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=781198428}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3636312210}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557148257}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.488+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3239019571}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2809777804}) 2024-09-16T15:21:00.500 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3284659517}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3545109122}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2522631220}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4269257101}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=297697613}) 2024-09-16T15:21:00.501 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1351774517}) 2024-09-16T15:21:00.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3708847184}) 2024-09-16T15:21:00.502 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.489+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1523221900}) 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.675+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1497811773}) 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2132071395}) 2024-09-16T15:21:00.678 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2695605326}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=158515624}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=991526430}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1002025217}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.679 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1433878539}) 2024-09-16T15:21:00.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4180145413}) 2024-09-16T15:21:00.680 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.676+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3629225691}) 2024-09-16T15:21:00.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.684 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3100577795}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3017221874}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=482110658}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2058883801}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=266655633}) 2024-09-16T15:21:00.685 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3562201895}) 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3803619498}) 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3243591893}) 2024-09-16T15:21:00.686 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.682+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1810392513}) 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3664499144}) 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2144864263}) 2024-09-16T15:21:00.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2972418217}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=571976792}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4250496991}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3266437620}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=276883056}) 2024-09-16T15:21:00.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:00.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1413755181}) 2024-09-16T15:21:00.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:00.711+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2664353338}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540688072}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1201026347}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=910860059}) 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.124 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1494325982}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3779639966}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=481137524}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2906611366}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=188767586}) 2024-09-16T15:21:01.125 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.122+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2285362445}) 2024-09-16T15:21:01.127 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=121922530}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1494073750}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3359708681}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=425608608}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.128 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1763846033}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3744326444}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370131251}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432290204}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.123+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1177311430}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4202533627}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.129 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2717278576}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=727237067}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3094129690}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=144571796}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.124+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1816529666}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2300012266}) 2024-09-16T15:21:01.130 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.131 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4050387187}) 2024-09-16T15:21:01.131 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.125+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4269742996}) 2024-09-16T15:21:01.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.132+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f579001b9a0 con 0x7f57a0133d10 2024-09-16T15:21:01.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.132+0000 7f5787fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:01.133 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.132+0000 7f5787fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2154229890}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3143836657}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2911713179}) 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.137 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1991635339}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3193444103}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2462443347}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3833407250}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3488551214}) 2024-09-16T15:21:01.138 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.135+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=269825728}) 2024-09-16T15:21:01.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1888448314}) 2024-09-16T15:21:01.150 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=521079916}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2933634025}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3756388642}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.136+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1693213537}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1427133777}) 2024-09-16T15:21:01.151 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3773501088}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3838062874}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.137+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1799080896}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1263315909}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1999758640}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3215059293}) 2024-09-16T15:21:01.152 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3941527367}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3549021322}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1394952793}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1096455035}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968388564}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.138+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3127812786}) 2024-09-16T15:21:01.153 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1965857183}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3668459262}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=497611845}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=418064485}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3565317810}) 2024-09-16T15:21:01.154 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2670123349}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1161347933}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065580224}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.140+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2789001274}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1921656665}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4203385966}) 2024-09-16T15:21:01.155 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2002395738}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3569758719}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1886674844}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3221006858}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=334236316}) 2024-09-16T15:21:01.156 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1269843350}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.141+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2038064752}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4219594784}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1321650868}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1938042644}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.157 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1378229763}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3165653414}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=309316482}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=294021190}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1674986828}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.143+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4054625856}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.158 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=90607551}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=855763780}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1782343459}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1550845094}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.144+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2057284235}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.159 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3994058898}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3407821558}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2546045940}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.145+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4258625390}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4101784119}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=205191002}) 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.160 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3625376223}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=459176490}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1095300156}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2015187506}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1462609853}) 2024-09-16T15:21:01.161 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=492261533}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.146+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2457378800}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2769673042}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1566339394}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1980662521}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1202125120}) 2024-09-16T15:21:01.162 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556017969}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1978081684}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4223006228}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1087755270}) 2024-09-16T15:21:01.163 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.147+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2251214065}) 2024-09-16T15:21:01.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.300+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a0061ae0 con 0x7f578c038880 2024-09-16T15:21:01.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.300+0000 7f5787fff640 1 -- 172.21.15.134:0/3773298615 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5794004040 con 0x7f578c038880 2024-09-16T15:21:01.302 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T15:21:01.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 msgr2=0x7f578c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 0x7f578c03ad40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5794034000 tx=0x7f579402e000 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 10 monclient: shutdown 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 msgr2=0x7f57a0132d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f579000e9e0 tx=0x7f579000eea0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 shutdown_connections 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> v2:172.21.15.162:6800/980547263 conn(0x7f578c038880 0x7f578c03ad40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57a0137290 0x7f57a00a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f57a0136eb0 0x7f57a00a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 --2- 172.21.15.134:0/3773298615 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f57a0133d10 0x7f57a0132d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 >> 172.21.15.134:0/3773298615 conn(0x7f57a00a99a0 msgr2=0x7f57a006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 shutdown_connections 2024-09-16T15:21:01.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.303+0000 7f57a619c640 1 -- 172.21.15.134:0/3773298615 wait complete. 2024-09-16T15:21:01.314 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T15:21:01.315 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:21:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.414+0000 7fc8b5412640 10 monclient: get_monmap_and_config 2024-09-16T15:21:01.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.414+0000 7fc8b5412640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.414+0000 7fc8b5412640 10 monclient: monmap: 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:01.415371+0000 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:01.415371+0000 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.416 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:01.417 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.420 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 Processor -- start 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 -- start start 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient: init 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient: _add_conns ranks=[9,4,10,3,5,6,8,2,7,1,0] 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc8b00a4220 0x7fc8b00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-a con 0x7fc8b00a4220 addr v2:172.21.15.134:3300/0 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-c con 0x7fc8b009c720 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 0x7fc8b0136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-f con 0x7fc8b009b1a0 addr v2:172.21.15.162:3300/0 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b009c390 con 0x7fc8b00a4220 2024-09-16T15:21:01.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b00a5080 con 0x7fc8b009b1a0 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00a5e20 con 0x7fc8b009c720 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8b5412640 10 monclient(hunting): authenticate will time out at 1998.004637s 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc8b00a4220 0x7fc8b00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33454/0 (socket says 172.21.15.134:33454) 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc8b00a4220 0x7fc8b00a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44244/0 (socket says 172.21.15.134:44244) 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 1 -- 172.21.15.134:0/3007279902 learned_addr learned my addr 172.21.15.134:0/3007279902 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): get_auth_request con 0x7fc8b009c720 auth_method 0 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request con 0x7fc8b00a4220 auth_method 0 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8ae7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.415+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_done global_id 124524 payload 274 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 -- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 msgr2=0x7fc8b00a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 -- 172.21.15.134:0/3007279902 >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 msgr2=0x7fc8b0136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/3007279902 >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 0x7fc8b0136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.423 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/3007279902 >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 0x7fc8b0136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 monclient: found mon.noname-a 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8aeffd640 1 -- 172.21.15.134:0/3007279902 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b004e730 con 0x7fc8b00a4220 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8af7fe640 10 monclient: get_auth_request con 0x7fc8b009b1a0 auth_method 0 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8ae7fc640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.416+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/3007279902 >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 0x7fc8b0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.418+0000 7fc8aeffd640 1 -- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b00a4220 msgr2=0x7fc8b00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.418+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b00a4220 0x7fc8b00a4600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0071a60 tx=0x7fc8a0029fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c1d6ba0f29744 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 1 -- 172.21.15.134:0/3007279902 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a002ad40 con 0x7fc8b00a4220 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:01.424 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: dump: 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.425 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 5 monclient: authenticate success, global_id 124524 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 1 -- 172.21.15.134:0/3007279902 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a00307b0 con 0x7fc8b00a4220 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 1 -- 172.21.15.134:0/3007279902 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a0030d90 con 0x7fc8b00a4220 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.426 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 4 set_mon_vals no callback set 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8adffb640 10 monclient: dump: 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.427 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 10 monclient: shutdown 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 -- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b00a4220 msgr2=0x7fc8b00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b00a4220 0x7fc8b00a4600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0071a60 tx=0x7fc8a0029fe0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 -- 172.21.15.134:0/3007279902 shutdown_connections 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 --2- 172.21.15.134:0/3007279902 >> v2:172.21.15.162:3300/0 conn(0x7fc8b009b1a0 0x7fc8b0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009c720 0x7fc8b00a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 --2- 172.21.15.134:0/3007279902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b00a4220 0x7fc8b00a4600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 -- 172.21.15.134:0/3007279902 >> 172.21.15.134:0/3007279902 conn(0x7fc8b00a9820 msgr2=0x7fc8b00a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.428 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 -- 172.21.15.134:0/3007279902 shutdown_connections 2024-09-16T15:21:01.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.419+0000 7fc8b5412640 1 -- 172.21.15.134:0/3007279902 wait complete. 2024-09-16T15:21:01.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.420+0000 7fc8b5412640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.420+0000 7fc8b5412640 10 monclient: monmap: 2024-09-16T15:21:01.433 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.433 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.420+0000 7fc8b5412640 1 Processor -- start 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.420+0000 7fc8b5412640 1 -- start start 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.420+0000 7fc8b5412640 10 monclient: init 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient: _add_conns ranks=[4,5,2,0,3,6,1,8,7] 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-c con 0x7fc8b009b1a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b00a4220 0x7fc8b0120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-h con 0x7fc8b00a4220 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0121090 0x7fc8b01214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): picked mon.noname-b con 0x7fc8b0121090 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b01cc380 con 0x7fc8b0121090 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00a2080 con 0x7fc8b009b1a0 2024-09-16T15:21:01.435 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0137e70 con 0x7fc8b00a4220 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8b5412640 10 monclient(hunting): authenticate will time out at 1998.010357s 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33470/0 (socket says 172.21.15.134:33470) 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 -- 172.21.15.134:0/507912897 learned_addr learned my addr 172.21.15.134:0/507912897 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request con 0x7fc8b009b1a0 auth_method 0 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0121090 0x7fc8b01214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.436 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request con 0x7fc8b0121090 auth_method 0 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b00a4220 0x7fc8b0120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 10 monclient(hunting): get_auth_request con 0x7fc8b00a4220 auth_method 0 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient(hunting): handle_auth_done global_id 124501 payload 824 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 -- 172.21.15.134:0/507912897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b00a4220 msgr2=0x7fc8b0120b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b00a4220 0x7fc8b0120b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.437 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 -- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0121090 msgr2=0x7fc8b01214f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8b0121090 0x7fc8b01214f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 monclient: found mon.noname-c 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8aeffd640 1 -- 172.21.15.134:0/507912897 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40094d0 con 0x7fc8b009b1a0 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.421+0000 7fc8ae7fc640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7fc8b00a4220 0x7fc8b0120b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.423+0000 7fc8aeffd640 1 -- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 msgr2=0x7fc8b0137270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.423+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0035040 tx=0x7fc8a00046c0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a0030d40 con 0x7fc8b009b1a0 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:21:01.438 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: dump: 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.439 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 5 monclient: authenticate success, global_id 124501 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0003a80 con 0x7fc8b009b1a0 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 10 monclient: _renew_subs 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a0009400 con 0x7fc8b009b1a0 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc8a0003bf0 con 0x7fc8b009b1a0 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:21:01.440 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8977fe640 10 monclient: dump: 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.441 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 10 monclient: _renew_subs 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b00a8e50 con 0x7fc8b009b1a0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.424+0000 7fc8ad7fa640 4 set_mon_vals no callback set 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8aeffd640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0035040 tx=0x7fc8a00046c0 comp rx=0 tx=0).queue_received will delay after 1.7265e+09 on 0x7fc8a006a5d0 osd_map(861..861 src has 1..861) v4 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fc8a0039040 con 0x7fc8b009b1a0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8b5412640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.134:3302/0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8b0061ae0 con 0x7fc8b009b1a0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8977fe640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fc8a006a5d0 con 0x7fc8b009b1a0 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8977fe640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc89c03e3d0 0x7fc89c0407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.442 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 --> v2:172.21.15.162:6805/3922313654 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc89c040e60 con 0x7fc89c03e3d0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8ae7fc640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8ae7fc640 10 monclient: get_auth_request con 0x7fc89c0388b0 auth_method 0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc89c03e3d0 0x7fc89c0407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.425+0000 7fc8af7fe640 10 monclient: get_auth_request con 0x7fc89c03e3d0 auth_method 0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.428+0000 7fc8ae7fc640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0066ff0 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.430+0000 7fc8af7fe640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc89c03e3d0 0x7fc89c0407f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.443 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.431+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== osd.3 v2:172.21.15.162:6805/3922313654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc89c040e60 con 0x7fc89c03e3d0 2024-09-16T15:21:01.444 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.443+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 --> v2:172.21.15.162:6805/3922313654 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc8b007a680 con 0x7fc89c03e3d0 2024-09-16T15:21:01.445 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.443+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== osd.3 v2:172.21.15.162:6805/3922313654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc8a8007cd0 con 0x7fc89c03e3d0 2024-09-16T15:21:01.445 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:21:01.445 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:21:01.445 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc89c03e3d0 msgr2=0x7fc89c0407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6805/3922313654 conn(0x7fc89c03e3d0 0x7fc89c0407f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 msgr2=0x7fc89c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 --2- 172.21.15.134:0/507912897 >> v2:172.21.15.162:6800/980547263 conn(0x7fc89c0388b0 0x7fc89c03ad70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0066ff0 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 10 monclient: shutdown 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 20 monclient: shutdown canceling and discarding version request 1 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 msgr2=0x7fc8b0137270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.446 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 --2- 172.21.15.134:0/507912897 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fc8b009b1a0 0x7fc8b0137270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0035040 tx=0x7fc8a00046c0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8af7fe640 1 -- 172.21.15.134:0/507912897 reap_dead start 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8977fe640 1 -- 172.21.15.134:0/507912897 <== mon.4 v2:172.21.15.134:3302/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (unknown 0 0 0) 0x7fc8a006ca60 con 0x7fc8b009b1a0 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8977fe640 10 monclient: discarding stray monitor message mon_get_version_reply(handle=1 version=861) v2 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 shutdown_connections 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 >> 172.21.15.134:0/507912897 conn(0x7fc8b00a9820 msgr2=0x7fc8b00a7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 shutdown_connections 2024-09-16T15:21:01.447 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.444+0000 7fc8b5412640 1 -- 172.21.15.134:0/507912897 wait complete. 2024-09-16T15:21:01.455 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T15:21:01.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.551+0000 7f72e6183640 10 monclient: get_monmap_and_config 2024-09-16T15:21:01.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.551+0000 7f72e6183640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.552 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.551+0000 7f72e6183640 10 monclient: monmap: 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:01.552267+0000 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:01.552267+0000 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.553 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:01.554 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:01.554 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 Processor -- start 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 -- start start 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient: init 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient: _add_conns ranks=[10,3,9,2,6,7,4,1,0,8,5] 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f72e0136e40 0x7f72e009c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-f con 0x7f72e0136e40 addr v2:172.21.15.162:3300/0 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0137220 0x7f72e0137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-g con 0x7f72e0137220 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f72e0135dc0 0x7f72e0136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-a con 0x7f72e0135dc0 addr v2:172.21.15.134:3300/0 2024-09-16T15:21:01.556 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e009abc0 con 0x7f72e0135dc0 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e009ba50 con 0x7f72e0136e40 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00a53c0 con 0x7f72e0137220 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72e6183640 10 monclient(hunting): authenticate will time out at 1998.141777s 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f72e0135dc0 0x7f72e0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f72e0135dc0 0x7f72e0136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44256/0 (socket says 172.21.15.134:44256) 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 1 -- 172.21.15.134:0/2768503685 learned_addr learned my addr 172.21.15.134:0/2768503685 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 10 monclient(hunting): get_auth_request con 0x7f72e0135dc0 auth_method 0 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.552+0000 7f72dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.557 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72d7fff640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0137220 0x7f72e0137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72df7fe640 1 --2- 172.21.15.134:0/2768503685 >> v2:172.21.15.162:3300/0 conn(0x7f72e0136e40 0x7f72e009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72d7fff640 10 monclient(hunting): get_auth_request con 0x7f72e0137220 auth_method 0 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72df7fe640 10 monclient(hunting): get_auth_request con 0x7f72e0136e40 auth_method 0 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient(hunting): handle_auth_done global_id 124533 payload 274 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 -- 172.21.15.134:0/2768503685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0137220 msgr2=0x7f72e0137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0137220 0x7f72e0137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.558 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 -- 172.21.15.134:0/2768503685 >> v2:172.21.15.162:3300/0 conn(0x7f72e0136e40 msgr2=0x7f72e009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 --2- 172.21.15.134:0/2768503685 >> v2:172.21.15.162:3300/0 conn(0x7f72e0136e40 0x7f72e009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 monclient: found mon.noname-a 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:01.559 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.553+0000 7f72dffff640 1 -- 172.21.15.134:0/2768503685 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0073250 con 0x7f72e0135dc0 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.555+0000 7f72dffff640 1 -- 172.21.15.134:0/2768503685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72e0135dc0 msgr2=0x7f72e0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.555+0000 7f72dffff640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72e0135dc0 0x7f72e0136220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72e0137f00 tx=0x7f72d802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c910d8afbdd70ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 1 -- 172.21.15.134:0/2768503685 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72d8007a70 con 0x7f72e0135dc0 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:01.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: dump: 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.565 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 1 -- 172.21.15.134:0/2768503685 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d802d050 con 0x7f72e0135dc0 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 5 monclient: authenticate success, global_id 124533 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 4 set_mon_vals no callback set 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 1 -- 172.21.15.134:0/2768503685 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72d8007bf0 con 0x7f72e0135dc0 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72deffd640 10 monclient: dump: 2024-09-16T15:21:01.566 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.567 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 10 monclient: shutdown 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 -- 172.21.15.134:0/2768503685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72e0135dc0 msgr2=0x7f72e0136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72e0135dc0 0x7f72e0136220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72e0137f00 tx=0x7f72d802a570 comp rx=0 tx=0).stop 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 -- 172.21.15.134:0/2768503685 shutdown_connections 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72e0135dc0 0x7f72e0136220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 --2- 172.21.15.134:0/2768503685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0137220 0x7f72e0137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 --2- 172.21.15.134:0/2768503685 >> v2:172.21.15.162:3300/0 conn(0x7f72e0136e40 0x7f72e009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 -- 172.21.15.134:0/2768503685 >> 172.21.15.134:0/2768503685 conn(0x7f72e00a6110 msgr2=0x7f72e00a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 -- 172.21.15.134:0/2768503685 shutdown_connections 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.556+0000 7f72e6183640 1 -- 172.21.15.134:0/2768503685 wait complete. 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.557+0000 7f72e6183640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.557+0000 7f72e6183640 10 monclient: monmap: 2024-09-16T15:21:01.568 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.569 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.557+0000 7f72e6183640 1 Processor -- start 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.557+0000 7f72e6183640 1 -- start start 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.557+0000 7f72e6183640 10 monclient: init 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient: _add_conns ranks=[3,8,1,7,2,4,5,6,0] 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0135dc0 0x7f72e0134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-g con 0x7f72e0135dc0 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-e con 0x7f72e0136e40 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e0137220 0x7f72e0121990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): picked mon.noname-f con 0x7f72e0137220 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00a89c0 con 0x7f72e0137220 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e01362c0 con 0x7f72e0135dc0 2024-09-16T15:21:01.570 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0136810 con 0x7f72e0136e40 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72e6183640 10 monclient(hunting): authenticate will time out at 1998.147195s 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3304/0 says I am v2:172.21.15.134:35470/0 (socket says 172.21.15.134:35470) 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 -- 172.21.15.134:0/3640388474 learned_addr learned my addr 172.21.15.134:0/3640388474 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): get_auth_request con 0x7f72e0136e40 auth_method 0 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0135dc0 0x7f72e0134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.571 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): get_auth_request con 0x7f72e0135dc0 auth_method 0 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e0137220 0x7f72e0121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): get_auth_request con 0x7f72e0137220 auth_method 0 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72df7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.572 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient(hunting): handle_auth_done global_id 85229 payload 824 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 -- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0135dc0 msgr2=0x7f72e0134eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0135dc0 0x7f72e0134eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 -- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e0137220 msgr2=0x7f72e0121990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e0137220 0x7f72e0121990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 monclient: found mon.noname-e 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.558+0000 7f72d7fff640 1 -- 172.21.15.134:0/3640388474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009b30 con 0x7f72e0136e40 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.560+0000 7f72d7fff640 1 -- 172.21.15.134:0/3640388474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 msgr2=0x7f72e01353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.560+0000 7f72d7fff640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d5f0 tx=0x7f72c800dab0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.560+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c8012c70 con 0x7f72e0136e40 2024-09-16T15:21:01.573 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.560+0000 7f72dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: dump: 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.574 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8012e10 con 0x7f72e0136e40 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 5 monclient: authenticate success, global_id 85229 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _renew_subs 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e01242a0 con 0x7f72e0136e40 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c80148a0 con 0x7f72e0136e40 2024-09-16T15:21:01.575 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 10 monclient: dump: 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.576 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _renew_subs 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72de7fc640 4 set_mon_vals no callback set 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e01258a0 con 0x7f72e0136e40 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f72c8014430 con 0x7f72e0136e40 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.561+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e0125400 con 0x7f72e0136e40 2024-09-16T15:21:01.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.562+0000 7f72dcff9640 1 --2- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 0x7f72c003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.562+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f72c804a540 con 0x7f72e0136e40 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.562+0000 7f72df7fe640 1 --2- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 0x7f72c003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.562+0000 7f72df7fe640 10 monclient: get_auth_request con 0x7f72c0038880 auth_method 0 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.564+0000 7f72df7fe640 1 --2- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 0x7f72c003ad40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72d0006fd0 tx=0x7f72d0008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.566+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f72c801e460 con 0x7f72e0136e40 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.566+0000 7f72dcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:01.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.566+0000 7f72dcff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:01.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2791412942}) 2024-09-16T15:21:01.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.694 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1566570656}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3095120405}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=983525078}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=598225515}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1597836588}) 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3188361224}) 2024-09-16T15:21:01.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.691+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=781946105}) 2024-09-16T15:21:01.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=852594433}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4061371807}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3957596219}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3629462278}) 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.703 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3376882934}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=602598762}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4293053436}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2867880737}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885560747}) 2024-09-16T15:21:01.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3579768956}) 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2973903460}) 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1944929383}) 2024-09-16T15:21:01.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4171080523}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2505641595}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=741053253}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3964427459}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1360309654}) 2024-09-16T15:21:01.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4095173922}) 2024-09-16T15:21:01.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.710+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4057879078}) 2024-09-16T15:21:01.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4075739851}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2163803295}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4249274330}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1454078746}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3627505750}) 2024-09-16T15:21:01.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=223762200}) 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2243299863}) 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=899006339}) 2024-09-16T15:21:01.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.729+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3795999089}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3829750230}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3622383946}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=90363172}) 2024-09-16T15:21:01.740 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1243426186}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3356726366}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2824947667}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1396629937}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=186461702}) 2024-09-16T15:21:01.741 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:01.736+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3147663468}) 2024-09-16T15:21:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.747+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e0061ae0 con 0x7f72c0038880 2024-09-16T15:21:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.748+0000 7f72dcff9640 1 -- 172.21.15.134:0/3640388474 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72d0004020 con 0x7f72c0038880 2024-09-16T15:21:01.749 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 msgr2=0x7f72c003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 0x7f72c003ad40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72d0006fd0 tx=0x7f72d0008040 comp rx=0 tx=0).stop 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 10 monclient: shutdown 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 msgr2=0x7f72e01353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.753 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d5f0 tx=0x7f72c800dab0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 shutdown_connections 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f72e0137220 0x7f72e0121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f72e0136e40 0x7f72e01353f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> v2:172.21.15.162:6800/980547263 conn(0x7f72c0038880 0x7f72c003ad40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 --2- 172.21.15.134:0/3640388474 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f72e0135dc0 0x7f72e0134eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.751+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 >> 172.21.15.134:0/3640388474 conn(0x7f72e00a6110 msgr2=0x7f72e009f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.752+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 shutdown_connections 2024-09-16T15:21:01.754 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.752+0000 7f72e6183640 1 -- 172.21.15.134:0/3640388474 wait complete. 2024-09-16T15:21:01.765 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T15:21:01.766 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:21:01.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 10 monclient: get_monmap_and_config 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 10 monclient: monmap: 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:01.868014+0000 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:01.868014+0000 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.869 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 1 Processor -- start 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 1 -- start start 2024-09-16T15:21:01.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.867+0000 7fd957cd7640 10 monclient: init 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient: _add_conns ranks=[9,0,1,3,7,4,10,8,2,5,6] 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9500a4220 0x7fd9500a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-a con 0x7fd9500a4220 addr v2:172.21.15.134:3300/0 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd95009c720 addr v1:172.21.15.134:6789/0 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7fd95009b1a0 addr v1:172.21.15.162:6789/0 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95004e730 con 0x7fd95009c720 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd950137e00 con 0x7fd95009b1a0 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95009c390 con 0x7fd9500a4220 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd957cd7640 10 monclient(hunting): authenticate will time out at 1998.457332s 2024-09-16T15:21:01.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9500a4220 0x7fd9500a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955272640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd95009c720 0x7fd950136fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59266/0 (socket says 172.21.15.134:59266) 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955272640 1 -- 172.21.15.134:0/3852540634 learned_addr learned my addr 172.21.15.134:0/3852540634 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): get_auth_request con 0x7fd9500a4220 auth_method 0 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient(hunting): handle_auth_done global_id 124542 payload 274 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 1 -- 172.21.15.134:0/3852540634 >> v1:172.21.15.162:6789/0 conn(0x7fd95009b1a0 legacy=0x7fd9501376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 1 -- 172.21.15.134:0/3852540634 >> v1:172.21.15.134:6789/0 conn(0x7fd95009c720 legacy=0x7fd950136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 1 monclient: found mon.noname-a 2024-09-16T15:21:01.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:01.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.868+0000 7fd955a73640 1 -- 172.21.15.134:0/3852540634 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500a5e20 con 0x7fd9500a4220 2024-09-16T15:21:01.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd955a73640 1 -- 172.21.15.134:0/3852540634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9500a4220 msgr2=0x7fd9500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd955a73640 1 --2- 172.21.15.134:0/3852540634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9500a4220 0x7fd9500a4600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097a0 tx=0x7fd94802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59562e2bec8b3ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 1 -- 172.21.15.134:0/3852540634 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd94802ae60 con 0x7fd9500a4220 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: dump: 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.880 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 1 -- 172.21.15.134:0/3852540634 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9480307b0 con 0x7fd9500a4220 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 1 -- 172.21.15.134:0/3852540634 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd948030d90 con 0x7fd9500a4220 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd954a71640 10 monclient: dump: 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.882 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd957cd7640 5 monclient: authenticate success, global_id 124542 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd957cd7640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.871+0000 7fd957cd7640 4 set_mon_vals no callback set 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 10 monclient: shutdown 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 -- 172.21.15.134:0/3852540634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9500a4220 msgr2=0x7fd9500a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 --2- 172.21.15.134:0/3852540634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9500a4220 0x7fd9500a4600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097a0 tx=0x7fd94802a060 comp rx=0 tx=0).stop 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 -- 172.21.15.134:0/3852540634 shutdown_connections 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 --2- 172.21.15.134:0/3852540634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9500a4220 0x7fd9500a4600 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 -- 172.21.15.134:0/3852540634 >> 172.21.15.134:0/3852540634 conn(0x7fd9500a9820 msgr2=0x7fd9500a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 -- 172.21.15.134:0/3852540634 shutdown_connections 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 1 -- 172.21.15.134:0/3852540634 wait complete. 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.872+0000 7fd957cd7640 10 monclient: build_initial_monmap 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient: monmap: 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:01.884 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 Processor -- start 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 -- start start 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient: init 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient: _add_conns ranks=[1,4,8,2,0,6,3,5,7] 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd95009b1a0 0x7fd9500a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-f con 0x7fd95009b1a0 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 0x7fd950135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-c con 0x7fd9500a4220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 0x7fd9501cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): picked mon.noname-e con 0x7fd9501357d0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd950122720 con 0x7fd95009b1a0 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd950135c50 con 0x7fd9500a4220 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:01.886 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fd950120330 con 0x7fd9501357d0 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd957cd7640 10 monclient(hunting): authenticate will time out at 1998.462918s 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.873+0000 7fd955272640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 0x7fd950135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 0x7fd950135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33472/0 (socket says 172.21.15.134:33472) 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 1 -- 172.21.15.134:0/1002819801 learned_addr learned my addr 172.21.15.134:0/1002819801 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): get_auth_request con 0x7fd9500a4220 auth_method 0 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 0x7fd9501cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): get_auth_request con 0x7fd9501357d0 auth_method 0 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.887 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd95009b1a0 0x7fd9500a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): get_auth_request con 0x7fd95009b1a0 auth_method 0 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955a73640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:01.888 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient(hunting): handle_auth_done global_id 85238 payload 824 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 -- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 msgr2=0x7fd950135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 0x7fd950135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 -- 172.21.15.134:0/1002819801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd95009b1a0 msgr2=0x7fd9500a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd95009b1a0 0x7fd9500a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient: _finish_hunting 0 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 monclient: found mon.noname-e 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd956274640 1 -- 172.21.15.134:0/1002819801 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948009400 con 0x7fd9501357d0 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.874+0000 7fd955272640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fd9500a4220 0x7fd950135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd956274640 1 -- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 msgr2=0x7fd9501cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd956274640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 0x7fd9501cb740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c780 tx=0x7fd94c00cc40 comp rx=0 tx=0).ready entity=mon.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd94c00acc0 con 0x7fd9501357d0 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: dump: 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.890 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: _finish_auth 0 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: _check_auth_tickets 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00ae40 con 0x7fd9501357d0 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd94c012890 con 0x7fd9501357d0 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd9467fc640 10 monclient: dump: 2024-09-16T15:21:01.891 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:01.892 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 5 monclient: authenticate success, global_id 85238 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 10 monclient: _renew_subs 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd947fff640 4 set_mon_vals no callback set 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd950122cb0 con 0x7fd9501357d0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 10 monclient: _renew_subs 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.876+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500a8a30 con 0x7fd9501357d0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.877+0000 7fd957cd7640 10 monclient: _send_mon_message to mon.e at v2:172.21.15.134:3304/0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.877+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd950061ae0 con 0x7fd9501357d0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.877+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fd94c012420 con 0x7fd9501357d0 2024-09-16T15:21:01.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.877+0000 7fd9467fc640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6800/980547263 conn(0x7fd934038880 0x7fd93403ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.877+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fd94c049f10 con 0x7fd9501357d0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd9467fc640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6801/1098590432 conn(0x7fd93403e3a0 0x7fd9340407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 --> v2:172.21.15.162:6801/1098590432 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd934040e30 con 0x7fd93403e3a0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== mon.8 v2:172.21.15.134:3304/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd94c04c4a0 con 0x7fd9501357d0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd9467fc640 10 monclient: handle_get_version_reply finishing 1 version 861 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd955a73640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6800/980547263 conn(0x7fd934038880 0x7fd93403ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd955a73640 10 monclient: get_auth_request con 0x7fd934038880 auth_method 0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd955272640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6801/1098590432 conn(0x7fd93403e3a0 0x7fd9340407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:01.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.878+0000 7fd955272640 10 monclient: get_auth_request con 0x7fd93403e3a0 auth_method 0 2024-09-16T15:21:01.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.880+0000 7fd955a73640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6800/980547263 conn(0x7fd934038880 0x7fd93403ad40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097a0 tx=0x7fd94802e000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.882+0000 7fd955272640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6801/1098590432 conn(0x7fd93403e3a0 0x7fd9340407c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:01.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.883+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== osd.4 v2:172.21.15.162:6801/1098590432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd934040e30 con 0x7fd93403e3a0 2024-09-16T15:21:01.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.895+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 --> v2:172.21.15.162:6801/1098590432 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd95007a680 con 0x7fd93403e3a0 2024-09-16T15:21:01.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd9467fc640 1 -- 172.21.15.134:0/1002819801 <== osd.4 v2:172.21.15.162:6801/1098590432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd940006c80 con 0x7fd93403e3a0 2024-09-16T15:21:01.897 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:21:01.898 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:21:01.898 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:21:01.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6801/1098590432 conn(0x7fd93403e3a0 msgr2=0x7fd9340407c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6801/1098590432 conn(0x7fd93403e3a0 0x7fd9340407c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6800/980547263 conn(0x7fd934038880 msgr2=0x7fd93403ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 --2- 172.21.15.134:0/1002819801 >> v2:172.21.15.162:6800/980547263 conn(0x7fd934038880 0x7fd93403ad40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097a0 tx=0x7fd94802e000 comp rx=0 tx=0).stop 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 10 monclient: shutdown 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 msgr2=0x7fd9501cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 --2- 172.21.15.134:0/1002819801 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7fd9501357d0 0x7fd9501cb740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c780 tx=0x7fd94c00cc40 comp rx=0 tx=0).stop 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd956274640 1 -- 172.21.15.134:0/1002819801 reap_dead start 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 shutdown_connections 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 >> 172.21.15.134:0/1002819801 conn(0x7fd9500a9820 msgr2=0x7fd950137640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 shutdown_connections 2024-09-16T15:21:01.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:01.896+0000 7fd957cd7640 1 -- 172.21.15.134:0/1002819801 wait complete. 2024-09-16T15:21:01.907 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.003+0000 7f4cb2e12640 10 monclient: get_monmap_and_config 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.003+0000 7f4cb2e12640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.003+0000 7f4cb2e12640 10 monclient: monmap: 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:02.004878+0000 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:02.004878+0000 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.005 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:02.006 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.004+0000 7f4cb2e12640 1 Processor -- start 2024-09-16T15:21:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.004+0000 7f4cb2e12640 1 -- start start 2024-09-16T15:21:02.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.004+0000 7f4cb2e12640 10 monclient: init 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient: _add_conns ranks=[6,10,8,1,7,3,2,4,9,5,0] 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-d con 0x7f4cac136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cac137220 0x7f4cac137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-f con 0x7f4cac137220 addr v2:172.21.15.162:3300/0 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 0x7f4cac136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-e con 0x7f4cac135dc0 addr [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cac09abc0 con 0x7f4cac137220 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac09ba50 con 0x7f4cac136e40 2024-09-16T15:21:02.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 1 -- --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac0a53c0 con 0x7f4cac135dc0 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb2e12640 10 monclient(hunting): authenticate will time out at 1998.594239s 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 --2- >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 0x7f4cac136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34836/0 (socket says 172.21.15.134:34836) 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2972824961 learned_addr learned my addr 172.21.15.134:0/2972824961 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): get_auth_request con 0x7f4cac136e40 auth_method 0 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): get_auth_request con 0x7f4cac135dc0 auth_method 0 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4ca3fff640 1 --2- 172.21.15.134:0/2972824961 >> v2:172.21.15.162:3300/0 conn(0x7f4cac137220 0x7f4cac137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4ca3fff640 10 monclient(hunting): get_auth_request con 0x7f4cac137220 auth_method 0 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4ca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4ca3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4ca3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient(hunting): handle_auth_done global_id 85247 payload 274 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 -- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 msgr2=0x7f4cac09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 -- 172.21.15.134:0/2972824961 >> v2:172.21.15.162:3300/0 conn(0x7f4cac137220 msgr2=0x7f4cac137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 --2- 172.21.15.134:0/2972824961 >> v2:172.21.15.162:3300/0 conn(0x7f4cac137220 0x7f4cac137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 monclient: found mon.noname-e 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 10 monclient: _send_mon_message to mon.noname-e at v2:172.21.15.134:3304/0 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb13af640 1 -- 172.21.15.134:0/2972824961 --> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac073250 con 0x7f4cac135dc0 2024-09-16T15:21:02.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.005+0000 7f4cb0bae640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb13af640 1 -- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 msgr2=0x7f4cac136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb13af640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 0x7f4cac136220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009dc0 tx=0x7f4ca402a4d0 comp rx=0 tx=0).ready entity=mon.8 client_cookie=475e8f9104ce02b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 1 -- 172.21.15.134:0/2972824961 <== mon.8 v2:172.21.15.134:3304/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca4007bc0 con 0x7f4cac135dc0 2024-09-16T15:21:02.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.017 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: dump: 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.018 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb2e12640 5 monclient: authenticate success, global_id 85247 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 1 -- 172.21.15.134:0/2972824961 <== mon.8 v2:172.21.15.134:3304/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca402d050 con 0x7f4cac135dc0 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb2e12640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb2e12640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4cb2e12640 4 set_mon_vals no callback set 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 1 -- 172.21.15.134:0/2972824961 <== mon.8 v2:172.21.15.134:3304/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca4007d80 con 0x7f4cac135dc0 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.019 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.008+0000 7f4ca37fe640 10 monclient: dump: 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.020 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 10 monclient: shutdown 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 msgr2=0x7f4cac136220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 0x7f4cac136220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009dc0 tx=0x7f4ca402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2972824961 shutdown_connections 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] conn(0x7f4cac135dc0 0x7f4cac136220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2972824961 >> v2:172.21.15.162:3300/0 conn(0x7f4cac137220 0x7f4cac137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2972824961 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4cac136e40 0x7f4cac09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2972824961 >> 172.21.15.134:0/2972824961 conn(0x7f4cac0a6110 msgr2=0x7f4cac0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2972824961 shutdown_connections 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2972824961 wait complete. 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.009+0000 7f4cb2e12640 10 monclient: monmap: 2024-09-16T15:21:02.021 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.022 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 Processor -- start 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 -- start start 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient: init 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient: _add_conns ranks=[2,4,0,7,6,5,1,8,3] 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-b con 0x7f4cac135dc0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-c con 0x7f4cac136e40 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 0x7f4cac121990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): picked mon.noname-a con 0x7f4cac137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1cc090 con 0x7f4cac137220 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.023 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac125350 con 0x7f4cac135dc0 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac1362a0 con 0x7f4cac136e40 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb2e12640 10 monclient(hunting): authenticate will time out at 1998.599725s 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59080/0 (socket says 172.21.15.134:59080) 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2700748253 learned_addr learned my addr 172.21.15.134:0/2700748253 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33474/0 (socket says 172.21.15.134:33474) 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): get_auth_request con 0x7f4cac135dc0 auth_method 0 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): get_auth_request con 0x7f4cac136e40 auth_method 0 2024-09-16T15:21:02.024 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 0x7f4cac121990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): get_auth_request con 0x7f4cac137220 auth_method 0 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4ca3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.025 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.010+0000 7f4cb13af640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 10 monclient(hunting): handle_auth_done global_id 114797 payload 824 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 msgr2=0x7f4cac135380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 msgr2=0x7f4cac121990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 0x7f4cac121990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 monclient: found mon.noname-b 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2700748253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80094d0 con 0x7f4cac135dc0 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4ca3fff640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.011+0000 7f4cb13af640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 0x7f4cac121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb0bae640 1 -- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 msgr2=0x7f4cac134e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.026 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb0bae640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800e9d0 tx=0x7f4ca800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca800cc70 con 0x7f4cac135dc0 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: dump: 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.027 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 5 monclient: authenticate success, global_id 114797 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca800cdf0 con 0x7f4cac135dc0 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 10 monclient: _renew_subs 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca4009a70 con 0x7f4cac135dc0 2024-09-16T15:21:02.028 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4ca8013d50 con 0x7f4cac135dc0 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca17fa640 10 monclient: dump: 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.029 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 10 monclient: _renew_subs 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac1367f0 con 0x7f4cac135dc0 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.013+0000 7f4ca2ffd640 4 set_mon_vals no callback set 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4ca8013470 con 0x7f4cac135dc0 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4cb2e12640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4cb2e12640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4cb2e12640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4ca17fa640 1 --2- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 0x7f4c9803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac09b770 con 0x7f4cac135dc0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f4ca804a3a0 con 0x7f4cac135dc0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.014+0000 7f4ca3fff640 1 --2- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 0x7f4c9803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.015+0000 7f4ca3fff640 10 monclient: get_auth_request con 0x7f4c980388b0 auth_method 0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.017+0000 7f4ca3fff640 1 --2- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 0x7f4c9803ad70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c007d10 tx=0x7f4c9c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.019+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4ca8017070 con 0x7f4cac135dc0 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.019+0000 7f4ca17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.019+0000 7f4ca17fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:02.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=177083126}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3194188685}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1062703484}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2999799043}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385559226}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.066 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2367895055}) 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1725657749}) 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3178855944}) 2024-09-16T15:21:02.067 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.063+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=37837246}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3018311879}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1859900913}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.068+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2818301008}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1928642069}) 2024-09-16T15:21:02.081 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3847021719}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1293692400}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1876513695}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127521185}) 2024-09-16T15:21:02.082 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=721254152}) 2024-09-16T15:21:02.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.203+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cac061ae0 con 0x7f4c980388b0 2024-09-16T15:21:02.205 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.204+0000 7f4ca17fa640 1 -- 172.21.15.134:0/2700748253 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4c9c010e80 con 0x7f4c980388b0 2024-09-16T15:21:02.205 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 msgr2=0x7f4c9803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 0x7f4c9803ad70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c007d10 tx=0x7f4c9c008040 comp rx=0 tx=0).stop 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 10 monclient: shutdown 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 msgr2=0x7f4cac134e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800e9d0 tx=0x7f4ca800ee90 comp rx=0 tx=0).stop 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 shutdown_connections 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c980388b0 0x7f4c9803ad70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac137220 0x7f4cac121990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4cac136e40 0x7f4cac135380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 --2- 172.21.15.134:0/2700748253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4cac135dc0 0x7f4cac134e40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 >> 172.21.15.134:0/2700748253 conn(0x7f4cac0a6110 msgr2=0x7f4cac06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 shutdown_connections 2024-09-16T15:21:02.209 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.206+0000 7f4cb2e12640 1 -- 172.21.15.134:0/2700748253 wait complete. 2024-09-16T15:21:02.218 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T15:21:02.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:21:02.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.316+0000 7f98a725a640 10 monclient: get_monmap_and_config 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.316+0000 7f98a725a640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.316+0000 7f98a725a640 10 monclient: monmap: 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:02.317447+0000 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:02.317447+0000 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.318 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:02.319 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 Processor -- start 2024-09-16T15:21:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 -- start start 2024-09-16T15:21:02.321 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient: init 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient: _add_conns ranks=[6,3,9,4,10,2,0,7,8,1,5] 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-d con 0x7f98a00a4220 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98a009c720 0x7f98a00a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-g con 0x7f98a009c720 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 0x7f98a0136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-a con 0x7f98a009b1a0 addr v2:172.21.15.134:3300/0 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a009c390 con 0x7f98a009b1a0 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00a5080 con 0x7f98a009c720 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00a5e20 con 0x7f98a00a4220 2024-09-16T15:21:02.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a725a640 10 monclient(hunting): authenticate will time out at 1998.906916s 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.317+0000 7f98a4ff6640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34844/0 (socket says 172.21.15.134:34844) 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 0x7f98a0136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 -- 172.21.15.134:0/580534189 learned_addr learned my addr 172.21.15.134:0/580534189 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): get_auth_request con 0x7f98a00a4220 auth_method 0 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): get_auth_request con 0x7f98a009b1a0 auth_method 0 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98a009c720 0x7f98a00a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): get_auth_request con 0x7f98a009c720 auth_method 0 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_done global_id 94956 payload 274 2024-09-16T15:21:02.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 -- 172.21.15.134:0/580534189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98a009c720 msgr2=0x7f98a00a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98a009c720 0x7f98a00a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 -- 172.21.15.134:0/580534189 >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 msgr2=0x7f98a0136fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/580534189 >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 0x7f98a0136fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 monclient: found mon.noname-d 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a4ff6640 1 -- 172.21.15.134:0/580534189 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a004e730 con 0x7f98a00a4220 2024-09-16T15:21:02.325 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.318+0000 7f98a57f7640 1 --2- 172.21.15.134:0/580534189 >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 0x7f98a0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a4ff6640 1 -- 172.21.15.134:0/580534189 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 msgr2=0x7f98a00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9894009e80 tx=0x7f989402a580 comp rx=0 tx=0).ready entity=mon.6 client_cookie=8d51bb5eac83e151 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 1 -- 172.21.15.134:0/580534189 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9894007a70 con 0x7f98a00a4220 2024-09-16T15:21:02.330 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: dump: 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.331 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 1 -- 172.21.15.134:0/580534189 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989402d050 con 0x7f98a00a4220 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 5 monclient: authenticate success, global_id 94956 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 1 -- 172.21.15.134:0/580534189 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9894007bf0 con 0x7f98a00a4220 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 4 set_mon_vals no callback set 2024-09-16T15:21:02.332 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f989f7fe640 10 monclient: dump: 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.333 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 10 monclient: shutdown 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 -- 172.21.15.134:0/580534189 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 msgr2=0x7f98a00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9894009e80 tx=0x7f989402a580 comp rx=0 tx=0).stop 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 -- 172.21.15.134:0/580534189 shutdown_connections 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98a009c720 0x7f98a00a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 --2- 172.21.15.134:0/580534189 >> v2:172.21.15.134:3300/0 conn(0x7f98a009b1a0 0x7f98a0136fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 --2- 172.21.15.134:0/580534189 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f98a00a4220 0x7f98a00a4600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 -- 172.21.15.134:0/580534189 >> 172.21.15.134:0/580534189 conn(0x7f98a00a9820 msgr2=0x7f98a00a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 -- 172.21.15.134:0/580534189 shutdown_connections 2024-09-16T15:21:02.334 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.321+0000 7f98a725a640 1 -- 172.21.15.134:0/580534189 wait complete. 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.322+0000 7f98a725a640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.322+0000 7f98a725a640 10 monclient: monmap: 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.335 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.322+0000 7f98a725a640 1 Processor -- start 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.322+0000 7f98a725a640 1 -- start start 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.322+0000 7f98a725a640 10 monclient: init 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient: _add_conns ranks=[4,0,1,3,2,6,7,5,8] 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f98a009b1a0 0x7f98a01cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-c con 0x7f98a009b1a0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 0x7f98a0137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-a con 0x7f98a00a4220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a0137760 0x7f98a0123cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): picked mon.noname-f con 0x7f98a0137760 addr [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a01cb820 con 0x7f98a00a4220 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00a8ea0 con 0x7f98a0137760 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0137be0 con 0x7f98a009b1a0 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a725a640 10 monclient(hunting): authenticate will time out at 1998.912439s 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 0x7f98a0137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 0x7f98a0137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44280/0 (socket says 172.21.15.134:44280) 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 -- 172.21.15.134:0/266565894 learned_addr learned my addr 172.21.15.134:0/266565894 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): get_auth_request con 0x7f98a00a4220 auth_method 0 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.337 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f98a009b1a0 0x7f98a01cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): get_auth_request con 0x7f98a009b1a0 auth_method 0 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a0137760 0x7f98a0123cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.338 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): get_auth_request con 0x7f98a0137760 auth_method 0 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a57f7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient(hunting): handle_auth_done global_id 124569 payload 824 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 -- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f98a009b1a0 msgr2=0x7f98a01cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f98a009b1a0 0x7f98a01cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 -- 172.21.15.134:0/266565894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a0137760 msgr2=0x7f98a0123cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a0137760 0x7f98a0123cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 monclient: found mon.noname-a 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f989ffff640 1 -- 172.21.15.134:0/266565894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898009400 con 0x7f98a00a4220 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.323+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f98a009b1a0 0x7f98a01cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989ffff640 1 -- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 msgr2=0x7f98a0137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989ffff640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 0x7f98a0137220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e980 tx=0x7f988c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f988c009670 con 0x7f98a00a4220 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: dump: 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.340 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c00cc10 con 0x7f98a00a4220 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 5 monclient: authenticate success, global_id 124569 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f988c00cd80 con 0x7f98a00a4220 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 10 monclient: _renew_subs 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9894009b30 con 0x7f98a00a4220 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989effd640 4 set_mon_vals no callback set 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f989d7fa640 10 monclient: dump: 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.342 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 10 monclient: _renew_subs 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.326+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0126000 con 0x7f98a00a4220 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f988c013470 con 0x7f98a00a4220 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a725a640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98a0061ae0 con 0x7f98a00a4220 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6800/980547263 conn(0x7f98840387e0 0x7f988403aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f988c04a2e0 con 0x7f98a00a4220 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6809/518841521 conn(0x7f988403e300 0x7f9884040720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 --> v2:172.21.15.162:6809/518841521 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9884040d90 con 0x7f988403e300 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=861) v2 ==== 24+0+0 (secure 0 0 0) 0x7f988c04c800 con 0x7f98a00a4220 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f989d7fa640 10 monclient: handle_get_version_reply finishing 1 version 861 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6800/980547263 conn(0x7f98840387e0 0x7f988403aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a4ff6640 10 monclient: get_auth_request con 0x7f98840387e0 auth_method 0 2024-09-16T15:21:02.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a57f7640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6809/518841521 conn(0x7f988403e300 0x7f9884040720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.327+0000 7f98a57f7640 10 monclient: get_auth_request con 0x7f988403e300 auth_method 0 2024-09-16T15:21:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.330+0000 7f98a4ff6640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6800/980547263 conn(0x7f98840387e0 0x7f988403aca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f98a00aa6e0 tx=0x7f9894009000 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.332+0000 7f98a57f7640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6809/518841521 conn(0x7f988403e300 0x7f9884040720 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.332+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== osd.5 v2:172.21.15.162:6809/518841521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9884040d90 con 0x7f988403e300 2024-09-16T15:21:02.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.344+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 --> v2:172.21.15.162:6809/518841521 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f98a007a680 con 0x7f988403e300 2024-09-16T15:21:02.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f989d7fa640 1 -- 172.21.15.134:0/266565894 <== osd.5 v2:172.21.15.162:6809/518841521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9898009910 con 0x7f988403e300 2024-09-16T15:21:02.346 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-16T15:21:02.346 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:21:02.346 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-16T15:21:02.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6809/518841521 conn(0x7f988403e300 msgr2=0x7f9884040720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6809/518841521 conn(0x7f988403e300 0x7f9884040720 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6800/980547263 conn(0x7f98840387e0 msgr2=0x7f988403aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 --2- 172.21.15.134:0/266565894 >> v2:172.21.15.162:6800/980547263 conn(0x7f98840387e0 0x7f988403aca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f98a00aa6e0 tx=0x7f9894009000 comp rx=0 tx=0).stop 2024-09-16T15:21:02.347 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 10 monclient: shutdown 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 msgr2=0x7f98a0137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 --2- 172.21.15.134:0/266565894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a00a4220 0x7f98a0137220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e980 tx=0x7f988c00ee40 comp rx=0 tx=0).stop 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a57f7640 1 -- 172.21.15.134:0/266565894 reap_dead start 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 shutdown_connections 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 >> 172.21.15.134:0/266565894 conn(0x7f98a00a9820 msgr2=0x7f98a00a7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 shutdown_connections 2024-09-16T15:21:02.348 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.345+0000 7f98a725a640 1 -- 172.21.15.134:0/266565894 wait complete. 2024-09-16T15:21:02.357 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T15:21:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.454+0000 7f823a718640 10 monclient: get_monmap_and_config 2024-09-16T15:21:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.454+0000 7f823a718640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.454+0000 7f823a718640 10 monclient: monmap: 2024-09-16T15:21:02.455 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:02.455236+0000 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:02.455236+0000 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.456 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:02.457 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:02.457 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 Processor -- start 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 -- start start 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient: init 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient: _add_conns ranks=[2,10,4,6,1,5,7,8,3,9,0] 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): picked mon.noname-b con 0x7f8234130ab0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82341305f0 0x7f8234133af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): picked mon.noname-f con 0x7f82341305f0 addr v2:172.21.15.162:3300/0 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 0x7f82341344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): picked mon.noname-c con 0x7f8234134030 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:02.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823409f600 con 0x7f82341305f0 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340673e0 con 0x7f8234130ab0 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f823409e740 con 0x7f8234134030 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f823a718640 10 monclient(hunting): authenticate will time out at 1999.044597s 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59092/0 (socket says 172.21.15.134:59092) 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 -- 172.21.15.134:0/1783612206 learned_addr learned my addr 172.21.15.134:0/1783612206 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): get_auth_request con 0x7f8234130ab0 auth_method 0 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.460 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 0x7f82341344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): get_auth_request con 0x7f8234134030 auth_method 0 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8238cb5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f82337fe640 1 --2- 172.21.15.134:0/1783612206 >> v2:172.21.15.162:3300/0 conn(0x7f82341305f0 0x7f8234133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f82337fe640 10 monclient(hunting): get_auth_request con 0x7f82341305f0 auth_method 0 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f82337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f82337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f82337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.461 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient(hunting): handle_auth_done global_id 114806 payload 274 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 -- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 msgr2=0x7f82341344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 0x7f82341344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 -- 172.21.15.134:0/1783612206 >> v2:172.21.15.162:3300/0 conn(0x7f82341305f0 msgr2=0x7f8234133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 --2- 172.21.15.134:0/1783612206 >> v2:172.21.15.162:3300/0 conn(0x7f82341305f0 0x7f8234133af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 1 monclient: found mon.noname-b 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.455+0000 7f8233fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.456+0000 7f8233fff640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.456+0000 7f8233fff640 1 -- 172.21.15.134:0/1783612206 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234073250 con 0x7f8234130ab0 2024-09-16T15:21:02.462 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.456+0000 7f8238cb5640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 0x7f82341344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8233fff640 1 -- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 msgr2=0x7f8234130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8233fff640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82240098a0 tx=0x7f822402a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce5bb1b2928f0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 1 -- 172.21.15.134:0/1783612206 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f822402ae70 con 0x7f8234130ab0 2024-09-16T15:21:02.467 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: dump: 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.468 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 1 -- 172.21.15.134:0/1783612206 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82240307b0 con 0x7f8234130ab0 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f823a718640 5 monclient: authenticate success, global_id 114806 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f823a718640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f823a718640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 1 -- 172.21.15.134:0/1783612206 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8224030d90 con 0x7f8234130ab0 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f823a718640 4 set_mon_vals no callback set 2024-09-16T15:21:02.469 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.458+0000 7f8232ffd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f8232ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f8232ffd640 10 monclient: dump: 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.470 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 10 monclient: shutdown 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 -- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 msgr2=0x7f8234130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82240098a0 tx=0x7f822402a150 comp rx=0 tx=0).stop 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 -- 172.21.15.134:0/1783612206 shutdown_connections 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f8234134030 0x7f82341344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 --2- 172.21.15.134:0/1783612206 >> v2:172.21.15.162:3300/0 conn(0x7f82341305f0 0x7f8234133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 --2- 172.21.15.134:0/1783612206 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f8234130e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 -- 172.21.15.134:0/1783612206 >> 172.21.15.134:0/1783612206 conn(0x7f82340a6400 msgr2=0x7f82340a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 -- 172.21.15.134:0/1783612206 shutdown_connections 2024-09-16T15:21:02.471 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.459+0000 7f823a718640 1 -- 172.21.15.134:0/1783612206 wait complete. 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.460+0000 7f823a718640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.460+0000 7f823a718640 10 monclient: monmap: 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.472 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.460+0000 7f823a718640 1 Processor -- start 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.460+0000 7f823a718640 1 -- start start 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.460+0000 7f823a718640 10 monclient: init 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient: _add_conns ranks=[6,2,5,1,3,0,7,4,8] 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): picked mon.noname-d con 0x7f82341305f0 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): picked mon.noname-b con 0x7f8234130ab0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 0x7f823412fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): picked mon.noname-h con 0x7f8234134030 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:21:02.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82340a6a50 con 0x7f8234130ab0 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82341c85c0 con 0x7f8234134030 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f8234132b20 con 0x7f82341305f0 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f823a718640 10 monclient(hunting): authenticate will time out at 1999.050118s 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59106/0 (socket says 172.21.15.134:59106) 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34846/0 (socket says 172.21.15.134:34846) 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 -- 172.21.15.134:0/684635002 learned_addr learned my addr 172.21.15.134:0/684635002 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): get_auth_request con 0x7f8234130ab0 auth_method 0 2024-09-16T15:21:02.474 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): get_auth_request con 0x7f82341305f0 auth_method 0 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 0x7f823412fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 10 monclient(hunting): get_auth_request con 0x7f8234134030 auth_method 0 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient(hunting): handle_auth_done global_id 114815 payload 824 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 -- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 msgr2=0x7f82341c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 -- 172.21.15.134:0/684635002 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 msgr2=0x7f823412fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 0x7f823412fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 monclient: found mon.noname-b 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f82337fe640 1 -- 172.21.15.134:0/684635002 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009a70 con 0x7f8234130ab0 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8233fff640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.461+0000 7f8238cb5640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 0x7f823412fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f82337fe640 1 -- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 msgr2=0x7f823412f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f82337fe640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8228006e80 tx=0x7f822800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8228004450 con 0x7f8234130ab0 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: dump: 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.477 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228007b70 con 0x7f8234130ab0 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 5 monclient: authenticate success, global_id 114815 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 10 monclient: _renew_subs 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82240094d0 con 0x7f8234130ab0 2024-09-16T15:21:02.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8228007ce0 con 0x7f8234130ab0 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f8230ff9640 10 monclient: dump: 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.479 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 10 monclient: _renew_subs 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f82327fc640 4 set_mon_vals no callback set 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.463+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8234071a20 con 0x7f8234130ab0 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.464+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f8228007700 con 0x7f8234130ab0 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.464+0000 7f823a718640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.464+0000 7f823a718640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.464+0000 7f823a718640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.464+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82340a0030 con 0x7f8234130ab0 2024-09-16T15:21:02.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.465+0000 7f8230ff9640 1 --2- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 0x7f822003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.465+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f822804a250 con 0x7f8234130ab0 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.465+0000 7f8233fff640 1 --2- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 0x7f822003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.465+0000 7f8233fff640 10 monclient: get_auth_request con 0x7f8220038880 auth_method 0 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.467+0000 7f8233fff640 1 --2- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 0x7f822003ad40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822403d040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.469+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f82280079b0 con 0x7f8234130ab0 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.469+0000 7f8230ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.469+0000 7f8230ff9640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:02.632 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.631+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8234061ae0 con 0x7f8220038880 2024-09-16T15:21:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.632+0000 7f8230ff9640 1 -- 172.21.15.134:0/684635002 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8224004160 con 0x7f8220038880 2024-09-16T15:21:02.633 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T15:21:02.635 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 msgr2=0x7f822003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 0x7f822003ad40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822403d040 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 10 monclient: shutdown 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 msgr2=0x7f823412f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8228006e80 tx=0x7f822800ee60 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 shutdown_connections 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> v2:172.21.15.162:6800/980547263 conn(0x7f8220038880 0x7f822003ad40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f8234134030 0x7f823412fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8234130ab0 0x7f823412f6b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 --2- 172.21.15.134:0/684635002 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f82341305f0 0x7f82341c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 >> 172.21.15.134:0/684635002 conn(0x7f82340a6400 msgr2=0x7f82341bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.636 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 shutdown_connections 2024-09-16T15:21:02.637 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.634+0000 7f823a718640 1 -- 172.21.15.134:0/684635002 wait complete. 2024-09-16T15:21:02.646 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1509149646}) 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=291521253}) 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.711 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2356194392}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=211277788}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=634201827}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=428742102}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3230164737}) 2024-09-16T15:21:02.712 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=815740480}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=621758845}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2864151149}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1657187901}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1755452209}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.713 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1378486989}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2029586961}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=805697784}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=211543642}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2701671129}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.693+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=432758828}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.714 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=7477824}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.694+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3280952505}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1076232896}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=923870975}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2158304796}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.715 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2609319118}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3515322116}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=557592546}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.695+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=229808100}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2369950285}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1131139853}) 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.716 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2044091419}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2346282911}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=982173876}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2702034100}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3362459798}) 2024-09-16T15:21:02.717 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3744795492}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.696+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=541064132}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3485725472}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1708363893}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=290253821}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.718 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1346039483}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=400635402}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3625757009}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1332811676}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4192920625}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.698+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2002822434}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1626579287}) 2024-09-16T15:21:02.719 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=290728258}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2818413611}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=48685763}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3667297265}) 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.720 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3283004865}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1397848607}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1939260402}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.700+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2938525831}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=483610409}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3832215088}) 2024-09-16T15:21:02.721 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1932499596}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3721010131}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3406701477}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4081475570}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2100920136}) 2024-09-16T15:21:02.722 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1514100349}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.701+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248744633}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=844441835}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3169687136}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2438683475}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1165664848}) 2024-09-16T15:21:02.723 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=283899582}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754973982}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3557928287}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693960153}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.703+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3789962662}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3355000673}) 2024-09-16T15:21:02.724 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1556062904}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1650266578}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3516505159}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2869702535}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.725 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1292867984}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4026596378}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173707731}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.704+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1140677298}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3769883384}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=171754136}) 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.726 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=476134696}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.705+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3337915536}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2946727959}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102417278}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2647447553}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320268818}) 2024-09-16T15:21:02.727 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.706+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2926558309}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2135483926}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4107828157}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2062259958}) 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=438319339}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1290982783}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2799613203}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1020500194}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1252625471}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.717+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=986402736}) 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.722+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.733 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.722+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.722+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=2,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=17,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=237268296,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=854399685,mon_config_key=1492070130,monmap=2838206317,osd_metadata=134110247,osd_pg_creating=1719352442,osd_snap=758207710}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.722+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.722+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=2,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=48} crc {logm=3815742121,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=1090687298,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=2949231881}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=17,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=49} crc {logm=3815742121,mdsmap=1531151339,mgr=1248521450,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3491890899,mon_config_key=1492070130,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=2584908680}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=2,mdsmap=3,mgr=5,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=15,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=55} crc {logm=3815742121,mdsmap=1531151339,mgr=2012037497,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=3966883682,mon_config_key=3602565677,monmap=2200900527,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=2430695736}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=2,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=15,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=3140514009,mgr=2295072758,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2425934214,mon_config_key=73932416,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1768409572}) 2024-09-16T15:21:02.734 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51} crc {logm=1113354332,mdsmap=1531151339,mgr=2295072758,mgr_command_descs=1973851347,mgrstat=1090687298,mon_config_key=362820133,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1672047797}) 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=2,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=15,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2425934214,mon_config_key=3884657887,monmap=3140514010,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=557650581}) 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=2,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=3815742121,mdsmap=1531151339,mgr=2166041845,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=862378756,mon_config_key=2328141726,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=1180084762}) 2024-09-16T15:21:02.735 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.723+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=2,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=16,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=49} crc {logm=3815742121,mdsmap=3585761405,mgr=3058068630,mgr_command_descs=1973851347,mgr_metadata=3299114077,mgrstat=2425934214,mon_config_key=649716617,monmap=2838206317,osd_metadata=3154723898,osd_pg_creating=1719352442,osd_snap=3132017131}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=3616172864}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=4211230064}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.742+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=100} crc {osd_snap=2063699617}) 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.758 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=100} crc {osd_snap=698234699}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=100} crc {osd_snap=314788423}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=354289562}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=2774196986}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=323518554}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.743+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=735463788}) 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.759 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=29,osdmap=71} crc {osd_snap=1107587044,osdmap=1017013392}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=479424712,osdmap=3713546980}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=557340219,osdmap=2076586176}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=3260934852,osdmap=2588120838}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.760 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=3110878392,osdmap=1724117769}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=2381181336,osdmap=897552569}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=1540241948,osdmap=4152909933}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=695361752,osdmap=1372035991}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.744+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=32,osdmap=68} crc {osd_snap=543387788,osdmap=2635098446}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=48295195}) 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.761 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3392781187}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2759600064}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2021784690}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1549508796}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2682522742}) 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.762 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2885905114}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1302118778}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.746+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2008107886}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=736406777}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2261346005}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.763 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2934673697}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=617471509}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3080982610}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=151034359}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.747+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.748+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1715736870}) 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.748+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.764 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.748+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2358063826}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.748+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3563382699}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3785837351}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1533539500}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=448312883}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2684379172}) 2024-09-16T15:21:02.765 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3805801019}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=751484982}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2523152873}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3304293227}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.749+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3465203574}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.750+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.750+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.766 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.750+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2475546344}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.750+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.750+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2138729422}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=52727608}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2656918291}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.767 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=988621194}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2815341078}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1352519503}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1471472324}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.751+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4108823860}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=622726009}) 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.768 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1615162396}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2726941327}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1139146901}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1697839071}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.769 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2160950920}) 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2915239306}) 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1746054103}) 2024-09-16T15:21:02.770 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:02.752+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=737784048}) 2024-09-16T15:21:02.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.760+0000 7f4ca6f9d640 10 monclient: get_monmap_and_config 2024-09-16T15:21:02.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.760+0000 7f4ca6f9d640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.760+0000 7f4ca6f9d640 10 monclient: monmap: 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:02.761762+0000 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:02.761762+0000 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.778 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.761+0000 7f4ca6f9d640 1 Processor -- start 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.761+0000 7f4ca6f9d640 1 -- start start 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.761+0000 7f4ca6f9d640 10 monclient: init 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient: _add_conns ranks=[5,4,2,0,3,10,1,8,9,7,6] 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136e40 0x7f4ca0137220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-h con 0x7f4ca0136e40 addr [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-c con 0x7f4ca0137760 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-b con 0x7f4ca00a4200 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:02.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a4b80 con 0x7f4ca00a4200 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a0f80 con 0x7f4ca0137760 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a1260 con 0x7f4ca0136e40 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca6f9d640 10 monclient(hunting): authenticate will time out at 1999.351093s 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:59122/0 (socket says 172.21.15.134:59122) 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:33486/0 (socket says 172.21.15.134:33486) 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 -- 172.21.15.134:0/1183681315 learned_addr learned my addr 172.21.15.134:0/1183681315 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.780 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): get_auth_request con 0x7f4ca00a4200 auth_method 0 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): get_auth_request con 0x7f4ca0137760 auth_method 0 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca5f9b640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136e40 0x7f4ca0137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca5f9b640 10 monclient(hunting): get_auth_request con 0x7f4ca0136e40 auth_method 0 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca5f9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca5f9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca5f9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient(hunting): handle_auth_done global_id 114824 payload 274 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 -- 172.21.15.134:0/1183681315 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136e40 msgr2=0x7f4ca0137220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136e40 0x7f4ca0137220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 -- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 msgr2=0x7f4ca00a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 monclient: found mon.noname-b 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca679c640 1 -- 172.21.15.134:0/1183681315 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0073250 con 0x7f4ca00a4200 2024-09-16T15:21:02.782 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.762+0000 7f4ca579a640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca679c640 1 -- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 msgr2=0x7f4ca00a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca679c640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009e80 tx=0x7f4c9802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21b08397108e311 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 1 -- 172.21.15.134:0/1183681315 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c98007a70 con 0x7f4ca00a4200 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: dump: 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.783 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 5 monclient: authenticate success, global_id 114824 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:02.784 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 1 -- 172.21.15.134:0/1183681315 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9802d050 con 0x7f4ca00a4200 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 4 set_mon_vals no callback set 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 1 -- 172.21.15.134:0/1183681315 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c98007bf0 con 0x7f4ca00a4200 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca4f99640 10 monclient: dump: 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.785 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 10 monclient: shutdown 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 msgr2=0x7f4ca00a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009e80 tx=0x7f4c9802a530 comp rx=0 tx=0).stop 2024-09-16T15:21:02.786 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1183681315 shutdown_connections 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ca00a4200 0x7f4ca00a4640 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7f4ca0137760 0x7f4ca00a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1183681315 >> [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] conn(0x7f4ca0136e40 0x7f4ca0137220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.765+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1183681315 >> 172.21.15.134:0/1183681315 conn(0x7f4ca0065e30 msgr2=0x7f4ca0066240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1183681315 shutdown_connections 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1183681315 wait complete. 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 10 monclient: build_initial_monmap 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 10 monclient: monmap: 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.787 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 1 Processor -- start 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 1 -- start start 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.766+0000 7f4ca6f9d640 10 monclient: init 2024-09-16T15:21:02.788 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient: _add_conns ranks=[3,6,7,8,1,0,5,2,4] 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ca00a4200 0x7f4ca01cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-g con 0x7f4ca00a4200 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-d con 0x7f4ca0136e40 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 0x7f4ca0136320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): picked mon.noname-i con 0x7f4ca0137760 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca0123570 con 0x7f4ca00a4200 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01237c0 con 0x7f4ca0136e40 2024-09-16T15:21:02.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca00a8c00 con 0x7f4ca0137760 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca6f9d640 10 monclient(hunting): authenticate will time out at 1999.356391s 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:34854/0 (socket says 172.21.15.134:34854) 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 -- 172.21.15.134:0/1853672860 learned_addr learned my addr 172.21.15.134:0/1853672860 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): get_auth_request con 0x7f4ca0136e40 auth_method 0 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:02.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca5f9b640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ca00a4200 0x7f4ca01cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca5f9b640 10 monclient(hunting): get_auth_request con 0x7f4ca00a4200 auth_method 0 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca679c640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 0x7f4ca0136320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca5f9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca5f9b640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca5f9b640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient(hunting): handle_auth_done global_id 94974 payload 824 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 -- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 msgr2=0x7f4ca0136320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 0x7f4ca0136320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 -- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ca00a4200 msgr2=0x7f4ca01cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ca00a4200 0x7f4ca01cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient: _finish_hunting 0 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 monclient: found mon.noname-d 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:21:02.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca579a640 1 -- 172.21.15.134:0/1853672860 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90009400 con 0x7f4ca0136e40 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca679c640 10 monclient: get_auth_request con 0x7f4ca0137760 auth_method 0 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.767+0000 7f4ca679c640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 0x7f4ca0136320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.769+0000 7f4ca579a640 1 -- 172.21.15.134:0/1853672860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 msgr2=0x7f4ca0135de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.769+0000 7f4ca579a640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e990 tx=0x7f4c9000ee50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9000cbf0 con 0x7f4ca0136e40 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: dump: 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.792 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: _finish_auth 0 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: _check_auth_tickets 2024-09-16T15:21:02.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9000cd70 con 0x7f4ca0136e40 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 5 monclient: authenticate success, global_id 94974 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4c9000cee0 con 0x7f4ca0136e40 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 10 monclient: _renew_subs 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c98009b30 con 0x7f4ca0136e40 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c97fff640 4 set_mon_vals no callback set 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 10 monclient: dump: 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:02.794 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:02.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 10 monclient: _renew_subs 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca0074b20 con 0x7f4ca0136e40 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.770+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f4c90013470 con 0x7f4ca0136e40 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca6f9d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca6f9d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca6f9d640 10 monclient: _send_mon_message to mon.d at v2:172.21.15.134:3303/0 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca0123040 con 0x7f4ca0136e40 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4c967fc640 1 --2- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 0x7f4c8403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f4c9004a2c0 con 0x7f4ca0136e40 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca5f9b640 1 --2- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 0x7f4c8403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.771+0000 7f4ca5f9b640 10 monclient: get_auth_request con 0x7f4c840388b0 auth_method 0 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.773+0000 7f4ca5f9b640 1 --2- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 0x7f4c8403ad70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:02.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.775+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mon.6 v2:172.21.15.134:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f4c9001b9a0 con 0x7f4ca0136e40 2024-09-16T15:21:02.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.775+0000 7f4c967fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:02.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.775+0000 7f4c967fc640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:02.935 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.933+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ca0061ae0 con 0x7f4c840388b0 2024-09-16T15:21:02.937 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.936+0000 7f4c967fc640 1 -- 172.21.15.134:0/1853672860 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54019 (secure 0 0 0) 0x7f4c9c00b230 con 0x7f4c840388b0 2024-09-16T15:21:02.937 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:21:02.938 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:21:02.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 msgr2=0x7f4c8403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 0x7f4c8403ad70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c006fd0 tx=0x7f4c9c008040 comp rx=0 tx=0).stop 2024-09-16T15:21:02.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 10 monclient: shutdown 2024-09-16T15:21:02.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:02.939 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 msgr2=0x7f4ca0135de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.937+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e990 tx=0x7f4c9000ee50 comp rx=0 tx=0).stop 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 shutdown_connections 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f4ca0137760 0x7f4ca0136320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f4ca0136e40 0x7f4ca0135de0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> v2:172.21.15.162:6800/980547263 conn(0x7f4c840388b0 0x7f4c8403ad70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 --2- 172.21.15.134:0/1853672860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ca00a4200 0x7f4ca01cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 >> 172.21.15.134:0/1853672860 conn(0x7f4ca0065e30 msgr2=0x7f4ca00a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 shutdown_connections 2024-09-16T15:21:02.940 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:02.938+0000 7f4ca6f9d640 1 -- 172.21.15.134:0/1853672860 wait complete. 2024-09-16T15:21:02.949 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1465,"stamp":"2024-09-16T15:21:02.411309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":363504,"kb_used_data":41508,"kb_used_omap":9,"kb_used_meta":321910,"kb_avail":565867536,"statfs":{"total":579820584960,"available":579448356864,"internally_reserved":0,"allocated":42504192,"data_stored":24192514,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":329636518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002500"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":1008,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715831+0000","last_change":"2024-09-16T15:00:45.968799+0000","last_active":"2024-09-16T15:19:44.715831+0000","last_peered":"2024-09-16T15:19:44.715831+0000","last_clean":"2024-09-16T15:19:44.715831+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:19:44.715831+0000","last_undegraded":"2024-09-16T15:19:44.715831+0000","last_fullsized":"2024-09-16T15:19:44.715831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:44.667324+0000","last_clean_scrub_stamp":"2024-09-16T15:00:44.667324+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:59.968816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":864,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076649+0000","last_change":"2024-09-16T15:00:49.323958+0000","last_active":"2024-09-16T15:19:43.076649+0000","last_peered":"2024-09-16T15:19:43.076649+0000","last_clean":"2024-09-16T15:19:43.076649+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:19:43.076649+0000","last_undegraded":"2024-09-16T15:19:43.076649+0000","last_fullsized":"2024-09-16T15:19:43.076649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:43:25.462263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":862,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.715726+0000","last_change":"2024-09-16T15:00:49.319607+0000","last_active":"2024-09-16T15:19:44.715726+0000","last_peered":"2024-09-16T15:19:44.715726+0000","last_clean":"2024-09-16T15:19:44.715726+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:19:44.715726+0000","last_undegraded":"2024-09-16T15:19:44.715726+0000","last_fullsized":"2024-09-16T15:19:44.715726+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:48.559120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":859,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.079648+0000","last_change":"2024-09-16T15:00:49.566752+0000","last_active":"2024-09-16T15:19:43.079648+0000","last_peered":"2024-09-16T15:19:43.079648+0000","last_clean":"2024-09-16T15:19:43.079648+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:19:43.079648+0000","last_undegraded":"2024-09-16T15:19:43.079648+0000","last_fullsized":"2024-09-16T15:19:43.079648+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:23.359447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":865,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.076560+0000","last_change":"2024-09-16T15:00:49.324450+0000","last_active":"2024-09-16T15:19:43.076560+0000","last_peered":"2024-09-16T15:19:43.076560+0000","last_clean":"2024-09-16T15:19:43.076560+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:19:43.076560+0000","last_undegraded":"2024-09-16T15:19:43.076560+0000","last_fullsized":"2024-09-16T15:19:43.076560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:26:16.292570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":866,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:00.376449+0000","last_change":"2024-09-16T15:21:00.376560+0000","last_active":"2024-09-16T15:21:00.376449+0000","last_peered":"2024-09-16T15:21:00.376449+0000","last_clean":"2024-09-16T15:21:00.376449+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:21:00.376449+0000","last_undegraded":"2024-09-16T15:21:00.376449+0000","last_fullsized":"2024-09-16T15:21:00.376449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:00.376415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:00.376415+0000","last_clean_scrub_stamp":"2024-09-16T15:21:00.376415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:21.405803+0000","scrub_duration":0.001206162,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":861,"reported_epoch":860,"state":"active+clean","last_fresh":"2024-09-16T15:19:43.080433+0000","last_change":"2024-09-16T15:00:49.319591+0000","last_active":"2024-09-16T15:19:43.080433+0000","last_peered":"2024-09-16T15:19:43.080433+0000","last_clean":"2024-09-16T15:19:43.080433+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:19:43.080433+0000","last_undegraded":"2024-09-16T15:19:43.080433+0000","last_fullsized":"2024-09-16T15:19:43.080433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:57.528133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":855,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:20:20.805323+0000","last_change":"2024-09-16T15:00:49.320197+0000","last_active":"2024-09-16T15:20:20.805323+0000","last_peered":"2024-09-16T15:20:20.805323+0000","last_clean":"2024-09-16T15:20:20.805323+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:20:20.805323+0000","last_undegraded":"2024-09-16T15:20:20.805323+0000","last_fullsized":"2024-09-16T15:20:20.805323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":861,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:19:44.717482+0000","last_change":"2024-09-16T15:00:49.319854+0000","last_active":"2024-09-16T15:19:44.717482+0000","last_peered":"2024-09-16T15:19:44.717482+0000","last_clean":"2024-09-16T15:19:44.717482+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:19:44.717482+0000","last_undegraded":"2024-09-16T15:19:44.717482+0000","last_fullsized":"2024-09-16T15:19:44.717482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:00:45.960564+0000","last_clean_scrub_stamp":"2024-09-16T15:00:45.960564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:18.193051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575095,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7069696,"data_stored":3855983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19400000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.31900000000000001,"15min":0.24199999999999999},"max":{"1min":0.66000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.48099999999999998}]},{"osd":1,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51800000000000002,"15min":0.497},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.23499999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":1.0860000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.26900000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.94999999999999996},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Sep 16 15:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.99299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":0.752,"15min":1.046},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.34699999999999998},"min":{"1min":0.248,"5min":0.23899999999999999,"15min":0.19900000000000001},"max":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.60099999999999998,"5min":0.628,"15min":0.63900000000000001},"last":0.372}]},{"osd":4,"last update":"Mon Sep 16 15:20:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.35399999999999998},"min":{"1min":0.221,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.58499999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.441,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.69599999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575098,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50356,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321484,"statfs":{"total":96636764160,"available":96585199616,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.22900000000000001},"max":{"1min":0.68300000000000005,"5min":0.81000000000000005,"15min":1.151},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.625,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.374,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63200000000000001,"5min":0.68400000000000005,"15min":1.2669999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.65000000000000002,"5min":0.753,"15min":0.88300000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.496,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":0.628,"5min":0.69199999999999995,"15min":1.391},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.46899999999999997},"min":{"1min":0.312,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.57899999999999996,"5min":0.79300000000000004,"15min":0.80400000000000005},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.217,"5min":0.193,"15min":0.193},"max":{"1min":0.48599999999999999,"5min":0.65500000000000003,"15min":1.6100000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.38200000000000001,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Sep 16 15:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.37,"15min":0.38200000000000001},"min":{"1min":0.219,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55800000000000005,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.442,"15min":0.44},"min":{"1min":0.27200000000000002,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.59299999999999997,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834575095,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.18099999999999999},"max":{"1min":0.438,"5min":0.56399999999999995,"15min":0.70899999999999996},"last":0.26500000000000001}]},{"osd":1,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.33000000000000002,"15min":0.35499999999999998},"min":{"1min":0.20200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.39200000000000002,"5min":0.496,"15min":2.2669999999999999},"last":0.246},{"interface":"front","average":{"1min":0.308,"5min":0.33600000000000002,"15min":0.33700000000000002},"min":{"1min":0.224,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42199999999999999,"5min":0.58899999999999997,"15min":0.72299999999999998},"last":0.23100000000000001}]},{"osd":3,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.80000000000000004},"last":0.52400000000000002}]},{"osd":4,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.33500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.83999999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.79900000000000004},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.70799999999999996,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":7065600,"data_stored":3854819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32900000000000001},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.156},"max":{"1min":0.42799999999999999,"5min":0.49099999999999999,"15min":0.88900000000000001},"last":0.215},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":1.262},"last":0.28499999999999998}]},{"osd":2,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.33800000000000002,"15min":0.35499999999999998},"min":{"1min":0.23999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.53600000000000003,"5min":0.879,"15min":4.6829999999999998},"last":0.253},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34200000000000003,"15min":0.39600000000000002},"min":{"1min":0.21099999999999999,"5min":0.19800000000000001,"15min":0.17000000000000001},"max":{"1min":0.55700000000000005,"5min":0.90700000000000003,"15min":12.194000000000001},"last":0.26700000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.377,"5min":0.246,"15min":0.246},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.024},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":1.129},"last":0.504}]},{"osd":4,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":0.748,"15min":4.6020000000000003},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.755,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.64700000000000002}]},{"osd":5,"last update":"Mon Sep 16 15:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.73299999999999998,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":1.198,"15min":1.2050000000000001},"last":0.65900000000000003}]}]},{"osd":4,"up_from":11,"seq":47244640501,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56064,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.39900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.60199999999999998,"5min":0.745,"15min":0.82099999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.46899999999999997},"min":{"1min":0.38300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.67500000000000004},"last":0.46800000000000003}]},{"osd":1,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.36799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.73499999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.76300000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Sep 16 15:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.69899999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.5},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77800000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.28599999999999998,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.25700000000000001,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.22900000000000001},"max":{"1min":0.625,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.41199999999999998}]}]},{"osd":0,"up_from":11,"seq":47244640502,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64308,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307532,"statfs":{"total":96636764160,"available":96570912768,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.47399999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34000000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.246}]},{"osd":2,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.35199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.53100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36499999999999999,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.27400000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.749,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.495},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.387,"15min":1.387},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Sep 16 15:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55300000000000005,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.77400000000000002,"5min":1.093,"15min":1.093},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.52900000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":0.80500000000000005,"5min":1.113,"15min":1.113},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.77500000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.52700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:21:02.953 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T15:00:44.667324+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=44, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.953 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.953 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T15:00:45.960564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=0, tm_sec=45, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=20, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:21:02.954 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T15:21:03.068 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3510166268}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2889422828}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3775666006}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2973911113}) 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.069 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2451586363}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1518127431}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2261828406}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3295962596}) 2024-09-16T15:21:03.070 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.066+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2712590875}) 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2427835140}) 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1856459189}) 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.072 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2260364975}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=569080048}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1450377373}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4010656961}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2738687971}) 2024-09-16T15:21:03.073 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3067512499}) 2024-09-16T15:21:03.074 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.069+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4244541879}) 2024-09-16T15:21:03.532 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=147954950}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4122283472}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3652963253}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=450686017}) 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.533 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1647175940}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2253405738}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3662396460}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1588229870}) 2024-09-16T15:21:03.534 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.530+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1035675408}) 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1468121367}) 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=770014025}) 2024-09-16T15:21:03.695 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2405460026}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1475785701}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2036458124}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1447599374}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.696 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=646116006}) 2024-09-16T15:21:03.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=839538945}) 2024-09-16T15:21:03.697 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.692+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2748689487}) 2024-09-16T15:21:03.704 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=497466334}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2286982035}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2617939959}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2981503139}) 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.705 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1118888993}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2255038283}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=276007265}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2807442981}) 2024-09-16T15:21:03.706 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.702+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=744873649}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2994598986}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=192315729}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3736772450}) 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.731 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1090618477}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2438890556}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4183804571}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3262347595}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2266367353}) 2024-09-16T15:21:03.732 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.728+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1522716338}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=76} crc {osdmap=358531987}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=76} crc {osdmap=4061213902}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=79} crc {osdmap=4227214351}) 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.748 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=78} crc {osdmap=3379575281}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=79} crc {osdmap=4034094331}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=78} crc {osdmap=2737355853}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=77} crc {osdmap=4059040311}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=78} crc {osdmap=3564910849}) 2024-09-16T15:21:03.749 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-16T15:21:03.745+0000 7f92456ac640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=75} crc {osdmap=3158462969}) 2024-09-16T15:21:22.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:21:23.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.062+0000 7f3157108640 10 monclient: get_monmap_and_config 2024-09-16T15:21:23.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.062+0000 7f3157108640 10 monclient: build_initial_monmap 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.062+0000 7f3157108640 10 monclient: monmap: 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:23.063193+0000 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:23.063193+0000 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:23.064 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:23.065 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 Processor -- start 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 -- start start 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient: init 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient: _add_conns ranks=[1,7,6,2,10,4,5,0,3,8,9] 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3150136030 addr v1:172.21.15.162:6789/0 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 0x7f315009c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): picked mon.noname-i con 0x7f315009bfe0 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): picked mon.noname-d con 0x7f315013bf80 addr [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3150073250 con 0x7f3150136030 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 -- --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f315009b600 con 0x7f315013bf80 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f31500a5740 con 0x7f315009bfe0 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f3157108640 10 monclient(hunting): authenticate will time out at 2019.652645s 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 1 --2- >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3303/0 says I am v2:172.21.15.134:55248/0 (socket says 172.21.15.134:55248) 2024-09-16T15:21:23.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 1 -- 172.21.15.134:0/2906999909 learned_addr learned my addr 172.21.15.134:0/2906999909 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): get_auth_request con 0x7f315013bf80 auth_method 0 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f314ffff640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 0x7f315009c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f314ffff640 10 monclient(hunting): get_auth_request con 0x7f315009bfe0 auth_method 0 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f314ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f314ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.063+0000 7f314ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 10 monclient(hunting): handle_auth_done global_id 94983 payload 274 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 1 -- 172.21.15.134:0/2906999909 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 msgr2=0x7f315009c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 0x7f315009c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 1 -- 172.21.15.134:0/2906999909 >> v1:172.21.15.162:6789/0 conn(0x7f3150136030 legacy=0x7f3150136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 10 monclient: _finish_hunting 0 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 1 monclient: found mon.noname-d 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.134:3303/0 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f31556a5640 1 -- 172.21.15.134:0/2906999909 --> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500a52c0 con 0x7f315013bf80 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.064+0000 7f314ffff640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 0x7f315009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.066+0000 7f31556a5640 1 -- 172.21.15.134:0/2906999909 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 msgr2=0x7f315013e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.066+0000 7f31556a5640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31400098a0 tx=0x7f314002a0d0 comp rx=0 tx=0).ready entity=mon.6 client_cookie=1e88a66940a877f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 1 -- 172.21.15.134:0/2906999909 <== mon.6 v2:172.21.15.134:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f314002adf0 con 0x7f315013bf80 2024-09-16T15:21:23.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: dump: 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.074 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: _finish_auth 0 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: _check_auth_tickets 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 1 -- 172.21.15.134:0/2906999909 <== mon.6 v2:172.21.15.134:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31400307b0 con 0x7f315013bf80 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 5 monclient: authenticate success, global_id 94983 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:23.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 1 -- 172.21.15.134:0/2906999909 <== mon.6 v2:172.21.15.134:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3140030d90 con 0x7f315013bf80 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 4 set_mon_vals no callback set 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f314f7fe640 10 monclient: dump: 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.076 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 10 monclient: shutdown 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 -- 172.21.15.134:0/2906999909 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 msgr2=0x7f315013e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31400098a0 tx=0x7f314002a0d0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 -- 172.21.15.134:0/2906999909 shutdown_connections 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] conn(0x7f315013bf80 0x7f315013e370 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 --2- 172.21.15.134:0/2906999909 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315009bfe0 0x7f315009c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.067+0000 7f3157108640 1 -- 172.21.15.134:0/2906999909 >> 172.21.15.134:0/2906999909 conn(0x7f31500a9ea0 msgr2=0x7f31500a5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.068+0000 7f3157108640 1 -- 172.21.15.134:0/2906999909 shutdown_connections 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.068+0000 7f3157108640 1 -- 172.21.15.134:0/2906999909 wait complete. 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.068+0000 7f3157108640 10 monclient: build_initial_monmap 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.068+0000 7f3157108640 10 monclient: monmap: 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:23.078 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 Processor -- start 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 -- start start 2024-09-16T15:21:23.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient: init 2024-09-16T15:21:23.083 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient: _add_conns ranks=[2,3,7,8,4,6,0,1,5] 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): picked mon.noname-b con 0x7f315009bfe0 addr [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3150136030 0x7f3150136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): picked mon.noname-g con 0x7f3150136030 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 --2- >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315013bf80 0x7f31501370f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): picked mon.noname-i con 0x7f315013bf80 addr [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31500a3100 con 0x7f315009bfe0 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31500a47a0 con 0x7f3150136030 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.084 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 1 -- --> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f31500a8240 con 0x7f315013bf80 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3157108640 10 monclient(hunting): authenticate will time out at 2019.658823s 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:43796/0 (socket says 172.21.15.134:43796) 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 1 -- 172.21.15.134:0/702754022 learned_addr learned my addr 172.21.15.134:0/702754022 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 10 monclient(hunting): get_auth_request con 0x7f315009bfe0 auth_method 0 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f3154ea4640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.069+0000 7f314ffff640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3150136030 0x7f3150136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315013bf80 0x7f31501370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): get_auth_request con 0x7f3150136030 auth_method 0 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): get_auth_request con 0x7f315013bf80 auth_method 0 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f31556a5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.086 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f314ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient(hunting): handle_auth_done global_id 114833 payload 824 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 -- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315013bf80 msgr2=0x7f31501370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315013bf80 0x7f31501370f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 -- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3150136030 msgr2=0x7f3150136bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3150136030 0x7f3150136bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient: _finish_hunting 0 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 monclient: found mon.noname-b 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.134:3301/0 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.070+0000 7f3154ea4640 1 -- 172.21.15.134:0/702754022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31400094d0 con 0x7f315009bfe0 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3154ea4640 1 -- 172.21.15.134:0/702754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 msgr2=0x7f315013b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3154ea4640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f314800c7c0 tx=0x7f314800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.087 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3148012420 con 0x7f315009bfe0 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: dump: 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.088 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: _finish_auth 0 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: _check_auth_tickets 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31480125a0 con 0x7f315009bfe0 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 5 monclient: authenticate success, global_id 114833 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 10 monclient: _renew_subs 2024-09-16T15:21:23.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3150070950 con 0x7f315009bfe0 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3148012bc0 con 0x7f315009bfe0 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314d7fa640 10 monclient: dump: 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.090 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 10 monclient: _renew_subs 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3150138e10 con 0x7f315009bfe0 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.072+0000 7f314effd640 4 set_mon_vals no callback set 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.073+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7f3148012d20 con 0x7f315009bfe0 2024-09-16T15:21:23.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.073+0000 7f3157108640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.073+0000 7f3157108640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.073+0000 7f3157108640 10 monclient: _send_mon_message to mon.b at v2:172.21.15.134:3301/0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.074+0000 7f314d7fa640 1 --2- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 0x7f313c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.073+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3150061ae0 con 0x7f315009bfe0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.074+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7f3148049d70 con 0x7f315009bfe0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.074+0000 7f314ffff640 1 --2- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 0x7f313c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.074+0000 7f314ffff640 10 monclient: get_auth_request con 0x7f313c0388b0 auth_method 0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.076+0000 7f314ffff640 1 --2- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 0x7f313c03ad70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f315009c8a0 tx=0x7f3144008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.078+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7f3150061ae0 con 0x7f315009bfe0 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.078+0000 7f314d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.078+0000 7f314d7fa640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:23.237 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.236+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 --> v2:172.21.15.162:6800/980547263 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f315007a680 con 0x7f313c0388b0 2024-09-16T15:21:23.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.239+0000 7f314d7fa640 1 -- 172.21.15.134:0/702754022 <== mgr.4120 v2:172.21.15.162:6800/980547263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+54299 (secure 0 0 0) 0x7f31440118b0 con 0x7f313c0388b0 2024-09-16T15:21:23.241 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-16T15:21:23.241 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 msgr2=0x7f313c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 0x7f313c03ad70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f315009c8a0 tx=0x7f3144008040 comp rx=0 tx=0).stop 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 10 monclient: shutdown 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 msgr2=0x7f315013b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f314800c7c0 tx=0x7f314800cc80 comp rx=0 tx=0).stop 2024-09-16T15:21:23.243 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 shutdown_connections 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] conn(0x7f315013bf80 0x7f31501370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> v2:172.21.15.162:6800/980547263 conn(0x7f313c0388b0 0x7f313c03ad70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3150136030 0x7f3150136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 --2- 172.21.15.134:0/702754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f315009bfe0 0x7f315013b1b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 >> 172.21.15.134:0/702754022 conn(0x7f31500a9ea0 msgr2=0x7f315009d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 shutdown_connections 2024-09-16T15:21:23.244 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.241+0000 7f3157108640 1 -- 172.21.15.134:0/702754022 wait complete. 2024-09-16T15:21:23.253 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":1475,"stamp":"2024-09-16T15:21:22.415659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":363480,"kb_used_data":41508,"kb_used_omap":9,"kb_used_meta":321910,"kb_avail":565867560,"statfs":{"total":579820584960,"available":579448381440,"internally_reserved":0,"allocated":42504192,"data_stored":24192514,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9562,"internal_metadata":329636518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002817"},"pg_stats":[{"pgid":"1.0","version":"861'119","reported_seq":1022,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:02.451116+0000","last_change":"2024-09-16T15:21:02.451198+0000","last_active":"2024-09-16T15:21:02.451116+0000","last_peered":"2024-09-16T15:21:02.451116+0000","last_clean":"2024-09-16T15:21:02.451116+0000","last_became_active":"2024-09-16T15:00:45.968553+0000","last_became_peered":"2024-09-16T15:00:45.968553+0000","last_unstale":"2024-09-16T15:21:02.451116+0000","last_undegraded":"2024-09-16T15:21:02.451116+0000","last_fullsized":"2024-09-16T15:21:02.451116+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"861'119","last_scrub_stamp":"2024-09-16T15:21:02.451096+0000","last_deep_scrub":"861'119","last_deep_scrub_stamp":"2024-09-16T15:21:02.451096+0000","last_clean_scrub_stamp":"2024-09-16T15:21:02.451096+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:58.409300+0000","scrub_duration":0.0040848940000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"861'2","reported_seq":877,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:06.518699+0000","last_change":"2024-09-16T15:21:06.518803+0000","last_active":"2024-09-16T15:21:06.518699+0000","last_peered":"2024-09-16T15:21:06.518699+0000","last_clean":"2024-09-16T15:21:06.518699+0000","last_became_active":"2024-09-16T15:00:46.973256+0000","last_became_peered":"2024-09-16T15:00:46.973256+0000","last_unstale":"2024-09-16T15:21:06.518699+0000","last_undegraded":"2024-09-16T15:21:06.518699+0000","last_fullsized":"2024-09-16T15:21:06.518699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"861'2","last_scrub_stamp":"2024-09-16T15:21:06.518677+0000","last_deep_scrub":"861'2","last_deep_scrub_stamp":"2024-09-16T15:21:06.518677+0000","last_clean_scrub_stamp":"2024-09-16T15:21:06.518677+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:02.010063+0000","scrub_duration":0.0032381200000000001,"objects_trimmed":0,"snaptrim_duration":0.00031043100000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":874,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:06.359671+0000","last_change":"2024-09-16T15:21:06.359788+0000","last_active":"2024-09-16T15:21:06.359671+0000","last_peered":"2024-09-16T15:21:06.359671+0000","last_clean":"2024-09-16T15:21:06.359671+0000","last_became_active":"2024-09-16T15:00:46.973102+0000","last_became_peered":"2024-09-16T15:00:46.973102+0000","last_unstale":"2024-09-16T15:21:06.359671+0000","last_undegraded":"2024-09-16T15:21:06.359671+0000","last_fullsized":"2024-09-16T15:21:06.359671+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:06.359651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:06.359651+0000","last_clean_scrub_stamp":"2024-09-16T15:21:06.359651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:59.708033+0000","scrub_duration":0.0091645170000000005,"objects_trimmed":0,"snaptrim_duration":0.00047147900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":871,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:02.726749+0000","last_change":"2024-09-16T15:21:02.726886+0000","last_active":"2024-09-16T15:21:02.726749+0000","last_peered":"2024-09-16T15:21:02.726749+0000","last_clean":"2024-09-16T15:21:02.726749+0000","last_became_active":"2024-09-16T15:00:46.973297+0000","last_became_peered":"2024-09-16T15:00:46.973297+0000","last_unstale":"2024-09-16T15:21:02.726749+0000","last_undegraded":"2024-09-16T15:21:02.726749+0000","last_fullsized":"2024-09-16T15:21:02.726749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:02.726712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:02.726712+0000","last_clean_scrub_stamp":"2024-09-16T15:21:02.726712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:08.928684+0000","scrub_duration":0.0019624180000000001,"objects_trimmed":0,"snaptrim_duration":0.00025202500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"861'3","reported_seq":878,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:05.508362+0000","last_change":"2024-09-16T15:21:05.508508+0000","last_active":"2024-09-16T15:21:05.508362+0000","last_peered":"2024-09-16T15:21:05.508362+0000","last_clean":"2024-09-16T15:21:05.508362+0000","last_became_active":"2024-09-16T15:00:46.975988+0000","last_became_peered":"2024-09-16T15:00:46.975988+0000","last_unstale":"2024-09-16T15:21:05.508362+0000","last_undegraded":"2024-09-16T15:21:05.508362+0000","last_fullsized":"2024-09-16T15:21:05.508362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"861'3","last_scrub_stamp":"2024-09-16T15:21:05.508308+0000","last_deep_scrub":"861'3","last_deep_scrub_stamp":"2024-09-16T15:21:05.508308+0000","last_clean_scrub_stamp":"2024-09-16T15:21:05.508308+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:32.834629+0000","scrub_duration":0.0039245540000000002,"objects_trimmed":0,"snaptrim_duration":0.00078247799999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":866,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:00.376449+0000","last_change":"2024-09-16T15:21:00.376560+0000","last_active":"2024-09-16T15:21:00.376449+0000","last_peered":"2024-09-16T15:21:00.376449+0000","last_clean":"2024-09-16T15:21:00.376449+0000","last_became_active":"2024-09-16T15:00:46.971641+0000","last_became_peered":"2024-09-16T15:00:46.971641+0000","last_unstale":"2024-09-16T15:21:00.376449+0000","last_undegraded":"2024-09-16T15:21:00.376449+0000","last_fullsized":"2024-09-16T15:21:00.376449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:00.376415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:00.376415+0000","last_clean_scrub_stamp":"2024-09-16T15:21:00.376415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:21.405803+0000","scrub_duration":0.001206162,"objects_trimmed":0,"snaptrim_duration":0.00029479499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":873,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:03.427117+0000","last_change":"2024-09-16T15:21:03.427251+0000","last_active":"2024-09-16T15:21:03.427117+0000","last_peered":"2024-09-16T15:21:03.427117+0000","last_clean":"2024-09-16T15:21:03.427117+0000","last_became_active":"2024-09-16T15:00:46.972414+0000","last_became_peered":"2024-09-16T15:00:46.972414+0000","last_unstale":"2024-09-16T15:21:03.427117+0000","last_undegraded":"2024-09-16T15:21:03.427117+0000","last_fullsized":"2024-09-16T15:21:03.427117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:03.427076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:03.427076+0000","last_clean_scrub_stamp":"2024-09-16T15:21:03.427076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:41.808572+0000","scrub_duration":0.0018678709999999999,"objects_trimmed":0,"snaptrim_duration":0.00044007699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":866,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:02.321267+0000","last_change":"2024-09-16T15:21:02.321345+0000","last_active":"2024-09-16T15:21:02.321267+0000","last_peered":"2024-09-16T15:21:02.321267+0000","last_clean":"2024-09-16T15:21:02.321267+0000","last_became_active":"2024-09-16T15:00:46.975755+0000","last_became_peered":"2024-09-16T15:00:46.975755+0000","last_unstale":"2024-09-16T15:21:02.321267+0000","last_undegraded":"2024-09-16T15:21:02.321267+0000","last_fullsized":"2024-09-16T15:21:02.321267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:02.321249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:02.321249+0000","last_clean_scrub_stamp":"2024-09-16T15:21:02.321249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:21.900445+0000","scrub_duration":0.0012842190000000001,"objects_trimmed":0,"snaptrim_duration":0.00047452099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":873,"reported_epoch":861,"state":"active+clean","last_fresh":"2024-09-16T15:21:05.377576+0000","last_change":"2024-09-16T15:21:05.377673+0000","last_active":"2024-09-16T15:21:05.377576+0000","last_peered":"2024-09-16T15:21:05.377576+0000","last_clean":"2024-09-16T15:21:05.377576+0000","last_became_active":"2024-09-16T15:00:46.975481+0000","last_became_peered":"2024-09-16T15:00:46.975481+0000","last_unstale":"2024-09-16T15:21:05.377576+0000","last_undegraded":"2024-09-16T15:21:05.377576+0000","last_fullsized":"2024-09-16T15:21:05.377576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:21:05.377554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:21:05.377554+0000","last_clean_scrub_stamp":"2024-09-16T15:21:05.377554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:51.552579+0000","scrub_duration":0.0014132960000000001,"objects_trimmed":0,"snaptrim_duration":0.00060041800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10784,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834575099,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":7069696,"data_stored":3855983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.46600000000000003},"min":{"1min":0.32100000000000001,"5min":0.23300000000000001,"15min":0.19400000000000001},"max":{"1min":0.621,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.48699999999999999,"15min":0.47799999999999998},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.24199999999999999},"max":{"1min":0.71199999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.5}]},{"osd":1,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.41799999999999998,"5min":0.32700000000000001,"15min":0.23499999999999999},"max":{"1min":0.69599999999999995,"5min":0.71799999999999997,"15min":1.0860000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.50700000000000001,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.26900000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.94999999999999996},"last":0.52000000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47999999999999998,"15min":0.47299999999999998},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.99299999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.41399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":0.752,"15min":1.046},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.374,"15min":0.35299999999999998},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.19900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.376},"min":{"1min":0.28599999999999998,"5min":0.224,"15min":0.21199999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38300000000000001,"15min":0.35799999999999998},"min":{"1min":0.29599999999999999,"5min":0.20599999999999999,"15min":0.19900000000000001},"max":{"1min":0.65200000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.434},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.33500000000000002,"5min":0.26100000000000001,"15min":0.23599999999999999},"max":{"1min":0.68000000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575102,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50352,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94321488,"statfs":{"total":96636764160,"available":96585203712,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.52900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.22900000000000001},"max":{"1min":0.68600000000000005,"5min":0.81000000000000005,"15min":0.91600000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.25800000000000001},"max":{"1min":0.61099999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.37}]},{"osd":1,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.316,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.65600000000000003,"5min":0.68400000000000005,"15min":0.875},"last":0.316},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.49199999999999999},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.24399999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.83899999999999997},"last":0.38500000000000001}]},{"osd":2,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.496},"min":{"1min":0.40899999999999997,"5min":0.29599999999999999,"15min":0.29099999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.95099999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.33800000000000002,"5min":0.28299999999999997,"15min":0.223},"max":{"1min":0.63700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.33800000000000002}]},{"osd":4,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.22700000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":1.6100000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.26800000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53600000000000003,"5min":0.61499999999999999,"15min":1.1140000000000001},"last":0.28899999999999998}]},{"osd":5,"last update":"Mon Sep 16 15:21:17 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38100000000000001,"15min":0.38200000000000001},"min":{"1min":0.26000000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.54600000000000004,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45700000000000002,"15min":0.443},"min":{"1min":0.32700000000000001,"5min":0.26200000000000001,"15min":0.19500000000000001},"max":{"1min":0.65000000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.32700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575099,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34000000000000002,"15min":0.32700000000000001},"min":{"1min":0.221,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45300000000000001,"5min":0.53800000000000003,"15min":0.68500000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.35099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.18099999999999999},"max":{"1min":0.438,"5min":0.56399999999999995,"15min":0.70899999999999996},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.33000000000000002,"15min":0.35499999999999998},"min":{"1min":0.20200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.39200000000000002,"5min":0.496,"15min":2.2669999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.308,"5min":0.33600000000000002,"15min":0.33700000000000002},"min":{"1min":0.224,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.42199999999999999,"5min":0.58899999999999997,"15min":0.72299999999999998},"last":0.52700000000000002}]},{"osd":3,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.36899999999999999,"5min":0.29499999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.502,"15min":0.496},"min":{"1min":0.35499999999999998,"5min":0.29199999999999998,"15min":0.27400000000000002},"max":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.80000000000000004},"last":0.64000000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.33500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.83999999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.79900000000000004},"last":0.625}]},{"osd":5,"last update":"Mon Sep 16 15:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.70799999999999996,"5min":0.81399999999999995,"15min":0.81999999999999995},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575100,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":7065600,"data_stored":3854819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.33300000000000002},"min":{"1min":0.189,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.88900000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.38600000000000001},"min":{"1min":0.23899999999999999,"5min":0.191,"15min":0.17000000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":1.262},"last":0.36499999999999999}]},{"osd":2,"last update":"Mon Sep 16 15:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":0.49299999999999999,"5min":0.59399999999999997,"15min":4.6829999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.33700000000000002,"15min":0.39900000000000002},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.57599999999999996,"5min":0.59699999999999998,"15min":12.194000000000001},"last":0.34200000000000003}]},{"osd":3,"last update":"Mon Sep 16 15:20:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.49099999999999999},"min":{"1min":0.377,"5min":0.246,"15min":0.246},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":1.024},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.27600000000000002},"max":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":1.129},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Sep 16 15:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.34000000000000002,"15min":0.28599999999999998},"max":{"1min":0.69099999999999995,"5min":0.748,"15min":4.6020000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.45000000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.71799999999999997,"5min":0.79600000000000004,"15min":2.9489999999999998},"last":0.66900000000000004}]},{"osd":5,"last update":"Mon Sep 16 15:21:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.33100000000000002,"15min":0.307},"max":{"1min":0.71099999999999997,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.442,"5min":0.33200000000000002,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":1.198,"15min":1.2050000000000001},"last":0.69299999999999995}]}]},{"osd":4,"up_from":11,"seq":47244640505,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56064,"kb_used_data":6900,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315776,"statfs":{"total":96636764160,"available":96579354624,"internally_reserved":0,"allocated":7065600,"data_stored":3855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47799999999999998,"15min":0.48599999999999999},"min":{"1min":0.39900000000000002,"5min":0.218,"15min":0.218},"max":{"1min":0.60199999999999998,"5min":0.745,"15min":0.82099999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.46800000000000003,"15min":0.46899999999999997},"min":{"1min":0.38300000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.61599999999999999,"5min":0.64900000000000002,"15min":0.67500000000000004},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46600000000000003},"min":{"1min":0.36799999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.73499999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.249,"15min":0.24299999999999999},"max":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.76300000000000001},"last":0.46000000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.432,"5min":0.29499999999999998,"15min":0.27800000000000002},"max":{"1min":0.69899999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.27600000000000002,"15min":0.26600000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.77800000000000002},"last":0.5}]},{"osd":3,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.28599999999999998,"5min":0.20899999999999999,"15min":0.20599999999999999},"max":{"1min":0.63600000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42999999999999999,"15min":0.42599999999999999},"min":{"1min":0.33800000000000002,"5min":0.25700000000000001,"15min":0.215},"max":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.70599999999999996},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Sep 16 15:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.38500000000000001,"15min":0.374},"min":{"1min":0.32800000000000001,"5min":0.22800000000000001,"15min":0.20399999999999999},"max":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.73799999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.32000000000000001,"5min":0.23899999999999999,"15min":0.22900000000000001},"max":{"1min":0.625,"5min":0.68799999999999994,"15min":0.76100000000000001},"last":0.38400000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640506,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64304,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307536,"statfs":{"total":96636764160,"available":96570916864,"internally_reserved":0,"allocated":7118848,"data_stored":4384892,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.47399999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.25},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34000000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":0.495,"5min":0.55000000000000004,"15min":1.0580000000000001},"last":0.27700000000000002}]},{"osd":2,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.35199999999999998},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.53100000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.314},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36499999999999999,"15min":0.38},"min":{"1min":0.25600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.498,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.26300000000000001}]},{"osd":3,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.26800000000000002},"max":{"1min":0.749,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.387,"15min":1.387},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Sep 16 15:21:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.48899999999999999,"5min":0.30499999999999999,"15min":0.27200000000000002},"max":{"1min":0.71699999999999997,"5min":1.093,"15min":1.093},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.45000000000000001,"5min":0.29399999999999998,"15min":0.28399999999999997},"max":{"1min":0.67800000000000005,"5min":1.113,"15min":1.113},"last":0.504}]},{"osd":5,"last update":"Mon Sep 16 15:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.77500000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.47199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5392,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:21:23.255 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T15:21:23.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.349+0000 7fdd71e87640 10 monclient: get_monmap_and_config 2024-09-16T15:21:23.350 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.349+0000 7fdd71e87640 10 monclient: build_initial_monmap 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.349+0000 7fdd71e87640 10 monclient: monmap: 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:21:23.350236+0000 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:21:23.350236+0000 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:0: v1:172.21.15.134:6789/0 mon.noname-a-legacy 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:1: v1:172.21.15.162:6789/0 mon.noname-f-legacy 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:23.351 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:23.352 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:23.352 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:23.352 INFO:teuthology.orchestra.run.smithi134.stderr:9: v2:172.21.15.134:3300/0 mon.noname-a 2024-09-16T15:21:23.352 INFO:teuthology.orchestra.run.smithi134.stderr:10: v2:172.21.15.162:3300/0 mon.noname-f 2024-09-16T15:21:23.352 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 Processor -- start 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 -- start start 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient: init 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient: _add_conns ranks=[3,4,0,9,5,1,2,6,7,10,8] 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c09c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-g con 0x7fdd6c136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-c con 0x7fdd6c137220 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fdd6c135dc0 addr v1:172.21.15.134:6789/0 2024-09-16T15:21:23.354 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c073250 con 0x7fdd6c135dc0 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09abc0 con 0x7fdd6c136e40 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09ba50 con 0x7fdd6c137220 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd71e87640 10 monclient(hunting): authenticate will time out at 2019.939598s 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6bfff640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fdd6c135dc0 0x7fdd6c136220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:59202/0 (socket says 172.21.15.134:59202) 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:46044/0 (socket says 172.21.15.134:46044) 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6bfff640 1 -- 172.21.15.134:0/2504732134 learned_addr learned my addr 172.21.15.134:0/2504732134 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): get_auth_request con 0x7fdd6c137220 auth_method 0 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6b7fe640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c09c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 10 monclient(hunting): handle_auth_done global_id 124555 payload 274 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 1 -- 172.21.15.134:0/2504732134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 msgr2=0x7fdd6c09c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.350+0000 7fdd6affd640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c09c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 1 -- 172.21.15.134:0/2504732134 >> v1:172.21.15.134:6789/0 conn(0x7fdd6c135dc0 legacy=0x7fdd6c136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 10 monclient: _finish_hunting 0 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 1 monclient: found mon.noname-c 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:23.356 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.134:3302/0 2024-09-16T15:21:23.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.351+0000 7fdd6affd640 1 -- 172.21.15.134:0/2504732134 --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0a53c0 con 0x7fdd6c137220 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6affd640 1 -- 172.21.15.134:0/2504732134 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 msgr2=0x7fdd6c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6affd640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009800 tx=0x7fdd5402a030 comp rx=0 tx=0).ready entity=mon.4 client_cookie=3c8384964ade1860 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 1 -- 172.21.15.134:0/2504732134 <== mon.4 v2:172.21.15.134:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd5402ae30 con 0x7fdd6c137220 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: dump: 2024-09-16T15:21:23.362 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.363 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: _finish_auth 0 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: _check_auth_tickets 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 1 -- 172.21.15.134:0/2504732134 <== mon.4 v2:172.21.15.134:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540307b0 con 0x7fdd6c137220 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd71e87640 5 monclient: authenticate success, global_id 124555 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd71e87640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd71e87640 10 monclient: get_monmap_and_config success 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 1 -- 172.21.15.134:0/2504732134 <== mon.4 v2:172.21.15.134:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd54030d90 con 0x7fdd6c137220 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd6a7fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.353+0000 7fdd71e87640 4 set_mon_vals no callback set 2024-09-16T15:21:23.364 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd6a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd6a7fc640 10 monclient: dump: 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.365 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 10 monclient: shutdown 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 -- 172.21.15.134:0/2504732134 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 msgr2=0x7fdd6c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009800 tx=0x7fdd5402a030 comp rx=0 tx=0).stop 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 -- 172.21.15.134:0/2504732134 shutdown_connections 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c137220 0x7fdd6c137600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 --2- 172.21.15.134:0/2504732134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c09c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 -- 172.21.15.134:0/2504732134 >> 172.21.15.134:0/2504732134 conn(0x7fdd6c0a6110 msgr2=0x7fdd6c0a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:23.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 -- 172.21.15.134:0/2504732134 shutdown_connections 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.354+0000 7fdd71e87640 1 -- 172.21.15.134:0/2504732134 wait complete. 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.355+0000 7fdd71e87640 10 monclient: build_initial_monmap 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.355+0000 7fdd71e87640 10 monclient: monmap: 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 0 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 0.000000 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:created 0.000000 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 0 (unknown) 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.noname-a 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.noname-f 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.noname-b 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.noname-g 2024-09-16T15:21:23.367 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.noname-c 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.noname-h 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.noname-d 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.noname-i 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.noname-e 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.355+0000 7fdd71e87640 1 Processor -- start 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.355+0000 7fdd71e87640 1 -- start start 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.355+0000 7fdd71e87640 10 monclient: init 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient: _reopen_session rank -1 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient: _add_conns ranks=[4,3,0,6,2,1,8,5,7] 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-c con 0x7fdd6c135dc0 addr [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c1201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.368 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-g con 0x7fdd6c136e40 addr [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 0x7fdd6c1206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): picked mon.noname-a con 0x7fdd6c137220 addr [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c122400 con 0x7fdd6c137220 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0a8ac0 con 0x7fdd6c136e40 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): start opening mon connection 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 1 -- --> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c136470 con 0x7fdd6c135dc0 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): _renew_subs 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd71e87640 10 monclient(hunting): authenticate will time out at 2019.945185s 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 0x7fdd6c1206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.369 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3302/0 says I am v2:172.21.15.134:46058/0 (socket says 172.21.15.134:46058) 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 1 -- 172.21.15.134:0/3215123807 learned_addr learned my addr 172.21.15.134:0/3215123807 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): get_auth_request con 0x7fdd6c137220 auth_method 0 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): get_auth_request con 0x7fdd6c135dc0 auth_method 0 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c1201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.370 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): get_auth_request con 0x7fdd6c136e40 auth_method 0 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): _init_auth method 2 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): _init_auth creating new auth 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient(hunting): handle_auth_done global_id 124578 payload 824 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 -- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 msgr2=0x7fdd6c11fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 -- 172.21.15.134:0/3215123807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 msgr2=0x7fdd6c1201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.371 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c1201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient: _finish_hunting 0 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 monclient: found mon.noname-a 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.134:3300/0 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6bfff640 1 -- 172.21.15.134:0/3215123807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54009400 con 0x7fdd6c137220 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.356+0000 7fdd6b7fe640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd6bfff640 1 -- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 msgr2=0x7fdd6c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd6bfff640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 0x7fdd6c1206e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400cb20 tx=0x7fdd6400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd6400abe0 con 0x7fdd6c137220 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: dump: 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.372 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: _finish_auth 0 2024-09-16T15:21:23.373 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: _check_auth_tickets 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6400ad60 con 0x7fdd6c137220 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd71e87640 5 monclient: authenticate success, global_id 124578 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd71e87640 10 monclient: _renew_subs 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd71e87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c121b30 con 0x7fdd6c137220 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.358+0000 7fdd63fff640 10 monclient: handle_config config(0 keys) v1 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdd64007950 con 0x7fdd6c137220 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd63fff640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd63fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd63fff640 10 monclient: dump: 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:epoch 1 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:fsid 9b832ad6-88ab-4e38-a956-73c2de3b6492 2024-09-16T15:21:23.374 INFO:teuthology.orchestra.run.smithi134.stderr:last_changed 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:created 2024-09-16T15:00:04.519049+0000 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:min_mon_release 18 (reef) 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:election_strategy: 1 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.f 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.b 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:3: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.g 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:4: [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] mon.c 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:5: [v2:172.21.15.162:3302/0,v1:172.21.15.162:6791/0] mon.h 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:6: [v2:172.21.15.134:3303/0,v1:172.21.15.134:6792/0] mon.d 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:7: [v2:172.21.15.162:3303/0,v1:172.21.15.162:6792/0] mon.i 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:8: [v2:172.21.15.134:3304/0,v1:172.21.15.134:6793/0] mon.e 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-16T15:21:23.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd69ffb640 4 set_mon_vals no callback set 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd71e87640 10 monclient: _renew_subs 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd71e87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.359+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c136960 con 0x7fdd6c137220 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd71e87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd71e87640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd71e87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 44782+0+0 (secure 0 0 0) 0x7fdd64007070 con 0x7fdd6c137220 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0a3390 con 0x7fdd6c137220 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd63fff640 1 --2- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 0x7fdd4c03ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 5 ==== osd_map(861..861 src has 1..861) v4 ==== 3598+0+0 (secure 0 0 0) 0x7fdd64049af0 con 0x7fdd6c137220 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd6b7fe640 1 --2- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 0x7fdd4c03ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:21:23.376 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.360+0000 7fdd6b7fe640 10 monclient: get_auth_request con 0x7fdd4c038880 auth_method 0 2024-09-16T15:21:23.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.363+0000 7fdd6b7fe640 1 --2- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 0x7fdd4c03ad40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006fd0 tx=0x7fdd5c008040 comp rx=0 tx=0).ready entity=mgr.4120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:21:23.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.364+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182695 (secure 0 0 0) 0x7fdd64007ab0 con 0x7fdd6c137220 2024-09-16T15:21:23.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.364+0000 7fdd63fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:21:23.377 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.364+0000 7fdd63fff640 10 monclient: _finish_command 1 = system:0 2024-09-16T15:21:23.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.523+0000 7fdd71e87640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-16T15:21:23.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.523+0000 7fdd71e87640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-16T15:21:23.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.523+0000 7fdd71e87640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.134:3300/0 2024-09-16T15:21:23.525 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.523+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdd6c09b770 con 0x7fdd6c137220 2024-09-16T15:21:23.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.684+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdd64007ab0 con 0x7fdd6c137220 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.684+0000 7fdd63fff640 10 monclient: handle_config config(1 keys) v1 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.684+0000 7fdd69ffb640 4 set_mon_vals no callback set 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.684+0000 7fdd69ffb640 10 set_mon_vals mon_health_to_clog = false 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.685+0000 7fdd63fff640 1 -- 172.21.15.134:0/3215123807 <== mon.0 v2:172.21.15.134:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdd64017030 con 0x7fdd6c137220 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.685+0000 7fdd63fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-16T15:21:23.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.685+0000 7fdd63fff640 10 monclient: _finish_command 2 = system:0 2024-09-16T15:21:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 msgr2=0x7fdd4c03ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 0x7fdd4c03ad40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c006fd0 tx=0x7fdd5c008040 comp rx=0 tx=0).stop 2024-09-16T15:21:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 10 monclient: shutdown 2024-09-16T15:21:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 20 monclient: shutdown discarding 0 pending message(s) 2024-09-16T15:21:23.689 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 msgr2=0x7fdd6c1206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 0x7fdd6c1206e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400cb20 tx=0x7fdd6400cf20 comp rx=0 tx=0).stop 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 shutdown_connections 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd6c137220 0x7fdd6c1206e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> v2:172.21.15.162:6800/980547263 conn(0x7fdd4c038880 0x7fdd4c03ad40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd6c136e40 0x7fdd6c1201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 --2- 172.21.15.134:0/3215123807 >> [v2:172.21.15.134:3302/0,v1:172.21.15.134:6791/0] conn(0x7fdd6c135dc0 0x7fdd6c11fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.687+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 >> 172.21.15.134:0/3215123807 conn(0x7fdd6c0a6110 msgr2=0x7fdd6c09f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.688+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 shutdown_connections 2024-09-16T15:21:23.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-16T15:21:23.688+0000 7fdd71e87640 1 -- 172.21.15.134:0/3215123807 wait complete. 2024-09-16T15:21:23.700 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T15:21:23.700 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T15:21:23.700 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T15:21:23.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:23.735 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T15:21:23.735 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T15:21:23.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:23.807 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T15:21:23.807 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T15:21:23.807 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:23.876 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T15:21:23.876 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T15:21:23.876 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:23.906 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T15:21:23.906 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T15:21:23.906 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:23.984 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T15:21:23.984 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T15:21:23.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.060 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T15:21:24.061 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T15:21:24.061 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T15:21:24.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.129 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T15:21:24.129 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T15:21:24.129 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:21:24.129 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.143 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:21:24.143 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:21:24.143 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.196 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:21:24.196 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T15:21:24.196 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.248 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T15:21:24.248 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-16T15:21:24.248 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.302 INFO:tasks.ceph.mon.d:Stopped 2024-09-16T15:21:24.302 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-16T15:21:24.303 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.356 INFO:tasks.ceph.mon.e:Stopped 2024-09-16T15:21:24.356 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-16T15:21:24.356 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.370 INFO:tasks.ceph.mon.f:Stopped 2024-09-16T15:21:24.371 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-16T15:21:24.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.428 INFO:tasks.ceph.mon.g:Stopped 2024-09-16T15:21:24.428 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-16T15:21:24.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.486 INFO:tasks.ceph.mon.h:Stopped 2024-09-16T15:21:24.486 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-16T15:21:24.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:21:24.540 INFO:tasks.ceph.mon.i:Stopped 2024-09-16T15:21:24.540 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T15:21:24.540 DEBUG:teuthology.orchestra.run.smithi134:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-16T15:21:24.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:21:24.589 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T15:21:24.779 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:21:24.779 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T15:21:24.885 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:21:24.885 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T15:21:24.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:21:24.981 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T15:21:25.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:21:25.131 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T15:21:25.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:21:25.211 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T15:21:25.260 INFO:tasks.ceph:Archiving mon data... 2024-09-16T15:21:25.261 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.a.tgz 2024-09-16T15:21:25.262 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T15:21:25.277 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.syA8SFeQb9 2024-09-16T15:21:25.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.syA8SFeQb9 2024-09-16T15:21:25.438 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.syA8SFeQb9 2024-09-16T15:21:25.520 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.syA8SFeQb9 is 555KB 2024-09-16T15:21:25.617 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.syA8SFeQb9 2024-09-16T15:21:25.632 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.b.tgz 2024-09-16T15:21:25.632 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T15:21:25.686 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.XXz2SMZGTy 2024-09-16T15:21:25.686 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.XXz2SMZGTy 2024-09-16T15:21:25.832 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.XXz2SMZGTy 2024-09-16T15:21:25.914 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.XXz2SMZGTy is 557KB 2024-09-16T15:21:26.011 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.XXz2SMZGTy 2024-09-16T15:21:26.026 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.c.tgz 2024-09-16T15:21:26.026 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T15:21:26.081 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.mLPcUL6ovH 2024-09-16T15:21:26.081 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.mLPcUL6ovH 2024-09-16T15:21:26.226 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.mLPcUL6ovH 2024-09-16T15:21:26.308 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.mLPcUL6ovH is 554KB 2024-09-16T15:21:26.405 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.mLPcUL6ovH 2024-09-16T15:21:26.420 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.d.tgz 2024-09-16T15:21:26.420 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T15:21:26.474 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.L1iJWRAJ7U 2024-09-16T15:21:26.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.L1iJWRAJ7U 2024-09-16T15:21:26.619 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.L1iJWRAJ7U 2024-09-16T15:21:26.701 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.L1iJWRAJ7U is 562KB 2024-09-16T15:21:26.771 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.L1iJWRAJ7U 2024-09-16T15:21:26.786 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.e.tgz 2024-09-16T15:21:26.786 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-16T15:21:26.841 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.vjbFFhNMyU 2024-09-16T15:21:26.841 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.vjbFFhNMyU 2024-09-16T15:21:26.987 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.vjbFFhNMyU 2024-09-16T15:21:27.069 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.vjbFFhNMyU is 565KB 2024-09-16T15:21:27.139 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.vjbFFhNMyU 2024-09-16T15:21:27.154 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.f.tgz 2024-09-16T15:21:27.154 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T15:21:27.173 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.rLWYfBEGMM 2024-09-16T15:21:27.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.rLWYfBEGMM 2024-09-16T15:21:27.337 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.rLWYfBEGMM 2024-09-16T15:21:27.421 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.rLWYfBEGMM is 557KB 2024-09-16T15:21:27.490 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.rLWYfBEGMM 2024-09-16T15:21:27.505 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.g.tgz 2024-09-16T15:21:27.505 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T15:21:27.560 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.PnLoAiJFhJ 2024-09-16T15:21:27.560 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.PnLoAiJFhJ 2024-09-16T15:21:27.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.PnLoAiJFhJ 2024-09-16T15:21:27.790 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.PnLoAiJFhJ is 565KB 2024-09-16T15:21:27.858 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.PnLoAiJFhJ 2024-09-16T15:21:27.873 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.h.tgz 2024-09-16T15:21:27.874 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T15:21:27.928 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.ZiQ76u4XWZ 2024-09-16T15:21:27.928 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.ZiQ76u4XWZ 2024-09-16T15:21:28.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.ZiQ76u4XWZ 2024-09-16T15:21:28.159 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.ZiQ76u4XWZ is 572KB 2024-09-16T15:21:28.229 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.ZiQ76u4XWZ 2024-09-16T15:21:28.244 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/data/mon.i.tgz 2024-09-16T15:21:28.244 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T15:21:28.299 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.o2NlVtClf0 2024-09-16T15:21:28.299 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.o2NlVtClf0 2024-09-16T15:21:28.446 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.o2NlVtClf0 2024-09-16T15:21:28.528 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.o2NlVtClf0 is 552KB 2024-09-16T15:21:28.596 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.o2NlVtClf0 2024-09-16T15:21:28.611 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T15:21:28.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:21:28.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:21:28.675 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T15:21:28.675 DEBUG:teuthology.orchestra.run.smithi134:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T15:21:28.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T15:21:28.712 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T15:21:28.741 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T15:21:28.746 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T15:21:28.748 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi134/crash 2024-09-16T15:21:28.748 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:21:28.786 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi162/crash 2024-09-16T15:21:28.787 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:21:28.811 INFO:tasks.ceph:Compressing logs... 2024-09-16T15:21:28.811 DEBUG:teuthology.orchestra.run.smithi134:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:21:28.831 DEBUG:teuthology.orchestra.run.smithi162:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:21:48.506 INFO:tasks.ceph:Archiving logs... 2024-09-16T15:21:48.506 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi134/log 2024-09-16T15:21:48.507 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:21:51.282 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi162/log 2024-09-16T15:21:51.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:21:53.401 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T15:21:53.410 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T15:21:53.410 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:21:53.413 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:21:53.505 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T15:21:53.513 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T15:21:53.513 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:21:53.516 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:21:53.530 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:21:53.533 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +88us[ +88us] +/- 47ms 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -353us[ -353us] +/- 62ms 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +232us[ +231us] +/- 63ms 2024-09-16T15:21:53.536 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:21:53.537 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:21:53.538 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-16T15:21:53.538 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +110us[ +121us] +/- 47ms 2024-09-16T15:21:53.538 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -341us[ -341us] +/- 63ms 2024-09-16T15:21:53.538 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +246us[ +257us] +/- 63ms 2024-09-16T15:21:53.538 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:21:53.538 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T15:21:53.591 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T15:21:53.592 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T15:21:53.599 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:21:53.674 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:21:53.741 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T15:21:53.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:21:53.770 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T15:21:53.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:21:53.799 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T15:21:53.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T15:21:53.817 INFO:teuthology.task.internal:Duration was 1700.582672 seconds 2024-09-16T15:21:53.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T15:21:53.826 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T15:21:53.826 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:21:53.829 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:21:53.866 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:21:53.878 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:21:54.231 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T15:21:54.231 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-16T15:21:54.232 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:21:54.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T15:21:54.253 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:21:54.293 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T15:21:54.293 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:21:54.296 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:21:54.361 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T15:21:54.361 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:21:54.363 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:21:54.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T15:21:54.516 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T15:21:54.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:21:54.541 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:21:54.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T15:21:54.576 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:21:54.584 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:21:54.604 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-16T15:21:54.629 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-09-16T15:21:54.652 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:21:54.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:21:54.683 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:21:54.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:21:54.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T15:21:54.716 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T15:21:54.717 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi134 2024-09-16T15:21:54.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:21:54.761 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906784/remote/smithi162 2024-09-16T15:21:54.761 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:21:54.794 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T15:21:54.795 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:21:54.797 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:21:54.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T15:21:54.855 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T15:21:54.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T15:21:54.864 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T15:21:54.864 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:21:54.866 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:21:54.879 INFO:teuthology.orchestra.run.smithi134.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:21 /home/ubuntu/cephtest 2024-09-16T15:21:54.898 INFO:teuthology.orchestra.run.smithi162.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:21 /home/ubuntu/cephtest 2024-09-16T15:21:54.900 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T15:21:54.910 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T15:21:54.932 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/snaps-few-objects} duration: 1700.5826721191406 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T15:21:54.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:21:55.021 INFO:teuthology.run:pass